运行quikstart进行模型推理报错

[sunt40]

This modeling file requires the following packages that were not found in your environment: flash_attn. Run pip install flash_attn

[sunt40]

安装不了这个包

[wangzilong1997]

。。。。mac m1 一样这个问题

[shams2023]

如何解决？

[wytitan]

trust_remote_code=True, use_flash_attention_2=True
我把这两个都移除了可以加载了，参考https://huggingface.co/microsoft/phi-1_5/discussions/72

我在纯CPU环境，改成了AutoModelForCausalLM.from_pretrained(model_name_or_path, device_map=device_map, torch_dtype=torch.float16)