企业官网建设流程全解析

以从huggingface下载模型uer/gpt2-chinese-cluecorpussmall为例

使用 HF_ENDPOINT进行下载

临时生效：

# 当前终端export HF_ENDPOINT=https://hf-mirror.com

永久生效：

echo'export HF_ENDPOINT=https://hf-mirror.com'>>~/.bashrc source~/.bashrc

然后执行如下python脚本：

fromtransformersimportAutoTokenizer,AutoModel model_name="uer/gpt2-chinese-cluecorpussmall"tokenizer=AutoTokenizer.from_pretrained(model_name)model=AutoModel.from_pretrained(model_name)

指定下载地址

export HF_HOME=/data/hf export HF_ENDPOINT=https://hf-mirror.com

完全离线使用

huggingface-cli download uer/gpt2-chinese-cluecorpussmall \--local-dir/data/models/uer-gpt2 \--local-dir-use-symlinksFalse

离线加载

tokenizer=AutoTokenizer.from_pretrained("/data/models/uer-gpt2",local_files_only=True)model=AutoModel.from_pretrained("/data/models/uer-gpt2",local_files_only=True)