Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
You can not select more than 25 topics
Topics must start with a chinese character,a letter or number, can include dashes ('-') and can be up to 35 characters long.
imyzx2017
d8623c9e86
|
2 years ago | |
|---|---|---|
| gpt2-changing-gates-gifs | 2 years ago | |
| src | 2 years ago | |
| Masked_pangu_eachTask.py | 2 years ago | |
| dataset_restore_data0.py | 2 years ago | |
| gate_pruning_pangu.py | 2 years ago | |
| grad_pangu_eachTask.py | 2 years ago | |
| jieba-0.42.1.tar.gz | 2 years ago | |
| ma-pre-start.sh | 2 years ago | |
| readme.md | 2 years ago | |
| sentencepiece-0.1.94-cp37-cp37m-linux_aarch64.whl | 2 years ago | |
| task_05_CFT_Masked_inference.py | 2 years ago | |
| tokenization_jieba.py | 2 years ago | |
| utils_fix.py | 2 years ago | |
readme.md
鹏程·盘古剪枝探索
「鹏程·盘古」在NPU服务器上的剪枝探索，在多头attention的head维度尝试了逐个head剪枝 / 全局随机剪枝 / 计算head_importance再剪枝的三种方法
[训练脚本]
环境要求
华为NPU ASCEND 910A 一张，框架环境为mindspore-1.3
采样剪枝方法
python task_05_CFT_Masked_inference.py
[逐个head剪枝]
修改TASK_METHOD=0
[逐个head剪枝]
修改TASK_METHOD=1
head重要性评估
python grad_pangu_eachTask.py
[遍历数据集计算head重要性分数矩阵]
【根据head_importance生成mask配置】
python task_05_CFT_Masked_inference.py
修改TASK_METHOD=3
[head重要性剪枝]
GatePruning in Megatron-GPT2-345M
GatePruning in 盘古350M
许可证
支持在鹏城云脑2上，或NPU服务器上，进行盘古2.6B大模型剪枝实验
Python Shell
Contributors (2)
