JinWang
Loading Heatmap…

JinWang commented on issue PCL-Platform.Inte.../AISynergy#20

梯度量化方法(BinarySGD+TrinarySGD)

2022-06-24: * 1、学习Pytorch中DDP模块的通信hook实现机制,参考完成BinarySGD代码编写,存在问题是:Pytorch中没有1bit数据类型 * 2、研究学习DeepSpeed中的1bit-Adam原理和实现源码,其中1bit数据采用了基于CUDA实现的cupy多维数组库来实现

1 week ago

JinWang opened issue PCL-Platform.Inte.../AISynergy#20

梯度量化方法(BinarySGD+TrinarySGD)

1 week ago

JinWang commented on issue PCL-Platform.Inte.../AISynergy#7

协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现

Colossal-AI框架模型训练效率仿真方法(基于事件时序建模的方法)主要步骤: 1、单机模拟多机多卡的分布式进程环境及模型加载;(已完成) 2、根据加载的模型获取每个进程的计算图; 3、根据计算图表示检测事件,包括计算和通信; 4、建立事件时序模型计算整体效率。

1 month ago

JinWang commented on issue PCL-Platform.Inte.../AISynergy#7

协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现

[5-20:DistIR仿真与Colossal-AI真实训练效率对比](https://git.openi.org.cn/PCL-Platform.Intelligence/AISynergy/src/branch/AISyn-simulator/examples/simulation_example)![](http://) (1)DistIR仿真MLP-small(4GPU,数据并行) ![图片](/attachments/0f6c359f-44cc-4cf8-b77d-9bf8b26bf0b5) (2)Colossal-AI真实执行MLP-small(4GPU,数据并行,只包括了forward和backward) ![图片](/attachments/a538198d-8977-498f-b64c-220b7f70eb5b) Colossal-AI Benchmark(MLP-small) ![图片](/attachments/f66c1b38-9b59-42b0-8437-411018e54e1c) (3)Colossal-AI GPT2真实训练效率(4GPU,数据并行) ![图片](/attachments/abe3bea0-06a0-4416-955a-aac2022402e2) 对比分析: * MLP-small模型真实执行效率随batch size变化规律与DistIR仿真结果不一致,初步分析是仿真方法DistIR的问题,正在分析定位; * Colossal-AI在4GPU上随着并行维度增加,速度降低,但是能够支持更大的batchsize

1 month ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 60a200ba9c 更新 'examples/simulation_example/README.md'

1 month ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • c0e5c56ee1 更新 'examples/simulation_example/README.md'

1 month ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • bac37058a7 更新 'examples/simulation_example/README.md'

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • e4653599ab 更新 'examples/simulation_example/README.md'

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 6b21ab0b6e 删除 'AISynergy-core/whls/X86_64/AISyncore-0.1.0-py3-none-any.whl'

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 37d1007c1a 删除 'AISynergy-core/whls/ARM/AISyncore-0.1.0-py3-none-any.whl'

2 months ago

JinWang commented on issue PCL-Platform.Inte.../AISynergy#7

协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现

[simulator README](https://git.openi.org.cn/PCL-Platform.Intelligence/AISynergy/src/branch/AISyn-simulator/examples/simulation_exampleb)

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 1062ce9b4f 更新 'examples/simulation_example/README.md'

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 3e35ddeb47 更新 'examples/simulation_example/README.md'

2 months ago

JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy

  • 72563b12e5 更新 'examples/simulation_example/README.md'

2 months ago