6828301b02 更新 'README.md'
PCL_JinWang
JinWang
- Joined on Apr 26, 2021
- Organization
-
-
-
42
-
0
-
-
5
-
0
-
-
3
-
0
-
-
27
-
9
-
-
24
-
0
-
-
32
-
2
-
-
11
-
23
-
-
76
-
5
-
Loading Heatmap…
JinWang pushed to master at LLM-Research/DataQuality
6 months ago
JinWang commented on issue PCL-Platform.Inte.../AISynergy#20
梯度量化方法（BinarySGD+TrinarySGD）2022-06-24: * 1、学习Pytorch中DDP模块的通信hook实现机制，参考完成BinarySGD代码编写，存在问题是：Pytorch中没有1bit数据类型 * 2、研究学习DeepSpeed中的1bit-Adam原理和实现源码，其中1bit数据采用了基于CUDA实现的cupy多维数组库来实现
1 year ago
JinWang commented on issue PCL-Platform.Inte.../AISynergy#7
协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现Colossal-AI框架模型训练效率仿真方法（基于事件时序建模的方法）主要步骤： 1、单机模拟多机多卡的分布式进程环境及模型加载；（已完成） 2、根据加载的模型获取每个进程的计算图； 3、根据计算图表示检测事件，包括计算和通信； 4、建立事件时序模型计算整体效率。
1 year ago
JinWang commented on issue PCL-Platform.Inte.../AISynergy#7
协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现[5-20：DistIR仿真与Colossal-AI真实训练效率对比](https://git.openi.org.cn/PCL-Platform.Intelligence/AISynergy/src/branch/AISyn-simulator/examples/simulation_example) （1）DistIR仿真MLP-small（4GPU，数据并行）  （2）Colossal-AI真实执行MLP-small(4GPU,数据并行,只包括了forward和backward)  Colossal-AI Benchmark（MLP-small）  （3）Colossal-AI GPT2真实训练效率（4GPU，数据并行）  对比分析： * MLP-small模型真实执行效率随batch size变化规律与DistIR仿真结果不一致，初步分析是仿真方法DistIR的问题，正在分析定位； * Colossal-AI在4GPU上随着并行维度增加，速度降低，但是能够支持更大的batchsize
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
- Compare 4 commits »
1 year ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago
JinWang commented on issue PCL-Platform.Inte.../AISynergy#7
协同计算任务各阶段耗时统计标准化+预执行回归建模仿真方案实现[simulator README](https://git.openi.org.cn/PCL-Platform.Intelligence/AISynergy/src/branch/AISyn-simulator/examples/simulation_exampleb)
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
- Compare 2 commits »
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago
JinWang pushed to AISyn-simulator at PCL-Platform.Inte.../AISynergy
2 years ago