PCL_JinWang JinWang
Loading Heatmap…

JinWang pushed to master at LLM-Research/DataQuality

1 year ago

JinWang pushed to master at LLM-Research/DataQuality

1 year ago

JinWang pushed to master at LLM-Research/DataQuality

1 year ago

JinWang commented on issue PCL-Platform.Inte.../AISynergy#20

梯度量化方法(BinarySGD+TrinarySGD)

2022-06-24: * 1、学习Pytorch中DDP模块的通信hook实现机制,参考完成BinarySGD代码编写,存在问题是:Pytorch中没有1bit数据类型 * 2、研究学习DeepSpeed中的1bit-Adam原理和实现源码,其中1bit数据采用了基于CUDA实现的cupy多维数组库来实现

2 years ago

JinWang opened issue PCL-Platform.Inte.../AISynergy#20

梯度量化方法(BinarySGD+TrinarySGD)

2 years ago