Are you sure you want to delete this task? Once this task is deleted, it cannot be recovered.
ver217 823f3b9cf4 | 1 year ago | |
---|---|---|
.. | ||
README.md | 1 year ago | |
__init__.py | 1 year ago | |
builder.py | 1 year ago | |
cpu_adam.py | 1 year ago | |
fused_optim.py | 1 year ago | |
layernorm.py | 1 year ago | |
moe.py | 1 year ago | |
multi_head_attn.py | 1 year ago | |
scaled_masked_softmax.py | 1 year ago | |
scaled_upper_triangle_masked_softmax.py | 1 year ago | |
utils.py | 1 year ago |
Building PyTorch extensions can be a difficult task for users not from the system background. It is definitely frustrating if the users encounter many strange technical jargons when install Colossal-AI. Therefore, we will provide two methods of building the PyTorch extensions for the users.
pip install
if CUDA_EXT=1
The first method is more suitable for users who are familiar with CUDA environment configurations. The second method is for those who are not as they only need to build the kernel which is required by their program.
These two methods have different advantages and disadvantages.
Method 1 is good because it allows the user to build all kernels during installation and directly import the kernel. They don't need to care about kernel building when running their program. However, installation may fail if they don't know how to configure their environments and this leads to much frustration.
Method 2 is good because it allows the user to only build the kernel they actually need, such that there is a lower probability that they encounter environment issue. However, it may slow down their program due to the first build and subsequence load.
The project DeepSpeed (https://github.com/microsoft/DeepSpeed) has proposed a solution) to support kernel-build during either installation or runtime.
We have adapted from DeepSpeed's solution to build extensions. The extension build requries two main functions from PyTorch:
torch.utils.cpp_extension.CUDAExtension
: used to build extensions in setup.py
during pip install
.torch.utils.cpp_extension.load
: used to build and load extension during runtimePlease note that the extension build by CUDAExtension
cannot be loaded by the load
function and load
will run its own build again (correct me if I am wrong).
Based on the DeepSpeed's work, we have make several modifications and improvements:
setup.py
) will be found in colossalai._C
When loading the built kernel, we will first check if the pre-built one exists. If not, the runtime build will be triggered.
No Description
Python Cuda C++ Shell C
Dear OpenI User
Thank you for your continuous support to the Openl Qizhi Community AI Collaboration Platform. In order to protect your usage rights and ensure network security, we updated the Openl Qizhi Community AI Collaboration Platform Usage Agreement in January 2024. The updated agreement specifies that users are prohibited from using intranet penetration tools. After you click "Agree and continue", you can continue to use our services. Thank you for your cooperation and understanding.
For more agreement content, please refer to the《Openl Qizhi Community AI Collaboration Platform Usage Agreement》