DeepSpeed：DeepSpeed是一个深度学习优化库，可让分布式培训变得轻松，高效，有效 （352个子文件） 
.clang-format 5KB CNAME 17B CODEOWNERS 141B ds_transformer_cuda.cpp 45KB cpu_adam.cpp 33KB utils.cpp 4KB fused_lamb_cuda.cpp 4KB fused_adam_frontend.cpp 781B flatten_unflatten.cpp 742B csrc 10B normalize_kernels.cu 75KB dropout_kernels.cu 30KB softmax_kernels.cu 23KB transform_kernels.cu 22KB fused_lamb_cuda_kernel.cu 15KB general_kernels.cu 14KB gelu_kernels.cu 12KB cublas_wrappers.cu 8KB multi_tensor_adam.cu 6KB custom_cuda_kernel.cu 528B multi_tensor_apply.cuh 5KB deepspeed 2B Dockerfile 8KB ds 105B ds_elastic 2KB ds_report 100B ds_ssh 438B Gemfile 612B
softmax_animation.gif 3.03MB layernorm_deepspeed.gif 2.44MB softmax_deepspeed.gif 2.3MB layernorm_pytorch.gif 1.75MB layernorm_animation.gif 1.35MB softmax_pytorch.gif 1.29MB .gitignore 372B .gitmodules 129B custom_cuda_layers.h 10KB gemm_test.h 9KB normalize_layer.h 7KB context.h 7KB strided_batch_gemm.h 6KB ds_transformer_cuda.h 6KB type_shim.h 5KB cpu_adam.h 5KB feed_forward.h 3KB cublas_wrappers.h 3KB dropout.h 2KB StopWatch.h 2KB softmax.h 2KB general_kernels.h 1KB Timer.h 1KB gelu.h 955B compat.h 287B
news-home.html 504B 404.html 419B index.html 26B MANIFEST.in 174B ds_config_func_bs8_zero2_offload.json 537B ds_config_func_bs8_zero0_gas3.json 526B ds_config_func_bs8_zero2_gas3.json 526B ds_config_func_bs8_zero2.json 512B ds_config_func_scheduler.json 449B ds_config_func_bs4_zero2_offload.json 422B ds_config_func_bs4_zero2.json 397B deepspeed_bsz24_fp16_config.json 344B deepspeed_bsz24_fp16_zero2_config.json 344B ds_config_perf_bs16.json 326B ds_config_perf_bs32.json 326B ds_config_perf_bs8.json 325B deepspeed_bsz24_fp32_config.json 298B ds_config_func_bs4_zero1.json 296B ds_config_func_bs8_no_zero.json 296B ds_config_func_bs8_zero1.json 296B ds_batch_config.json 277B azure_config.json 146B LICENSE 1KB Gemfile.lock 8KB Makefile 638B config-json.md 21KB sparse-attention.md 18KB index.md 17KB 2020-05-28-fastest-bert-training.md 17KB megatron.md 17KB bert-pretraining.md 16KB bert-finetuning.md 16KB README.md 14KB 2020-09-09-onebit-adam-blog-post.md 13KB onebit-adam.md 13KB features.md 12KB pipeline.md 12KB cifar-10.md 12KB README.md 12KB getting-started.md 10KB 2020-09-09-sparse-attention.md 9KB transformer_kernel.md 9KB progressive_layer_dropping.md 8KB zero.md 7KB lrrt.md 7KB 1Cycle.md 6KB azure.md 6KB共 352 条
- 1
- 2
- 3
- 4
评论10
最新资源