顶点：PyTorch扩展：在Pytorch中易于混合精度和分布式培训的工具
需积分: 30 41 浏览量
2021-02-20
02:24:01
上传
评论
收藏 648KB ZIP 举报
顶点：PyTorch扩展：在Pytorch中易于混合精度和分布式培训的工具 （294个子文件） 
encdec_multihead_attn_norm_add.cpp 12KB self_multihead_attn_norm_add.cpp 10KB encdec_multihead_attn.cpp 9KB self_multihead_attn_bias_additive_mask.cpp 7KB self_multihead_attn_bias.cpp 7KB interface.cpp 7KB self_multihead_attn.cpp 7KB layer_norm_cuda.cpp 6KB syncbn.cpp 6KB fused_adam_cuda.cpp 5KB mlp.cpp 4KB masked_softmax_dropout.cpp 4KB amp_C_frontend.cpp 4KB additive_masked_softmax_dropout.cpp 4KB interface.cpp 2KB multi_tensor_distopt_lamb.cpp 1KB flatten_unflatten.cpp 584B fused_lamb_cuda.cpp 562B multi_tensor_distopt_adam.cpp 560B pytorch_theme.css 2KB welford.cu 53KB mlp_cuda.cu 41KB fused_adam_cuda_kernel.cu 34KB encdec_multihead_attn_norm_add_cuda.cu 30KB self_multihead_attn_norm_add_cuda.cu 25KB encdec_multihead_attn_cuda.cu 25KB layer_norm_cuda_kernel.cu 24KB xentropy_kernel.cu 24KB self_multihead_attn_bias_cuda.cu 21KB self_multihead_attn_cuda.cu 20KB self_multihead_attn_bias_additive_mask_cuda.cu 20KB multi_tensor_distopt_lamb_kernel.cu 14KB multi_tensor_lamb.cu 12KB multi_tensor_l2norm_kernel.cu 12KB batch_norm_add_relu.cu 12KB batch_norm.cu 11KB fused_lamb_cuda_kernel.cu 8KB multi_tensor_sgd_kernel.cu 8KB multi_tensor_distopt_adam_kernel.cu 7KB masked_softmax_dropout_cuda.cu 6KB multi_tensor_novograd.cu 5KB additive_masked_softmax_dropout_cuda.cu 5KB multi_tensor_axpby_kernel.cu 5KB multi_tensor_adam.cu 4KB multi_tensor_lamb_stage_1.cu 4KB multi_tensor_scale_kernel.cu 4KB ipc.cu 4KB multi_tensor_lamb_stage_2.cu 3KB multi_tensor_adagrad.cu 3KB multi_tensor_apply.cuh 5KB Dockerfile 760B .gitignore 38B .gitignore 31B .gitmodules 166B softmax.h 116KB nhwc_batch_norm_kernel.h 109KB strided_batched_gemm.h 34KB batch_norm.h 28KB batch_norm_add_relu.h 26KB layer_norm.h 23KB dropout.h 12KB type_shim.h 5KB philox.h 3KB cuda_utils.h 288B compat.h 140B
layout.html 1019B LICENSE 1KB Makefile 947B README.md 17KB README.md 9KB README.md 6KB README.md 3KB README.md 3KB README.md 2KB README.md 2KB README.md 2KB README.md 2KB README.md 2KB README.md 1KB README.md 762B FAQs.md 529B README.md 446B README.md 393B README.md 315B README.md 124B README.md 22B README.md 22B .nojekyll 0B
MHA_bwd.png 85KB MHA_fwd.png 82KB nv-pytorch2.png 6KB test_fused_sgd.py 38KB test_multiple_models_optimizers_losses.py 36KB distributed_fused_lamb.py 35KB distributed_fused_adam.py 34KB distributed_fused_adam_v2.py 31KB distributed.py 30KB fp16_optimizer.py 28KB setup.py 27KB test_pyprof_nvtx.py 27KB共 294 条
- 1
- 2
- 3
资源评论
