NVIDIA开源面向精简混合精度和分布式训练的Pytorch扩展-python
需积分: 38 147 浏览量
2021-06-18
18:06:57
上传
评论
收藏 760KB ZIP 举报
NVIDIA开源面向精简混合精度和分布式训练的Pytorch扩展-python （341个子文件） 
bottleneck.cpp 66KB encdec_multihead_attn_norm_add.cpp 12KB fmha_api.cpp 11KB self_multihead_attn_norm_add.cpp 10KB encdec_multihead_attn.cpp 9KB self_multihead_attn_bias_additive_mask.cpp 7KB self_multihead_attn_bias.cpp 7KB interface.cpp 7KB self_multihead_attn.cpp 7KB layer_norm_cuda.cpp 6KB syncbn.cpp 6KB fused_adam_cuda.cpp 5KB mlp.cpp 5KB masked_softmax_dropout.cpp 4KB amp_C_frontend.cpp 4KB additive_masked_softmax_dropout.cpp 4KB ln_api.cpp 3KB transducer_loss.cpp 2KB transducer_joint.cpp 2KB interface.cpp 2KB multi_tensor_distopt_lamb.cpp 1KB flatten_unflatten.cpp 584B fused_lamb_cuda.cpp 562B multi_tensor_distopt_adam.cpp 560B pytorch_theme.css 2KB welford.cu 53KB mlp_cuda.cu 51KB fused_adam_cuda_kernel.cu 34KB encdec_multihead_attn_norm_add_cuda.cu 30KB transducer_joint_kernel.cu 27KB transducer_loss_kernel.cu 26KB self_multihead_attn_norm_add_cuda.cu 25KB encdec_multihead_attn_cuda.cu 25KB layer_norm_cuda_kernel.cu 24KB xentropy_kernel.cu 24KB self_multihead_attn_bias_cuda.cu 21KB self_multihead_attn_cuda.cu 20KB self_multihead_attn_bias_additive_mask_cuda.cu 20KB ln_bwd_semi_cuda_kernel.cu 16KB multi_tensor_distopt_lamb_kernel.cu 15KB multi_tensor_l2norm_kernel.cu 13KB multi_tensor_lamb.cu 12KB batch_norm_add_relu.cu 12KB batch_norm.cu 11KB fused_lamb_cuda_kernel.cu 8KB multi_tensor_sgd_kernel.cu 8KB multi_tensor_distopt_adam_kernel.cu 7KB masked_softmax_dropout_cuda.cu 6KB ln_fwd_cuda_kernel.cu 5KB multi_tensor_novograd.cu 5KB additive_masked_softmax_dropout_cuda.cu 5KB multi_tensor_axpby_kernel.cu 5KB multi_tensor_adam.cu 4KB multi_tensor_lamb_stage_1.cu 4KB multi_tensor_scale_kernel.cu 4KB ipc.cu 4KB multi_tensor_lamb_stage_2.cu 3KB fmha_dgrad_fp16_512_64_kernel.sm80.cu 3KB fmha_dgrad_fp16_128_64_kernel.sm80.cu 3KB fmha_dgrad_fp16_256_64_kernel.sm80.cu 3KB fmha_dgrad_fp16_384_64_kernel.sm80.cu 3KB fmha_fprop_fp16_256_64_kernel.sm80.cu 3KB fmha_fprop_fp16_128_64_kernel.sm80.cu 3KB fmha_fprop_fp16_512_64_kernel.sm80.cu 3KB fmha_fprop_fp16_384_64_kernel.sm80.cu 3KB multi_tensor_adagrad.cu 3KB multi_tensor_apply.cuh 5KB utils.cuh 3KB Dockerfile 760B .gitignore 51B .gitignore 31B .gitmodules 306B softmax.h 116KB nhwc_batch_norm_kernel.h 109KB smem_tile.h 50KB strided_batched_gemm.h 34KB utils.h 31KB batch_norm.h 28KB batch_norm_add_relu.h 26KB fmha_dgrad_kernel_1xN_reload.h 24KB layer_norm.h 23KB softmax.h 18KB gmem_tile.h 16KB fmha_fprop_kernel_1xN.h 13KB fmha_fprop_kernel_1xN_reload_v.h 13KB dropout.h 12KB gemm.h 12KB kernel_traits.h 5KB type_shim.h 5KB fmha_utils.h 4KB fmha_kernel.h 4KB fmha.h 3KB mask.h 3KB philox.h 3KB ln_kernel_traits.h 910B cuda_utils.h 288B compat.h 140B
layout.html 1019B LICENSE 1KB Makefile 947B共 341 条
- 1
- 2
- 3
- 4
资源评论
