TensorRT-tensorrt插件Plugin的自动生成工具-Autogen-优质算法部署项目实战.zip

 # CUTLASS 2.7 _CUTLASS 2.7 - September 2021_ CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-multiplication (GEMM) and related computations at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS and cuDNN. CUTLASS decomposes these "moving parts" into reusable, modular software components abstracted by C++ template classes. These thread-wide, warp-wide, block-wide, and device-wide primitives can be specialized and tuned via custom tiling sizes, data types, and other algorithmic policy. The resulting flexibility simplifies their use as building blocks within custom kernels and applications. To support a wide variety of applications, CUTLASS provides extensive support for mixed-precision computations, providing specialized data-movement and multiply-accumulate abstractions for half-precision floating point (FP16), BFloat16 (BF16), Tensor Float 32 (TF32), single-precision floating point (FP32), double-precision floating point (FP64) types, integer data types (4b and 8b), and binary data types (1b). CUTLASS demonstrates warp-synchronous matrix multiply operations targeting the programmable, high-throughput _Tensor Cores_ implemented by NVIDIA's Volta, Turing, and Ampere architectures. CUTLASS implements high-performance Convolution via the implicit GEMM algorithm. Implicit GEMM is the formulation of a convolution operation as a GEMM thereby taking advantage of CUTLASS's modular GEMM pipeline. This allows CUTLASS to build convolutions by reusing highly optimized warp-wide GEMM components and below. See the [Quick Start Guide](/media/docs/quickstart.md) to get started quickly. See the [functionality listing](/media/docs/functionality.md) for the list of operations supported at each level of the execution model hierarchy. See the [CHANGELOG](CHANGELOG.md) for descriptions of recent updates. # What's New in CUTLASS 2.7 CUTLASS 2.7 is a minor update to CUTLASS adding: - Mainloop fusion for GEMM: [summation over A or B](/examples/23_ampere_gemm_operand_reduction_fusion/ampere_gemm_operand_reduction_fusion.cu) - [Optimizations for strided DGRAD](/include/cutlass/conv/kernel/default_conv2d_dgrad.h) - [Half-precision GELU_taylor activation functions](/include/cutlass/epilogue/thread/activation.h#L196) - Tuning and bug fixes to [fused GEMM + GEMM example](/examples/13_two_tensor_op_fusion/) - Support for smaller than 128b aligned Convolutions: [see examples](test/unit/conv/device/conv2d_fprop_implicit_gemm_f16nhwc_f16nhwc_f16nhwc_tensor_op_f16_sm80.cu#L272) - Caching of results to accelerate Convolution [unit tests](test/unit/conv/device/cache_testbed_output.h) - Numerous updates from the community (thanks!) # What's New in CUTLASS 2.6 CUTLASS 2.6 is a minor update to CUTLASS adding: - Fused [broadcast](test/unit/gemm/device/gemm_with_broadcast_f16n_f16n_f16n_tensorop_f32_sm75.cu) and [reductions](/test/unit/gemm/device/gemm_with_reduction_f16n_f16n_f16n_tensorop_f32_sm75.cu) in the epilogues of GEMM and Convolution - [Quaternion-valued GEMM](/examples/21_quaternion_gemm/quaternion_gemm.cu) and [Convolution](/examples/22_quaternion_conv/quaternion_conv.cu) in single-precision - [New strided Dgrad](test/unit/conv/device/conv2d_strided_dgrad_implicit_gemm_f16nhwc_f16nhwc_f32nhwc_tensor_op_f32_sm80.cu) implementation offers up to 4x performance improvements over previous strided Dgrad - 64-bit strides for large tensor allocations - [General affine layouts](/examples/18_ampere_fp64_tensorop_affine2_gemm/ampere_fp64_tensorop_affine2_gemm.cu) fp64 tensor core and simt GEMM - [Batched GEMV](/test/unit/gemm/device/gemv.cu) preview implementation - Enhanced functionality, boosted performance, and bug fixes in the epilogue. - Optimal performance when compiled with the [CUDA 11.4 Toolkit](https://developer.nvidia.com/cuda-toolkit) - Adopt new L2 prefetch feature in [ptx instruction](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#ptx-isa-version-7-4). - Enhanced Clang support and the combination of Clang 13 and CUDA 11.4 can build and run kernels from Pascal and Ampere. - Numerous updates from the community (thanks!) # What's New in CUTLASS 2.5 CUTLASS 2.5 is a minor update to CUTLASS adding: - [Tensor reductions](/test/unit/reduction/device/tensor_reduce_contiguous.cu) - [Optimizations for 3-D convolution](include/cutlass/conv/threadblock/conv3d_fprop_activation_tile_access_iterator_optimized.h) - [Fused Convolution+Convolution example](/examples/13_two_tensor_op_fusion/README.md) # What's New in CUTLASS 2.4 CUTLASS 2.4 is a significant update to CUTLASS adding: - 1-D, 2-D, and 3-D convolution targeting Tensor and CUDA cores for NVIDIA Ampere, Turing, and Volta GPU architectures - CUTLASS profiler support for convolution - [Documentation](/media/docs/implicit_gemm_convolution.md) describing Implicit GEMM Convolution algorithm and implementation # What's New in CUTLASS 2.3 CUTLASS 2.3 is a minor update to CUTLASS adding: - GEMMs targeting structured [Sparse Tensor Cores](test/unit/gemm/device/gemm_f16n_f16n_f32t_tensor_op_f32_sparse_sm80.cu) in NVIDIA Ampere Architecture GPUs - Fast SGEMM kernels targeting GeForce RTX 30-series CUDA Cores - Intended to be compiled with [CUDA 11.1 Toolkit](https://developer.nvidia.com/cuda-toolkit) or later # What's New in CUTLASS 2.2 CUTLASS 2.2 is a significant update to CUTLASS adding: - Coverage of [NVIDIA Ampere Architecture features](https://devblogs.nvidia.com/nvidia-ampere-architecture-in-depth/) - Tensor Core-accelerated GEMMs targeting Tensor Float 32, BFloat16, and double-precision data types - Deep software pipelines using asynchronous copy - Described in [GTC 2020 Webinar (SR 21745)](https://developer.nvidia.com/gtc/2020/video/s21745) - Intended to be compiled with [CUDA 11 Toolkit](https://developer.nvidia.com/cuda-toolkit) or later # What's New in CUTLASS 2.1 CUTLASS 2.1 is a minor update to CUTLASS adding: - [Planar complex GEMM kernels](/examples/10_planar_complex/planar_complex.cu) targeting Volta and Turing Tensor Cores - BLAS-style API to launch kernels compiled into the [CUTLASS Library](/media/docs/quickstart.md#cutlass-library) # What's New in CUTLASS 2.0 CUTLASS 2.0 is a substantial refactoring from the previous version, intended to offer: - Better performance over 1.x, particularly for kernels targeting Turing Tensor Cores - Robust and durable templates that reliably span the design space - Encapsulated functionality that may be reusable in other contexts **See the [CHANGELOG](CHANGELOG.md) for more details.** # Performance <p align="center"><img src=/media/images/cutlass-performance-plot.png></p> CUTLASS primitives are very efficient. When used to construct device-wide GEMM kernels, they exhibit performance comparable to cuBLAS for scalar GEMM computations. The above figure shows CUTLASS performance relative to cuBLAS for large matrix dimensions on an NVIDIA GeForce 2080 Ti, an NVIDIA A100, and an NVIDIA TitanV using CUDA 11.0 Toolkit. Tensor Core operations are implemented using CUDA's [mma instruction](https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#warp-level-matrix-instructions-mma). # Compatibility CUTLASS requires a C++11 host compiler and performs best when built with the [CUDA 11.4 Toolkit](https://developer.nvidia.com/cuda-toolkit). It is also compatible with CUDA 10.2, CUDA 11.0, CUDA 11.1, CUDA 11.2, and CUDA 11.3. We have tested the following environments. |**Operating System** | **Compiler** | |-----------------|----------| | Windows 10 | Microsoft Visual Studio 2015| | | Microsoft Visual Studio 2017| | Ubuntu 16.04 | GCC 5.4.0 | | Ubuntu 18.04 | GCC 7.5.0 | | Ubuntu 20.04 | GCC 10.2.0 | Additionally, CUTLASS may be built with clang. See [these instructi