DeepGEMM: clean and efficient FP8 GEMM kernels with fine-grained scaling

📚200+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).

Sample codes for my CUDA programming book

This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, s…

Optimizing SGEMM kernel functions on NVIDIA GPUs to a close-to-cuBLAS performance.

CUDA C 编程权威指南代码实现 包含了书上第二章到第八章的大部分代码实现和作者笔记，全由作者本人手动实现，难免有错误的地方，请大家谨慎参考，非常欢迎对错误的指正。 如果有帮助的话请Star一下，对作者帮助很大，谢谢！

Yinghan's Code Sample

基于《cuda编程-基础与实践》（樊哲勇 著）的cuda学习之路。

GEMM and Winograd based convolutions using CUTLASS