As part of the ongoing #OpenSourceWeek, DeepSeek announced the release of DeepGEMM, a cutting-edge library designed for efficient FP8 General Matrix Multiplications (GEMMs). This library is tailored to support both dense and Mix-of-Experts (MoE) GEMMs, making it a powerful tool for V3\/R1 training and inference. With DeepGEMM, we aim to push the boundaries of performance and efficiency in AI workloads, furthering our commitment to advancing open-source innovation in the field.<\/p>\n \ud83d\ude80 Day 3 of #OpenSourceWeek<\/a>: DeepGEMM<\/p>\n Introducing DeepGEMM \u2013 an FP8 GEMM library that supports both dense and MoE GEMMs, powering V3\/R1 training and inference.<\/p>\n \u26a1 Up to 1350+ FP8 TFLOPS on Hopper GPUs \u2014 DeepSeek (@deepseek_ai) February 26, 2025<\/a><\/p><\/blockquote>\n<\/div>\n<\/figure>\n This release marks Day 3 of our Open Source Week celebrations, following the successful launches of DeepSeek FlashML on Day 1 <\/a>and DeepSeek DeepEP on Day 2<\/a>.<\/p>\n General Matrix Multiplication (GEMM) is a operation that takes two matrices and multiplies them by storing the result into a third matrix. It is a fundamental operation in Linear Algebra, widely used in various applications. Its formula is\u00a0<\/p>\n GEMM is critical for optimizing the performance of the models. It is particularly useful in Deep learning, where it is mostly used in training and inference of neural networks. <\/p>\n This image depicts GEMM (General Matrix Multiplication), showing matrices A, B, and the resulting C. It highlights tiling, dividing matrices into smaller blocks (Mtile, Ntile, Ktile) for optimized cache usage. The blue and yellow tiles illustrate the multiplication process, contributing to the green \u201cBlock_m,n\u201d tile in C. This technique improves performance by enhancing data locality and parallelism.<\/p>\n FP8, or 8-bit floating point, is a format designed for high-performance computing which allows reduced precision as well as efficient representation of numerical data with real values. Huge datasets can result in high computational overload in machine learning and deep learning applications, this is where FP8 plays a vital role by reducing the computational complexity.<\/p>\n The FP8 format typically consists of:<\/p>\n This compact representation allows for faster computations and reduced memory usage, making it ideal for training large models on modern hardware. The trade-off is a potential loss of precision, but in many deep learning scenarios, this loss is acceptable and can even lead to improved performance due to reduced computational load.<\/p>\n This image illustrates FP8 (8-bit Floating Point) formats, specifically E4M3 and E5M2, alongside FP16 and BF16 for comparison. It shows how FP8 representations allocate bits for sign, exponent, and mantissa, affecting precision and range. E4M3 uses 4 exponent bits and 3 mantissa bits, while E5M2 uses 5 and 2 respectively. The image highlights the trade-offs in precision and range between different floating-point formats, with FP8 offering reduced precision but lower memory footprint.<\/p>\n DeepGEMM addresses the challenges in Matrix Multiplication by providing a lightweight, high-performance library that is easy to use and flexible enough to handle a variety of GEMM operations.<\/p>\n DeepGEMM stands out with its impressive features:<\/p>\n DeepGEMM has been rigorously tested across various matrix shapes, demonstrating significant speedups compared to existing implementations. Below is a summary of performance metrics:<\/p>\n \n
\u2705 No heavy dependency, as clean as a tutorial
\u2705 Fully Just-In-Time compiled\u2026<\/p>\nWhat is GEMM?<\/h2>\n
![\"\"](\"https:\/\/cdn.analyticsvidhya.com\/wp-content\/uploads\/2025\/02\/What-is-GEMM_.webp\")
<\/figure>\n<\/div>\nWhat is FP8?<\/h2>\n
\n
Need for DeepGEMM<\/h2>\n
\n
Key Features of DeepGEMM<\/h2>\n
\n
Performance Metrics<\/h2>\n
\n
\n
\n \n \n
![\"Harsh](\"https:\/\/av-eks-lekhak.s3.amazonaws.com\/media\/lekhak-profile-images\/converted_image_0fBqNLi.webp\")
<\/p>\n