{"id":7046132,"date":"2026-08-24T14:01:40","date_gmt":"2026-08-24T14:01:40","guid":{"rendered":"https:\/\/peraltafinancing.com\/analytics\/moonshot-ais-2-8t-open-weight-model\/"},"modified":"2026-08-24T14:01:40","modified_gmt":"2026-08-24T14:01:40","slug":"moonshot-ais-2-8t-open-weight-model","status":"publish","type":"post","link":"https:\/\/fivemor.com\/?p=7046132","title":{"rendered":"Moonshot AI\u2019s 2.8T Open-Weight Model"},"content":{"rendered":"


\n<\/p>\n

\n

Moonshot AI\u2019s Kimi K3 is a 2.8-trillion-parameter open-weight model built with a Mixture-of-Experts architecture. It activates only a small fraction of its parameters per token, helping reduce inference costs while delivering strong coding and agentic performance.<\/p>\n

K3 combines near-frontier capabilities, open weights<\/mark>, and lower API pricing, making it an interesting alternative to proprietary models. In this article, we\u2019ll explore how K3 works and show you two simple ways to use it: through Kimi\u2019s app or the Fireworks AI API.<\/p>\n

What Makes K3 Different?<\/h2>\n
\n\n\n\n\n\n\n\n\n\n
Specification<\/th>\nDetails<\/th>\n<\/tr>\n<\/thead>\n
Total Parameters<\/strong><\/td>\n2.8 trillion<\/td>\n<\/tr>\n
Active Experts per Token<\/strong><\/td>\n16 of 896 (~1.8% of total experts)<\/td>\n<\/tr>\n
Context Window<\/strong><\/td>\n1,048,576 tokens (1M)<\/td>\n<\/tr>\n
Modality<\/strong><\/td>\nNative text + vision<\/td>\n<\/tr>\n
License<\/strong><\/td>\nOpen-weight; first open 3T-class model<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

Two decisions in architecture are notable:<\/strong>\u00a0<\/p>\n

    \n
  • Kimi Delta Attention (KDA): <\/strong>A hybrid linear attention technique created to render the context window of 1M tokens feasible instead of quadratic.\u00a0<\/li>\n
  • Attention Residuals:<\/strong> A complete substitute for standard residual connections that Moonshot claims lead to more consistent results as the model grows larger.\u00a0<\/li>\n<\/ul>\n

    K3 uses a sparse Mixture-of-Experts architecture that activates only a small fraction of its parameters for each token. This gives the model a massive parameter count while keeping inference relatively efficient compared with other trillion-parameter models. Early benchmarks suggest that K3 can compete with advanced proprietary models on coding and agentic tasks, with particularly strong performance on long-context coding and blind coding evaluations.<\/p>\n

    How to Access K3 and What it Costs?\u00a0<\/h2>\n

    There are two ways to access K3, and each is suited to a different use case.<\/p>\n

      \n
    • Kimi App:<\/strong> Available on iOS and Android through Kimi.com, as well as through the Kimi Work desktop application. No developer account is required, and you can start with the free tier, which comes with usage limits. It\u2019s a good option if you simply want to explore the model and see how it performs.<\/li>\n
    • API Access:<\/strong> Available through the Kimi web platform and Fireworks AI. This option uses pay-per-token pricing and is better suited for developers who want to integrate K3 into their own applications or workflows.<\/li>\n<\/ul>\n

      Fireworks offers three serving paths, depending on how you want to balance latency and cost:<\/p>\n

      \n\n\n\n\n\n\n\n
      Path<\/th>\nUncached Input<\/th>\nCached Input<\/th>\nOutput<\/th>\n<\/tr>\n<\/thead>\n
      Standard<\/strong><\/td>\n$3.00 \/ M<\/td>\n$0.30 \/ M<\/td>\n$15.00 \/ M<\/td>\n<\/tr>\n
      Priority<\/strong><\/td>\n$3.75 \/ M<\/td>\n$0.375 \/ M<\/td>\n$18.75 \/ M<\/td>\n<\/tr>\n
      Fast<\/strong><\/td>\n$4.50 \/ M<\/td>\n$0.45 \/ M<\/td>\n$22.50 \/ M<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

      Cached input is priced at roughly one-tenth the cost of non-cached input. At scale, it\u2019s therefore worth structuring your prompts, including system prompts and reusable context, to take advantage of caching.<\/p>\n

      For most use cases, Standard<\/strong> is the best starting point. Choose Fast<\/strong> when latency is critical, or Priority<\/strong> when you need higher reliability.<\/p>\n

      If you prefer to self-host K3, Moonshot recommends a supernode with at least 64 accelerators<\/strong> to create a high-speed environment where the experts can operate within a single domain. This is a significantly larger infrastructure commitment than self-hosting models in the 700B\u20131.6T parameter range.<\/p>\n

      Now that we know everything about the model, let\u2019s give it a try:<\/p>\n

      Task 1: Talking to K3 Directly (No API Key Needed)\u00a0<\/h2>\n

      The quickest way to experience K3 is through Kimi\u2019s consumer interface. You can use the Kimi app on iOS and Android, access it through kimi.com, or use the Kimi desktop app for work. No developer account or billing details are required. You can simply open the chat interface and start experimenting.<\/p>\n

      For a more meaningful test, don\u2019t just ask K3 questions about general knowledge. Instead, give it a small project or ask it to build a simple system. That\u2019s where K3\u2019s coding and agentic capabilities really come into play.<\/p>\n

      Input Prompt:<\/strong> <\/p>\n

      \n

      Build a single-file HTML page for a typing speed test \u2014 it should show a random sentence, start a timer on the first keystroke, and display words-per-minute and accuracy when I finish typing. Make it look clean, not default-Bootstrap. Then explain the trickiest part of the logic in two sentences.<\/p>\n<\/blockquote>\n

      Output:<\/strong><\/p>\n

      \n