![\"DeepSeek](\"https:\/\/static.wixstatic.com\/media\/d2c101_5974467bd3d64824bc2d7e5045060618~mv2.png\/v1\/fill\/w_48,h_28,al_c,q_85,usm_0.66_1.00_0.01,blur_2,enc_avif,quality_auto\/d2c101_5974467bd3d64824bc2d7e5045060618~mv2.png\")
<\/wow-image><\/div>\n<\/div>\n<\/figure>\n<\/div>\n<\/div>\n<\/div>\nA week on from the soon-to-be-legendary <\/span>market meltdown DeepSeek generated<\/span><\/a>, the noise is subsiding. With more data coming to light, we can finally start to parse out what\u2019s what, far from the money-people\u2019s frenzy and the swanky headlines.<\/span><\/span><\/p>\n<\/div>\n No. Many headlines have shared a $6m training cost figure for DeepSeek V3. <\/span>This is wrong<\/span><\/a>. The $6m does not include \u201c<\/span>costs associated with prior research and ablation experiments on architectures, algorithms and data<\/span><\/em>\u201d. The pre-training cost is a very narrow portion of the total cost. Excluded are important pieces of the puzzle like R&D and TCO of the hardware itself. Excluded are also potential subsidies from the Chinese state.<\/span><\/span><\/p>\n<\/div>\n Furthermore, comparison of training costs between models trained at different times is inherently flawed: training costs have been improving non-stop. First movers have always spent more\u2026 and it seems that <\/span>DeepSeek may have \u201c<\/span><\/a>distilled<\/span><\/em><\/a>\u201d OpenAI\u2019s models<\/span><\/a>. Newcomers stand on the shoulder of giants, that\u2019s just how science works.<\/span><\/span><\/p>\n<\/div>\n As Cohere co-founder <\/span>Nick Frosst said in late January<\/span><\/a>, \u201c<\/span>It\u2019s been clear for some time now that innovating and creating greater efficiencies\u200a\u2014\u200arather than just throwing unlimited compute at the problem\u200a\u2014\u200awill spur the next round of technology breakthroughs. This is a clarifying moment when people are realizing what\u2019s long been obvious<\/span><\/em>\u201d.<\/span><\/span><\/p>\n<\/div>\n We know that R1 is comparable to OpenAI\u2019s o1 from a quality perspective (on <\/span>some<\/span><\/em>\u00a0benchmarks, not all), although it lags o3.<\/span><\/span><\/p>\n<\/div>\n We also know that DeepSeek\u2019s models incorporate important breakthroughs, highlighting a path to more cost-effective AI.<\/span><\/span><\/p>\n<\/div>\n Of note\u00a0: FP8 mixed precision training, Multi-head Latent Attention (MLA), Multi-Token Prediction (MTP) and Auxiliary-loss-free load balancing\u2026 all of which increase efficiency.\u00a0<\/span><\/span><\/p>\n<\/div>\n There are however several factors ensuring that DeepSeek\u2019s overall GPU requirements are unlikely to decline:<\/span><\/span><\/p>\n<\/div>\n FP8 mixed-precision training is excellent for large language models (LLMs) due to its efficiency in handling massive datasets and parameter counts. For more complex tasks however, that require higher numerical stability, precision, or dynamic range, the same cannot be said.<\/span><\/span><\/p>\n<\/li>\n Multi-Token Prediction allows DeepSeek to predict multiple tokens simultaneously, improving inference throughput by up to 1.8x. This reduces the per-task GPU load during real-time applications like chatbots or coding assistants. This however comes with challenges like prediction errors, low acceptance rates, and increased verification complexity. These issues make it less suitable for applications requiring strict accuracy, fine-grained control, or high-context dependency (e.g., code generation, formal logic constructs).<\/span><\/span><\/p>\n<\/li>\n<\/ul>\n<\/div>\n This market is heated and looking for a reason to sneeze. The fact this news came from China is a bigger trigger than technological improvements.<\/span><\/span><\/p>\n<\/div>\n Firstly, we don\u2019t have enough power, capital, data centers, or chips to meet the demand as currently forecasted in the most optimistic cases (80 GW+). Thus, improvements had to happen.<\/span><\/span><\/p>\n<\/li>\n In addition, the market cannot <\/span>really <\/span><\/em>support 300kw racks with today\u2019s technology (liquid cooling is great but costly and carries risks). If efficiency improvements slow the curve of increasing power densities, we\u2019ll be very lucky indeed\u200a\u2014\u200athe alternative is a bunch of rapidly obsolete data center capacity and rapidly increasing per MW build costs.<\/span><\/span><\/p>\n<\/li>\n 30% improvement doesn\u2019t mean we need 30% fewer chips or data centers. It means we get 30% more power. Goldman Sachs has lamented that AI isn\u2019t delivering enough ROI\u200a\u2014\u200awell, this increases the gain significantly. There is a logical fallacy in IT, that a 10% efficiency gain means 10% less data center and 10% less server machines. And yet, that has been repeatedly disproven.<\/span><\/span><\/p>\n<\/li>\n<\/ul>\n<\/div>\n Yes. It\u2019s an over-used model, but useful. Any shift toward cheaper, more powerful, and less energy-intensive algorithms has the potential to significantly expand AI adoption \/ the total-addressable-market, which could ultimately fuel demand for both large-scale and distributed data center infrastructures.\u00a0<\/span><\/span><\/p>\n<\/div>\n That, in turn, means that A.I. companies may be able to achieve very powerful capabilities with far less investment than previously thought. And it suggests that we may soon see a flood of investment into smaller A.I. start-ups, and much more competition for the giants of Silicon Valley (which, because of the enormous costs of training their models, have mostly been competing with each other until now).<\/span><\/span><\/p>\n<\/div>\n Efficiencies reduce cost per task, but total GPU utilization increases as more tasks, larger models, and broader applications are adopted.<\/span><\/span><\/p>\n<\/div>\n This view has been backed by a few ecosystem players. \u201c<\/span>\u2060Because the value of having a more intelligent system is so high,<\/span><\/em>\u201d <\/span>wrote<\/span><\/a>\u00a0Anthropic cofounder Dario Amodei, it \u201c<\/span>causes companies to spend more, not less, on training models<\/span><\/em>\u201d. <\/span>Baxtel VP of Sales & Operations Mitch Lenzi<\/span><\/a>\u00a0would opine, saying that \u201c<\/span>innovation in AI doesn\u2019t reduce demand\u200a\u2014\u200ait fuels it<\/span><\/em>. <\/span>As AI becomes more accessible and cost-effective, the industry will see continued expansion, maintaining the need for high-performance data center infrastructure<\/span><\/em>\u201d.<\/span><\/span><\/p>\n<\/div>\n Training a model is only the beginning. Using it also uses compute \/ energy. In the same way that creating Google is one thing, but searching it is another.<\/span><\/span><\/p>\n<\/div>\n The first thing to say on this is that the cost of inferencing always needed to scale down to reach mass AI adoption. In fact, inference costs have been coming down every months; gains in efficiency are a given, and the landscape is not drastically changed for those who understand it. New breakthroughs (likely due to geopolitical tensions) can only accelerate what is already underway.<\/span><\/span><\/p>\n<\/div>\n BUT<\/span><\/em>\u2026 there are some indications that Deepseek models are less efficient for inference than they let on. In fact, the energy it saves in training is offset by its more intensive techniques for answering questions, and by the long answers they produce.<\/span><\/span><\/p>\n<\/div>\n Some preliminary tests have shown that, overall, when tested on 40 prompts, DeepSeek has a similar energy <\/span>efficiency<\/span><\/em>\u00a0to Meta models, but tends to generate much longer responses and therefore uses 87% more energy.<\/span><\/span><\/p>\n<\/div>\nIs the 30x price reduction a fair assessment?<\/span><\/span><\/h2>\n<\/p>\n
What do we know about DeepSeek performance?\u00a0<\/span><\/span><\/h2>\n<\/p>\n
\n
So\u2026 are the training improvements real?<\/span><\/span><\/h2>\n<\/p>\n
\n
Wait, are you talking about this Jevon\u2019s Paradox I keep hearing\u00a0about?<\/span><\/span><\/h2>\n<\/p>\n
Speaking of cost per task: what about inferencing?<\/span><\/span><\/h2>\n<\/p>\n