Kimi K3 boasts 2.8 trillion parameters, effectively activating 16 out of 896 experts when processing each token, employing MXFP4 weights and MXFP8 activation values. The sparse expert architecture reduces actual computational load, and low-precision weights compress storage usage, but it still cannot eliminate the memory pressure caused by the massive parameter scale. A single DGX B200 has 1.44TB of memory, theoretically only approaching the basic size of a four-bit weight in K3; after accounting for cache, activation values, and runtime overhead, it's difficult to fully handle this. The B300 and MI355X each offer 288GB of memory per GPU, making them more suitable for deploying ultra-large models.
A greater bottleneck lies in communication. While multiple B200s can collectively hold the model, experts distributed across different GPUs need to frequently exchange data. Cross-server data transfer efficiency is significantly lower than the high-speed NVLink within the GB300 NVL72. Therefore, Kimi officially recommends using supernodes composed of 64 or more accelerators; high-density deployment also requires supporting high-speed interconnects, parallel software stacks, and corresponding power supply and cooling systems.
In summary: saves computing power, consumes a lot of video memory, and emphasizes interconnectivity.
