Moore Threads Scales AI Systems to Support 10,000 Accelerators Amid US Export Restrictions

Moore Threads Scales AI Systems to Support 10,000 Accelerators Amid US Export Restrictions
Image credit: Moore Threads website 

Chinese GPU vendor Moore Threads has announced a significant advancement in its datacenter-focused AI systems, now capable of supporting clusters of up to 10,000 accelerators. This represents a tenfold increase from the previous year. The company revealed this development in a statement to the South China Morning Post, positioning itself as a key player in providing Chinese buyers with alternatives to technologies restricted by US export policies.

In 2023, Moore Threads was added to the US Entities list, which restricts its access to American technologies without a special license. Despite these limitations, the four-year-old startup aims to lead the Chinese market in AI accelerators without relying on US tech. In December 2023, it introduced the MTT S4000 chips, featuring 8,192 vector cores, 128 tensor cores, and a memory bandwidth of 768GB/sec with 48GB of vRAM per card.

Initially, Moore Threads' technology supported clusters of up to 1,000 units. To reach the current 10,000 GPU cluster capability, 1,250 MCCX D800 servers would need to be interconnected using a high-speed network. Each server in its KUAE clusters is equipped with dual 400Gbit/sec InfiniBand, although future configurations might require homegrown alternatives due to ongoing US restrictions.

While Moore Threads has made significant strides, its technology still lags behind Nvidia’s leading products. Nvidia’s H200, for example, offers nearly three times the memory and ten times the dense floating point performance of Moore Threads’ S4000. Nonetheless, US trade restrictions have limited Nvidia's ability to sell its most powerful accelerators in China, creating a more level playing field.

Despite these challenges, Moore Threads continues to progress and has secured 2.5 billion yuan ($343.9 million) in financing. The company has also formed partnerships with state-run telecoms, including China Mobile, China Unicom, and China Energy Engineering Corp., to build compute clusters. 

China aims to boost its national compute capacity by 30 percent to 300 exaFLOPS by 2025, and Moore Threads' advancements are likely to support this goal. However, the lack of a domestic high-bandwidth memory supplier and ongoing sanctions pose significant hurdles. ChangXin Memory Technologies (CXMT) may soon produce HBM stacks in China, potentially changing the landscape for Moore Threads and its competition with US chipmakers.