Intel announces Gaudi 3 AI chip, claims it's 50% faster than NVIDIA's H100

Intel has unveiled its latest AI chip, the Gaudi 3, in a bid to challenge NVIDIA's leadership in the AI industry. The new chip promises significant performance and efficiency improvements over NVIDIA's H100 GPU, the latter has a faster HGX H200 too.

According to Intel, the Gaudi 3 is more than twice as energy efficient and can perform AI tasks one and a half times faster than the H100. This improved efficiency could help reduce the exorbitant costs associated with training large AI models. The chip also comes in different modules, including an 8-chip configuration on a motherboard and a card compatible with existing Intel server designs.

Intel says it has tested the chip on popular AI models from Meta's Llama 2 and Falcon project, showing that Gaudi 3 is capable of training transformers and running complex applications.

According to Intel benchmarks, Gaudi 3 is on average 50% faster at training large language models such as Llama with 7 billion and 13 billion parameters and GPT-3 with 175 billion parameters. In addition, Gaudi 3 is expected to achieve 40% better power efficiency than the H100 for inference.

In comparison to Nvidia H100, Intel Gaudi 3 is projected to deliver 50% faster time-to-train on average across Llama2 models with 7B and 13B parameters, and GPT-3 175B parameter model. Additionally, Intel Gaudi 3 accelerator inference throughput is projected to outperform the H100 by 50% on average1 and 40% for inference power-efficiency averaged2 across Llama 7B and 70B parameters, and Falcon 180B parameter models.

The full spec details of the Gaudi 3 are given in the image below:

Gaudi 3 connects to its host CPU via a high-speed PCIe Gen 5 x16 link. Intel is also developing liquid-cooled variants for even higher performance.

The launch marks Intel's most serious challenge yet to NVIDIA's dominance, which is estimated at around 80% of the AI chip market. With customers like Dell, HPE, and Supermicro on board to build servers with the new chips, Intel is aiming to steal a significant share.

"Gaudi 3 will be available to OEMs – including Dell Technologies, HPE, Lenovo and Supermicro – in the second quarter of 2024," Intel wrote in the blog post.