Microsoft's ND H100 v5 VM is generally available, Azure OpenAI Service is in more regions

In March, Microsoft announced that it was collaborating with NVIDIA to create the Azure ND H100 v5 Virtual Machine (VM) series. Today, Microsoft announced that those new supercomputers are now generally available, offering business access to faster performance for generative AI tasks,

The new VMs uses NVIDIA's NVIDIA Tensor Core H100 GPUs and NVIDIA Quantum-2 InfiniBand networking. At the moment, they are available for access in both the East and West United States Azure regions. Here's what businesses can expect when they sign up to access these new and fast AI-based VMs.

Supercomputing GPUs: Equipped with eight NVIDIA Tensor Core H100 GPUs, these VMs promise significantly faster AI model performance than previous generations, empowering businesses with unmatched computational power.

Next-generation computer processing unit (CPU): Understanding the criticality of CPU performance for AI training and inference, we have chosen the 4th Gen Intel Xeon Scalable processors as the foundation of these VMs, ensuring optimal processing speed

Low-latency networking: The inclusion of NVIDIA Quantum-2 CX7 InfiniBand with 400GB/s crossnode bandwidth ensures seamless performance across the GPUs, matching the capabilities of top-performing supercomputers globally.

Optimized host to GPU performance: With PCIe Gen5 providing 64GB/s bandwidth per GPU, Azure achieves significant performance advantages between CPU and GPU.

Large scale memory and memory bandwidth: DDR5 memory is at the core of these VMs, delivering greater data transfer speeds and efficiency, making them ideal for workloads with larger datasets.

Microsoft added that it is working to add hundreds of thousands of these new GPUs for its VMs over the next year.

In addition to the new generative AI Azure virtual machines, Microsoft has also announced an expansion of its Azure OpenAI Service. It is now available in the Canada East, East United States 2, Japan East, and UK South regions.

The service, as we have reported on before, will help businesses create their own AI-powered apps. Since it was first announced in March, Microsoft has said it has seen great responses from businesses:

We now proudly serve over 11,000 customers, attracting an average of 100 new customers daily this quarter. This remarkable growth is a testament to the value and scalability our service brings to businesses eager to harness the potential of AI for their unique needs.

Microsoft also added that the most powerful version of the ChatGPT AI platform, GPT-4, will also be available for the new Azure OpenAI Service regions.