NVIDIA has unveiled its latest flagship GPU architecture NVIDIA H200, delivering major leaps in AI performance and memory bandwidth. Equipped with 141GB of ultra-fast HBM3e memory, the H200 GPU provides nearly 2x the speed of its predecessor for generative AI workloads.
The new H200 represents the continued evolution of NVIDIA’s Hopper architecture for data center and high performance computing applications. It is tailored to handle the exploding model sizes and training demands of large language models and computer vision AI.
NVIDIA states the key upgrade enabling the H200’s massive performance gains is its new HBM3e memory. With a 4.8 terabytes per second bandwidth, HBM3e offers 1.4x more bandwidth versus the H100’s HBM3 memory.
Total on-board memory also grows substantially from 80GB to 141GB, providing ample capacity to process ever growing datasets required for state-of-the-art AI.
Leveraging the ample memory bandwidth and capacity, the H200 delivers nearly 2x faster inferencing speed on advanced AI models like Anthropic’s 70 billion parameter Llama language model. This supercharges response times for natural language processing.
NVIDIA expects the H200 to begin shipping in Q2 2024, giving data centers over a year to prepare infrastructure for the cutting-edge accelerator cards. H200-powered servers will be available from all major manufacturers and cloud providers.
The NVIDIA H200 represents the bleeding edge of innovation needed to satisfy the insatiable demands of today’s AI algorithms. As Elon Musk’s Tesla knows very well, neural network training for autonomous driving requires processing unfathomable amounts of visual data.
Hopper architecture enhancements like HBM3e memory will enable developing ever smarter AI models to power technologies like full self-driving cars. NVIDIA is pushing the boundaries of what’s possible in parallel computing.
To ensure seamless upgrades, the H200 GPU will be compatible with existing H100 server board configurations. IT teams can slot the new accelerator cards into HGX H100 chassis and rack systems without modification.
With AI playing an increasingly pivotal role across science and industry, NVIDIA’s constant GPU innovation is crucial. The 2x performance leap from the H100 to H200 underscores NVIDIA’s commitment to providing the world’s most advanced AI processors.
As companies race to leverage AI, the H200’s unmatched memory and throughput will likely make it the highest demand data center GPU yet upon launch. Expect supply to be extremely tight initially for the cutting-edge hardware.
Once again, NVIDIA has delivered a next-generation GPU that redefines speed and capacity boundaries, H200 will equip researchers to develop ever more capable generative AI that is key to major innovations on the horizon.
EDITOR PICKED: NVIDIA Unveils Eureka AI to Enable Complex Robot Actions