Reducing High-Bandwidth Memory Bottlenecks in JAX-Based LLM Training with Host Offloading
Large language model (LLM) training workloads increasingly run into GPU memory limits before compute is fully used.
Tag archive
infrastructure
6 confirmed signals.
AI Model Co-Design: Hardware-Friendly LLM Design
NVIDIA Generative AI Blog published a source-backed update on AI Model Co-Design: Hardware-Friendly LLM Design.
Accelerating End-to-End Co-Folding Performance with NVIDIA BioNeMo Agent Toolkit
NVIDIA Generative AI Blog published a source-backed update on Accelerating End-to-End Co-Folding Performance with NVIDIA BioNeMo Agent Toolkit.
NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads
NVIDIA Generative AI Blog published a source-backed update on NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads.
Scaling AI Inference Across Multiple GPUs Using NVIDIA TensorRT with Multi-Device Inference Support
Generative AI workloads are rapidly outgrowing the memory and compute budget of single GPUs.
Boost Inference Performance up to 15x on NVIDIA Blackwell Using DFlash Speculative Decoding
NVIDIA describes DFlash speculative decoding for Blackwell inference throughput. Treat the headline speedup as vendor-benchmark context until workload shape, quality tradeoffs, and deployment constraints are verified.