Pliops expands AI's context windows with 3D NAND-based accelerator – can accelerate certain inference workflows by up to eight times

Pliops expands AI's context windows with 3D NAND-based accelerator – can accelerate certain inference workflows by up to eight times - WorldNL Magazine

(Image credit: Pliops) As language models grow in complexity and their context windows expand, GPU-attached high bandwidth memory (HBM) becomes a bottleneck, forcing systems to repeatedly recalculate data that no longer fits in onboard HBM. Pliops has addressed this challenge with its XDP LightningAI device and FusIOnX software, which store precomputed context on fast SSDs and retrieve it instantly when needed, reports Blocks and Files. The com…

Pliops expands AI's context windows with 3D NAND-based accelerator – can accelerate certain inference workflows by up to eight times

Pliops claims its XDP LightningAI card and FusIOnX software accelerate large language model inference by offloading context data to SSDs, reducing redundant computation, and boosting vLLM throughput by up to eight times while avoiding the need for additional GPUs.