Tag
7 articles
PrismML has compressed a 27-billion-parameter AI model to under 4 GB, enabling it to run on an iPhone. Apple is reportedly testing the technology, which could advance on-device AI capabilities.
NVIDIA introduces Nemotron-Labs-3-Puzzle-75B-A9B, a compressed hybrid MoE LLM delivering 2.03x server throughput, leveraging hardware-aware compression and knowledge distillation.
Learn how NVIDIA's new AI model Nemotron-Labs-3-Puzzle-75B-A9B uses compression and smart design to work faster and more efficiently than previous versions, without sacrificing quality.
A new AI model called VibeThinker-3B shows that logical reasoning can be made small and efficient, while factual knowledge still requires larger models. This discovery could make AI more accessible and powerful.
As KV cache memory outpaces model weights in large language models, three compression techniques—TurboQuant, OSCAR, and EpiCache—are emerging as key contenders. While each offers distinct methods for optimization, they are seen as complementary rather than competitive.
Learn to compress instruction-tuned language models using FP8, GPTQ, and SmoothQuant quantization techniques with llmcompressor, and benchmark their performance.
Google introduces TurboQuant, a new compression algorithm that reduces LLM key-value cache memory by 6x and delivers up to 8x speedup without accuracy loss.
