Tag

#inference

13 articles

Fractile raises $220m to take its in-memory-compute inference chip into production

London-based AI chip startup Fractile has raised $220 million to bring its in-memory computing inference chip to production, with support from Accel and Pat Gelsinger.

May 1320

Meta and Stanford Researchers Propose Fast Byte Latent Transformer That Reduces Inference Memory Bandwidth by Over 50% Without Tokenization

Meta and Stanford researchers introduce the Fast Byte Latent Transformer, reducing inference memory bandwidth by over 50% without subword tokenization.

May 1118

LightSeek Foundation Releases TokenSpeed, an Open-Source LLM Inference Engine Targeting TensorRT-LLM-Level Performance for Agentic Workloads

Learn how to set up and use TokenSpeed, an open-source LLM inference engine optimized for agentic workloads, with step-by-step instructions for beginners.

May 97

Nebius paid $643 million for 20 people because inference is where the money is

This article explains the concept of inference optimization in AI, why it's critical for modern AI deployment, and how companies like Nebius are investing heavily in this area.

May 110

Google is building a four-partner chip supply chain to challenge Nvidia in AI inference

This article explains Google's strategy to challenge Nvidia in AI inference by building a diverse chip supply chain with four partners, spanning multiple chip generations from Ironwood to TPU v8.

Apr 2040

tech

Google is in talks with Marvell to build custom AI inference chips as it diversifies beyond Broadcom

Learn how to simulate AI chip inference execution with custom memory processing units and inference-optimized architectures, similar to Google's custom chips with Marvell.

Apr 1942

tools

NVIDIA Releases AITune: An Open-Source Inference Toolkit That Automatically Finds the Fastest Inference Backend for Any PyTorch Model

NVIDIA releases AITune, an open-source toolkit that automatically identifies the fastest inference backend for PyTorch models, streamlining deployment and enhancing performance.

Apr 1054

Sigmoid vs ReLU Activation Functions: The Inference Cost of Losing Geometric Context

This article explains how sigmoid and ReLU activation functions affect geometric context preservation in neural networks, and why this matters for inference accuracy.

Apr 857

tech

AI chip startup Rebellions raises $400 million at $2.3B valuation in pre-IPO round

AI chip startup Rebellions raises $400 million at $2.3B valuation, positioning itself as a challenger to Nvidia's dominance in AI inference hardware.

Mar 3061

tech

GTC 2026: With Groq 3 LPX, Nvidia adds dedicated inference hardware to its platform for the first time

Learn how to deploy machine learning models on Nvidia's new Vera Rubin platform with dedicated Groq 3 LPX inference chips using Docker containers and ONNX export.

Mar 1788

tech

Meta unveils four generations of custom AI chips to cut inference costs for billions of users

Meta has unveiled four new generations of custom AI chips aimed at reducing inference costs and decreasing reliance on external GPU suppliers like Nvidia and AMD.

Mar 1273

DeepSeek Signals Next-Gen R2 Model, Unveils Novel Approach to Scaling Inference with SPCT

Learn about SPCT (Sparse Prompt Compression Technique), a new method developed by DeepSeek AI that improves the scalability of reward models during inference, making AI systems more efficient and cost-effective.

Feb 27115