Tag

#LLM

38 articles

Validating Distributed LLM Serving Benchmarks with NVIDIA srt-slurm, SLURM Recipes, Parameter Sweeps, and Pareto Analysis

NVIDIA's srt-slurm framework simplifies distributed LLM serving benchmarking using SLURM, enabling reproducible workflows and advanced performance analysis.

Jul 214

tools

10 Open-Source No-Code AI Platforms for Building LLM Apps, RAG Systems, and AI Agents

A roundup of 10 open-source no-code AI platforms that simplify the creation of LLM apps, RAG systems, and AI agents using visual and plain-English tools.

Jul 1819

Sakana AI Launches Sakana Fugu: An Orchestration Model That Routes Tasks Across a Swappable Pool of Frontier LLMs

Sakana AI has launched Sakana Fugu, an orchestration model that dynamically routes tasks across a swappable pool of frontier LLMs, achieving strong performance on coding, reasoning, and agentic benchmarks.

Jun 2246

Sakana AI's Fugu orchestrates multiple LLMs to match Anthropic's Fable and Mythos benchmarks

Learn to build a simple multi-LLM orchestration system that dynamically selects the best AI model for each question, similar to Sakana AI's Fugu system.

Jun 2157

tech

Google Cloud Introduces Open Knowledge Format (OKF): A Vendor-Neutral Markdown Spec for Giving AI Agents Curated Context

This explainer introduces the Open Knowledge Format (OKF), a vendor-neutral specification for structuring knowledge for AI agents. It explains how OKF uses Markdown and YAML to create curated knowledge bundles, distinguishing it from traditional RAG systems.

Jun 1552

tools

NVIDIA garak Tutorial: Build a Complete Defensive LLM Red-Teaming Workflow with Custom Probes and Detectors

NVIDIA's garak framework offers a comprehensive solution for defensive LLM red-teaming, enabling organizations to identify vulnerabilities and enhance model safety.

Jun 645

ChatGPT now saves narrative dossiers about you sorted by work, hobbies, and travel preferences

This article explains OpenAI's new Dreaming memory system that builds coherent user profiles from conversations, improving information retention from 52.2% to 75.1%.

Jun 453

Asana acquires no-code agent-builder Stack AI

This article explains how no-code AI agent builders like Stack AI work and why they're transforming enterprise workflow automation by enabling non-technical users to create sophisticated autonomous AI systems.

May 2854

MEMO: A Modular Framework for Training a Dedicated Memory Model on New Knowledge Without Modifying LLM Parameters

Researchers from NUS, MIT, and A*STAR introduce MEMO, a modular framework that enables LLMs to learn new knowledge without modifying core parameters.

May 2662

Meet EAGLE 3.1: The Speculative Decoding Algorithm That Fixes Attention Drift in LLM Inference

EAGLE 3.1, developed by the EAGLE team, vLLM, and TorchSpec, tackles attention drift in LLM inference, enhancing speculative decoding stability for production use.

May 2654

Together AI Open-Sources OSCAR: An Attention-Aware 2-Bit KV Cache Quantization System for Long-Context LLM Serving

Together AI open-sources OSCAR, an attention-aware 2-bit KV cache quantization system that significantly reduces memory usage and improves decoding speed for long-context LLMs.

May 2559

Build a Complete Langfuse Observability and Evaluation Pipeline for Tracing, Prompt Management, Scoring, and Experiments

This article explains how to build a complete Langfuse observability and evaluation pipeline for LLM development, covering tracing, prompt management, scoring, and experimentation.

May 2472