OpenAI and Broadcom announce chip designed for LLM inference at scale

OpenAI and Broadcom have announced a new chip, Jalapeño, designed for large language model inference in data centers. The chip's development took nine months and is intended to deliver better performance per watt than current state-of-the-art.

The Jalapeño chip is designed to be deployed at large data centers and is more specialized for the current needs of large language models (LLMs) than existing chips, according to OpenAI and Broadcom^[1]. OpenAI claims that early testing shows Jalapeño will deliver substantially better performance per watt, although a detailed technical report is pending. Meanwhile, researchers have proposed ContextForge, a system for context recycling that improves LLM performance in long-horizon tasks by maintaining task-relevant information across turns and reducing token overhead^[2]. ContextForge combines structured query generation, external memory retrieval, and controlled synthesis to preserve answer quality while reducing token consumption. This development comes as LLMs face challenges due to context window limitations and inefficient token usage, leading to degraded performance over long conversational horizons.