Jalapeño is an ASIC for OpenAI’s inference workloads, designed from the ground up around real software and workloads, with AI helping accelerate the process from RTL to tape-out in nine months.
Its key strength is controlling performance per watt and cost to match its own workloads. However, it is not yet a product that general users can buy and use.
Jalapeño is an ASIC for OpenAI’s inference workloads, designed from the ground up around real software and workloads, with AI helping accelerate the process from RTL to tape-out in nine months.
Its key strength is controlling performance per watt and cost to match its own workloads. However, it is not yet a product that general users can buy and use.
Visualizing Jalapeño
Think of Jalapeño as a specialized inference chip that works alongside HBM4 memory through high-speed interconnects in OpenAI’s computing systems. AI assists with the design behind the scenes, from RTL through tape-out.
The image should show the scale in an easy-to-understand way: Jalapeño serves as the main processing component, HBM4 supplies data to the chip, and the connection between them is critical to overall performance.
Visualizing Jalapeño
Think of Jalapeño as a specialized inference chip that works alongside HBM4 memory through high-speed interconnects in OpenAI’s computing systems. AI assists with the design behind the scenes, from RTL through tape-out.
The image should show the scale in an easy-to-understand way: Jalapeño serves as the main processing component, HBM4 supplies data to the chip, and the connection between them is critical to overall performance.
When GPUs Are No Longer Enough for Our Workloads
The team must handle enormous numbers of requests from language models and agents, pushing existing GPUs up against limits in power consumption, latency, and hardware queues. Some workloads are not slow because of software alone; they are slow because they must compete for the same resources.
This raises an important question: if general-purpose GPUs no longer meet the requirements, should we build a chip for our own workloads? An ASIC could make the design more closely match the workload, but the trade-off is that all the cost and risk must be taken on internally.
When GPUs Are No Longer Enough for Our Workloads
The team must handle enormous numbers of requests from language models and agents, pushing existing GPUs up against limits in power consumption, latency, and hardware queues. Some workloads are not slow because of software alone; they are slow because they must compete for the same resources.
This raises an important question: if general-purpose GPUs no longer meet the requirements, should we build a chip for our own workloads? An ASIC could make the design more closely match the workload, but the trade-off is that all the cost and risk must be taken on internally.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is an ASIC specifically for inference. It is not a general-purpose GPU and not a chip for training models. Its purpose is to accelerate model responses in a way that better matches OpenAI’s real workloads.
The chip therefore has to work alongside models, software, networking, and data centers. It is not an isolated piece of hardware. Broadcom is involved in its development and manufacturing, allowing OpenAI to manage the process from chip design through deployment in real-world systems.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is an ASIC specifically for inference. It is not a general-purpose GPU and not a chip for training models. Its purpose is to accelerate model responses in a way that better matches OpenAI’s real workloads.
The chip therefore has to work alongside models, software, networking, and data centers. It is not an isolated piece of hardware. Broadcom is involved in its development and manufacturing, allowing OpenAI to manage the process from chip design through deployment in real-world systems.
From Relying on GPUs to a Chip Built for This Work
GPUs are well suited to a wide range of tasks, but that flexibility comes at the cost of adapting the system to the model’s processing patterns. Jalapeño, by contrast, is designed around OpenAI’s workloads, creating an opportunity to address bottlenecks between the chip, software, and data center more directly.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on OpenAI’s AI workloads |
| Workload type | AI and general computing workloads | Running OpenAI models |
| Flexibility | High | Focused on specialized workloads |
| Memory | Designed around general systems | Arranged to match the models |
| Power consumption | Depends on the overall system | Aims to reduce energy per workload |
| Response time | Bottlenecks from surrounding systems | Optimized for faster responses |
| Adaptation to new models | Relies more heavily on software | Designed together with the software |
From Relying on GPUs to a Chip Built for This Work
GPUs are well suited to a wide range of tasks, but that flexibility comes at the cost of adapting the system to the model’s processing patterns. Jalapeño, by contrast, is designed around OpenAI’s workloads, creating an opportunity to address bottlenecks between the chip, software, and data center more directly.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on OpenAI’s AI workloads |
| Workload type | AI and general computing workloads | Running OpenAI models |
| Flexibility | High | Focused on specialized workloads |
| Memory | Designed around general systems | Arranged to match the models |
| Power consumption | Depends on the overall system | Aims to reduce energy per workload |
| Response time | Bottlenecks from surrounding systems | Optimized for faster responses |
| Adaptation to new models | Relies more heavily on software | Designed together with the software |
How Much Can AI Really Help Design Chips?
AI is useful as an engineering assistant, helping write and verify RTL, suggest architectural alternatives, identify errors, and point out areas where the design may not meet its targets.
For example, engineers might ask AI to draft part of a circuit, then use OpenAI software to test and analyze multiple approaches. People still have to verify correctness, choose solutions that fit the chip’s constraints, and make the final decisions themselves.
The key is not having AI design everything on its own, but using it to reduce repetitive work and accelerate experimentation, giving engineers more time to focus on the system as a whole;
How Much Can AI Really Help Design Chips?
AI is useful as an engineering assistant, helping write and verify RTL, suggest architectural alternatives, identify errors, and point out areas where the design may not meet its targets.
For example, engineers might ask AI to draft part of a circuit, then use OpenAI software to test and analyze multiple approaches. People still have to verify correctness, choose solutions that fit the chip’s constraints, and make the final decisions themselves.
The key is not having AI design everything on its own, but using it to reduce repetitive work and accelerate experimentation, giving engineers more time to focus on the system as a whole;
Four Ways Jalapeño Differs in Real-World Use
-
Placing the KV cache close to the processing units helps long chats and continuous agent workloads run more smoothly because data does not have to travel long distances as often.
-
Using high-bandwidth HBM4 memory is suitable for periods when many users submit requests simultaneously, reducing bottlenecks during processing.
-
It is designed specifically for inference, focusing on reducing latency when generating responses one token at a time, making it suitable for interactive workloads.
-
System-level interconnects support combining chips in large racks and pods, allowing services to scale systematically as workloads grow.
Four Ways Jalapeño Differs in Real-World Use
-
Placing the KV cache close to the processing units helps long chats and continuous agent workloads run more smoothly because data does not have to travel long distances as often.
-
Using high-bandwidth HBM4 memory is suitable for periods when many users submit requests simultaneously, reducing bottlenecks during processing.
-
It is designed specifically for inference, focusing on reducing latency when generating responses one token at a time, making it suitable for interactive workloads.
-
System-level interconnects support combining chips in large racks and pods, allowing services to scale systematically as workloads grow.
How Does Jalapeño Compare with Other Chip Options?
| Factor | Jalapeño | NVIDIA GB300/Blackwell and specialized chips |
|---|---|---|
| Performance per watt | Designed with a focus on inference | Powerful and supports a wide range of workloads |
| Latency | Focused on responding one token at a time | Depends on the system and configuration |
| Model compatibility | Well suited to supported workloads | Supports a broader range of models and tools |
| Availability | Real-world deployment remains to be seen | Clearer commercial platforms |
| Software ecosystem | Still building its supporting ecosystem | Mature, with many available tools |
| Dependence on a single provider | Risk of being locked into OpenAI’s system | Options remain available from multiple providers |
Jalapeño’s selling point is that it can perform inference in a way that more closely matches the workload. The trade-off is that its flexibility and ecosystem may still lag behind competitors. This strategy becomes compelling when OpenAI can control the models, software, and hardware itself.
How Does Jalapeño Compare with Other Chip Options?
| Factor | Jalapeño | NVIDIA GB300/Blackwell and specialized chips |
|---|---|---|
| Performance per watt | Designed with a focus on inference | Powerful and supports a wide range of workloads |
| Latency | Focused on responding one token at a time | Depends on the system and configuration |
| Model compatibility | Well suited to supported workloads | Supports a broader range of models and tools |
| Availability | Real-world deployment remains to be seen | Clearer commercial platforms |
| Software ecosystem | Still building its supporting ecosystem | Mature, with many available tools |
| Dependence on a single provider | Risk of being locked into OpenAI’s system | Options remain available from multiple providers |
Jalapeño’s selling point is that it can perform inference in a way that more closely matches the workload. The trade-off is that its flexibility and ecosystem may still lag behind competitors. This strategy becomes compelling when OpenAI can control the models, software, and hardware itself.
Clear Strengths and Limitations to Watch
Jalapeño’s main strength is its direct alignment with OpenAI’s workloads. This could help control long-term costs, shorten development cycles, and improve power management in real-world use.
Pros
- +Designed for real workloads
- +Potential to reduce costs and development cycles
- +Focused on energy efficiency
Cons
- −Better suited to specialized workloads than general use
- −Limited testing data is available
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Clear Strengths and Limitations to Watch
Jalapeño’s main strength is its direct alignment with OpenAI’s workloads. This could help control long-term costs, shorten development cycles, and improve power management in real-world use.
Pros
- +Designed for real workloads
- +Potential to reduce costs and development cycles
- +Focused on energy efficiency
Cons
- −Better suited to specialized workloads than general use
- −Limited testing data is available
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Costs That Do Not Appear on the Spec Sheet
The cost of an ASIC does not end with the chip itself. It also includes development, tape-out, HBM4 manufacturing, cooling systems, power, networking, and rack installation. The more specialized the system, the more carefully its design and maintenance costs must be planned over the long term.
Another major cost comes from dependence on a specialized software stack. If the models or tools change, both the software and hardware may need to be modified, making the total cost far higher than the chip’s listed price.
Costs That Do Not Appear on the Spec Sheet
The cost of an ASIC does not end with the chip itself. It also includes development, tape-out, HBM4 manufacturing, cooling systems, power, networking, and rack installation. The more specialized the system, the more carefully its design and maintenance costs must be planned over the long term.
Another major cost comes from dependence on a specialized software stack. If the models or tools change, both the software and hardware may need to be modified, making the total cost far higher than the chip’s listed price.
What Jalapeño Says About the Future of Hardware Design
Jalapeño suggests that the future may not be determined solely by building chips that are faster than GPUs. Models, software, chips, memory, and data centers must be designed to work as one system, from RTL to tape-out, with AI helping accelerate engineering work.
The key factors are therefore performance per watt and cost control, while maintaining chip-level reliability. It is not simply about completing prototypes faster. We will need to see how much this approach can truly reduce development time without increasing risks during manufacturing and real-world deployment.
What Jalapeño Says About the Future of Hardware Design
Jalapeño suggests that the future may not be determined solely by building chips that are faster than GPUs. Models, software, chips, memory, and data centers must be designed to work as one system, from RTL to tape-out, with AI helping accelerate engineering work.
The key factors are therefore performance per watt and cost control, while maintaining chip-level reliability. It is not simply about completing prototypes faster. We will need to see how much this approach can truly reduce development time without increasing risks during manufacturing and real-world deployment.
Visualizing Jalapeño
The image should show Jalapeño as an AI ASIC connected to HBM4 and other chips through high-speed data paths, clearly illustrating its approximate size, position within the system, and the role of each component.
Visualizing Jalapeño
The image should show Jalapeño as an AI ASIC connected to HBM4 and other chips through high-speed data paths, clearly illustrating its approximate size, position within the system, and the role of each component.
When GPUs Are No Longer Enough for Our Workloads
When the team must support enormous numbers of requests from language models and agents, GPUs are not only about computing power. They also bring electricity costs, latency, and longer hardware queues. Some workloads therefore become slower even when the system already has GPUs.
This raises an important question: should the team continue buying more GPUs, or build a specialized chip so AI workloads can run more efficiently? The Jalapeño concept begins with real operational problems, rather than simply trying to build the most powerful chip.
When GPUs Are No Longer Enough for Our Workloads
When the team must support enormous numbers of requests from language models and agents, GPUs are not only about computing power. They also bring electricity costs, latency, and longer hardware queues. Some workloads therefore become slower even when the system already has GPUs.
This raises an important question: should the team continue buying more GPUs, or build a specialized chip so AI workloads can run more efficiently? The Jalapeño concept begins with real operational problems, rather than simply trying to build the most powerful chip.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is a specialized inference chip for running models with workload-specific responsiveness. It is not a general-purpose GPU and not a chip for training models.
It is part of a system that works with OpenAI’s models, software, networks, and data centers. Broadcom is helping develop and manufacture it. This positioning means Jalapeño emphasizes suitability for real workloads over being an all-purpose chip.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is a specialized inference chip for running models with workload-specific responsiveness. It is not a general-purpose GPU and not a chip for training models.
It is part of a system that works with OpenAI’s models, software, networks, and data centers. Broadcom is helping develop and manufacture it. This positioning means Jalapeño emphasizes suitability for real workloads over being an all-purpose chip.
From Relying on GPUs to a Chip Built for This Work
General-purpose accelerators offer broad support and are easy to deploy, but they require a design that is not specifically optimized for model responses. Jalapeño is designed from the beginning around OpenAI’s model-running workloads, giving it the potential to reduce unnecessary work during response generation.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on model-running workloads |
| Workload type | Can train and run a wide range of workloads | Focused on model responses |
| Flexibility | High | Focused on specialized workloads |
| Memory | Managed generally | Designed around model patterns |
| Power consumption | Includes overhead depending on usage | Aims to reduce overhead |
| Response time | Depends on the overall workload | Aims to respond directly to the workload |
| Adaptation to new models | More flexible | Requires co-design with software |
From Relying on GPUs to a Chip Built for This Work
General-purpose accelerators offer broad support and are easy to deploy, but they require a design that is not specifically optimized for model responses. Jalapeño is designed from the beginning around OpenAI’s model-running workloads, giving it the potential to reduce unnecessary work during response generation.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on model-running workloads |
| Workload type | Can train and run a wide range of workloads | Focused on model responses |
| Flexibility | High | Focused on specialized workloads |
| Memory | Managed generally | Designed around model patterns |
| Power consumption | Includes overhead depending on usage | Aims to reduce overhead |
| Response time | Depends on the overall workload | Aims to respond directly to the workload |
| Adaptation to new models | More flexible | Requires co-design with software |
How Much Can AI Really Help Design Chips?
AI can draft RTL and check repetitive code more quickly, while also exploring architectural alternatives, such as adjusting data paths or memory usage for Jalapeño’s ASIC workloads. Engineers can use AI to analyze errors in simulation results and identify problems before formal verification.
AI also works alongside OpenAI’s software. When the team changes a model or workload pattern, AI can help determine which instructions and data flows the chip should support. People still define the constraints, review the results, and make the final decisions. AI is therefore more like an assistant that accelerates design iterations than a complete replacement for human designers.
How Much Can AI Really Help Design Chips?
AI can draft RTL and check repetitive code more quickly, while also exploring architectural alternatives, such as adjusting data paths or memory usage for Jalapeño’s ASIC workloads. Engineers can use AI to analyze errors in simulation results and identify problems before formal verification.
AI also works alongside OpenAI’s software. When the team changes a model or workload pattern, AI can help determine which instructions and data flows the chip should support. People still define the constraints, review the results, and make the final decisions. AI is therefore more like an assistant that accelerates design iterations than a complete replacement for human designers.
Four Ways Jalapeño Differs in Real-World Use
Jalapeño places the KV cache close to the processing units, making it suitable for long chats and agents that need to retain context continuously while reducing the time spent moving data around the system.
High-bandwidth HBM4 helps handle large numbers of simultaneous requests. Meanwhile, the inference-focused design aims to reduce latency when generating responses one token at a time, making the system feel more responsive.
Another feature is system-level connectivity for large racks and pods. This allows services to scale in groups rather than simply adding individual chips, making it suitable for AI workloads with users arriving continuously throughout the day.
Four Ways Jalapeño Differs in Real-World Use
Jalapeño places the KV cache close to the processing units, making it suitable for long chats and agents that need to retain context continuously while reducing the time spent moving data around the system.
High-bandwidth HBM4 helps handle large numbers of simultaneous requests. Meanwhile, the inference-focused design aims to reduce latency when generating responses one token at a time, making the system feel more responsive.
Another feature is system-level connectivity for large racks and pods. This allows services to scale in groups rather than simply adding individual chips, making it suitable for AI workloads with users arriving continuously throughout the day.
How Does Jalapeño Compare with Other Chip Options?
Jalapeño is interesting because it is designed specifically for inference, but there are not yet confirmed figures showing that it is superior in actual performance per watt or latency. The choice therefore depends as much on the models, software, and system readiness as on the chip itself.
| Factor | Jalapeño | NVIDIA GB300/Blackwell and Google/cloud ASIC |
|---|---|---|
| Performance per watt | Has a clear objective for inference | More market data and real-world usage |
| Latency | Focused on one-token-at-a-time responses | Depends on system and model configuration |
| Model compatibility | Software support remains to be seen | Broader support |
| Availability | Still uncertain | Clearer deployment channels |
| Software ecosystem | Still under development | Stronger |
| Single-provider risk | Risk of being tied to the platform | Still carries risks as well |
How Does Jalapeño Compare with Other Chip Options?
Jalapeño is interesting because it is designed specifically for inference, but there are not yet confirmed figures showing that it is superior in actual performance per watt or latency. The choice therefore depends as much on the models, software, and system readiness as on the chip itself.
| Factor | Jalapeño | NVIDIA GB300/Blackwell and Google/cloud ASIC |
|---|---|---|
| Performance per watt | Has a clear objective for inference | More market data and real-world usage |
| Latency | Focused on one-token-at-a-time responses | Depends on system and model configuration |
| Model compatibility | Software support remains to be seen | Broader support |
| Availability | Still uncertain | Clearer deployment channels |
| Software ecosystem | Still under development | Stronger |
| Single-provider risk | Risk of being tied to the platform | Still carries risks as well |
Clear Strengths and Limitations to Watch
Jalapeño ASIC appears to have a clear advantage in being designed for specific AI workloads. It could help control long-term costs, reduce development steps, and target energy efficiency more precisely. However, its value will depend on real workloads and deployment within OpenAI’s systems.
Pros
- +Designed for AI workloads
- +Controls long-term costs and shortens development cycles
- +Focused on performance per watt
Cons
- −Specialized use with limited flexibility
- −Limited real-world testing data
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Clear Strengths and Limitations to Watch
Jalapeño ASIC appears to have a clear advantage in being designed for specific AI workloads. It could help control long-term costs, reduce development steps, and target energy efficiency more precisely. However, its value will depend on real workloads and deployment within OpenAI’s systems.
Pros
- +Designed for AI workloads
- +Controls long-term costs and shortens development cycles
- +Focused on performance per watt
Cons
- −Specialized use with limited flexibility
- −Limited real-world testing data
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Costs That Do Not Appear on the Spec Sheet
The cost of a system like this does not end with ASIC design. It also includes tape-out, manufacturing high-bandwidth memory chips, and testing before real-world deployment. The more specialized the chip, the more investment is required in cooling systems, power supplies, and rack networking to keep everything running continuously.
There are also rack installation, data-center space, maintenance, and spare-parts costs to account for. An easily overlooked expense is dependence on a specialized software stack. When moving systems or changing model architectures, nearly all the code and processes may need to be revised.
Costs That Do Not Appear on the Spec Sheet
The cost of a system like this does not end with ASIC design. It also includes tape-out, manufacturing high-bandwidth memory chips, and testing before real-world deployment. The more specialized the chip, the more investment is required in cooling systems, power supplies, and rack networking to keep everything running continuously.
There are also rack installation, data-center space, maintenance, and spare-parts costs to account for. An easily overlooked expense is dependence on a specialized software stack. When moving systems or changing model architectures, nearly all the code and processes may need to be revised.
What Jalapeño Says About the Future of Hardware Design
The key issue may not be having a chip that is faster than a GPU, but designing models, software, chips, memory, and data centers to work as one system from the beginning.
Jalapeño is therefore a test of how much AI can genuinely reduce engineering development time while maintaining chip-level reliability. It is worth watching because faster design only matters when the result is ready for real-world use.
What Jalapeño Says About the Future of Hardware Design
The key issue may not be having a chip that is faster than a GPU, but designing models, software, chips, memory, and data centers to work as one system from the beginning.
Jalapeño is therefore a test of how much AI can genuinely reduce engineering development time while maintaining chip-level reliability. It is worth watching because faster design only matters when the result is ready for real-world use. Jalapeño is an ASIC for OpenAI’s inference workloads, designed from the ground up around real software and workloads, with AI helping accelerate the process from RTL to tape-out in nine months.
Its key strength is controlling performance per watt and cost to match its own workloads. However, it is not yet a product that general users can buy and use.
Jalapeño is an ASIC for OpenAI’s inference workloads, designed from the ground up around real software and workloads, with AI helping accelerate the process from RTL to tape-out in nine months.
Its key strength is controlling performance per watt and cost to match its own workloads. However, it is not yet a product that general users can buy and use.
Visualizing Jalapeño
Think of Jalapeño as a specialized inference chip that works alongside HBM4 memory through high-speed interconnects in OpenAI’s computing systems. AI assists with the design behind the scenes, from RTL through tape-out.
The image should show the scale in an easy-to-understand way: Jalapeño serves as the main processing component, HBM4 supplies data to the chip, and the connection between them is critical to overall performance.
Visualizing Jalapeño
Think of Jalapeño as a specialized inference chip that works alongside HBM4 memory through high-speed interconnects in OpenAI’s computing systems. AI assists with the design behind the scenes, from RTL through tape-out.
The image should show the scale in an easy-to-understand way: Jalapeño serves as the main processing component, HBM4 supplies data to the chip, and the connection between them is critical to overall performance.
When GPUs Are No Longer Enough for Our Workloads
The team must handle enormous numbers of requests from language models and agents, pushing existing GPUs up against limits in power consumption, latency, and hardware queues. Some workloads are not slow because of software alone; they are slow because they must compete for the same resources.
This raises an important question: if general-purpose GPUs no longer meet the requirements, should we build a chip for our own workloads? An ASIC could make the design more closely match the workload, but the trade-off is that all the cost and risk must be taken on internally.
When GPUs Are No Longer Enough for Our Workloads
The team must handle enormous numbers of requests from language models and agents, pushing existing GPUs up against limits in power consumption, latency, and hardware queues. Some workloads are not slow because of software alone; they are slow because they must compete for the same resources.
This raises an important question: if general-purpose GPUs no longer meet the requirements, should we build a chip for our own workloads? An ASIC could make the design more closely match the workload, but the trade-off is that all the cost and risk must be taken on internally.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is an ASIC specifically for inference. It is not a general-purpose GPU and not a chip for training models. Its purpose is to accelerate model responses in a way that better matches OpenAI’s real workloads.
The chip therefore has to work alongside models, software, networking, and data centers. It is not an isolated piece of hardware. Broadcom is involved in its development and manufacturing, allowing OpenAI to manage the process from chip design through deployment in real-world systems.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is an ASIC specifically for inference. It is not a general-purpose GPU and not a chip for training models. Its purpose is to accelerate model responses in a way that better matches OpenAI’s real workloads.
The chip therefore has to work alongside models, software, networking, and data centers. It is not an isolated piece of hardware. Broadcom is involved in its development and manufacturing, allowing OpenAI to manage the process from chip design through deployment in real-world systems.
From Relying on GPUs to a Chip Built for This Work
GPUs are well suited to a wide range of tasks, but that flexibility comes at the cost of adapting the system to the model’s processing patterns. Jalapeño, by contrast, is designed around OpenAI’s workloads, creating an opportunity to address bottlenecks between the chip, software, and data center more directly.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on OpenAI’s AI workloads |
| Workload type | AI and general computing workloads | Running OpenAI models |
| Flexibility | High | Focused on specialized workloads |
| Memory | Designed around general systems | Arranged to match the models |
| Power consumption | Depends on the overall system | Aims to reduce energy per workload |
| Response time | Bottlenecks from surrounding systems | Optimized for faster responses |
| Adaptation to new models | Relies more heavily on software | Designed together with the software |
From Relying on GPUs to a Chip Built for This Work
GPUs are well suited to a wide range of tasks, but that flexibility comes at the cost of adapting the system to the model’s processing patterns. Jalapeño, by contrast, is designed around OpenAI’s workloads, creating an opportunity to address bottlenecks between the chip, software, and data center more directly.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on OpenAI’s AI workloads |
| Workload type | AI and general computing workloads | Running OpenAI models |
| Flexibility | High | Focused on specialized workloads |
| Memory | Designed around general systems | Arranged to match the models |
| Power consumption | Depends on the overall system | Aims to reduce energy per workload |
| Response time | Bottlenecks from surrounding systems | Optimized for faster responses |
| Adaptation to new models | Relies more heavily on software | Designed together with the software |
How Much Can AI Really Help Design Chips?
AI is useful as an engineering assistant, helping write and verify RTL, suggest architectural alternatives, identify errors, and point out areas where the design may not meet its targets.
For example, engineers might ask AI to draft part of a circuit, then use OpenAI software to test and analyze multiple approaches. People still have to verify correctness, choose solutions that fit the chip’s constraints, and make the final decisions themselves.
The key is not having AI design everything on its own, but using it to reduce repetitive work and accelerate experimentation, giving engineers more time to focus on the system as a whole;
How Much Can AI Really Help Design Chips?
AI is useful as an engineering assistant, helping write and verify RTL, suggest architectural alternatives, identify errors, and point out areas where the design may not meet its targets.
For example, engineers might ask AI to draft part of a circuit, then use OpenAI software to test and analyze multiple approaches. People still have to verify correctness, choose solutions that fit the chip’s constraints, and make the final decisions themselves.
The key is not having AI design everything on its own, but using it to reduce repetitive work and accelerate experimentation, giving engineers more time to focus on the system as a whole;
Four Ways Jalapeño Differs in Real-World Use
-
Placing the KV cache close to the processing units helps long chats and continuous agent workloads run more smoothly because data does not have to travel long distances as often.
-
Using high-bandwidth HBM4 memory is suitable for periods when many users submit requests simultaneously, reducing bottlenecks during processing.
-
It is designed specifically for inference, focusing on reducing latency when generating responses one token at a time, making it suitable for interactive workloads.
-
System-level interconnects support combining chips in large racks and pods, allowing services to scale systematically as workloads grow.
Four Ways Jalapeño Differs in Real-World Use
-
Placing the KV cache close to the processing units helps long chats and continuous agent workloads run more smoothly because data does not have to travel long distances as often.
-
Using high-bandwidth HBM4 memory is suitable for periods when many users submit requests simultaneously, reducing bottlenecks during processing.
-
It is designed specifically for inference, focusing on reducing latency when generating responses one token at a time, making it suitable for interactive workloads.
-
System-level interconnects support combining chips in large racks and pods, allowing services to scale systematically as workloads grow.
How Does Jalapeño Compare with Other Chip Options?
| Factor | Jalapeño | NVIDIA GB300/Blackwell and specialized chips |
|---|---|---|
| Performance per watt | Designed with a focus on inference | Powerful and supports a wide range of workloads |
| Latency | Focused on responding one token at a time | Depends on the system and configuration |
| Model compatibility | Well suited to supported workloads | Supports a broader range of models and tools |
| Availability | Real-world deployment remains to be seen | Clearer commercial platforms |
| Software ecosystem | Still building its supporting ecosystem | Mature, with many available tools |
| Dependence on a single provider | Risk of being locked into OpenAI’s system | Options remain available from multiple providers |
Jalapeño’s selling point is that it can perform inference in a way that more closely matches the workload. The trade-off is that its flexibility and ecosystem may still lag behind competitors. This strategy becomes compelling when OpenAI can control the models, software, and hardware itself.
How Does Jalapeño Compare with Other Chip Options?
| Factor | Jalapeño | NVIDIA GB300/Blackwell and specialized chips |
|---|---|---|
| Performance per watt | Designed with a focus on inference | Powerful and supports a wide range of workloads |
| Latency | Focused on responding one token at a time | Depends on the system and configuration |
| Model compatibility | Well suited to supported workloads | Supports a broader range of models and tools |
| Availability | Real-world deployment remains to be seen | Clearer commercial platforms |
| Software ecosystem | Still building its supporting ecosystem | Mature, with many available tools |
| Dependence on a single provider | Risk of being locked into OpenAI’s system | Options remain available from multiple providers |
Jalapeño’s selling point is that it can perform inference in a way that more closely matches the workload. The trade-off is that its flexibility and ecosystem may still lag behind competitors. This strategy becomes compelling when OpenAI can control the models, software, and hardware itself.
Clear Strengths and Limitations to Watch
Jalapeño’s main strength is its direct alignment with OpenAI’s workloads. This could help control long-term costs, shorten development cycles, and improve power management in real-world use.
Pros
- +Designed for real workloads
- +Potential to reduce costs and development cycles
- +Focused on energy efficiency
Cons
- −Better suited to specialized workloads than general use
- −Limited testing data is available
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Clear Strengths and Limitations to Watch
Jalapeño’s main strength is its direct alignment with OpenAI’s workloads. This could help control long-term costs, shorten development cycles, and improve power management in real-world use.
Pros
- +Designed for real workloads
- +Potential to reduce costs and development cycles
- +Focused on energy efficiency
Cons
- −Better suited to specialized workloads than general use
- −Limited testing data is available
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Costs That Do Not Appear on the Spec Sheet
The cost of an ASIC does not end with the chip itself. It also includes development, tape-out, HBM4 manufacturing, cooling systems, power, networking, and rack installation. The more specialized the system, the more carefully its design and maintenance costs must be planned over the long term.
Another major cost comes from dependence on a specialized software stack. If the models or tools change, both the software and hardware may need to be modified, making the total cost far higher than the chip’s listed price.
Costs That Do Not Appear on the Spec Sheet
The cost of an ASIC does not end with the chip itself. It also includes development, tape-out, HBM4 manufacturing, cooling systems, power, networking, and rack installation. The more specialized the system, the more carefully its design and maintenance costs must be planned over the long term.
Another major cost comes from dependence on a specialized software stack. If the models or tools change, both the software and hardware may need to be modified, making the total cost far higher than the chip’s listed price.
What Jalapeño Says About the Future of Hardware Design
Jalapeño suggests that the future may not be determined solely by building chips that are faster than GPUs. Models, software, chips, memory, and data centers must be designed to work as one system, from RTL to tape-out, with AI helping accelerate engineering work.
The key factors are therefore performance per watt and cost control, while maintaining chip-level reliability. It is not simply about completing prototypes faster. We will need to see how much this approach can truly reduce development time without increasing risks during manufacturing and real-world deployment.
What Jalapeño Says About the Future of Hardware Design
Jalapeño suggests that the future may not be determined solely by building chips that are faster than GPUs. Models, software, chips, memory, and data centers must be designed to work as one system, from RTL to tape-out, with AI helping accelerate engineering work.
The key factors are therefore performance per watt and cost control, while maintaining chip-level reliability. It is not simply about completing prototypes faster. We will need to see how much this approach can truly reduce development time without increasing risks during manufacturing and real-world deployment.
Visualizing Jalapeño
The image should show Jalapeño as an AI ASIC connected to HBM4 and other chips through high-speed data paths, clearly illustrating its approximate size, position within the system, and the role of each component.
Visualizing Jalapeño
The image should show Jalapeño as an AI ASIC connected to HBM4 and other chips through high-speed data paths, clearly illustrating its approximate size, position within the system, and the role of each component.
When GPUs Are No Longer Enough for Our Workloads
When the team must support enormous numbers of requests from language models and agents, GPUs are not only about computing power. They also bring electricity costs, latency, and longer hardware queues. Some workloads therefore become slower even when the system already has GPUs.
This raises an important question: should the team continue buying more GPUs, or build a specialized chip so AI workloads can run more efficiently? The Jalapeño concept begins with real operational problems, rather than simply trying to build the most powerful chip.
When GPUs Are No Longer Enough for Our Workloads
When the team must support enormous numbers of requests from language models and agents, GPUs are not only about computing power. They also bring electricity costs, latency, and longer hardware queues. Some workloads therefore become slower even when the system already has GPUs.
This raises an important question: should the team continue buying more GPUs, or build a specialized chip so AI workloads can run more efficiently? The Jalapeño concept begins with real operational problems, rather than simply trying to build the most powerful chip.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is a specialized inference chip for running models with workload-specific responsiveness. It is not a general-purpose GPU and not a chip for training models.
It is part of a system that works with OpenAI’s models, software, networks, and data centers. Broadcom is helping develop and manufacture it. This positioning means Jalapeño emphasizes suitability for real workloads over being an all-purpose chip.
Where Jalapeño Fits in OpenAI’s Hardware Strategy
Jalapeño is a specialized inference chip for running models with workload-specific responsiveness. It is not a general-purpose GPU and not a chip for training models.
It is part of a system that works with OpenAI’s models, software, networks, and data centers. Broadcom is helping develop and manufacture it. This positioning means Jalapeño emphasizes suitability for real workloads over being an all-purpose chip.
From Relying on GPUs to a Chip Built for This Work
General-purpose accelerators offer broad support and are easy to deploy, but they require a design that is not specifically optimized for model responses. Jalapeño is designed from the beginning around OpenAI’s model-running workloads, giving it the potential to reduce unnecessary work during response generation.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on model-running workloads |
| Workload type | Can train and run a wide range of workloads | Focused on model responses |
| Flexibility | High | Focused on specialized workloads |
| Memory | Managed generally | Designed around model patterns |
| Power consumption | Includes overhead depending on usage | Aims to reduce overhead |
| Response time | Depends on the overall workload | Aims to respond directly to the workload |
| Adaptation to new models | More flexible | Requires co-design with software |
From Relying on GPUs to a Chip Built for This Work
General-purpose accelerators offer broad support and are easy to deploy, but they require a design that is not specifically optimized for model responses. Jalapeño is designed from the beginning around OpenAI’s model-running workloads, giving it the potential to reduce unnecessary work during response generation.
| Factor | General-purpose accelerator | Jalapeño |
|---|---|---|
| Design objective | Supports many types of workloads | Focused on model-running workloads |
| Workload type | Can train and run a wide range of workloads | Focused on model responses |
| Flexibility | High | Focused on specialized workloads |
| Memory | Managed generally | Designed around model patterns |
| Power consumption | Includes overhead depending on usage | Aims to reduce overhead |
| Response time | Depends on the overall workload | Aims to respond directly to the workload |
| Adaptation to new models | More flexible | Requires co-design with software |
How Much Can AI Really Help Design Chips?
AI can draft RTL and check repetitive code more quickly, while also exploring architectural alternatives, such as adjusting data paths or memory usage for Jalapeño’s ASIC workloads. Engineers can use AI to analyze errors in simulation results and identify problems before formal verification.
AI also works alongside OpenAI’s software. When the team changes a model or workload pattern, AI can help determine which instructions and data flows the chip should support. People still define the constraints, review the results, and make the final decisions. AI is therefore more like an assistant that accelerates design iterations than a complete replacement for human designers.
How Much Can AI Really Help Design Chips?
AI can draft RTL and check repetitive code more quickly, while also exploring architectural alternatives, such as adjusting data paths or memory usage for Jalapeño’s ASIC workloads. Engineers can use AI to analyze errors in simulation results and identify problems before formal verification.
AI also works alongside OpenAI’s software. When the team changes a model or workload pattern, AI can help determine which instructions and data flows the chip should support. People still define the constraints, review the results, and make the final decisions. AI is therefore more like an assistant that accelerates design iterations than a complete replacement for human designers.
Four Ways Jalapeño Differs in Real-World Use
Jalapeño places the KV cache close to the processing units, making it suitable for long chats and agents that need to retain context continuously while reducing the time spent moving data around the system.
High-bandwidth HBM4 helps handle large numbers of simultaneous requests. Meanwhile, the inference-focused design aims to reduce latency when generating responses one token at a time, making the system feel more responsive.
Another feature is system-level connectivity for large racks and pods. This allows services to scale in groups rather than simply adding individual chips, making it suitable for AI workloads with users arriving continuously throughout the day.
Four Ways Jalapeño Differs in Real-World Use
Jalapeño places the KV cache close to the processing units, making it suitable for long chats and agents that need to retain context continuously while reducing the time spent moving data around the system.
High-bandwidth HBM4 helps handle large numbers of simultaneous requests. Meanwhile, the inference-focused design aims to reduce latency when generating responses one token at a time, making the system feel more responsive.
Another feature is system-level connectivity for large racks and pods. This allows services to scale in groups rather than simply adding individual chips, making it suitable for AI workloads with users arriving continuously throughout the day.
How Does Jalapeño Compare with Other Chip Options?
Jalapeño is interesting because it is designed specifically for inference, but there are not yet confirmed figures showing that it is superior in actual performance per watt or latency. The choice therefore depends as much on the models, software, and system readiness as on the chip itself.
| Factor | Jalapeño | NVIDIA GB300/Blackwell and Google/cloud ASIC |
|---|---|---|
| Performance per watt | Has a clear objective for inference | More market data and real-world usage |
| Latency | Focused on one-token-at-a-time responses | Depends on system and model configuration |
| Model compatibility | Software support remains to be seen | Broader support |
| Availability | Still uncertain | Clearer deployment channels |
| Software ecosystem | Still under development | Stronger |
| Single-provider risk | Risk of being tied to the platform | Still carries risks as well |
How Does Jalapeño Compare with Other Chip Options?
Jalapeño is interesting because it is designed specifically for inference, but there are not yet confirmed figures showing that it is superior in actual performance per watt or latency. The choice therefore depends as much on the models, software, and system readiness as on the chip itself.
| Factor | Jalapeño | NVIDIA GB300/Blackwell and Google/cloud ASIC |
|---|---|---|
| Performance per watt | Has a clear objective for inference | More market data and real-world usage |
| Latency | Focused on one-token-at-a-time responses | Depends on system and model configuration |
| Model compatibility | Software support remains to be seen | Broader support |
| Availability | Still uncertain | Clearer deployment channels |
| Software ecosystem | Still under development | Stronger |
| Single-provider risk | Risk of being tied to the platform | Still carries risks as well |
Clear Strengths and Limitations to Watch
Jalapeño ASIC appears to have a clear advantage in being designed for specific AI workloads. It could help control long-term costs, reduce development steps, and target energy efficiency more precisely. However, its value will depend on real workloads and deployment within OpenAI’s systems.
Pros
- +Designed for AI workloads
- +Controls long-term costs and shortens development cycles
- +Focused on performance per watt
Cons
- −Specialized use with limited flexibility
- −Limited real-world testing data
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Clear Strengths and Limitations to Watch
Jalapeño ASIC appears to have a clear advantage in being designed for specific AI workloads. It could help control long-term costs, reduce development steps, and target energy efficiency more precisely. However, its value will depend on real workloads and deployment within OpenAI’s systems.
Pros
- +Designed for AI workloads
- +Controls long-term costs and shortens development cycles
- +Focused on performance per watt
Cons
- −Specialized use with limited flexibility
- −Limited real-world testing data
- −Depends on OpenAI software
- −May be difficult to adapt when model architectures change
Costs That Do Not Appear on the Spec Sheet
The cost of a system like this does not end with ASIC design. It also includes tape-out, manufacturing high-bandwidth memory chips, and testing before real-world deployment. The more specialized the chip, the more investment is required in cooling systems, power supplies, and rack networking to keep everything running continuously.
There are also rack installation, data-center space, maintenance, and spare-parts costs to account for. An easily overlooked expense is dependence on a specialized software stack. When moving systems or changing model architectures, nearly all the code and processes may need to be revised.
Costs That Do Not Appear on the Spec Sheet
The cost of a system like this does not end with ASIC design. It also includes tape-out, manufacturing high-bandwidth memory chips, and testing before real-world deployment. The more specialized the chip, the more investment is required in cooling systems, power supplies, and rack networking to keep everything running continuously.
There are also rack installation, data-center space, maintenance, and spare-parts costs to account for. An easily overlooked expense is dependence on a specialized software stack. When moving systems or changing model architectures, nearly all the code and processes may need to be revised.
What Jalapeño Says About the Future of Hardware Design
The key issue may not be having a chip that is faster than a GPU, but designing models, software, chips, memory, and data centers to work as one system from the beginning.
Jalapeño is therefore a test of how much AI can genuinely reduce engineering development time while maintaining chip-level reliability. It is worth watching because faster design only matters when the result is ready for real-world use.
What Jalapeño Says About the Future of Hardware Design
The key issue may not be having a chip that is faster than a GPU, but designing models, software, chips, memory, and data centers to work as one system from the beginning.
Jalapeño is therefore a test of how much AI can genuinely reduce engineering development time while maintaining chip-level reliability. It is worth watching because faster design only matters when the result is ready for real-world use.