AI That Designs Its Own Chips: Ricursive's Anna Goldie and Azalia Mirhoseini

you One of the themes that we've heard throughout the day is that neural nets are replacing a lot of traditional tools. And I think one of the most exciting application categories, where we've actually seen that come to life, is within chip design, where neural nets are now becoming superhuman at certain parts of the semiconductor design process. And so I'm thrilled to introduce Anna and Azalea. They were the co-creators of Alpha Chip, which did exactly this. I'm at Google and was used on multiple generations of TPU and I've now started a company to build this.

Thank you and welcome Anna and Azalea. Thank you. Thank you so much. Hi everyone. Azali and I are so excited to be here today to talk about our new company, Recursive Intelligence, where we're doing AI for chip design and chip design for AI. We've been working together closely for the last 10 years across Google Brain, Anthropic, deep mind and then Because it wasn't enough to work at one institution in parallel, I started my PhD while continuing to work full-time, and Azalea joined the Stanford faculty. So, we have been working in many places together for a long time.

But our thesis in this company... is that chips are the fuel for AI. and that we should be using AI to design to optimize and automate the chip design process. and close this recursive self-improving loop between AI in its physical substrate. We started this direction in 2018 with our work on AlphaChip. where we developed a deep reinforcement learning agent that was capable of generating superhuman chip playouts. This work was published in Nature, but the interesting part about it, in our opinion, was it was actually used in the tape out of real chips.

So the last four generations now of Google's AI accelerator chips TPU data centers, CPUs called Axion. Pixel phones and also autonomous vehicle chips. And in addition to adoption by external companies like MediaTek, Thank you. And so we decided to start this company to take this work to the next level. and take on all of the chip design workflow. We see the company in three phases. So currently we're in phase one. where we want to accelerate the chip design process, So today there are two long poles. One is physical design. which is placing the billions of standard cells, or billions of transistors, and routing billions of these components onto a chip canvas.

and design verification, which is verifying the correctness of the logic of that chip. Each of these can take up to a year and involves hundreds or thousands of human experts. And the stakes are extremely high. So-- We've heard estimates like one day of delay of an NVIDIA Chip costs like a Blackwell cost the company something like $225 million in lost opportunity cost. So you want to help existing chip makers get to market faster, build faster, cheaper, and more environmentally friendly chips. But I think in phase two of the company, we want to democratize chip design.

So you want to become a platform for designing new hardware where we can take as input like a workload, say, like the next Claude model, design an accelerates that workload and then do the entire design process all the way to GDS2Clean, which is the format that we send to the fabs for manufacturing. In that case, we can massively unlock the number of customers that we could serve, like any company that has a workload that they serve at sufficient scale could benefit from custom chips, even if they don't have teams of hundreds or thousands of human experts.

And then phase three of the company would be vertical integration. So if we have this capability to quickly design highly performant chips, Why not build our own chips? Why not train our own models and co-evolve them? and serve intelligence at a price or a capability that would be impossible to match. So Azalia is going to talk a bit more about our approach in this company. If I can stop on this slide. So on the right side, we are showing you the flow, the traditional flow for chip design. It starts with architecture design, and it goes all the way to sign off, and that's what you send to the fabs.

And as you can see, there are many components here, and the way these-- kind of phases are done today with human experts in the loop, working with commercial tools that sometimes takes days to run for a single iteration of an optimization. So our approach here at Recursive is to first redesign the way these tools perform, make them 100,000x faster, and then they're primed to be used with AI because, as you know, our AIs really like fast iteration loops, and they can just exponentially learn more and co-optimize across a very, very large space if we enable them to do so.

So by co-designing across this stack, what this enables is unlocking massive performance improvements and time to market, which comes from both the co-design and the automation. To just show you a glimpse into what we are building, here is an STA, a static timing analysis engine. This is a very challenging component of physical design. And what we are showing here is that we have built a tool that correlates with commercial tools, very high fidelity and What we can do here is do so a thousand X faster. Now imagine if you are doing an AI tool use or an RL loop, we now have this kind of feedback signal that we can use in the optimization.

We can do a lot more with it. Here is an example of how our outer loop, our AI, works with this tool. Like here, early on, is what we get with a single iteration of our inner loop. But as the AI optimizes around the recipes that are possible to use these tools with, we can get significantly more performance. So taking a step back, what Recursive is enabling is a new era, which we call designless, just like fabulous. AND I THINK THAT'S A GOOD IDEA. enabled by companies like TSMC, made it such that we can have NVIDIA, Apple, other companies focus on designing chips and send off the designs for fabrication elsewhere.

We want to be the platform for chip design. companies can focus on the application modeling and other layers, and we can be the compute and hardware that enable those applications. And the impact would be that we can democratize chip design and enable a lot more variety and performance types of chips possible. so so Right now, there are a few mainstream chips for AI inference. But as you can imagine, and as a lot of talks and conversations today allude to, we are going to need a whole lot more performance in the coming years.

And one way to unlock a lot of performance is through customization. So when we build chips that are truly customized to the workloads that we are serving, and we at Recursive want to be the platform that enables this Cambrian explosion of chips more variety of chips that are really custom to the types of workloads that the companies and the users care about. And you can imagine these chips can enable a very large workload like a frontier model or they can enable like a very low power or high throughput or other kind of variations of performance that we would need.

Hmm. And finally, we have an amazing team. Our company is a little unusual in that we have this subset of people who are very expert in LLM. They have worked on... called Gemini, Grok, and such in the past, and now we have put them together with these experts in chip design. And so it's a very great mix, and we are very glad that we get to build together. Yeah, and with that, we can conclude the talk. Happy to answer questions. Can you work on the shape of the chip placements that you end up seeing out of these walls?

Oh shoot, we should have shown some of the layouts. So I think just like an alpha chip, we're seeing these kind of curved, organic-looking sheets to our placements. So human experts, they tend to make these very aligned regular looking placements, but the AI generated ones look more like organic, curved, which minimizes wire length, improves performance. But it was kind of shocking to physical design engineers when they first saw them. A question on the cost of these specialized chips. I totally understand that you could make a better chip with AI and have better placements and stuff.

But I think Azalea was also making the point that you could make specialized chips. How does the scale work out? I don't know the first principles of chip design. Can you make thousands of different chips and make them as cheaply as one hopper architecture? Yeah, go ahead. Yes, there is definitely what we are doing here is like we are going into a new regime where we have we can work compute to our advantage. So by scaling compute, we can reduce the run times to design the chip and also to enable more performance chips.

So we are basically introducing a knob. is working itself, right? Even a 1% improvement in a chip that serves a frontier model is a massive gain and success if you could enable that. But at different scales, we have different performance gains. And again, what we are talking here is that we are using compute to bring automations and better performance, and that's a knob that we can... Play bit. Thank you.