342 Transistors for Every Person In the World: Cerebras 2nd Gen Wafer Scale Engine Teased

One of the highlights of Hot Chips from 2019 was the startup Cerebras showcasing its product – a large ‘wafer-scale’ AI chip that was literally the size of a wafer. The chip itself was rectangular, but it was cut from a single wafer, and contained 400,000 cores, 1.2 trillion transistors, 46225 mm² of silicon, and was built on TSMC’s 16 nm process.

The whole thing created a very big buzz, and later that year the company showed of the first system, CS-1, which is a 15U unit in order to power a single chip. Power consumption was in the 15 kW range, and the unit cost a few million. They are already being deployed by research institutions.

Initially Cerebras was set to announce its 2^nd generation product here at Hot Chips this year, however the time-scale didn’t align exactly, so the company is instead going through its software procedures. But at the end of the slide deck, there’s a special slide with some details about the next generation.

Obviously when doing wafer scale, you can’t just add more die area, so the only way is to optimize die area per core and take advantage of smaller process nodes. That means for TSMC 7nm, there are now 850,000 cores and 2.6 trillion transistors. Cerebras has had to develop new technologies to deal with multi-reticle designs, but they succeeded with the first gen, and transferred the learnings to the new chip. We’re expecting more details about this new product later this year.