Notes on Hardware and Silicon Industry

on Notes, Hardware, and Silicon

In this post, I will be talking about rapid changes in the silicon and hardware industry and the reasons why I think we will see the next wave of innovation in this space.

These predictions are based on discussions and what I have been reading past few months. see sources at the end of the post for details.

A few months ago (June 2020), I was talking to my friend Vaishali about what is going in the Linux kernel world. She works with the Linux kernel closely and we started talking about excitement for ARM in kernel maintainers and contributors.

Discussion took an interesting turn and we started talking about chiplets, co-located accelerators and future SOC, chips, and startups working in this space.

Discussions mostly revolved around:

  • NUVIA, also called “Tesla of Silicon”,
  • Rise of new rack-scale computer companies like Oxide Computer Company,
  • Rise of accelerators, and use of GPUs as general-purpose accelerators,
  • GPU companies expanding into purpose-built accelerators,
  • Hyperscalers making their own chips.
  • How Apple is going all-in on ARM, along with Microsoft and Amazon (maybe Google as well, nothing public yet)

Apologies for the buzzword soup, I will try to explain things in details in the coming paragraphs 🙂

I am trying to break it down into sections, and explain all the changes we are seeing in this space, we can think of these sections as big moves that are inter-dependent, but essential for the growth of emerging silicon ecosystem.

Rise of ARM

Big players like Apple, Microsoft, Amazon, and Google are investing heavily in the ARM ecosystem. NVIDIA Acquiring ARM for $40 Billion is the big move that indicates the direction of the industry.

Along with that, big Hyperscalers are betting big on ARM. AWS made its public move by acquiring Annapurna Labs(Israeli startup working on chips) for an estimated $350-$400 million in 2015-16. Annapurna Labs was acquired to bring silicon design capabilities in AWS.

As a result, AWS now offers Graviton, and Graviton2 chips which are ARM-based, and available to AWS customers at cheaper rates compared to Intel chips.

I am sure Azure and Google Cloud are also planning to release ARM-based chips, probably designed by in-house teams, few acquisitions, and some industry collaboration.

Abode is investing heavily in porting software to ARM (Lightroom 4.1 runs natively on Apple M1), and we all know about Apple Silicon-based M1 laptops, and how fast they are.

ARM dominates the mobile and IoT market, and now we are seeing that an ARM takeover is happening in Hyperscaler, and PC market.

Windows is getting support for ARM, and Microsoft is joining forces with Qualcomm, and PC makers to ARM-based computers in the market.

Rise of Accelerators

We are seeing a rise of dedicated accelerators and a new breed of workload-specific SOCs.

Chip designers are trying to get around the performance bottlenecks by co-locating data and compute. See more on chiplets in ACM, March Issue.

We are seeing companies like Tesla building in-house chips to power the self-driving capabilities, and benchmarks show that these chips are extremely good at the job.

Google and NVIDIA have built accelerators for machine learning workloads, and we are seeing the rise of ML processors in mobile SoCs

The more the merrier

When you pack warehouses full of computers, you care about things that normal people don’t even think about. Along with raw per-core performance, heat emission, and power consumption are important for hyperscalers.

Hyperscalers used to be the customers of chipmakers, but they realized that having more control over things like heat and power consumption is important.

When you have a warehouse full of computers, lower power consumption, and less heat can save millions in the power bill.

Because of these, and many other reasons we are seeing hyperscalers investing in their own chips.

AWS, Apple Silicon, Google Silicon, and Tesla making their own chips. We see startups like Nuvia (got acquired by Qualcomm), also working on it.

We are seeing the rise of new Chip Makers, and as the saying goes “the more the merrier”. I am certain this will result in innovation in the space.

The Hardware Software Interface & Rise of Open Hardware, and Firmware

We are seeing a new wave of hardware-based exploits. Meltdown and Spectre opened the pandora’s box, and now we are seeing that x86 optimization techniques like speculative execution are being exploited.

Now hardware will be the place where folks have started looking for optimizations, and security issues. We are seeing the financial industry building their own hardware, and publicly talking about it.

Startups like Oxide computer company are planning to de-blot and modernize servers, projects like Open Compute, and the rise of open firmware means that now it’s possible to design your own hardware and get it built. The barrier to entry is lowered, and now you have a choice.

Closing Note

We are seeing a surge in open source tools, open firmware, and open hardware. The market is ripe for disruption, and we are seeing lots of changes in this space. We are seeing a new wave where x86 is being left behind and big market players are betting billions on ARM.

My Predictions

  • ARM going to be big in the coming years (US Govt is testing the waters)
  • RISC-V and open firmware will shake up silicon space
  • Commoditization of fabrication will happen
  • Many-Core, Low Power, Purpose-built chips will show up.
  • Programmable Hardware will make a comeback for those performance critical cases (HFT folks are already doing it)
  • Rise of Open Firmware & Open tooling to design your own SoC (System on a Chip)
  • Rise of Security Issues in closed hardware and firmware will drive open firmware adoption

Sources

Read them, verify them, and if you find any errors let me know on Twitter :)

Press & Publications
YouTube
Tooling & Tech
Tweets

Continue Reading