Notes on Hardware and Silicon Industry
In this post, I will be talking about rapid changes in the silicon and hardware industry and the reasons why I think we will see the next wave of innovation in this space.
These predictions are based on discussions and what I have been reading past few months. see sources at the end of the post for details.
A few months ago (June 2020), I was talking to my friend Vaishali about what is going in the Linux kernel world. She works with the Linux kernel closely and we started talking about excitement for ARM in kernel maintainers and contributors.
Discussion took an interesting turn and we started talking about chiplets, co-located accelerators and future SOC, chips, and startups working in this space.
Discussions mostly revolved around:
- NUVIA, also called “Tesla of Silicon”,
- Rise of new rack-scale computer companies like Oxide Computer Company,
- Rise of accelerators, and use of GPUs as general-purpose accelerators,
- GPU companies expanding into purpose-built accelerators,
- Hyperscalers making their own chips.
- How Apple is going all-in on ARM, along with Microsoft and Amazon (maybe Google as well, nothing public yet)
Apologies for the buzzword soup, I will try to explain things in details in the coming paragraphs 🙂
I am trying to break it down into sections, and explain all the changes we are seeing in this space, we can think of these sections as big moves that are inter-dependent, but essential for the growth of emerging silicon ecosystem.
Rise of ARM
Big players like Apple, Microsoft, Amazon, and Google are investing heavily in the ARM ecosystem. NVIDIA Acquiring ARM for $40 Billion is the big move that indicates the direction of the industry.
Along with that, big Hyperscalers are betting big on ARM. AWS made its public move by acquiring Annapurna Labs(Israeli startup working on chips) for an estimated $350-$400 million in 2015-16. Annapurna Labs was acquired to bring silicon design capabilities in AWS.
As a result, AWS now offers Graviton, and Graviton2 chips which are ARM-based, and available to AWS customers at cheaper rates compared to Intel chips.
I am sure Azure and Google Cloud are also planning to release ARM-based chips, probably designed by in-house teams, few acquisitions, and some industry collaboration.
Abode is investing heavily in porting software to ARM (Lightroom 4.1 runs natively on Apple M1), and we all know about Apple Silicon-based M1 laptops, and how fast they are.
ARM dominates the mobile and IoT market, and now we are seeing that an ARM takeover is happening in Hyperscaler, and PC market.
Windows is getting support for ARM, and Microsoft is joining forces with Qualcomm, and PC makers to ARM-based computers in the market.
Rise of Accelerators
We are seeing a rise of dedicated accelerators and a new breed of workload-specific SOCs.
Chip designers are trying to get around the performance bottlenecks by co-locating data and compute. See more on chiplets in ACM, March Issue.
We are seeing companies like Tesla building in-house chips to power the self-driving capabilities, and benchmarks show that these chips are extremely good at the job.
Google and NVIDIA have built accelerators for machine learning workloads, and we are seeing the rise of ML processors in mobile SoCs
The more the merrier
When you pack warehouses full of computers, you care about things that normal people don’t even think about. Along with raw per-core performance, heat emission, and power consumption are important for hyperscalers.
Hyperscalers used to be the customers of chipmakers, but they realized that having more control over things like heat and power consumption is important.
When you have a warehouse full of computers, lower power consumption, and less heat can save millions in the power bill.
Because of these, and many other reasons we are seeing hyperscalers investing in their own chips.
AWS, Apple Silicon, Google Silicon, and Tesla making their own chips. We see startups like Nuvia (got acquired by Qualcomm), also working on it.
We are seeing the rise of new Chip Makers, and as the saying goes “the more the merrier”. I am certain this will result in innovation in the space.
The Hardware Software Interface & Rise of Open Hardware, and Firmware
We are seeing a new wave of hardware-based exploits. Meltdown and Spectre opened the pandora’s box, and now we are seeing that x86 optimization techniques like speculative execution are being exploited.
Now hardware will be the place where folks have started looking for optimizations, and security issues. We are seeing the financial industry building their own hardware, and publicly talking about it.
Startups like Oxide computer company are planning to de-blot and modernize servers, projects like Open Compute, and the rise of open firmware means that now it’s possible to design your own hardware and get it built. The barrier to entry is lowered, and now you have a choice.
We are seeing a surge in open source tools, open firmware, and open hardware. The market is ripe for disruption, and we are seeing lots of changes in this space. We are seeing a new wave where x86 is being left behind and big market players are betting billions on ARM.
- ARM going to be big in the coming years (US Govt is testing the waters)
- RISC-V and open firmware will shake up silicon space
- Commoditization of fabrication will happen
- Many-Core, Low Power, Purpose-built chips will show up.
- Programmable Hardware will make a comeback for those performance critical cases (HFT folks are already doing it)
- Rise of Open Firmware & Open tooling to design your own SoC (System on a Chip)
- Rise of Security Issues in closed hardware and firmware will drive open firmware adoption
Read them, verify them, and if you find any errors let me know on Twitter :)
Press & Publications
- Apple, ARM, and Intel - Stratechery by Ben Thompson
- Jim Keller Resigns from Intel, Effective Immediately
- Amazon buys secretive chip maker Annapurna Labs for $350 million - ExtremeTech
- If you wanna make your own open-source chip, just Google it. Literally. Web giant says it’ll fab them for free • The Register
- Google Partners with SkyWater and Efabless to Enable Open Source Manufacturing of Custom ASICs - Skywater Technology
- Apple’s Courage With MacOS On ARM Will Lift The Industry
- The Future of American Industry Depends on Open Source Tech - WIRED
- NVIDIA to Acquire Arm for $40 Billion, Creating World’s Premier Computing Company for the Age of AI - NVIDIA Newsroom
- The Future of Operating Systems on RISC-V
- Why Semiconductor Development Matters - steve cheney, technology, business & strategy
- Is it time to SoC the CPU?
- Qualcomm Acquires NUVIA To Accelerate Its Future CPUs With Support From 18 Partners
- Notes on technology in the 2020s - Eli Dourado
- Apple Silicon: The Passing of Wintel - by Jean-Louis Gassée - Monday Note
- Virtual Media Vulnerability in BMC Opens Servers to Remote Attack - Eclypsium
- Qualcomm Execs Say Apple M1 Further Validates Windows On Arm, And They’re Right - HotHardware
- Intel Problem’s - Stratechery by Ben Thompson
- Intel’s Missed Opportunity - Stratechery by Ben Thompson
- Meet Raspberry Silicon: Raspberry Pi Pico now on sale at $4 - Raspberry Pi
- Tesla Autonomy Day - YouTube - Tesla building it’s own chips with lots of co-processors
- Astra and Sandia’s 725E Data Center - YouTube - US Govt investing big into ARM
- Intel is in serious trouble. ARM is the Future. - YouTube
- Goodbye x86. The FUTURE is RISC-V - YouTube
- The SQ1 is just the beginning. ARM is coming for the PC - YouTube
- RISC vs CISC Computer Architectures (David Patterson) - AI Podcast Clips with Lex Fridman - YouTube
- How Machine Learning Changed Computer Architecture Design (David Patterson) - AI Clips with Lex - YouTube
- New Golden Age for Computer Architectures - Dave Patterson (UC Berkeley) - YouTube
- Programmable Hardware with Andy Ray - YouTube
- RISC-V is the future of computing - Chris Lattner and Lex Fridman - YouTube
- Forget x86; OpenPower is it! Talos II Secure Workstation! - YouTube
- OSFC 2019 - Common BMC vulnerabilities and how to avoid repeating them - YouTube
- Intel - From Inventors of the CPU to Laughing Stock [Part 1]
Tooling & Tech
- Chisel/FIRRTL: Home
- OpenPOWER Foundation
- The Innovator’s Edge - SkyWater Technology
- About - Optimyze.cloud