CPU steal time and other caveats of virtualization
In this post, we will talk about downsides of virtualization in the context of cloud environments.
Cloud is great, we make an API call, and pay as we go, what’s not to like
Well, few things, which are obviously not mentioned in marketing materials, and are not commonly talked about on the web as much.
We will talk about CPU Steal Time and it’s partners in crime in virtualization
CPU Steal Time
Understand that your hardware is not real.
Wait, so they lied to me that I have 16 core machine?
Not really, to be correct you have 16 Virtual cores, the same physical machines have way more the 16 cores and it is shared with other VMs
Why do I care, I have the cores that I paid for?
Well, as I said they are virtual cores so other workload running on it can impact you.
How?
Allow me to introduce CPU steal time, the hypervisor can steal your CPU when it needs to do other important stuff.
Steal time is the percentage of time a virtual CPU waits for a real CPU while the hypervisor is servicing another virtual processor.
– ibm.com
Further Reading:
- Detecting CPU steal time in guest virtual machines
- Understanding CPU Steal Time - when should you be worried?
- Who Stole my CPU? Few Basics on CPU Steal Time
- Understanding AWS stolen CPU and how it affects your apps
The Noisy Neighbor Problem
Other misbehaving workloads running on the same physical hardware can impact things that hypervisors can’t set constraints on, like cache, network links, etc.
This is something to keep in mind when running on cloud environments
Further Reading:
Security Implications
Side-channel attacks are a real threat when running in the cloud, research shows that it’s very much real.
Meltdown and Spectre showed us that speculative execution based attacks are real, and new research in this direction shows that this threat is very real.
Further Reading:
- Hello from the Other Side: SSH over Robust Cache Covert Channels in the Cloud: Paper, Talk, and Media Coverage
- Hardware vulnerabilities in cloud-native environments
- Leveraging KVM Events to Detect Cache-Based Side Channel Attacks in a Virtualization Environment
- A Placement Vulnerability Study in Multi-Tenant Public Clouds
Dedicated VMs
All cloud providers have a way to provision dedicated VMs(GCP calls then sole-tenant nodes) where physical machine runs only your workload.
They are expensive, but this is an option you have when you want extra security when running on cloud environments.
Closing Note
Cloud has more upsides than downsides but we should be aware of downsides and our options when we have to make a call where we can’t live with one of the downsides
Send your questions and corrections on twitter @electron0zero
Hopefully, it was helpful. Stay in and Stay Safe 👋