AWS re:Invent 2018: Powering Next-Gen EC2 Instances: Deep Dive into the Nitro System (CMP303-R1)

Hi! I'm from the EC2 engineering team. This is a super challenging problem for general purpose virtualization stacks. EC2 has been working to avoid any guest VM interruptions. With the Nitro hypervisor, there are no management tasks that share CPUs with guest workloads. See more here, including data from a real customer with real-time requirements running on EC2 under the Nitro hypervisor: ▶ this video

A longer history would start with IBM mainframes. More recently, IBM Ultravisor shipped in OpenPower firmware, mediating KVM VMs, link & link 2018 video by Ian Pratt covers Xen, uXen and AX (2005-2015), link . Citrix acquired XenSource. Pratt left to work at Bromium, acquired by HP (which previously acquired BIOS company from Bromium co-founder). The former CTO of XenSource co-founded Qumranet (KVM), acquired by RedHat. AWS began with Xen, then migrated to a subset of KVM. Nitro used Arm hardware to virtualize I/O (storage, network) paths, leaving KVM responsible for x86 CPU and memory virtualization, ▶ this video & link . Parallels could be drawn to the Apple T2 enclave (Arm) coprocessor being used for disk encryption on x86 Apple Macbooks. Under the "Confidential Computing" umbrella, Intel has TDX and a new (closed?) hypervisor on servers, using SGX and new hardware privilege levels. Apple recently added Secure eXclaves to iOS, and Apple Silicon hardware supports nested virtualization, which is what Google pKVM uses on Pixel (and upcoming ChromeOS?) devices, link For production code, pKVM deserves attention because it's open (upstreamed to mainline Linux), exists in the real world (Pixel phones), stands in stark contrast to Apple's neutered iPads and has the potential to improve upon TrustZone security, link . Finally, to bring this thread back to Barrelfish, Google OpenTitan open silicon root of trust (OCP servers, Chromebooks) is partly under Pulp Platform research

DPUs/SmartNICs are a partial industry response to AWS Nitro (based on Annapurna acquisition) which performs I/O (network/storage) virtualization. Nvidia DPU is based on Mellanox acquisition AMD DPU is based on Pensado acquisition Intel has an in-house DPU There was a 2018 talk on AWS Nitro, ▶ this video Cadence summary: link James Hamilton summary: link The founder of Annapurna-Nitro is now at link , which has created the NVMEoF (NVME over Fabric, i.e. Ethernet or FiberChannel) standard. This is implemented via a software driver or hardware accelerator. Both DPUs and NVMEoF can be viewed as an attempt to standardize "composable data center" architectures pioneered by AWS Nitro. > but this is HN... what other uses can we put through this / what advantages does this physical architecture give us? A good place to start is the decade-old NetFPGA SmartNIC research project from University of Cambridge, now in the 5th generation of hardware, with earlier boards sometimes available on eBay. link A line-rate, flexible, and open platform for research, and classroom experimentation. More than 3,500 NetFPGA systems have been deployed at over 300 institutions in over 60 countries around the world. Several hundred papers have been published: link

Hi Jeff! Eeeeeek! I'd love to know so much more about the Nitro acceleration. All these accelerated fabrics are so interesting. * What does the Nitro accelerator look like to the host? . Does the Nitro accelerator present as NVMe devices to the OS host, or is there a more custom thing it presents as? Does the Nitro accelerator use SR-IOV to or something else to present as many different PCIe adapters, per-drive PCIe, or a single PCIe device, or no PCIe devices at all, something else entirely (and if so what)? Are there custom virt-io drivers powering the VMs? How much change has gone into these interfaces in the newest iterations, or have these interface channels remained stable? * What is the over the wire communication? Related to the above; ultimately the VM's see NVMe, & how far down the stack/across the network does that go? Is what's on the wire NVMe based, or something else; is it custom? What trade-offs were there, what protocols inspired the teams? Originally at launch it seemed like there was a custom remote protocol[1]; has that stayed? What drove the protocol evolution/change over time? What's new & changed? * What do the storage arrays look like; are they also PCs based? Or do the flash arrays connect via accelerators too? Are these FPGA-based or hard silicon? Are there standard flash controllers in use, or is this custom? How many channels of flash will one accelerator have connected to it? How much has the storage array architecture changed since Nitro was firs

Full post from eclypsium link . Also I assume aws baremetal servers are not vulnerable due to the nitro security card ( ▶ this video ).