Enabling AMD Secure Nested Paging (SEV-SNP) on ThinkSystem Servers

Trusted execution environments have become increasingly common for the execution of security critical code. In their processors, AMD first introduced Secure Encrypted Virtualization (SEV) in 2016, and then introduced Encrypted State (SEV-ES) to encrypt CPU register state of virtual machines (VM) in 2017. The third generation of SEV, Secure Nested Paging (SEV-SNP), enhances memory integrity protection for the malicious attacks from hypervisor.

SEV-SNP is supported on AMD EYPC processors starting with the AMD EPYC 7003 series processors.

AMD SEV-SNP offers powerful and flexible support for the isolation of a guest virtual machine from an untrusted host operating system. It is very useful in public cloud and any untrusted host scenario. Major public cloud vendors already used it in their products, including Amazon Web Services (AWS) and Google Cloud.

This paper describes how to enable SEV-SNP on an AMD-based ThinkSystem server running Red Hat Enterprise Linux (RHEL) 9.2.

This section will show how to protect the guest VM via SEV-SNP function and what threats can be prevented.

In Figure 1, “AMD Hardware and Firmware” and “SEV-SNP VM” are considered trusted in the measurement process, even though the hypervisor is untrusted.

Figure 1. Threat model

Under the attestation process of SEV-SNP, only a guest owner (third-party) can decide whether the guest is trusted or not based on attestation reports.

Figure 2. Attestation Process

Table 1 lists potential threads mitigated by SEV-SNP.

In this section:

• UEFI configuration via System Setup
• UEFI configuration via OneCLI
• Operating System configuration

UEFI configuration via System Setup

The steps to activate SEV-SNP in UEFI are as follows:

1. Press F1 during boot to enter System Setup
2. In the Processors section, enable these items as shown in the figure below.
   • SVM Mode: Enable
   • SEV-SNP Support : Enable

   
   Figure 3. Processor settings in System Setup

3. In the Memory section, enable these items as shown in the figure below.
   • SMEE: Enable
   • SEV-ES ASID Count: AUTO
   • SEV-ES ASID Space Limit Control: Manual
   • SEV-ES ASID Space Limit: 10
   • SEV Control: Enable

   
   Figure 4. Memory settings in System Setup

UEFI configuration via OneCLI

As an alternative to System Setup, you can use the OneCLI command line tool, which can be downloaded from:
https://support.lenovo.com/us/en/solutions/HT116433

1. Create a configuration file, as follows:

   [root@sev-snp ~]# cat > snp_uefi.txt << EOF 
   set Processors.SEV-SNPSupport enable
   set Memory.SMEE Enable
   set Memory.SEVASIDCount AUTO
   set Memory.SEV-ESASIDSpaceLimitControl Manual
   set Memory.SEV-ESASIDSpaceLimit 10
   set Memory.SEVControl Enable set Processors.SVMMode Enable
   EOF
   [root@sev-snp ~]#
   

2. Set up UEFI config via Onecli command: 

   
   Figure 5. Issuing the OneCLI command to run the configuration file

3. Restart the server to apply the configuration.

Operating System configuration

As RHEL 9.2 inbox kernel and QEMU hypervisor still do not fully support this feature, users need to compile it by themselves. Ensure your system has access to the Internet and source code will be downloaded automatically during compiling.

1. Register your system and enable repository “codeready-builder-for-rhel-9-x86_64-rpms" using the following commands:

   [root@sev-snp ~]# subscription-manager register --username XXX --password XXX
   This system is already registered. Use --force to override
   [root@sev-snp ~]#
   [root@sev-snp ~]# subscription-manager repos --enable codeready-builder-for-rhel-9-x86_64-rpms
   

2. Install the necessary packages for compiling:

   [root@sev-snp ~]# yum install -y ninja-build.x86_64 gthread libgib* glib-devel.x86_64 \
   > PackageKit-glib.x86_64 PackageKit-glib-devel.x86_64 pixman pixman-devel.x86_64 \
   > nasm.x86_64 uuid-devel.x86_64 glibc-static acpica-tools  perl dwarves pkgconfig
   [root@sev-snp ~]# pip install meson; ln -s /usr/lib64/libuuid.so.1.3.0 /usr/lib64/libuuid.so; ldconfig
   

3. Build Linux kernel, QEMU and other components with the following command

   # git clone https://github.com/AMDESE/AMDSEV.git
   # cd AMDSEV; git checkout snp-latest
   # ./build.sh –package
   

Follow these steps enable and verify SEV-SNP on a host OS:

1. Install the compiled kernel

   # cd snp-release-<DATE>
   # sudo cp kvm.conf /etc/modprobe.d/
   # rpm -ivh $(find . -name "kernel*host*" | grep -v headers)
   

2. Modify the SNP kernel to the default boot entry

   # grubby --default-kernel                 # Get current default boot entry 
   # grubby --info ALL                       # Get all the boot entry
   # grubby --set-default-index=ENTRY-INDEX  # Set the SNP kernel entry index to the default
   

3. Reboot the server

   # reboot
   

4. Verify the feature was enabled from driver layer:

   [root@sev-snp ~]# cat /sys/module/kvm_amd/parameters/sev_snp 
   Y
   [root@sev-snp ~]#
   

5. Verify the dmesg log shows the SEV-SNP support information:

   [root@sev-snp ~]# dmesg I grep SEV-SNP
   [    0.569182] SEV-SNP: RMP table physical address [0x000000009b700000 - 0x00000000a3cfffff]
   [    3.905529] ccp 0000:23:00.1: SEV-SNP API:1.55 build : 14
   [   15.047076] kvm_amd: SEV-ES and SEV-SNP supported: 9 ASIDs
   

Follow these steps to enable and verify SEV-SNP on guest OS.

1. Create SEV-SNP VM with the following commands

   # qemu-img create -f qcow2 /home/rh9.qcow2 40G    #Create your qcow2 file for guest storage
   # cd AMDSEV/snp-release-
   # sed -i "s/CONSOLE=.*$/CONSOLE=\"virtio\"/" launch-qemu.sh
   # sed -i "s/readonly/readonly=on/" launch-qemu.sh
   # ./launch-qemu.sh -hda /home/rh9.qcow2 -cdrom home/RHEL-9.2.0-20230414.17-x86_64-dvd1.iso
   

   
   Figure 6.

2. Finish the installation via VNC viewer based on the output about VNC server address.

   
   Figure 7. Launch VNC viewer

3. Launch the guest OS

   # ./launch-qemu.sh -hda  /home/rh9.qcow2 -sev-snp   #Launch the guest 
   

4. Access the guest via VNC viewer based on the output about VNC server address.

   
   Figure 8.

5. If SEV-SNP is enabled properly in a VM, the log “Memory Encryption Features active:” must include the string “SEV-SNP” in OS log (dmesg):

   [root@snp-guest ~]# dmesg | grep -i SEV-SNP
   [    0.2712931 Memory Encryption Features active: AMD SEU SEV-ES SEV-SNP
   [root@snp-guest ~]#
   

For more information, see these resources:

• AMD SEV-SNP:Strengthening VM Isolation
  https://www.amd.com/content/dam/amd/en/documents/epyc-business-docs/white-papers/SEV-SNP-strengthening-vm-isolation-with-integrity-protection-and-more.pdf
• The github of SEV-SNP
  https://github.com/AMDESE/AMDSEV/tree/snp-latest
• Introduction to confidential virtual machines
  https://www.redhat.com/en/blog/introduction-confidential-virtual-machines
• AMD SEV-SNP Attestation: Establishing Trust in Guests
  https://www.amd.com/content/dam/amd/en/documents/developer/lss-snp-attestation.pdf

Song Shang is a Linux Engineer in Lenovo Infrastructure Solutions Group, based in Beijing, China.

Thanks to the following people for their contributions to this project:

• David Watts, Lenovo Press
• Adrian Huang, Lenovo Linux Engineer
• Gary Cudak, Lenovo Lead Architect