GPU Passthrough on OpenPOWER™
QEMU KVM VFIO Passthrough and AMD GPU Demonstration

This video demonstrates effective use of virtual machines with direct GPU access to get the most out of your Talos™ machine.

OpenPOWER fully supports advanced virtualization technologies, including PCI device passthrough when used with a PowerNV system, KVM, and the HV module. We demonstrate the power of this virtualization support below by launching two separate virtual machines, each with its own dedicated AMD Radeon GPU attached, and putting both virtual machines and GPUs under moderate load simultaneously.

System Configuration

Demonstration

For the best viewing experience, expand the video to full screen mode on your device. A screen resolution of at least 1920 x 1080 pixels is recommended.

NOTE: This video is available in native 4K resolution for those with 4K-compatible devices. The high-resolution 1080p version below is scaled down to 1/4 of the 4K native resolution, and some fine and/or text detail may have been lost in the scaling process.

 
General Notes

  • This radeon driver patch is required for proper GPU start on non-x86 platforms, and was applied to the radeon driver used in this demonstration
  • This radeon driver patch is required for QEMU VFIO support on all platforms, and was applied to the radeon driver used in this demonstration
  • The Radeon R9 290X GPUs (Hawaii class) used in this demonstration contain a hardware reset bug that precludes stable long term use in virtualized environments. Specifically, they do not respond to the PCIe reset signal, and therefore cannot be reinitialized without a host power cycle. This affects all architectures including x86, and subsequent AMD cards do not have this issue.
VFIO Configuration and QEMU Launch

The following commands will isolate the GPU at PCI bus location 0000:01:00 (GPU core [1002:67b0], HDMI audio [1002:aac8]), then prepare it for attachment to a virtual machine:

modprobe vfio-pci echo "1002 67b0" > /sys/bus/pci/drivers/vfio-pci/new_id echo "1002 aac8" > /sys/bus/pci/drivers/vfio-pci/new_id echo 0000:01:00.0 > /sys/bus/pci/devices/0000\:00\:00.0/0000\:01\:00.0/driver/unbind echo 0000:01:00.1 > /sys/bus/pci/devices/0000\:00\:00.0/0000\:01\:00.1/driver/unbind echo 0000:01:00.0 > /sys/bus/pci/drivers/vfio-pci/bind echo 0000:01:00.1 > /sys/bus/pci/drivers/vfio-pci/bind echo "1002 67b0" > /sys/bus/pci/drivers/vfio-pci/remove_id echo "1002 aac8" > /sys/bus/pci/drivers/vfio-pci/remove_id

Once the GPU has been prepared, it can be attached to a newly started virtual machine via the "-device vfio-pci" arguments to QEMU:

qemu-system-ppc64 --enable-kvm -M pseries -cpu host -m 32G -device vfio-pci,host=0000:01:00.0 -device vfio-pci,host=0000:01:00.1 qemu-pcie-passthrough-vm-1.qcow2
 
All data and information provided on this page is for informational purposes only. Raptor Engineering makes no representations as to accuracy, completeness, currentness, suitability, or validity of any information on this page and will not be liable for any errors, omissions, or delays in this information or any losses, injuries, or damages arising from its display or use. All information is provided on an as-is basis. All product names, logos, and brands are property of their respective owners. All company, product and service names used in this page are for identification purposes only. Use of these names, logos, and brands does not imply endorsement.
Edited and encoded on OpenPOWER using libre software!

Back to the Talos™ prerelease page

 

Design and contents copyright
© 2009 - 2021 Raptor Engineering, LLC.
All rights reserved.
No pages or files may be distributed without express written permission.

This website makes minimal use of cookies.
Use of this site constitutes acceptance of this policy.

Subscribe via GNU Social
Design and contents copyright
© 2009 - 2021 Raptor Engineering, LLC.
All rights reserved.
No pages or files may be distributed without express written permission.

This website makes minimal use of cookies.
Use of this site constitutes acceptance of this policy.