Skip to content

Commit

Permalink
feat: add support for dynamic directpath i/o
Browse files Browse the repository at this point in the history 
Adding support for vSphere Dynamic DirectPath I/O.

Co-Authored-By: Ryan Johnson <[email protected]>
  • Loading branch information
@kshivakumar-nvidia @tenthirtyam
kshivakumar-nvidia and tenthirtyam committed May 1, 2024
1 parent de0c305 commit c53c318
Show file tree
Hide file tree
Showing 12 changed files with 337 additions and 41 deletions.
6 changes: 5 additions & 1 deletion .web-docs/components/builder/vsphere-clone/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -750,7 +750,11 @@ wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/foo/bar/preseed.cfg
- `displays` (int32) - Number of video displays. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-789C3913-1053-4850-A0F0-E29C3D32B6DA.html)
for supported maximums. Defaults to 1.

- `vgpu_profile` (string) - vGPU profile for accelerated graphics. See [NVIDIA GRID vGPU documentation](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#configure-vmware-vsphere-vm-with-vgpu)
- `pci_passthrough_allowed_device` ([]PCIPassthroughAllowedDevice) - Configure Dynamic DirectPath I/O [PCI Passthrough](#pci-passthrough-configuration) for
virtual machine. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-5B3CAB26-5D06-4A99-92A0-3A04C69CE64B.html)

- `vgpu_profile` (string) - vGPU profile for accelerated graphics.
vGPU profile for accelerated graphics. See [NVIDIA GRID vGPU documentation](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#configure-vmware-vsphere-vm-with-vgpu)
for examples of profile names. Defaults to none.

- `NestedHV` (bool) - Enable nested hardware virtualization for VM. Defaults to `false`.
Expand Down
60 changes: 58 additions & 2 deletions .web-docs/components/builder/vsphere-iso/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -416,7 +416,11 @@ wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/foo/bar/preseed.cfg
- `displays` (int32) - Number of video displays. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-789C3913-1053-4850-A0F0-E29C3D32B6DA.html)
for supported maximums. Defaults to 1.

- `vgpu_profile` (string) - vGPU profile for accelerated graphics. See [NVIDIA GRID vGPU documentation](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#configure-vmware-vsphere-vm-with-vgpu)
- `pci_passthrough_allowed_device` ([]PCIPassthroughAllowedDevice) - Configure Dynamic DirectPath I/O [PCI Passthrough](#pci-passthrough-configuration) for
virtual machine. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-5B3CAB26-5D06-4A99-92A0-3A04C69CE64B.html)

- `vgpu_profile` (string) - vGPU profile for accelerated graphics.
vGPU profile for accelerated graphics. See [NVIDIA GRID vGPU documentation](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#configure-vmware-vsphere-vm-with-vgpu)
for examples of profile names. Defaults to none.

- `NestedHV` (bool) - Enable nested hardware virtualization for VM. Defaults to `false`.
Expand All @@ -432,6 +436,59 @@ wget http://{{ .HTTPIP }}:{{ .HTTPPort }}/foo/bar/preseed.cfg
<!-- End of code generated from the comments of the HardwareConfig struct in builder/vsphere/common/step_hardware.go; -->


### PCI Passthrough Configuration

<!-- Code generated from the comments of the PCIPassthroughAllowedDevice struct in builder/vsphere/common/step_hardware.go; DO NOT EDIT MANUALLY -->

Dynamic DirectPath I/O is component of the Assignable Hardware framework in VMware vSphere.
Dynamic DirectPath I/O enables the Assignable Hardware intelligence for passthrough devices and
the hardware address of the PCIe device is no longer directly mapped to the virtual machine
configuration. Instead, the attributes, or capabilities, are exposed to the virtual machine.

JSON

```json

{
"pci_passthrough_allowed_device": {
"vendor_id": "8086",
"device_id": "100e",
"sub_device_id": "8086",
"sub_vendor_id": "100e"
}
}

```

HCL2

```hcl
pci_passthrough_allowed_device {
"vendor_id": "8086",
"device_id": "100e",
"sub_device_id": "8086",
"sub_vendor_id": "100e"
}
```

<!-- End of code generated from the comments of the PCIPassthroughAllowedDevice struct in builder/vsphere/common/step_hardware.go; -->


<!-- Code generated from the comments of the PCIPassthroughAllowedDevice struct in builder/vsphere/common/step_hardware.go; DO NOT EDIT MANUALLY -->

- `vendor_id` (string) - The sub-vendor ID of the PCI device.

- `device_id` (string) - The vendor ID of the PCI device.

- `sub_vendor_id` (string) - The sub-vendor ID of the PCI device.

- `sub_device_id` (string) - The sub-device ID of the PCI device.

<!-- End of code generated from the comments of the PCIPassthroughAllowedDevice struct in builder/vsphere/common/step_hardware.go; -->


## Location Configuration

<!-- Code generated from the comments of the LocationConfig struct in builder/vsphere/common/config_location.go; DO NOT EDIT MANUALLY -->
Expand Down Expand Up @@ -688,7 +745,6 @@ iso_paths = [
]
```


<!-- Code generated from the comments of the CDRomConfig struct in builder/vsphere/common/step_add_cdrom.go; DO NOT EDIT MANUALLY -->

- `cdrom_type` (string) - Which controller to use. Example: `sata`. Defaults to `ide`.
Expand Down
2 changes: 2 additions & 0 deletions builder/vsphere/clone/config.hcl2spec.go
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

58 changes: 56 additions & 2 deletions builder/vsphere/common/step_hardware.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,63 @@
// SPDX-License-Identifier: MPL-2.0

//go:generate packer-sdc struct-markdown
//go:generate packer-sdc mapstructure-to-hcl2 -type HardwareConfig
//go:generate packer-sdc mapstructure-to-hcl2 -type HardwareConfig,PCIPassthroughAllowedDevice

package common

import (
"context"
"fmt"
"reflect"

"github.com/hashicorp/packer-plugin-sdk/multistep"
packersdk "github.com/hashicorp/packer-plugin-sdk/packer"
"github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/driver"
)

// Dynamic DirectPath I/O is component of the Assignable Hardware framework in VMware vSphere.
// Dynamic DirectPath I/O enables the Assignable Hardware intelligence for passthrough devices and
// the hardware address of the PCIe device is no longer directly mapped to the virtual machine
// configuration. Instead, the attributes, or capabilities, are exposed to the virtual machine.
//
// JSON
//
// ```json
//
// {
// "pci_passthrough_allowed_device": {
// "vendor_id": "8086",
// "device_id": "100e",
// "sub_device_id": "8086",
// "sub_vendor_id": "100e"
// }
// }
//
// ```
//
// HCL2
//
// ```hcl
//
// pci_passthrough_allowed_device {
// "vendor_id": "8086",
// "device_id": "100e",
// "sub_device_id": "8086",
// "sub_vendor_id": "100e"
// }
//
// ```
type PCIPassthroughAllowedDevice struct {
// The sub-vendor ID of the PCI device.
VendorId string `mapstructure:"vendor_id"`
// The vendor ID of the PCI device.
DeviceId string `mapstructure:"device_id"`
// The sub-vendor ID of the PCI device.
SubVendorId string `mapstructure:"sub_vendor_id"`
// The sub-device ID of the PCI device.
SubDeviceId string `mapstructure:"sub_device_id"`
}

type HardwareConfig struct {
// Number of CPU cores.
CPUs int32 `mapstructure:"CPUs"`
Expand All @@ -41,6 +85,10 @@ type HardwareConfig struct {
// Number of video displays. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/8.0/vsphere-vm-administration/GUID-789C3913-1053-4850-A0F0-E29C3D32B6DA.html)
// for supported maximums. Defaults to 1.
Displays int32 `mapstructure:"displays"`
// Configure Dynamic DirectPath I/O [PCI Passthrough](#pci-passthrough-configuration) for
// virtual machine. See [vSphere documentation](https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.vm_admin.doc/GUID-5B3CAB26-5D06-4A99-92A0-3A04C69CE64B.html)
AllowedDevices []PCIPassthroughAllowedDevice `mapstructure:"pci_passthrough_allowed_device"`
// vGPU profile for accelerated graphics.
// vGPU profile for accelerated graphics. See [NVIDIA GRID vGPU documentation](https://docs.nvidia.com/grid/latest/grid-vgpu-user-guide/index.html#configure-vmware-vsphere-vm-with-vgpu)
// for examples of profile names. Defaults to none.
VGPUProfile string `mapstructure:"vgpu_profile"`
Expand Down Expand Up @@ -86,9 +134,14 @@ func (s *StepConfigureHardware) Run(_ context.Context, state multistep.StateBag)
ui := state.Get("ui").(packersdk.Ui)
vm := state.Get("vm").(driver.VirtualMachine)

if *s.Config != (HardwareConfig{}) {
if !reflect.DeepEqual(*s.Config, HardwareConfig{}) {
ui.Say("Customizing hardware...")

var allowedDevices []driver.PCIPassthroughAllowedDevice
for _, device := range s.Config.AllowedDevices {
allowedDevices = append(allowedDevices, driver.PCIPassthroughAllowedDevice(device))

Check failure on line 142 in builder/vsphere/common/step_hardware.go

View workflow job for this annotation

GitHub Actions / Linux Go tests 

cannot convert device (variable of type PCIPassthroughAllowedDevice) to type "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/driver".PCIPassthroughAllowedDevice

Check failure on line 142 in builder/vsphere/common/step_hardware.go

View workflow job for this annotation

GitHub Actions / Darwin Go tests 

cannot convert device (variable of type PCIPassthroughAllowedDevice) to type "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/driver".PCIPassthroughAllowedDevice

Check failure on line 142 in builder/vsphere/common/step_hardware.go

View workflow job for this annotation

GitHub Actions / Windows Go tests 

cannot convert device (variable of type PCIPassthroughAllowedDevice) to type "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/driver".PCIPassthroughAllowedDevice
}

err := vm.Configure(&driver.HardwareConfig{
CPUs: s.Config.CPUs,
CpuCores: s.Config.CpuCores,
Expand All @@ -102,6 +155,7 @@ func (s *StepConfigureHardware) Run(_ context.Context, state multistep.StateBag)
MemoryHotAddEnabled: s.Config.MemoryHotAddEnabled,
VideoRAM: s.Config.VideoRAM,
Displays: s.Config.Displays,
AllowedDevices: allowedDevices,
VGPUProfile: s.Config.VGPUProfile,
Firmware: s.Config.Firmware,
ForceBIOSSetup: s.Config.ForceBIOSSetup,
Expand Down
Loading

0 comments on commit c53c318

Please sign in to comment.