Provider deletes PVs created by proxmox-csi-plugin

[TomyLobo]

I'm doing the following:

1. (Telmate/terraform-provider-proxmox) Create a machine on Proxmox
   • The VM has an SCSI disk named scsi0
2. (siderolabs/terraform-provider-talos) Install and bootstrap a Talos/Kubernetes cluster
   • Installing to /dev/sda
3. (sergelogvinov/proxmox-csi-plugin) Provision PersistentVolumes to fulfill PersistentVolumeClaims with accessMode ReadWriteOnce
   • This provisions extra SCSI disks starting from scsi1, i.e. /dev/sdb
   • These disks belong to a non-existant vmid 9999 and are attached to the VM that runs the pods that need to mount the volume.

This works fine, but if I then run terraform plan again, it says it wants to remove the disks provisioned by proxmox-csi-plugin.

A workaround I tried was to put disks (or even just disks[0].scsi) into ignore_changes.
That works to prevent Terraform from removing my disks.
However, that also prevents me from increasing the size of scsi0 in order to, for instance, accommodate more and/or larger container images than originally anticipated.

[Tinyblargon]

@TomyLobo are they normal disks?
Is it always vmid 9999?

Maybe with a bit of collaboration with proxmox-csi-plugin this could work but we would require a way to know it created the disk, for example if they prefix the serial with externally-managed we would know not to touch the disk.

Before we should consider moving forward with this the following has to be done:

1. Rework the Qemu disks in the SDK so not configured does not mean remove, but don't change like everywhere else in that project.
2. Check if we can suppress the terraform state based on a sub property like serial.

[TomyLobo]

The Proxmox web interface says this:

Hard Disk (scsi1)	ssd1:vm-9999-pvc-03d18b7a-485a-4102-b896-6aeb86198b9f,backup=0,iothread=1,size=1G,wwn=0x5056432d49443031

I guess by serial, you mean the wwn.
If I reinterpret that as ASCII (UTF-8, actually), I get the following:

$ sed 's/^0x//; s/../& /g' <<< '0x5056432d49443031' | xxd -r -p; echo
PVC-ID01

That could be used to detect it.

Additionally, the disk's name contains the PV name and that's generated by proxmox-csi-plugin.
That seems to always start with "pvc-", followed by a UUID, so that could be detected.
Haven't found the code section that generates those names, though. I suspect it's in some library.

P.S.: my thought process here, and why I took some time to answer, was this:

1. that's only a 2-digit number, I wonder what happens if I create more than 100 volumes per node
2. find the code section that generates it, see it's actually only using lun 1-30 (Proxmox's limit is 0-30, but lun 0 is usually used for the system disk anyway)
3. find another code section where it's actually even lower (24)
4. oh shit, that might not be enough for the things we plan to do with the cluster