StarWind Software

Dear sir/miss,

StarLVM plugin (starwind-proxmox-plugin-pve9 1.1.41, PVE 9.2.21): snapshot rollback recreates the disk LVs with lvcreate -kn -n vm-<id>-disk-<n> -s standard/snap_vm-<id>-disk-<n>_<snapname>, which clears ACTIVATION_SKIP. All other plugin operations use -ky. With the shared VG, LVM event autoactivation (vgchange -aay --autoactivation event) on any node that boots then activates these thin LVs and their pool while the VM runs on another node. On 2026-10-05 this happened on two nodes during a rolling reboot, the pool was active on two nodes at once, and the thin pool metadata was corrupted (thin_check status 64 on next activation). We have the LVM archives, journals and the original tmeta (as an image file) available. Please fix the rollback to use -ky, and confirm whether vgchange --setautoactivation n on the shared VG is supported with your plugin.

Hi,

Can you please pull the plugin logs from both nodes?
Use /opt/starwind/starwind-proxmox-plugin/collect_pve_logs.sh

Upload them here https://www.starwindsoftware.com/support-form. 1560299 - is your reference

Yaroslav,

I've uploaded the logs as requested. I think the code below could solve the issue, be careful, as I might've made mistakes though

Yaroslav,

Another issue came up when searching through the logs (partially Claude assisted, but found myself in dmesg).

The LVM commands keep running after the cluster storage lock expires
These operations run their LVM commands inside cluster_lock_storage(), so that only one node changes the shared VG at a time:
- volume_resize (line 621)
- volume_snapshot (line 640)
- volume_snapshot_rollback (line 660)
- volume_snapshot_delete (line 676)
- rename_snapshot (line 690)
- StarLvmMonitor.pl (line 116)

PVE's cfs_lock sets alarm(60) around the locked code (pve-cluster, src/PVE/Cluster.pm: "fixed command timeout: cfs locks have a timeout of 120"). run_command (pve-common, src/PVE/Cmd.pm) only installs its own SIGALRM handler and kills the child when it is given a "timeout" parameter. The plugin never passes one. When the 60 s alarm fires, the Perl code dies and cfs_lock releases the lock, but the lvcreate, lvremove or lvextend child keeps running. From then on, other nodes can take the storage lock and write VG metadata at the same time as the still-running command. Without lvmlockd, this can corrupt the metadata of the whole VG, not just one pool.

Thin pool operations easily take longer than 60 s under load, because each thin volume and the pool are suspended in turn.

Evidence
On 2026-10-10 on idkstbabshv05, StarLvmMonitor.pl started "lvextend --use-policies standard/lvmth-1033". It stayed in D state in thin_presuspend/thin_postsuspend for about 30 minutes. By the code above, the lock was released after 60 s, so the extend ran unlocked for about 29 minutes.

Expected
The lock is held until the LVM command has finished, or the operation fails without leaving a running LVM command behind. Alternatively, document and support lvmlockd for the shared VG.

Claude's suggested fix for this issue is to wrap every LVM command that runs under the lock in a helper. The 60 second abort is replaced by a timer that renews the lock every 30 seconds and only would returns when the command is really finished.

AFAIK, a timeout extension for the plugin is already on the to-do list. Please wait for the next build.
Please tell me more about the system
1. storage provider
2. protocol over which the storage is connected.
3. Proxmox version.

yaroslav (staff) wrote: ↑

Thu Oct 15, 2026 3:08 pm

AFAIK, a timeout extension for the plugin is already on the to-do list. Please wait for the next build.
Please tell me more about the system
1. storage provider
2. protocol over which the storage is connected.
3. Proxmox version.

1. Fujitsu Siemens Eternus with a 4x Dell PowerEdge R740 and 6x Fujitsu servers (2 separate clusters with 2 separate volumes)
2. We're using iSCSI 10Gbps with Multipath to 2x Cisco Nexus 9300 series switches.
3. Proxmox version on all hosts is 9.2.21

Another few issues that I've encountered based on the logs
- During live migration, the thin pool is active on two nodes
- LVM commands keep running after the cluster storage lock expires
- StarLvmMonitor.pl reports success when the extend did not run
- Pools left active on other nodes are not detected, and their space is reused

Evidence of these bugs can be found in the logs
- We hit the same pattern with a longer window. On 2026-10-05, shv02 activated lvmth-1005 at boot (autoactivation, 11:24:35) while VM 1005 was running on shv05. VM 1005 was live-migrated onto shv02 at 11:29-11:30. That VM had had a snapshot since 2026-09-17, so most of its writes were copy-on-write. Days later, the pool failed thin_check with status 64.

- After lvmth-1005 was removed and recreated on shv05 (2026-10-15), shv03 still had live dm mappings of the old lvmth-1005 pool and disks. The VM was not running there. We removed them by hand. An earlier case on shv02 left orphaned dm mappings of a deleted pool (lvmth-1001), with "Failed to suspend ... with queued messages" when the VMID was reused.

- On 2026-10-10 on shv05, StarLvmMonitor.pl started "lvextend --use-policies standard/lvmth-1033". It stayed in D state in thin_presuspend/thin_postsuspend for about 30 minutes, during a heavy initial write into a new VM. By the code above, the lock was released after 60 s, so the extend ran unlocked for about 29 minutes.

I've also put these details in the ticket via an e-mail update.