Unfortunately we had data corruption again, this time while live migrating virtual machines.
What happened
A rolling update (node evacuation by live migration, then a reboot) moved
VM 2025 (Windows Server, 4 GiB RAM, pool legacy/lvmth-2025, no snapshots)
twice:
- 06:39:15 lhv02 -> lhv03 (evacuating lhv02)
- 06:53:06 lhv03 -> lhv02 (evacuating lhv03)
Second migration, all times CEST on 2026-10-23. Sources: the qmigrate task log
on lhv03, the journal on lhv03, and the kernel log on lhv02.
06:53:06 lhv03 qmigrate 2025 starts
06:53:08 lhv03 "starting VM 2025 on remote node 'idkstbablhv02'"
06:53:11 lhv02 lvmth-2025 activated (tdata dm-46, tpool 252:47),
so the target loads the pool metadata here
06:53:15 lhv03 RAM transfer starts; the guest keeps running and writing
on lhv03 for the next 55 seconds (6.4 GiB sent for
4.0 GiB RAM, xbzrle active, downtime auto-raised to
200 ms: a busy guest)
06:54:10 lhv03 "migration completed", downtime 140 ms
-> VM 2025 resumes on lhv02
06:54:10 lhv03 " deactivating vm-2025-disk-2/1/0..."
06:54:13 lhv03 dmeventd: "No longer monitoring thin pool
legacy-lvmth--2025-tpool." (pool released on source)
06:55:41 lhv02 kernel:
device-mapper: space map common: metadata block out of bounds
device-mapper: thin: 252:47: metadata operation
'dm_thin_insert_block' failed: error = -22
device-mapper: thin: 252:47: aborting current metadata transaction
device-mapper: thin: 252:47: switching pool to read-only mode
07:11 lhv02 Buffer I/O errors on dm-49, "lost async page write"
The guest ended in Windows Automatic Repair. On 2026-10-26, "lvchange -an"
reported "Integrity check of metadata for pool legacy/lvmth-2025 failed". We
removed the pool and restored the VM from backup.
What we ruled out
- Space: the VG had 2.95 TiB free, the pool was at 63.69% data and 29.49%
metadata. (The restore attempt also printed "free space in thin pool
reached threshold"; with the pool at 64% we read that as LVM refusing a
pool it could not write to, not a real space problem.)
- Snapshots: VM 2025 had none.
- LVM autoactivation (the lvmth-1005 path): on all 4 nodes, every early pool
activation on 2026-10-23 matches a migration or startall to the second,
and every boot activated 0 LVs.
- A third node: lvmth-2025 was active only on lhv02 afterwards.
- No other dm-thin errors on any node. Every other active pool is rw, has a
matching kernel and LVM transaction id, and no needs_check.
About 31 other VMs made the same migrations that morning without a visible
error.
Root cause
This is exactly Bug 1 from our earlier report, now with timing.
The target loaded the pool metadata at 06:53:11. The source kept allocating
and committing for another 59 seconds. The VM then resumed on the target with
an in-kernel copy of the metadata that was about a minute old. The source
released the pool 3 seconds after the resume. The target's next metadata
operation ran into blocks the source had changed, and the pool went
read-only.
Three corrections to the earlier report
- No snapshot is needed. VM 2025 had none, so writes to blocks that were
never provisioned are enough. The pool had about 36% of its data space
unallocated.
- The trigger here was the live migration itself, not boot autoactivation.
The lvmth-1005 case (autoactivation after a rollback stripped
ACTIVATION_SKIP, Bug 1 option B and the -kn rollback issue) is a second
path to the same condition: a pool active on two nodes.
- Damage that doesn't show up as a metadata error is silent: lost mappings
for writes made during the migration window, or blocks handed out twice.
We cannot rule this out for the other VMs that were live-migrated, and we
will thin_check their pools at their next planned restart.
Why the plugin can't fix this without help from Proxmox
The only safe moment to move the pool is after the source has paused and
flushed, and before the target resumes. qemu-server (9.2.11 to 9.2.13)
calls no storage plugin in that window:
- target: vm_start_nolock sets $ENV{PVE_MIGRATED_FROM} and calls
activate_volumes BEFORE the RAM transfer (QemuServer.pm, around line 5632)
- resume: phase3_cleanup sends "resume" to the target (QemuMigrate.pm,
around line 1749); vm_resume (QemuServer/RunState.pm) makes no storage call
- source: deactivate_volumes only runs in vm_stop_cleanup, AFTER the resume
(QemuMigrate.pm, around line 1872)
So the plugin can either refuse the dual activation (this rules out live
migration, our option B) or accept the race. A safe handover needs a hook in
qemu-server. This is option A from our earlier report, which we have now
implemented.
Proposed patch
Note: This patch is made with the assistance of Claude AI, so be careful and check it thoroughly.
Code: Select all
--- a/usr/share/perl5/PVE/Storage/Custom/StarLvmPlugin.pm
+++ b/usr/share/perl5/PVE/Storage/Custom/StarLvmPlugin.pm
@@ -16,9 +16,19 @@
use PVE::Tools qw(run_command trim);
use PVE::Storage::Plugin;
use PVE::Storage::LVMPlugin;
+use PVE::INotify;
+use PVE::SSHInfo;
my $PLUGIN_VERSION = '1.6';
+# Live migration handover, opt-in. A dm-thin pool loads its metadata once, when it is activated.
+# Live migration activates the pool on the target while the source still writes, so the target
+# would resume with stale metadata. With handover the target keeps the pool read-only until the
+# source has released it, then reloads it if the source changed anything meanwhile.
+# Needs qemu-server with storage_migration_handover() on every node.
+our $HANDOVER_FLAG = '/etc/pve/starlvm-live-handover';
+our $HANDOVER_STATE_DIR = '/run/starlvm-handover';
+
use base qw(PVE::Storage::LVMPlugin);
sub api {
@@ -403,6 +413,336 @@
$class->SUPER::activate_storage($storeid, $scfg, $cache);
}
+sub handover_enabled {
+ return 0 if !-e $HANDOVER_FLAG;
+ # without the qemu-server hook nobody would switch the pool back to read-write
+ return eval {
+ require PVE::QemuMigrate;
+ PVE::QemuMigrate->can('storage_migration_handover');
+ } ? 1 : 0;
+}
+
+my sub handover_state_file {
+ my ($vg, $pool) = @_;
+ return "$HANDOVER_STATE_DIR/$vg-$pool";
+}
+
+# device-mapper name of an LV, e.g. legacy/lvmth-2025_tmeta -> legacy-lvmth--2025_tmeta
+my sub dm_name {
+ my ($vg, $lv) = @_;
+ return join('-', map { s/-/--/gr } ($vg, $lv));
+}
+
+my sub dm_exists {
+ my ($name) = @_;
+ my $found = 0;
+ eval {
+ run_command(['/sbin/dmsetup', 'info', '-c', '--noheadings', '-o', 'name', $name],
+ outfunc => sub { $found = 1 if trim(shift) eq $name }, errfunc => sub {});
+ };
+ return $found;
+}
+
+my sub dm_table {
+ my ($name) = @_;
+ my @lines;
+ run_command(['/sbin/dmsetup', 'table', $name],
+ outfunc => sub { push @lines, trim(shift) },
+ errmsg => "dmsetup table '$name' failed");
+ die "expected one table line for '$name', got " . scalar(@lines) . "\n" if @lines != 1;
+ return $lines[0];
+}
+
+# load a new table into an existing device, whoever has it open keeps using the same device
+my sub dm_swap_table {
+ my ($name, $table) = @_;
+ run_command(['/sbin/dmsetup', 'load', $name, '--table', $table], errmsg => "dmsetup load '$name' failed");
+ eval {
+ run_command(['/sbin/dmsetup', 'suspend', $name], timeout => 60, errmsg => "dmsetup suspend '$name' failed");
+ };
+ if (my $err = $@) {
+ eval { run_command(['/sbin/dmsetup', 'clear', $name]) };
+ die $err;
+ }
+ run_command(['/sbin/dmsetup', 'resume', $name], errmsg => "dmsetup resume '$name' failed");
+}
+
+# point a thin LV at the error target, so it no longer holds a reference on its pool
+my sub thin_lv_detach {
+ my ($vg, $lv) = @_;
+ my $name = dm_name($vg, $lv);
+ return if !dm_exists($name);
+ my $table = dm_table($name);
+ return if $table =~ m/^0 \d+ error$/;
+ die "unexpected table for '$name': $table\n" if $table !~ m/^0 (\d+) thin /;
+ dm_swap_table($name, "0 $1 error");
+}
+
+my sub pool_set_read_only {
+ my ($vg, $pool, $read_only) = @_;
+ my $name = dm_name($vg, $pool) . '-tpool';
+ my $table = dm_table($name);
+ # 0 <len> thin-pool <metadata dev> <data dev> <block size> <low water mark> <#features> [<feature>...]
+ my @f = split(/\s+/, $table);
+ die "unexpected table for '$name': $table\n"
+ if @f < 8 || $f[2] ne 'thin-pool' || $f[7] !~ m/^\d+$/ || @f != 8 + $f[7];
+ my @features = grep { $_ ne 'read_only' } @f[8 .. $#f];
+ push @features, 'read_only' if $read_only;
+ my $new = join(' ', @f[0 .. 6], scalar(@features), @features);
+ dm_swap_table($name, $new) if $new ne $table;
+}
+
+my sub pool_check_healthy {
+ my ($vg, $pool) = @_;
+ my $name = dm_name($vg, $pool) . '-tpool';
+ my $status = '';
+ # --noflush: a plain status commits the pool metadata
+ run_command(['/sbin/dmsetup', 'status', '--noflush', $name],
+ outfunc => sub { $status = trim(shift) },
+ errmsg => "dmsetup status '$name' failed");
+ # 0 <len> thin-pool <transaction id> <meta used/total> <data used/total> <held root> <mode> ...
+ my @f = split(/\s+/, $status);
+ die "unexpected status for '$name': $status\n" if @f < 8 || $f[2] ne 'thin-pool';
+ die "thin pool $vg/$pool is in mode '$f[7]', expected rw\n" if $f[7] ne 'rw';
+ die "thin pool $vg/$pool needs a check\n" if grep { $_ eq 'needs_check' } @f;
+}
+
+# Fingerprint of the on-disk superblock of a thin pool, it changes with every metadata commit.
+# Read straight from the PV, so it also works while the pool is inactive on this node.
+my sub pool_superblock_digest {
+ my ($vg, $pool) = @_;
+
+ my ($pv, $pe, $extent_size);
+ run_command(['/sbin/lvs', '-a', '--segments', '--noheadings', '--units', 'b', '--nosuffix',
+ '--separator', '|', '-o', 'seg_start_pe,seg_pe_ranges,vg_extent_size',
+ '-S', "lv_name=${pool}_tmeta", $vg],
+ outfunc => sub {
+ my ($start, $ranges, $size) = split(/\|/, trim(shift));
+ return if !defined($start) || $start ne '0'; # the superblock is block 0 of the LV
+ ($pv, $pe) = $ranges =~ m|^(/dev/[-/\w.]+):(\d+)-\d+$|;
+ $extent_size = $size;
+ },
+ errmsg => "lvs for $vg/${pool}_tmeta failed");
+ die "can't locate the metadata of $vg/$pool\n" if !$pv || !defined($pe) || !$extent_size;
+
+ my $pe_start;
+ run_command(['/sbin/pvs', '--noheadings', '--units', 'b', '--nosuffix', '-o', 'pe_start', $pv],
+ outfunc => sub { $pe_start = trim(shift) },
+ errmsg => "pvs for $pv failed");
+ die "can't read pe_start of $pv\n" if !defined($pe_start) || $pe_start !~ m/^\d+$/;
+
+ my $offset = $pe_start + $pe * $extent_size;
+ my $digest;
+ run_command([['/bin/dd', "if=$pv", 'bs=4096', 'count=1', "skip=$offset",
+ 'iflag=direct,skip_bytes', 'status=none'], ['/usr/bin/sha256sum']],
+ outfunc => sub { $digest = $1 if $_[0] =~ m/^([0-9a-f]{64})\b/ },
+ errmsg => "reading the superblock of $vg/$pool failed");
+ die "no superblock digest for $vg/$pool\n" if !$digest;
+ return $digest;
+}
+
+# take a pool off this node: thin LVs stay open for QEMU but point at the error target,
+# removing the pool stack suspends the pool, which commits its metadata a last time
+my sub handover_release {
+ my ($vg, $pool, $volumes) = @_;
+ run_command(['/sbin/lvchange', '--monitor', 'n', "$vg/$pool"], errmsg => "monitor=no for pool '$vg/$pool' error");
+ thin_lv_detach($vg, $_) for @$volumes;
+ my $base = dm_name($vg, $pool);
+ for my $name ($base, "$base-tpool", dm_name($vg, "${pool}_tdata"), dm_name($vg, "${pool}_tmeta")) {
+ next if !dm_exists($name);
+ run_command(['/sbin/dmsetup', 'remove', $name], timeout => 60, errmsg => "dmsetup remove '$name' failed");
+ }
+}
+
+# activate a pool fresh (LVM runs thin_check) and map the thin LVs onto it again
+my sub handover_reattach {
+ my ($vg, $pool, $volumes) = @_;
+ my $vol = "$vg/$pool";
+ run_command(['/sbin/lvchange', '-aly', '-K', '--monitor', 'y', $vol], errmsg => "activating thin pool '$vol' failed");
+ for my $lv (@$volumes) {
+ run_command(['/sbin/lvchange', '--refresh', "$vg/$lv"], errmsg => "can't refresh LV '$vg/$lv'");
+ my $table = dm_table(dm_name($vg, $lv));
+ die "$vg/$lv is not mapped to its pool after the refresh: $table\n" if $table !~ m/^0 \d+ thin /;
+ }
+ pool_check_healthy($vg, $pool);
+}
+
+my sub ssh_node_cmd {
+ my ($node, $cmd) = @_;
+ my $ssh = PVE::SSHInfo::ssh_info_to_command(PVE::SSHInfo::get_ssh_info($node));
+ return [@$ssh, '--', PVE::Tools::cmd2string($cmd)];
+}
+
+# the source must run the qemu-server hook, otherwise the pool would stay read-only here
+my sub handover_source_ready {
+ my ($source) = @_;
+ my $check = ['/usr/bin/perl', '-MPVE::QemuMigrate', '-e',
+ 'exit(PVE::QemuMigrate->can(q{storage_migration_handover}) ? 0 : 1)'];
+ eval { run_command(ssh_node_cmd($source, $check), timeout => 30, outfunc => sub {}, errfunc => sub {}) };
+ die "source node '$source' has no qemu-server with storage_migration_handover(), refusing\n" if $@;
+}
+
+# incoming live migration: load the pool read-only and remember the superblock fingerprint
+my sub handover_prepare_pool {
+ my ($class, $storeid, $vg, $pool, $v, $source) = @_;
+ my $vol = "$vg/$pool";
+ my $state_file = handover_state_file($vg, $pool);
+
+ if ($v->{$pool}->{lv_state} eq 'a') {
+ my $state = -e $state_file ? PVE::Tools::file_get_contents($state_file) : '';
+ return if $state =~ m/^source=\Q$source\E$/m; # an earlier volume of this migration
+ # active here since before this migration, so its metadata is of unknown age
+ die "thin pool '$vol' is already active on this node, refusing the incoming live migration\n";
+ }
+
+ handover_source_ready($source);
+
+ # the storage lock keeps the auto-extend monitor from resizing the pool in between
+ $class->cluster_lock_storage($storeid, 1, 60, sub {
+ # fingerprint first: when the source commits before the kernel loads the metadata, the
+ # handover sees a change and reloads, instead of trusting a fingerprint that is too new
+ my $digest = pool_superblock_digest($vg, $pool);
+ run_command(['/sbin/lvchange', '-aly', '-K', '--monitor', 'n', $vol], errmsg => "activating thin pool '$vol' failed");
+ $v->{$pool}->{lv_state} = 'a'; # update cached state
+ pool_set_read_only($vg, $pool, 1);
+ mkdir($HANDOVER_STATE_DIR, 0700);
+ PVE::Tools::file_set_contents($state_file, "source=$source\ndigest=$digest\n");
+ });
+}
+
+# target side, called by migration_handover() over SSH once the source has released the pool
+my sub handover_attach {
+ my ($vg, $pool, $source, $volumes) = @_;
+
+ my $state_file = handover_state_file($vg, $pool);
+ my $state = -e $state_file ? PVE::Tools::file_get_contents($state_file) : '';
+ my ($from) = $state =~ m/^source=(\S+)$/m;
+ my ($digest) = $state =~ m/^digest=([0-9a-f]{64})$/m;
+ die "no handover state for $vg/$pool on this node\n" if !$from || !$digest;
+ die "handover state of $vg/$pool belongs to '$from', not '$source'\n" if $from ne $source;
+
+ my $mode;
+ if (pool_superblock_digest($vg, $pool) eq $digest) {
+ # the source changed nothing after this node loaded the metadata
+ pool_set_read_only($vg, $pool, 0);
+ run_command(['/sbin/lvchange', '--monitor', 'y', "$vg/$pool"], errmsg => "monitor=yes for pool '$vg/$pool' error");
+ for my $lv (@$volumes) {
+ my $table = dm_table(dm_name($vg, $lv));
+ die "$vg/$lv is not mapped to its pool: $table\n" if $table !~ m/^0 \d+ thin /;
+ }
+ pool_check_healthy($vg, $pool);
+ $mode = 'unchanged';
+ } else {
+ # the source committed after this node loaded the metadata, so load it again
+ handover_release($vg, $pool, $volumes);
+ handover_reattach($vg, $pool, $volumes);
+ $mode = 'reloaded';
+ }
+
+ unlink($state_file);
+ return $mode;
+}
+
+sub handover_attach_cli {
+ my ($vg, $pool, $source, @volumes) = @_;
+
+ my $mode = eval { handover_attach($vg, $pool, $source, \@volumes) };
+ if (my $err = $@) {
+ chomp $err;
+ # take the pool off this node again, so the source can safely use it
+ eval {
+ handover_release($vg, $pool, \@volumes);
+ unlink(handover_state_file($vg, $pool));
+ };
+ if (my $err2 = $@) {
+ chomp $err2;
+ print "STARLVM-HANDOVER UNKNOWN $err - releasing failed too: $err2\n";
+ exit(3);
+ }
+ print "STARLVM-HANDOVER RELEASED $err\n";
+ exit(2);
+ }
+ print "STARLVM-HANDOVER OK $mode\n";
+ exit(0);
+}
+
+my sub handover_call_target {
+ my ($vg, $pool, $volumes, $target) = @_;
+
+ my $source = PVE::INotify::nodename();
+ my $cmd = ['/usr/bin/perl', '-MPVE::Storage', '-e',
+ 'PVE::Storage::Custom::StarLvmPlugin::handover_attach_cli(@ARGV)',
+ $vg, $pool, $source, @$volumes];
+
+ my ($status, $msg) = ('UNKNOWN', 'no answer from the target');
+ eval {
+ run_command(ssh_node_cmd($target, $cmd), timeout => 300, noerr => 1,
+ outfunc => sub {
+ my $line = shift;
+ if ($line =~ m/^STARLVM-HANDOVER (OK|RELEASED|UNKNOWN)\s*(.*)$/) {
+ ($status, $msg) = ($1, $2);
+ } else {
+ print "[$target] $line\n";
+ }
+ },
+ errfunc => sub { print STDERR "[$target] $_[0]\n" });
+ };
+ $msg = "$@" if $@ && $status eq 'UNKNOWN';
+ chomp $msg;
+ return ($status, $msg);
+}
+
+# Called by qemu-server (storage_migration_handover) on the source node, after the live
+# migration converged and before the target resumes. The source VM is paused and flushed.
+sub migration_handover {
+ my ($class, $storeid, $scfg, $vmid, $volnames, $target) = @_;
+
+ return if !-e $HANDOVER_FLAG; # without the flag the target didn't load the pool read-only
+
+ my $vg = $scfg->{vgname};
+ my $lvs = lvm_list_volumes($vg)->{$vg} or die "no such vg $vg\n";
+
+ my $pools = {};
+ for my $volname (@$volnames) {
+ my $info = $lvs->{$volname} or die "no such volume $vg/$volname\n";
+ my $pool = $info->{pool_lv} or die "$vg/$volname is not a thin volume\n";
+ push $pools->{$pool}->@*, $volname;
+ }
+ return if !%$pools;
+ # with one pool per VM this doesn't happen; refusing here still lets the source resume
+ die "volumes in more than one thin pool can't be handed over\n" if scalar(keys %$pools) > 1;
+ my ($pool) = keys %$pools;
+ my $volumes = $pools->{$pool};
+
+ # the pool can only leave this node when nothing else here still uses it
+ my %moving = map { $_ => 1 } @$volumes;
+ for my $lv (sort keys %$lvs) {
+ next if ($lvs->{$lv}->{pool_lv} // '') ne $pool || $lvs->{$lv}->{lv_state} ne 'a';
+ die "$vg/$lv in pool $pool is active here but not part of this migration\n" if !$moving{$lv};
+ }
+
+ eval { handover_release($vg, $pool, $volumes) };
+ if (my $err = $@) {
+ eval { handover_reattach($vg, $pool, $volumes) };
+ die { msg => "releasing $vg/$pool failed ($err) and re-attaching it failed too: $@", keep_paused => 1 }
+ if $@;
+ die "releasing $vg/$pool failed: $err";
+ }
+
+ my ($status, $msg) = handover_call_target($vg, $pool, $volumes, $target);
+ if ($status eq 'OK') {
+ print " handed over $vg/$pool to $target ($msg)\n";
+ return;
+ }
+ if ($status eq 'RELEASED') {
+ eval { handover_reattach($vg, $pool, $volumes) };
+ die { msg => "$target refused ($msg) and re-attaching $vg/$pool here failed: $@", keep_paused => 1 }
+ if $@;
+ die "$target refused the handover of $vg/$pool: $msg\n";
+ }
+ die { msg => "state of $vg/$pool on $target is unknown: $msg", keep_paused => 1 };
+}
+
sub activate_volume {
my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
@@ -427,9 +767,15 @@
no locale;
die "Invalid pool name $pool" if $pool !~ /^(\w[-\w]*\w)$/;
- my $vol = "$vg/$1";
+ my $pool_name = $1;
+ my $vol = "$vg/$pool_name";
print "path $vol tainted!\n" if Scalar::Util::tainted($vol);
+ # target of a live migration: the source still writes to the pool until the handover
+ my ($source) = ($ENV{PVE_MIGRATED_FROM} // '') =~ m/^([-\w.]+)$/;
+ my $incoming = !$isBase && !$snapname && $source && handover_enabled();
+ handover_prepare_pool($class, $storeid, $vg, $pool_name, $v, $source) if $incoming;
+
return undef if $v->{$lv}->{lv_state} eq 'a';
if ($isBase) {
@@ -437,6 +783,8 @@
run_command(['/sbin/lvchange', '-an', '-K', '--monitor', 'n', $vol], errmsg => "deactivating thin pool '$vol' failed");
$v->{$pool}->{lv_state} = '-'; # update cached state
}
+ } elsif ($incoming) {
+ # handover_prepare_pool() activated the pool read-only, a refresh here would undo that
} else {
if ($v->{$pool}->{lv_state} ne 'a') {
run_command(['/sbin/lvchange', '-aly', '-K', '--monitor', 'y', $vol], errmsg => "activating thin pool '$vol' failed");
@@ -451,10 +799,14 @@
}
my $p = "$vg/$lv";
- run_command(['/sbin/lvchange', '-aly', '-K', $p], errmsg => "activating LV '$p' failed");
+ my @nomonitor = $incoming ? ('--config', 'activation/monitoring=0') : ();
+ run_command(['/sbin/lvchange', '-aly', '-K', @nomonitor, $p], errmsg => "activating LV '$p' failed");
$v->{$lv}->{lv_state} = 'a'; # update cached state
- run_command(['/sbin/lvchange', '--refresh', $p], errmsg => "can't refresh LV '$p' for activation");
+ run_command(['/sbin/lvchange', '--refresh', @nomonitor, $p], errmsg => "can't refresh LV '$p' for activation");
+
+ # the refresh can reload the pool table, so make sure the pool is still read-only
+ pool_set_read_only($vg, $pool_name, 1) if $incoming;
}
sub deactivate_volume {
@@ -509,6 +861,7 @@
run_command(['/sbin/lvchange', '-an', $p], errmsg => "deactivate_volume pool '$p' error");
$v->{$pool}->{lv_state} = '-'; # update cached state
run_command(['/sbin/lvchange', '--refresh', $p], errmsg => "can't refresh LV pool '$p' for deactivation");
+ unlink(handover_state_file($vg, $pool)); # an incoming migration that was aborted
}
}
It compiles and passes 33 behaviour tests against a simulated dmsetup/LVM (tests/handover.). It is NOT tested on hardware yet, and shouldn't be used in production, but as a reference.
I've made a post on the Proxmox forums about this too:
https://forum.proxmox.com/threads/live- ... er.187272/
