Disk full

[CCampbell]

So I downloaded the eval version of Starwind and I'm testing it out. My specs are Windows 2012 R2 (bare-metal/non-VM) on a Dell PE 2950 (Dual Quad core 2.0Ghz) w. 48GB of RAM and ~1.5TB of SSD disks in a RAID array. My iSCSI port is an Intel 10GB NIC. Management is the onboard 1GB NIC.

I setup a deduplicated disk for my VMWare cluster of about 1.25TB in Starwinds. So I had some spare space left on the D drive. Things went fine for awhile. I used it to test vMotions mostly, but ran a couple of cloned test VMs to test the de-duplication features as well. But now I have a file on my d drive ending in .spdata that's 1.5TB and all the drive space is used. Meanwhile at the time it did this I didn't have anything running on it. I'm trying to understand why the file would just grow past the limit I set for it and then just blow up. I've attempted to remount it now after a rebooting the server running Starwinds but my ESX hosts just act like the iSCSI storage doesn't have a VMFS volume on it anymore and are offering to format it for me.
Help? Thoughts?

[anton (staff)]

StarWind LSFS volumes need to be INSANELY overprovisioned as we do redirect-on-write basically and untill trimmed your disks has history of all writes you had applied (automation of this process is what we're busy with right now). So don't give away more then 50% of your disk as LSFS volume for now.

[CCampbell]

So is an LSFS volume unique to Dedupe disks? Or is that an underlying the whole Starwinds product technology and I should account for that in all disk types?

[anton (staff)]

FLAT (image files) are immune to this as they are thick-provisioned so pre-allocated.

So is an LSFS volume unique to Dedupe disks? Or is that an underlying the whole Starwinds product technology and I should account for that in all disk types?

[fbifido]

Hi,

Had a similar problem, but I disable dedup, to see if that was the problem. But now I know its an LSFS problem.
made an LSFS LUN 1.17TB,
created VM disk of 1024MB,
copy 500GB of documents to this disk,

Now in my VM that is using this LUN, on the disk I have 557GB Free, but on the iSCSI physical disk it say I used 1.2TB,

lsfs_size_on_disk.png (94.31 KiB) Viewed 14942 times

This seem strange to me, so I right-click the disk in my VM and select properties, used=466GB, free=557GB, total=0.99TB,
but my backup software says it copied 757GB of data and it should also keep a copy of all old documents for 3 months.

lsfs_size_on_disk2.png (211.32 KiB) Viewed 14942 times

This was not adding up, so I went into the disk (drive E:) and did a Ctrl+A in the root of that drive, the right-click, then select properties
that's when I remember that I had enabled compression and dedup inside the VM(file server with win2012R2).

This is somewhat brain-mashing, I have:
1.1M Files
77K Folders
size of all files = 814GB
size of all files on Disk = 6.02GB

I for one can not explain all these different numbers/reading, so am looking for a way to get back some of the free space on my iSCSI Physical Disks, and I found 2:

1) sdelete v1.61 (non-gui) http://technet.microsoft.com/en-us/sysi ... 97443.aspx
2) perfectstorage [ ... ]

I am going to try them, to see which is better.

Thanks

[Anatoly (staff)]

Thank you for the information. We`ll look forward to hear back from you.

Also I~d like to ask community to post the beta related question on our Beta forum that you can find by using the link below:
http://www.starwindsoftware.com/forums/beta-f17/

Thank you

[fbifido]

Also I~d like to ask community to post the beta related question on our Beta forum that you can find by using the link below: http://www.starwindsoftware.com/forums/beta-f17/

Thank you

Sorry about that, can my post be moved ?

*** Update ****
I ran sdelete -z e:
it tock a while, this program was run from inside the VM.

The pic below is from my iSCSI LUN storage directory, that my E: is on.

lsfs_size_on_disk3.png (14.33 KiB) Viewed 14901 times

the files(*.spspx) on my storage when up, not down.
how I have 136 files from pre-128-files
with 1.37TB in size from 1.29TB.

[fbifido]

*** Update #2 ****

Now I went on the esxi 5.5 server using putty

cd /vmfs/volumes/
esxcli storage vmfs unmap -l "Files"

it took about 2 hrs, with just the one VM disk on it, but I notice something strange, see if you could spot it:

lsfs_size_on_disk4.png (20.52 KiB) Viewed 14888 times

I did select thin provisioned when creating this Disk, so why is it flat.
Now I have to restart my test again.

Question: What is the best thin option to use in esxi 5.5 ?

forgot to state that the amount of .spspx files still remain the same(136) as well as the size(1.37TB).
so unmap did not work for me, mainly I thing because of the flat file on the disk image.

[fbifido]

**** update #3 *****

Just check my VM re the disk & flat attach to file name.
it is indeed thin, so why ?

lsfs_size_on_disk5.png (27.68 KiB) Viewed 14889 times

[fbifido]

**** Update #4 *****
Ok, just added another VM thin provisioned disk, but added it to a dedup LUN, its only 800GB.
this is how the setup looks now:

lsfs_size_on_disk6a.png (93.26 KiB) Viewed 14875 times

The datastore say the new disk is using 116MB
The iSCSI physical disk says that LUN is using 8GB
while the esxi command prompt says I am using the full 700GB allocated.

this is the datastore:

lsfs_size_on_disk7.png (20.12 KiB) Viewed 14875 times

this is the SW-iSCSI LUN:

lsfs_size_on_disk7a.png (15.28 KiB) Viewed 14874 times

I will move the documents from E: drive to F: drive

[fbifido]

The file are now moving from the Files LUN to the Emails LUN
started at 20-03-2014 @ 3PM

lsfs_size_on_disk7b.png (57.09 KiB) Viewed 14882 times

it's now coping at a about 22MB/s to 32MB/s

lsfs_size_on_disk7c-bandwidth.png (17.5 KiB) Viewed 14882 times

lsfs_size_on_disk7c-cpu&ram.png (23.31 KiB) Viewed 14881 times

[fbifido]

lsfs_size_on_disk7c-iops.png (21.16 KiB) Viewed 14878 times

[fbifido]

**** Update #5 ****
its was taking too long so I just stop the move, at around 400GB, then format my e: drive in the VM
run sdelete -z e:\
the ran, on the esxi server, esxcli ...unmap -l "Files"

after all of that, my Storage server did not reduce the space use,
its says 133 Files @ 1.34TB

vcenter is as it was, with the Files LUN still @ 167GB free. but the Emails LUN is now @ 415GB Free

Next I will try to delete the Files.vmdk from the Files datastore, if that does not free up disk space, then I will delete the Files LUN and use NFS datastore, to free up disk space.

I will now look at the Emails LUN, because it has dedup enable, I will see how it works out:
size that vcenter report ?
size that SW iSCSI is using (.spspx) ?
size of directory the Emails LUN is located ?
size of drive in VM (free & Used) ?

[Anatoly (staff)]

R&D confirmed that we can have that issue. QA department is testing the fix for it right now. As soon as we`ll be sure that it works we will update RC or put the fix into the RTM build.

[fbifido]

**** UPDATE #?? *****

Manage to get the sdelete to work from within the VM.
1. do a full format, not quick, on the disk
2. run the sdelete -z <drive letter> ( eg: e:)
3. on esxi run the esxcli ... unmap -l command
4. run sdelete again, now it will work

BUT, this is a big but, no pond intended, VMware will not see any change is free space.
I do not know if it's a VMware issue or SW.

lsfs_size_on_disk8.png (129.14 KiB) Viewed 14799 times