Page 1 of 2

Disk full

Posted: Wed Mar 05, 2014 7:40 pm
by CCampbell
So I downloaded the eval version of Starwind and I'm testing it out. My specs are Windows 2012 R2 (bare-metal/non-VM) on a Dell PE 2950 (Dual Quad core 2.0Ghz) w. 48GB of RAM and ~1.5TB of SSD disks in a RAID array. My iSCSI port is an Intel 10GB NIC. Management is the onboard 1GB NIC.

I setup a deduplicated disk for my VMWare cluster of about 1.25TB in Starwinds. So I had some spare space left on the D drive. Things went fine for awhile. I used it to test vMotions mostly, but ran a couple of cloned test VMs to test the de-duplication features as well. But now I have a file on my d drive ending in .spdata that's 1.5TB and all the drive space is used. Meanwhile at the time it did this I didn't have anything running on it. I'm trying to understand why the file would just grow past the limit I set for it and then just blow up. I've attempted to remount it now after a rebooting the server running Starwinds but my ESX hosts just act like the iSCSI storage doesn't have a VMFS volume on it anymore and are offering to format it for me.
Help? Thoughts?

Re: Disk full

Posted: Wed Mar 05, 2014 8:42 pm
by anton (staff)
StarWind LSFS volumes need to be INSANELY overprovisioned as we do redirect-on-write basically and untill trimmed your disks has history of all writes you had applied (automation of this process is what we're busy with right now). So don't give away more then 50% of your disk as LSFS volume for now.

Re: Disk full

Posted: Wed Mar 05, 2014 9:17 pm
by CCampbell
So is an LSFS volume unique to Dedupe disks? Or is that an underlying the whole Starwinds product technology and I should account for that in all disk types?

Re: Disk full

Posted: Wed Mar 05, 2014 9:30 pm
by anton (staff)
FLAT (image files) are immune to this as they are thick-provisioned so pre-allocated.
CCampbell wrote:So is an LSFS volume unique to Dedupe disks? Or is that an underlying the whole Starwinds product technology and I should account for that in all disk types?

Re: Disk full

Posted: Wed Mar 19, 2014 4:30 pm
by fbifido
Hi,

Had a similar problem, but I disable dedup, to see if that was the problem. But now I know its an LSFS problem.
made an LSFS LUN 1.17TB,
created VM disk of 1024MB,
copy 500GB of documents to this disk,

Now in my VM that is using this LUN, on the disk I have 557GB Free, but on the iSCSI physical disk it say I used 1.2TB,
lsfs_size_on_disk.png
lsfs_size_on_disk.png (94.31 KiB) Viewed 9876 times
This seem strange to me, so I right-click the disk in my VM and select properties, used=466GB, free=557GB, total=0.99TB,
but my backup software says it copied 757GB of data and it should also keep a copy of all old documents for 3 months.
lsfs_size_on_disk2.png
lsfs_size_on_disk2.png (211.32 KiB) Viewed 9876 times
This was not adding up, so I went into the disk (drive E:) and did a Ctrl+A in the root of that drive, the right-click, then select properties
that's when I remember that I had enabled compression and dedup inside the VM(file server with win2012R2).

This is somewhat brain-mashing, I have:
1.1M Files
77K Folders
size of all files = 814GB
size of all files on Disk = 6.02GB

I for one can not explain all these different numbers/reading, so am looking for a way to get back some of the free space on my iSCSI Physical Disks, and I found 2:

1) sdelete v1.61 (non-gui) http://technet.microsoft.com/en-us/sysi ... 97443.aspx
2) perfectstorage [ ... ]

I am going to try them, to see which is better.

Thanks

Re: Disk full

Posted: Thu Mar 20, 2014 10:14 am
by Anatoly (staff)
Thank you for the information. We`ll look forward to hear back from you.

Also I~d like to ask community to post the beta related question on our Beta forum that you can find by using the link below:
http://www.starwindsoftware.com/forums/beta-f17/

Thank you

Re: Disk full

Posted: Thu Mar 20, 2014 4:46 pm
by fbifido
Anatoly (staff) wrote: Also I~d like to ask community to post the beta related question on our Beta forum that you can find by using the link below: http://www.starwindsoftware.com/forums/beta-f17/

Thank you
Sorry about that, can my post be moved ?

*** Update ****
I ran sdelete -z e:
it tock a while, this program was run from inside the VM.

The pic below is from my iSCSI LUN storage directory, that my E: is on.
lsfs_size_on_disk3.png
lsfs_size_on_disk3.png (14.33 KiB) Viewed 9835 times
the files(*.spspx) on my storage when up, not down.
how I have 136 files from pre-128-files
with 1.37TB in size from 1.29TB.

Re: Disk full

Posted: Thu Mar 20, 2014 7:28 pm
by fbifido
*** Update #2 ****

Now I went on the esxi 5.5 server using putty

cd /vmfs/volumes/
esxcli storage vmfs unmap -l "Files"

it took about 2 hrs, with just the one VM disk on it, but I notice something strange, see if you could spot it:
lsfs_size_on_disk4.png
lsfs_size_on_disk4.png (20.52 KiB) Viewed 9822 times
I did select thin provisioned when creating this Disk, so why is it flat.
Now I have to restart my test again.

Question: What is the best thin option to use in esxi 5.5 ?

forgot to state that the amount of .spspx files still remain the same(136) as well as the size(1.37TB).
so unmap did not work for me, mainly I thing because of the flat file on the disk image.

Re: Disk full

Posted: Thu Mar 20, 2014 7:46 pm
by fbifido
**** update #3 *****

Just check my VM re the disk & flat attach to file name.
it is indeed thin, so why ?
lsfs_size_on_disk5.png
lsfs_size_on_disk5.png (27.68 KiB) Viewed 9823 times

Re: Disk full

Posted: Thu Mar 20, 2014 8:19 pm
by fbifido
**** Update #4 *****
Ok, just added another VM thin provisioned disk, but added it to a dedup LUN, its only 800GB.
this is how the setup looks now:
lsfs_size_on_disk6a.png
lsfs_size_on_disk6a.png (93.26 KiB) Viewed 9809 times
The datastore say the new disk is using 116MB
The iSCSI physical disk says that LUN is using 8GB
while the esxi command prompt says I am using the full 700GB allocated.

this is the datastore:
lsfs_size_on_disk7.png
lsfs_size_on_disk7.png (20.12 KiB) Viewed 9809 times
this is the SW-iSCSI LUN:
lsfs_size_on_disk7a.png
lsfs_size_on_disk7a.png (15.28 KiB) Viewed 9808 times
I will move the documents from E: drive to F: drive

Re: Disk full

Posted: Thu Mar 20, 2014 8:56 pm
by fbifido
The file are now moving from the Files LUN to the Emails LUN
started at 20-03-2014 @ 3PM
lsfs_size_on_disk7b.png
lsfs_size_on_disk7b.png (57.09 KiB) Viewed 9816 times
it's now coping at a about 22MB/s to 32MB/s
lsfs_size_on_disk7c-bandwidth.png
lsfs_size_on_disk7c-bandwidth.png (17.5 KiB) Viewed 9816 times
lsfs_size_on_disk7c-cpu&ram.png
lsfs_size_on_disk7c-cpu&ram.png (23.31 KiB) Viewed 9815 times

Re: Disk full

Posted: Thu Mar 20, 2014 8:56 pm
by fbifido
lsfs_size_on_disk7c-iops.png
lsfs_size_on_disk7c-iops.png (21.16 KiB) Viewed 9812 times

Re: Disk full

Posted: Fri Mar 21, 2014 5:49 pm
by fbifido
**** Update #5 ****
its was taking too long so I just stop the move, at around 400GB, then format my e: drive in the VM
run sdelete -z e:\
the ran, on the esxi server, esxcli ...unmap -l "Files"

after all of that, my Storage server did not reduce the space use,
its says 133 Files @ 1.34TB

vcenter is as it was, with the Files LUN still @ 167GB free. but the Emails LUN is now @ 415GB Free

Next I will try to delete the Files.vmdk from the Files datastore, if that does not free up disk space, then I will delete the Files LUN and use NFS datastore, to free up disk space.

I will now look at the Emails LUN, because it has dedup enable, I will see how it works out:
size that vcenter report ?
size that SW iSCSI is using (.spspx) ?
size of directory the Emails LUN is located ?
size of drive in VM (free & Used) ?

Re: Disk full

Posted: Mon Mar 24, 2014 10:58 am
by Anatoly (staff)
R&D confirmed that we can have that issue. QA department is testing the fix for it right now. As soon as we`ll be sure that it works we will update RC or put the fix into the RTM build.

Re: Disk full

Posted: Mon Mar 24, 2014 3:36 pm
by fbifido
**** UPDATE #?? *****

Manage to get the sdelete to work from within the VM.
1. do a full format, not quick, on the disk
2. run the sdelete -z <drive letter> ( eg: e:)
3. on esxi run the esxcli ... unmap -l command
4. run sdelete again, now it will work

BUT, this is a big but, no pond intended, VMware will not see any change is free space.
I do not know if it's a VMware issue or SW.
lsfs_size_on_disk8.png
lsfs_size_on_disk8.png (129.14 KiB) Viewed 9733 times