Page 1 of 1

Which dedup better?

Posted: Thu Oct 24, 2013 7:37 pm
by fbifido
Hi,

Just like to ask a question about StarWind iSCSI dedup.

Which of the 2 below dedup better, as in give the max amount of storage savings

1) one very large file, say 20GB, be it a doc, zip, iso, pdf or mp3
2) a bunch of small files, say about 20 - 1GB file, be it the same type as above

note-1) knowing that this is a technical question, let me say for a large file, I created a ISO file with all of windows os.
windows 8.1 core,pro,ent
windows 8.0 all
windows 7 all
windows server 2008 R2 all
windows server 2012 all
windows server 2012 R2 all
hyper-v 2012 r2

There are free software that allows you to do this. Now all those OS are now in one big iso file, with about 18 are so single os option to select and install.

note-2) using the info from above, say we extract each of those 18 plus option to each a file on its own, we will have a directory it say 18 or more iso files.


note-1a) the same can be done using pdf. I did it once using my outlook, I had about 200 or so emails that I needed to send to someone I a did not want to print them and to pdf each email was out of the question, and most of them had attachments(one or more), so I use acrobat pro 10(at that time) to create a book-like-pdf with all 200 emails, it seems to keep all the email structure including time,date, etc....
but it was around 3GB, if I had use max compress maybe smaller.

Re: Which dedup better?

Posted: Fri Oct 25, 2013 10:16 am
by anton (staff)
1) Everything you list suck except ISO :)

DOCs are zipped XMLs, ZIP, PDF and MP3 are all compressed. So you'll have ZERO savings.

2) For bunch of small files you'd better use Windows built-in off-line deduplication. StarWind dedupe is for I/O acceleration (reduced writes) and space and life savings with flash and in-memory operations.
Windows cannot dedupe live VMs (StarWind *CAN*) and StarWind is very expensive because of an in-memory RAM hash tables (still smaller then with say ZFS but still space wasted) for a big ice cold file dumps.
We're COMPLIMENTARY to Windows built-in dedupe and not competitive thing. So... File dumps, backup volumes -> use Windows. Live VMs -> use StarWind.

Re: Which dedup better?

Posted: Sat Oct 26, 2013 7:59 pm
by fbifido
So, if i set the L1 cache from 128MB to 256mb i will get a boost in dedup?

I don't no the diff dedup tech out there, but vmworld 2013 did a video on dedup tech that emc and other venders use:

can you tell me how diff is your dedup re the info in this video:
http://www.youtube.com/watch?v=vMB2fs1o ... rLa4_ULwpo

thanks

Re: Which dedup better?

Posted: Sat Oct 26, 2013 8:28 pm
by anton (staff)
Spoofing has no effect on in-line dedupe.
fbifido wrote:So, if i set the L1 cache from 128MB to 256mb i will get a boost in dedup?

I don't no the diff dedup tech out there, but vmworld 2013 did a video on dedup tech that emc and other venders use:

can you tell me how diff is your dedup re the info in this video:
http://www.youtube.com/watch?v=vMB2fs1o ... rLa4_ULwpo

thanks

Re: Which dedup better?

Posted: Sun Oct 27, 2013 2:28 am
by fbifido
VMware is now moving to use their new external storage technology
Virtual Volume
http://www.youtube.com/watch?v=651460_8 ... rLa4_ULwpo

will StarWind v8 support this on release, because most of the main hardware storage vender have already have this build in?

Re: Which dedup better?

Posted: Sun Oct 27, 2013 10:00 am
by anton (staff)
vVol is another name for "one VM = one LUN"

It's not about storage it's about management.
fbifido wrote:VMware is now moving to use their new external storage technology
Virtual Volume
http://www.youtube.com/watch?v=651460_8 ... rLa4_ULwpo

will StarWind v8 support this on release, because most of the main hardware storage vender have already have this build in?