Which dedup better?

Public beta (bugs, reports, suggestions, features and requests)

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Post Reply
User avatar
fbifido
Posts: 125
Joined: Thu Sep 05, 2013 7:33 am

Thu Oct 24, 2013 7:37 pm

Hi,

Just like to ask a question about StarWind iSCSI dedup.

Which of the 2 below dedup better, as in give the max amount of storage savings

1) one very large file, say 20GB, be it a doc, zip, iso, pdf or mp3
2) a bunch of small files, say about 20 - 1GB file, be it the same type as above

note-1) knowing that this is a technical question, let me say for a large file, I created a ISO file with all of windows os.
windows 8.1 core,pro,ent
windows 8.0 all
windows 7 all
windows server 2008 R2 all
windows server 2012 all
windows server 2012 R2 all
hyper-v 2012 r2

There are free software that allows you to do this. Now all those OS are now in one big iso file, with about 18 are so single os option to select and install.

note-2) using the info from above, say we extract each of those 18 plus option to each a file on its own, we will have a directory it say 18 or more iso files.


note-1a) the same can be done using pdf. I did it once using my outlook, I had about 200 or so emails that I needed to send to someone I a did not want to print them and to pdf each email was out of the question, and most of them had attachments(one or more), so I use acrobat pro 10(at that time) to create a book-like-pdf with all 200 emails, it seems to keep all the email structure including time,date, etc....
but it was around 3GB, if I had use max compress maybe smaller.
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Fri Oct 25, 2013 10:16 am

1) Everything you list suck except ISO :)

DOCs are zipped XMLs, ZIP, PDF and MP3 are all compressed. So you'll have ZERO savings.

2) For bunch of small files you'd better use Windows built-in off-line deduplication. StarWind dedupe is for I/O acceleration (reduced writes) and space and life savings with flash and in-memory operations.
Windows cannot dedupe live VMs (StarWind *CAN*) and StarWind is very expensive because of an in-memory RAM hash tables (still smaller then with say ZFS but still space wasted) for a big ice cold file dumps.
We're COMPLIMENTARY to Windows built-in dedupe and not competitive thing. So... File dumps, backup volumes -> use Windows. Live VMs -> use StarWind.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
fbifido
Posts: 125
Joined: Thu Sep 05, 2013 7:33 am

Sat Oct 26, 2013 7:59 pm

So, if i set the L1 cache from 128MB to 256mb i will get a boost in dedup?

I don't no the diff dedup tech out there, but vmworld 2013 did a video on dedup tech that emc and other venders use:

can you tell me how diff is your dedup re the info in this video:
http://www.youtube.com/watch?v=vMB2fs1o ... rLa4_ULwpo

thanks
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Sat Oct 26, 2013 8:28 pm

Spoofing has no effect on in-line dedupe.
fbifido wrote:So, if i set the L1 cache from 128MB to 256mb i will get a boost in dedup?

I don't no the diff dedup tech out there, but vmworld 2013 did a video on dedup tech that emc and other venders use:

can you tell me how diff is your dedup re the info in this video:
http://www.youtube.com/watch?v=vMB2fs1o ... rLa4_ULwpo

thanks
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
fbifido
Posts: 125
Joined: Thu Sep 05, 2013 7:33 am

Sun Oct 27, 2013 2:28 am

VMware is now moving to use their new external storage technology
Virtual Volume
http://www.youtube.com/watch?v=651460_8 ... rLa4_ULwpo

will StarWind v8 support this on release, because most of the main hardware storage vender have already have this build in?
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Sun Oct 27, 2013 10:00 am

vVol is another name for "one VM = one LUN"

It's not about storage it's about management.
fbifido wrote:VMware is now moving to use their new external storage technology
Virtual Volume
http://www.youtube.com/watch?v=651460_8 ... rLa4_ULwpo

will StarWind v8 support this on release, because most of the main hardware storage vender have already have this build in?
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Post Reply