Page 1 of 1

L2 Cache and Dedup Questions

Posted: Sat Feb 01, 2014 4:10 pm
by Rusty
Hello,

I understand that V8 has Dedup support, but that in the previous version(s), Dedup did not work in HA mode. I wanted to confirm that this will work in v8. I also wanted to get an idea of whether or not Dedup is recommended for a VM/cloud environment. In talking to your engineers, they did not recommend Dedup for volumes larger than 6TB of actual data (this means a lot more data pre-Dedup).

So, here are my questions about Dedup:
  • Will it work in HA mode?
  • What limitations does it have -- i.e. is it dangerous to use it with data sets larger than a certain size?
  • Any caveats or reasons not to recommend it?
  • Your engineers stated that at some point, Starwind might support using SSD for Dedup metadata to speed things up, even if LUNs are not located on SSD. Is this possible?
I also have some questions about L2 caching:
  • Can you assign one SSD per volume as an L2 cache?
  • Can the L2 cache be a RAID array of SSDs, or does it for some reason require a single SSD?
  • Any size limitation on L2 cache?
Thanks,
Russ

Re: L2 Cache and Dedup Questions

Posted: Sun Feb 02, 2014 12:54 pm
by anton (staff)
1) Yes, deduplication is now part of our Log-Structured File System (LSFS) and works in HA config.
2) There are no known limitations of a working set however LSFS is still "experimental" feature with V8 so you may think twice before putting it into hardcore production.
3) See 2) and use backups.
4) We'll make this scenario (separate hash database on flash and primary storage on spindle) available @ some point but in general it has very little to zero sense... Unless you in-line dedupe expensive and fast flash and memory saving big $$$ doing in-line dedupe with slow and inexpensive SATA is pointless. Better use it AS IS (LSFS but no deduplication) and use flash as a flash cache. More IOPS.

1) You assign cache set per LUN and not per volume. We don't control file system above us (ReFS, NTFS, VMFS etc)
2) Yes, if you'll create a software or hardware RAID over flash you have we would be able to place our container file on top of it to use as a cache.
3) No, flash cache is not limited with anything (except physical availability of corresponding memory type, RAM cache is the same here).

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 3:59 am
by Rusty
Ok, thanks for the information. A couple of follow-up questions about failures:
  1. If a LUN fails due to underlying disk failure, will that node mark the LUN as bad and disable it (but allow HA nodes to continue to serve it)?
  2. If the L2 cache fails (either an SSD or a set of SSDs in a RAID -- let's say RAID0) -- will Starwind, or can Starwind mark that LUN as bad and allow other HA nodes to continue to serve it?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 12:51 pm
by anton (staff)
1) If LUN fails all I/O would be re-routed by client MPIO stack to HA partners.

2) If L2 cache would fail (we use strong checksums to make sure data is valid) then we'll simply disable it (reporting to admin of course). If L1 cache fails whole node with all LUNs would be "out" because obvious reasons (you cannot trust to main memory anymore).
Rusty wrote:Ok, thanks for the information. A couple of follow-up questions about failures:
  1. If a LUN fails due to underlying disk failure, will that node mark the LUN as bad and disable it (but allow HA nodes to continue to serve it)?
  2. If the L2 cache fails (either an SSD or a set of SSDs in a RAID -- let's say RAID0) -- will Starwind, or can Starwind mark that LUN as bad and allow other HA nodes to continue to serve it?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 5:08 pm
by Rusty
Great, thanks.

Is there any way to force failure if the L2 cache dies? I.e. we setup a RAID0 array of 4 SSDs as L2 cache -- if this goes down, the performance of any LUNs using it will probably drop dramatically (and we'd prefer to fail over completely to working nodes)?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 6:50 pm
by anton (staff)
That's not implemented currently but would be trivial to be added.
Rusty wrote:Great, thanks.

Is there any way to force failure if the L2 cache dies? I.e. we setup a RAID0 array of 4 SSDs as L2 cache -- if this goes down, the performance of any LUNs using it will probably drop dramatically (and we'd prefer to fail over completely to working nodes)?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 7:55 pm
by Rusty
That's not implemented currently but would be trivial to be added.
Can I take that as the feature request having been officially added? :)

Could this maybe be part of the v8 stable release?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 8:00 pm
by Rusty
anton (staff) wrote: 2) There are no known limitations of a working set however LSFS is still "experimental" feature with V8 so you may think twice before putting it into hardcore production.
3) See 2) and use backups.

I understand that LSFS (and therefore dedup) is considered experimental. A few more questions came up in our discussions:
  1. Is L1 and L2 caching considered fully production ready?
  2. Can the L1 and L2 caches be expanded dynamically (i.e. we add more SSDs to a RAID10 volume used as a cache, then we expand the volume after adding drives, then expand L2 cache)? Or, we shut server, add more RAM, and increase L1 cache size?

Re: L2 Cache and Dedup Questions

Posted: Mon Feb 03, 2014 8:02 pm
by anton (staff)
Yes, because with a first public release we want to get waaaay more feedback then we currently have.

Cache is unified (architecture) L1 and L2, tested for years so is fully production-ready.

Yes but not with an initial V8 release probably (we don't know for sure yet).
Rusty wrote:
anton (staff) wrote: 2) There are no known limitations of a working set however LSFS is still "experimental" feature with V8 so you may think twice before putting it into hardcore production.
3) See 2) and use backups.

I understand that LSFS (and therefore dedup) is considered experimental. A few more questions came up in our discussions:
  1. Is L1 and L2 caching considered fully production ready?
  2. Can the L1 and L2 caches be expanded dynamically (i.e. we add more SSDs to a RAID10 volume used as a cache, then we expand the volume after adding drives, then expand L2 cache)? Or, we shut server, add more RAM, and increase L1 cache size?

Re: L2 Cache and Dedup Questions

Posted: Wed Feb 05, 2014 2:27 am
by Rusty
Thanks Anton,

Do you know if the ability to mark a LUN as "failed" and therefore disable it if L2 cache fails (i.e. failed SSD or RAID0) will be included in the production release of V8? This may influence some of our hardware decisions. I think that this feature would be extremely useful to us and save us money on SSDs because we could do RAID0.

Thanks!

Re: L2 Cache and Dedup Questions

Posted: Wed Feb 05, 2014 11:59 am
by anton (staff)
Was not on the schedule for V8 release but we'll do it. I'll kick Alex :)
Rusty wrote:Thanks Anton,

Do you know if the ability to mark a LUN as "failed" and therefore disable it if L2 cache fails (i.e. failed SSD or RAID0) will be included in the production release of V8? This may influence some of our hardware decisions. I think that this feature would be extremely useful to us and save us money on SSDs because we could do RAID0.

Thanks!

Re: L2 Cache and Dedup Questions

Posted: Mon Aug 04, 2014 1:57 pm
by robnicholson
I understand that LSFS (and therefore dedup) is considered experimental. A few more questions came up in our discussions:
I assume this isn't the case any more and the above comment was on a beta?

Cheers, Rob.

Re: L2 Cache and Dedup Questions

Posted: Tue Aug 05, 2014 11:28 am
by anton (staff)
V8 is released so this functionality is not in beta anymore. We do find and fix "flaw-by-design" issues but that will take a time (think about ZFS turning mature only after ±28 release).
robnicholson wrote:
I understand that LSFS (and therefore dedup) is considered experimental. A few more questions came up in our discussions:
I assume this isn't the case any more and the above comment was on a beta?

Cheers, Rob.