Drive Pooling, SMB and iSCSI?

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

Locked
ironhead65
Posts: 1
Joined: Fri Apr 05, 2013 5:46 pm

Fri Apr 05, 2013 6:21 pm

So I have kind of a unique situation. I think that if I had more time to dedicate to this I might learn the answer myself through trial and error.
After speaking to Richard Pierro, he mentioned that I might want to post here in the forums to get a better understanding more quickly.

Essentially, I have a server that is running drive pooling software. As such I end up with a drive that is made up of multiple drives. This pool contains all the video files that are served throughout the network as well as user's personal files. This pool is residing on a lower power machine, which has hardware TCP/IP offloading NICS, as well as other hardware accelerators (by design) so the unit can be as low power as possible, yet still be able to serve the network without problem.

This machine has 1 NIC and due to the file duplication capabilities, as well as the ability to grow/replace drives without rebuilding the entire network, it has become the main place that all information is kept. It is Gigabit. It's job is to serve out files using SMB (for the Windows machines), as well as the video files, etc.

The second machine is a high power Xeon Server, which runs ESXi. it has under/overclock ability and can be very low in power. One of the VMs is the machine that does the transcoding for the network. The way the transcoding software is designed, it can be aware of all local files as well as "network" files over SMB. The problem I have is that all files that are referenced over the SMB shares are re-assesed each time the Transcoding software is restarted. That is OK for small pools. The video pool is well over 6TB. That means each time I restart the server, it could be hours before the video transcoder is ready.

My goal was to run iSCSI or NFS to trick the machine into thinking the video files were local. My problems with NFS is that I am not sure that a NFS hard-link in Windows is going to trick the software into thinking the files are local. I KNOW that iSCSI will be represented as local.

Questions:
*If I were to run the iSCSI target on the storage pool machine, do you think there will be performance issues? It is an Atom dual core 2.1Ghz machine with only 2GB of ram. I wanted more, but the motherboard that fit the case we are using is limiting us. I keep flipping OSes, but it is either Windows 2008 Storage Server or WHS 2012. Right now the install is WHS 2012. The storage pool software is from StableBit.

*I know that iSCSI and NFS are not the same thing. NFS is kind of like SMB, except that in Linux the files actually "do" show up as local. I am not sure of that in Windows, except that when you perform the mount (after installing the NFS client), you can choose a hard or soft link.

iSCSI on the other hand gives "block" level access to the connected drive space. So, if I wanted to use iSCSI, I would have to provide either a virtual hard drive file located on the storage pool or dedicate the pool to the iSCSI target. It is not desired to local all these video files into a Virtual Hard drive, especially since it will grow so large there might end up being issues. I would not be able to share the pool with SMB shares. Or can it? The desire is for both SMB and iSCSI to "point" to the same drive, not the same folders ON the drive. Similar to how NFS appears to work. I would NOT be able to share the same folders as both NFS and SMB.

From the pool's perspective it might be drive D:
*Can I put folders on that pool and have 1 folder share out through iSCSI to the transcoder, and the other folders act as SMB shares?
*logging into the server, opening file explorer I would go to D and see the movies folder and the user data folder.
*If I did dedicate the pool to iSCSI, would it goof up the storage pool, or would it goof up the iSCSI connectors/target when I used the pool's ability to eject a bad drive, or add in a new drive (i.e. a 8TB drive might drop down to 6TB, or 8TB might grow to 10TB+)?

My other option is the install yet enough VM and run the transcoder under Linux - hoping that NFS would behave as "local" to limit reboot time to minutes, not hours. Not preferred as we are forced to have at least 1 windows based server for an application that runs windows only, and I would like to NOT have yet another VM running.

Thank you for your time!

--Dan
User avatar
anton (staff)
Site Admin
Posts: 4008
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Sat Apr 06, 2013 8:04 pm

You can host StarWind image files on your subfolder (except we never worked with StableBit and I'm not sure we'll support this in the production) so the whole disk would be SMB (or NFS) accessed and some subset of files will be actually used to feed storage as iSCSI block devices.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
jeddyatcc
Posts: 49
Joined: Wed Apr 25, 2012 11:52 pm

Mon Apr 08, 2013 12:47 pm

iSCSI is a block device, you cannot "share" it. Even if you used a RAW disk, this disk can't be used by StarWind and SMB at the same time(there are ways, but performance would suffer). I would recommend testing NFS under windows as I do think it works exactly the way you want. My question always comes back to support though, Why does the software check all of the files on the Share?, Is there a way to stop that from happening?, Does the software support NFS on Windows?
User avatar
Anatoly (staff)
Staff
Posts: 1675
Joined: Tue Mar 01, 2011 8:28 am
Contact:

Wed Apr 10, 2013 10:14 am

Why does the software check all of the files on the Share? Is there a way to stop that from happening?
Would you please clarify that?
Does the software support NFS on Windows?
Users do can create the NFS that will store the data on StarWind devices, but it is impossible to store StarWind files on NFS.
Best regards,
Anatoly Vilchinsky
Global Engineering and Support Manager
www.starwind.com
av@starwind.com
Locked