Massive data corruption after SW reboot

Software-based VM-centric and flash-friendly VM storage + free version

Moderators: anton (staff), art (staff), Max (staff), Anatoly (staff)

craggy
Posts: 55
Joined: Tue Oct 30, 2012 3:33 pm

Mon Sep 14, 2015 11:32 am

Hi

Yes we have been told that since and are planning to disable the WB cache once we can safely flush the current cache data to disk and stop the service without corruption occurring.
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Mon Sep 14, 2015 11:35 am

R&D should give you API entry calls and PowerShell script sample on how to do that.
craggy wrote:Hi

Yes we have been told that since and are planning to disable the WB cache once we can safely flush the current cache data to disk and stop the service without corruption occurring.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
User avatar
anton (staff)
Site Admin
Posts: 4010
Joined: Fri Jun 18, 2004 12:03 am
Location: British Virgin Islands
Contact:

Mon Sep 14, 2015 11:35 am

R&D should give you API entry calls and PowerShell script sample on how to do that.
craggy wrote:Hi

Yes we have been told that since and are planning to disable the WB cache once we can safely flush the current cache data to disk and stop the service without corruption occurring.
Regards,
Anton Kolomyeytsev

Chief Technology Officer & Chief Architect, StarWind Software

Image
Vladislav (Staff)
Staff
Posts: 180
Joined: Fri Feb 27, 2015 4:31 pm

Mon Sep 14, 2015 12:19 pm

We have finished the script.

In order to manually flush cache of your StarWind device, you should do the following:

- Download FlushCache.ps1 file attached.
- Open file with Wordpad / Notepad ++.
- Change $deviceName="imagefile1" value to correspond the name of your StarWind device.
- Save file.
- Disconnect all clients from StarWind device. The best way to perform this is to disconnect all targets in ISCSI Initiator.
- Open starwindX Powershell.
- Run FlushCache.ps1

If the script was successfully executed you will receive "Start flushing... Completed!" output in Powershell console. At this moment, the flushing process has begun.

The total capacity of your WB cache is L1 8GB + L2 60GB = 68GB which means your underlying physical storage will now perform the 68GB write operation.
If you are aware of your physical storage performance, you can calculate the approximate tame it takes to perform such operation.
For example with the transfer rate of 100 MB/sec it will take approximately 11 minutes to write 68GB of data.
Therefore, we recommend you to wait at least 15-20 minutes until flushing is finished.

After that, you can change your L2 cache mode to wright-through and restart StarWind service.

Please, contact us directly by e-mail: support@starwind.com in case if you face any difficulties.
Attachments
FlushCache.rar
(1.06 KiB) Downloaded 337 times
craggy
Posts: 55
Joined: Tue Oct 30, 2012 3:33 pm

Tue Sep 15, 2015 11:12 am

Cheers guys, going to test this on our lab environment before we schedule any production downtime.
User avatar
darklight
Posts: 185
Joined: Tue Jun 02, 2015 2:04 pm

Wed Sep 16, 2015 8:57 am

Wow, interesting script :) Will definitely take a look :)
craggy
Posts: 55
Joined: Tue Oct 30, 2012 3:33 pm

Mon Sep 21, 2015 2:06 pm

Hi

We've tried to run this script in a test environment and keep getting the following error:

Start flushing...Exception The variable '$raw' cannot be retrieved because it has not been set.

Any ideas?
craggy
Posts: 55
Joined: Tue Oct 30, 2012 3:33 pm

Mon Sep 21, 2015 3:11 pm

Update:

So I created a new test flat disk with L2 WB cache and ran the powershell script. It finished successfully.
However, when I try run the script on any of the existing image files I get the error above.
Vladislav (Staff)
Staff
Posts: 180
Joined: Fri Feb 27, 2015 4:31 pm

Wed Sep 23, 2015 9:27 am

Community update:

We have requested more information from craggy regarding existing device settings.
Vladislav (Staff)
Staff
Posts: 180
Joined: Fri Feb 27, 2015 4:31 pm

Tue Sep 29, 2015 12:11 pm

Quick community update:

Script fails to run on the particular craggy's device. We are now investigating why.
schraepf
Posts: 2
Joined: Wed May 27, 2015 12:14 pm

Wed Oct 21, 2015 4:47 pm

Any update on this issue? Interested to learn about @craggy's system specs.
Vladislav (Staff)
Staff
Posts: 180
Joined: Fri Feb 27, 2015 4:31 pm

Thu Oct 22, 2015 5:15 pm

Hello.

Just a quick update.

We have changed the initial FlushCache script and it was successfully executed in our test environment.

However, we have faced an issue on Craggy's setup, when performance monitor shows near zero activity during script execution as well as during regular file level read-write operations.

We are going to arrange a remote session to check Craggy's configuration once again and possibly locate the issue.
Post Reply