DPM2012, CSV and Hardware VSS

[pit]

Hi!

We are using new Windows Server 2012 Failover Cluster with CSV + DPM 2012 (SP1 Beta, v4.0.3152.0) + StarWind iSCSI SAN Software v6.0.0 (Build 20120927, [SwSAN], Win64) + Starwind Hardware VSS Provider installed on all nodes (initiator). Device is IBV.

When DPM starts to make backups, we receive VSS errors 22 + 12292 (@node event log):

Log Name: Application
Source: VSS
Date: 19.12.2012 11:38:58
Event ID: 22
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: node00
Description:
Volume Shadow Copy Service error: A critical component required by the Volume Shadow Copy service is not registered. This might happened if an error occurred during Windows setup or during installation of a Shadow Copy provider. The error returned from CoCreateInstance on class with CLSID {b319214a-c160-4689-ac7c-4d0d65359e11} and Name HWPRV is [0x80040154, Class not registered
].

Operation:
Creating instance of hardware provider
Obtain a callable interface for this provider
Add a Volume to a Shadow Copy Set

Context:
Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
Class ID: {b319214a-c160-4689-ac7c-4d0d65359e11}
Snapshot Context: 4194304
Execution Context: Coordinator
-----------------------

Log Name: Application
Source: VSS
Date: 19.12.2012 11:38:58
Event ID: 12292
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: node00
Description:
Volume Shadow Copy Service error: Error creating the Shadow Copy Provider COM class with CLSID {b319214a-c160-4689-ac7c-4d0d65359e11} [0x80040154, Class not registered
].

Operation:
Creating instance of hardware provider
Obtain a callable interface for this provider
Add a Volume to a Shadow Copy Set

Context:
Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
Class ID: {b319214a-c160-4689-ac7c-4d0d65359e11}
Snapshot Context: 4194304
Execution Context: Coordinator
-------------------

[anton (staff)]

Are you able to take a snapshots and work with DPM using default Microsoft software VSS or is it a complete show stopper issue? Any chance to see StarWind logs from failing attempts?

[pit]

Hello & Happy New Year!

We didn't try to use MS Software VSS yet because we chose StarWind because of the possibility of Hardware VSS. So the question is: Does your Hardware VSS provider support CSV/Win2012/DPM2012? Do you have a white paper / how-to?

[tux-]

I seem to have the very same problem. Lack of a good backup solution is really what stops me from choosing for Starwind so far. For the rest, I really like what I have seen so far.
In a test setup, I tried Server 2012, VM on CSV and DPM 2012 SP1.

Code: Select all

Volume Shadow Copy Service error: A critical component required by the Volume Shadow Copy service is not registered. This might happened if an error occurred during Windows setup or during installation of a Shadow Copy provider. The error returned from CoCreateInstance on class with CLSID {b319214a-c160-4689-ac7c-4d0d65359e11} and Name HWPRV is [0x80040154, Class not registered
]. 

Operation:
   Creating instance of hardware provider
   Obtain a callable interface for this provider
   Add a Volume to a Shadow Copy Set

Context:
   Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
   Provider ID: {ca6fbd59-0d58-420d-962f-2778e4ec8583}
   Class ID: {b319214a-c160-4689-ac7c-4d0d65359e11}
   Snapshot Context: 4194304
   Execution Context: Coordinator

I really hope this issue get's resolved..

[tux-]

It seems that the problem originates in the installation / registration phase of the provider. I tried re-registring the VSS hw provider, with the following result. Hopefully that's of any help. If you need more information for debugging, please feel free to ask.

Code: Select all

C:\VssHWProviderStarWind>cscript register_app.vbs -register "VssHWProviderStarWi
nd" C:\VssHWProviderStarWind\hardwareprovider.dll "VSS HW StarWind Provider"
Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.


REGISTER_APP.VBS version 1.6 for Windows Server 2008
Copyright (C) Microsoft Corporation 2002-2003. All rights reserved.

Unregistering the existing application...
- Create the catalog object
- Get the Applications collection
- Populate...
- Search for VssHWProviderStarWind application...
- Application VssHWProviderStarWind removed!
- Saving changes...
Done.
Creating a new COM+ application:
- Creating the catalog object
- Get the Applications collection
- Populate...
- Add new application object
- Set app name = VssHWProviderStarWind
- Set app description = VSS HW StarWind Provider
- Set app access check = true
- Set encrypted COM communication = true
- Set secure references = true
- Set impersonation = false
- Save changes...
- Create Windows service running as Local System
- Add the DLL component

[b]ERROR:
- Error code: -2146368511 [0x80110401]
- Exit code: 113
- Description:
- Source:
- Help file:
- Help context: 0
- COM+ Errors detected: (2)
   * (COM+ ERROR 0) on C:\VssHWProviderStarWind\hardwareprovider.dll
       ErrorCode: -2146368486 [0x8011041A]
       MajorRef: C:\VssHWProviderStarWind\hardwareprovider.dll
       MinorRef: <invalid>
   * (COM+ ERROR 1) on StarWindProvider
       ErrorCode: -2146368486 [0x8011041A]
       MajorRef: {3036D363-CB9B-44B1-8657-53C5B6DEFEF5}
       MinorRef: <invalid>[/b]

C:\VssHWProviderStarWind>set EVENT_LOG=HKLM\SYSTEM\CurrentControlSet\Services\Ev
entlog\Application\VssHWProviderStarWind

C:\VssHWProviderStarWind>reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\Even
tlog\Application\VssHWProviderStarWind /f
The operation completed successfully.

C:\VssHWProviderStarWind>reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\Even
tlog\Application\VssHWProviderStarWind /f /v CustomSource /t REG_DWORD /d 1
The operation completed successfully.

C:\VssHWProviderStarWind>reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\Even
tlog\Application\VssHWProviderStarWind /f /v EventMessageFile /t REG_EXPAND_SZ /
d C:\VssHWProviderStarWind\hardwareprovider.dll
The operation completed successfully.

C:\VssHWProviderStarWind>reg.exe add HKLM\SYSTEM\CurrentControlSet\Services\Even
tlog\Application\VssHWProviderStarWind /f /v TypesSupported /t REG_DWORD /d 7
The operation completed successfully.

[pit]

Hi Tux!

okay, thanx ... and the result? Does DPM 2012 work now without troubles?

regards,
Lukas

[Anatoly (staff)]

tux, can we ask you to get us update on this?

[tux-]

Unfortunately, no result as you can see in the logfile in my post. The COM object cannot be registered. I sent a PM to Anatoly, in case Starwind support want to follow-up on this issue.
Fingers crossed they will solve this

[Anatoly (staff)]

OK, we have updated the VSS provider. Would you be so kind to download it and install, and see if it will fix your problem?

Thanks

[tux-]

Great work guys! It's working now.
There are two minor issues I encountered:

1. It seems that the auto-recovery phase of the snapshot still fails. It's only logged when backup up a Windows VM. The following is logged in the event log:

Code: Select all

The following information was included with the event: 

\\?\Volume{51e2aeb3-5b0e-11e2-b1ce-d48564528c47}\
{36232844-18cd-43a1-ae47-dd7e2ca8f8a0}


Operation:
   Revert a Shadow Copy

Context:
   Execution Context: Coordinator

The handle is invalid

2. It seems that snapshots are not deleted automatically. This will result in a snapshot clutter on the SAN, and possibly performance impact.

Thanks,
Remco.

[pit]

Where can I download the newest release of VSS provider? I found only v4.5 (20121008) ...

[Anatoly (staff)]

Where can I download the newest release of VSS provider? I found only v4.5 (20121008) ...

You can download it from the Customers page on our website. Here is the link:
http://www.starwindsoftware.com/customer-page

[pit]

OK, we use DPM 2012 SP1 RTM + new VSS provider 4.5. New error on node:

Log Name: Application
Source: VSS
Date: 15.01.2013 09:30:04
Event ID: 12362
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: node00
Description:
A Shadow Copy LUN was not detected in the system and did not arrive.

LUN ID {f9474270-457d-4b26-b062-9b1b10268e55}

Version 0x0000000000000001
Device Type 0x0000000000000000
Device TypeModifier 0x0000000000000000
Command Queueing 0x0000000000000001
Bus Type 0x0000000000000009
Vendor Id ROCKET
Product Id IBVolume
Product Revision 0001
Serial Number 6F104C962B93BBB

Storage Identifiers
Version 16
Identifier Count 1

Identifier 0
CodeSet "VDSStorageIdCodeSetBinary" (1)
Type "VDSStorageIdTypeVendorSpecific" (0)
Byte Count 24

03 00 00 00 39 46 38 32 46 30 30 36 46 31 43 38 ....9F82F006F1C8
39 35 36 00 00 00 00 00 ## ## ## ## ## ## ## ## 956.....########

Error on DPM:

Log Name: DPM Alerts
Source: DPM-EM
Date: 15.01.2013 09:53:47
Event ID: 3106
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: dpm
Description:

The replica of Backup Using Child Partition Snapshot\dc00 on dc00.xx.local.cluster00.xx.local is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. (ID: 3106)
The VSS application writer or the VSS provider is in a bad state. Either it was already in a bad state or it entered a bad state during the current operation. (ID: 30111)

Do you also want the Starwind server log? (support)

[Anatoly (staff)]

There are two minor issues I encountered:

1. It seems that the auto-recovery phase of the snapshot still fails. It's only logged when backup up a Windows VM. The following is logged in the event log:

CODE: SELECT ALL
The following information was included with the event:

\\?\Volume{51e2aeb3-5b0e-11e2-b1ce-d48564528c47}\
{36232844-18cd-43a1-ae47-dd7e2ca8f8a0}


Operation:
Revert a Shadow Copy

Context:
Execution Context: Coordinator

The handle is invalid


2. It seems that snapshots are not deleted automatically. This will result in a snapshot clutter on the SAN, and possibly performance impact.

tux, can I ask you what commands have you used? Is it possible to share the entire log with us?

[Anatoly (staff)]

The replica of Backup Using Child Partition Snapshot\dc00 on dc00.xx.local.cluster00.xx.local is inconsistent with the protected data source. All protection activities for data source will fail until the replica is synchronized with consistency check. (ID: 3106)
The VSS application writer or the VSS provider is in a bad state. Either it was already in a bad state or it entered a bad state during the current operation. (ID: 30111)

Do you also want the Starwind server log? (support)

pit, can I ask you to rovide detailed description of configuration and sequence of actions that were performed? We need this to re-create your issue in our test-lab.

Thank you