StarWind iSCSI SAN
StarWind Native SAN for Hyper-V
 

KeepInCloud, KeepInStorage

StarWind VTL, VTL Free, VTL Appliance

Moderators: art (staff), anton (staff), Anatoly (staff), Max (staff)

KeepInCloud, KeepInStorage

Postby linchesterbr » Fri Dec 21, 2018 7:30 pm

Hello Users.

Someone can explain what means the following parameters and possible values for the VTLReplicationSettings.ps1 (PowerShell Script)?:
$settings.KeepInCloud=
$settings.KeepInStorage1=
$settings.KeepInStorage2=

Another thing. Should I create a Glacier Vault to move the files from S3 or do API this automatically? Beacause documentation says just about S3 bucket creation, but nothing about glacier.

Thanks in advance.
linchesterbr
 
Posts: 1
Joined: Fri Dec 21, 2018 7:22 pm

Re: KeepInCloud, KeepInStorage

Postby Oleg(staff) » Thu Jan 03, 2019 2:36 pm

Hello,
Someone can explain what means the following parameters and possible values for the VTLReplicationSettings.ps1 (PowerShell Script)?:
$settings.KeepInCloud=
$settings.KeepInStorage1=
$settings.KeepInStorage2=


$settings.KeepInCloud - The time for saving in the cloud, days. The data will be removed from the cloud after this time expiration (It is working for all the types).

$settings.KeepInStorage1=
$settings.KeepInStorage2=
The time for migration to different storage type.
For S3 only KeepInStorage1 parameter is working, it is the time of keeping on S3, after this time expiration the data is moving to Glacier
For Azure KeepInStorage1 that is the time of keeping data on Hot till the moment of moving to Cold, KeepInStorage2that is the time of keeping data on Cold till moving to Archive.

Another thing. Should I create a Glacier Vault to move the files from S3 or do API this automatically? Beacause documentation says just about S3 bucket creation, but nothing about glacier.

In the case of S3 you are making/changing the rule for the bucket, then Amazon is making everything by itself. The rule will work to all files in the bucket.
Oleg(staff)
Staff
 
Posts: 567
Joined: Fri Nov 24, 2017 7:52 am

Re: KeepInCloud, KeepInStorage

Postby willn333 » Tue Aug 06, 2019 3:13 am

Is there a setting in powershell for the option "Create new empty tapes automatically when existing tape removed from VTL for replication"?
willn333
 
Posts: 8
Joined: Tue Aug 06, 2019 2:43 am

Re: KeepInCloud, KeepInStorage

Postby Boris (staff) » Tue Aug 06, 2019 2:42 pm

Check this option on the example of Veeam B&R:
https://www.starwindsoftware.com/resour ... e418c70ca9
You need to the screenshot at step 9 of the Veeam block. Veeam has got that setting, which seems to be exactly what you are looking for.
Boris (staff)
Staff
 
Posts: 805
Joined: Fri Jul 28, 2017 8:18 am

Re: KeepInCloud, KeepInStorage

Postby willn333 » Wed Aug 07, 2019 2:06 am

Boris (staff) wrote:Check this option on the example of Veeam B&R:
https://www.starwindsoftware.com/resour ... e418c70ca9
You need to the screenshot at step 9 of the Veeam block. Veeam has got that setting, which seems to be exactly what you are looking for.


I don't think this is the same thing. Veeam wouldn't be able to just create a new tape in the VTL. That is done with the powershell commands:

Code: Select all
$tapeType = [StarWindTapeType]::STARWIND_TAPE_ULTRIUM_LTO8            
   $tape = Add-VirtualTape -VTLDevice $device -tapeType $tapeType -tapeSize $tapeSize -maxDataFileSize $maxDataFileSize


What I am trying to do is have a workflow of... backing up files to tape and eject them so they go to the offline shelf and get uploaded to the cloud. After a retention period of 30 days, reinsert the offline tapes into slots so that I can reuse them and remove the data from the cloud.

Is that possible?
willn333
 
Posts: 8
Joined: Tue Aug 06, 2019 2:43 am

Re: KeepInCloud, KeepInStorage

Postby Boris (staff) » Wed Aug 07, 2019 7:24 pm

The Veeam feature I outlined does work for numerous customers that have it configured in their environments, and I do not have any grounds for suspecting this functionality to be faulty (unless this is specific to your particular environment). Have you tried configuring this before stating that "Veeam wouldn't be able to just create a new tape in the VTL"? If so, something is definitely wrong there, as it has been working flawlessly for hundreds of others.
With Veeam, you also have an option of ejecting the tape after the backup is completed, so you can later query the VTL device to get the list of offline shelf tapes as we discussed in viewtopic.php?f=5&t=5356&p=29954#p29954 I believe other backup software may have that functionality as well.
Boris (staff)
Staff
 
Posts: 805
Joined: Fri Jul 28, 2017 8:18 am

Re: KeepInCloud, KeepInStorage

Postby willn333 » Thu Aug 08, 2019 6:44 pm

Boris (staff) wrote:The Veeam feature I outlined does work for numerous customers that have it configured in their environments, and I do not have any grounds for suspecting this functionality to be faulty (unless this is specific to your particular environment). Have you tried configuring this before stating that "Veeam wouldn't be able to just create a new tape in the VTL"? If so, something is definitely wrong there, as it has been working flawlessly for hundreds of others.
With Veeam, you also have an option of ejecting the tape after the backup is completed, so you can later query the VTL device to get the list of offline shelf tapes as we discussed in viewtopic.php?f=5&t=5356&p=29954#p29954 I believe other backup software may have that functionality as well.


Not sure if we are on the same page here. Bascially, I ran the Veeam job with no tapes in any slots but the ones that were in the offline shelf and I get a message from the Veeam job of "[Warning] Insert a tape to continue".

Veeam doesn't have the ability to create tapes in the VTL, nor does it have the ability to load a tape from the offline shelf. They are completely separate systems.

Any other ideas?
willn333
 
Posts: 8
Joined: Tue Aug 06, 2019 2:43 am

Re: KeepInCloud, KeepInStorage

Postby Boris (staff) » Mon Aug 12, 2019 7:54 am

We are on the same page, but in its different parts, if I may say so...
First of all, you will need a tape available for Veeam to write backups to. Once this backup is successful, the tape gets ejected (if the corresponding option is enabled in the backup job config) and here VTL tape creation kicks in (again - if it is configured to do so). If you do not have any tapes online, this will never work, no matter how long you would wait for this.
This does work for everyone using StarWind VTL and Veeam. I am sure this will do so for you, too, once you configure everything properly. Just post back any questions you have after you follow the guide on configuring Veeam B&R to be used with VTL that I posted last week.
Boris (staff)
Staff
 
Posts: 805
Joined: Fri Jul 28, 2017 8:18 am

Re: KeepInCloud, KeepInStorage

Postby willn333 » Tue Aug 13, 2019 12:26 am

Boris (staff) wrote:We are on the same page, but in its different parts, if I may say so...
First of all, you will need a tape available for Veeam to write backups to. Once this backup is successful, the tape gets ejected (if the corresponding option is enabled in the backup job config) and here VTL tape creation kicks in (again - if it is configured to do so). If you do not have any tapes online, this will never work, no matter how long you would wait for this.
This does work for everyone using StarWind VTL and Veeam. I am sure this will do so for you, too, once you configure everything properly. Just post back any questions you have after you follow the guide on configuring Veeam B&R to be used with VTL that I posted last week.


The reason this works is the setting in cloud replication on the StarWind side that I asked about in my previous post.

This setting to create a new tape when the existing one is ejected. You never answered if that setting is available via powershell. In looking at the sample scripts, it is not part of the settings properties and when I do a listing of all the settings for the CloudReplication Settings, it is only these:

Script:
Code: Select all
Import-Module StarWindX

$server = New-SWServer -host 127.0.0.1 -port 3261 -user root -password starwind

try
{
    $server.Connect()

   #
   # common parameters
   #
   $deviceName = "vtl1"
   $device = $null
      
   foreach($dev in $server.Devices)
   {
      
      if( $dev.Name.Equals($deviceName) -and $dev.DeviceType.Equals("VTL") )
      {
         $device = $dev
         break
      }
   }

   if( !$device )
   {
      Write-Host "Device '$($deviceName)' not found" -foreground red
      return
   }

   $device

   $settings = new-object -ComObject StarWindX.VTLReplicationSettings
   $settings   
}
catch
{
    Write-Host $_ -foreground red
}
finally
{
   $server.Disconnect()
}


Output:
Code: Select all
Name                   : vtl1
DeviceType             : VTL
DeviceId               : 0x0000004D8A100440
File                   :
TargetName             : iqn.2008-08.com.starwindsoftware:veeambackup-targetvtl1
TargetId               : 0x0000004D8A4DFA80
Size                   : empty
CacheMode              : empty
CacheSize              : empty
CacheBlockExpiryPeriod : empty
Exists                 : True
DeviceLUN              : 0
IsSnapshotsSupported   : False
Snapshots              :
SectorSize             :
State                  : 0
Tapes                  : System.__ComObject
AvailableSlots         : 96
TransportSlots         : 1
DriveSlots             : 4
ImportExportSlots      : 3
StorageSlots           : 96
Slots                  : System.__ComObject
ReplicationSettings    : System.__ComObject
DriveType              : 8

Target           : 0
AccessKey        :
SecretAccessKey  :
RegionName       :
ContainerName    :
KeepLocal        : -1
KeepInCloud      : -1
KeepInStorage1   : -1
KeepInStorage2   : -1
DelayBeforeStart : -1
ServiceUrl       :


As you can see, there is no setting in powershell for the "Create new empty tapes automatically when existing tape removed from VTL for replication."

Image
willn333
 
Posts: 8
Joined: Tue Aug 06, 2019 2:43 am

Re: KeepInCloud, KeepInStorage

Postby Michael (staff) » Fri Aug 16, 2019 4:49 pm

Hello willn333,
The option "Create new empty tapes automatically when existing tape removed from VTL for replication." can be changed in VTL device configuration file. For example in my case it's located here: "E:\VTL\VTL.swdsk"
Just stop StarWind Virtual SAN and CloudReplicator services, open VTL.swdsk, locate the line "<createTapeOnExport>false</createTapeOnExport>", change "false" to "true", save file and start services back.
Let us know about the result.
Michael (staff)
Staff
 
Posts: 310
Joined: Thu Jul 21, 2016 10:16 am

Re: KeepInCloud, KeepInStorage

Postby dgbegfb » Tue Mar 31, 2020 11:31 am

Michael (staff) wrote:Hello willn333,
The option "Create new empty tapes automatically when existing tape removed from VTL for replication." can be changed in VTL device configuration file. For example in my case it's located here: "E:\VTL\VTL.swdsk"
Just stop StarWind Virtual SAN and CloudReplicator services, open VTL.swdsk, locate the line "<createTapeOnExport>false</createTapeOnExport>", change "false" to "true", save file and start services back.
Let us know about the result.


Hi Michael,
Thank you for the workaround, it really works as expected but there're two issues:
- each new tape creates with default maximum tape size and default part size.
- each time after VTLReplicationSettings.ps1 script execution the line "<createTapeOnExport>true</createTapeOnExport>" reverts to "false" state. It gives me a clue, may it can be set trough powershell script?

Is there any way to adjust such behaviour?
dgbegfb
 
Posts: 8
Joined: Tue Mar 31, 2020 11:14 am

Re: KeepInCloud, KeepInStorage

Postby yaroslav (staff) » Tue Apr 07, 2020 8:06 am

Sorry for such a delayed response.
Happy to know it finally works!
- each new tape creates with default maximum tape size and default part size.

That you can tune in the AddVirtualTape.ps1 script. See the lines to edit in the image below.
"<createTapeOnExport>true</createTapeOnExport>" reverts to "false" state

Well, these parameters cannot be modified in StarWindX yet.
I have created a feature request. Thanks for your effort to improve StarWind VTL!
Attachments
image (10).png
image (10).png (5.67 KiB) Viewed 2624 times
yaroslav (staff)
Staff
 
Posts: 481
Joined: Mon Nov 18, 2019 11:11 am

Re: KeepInCloud, KeepInStorage

Postby dgbegfb » Tue Apr 07, 2020 8:47 am

Super, thanks. We're all in profit from such collaboration.

But such modification of AddVirtualTape script doesn't affect an automatic tape creation after the export.
dgbegfb
 
Posts: 8
Joined: Tue Mar 31, 2020 11:14 am

Re: KeepInCloud, KeepInStorage

Postby yaroslav (staff) » Tue Apr 07, 2020 9:38 am

But such modification of AddVirtualTape script doesn't affect an automatic tape creation after the export.

Sorry, got your question wrong. R&D guys are planning to modify the script so that tape that is created has identical properties as the offloaded one.
One more feature request I guess.

Thanks.
yaroslav (staff)
Staff
 
Posts: 481
Joined: Mon Nov 18, 2019 11:11 am

Re: KeepInCloud, KeepInStorage

Postby vondess » Sat May 16, 2020 4:38 pm

Thanks for all your responses. I have also found a response to my question here. I am from [ ... ], and I really like such forums. I have found a lot of technical information in similar threads. It is a good way to help each other. Sometimes an average user can share a lot of helpful tips. And it is great.
Last edited by vondess on Sun May 17, 2020 2:39 pm, edited 1 time in total.
vondess
 
Posts: 2
Joined: Fri May 15, 2020 8:43 pm

Next

Return to StarWind Virtual Tape Library (VTL) & StarWind VTL Free, StarWind VTL Appliance

Who is online

Users browsing this forum: No registered users and 2 guests