Restoring Tapes from Azure Blob Archive tier storage

dgbegfb · Tue Mar 31, 2020 12:07 pm

Hi guys,

I started using VTL 8.0 13481 free license with Veeam and Azure blob storage. I've managed to upload some tapes to Azure archive tier storage directly and restore from there. Doing that I found out that StarWind VTL during rehydration azure archive tier changes blobs tier to "cool" and that is non cost efficient way in this case because after that I have to pay either for early deletion from cool tier or for deletion and read operation from cool tier.

Could you please change the logic of the restoration process from Azure in the part of rehydration to much more cost efficient: "archive tire" -> "hot tier" or to add additional adjustment to the tape restoration?

Thu Apr 02, 2020 11:51 am

Greetings,

Thank you for your question. Let me clarify that with our R&D team.

Mon Apr 06, 2020 11:13 am

I believe that is the way StarWind Cloud Replicator works. Transferring from archive straight to hot costed more than moving data to cold tier as intermediate when the solution was designed.
Delegated to R&D team to see if this logic should be changed.

dgbegfb · Fri Apr 10, 2020 2:10 pm

Do you have any feedback from R&D team? If you need I can proof that an archive->hot tier recover process is more cost efficient than the others.

Can I expect to see new features in the next release? Next August?

Fri Apr 10, 2020 3:02 pm

Hi,

Yes, I do. Here is what they said

I believe that is the way StarWind Cloud Replicator works. Transferring from archive straight to hot costed more than moving data to cold tier as intermediate when the solution was designed.

I have asked them to see if the data path can be changed in the future builds and do some calculations to make it more cost-efficient.

Can I expect to see new features in the next release? Next August?

Sorry, but I cannot provide any ETA at this point

dgbegfb · Fri Apr 10, 2020 7:42 pm

yaroslav (staff) wrote:Hi,

Yes, I do. Here is what they said
I believe that is the way StarWind Cloud Replicator works. Transferring from archive straight to hot costed more than moving data to cold tier as intermediate when the solution was designed.

That's a delusion! Here you are https://docs.microsoft.com/en-us/azure/ ... el-tiering
"When a blob is moved to a warmer tier (archive->cool, archive->hot, or cool->hot), the operation is billed as a read from the source tier, where the read operation (per 10,000) and data retrieval (per GB) charges of the source tier apply. Early deletion charges for any blob moved out of the cool or archive tier may apply as well."

It means that the cost of moving data from archive tier to whatever warmer is the same cause it consists from the archive tier read operations.

But if the data moves to a cool tier for recover I have to pay for data retrieval, read operations and early deletion once again contrary to hot tier.

Do you see that I mean?

Sun Apr 12, 2020 10:26 am

Yes, I believe I do.
I have already delegated changing the data path to R&D.
I do not have any ETA though.
If there is any news, I will post them here.
Thank you for your effort! We really appreciate that.

leonardocap · Mon Apr 13, 2020 9:50 am

dgbegfb wrote:Hi guys,

I started using VTL 8.0 13481 free license with Veeam and Azure blob storage. I've managed to upload some tapes to Azure archive tier storage directly and restore from there. Doing that I found out that StarWind VTL during rehydration azure archive tier changes blobs tier to "cool" and that is non cost efficient way in this case because after that I have to pay either for early deletion from cool tier or for deletion and read operation from cool tier.

Could you please change the logic of the restoration process from Azure in the part of rehydration to much more cost efficient: "archive tire" -> "hot tier" or to add additional adjustment to the tape restoration?

yeah, please let me know if it is really possible.

Mon Apr 13, 2020 9:58 am

I will keep inform you once the update arrives.

dgbegfb · Thu Mar 25, 2021 4:57 pm

Ahaha... LOL.

"Version 8 (build 14033) Release Notes
MS Azure support: Added the option to upload virtual files to Archive tier directly without uploading to Hot/Cool tier and then moving it to Archive tier in the cloud."

Guys, whether you have been writing a wrong code for a year or someone who write release notes doesn't know what you have done at all

Mon Mar 29, 2021 10:16 am

Hi,

You are correct, the mechanism of uploading the virtual tapes to Azure was changed in the recent build, but, unfortunately, the same for restoring is still under development. We are going to implement your request in the next release build. Stay tuned!

dgbegfb · Fri Oct 27, 2023 10:43 am

Guys,

it looks like you drop down your VTL product. The latest release was in 2021. Is it truth or we cat expect future releases?

As of me I really need an optimal restore process from Azure Archive tire storage.

Mon Oct 30, 2023 11:08 am

Hi,

Thanks for coming back to this thread.
No, the project is not abandoned. The request is still there and should be implemented in the next CVM releases stay tuned!!

dgbegfb · Thu Dec 14, 2023 12:44 pm

Hi guys.

Finally I've managed to make some tests and can proof that your VTL software works with Azure archive storage as I asked about (it writes directly to archive tier and rehydrate data directly to hot tier). I have to stated everything works flawlessly, it's a state of art

Meanwhile I have 3 feature request in order to further improve Azure integration.

1. in the case of immutability at Azure is turned on and I try to delete tape from cloud in VTL it is successfully deleted from VTL DB but not from Azure. There's no error handling in VTL and it's a bad disign.

2. VTL rehydrate archive tape using Set Blob Tier to the same container. It leads to early deletion and impossibility of using life cycle management in the case of immutability. Microsoft recommends using Copy Blob command to Hot tier and if it can be done to another container it will be possible to delete it automatically.

3. using VTL to upload/download tape to/from Azure it uses much many write/read operation in comparison to make the same operation by web gui. I have a chart to compare it if you will need. It just the question of money.

Hope to see improvements in the upcoming release

Any way your SW is awesome and the best VTL add-on to Veeeam! Thank you.

Thu Dec 14, 2023 4:00 pm

Thank you for your feedback.

1. in the case of immutability at Azure is turned on and I try to delete tape from cloud in VTL it is successfully deleted from VTL DB but not from Azure. There's no error handling in VTL and it's a bad design.

VTL tapes are not immutable by design. So, is your request about introducing immutability on the VTL level?

3. using VTL to upload/download tape to/from Azure it uses much many write/read operation in comparison to make the same operation by web gui. I have a chart to compare it if you will need. It just the question of money.

Yes, please I will be happy to have that chart. Thanks!