Copy content from one Azure Storage account to another

Q. How can I copy all contents from an Azure Storage account container to another Azure Storage account?

A. The easiest way to copy between storage accounts is to use the Azure asynchronous server-side copy capability which is exposed through the Start-CopyAzureStorageBlob cmdlet. This means all the data is sent directly between the Azure locations and not via your location. It's also intelligent which means even though a VHD may be 1 TB in size, if it only contains 100 MB of data it will only copy the 100 MB which will save you egress network charges.

Below is some PowerShell scripts I created that copy everything in a container to a container in another storage account. This can be across subscriptions and across regions. Note you need to replace the variables at the start with your own storage account names and storage account keys (which can be found through the Azure portals). You can also change the source and destination containers.

#Server side storage copy
$SourceStorageAccount = "storageaccount1"
$SourceStorageKey = "yourKey1=="
$DestStorageAccount = "storageaccount2"
$DestStorageKey = "yourKey2=="
$SourceStorageContainer = 'vhds'
$DestStorageContainer = 'vhds'
$SourceStorageContext = New-AzureStorageContext –StorageAccountName $SourceStorageAccount -StorageAccountKey $SourceStorageKey
$DestStorageContext = New-AzureStorageContext –StorageAccountName $DestStorageAccount -StorageAccountKey $DestStorageKey

$Blobs = Get-AzureStorageBlob -Context $SourceStorageContext -Container $SourceStorageContainer
$BlobCpyAry = @() #Create array of objects

#Do the copy of everything
foreach ($Blob in $Blobs)
{
   Write-Output "Moving $Blob.Name"
   $BlobCopy = Start-CopyAzureStorageBlob -Context $SourceStorageContext -SrcContainer $SourceStorageContainer -SrcBlob $Blob.Name `
      -DestContext $DestStorageContext -DestContainer $DestStorageContainer -DestBlob $Blob.Name
   $BlobCpyAry += $BlobCopy
}

#Check Status
foreach ($BlobCopy in $BlobCpyAry)
{
   #Could ignore all rest and just run $BlobCopy | Get-AzureStorageBlobCopyState but I prefer output with % copied
   $CopyState = $BlobCopy | Get-AzureStorageBlobCopyState
   $Message = $CopyState.Source.AbsolutePath + " " + $CopyState.Status + " {0:N2}%" -f (($CopyState.BytesCopied/$CopyState.TotalBytes)*100) 
   Write-Output $Message
}

This will start the server-side copy and storage each copy operational request as part of an array. This is used in the final piece of code to check the status. This can be run multiple times to check status, i.e. re-run the part below:

#Check Status
foreach ($BlobCopy in $BlobCpyAry)
{
   #Could ignore all rest and just run $BlobCopy | Get-AzureStorageBlobCopyState but I prefer output with % copied
   $CopyState = $BlobCopy | Get-AzureStorageBlobCopyState
   $Message = $CopyState.Source.AbsolutePath + " " + $CopyState.Status + " {0:N2}%" -f (($CopyState.BytesCopied/$CopyState.TotalBytes)*100) 
   Write-Output $Message
}

Below is an example execution where I check the status multiple times. My source container had 6 files.

Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name
Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name
Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name
Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name
Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name
Moving Microsoft.WindowsAzure.Commands.Storage.Model.ResourceModel.AzureStorageBlob.Name

/vhds/Minecraft-20150417-083952.vhd Pending 0.00%
/vhds/Minecraft-Data2.vhd Pending 0.00%
/vhds/Minecraft-Data3.vhd Pending 0.00%
/vhds/Minecraft-Data4.vhd Pending 0.06%
/vhds/Minecraft-Savill-Minecraft-os-1429277175051.vhd Pending 0.00%
/vhds/Minecraft-Savill.Minecraft-Savill.Minecraft.status Success 100.00%

/vhds/Minecraft-20150417-083952.vhd Pending 0.16%
/vhds/Minecraft-Data2.vhd Pending 0.16%
/vhds/Minecraft-Data3.vhd Pending 0.16%
/vhds/Minecraft-Data4.vhd Pending 12.60%
/vhds/Minecraft-Savill-Minecraft-os-1429277175051.vhd Pending 0.44%
/vhds/Minecraft-Savill.Minecraft-Savill.Minecraft.status Success 100.00%

/vhds/Minecraft-20150417-083952.vhd Success 100.00%
/vhds/Minecraft-Data2.vhd Success 100.00%
/vhds/Minecraft-Data3.vhd Success 100.00%
/vhds/Minecraft-Data4.vhd Success 100.00%
/vhds/Minecraft-Savill-Minecraft-os-1429277175051.vhd Pending 1.37%
/vhds/Minecraft-Savill.Minecraft-Savill.Minecraft.status Success 100.00%

/vhds/Minecraft-20150417-083952.vhd Success 100.00%
/vhds/Minecraft-Data2.vhd Success 100.00%
/vhds/Minecraft-Data3.vhd Success 100.00%
/vhds/Minecraft-Data4.vhd Success 100.00%
/vhds/Minecraft-Savill-Minecraft-os-1429277175051.vhd Pending 100.00%
/vhds/Minecraft-Savill.Minecraft-Savill.Minecraft.status Success 100.00%

Note that even though my Data files were all 1 TB they finished much faster than the 127 GB OS drive since the OS drive had around 20 GB of content while each data drive only had around 50 MB of content showing only written data is copied.

Comments

Plain text