There are several options when you want to move files in or out of an Azure Storage account to a Red Hat Linux (RHEL) server. Below is a quick break down of the most commonly used options.
Azure CLI
Azure CLI is design to run on Linux or Windows so is the ideal tool when a Linux machine is involved. Azure CLI 2.0 (https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) is the latest and preferred version. See https://docs.microsoft.com/en-us/azure/storage/common/storage-azure-cli#create-and-manage-blobs for syntax.
Example:
#Start the copy of the blob
az storage blob copy start –account-name ${storage_account} –account-key ${key} –source-account-name ${source_account} –source-account-key ${source_key} –source-container ${blob_container} –source-blob ${blob_name} –destination-container $target_container –destination-blob $target_blob_name
#Now wait while this copy happening. This could take a while.
while [ $(az storage blob show –account-name ${storage_account} –account-key ${key} –container-name $target_container –name $target_blob_name| jq .properties.copy.status| tr -d ‘”‘) == “pending” ]; do
progress=$(az storage blob show –account-name ${storage_account} –account-key ${key} –container-name ${target_container} –name ${target_blob_name} | jq -r .properties.copy.progress)
done_count=$(echo $progress | sed ‘s,\([0-9]*\)/\([0-9]*\),\1,g’)
total_count=$(echo $progress | sed ‘s,\([0-9]*\)/\([0-9]*\),\2,g’)
progress_percent=$((100 * $done_count / $total_count))
echo “Copying: $progress ($progress_percent %)” && sleep 5
done
echo “Copying done”
AzCopy
AzCopy on Linux is a command-line utility designed for copying data to/from Azure Blob and File storage using simple commands. See https://docs.microsoft.com/en-us/azure/storage/common/storage-use-azcopy-linux?toc=%2fazure%2fstorage%2ffiles%2ftoc.json. Specific steps for the file copy are here.
To Install on RHEL, you first need to install .NET Core 1.1.1
- sudo yum install rh-dotnetcore11
- scl enable rh-dotnetcore11 bash
- source scl_source enable rh-dotnetcore11
Then you install AzCopy per the article.
- wget -O azcopy.tar.gz https://aka.ms/downloadazcopyprlinux
- tar -xf azcopy.tar.gz
- sudo ./install.sh
Example from RHEL 7.4:
Wget
The final approach is to downloading is to generate a SAS token, append it to the blob URL and simply use wget in Linux.
See https://blogs.msdn.microsoft.com/nicole_welch/2017/05/using-azure-cli-1-0-to-copy-a-file-between-subscriptions/ for details on how to generate the SAS token and append it to the the blob URL (aka, “sourceSAS”).
*previously posed on https://blogs.msdn.microsoft.com/nicole_welch/2017/09/moving-files-between-azure-storage-and-rhel/
Nicole Welch 