The Azure SDK team reworked the SDK for Go and released a bunch of Go modules to interact with popular cloud services - not just from a management perspective. In this post, we will take a closer look at the Go-modules for interacting with blob storage capabilities offered by Azure Storage Account.

When interacting with Azure services, we must authenticate. The Azure SDK for Go supports different authentication patterns and flows. For demonstration purposes and to keep things simple, we’ll address two different authentication approaches as part of this article:

  • Authentication by re-using Azure CLI credentials
  • Authentication using Azure Storage account access keys

General authentication patterns like re-using Azure CLI credentials (or leveraging Managed Service Identities MSIs) can be implemented using Go’s azidentity module. In contrast, Azure services like Azure Storage Accounts support built-in authentication mechanisms like connection strings or access keys. Those authentication capabilities are typically part of the service module (azblob here). In real-world scenarios, you should always use techniques like MSIs or Role-Based-Access Control (RBAC) for authentication.

The sample application

Samples shown here are taken from azb, a simple CLI I built to easily backup files to an Azure Storage Account. The CLI allows users to:

  • Upload files to a configurable Azure Storage Account
  • Download previously uploaded blobs from that particular Azure Storage Account
  • List all blobs in the uploads container
  • Delete a blob from the uploads container

You can find azb on GitHub at ThorstenHans/azb.

Provision an Azure Storage Account for testing purposes

We must have access to an Azure Storage Account. For demonstration purposes, let’s quickly spin up a new Azure Storage Account and a container (folder) inside of that Storage Account using Azure CLI. Additionally, the script will create a new role assignment that assigns the Storage Blob Data Contributor role to the user currently signed in with Azure CLI. If you’re unfamiliar with Azure CLI, you can use Azure Portal or other management interfaces to create the Azure Storage Account.

In the following snippet, two important variables are defined. storageAccountName and storageAccountKey, we will use those variables later to pass the necessary information to our Go application:

rgName=rg-blog-blob-sample
location=germanywestcentral
storageAccountName=sablogblob2022

# Login
az login

# Select desired Azure Subscription
az account set --subscription <SUB_NAME_OR_ID>

# create a Resource Group
az group create -n $rgName -l $location

# create a Storage Account
saId=$(az storage account create -n $storageAccountName \
    -g $rgName \
    -l $location \
    --query "id" \
    -otsv)

# Make the current user a "Storage Blob Data Contributor"
currentUserId=$(az ad signed-in-user show --query "id" -otsv)
az role assignment create --scope $saId \
    --role "Storage Blob Data Contributor" \
    --assignee $currentUserId

# grab primary Storage Account Key
storageAccountKey=$(az storage account keys list -n $storageAccountName -g $rgName --query "[?keyName == 'key1'].value" -otsv)

az storage container create -n uploads -g $rgName \
   --account-name $storageAccountName \
   --account-key $storageAccountKey \
   -onone

# print storage account name
echo $storageAccountName
echo $storageAccountKey

With the storage account and the uploads container (folder) in place, we can use the Azure Storage Account from within Go.

Install necessary Go modules

To interact with Azure Storage Account and implement different authentication patterns within a Go application, we have to install the necessary Go module (azblob) with go get:

# install Go-module to interact with Azure Storage Account
go get github.com/Azure/azure-sdk-for-go/sdk/storage/azblob

# install Go-module for common Azure authentication patterns 
go get github.com/Azure/azure-sdk-for-go/sdk/azidentity

If you want to interact with other Azure services using Go, consult the list of all modules available as part of the Azure SDK for Go.

Implement Authentication

As mentioned at the article’s beginning, we can consult azidentity or the service module (azblob) to authenticate. In the following paragraphs, we will implement two authentication patterns and hide the authentication strategy behind a simple boolean useAzureCliAuth to easily swap the procedure.

Authentication with azidentity

Among many other supported authentication patterns, we can use theNewAzureCLICredential function of azidentity to get a credential set re-using existing authentication material from Azure CLI. We can use those credentials to create a new instance of azblob.Client as the following snippet demonstrates:

func getClient(accountName string) (*azblob.Client, error) {
    cred, err := azidentity.NewAzureCLICredential(nil)
    if err != nil {
        return nil, err
    }
    url := fmt.Sprintf("https://%s.blob.core.windows.net/", accountName)
    return azblob.NewClient(url, cred, nil)
}

Authentication with azblob

To authenticate using Azure Storage Account keys, you can use the NewSharedKeyCredential and NewClientWithSharedKeyCredential methods provided by azblob:

func getClient(accountName string, accountKey string) (*azblob.Client, error) {
    cred, err := azblob.NewSharedKeyCredential(accountName, accountKey)
    if err != nil {
        return nil, err
    }
    url := fmt.Sprintf("https://%s.blob.core.windows.net/", accountName)
    return azblob.NewClientWithSharedKeyCredential(url, cred, nil)
}

Combine multiple authentication strategies

Now that we know how to use azidentity and azblob for authentication, we can combine both strategies and use the desired implementation by simply modifying a boolean. The following code assumes that all necessary parameters are directly passed into the getClient function for demonstration purposes. In an actual implementation, you would typically introduce some sort of configuration struct to encapsulate this information and streamline your code:

func getClient(accountName string, accountKey string, useCliAuth bool) (*azbolb.Client, error) {
    url := fmt.Sprintf("https://%s.blob.core.windows.net/", accountName)
    if useCliAuth {
        cred, err := azidentity.NewAzureCLICredential(nil)
        if err != nil {
            return nil, err
        }
        return azblob.NewClient(url, cred, nil)
    }
    cred, err := azblob.NewSharedKeyCredential(accountName, accountKey)
    if err != nil {
        return nil, err
    }
    return azblob.NewClientWithSharedKeyCredential(url, cred, nil)
}

In contrast to the previous samples, let’s assume that all configuration data provided in upcoming snippets is provided by a cfg variable representing an instance of the following Configuration struct:

type Configuration struct {
    UseCliAuth bool
    StorageAccountName string
    StorageAccountKey string
}

With the authenticated azblob.Client in place, we can implement use-cases specific to the Azure Storage Account like upload, download, and listBlobs

Upload a blob to Azure Storage Account

First, let’s see how we can upload a file from the local machine to Azure blob storage. With an authenticated client ( *azblob.Client), we can use the UploadFile method and let the SDK do the heavy lifting for us:

func Upload(fileName string) error {
    containerName := "uploads"
    client, err := getClient(cfg)
    if err != nil {
        return err
    }
    
    file, err := os.OpenFile(fileName, os.O_RDONLY, 0)
    if err != nil {
        return err
    }
    defer file.Close()

    _, err = client.UploadFile(context.Background(), containerName, fileName, file, nil)
    return err
}

Download a blob from Azure Storage Account

We can download blobs from Azure Blob Storage and store them on the local machine. To do so, we must know the name of the desired blob, and we must come up with a file handle where the SDK could store the contents of the blob to:

func Download(blobName string, destination string) error {
    containerName := "uploads"
    client, err := getClient(cfg)
    if err != nil {
        return err
    }
    target := path.Join(destination, blobName)
    d, err := os.Create(target)
    if err != nil {
        return err
    }
    defer d.Close()

    _, err = client.DownloadFile(context.Background(), containerName, blobName, d, nil)
    return err
}

List all blobs of a particular container in Azure Storage Account

When listing items stored in different Azure services, all available Azure SDK modules use so-called pagers to deliver items in paged results. By iterating over the pages, we can access all blobs in a particular container:

func ListBlobs() error {
    containerName := "uploads"
    creds, err := getClient(cfg)
    if err != nil {
        return err
    }
    pager := client.NewListBlobsFlatPager(containerName, nil)
    for pager.More() {
        page, err := pager.NextPage(context.Background())
        if err != nil {
            return err
        }
        for _, blob := range page.Segment.BlobItems {
            fmt.Println(*blob.Name)
        }
    }
    return nil
}

Delete a particular blob from Azure Storage Account

Last but not least, let’s see how to delete a particular blob from Azure Blob Storage.

func DeleteBlob(blobName string) error {
    containerName := "uploads"
    client, err := getClient(cfg)
    if err != nil {
        return err
    }
    
    _, err = client.DeleteBlob(context.Background(), containerName, blobName, nil)
    return err
}

Conclusion

We can easily interact with Azure services like Azure Storage Accounts using the Azure SDK for Go. We must pull the desired Go modules and use the authentication strategy before interacting with the service. Unfortunately, only a small fraction of Azure services provided SDKs for data plane activities. Besides Azure Storage Account (Blobs), the Azure SDK for Go allows you to interact with:

I hope to see additional services being supported by Azure SDK for Go. However, I don’t think we will see feature parity with .NET and JavaScript counterparts soon.