Open Source Sunday: The Azure Test CLI
Open Source Sunday is a series covering my work on open source software. Over the years I’ve developed software for some pretty cool and sometimes high profile companies, but rarely for the open source community. I’d like to use this series to share valuable tools I’ve created or contributed to.
My Honest Opinion of Azure
Recently I’ve been working with Microsoft’s Azure Cloud. For the uninitiated its a competitor to other cloud platforms like Amazon Web Services and Google Cloud. My employer initially chose to use Azure to align our company with key partners, but that soon fell through. The main reason we work with it today is cost. Microsoft was relatively late to the cloud game, so their platform is lacking in functionality and documentation compared to the older players. They compete by offering a cheaper solution now as they mature their offerings.
As I mentioned documentation in Azure cloud is a pain. If you find the support page for the feature you’re working with, there’s a significant chance you’re looking at outdated docs. Since their user base is far behind, so is the info in the blogosphere. This ultimately drags on developer performance, impacting a team’s ability to move as quickly with say AWS, who’s documentation and blog presence is as good as it gets.
Another area azure lacks in is tooling. Yes they have many libraries to integrate with their services and many CLIs too. These tools are lacking in functionality when compared to the older cloud providers. In one case was testing. Their CLI’s don’t allow you integrate directly with many of their popular services. There’s no way to interact with Azure’s blob storage, service bus, or eventhub with their CLIs. This was a major pain as we attempted to troubleshoot these offerings as developers or manually test as QA engineers, slowing down both development and testing.
Enter the Azure Test CLI
Frustrated with the state of things, I decided to take it upon myself to build something that could help. I poked around the available libraries and found that Azure had decent bindings to these services in their python library. I jumped in head first and learned the CLI python framework click and developed a tool that filled this painful void.
What I ended up with was a surprisingly powerful python cli capable of interacting with live instances of blob storage, service bus, and eventhub. I’m going to share with you what it does and how to use it.
Installation
Before we get started it’s required you have python 3 installed locally. I’m not going to instruct you how to do this. A simple google search should yield some easy instructions.
Once you’ve got python 3 installed, to locally install the cli run the following command.
$ pip install azure-test-cli
Great! The cli is ready to be executed.
Now, to see the top level options enter the following into your command line.
$ aztest --help Usage: aztest [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: eventhub Perform eventhub tests servicebus Perform servicebus tests storage Perform blobstorage tests
As you can see, we’ve been given 3 distinct parts of the help page.
Usage - How you use this command
Options - What options you can feed this command
Commands - What subcommands you can run
The cli structure is broken down by distinct azure service, eventhub, servicebus, and blob storage. At the moment it only supports these 3 services, but stay tuned for more on that.
Event Hub
The tool provides two operations for eventhub, receive and send.
Usage: aztest eventhub [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: receive Receive messages from eventhub send Send messages to eventhub
As you can see there are two operations available with eventhub, receive and test. They’re pretty self explanatory. Receive allows you to consume records from eventhub and send allows you to push events to eventhub.
Receive
Receive events from live eventhub. It will start receiving from zero offset.
Usage: aztest eventhub receive [OPTIONS]
Options:
-s, --eventhub_namespace TEXT Name of event hub namespace [required]
-n, --eventhub_name TEXT Name of eventhub [required]
-p, --eventhub_sas_name TEXT Name of eventhub SAS policy with listen
rights [required]
-k, --eventhub_sas_key TEXT Key value of eventhub SAS policy with listen
rights [required]
-c, --consumer_group TEXT Name of event hub consumer group [required]
--help Show this message and exit.Send
Send events to an eventhub. You can provide a repeat to send the same message multiple times.
Usage: aztest eventhub send [OPTIONS]
Options:
-s, --eventhub_namespace TEXT Name of event hub namespace [required]
-n, --eventhub_name TEXT Name of eventhub [required]
-p, --eventhub_sas_name TEXT Name of eventhub SAS policy with send rights
[required]
-k, --eventhub_sas_key TEXT Key value of eventhub SAS policy with send
rights [required]
-m, --message TEXT Message to send. Must be in quotes
[required]
-r, --repeat INTEGER Count of times to repeat the send of the
message
--help Show this message and exit.Service Bus
There are two groups of commands for services bus, queue and topic.
Usage: aztest servicebus [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: queue Perform servicebus queue tests topic Perform servicebus topic tests
Service Bus - Queue
There are two operations for service bus queues: receive and send.
Usage: aztest servicebus queue [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: receive Receive messages onto servicebus queue send Send messages onto servicebus queue
Receive
Receive messages off a servicebus queue.
Usage: aztest servicebus queue receive [OPTIONS]
Options:
-s, --servicebus_namespace TEXT
Name of servicebus namespace [required]
-n, --servicebus_queue_name TEXT
Name of queue [required]
-p, --servicebus_sas_name TEXT Name of servicebus SAS policy with listen
access [required]
-k, --servicebus_sas_key TEXT Key value of servicebus SAS policy with
listen access [required]
--help Show this message and exit.Send
Send messages onto a servicebus queue.
Usage: aztest servicebus queue send [OPTIONS]
Options:
-s, --servicebus_namespace TEXT
Name of servicebus namespace [required]
-n, --servicebus_queue_name TEXT
Name of queue [required]
-p, --servicebus_sas_name TEXT Name of servicebus SAS policy with send
access [required]
-k, --servicebus_sas_key TEXT Key value of servicebus SAS policy with send
access [required]
-m, --message TEXT Message to send. Must be in quotes
[required]
-r, --repeat INTEGER Count of times to repeat the send of the
message
--help Show this message and exit.Service Bus - Topic
There are two operations for service bus topic receive and send
Receive
Receive messages from a service bus topic.
Usage: aztest servicebus topic receive [OPTIONS]
Options:
-s, --servicebus_namespace TEXT
Name of servicebus namespace [required]
-n, --servicebus_topic_name TEXT
Name of topic [required]
-p, --servicebus_sas_name TEXT Name of servicebus SAS policy with listen
access [required]
-k, --servicebus_sas_key TEXT Key value of servicebus SAS policy with
listen access [required]Send
Send messages onto a servicebus topic
Usage: aztest servicebus topic send [OPTIONS]
Options:
-s, --servicebus_namespace TEXT
Name of servicebus namespace [required]
-n, --servicebus_topic_name TEXT
Name of topic [required]
-p, --servicebus_sas_name TEXT Name of servicebus SAS policy with send
access [required]
-k, --servicebus_sas_key TEXT Key value of servicebus SAS policy with send
access [required]
-m, --message TEXT Message to send. Must be in quotes
[required]
-r, --repeat INTEGER Count of times to repeat the send of the
message
--help Show this message and exit.Blob Storage
Blob storage is broken up by blob type: block blobs and append blobs.
Usage: aztest storage [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: appendblob Perform append blob tests blockblob Perform block blob tests
Append Blobs
Append blobs allow you append data to existing blob files. The CLI provides four operations: append, delete, download, and stream.
Usage: aztest storage appendblob [OPTIONS] COMMAND [ARGS]... Options: --help Show this message and exit. Commands: append Upload content to an append blob delete Delete an append blob download Download append blob to local file stream Stream append blob to output
Append
Append content to an append blob, create one if it does not exist. Provides a repeat option to append the same content multiple times.
Usage: aztest storage appendblob append [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Name of SAS policy with write access [required] -c, --container TEXT Blob container name [required] -b, --blob_path TEXT Blob path of uploaded file [required] -f, --file_path TEXT Content to upload to path [required] -r, --repeat INTEGER Count of times to repeat the append the content --help Show this message and exit.
Download
Download an append blob to local path.
Usage: aztest storage appendblob download [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Name of SAS policy with write access [required] -c, --container TEXT Blob container name [required] -b, --blob_path TEXT Blob path of uploaded file [required] -f, --file_path TEXT Content to upload to path [required] --help Show this message and exit.
Delete
Delete an existing blob.
Usage: aztest storage appendblob delete [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Name of access key [required] -c, --container TEXT Blob container name [required] -b, --blob_path TEXT Blob path of file [required]
Stream
Stream append updates from an append blob. If the blob doesn’t exist, the tool will wait 60 seconds for it to be created before timing out.
Usage: aztest storage appendblob stream [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Name of SAS policy with write access [required] -c, --container TEXT Blob container name [required] -b, --blob_path TEXT Blob path of uploaded file [required]
Block Blobs
Block blobs let you upload large blobs efficiently. There are two operation available at the moment: upload and download.
Upload
Upload a block blob from a local file.
Usage: aztest storage blockblob upload [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Storage access key [required] -c, --container TEXT Blob container name [required] -b, --path TEXT Blob path of uploaded file [required] -f, --file TEXT Content to upload to path [required]
Download
Download a blockblob to a local file
Usage: aztest storage blockblob download [OPTIONS] Options: -a, --storage_account TEXT Name of blob account [required] -k, --storage_key TEXT Storage access key [required] -c, --container TEXT Blob container name [required] -b, --path TEXT Blob path of uploaded file [required] -f, --file TEXT Content to upload to path [required]
A Work In Progress
Though this tool yielded instant value for myself and co-workers, this is still a work in progress. I’ve open-sourced this project at on github for anyone to fork. If you have any issues please add an issue there or leave a comment below. The same goes if you have any ideas of how I could improve or expand this tool.
This project was fun, relatively easy, and surprisingly helpful when troubleshooting azure based applications. Hopefully this tool helps you in the future.