Efficiently Tagging AWS Resources Using CLI to Better Manage Resources and Billing Costs

It is common when organizations have large workloads based on on a multitude of AWS services, they may lose track of how resources are being used. In a nutshell, identifying resources can take rigorous effort. On AWS, utilization and cost go hand in hand and tagging helps ensure that the resources are managed efficiently. In fact, one could also build insightful reports/dashboards with the tags in place.

Tagging Strategy:

For tags to be effective at scale they need to be strategically managed. Many organizations group tags into different categories like technical, business, security and automation, etc. A typical set of tags could be:

1. Name
2. Owner
3. Application/Project/Product
4. Environment
5. Client/Customer

For more on creative tagging strategies, please read this.

Prerequisites: AWS CLI configured.

Getting all untagged resources using CLI:

As of this writing, there is no CLI command to list all untagged resources. One could follow the below steps to get the list.

Step1: List all the resources in AWS and write them to a text file

aws lambda list-functions --profile PROFILE_NAME &>> resourcesList.txt

Note: The above command is for listing details of lambda resources. The command and its output might vary with other resources. Read more here.

&>> appends the output of the command to resourcesList.txt file in the current working directory.

The output of the above command is a JSON object that looks like this:

{
    "Functions": [
        {
            "FunctionName": "Chat-Conversation-POST",
            "FunctionArn": "arn:aws:lambda:us-west-2:89XXXXXXXX14:function:Chat-Conversation-POST",
            "Runtime": "nodejs8.10",
            "Role": "arn:aws:iam::89XXXXXXXX14:role/chat-lambda-data",
            "Handler": "index.handler",
            "CodeSize": 474,
            "Description": "",
            "Timeout": 15,
            "MemorySize": 128,
            "LastModified": "2019-05-02T13:20:53.887+0000",
            "CodeSha256": "h1bxXaXXXXXXxxxxxxxxxXxxXxxxxxxXXXxxxxxxmGg=",
            "Version": "$LATEST",
            "TracingConfig": {
                "Mode": "PassThrough"
            },
            "RevisionId": "f447bca3-06f9-49d8-8a5d-c740f6aec405"
        },
        {
            "FunctionName": "Chat-Conversation-GET",
            "FunctionArn": "arn:aws:lambda:us-west-2:89XXXXXXXX14:function:Chat-Conversation-GET",
            "Runtime": "nodejs8.10",
            "Role": "arn:aws:iam::89XXXXXXXX14:role/service-role/chat-lambda-data",
            "Handler": "index.handler",
            "CodeSize": 785,
            "Description": "",
            "Timeout": 25,
            "MemorySize": 128,
            "LastModified": "2019-05-04T14:23:07.002+0000",
            "CodeSha256": "h1bxXaXXXXXXxxxxxxxxxXxxXxxxxxxXXXxxxxxxmGg=",
            "Version": "$LATEST",
            "VpcConfig": {
                "SubnetIds": [],
                "SecurityGroupIds": [],
                "VpcId": ""
            },
            "TracingConfig": {
                "Mode": "PassThrough"
            },
            "RevisionId": "210dd3fa-ba47-4e06-ab53-e34aa793b344"
        }
    ]
}

Now, one could either use multiple selection (ctrl+d) in Sublime or a python script to extract the list of resource ARN/names.

Step2: Iterate this list of resource names, and fetch tagging details for all of them & append the output of these commands to a file.

echo RESOURCE_NAME: &>> tagsList.txt

aws lambda list-tags --resource arn:aws:lambda:us-west-2:89XXXXXXXX14:function:RESOURCE_NAME --profile PROFILE_NAME &>> tagsList.txt

The output of the above command is also a JSON object:

{
    "Tags": {}
}

As you can see there is no name attribute here so, we add the resource name before command output using echo:

RESOURCE_NAME:{
    "Tags": {}
}

Let us say, the resource names we have got in resourcesList.txt are as follows:

• new-client-acquisition
• initiate-raw-file-ingestion
• initiate-raw-crawler
• raw-refined-transform
• initiate-refined-crawler
• check-status

Creating commands for the above resources in sublime:

Step3: Extract resources with no tags from the tagsList.txt file.

Untagged = all – tagged

From the resourcesList.txt we get all the resource names, and from the tagsList.txt we get all tagged resources. You could use both these lists to get the untagged resources.

Step4: Preparing and Updating the tags

aws lambda tag-resource --resource arn:aws:lambda:us-west-2:89XXXXXXXX14:function:RESOURCE_NAME --tags Environment=prod,Project=sales,Name=RESOURCE_NAME --profile PROFILE_NAME

Create multiple commands for each resource name with the above template.

Once you create all the commands just copy and paste them in the terminal. That would update all the resources with new tags.

This is pretty much the steps involved in tagging resources, maybe a few tweaks have to be made depending on the AWS service.

Note: Output of all the above commands are executed with default region name specified in the AWS CLI profile, if not specified in the command.

One other way of tagging resources on AWS is using Tag Editor in Resource Groups. I found it hard to work with as one couldn’t sophisticatedly search, filter or group resource names.

I hope it was helpful. For any queries or if you know a better way of tagging let us know in the comment section. Happy to discuss it further.

Thank-you!

This story is authored by Koushik. Koushik is a software engineer and a keen data science and machine learning enthusiast.