Control and Manage AWS costs efficiently
There are many organizations looking for ways to find control the Cloud cost. Its very challenging to maintain and monitor the consumption of resources for each cloud provider unless you have solid Cloud Financial Management (CMF) Framework. So let us see how to build strong CMF, here am taking AWS as cloud platform and explained various cost monitoring and controlling mechanism
According to AWS, their recommendation for CMF are
See : -
Account & tagging strategy Cost reporting & monitoring Showback & Chargeback Efficiency/value KPIs
Save:-
Cost aware architecture, design & service selection Match capacity with demand Choose the right pricing model Resource governance
Plan:-
Strategic fit Business case & value articulation POC based cost estimation Budgeting & forecasting variable cloud spend
Run:-
Partnership between Finance & Technology organizations Invest in people, processes, governance & tools Celebrate wins
Before we moved to cost management framework, let us understand the AWS pricing model
Organize your AWS resources as per below recommendations
Using tags for visibility and chargeback
A tag is a label that you assign to an AWS resource. A tag consists of a key and a value, both of which you define. For example, if you have two Amazon EC2 instances, you might assign both a tag key of “Stack.” But the value of “Stack” might be “Testing” for one and “Production” for the other.
Tagging can help you organize your resources and enables you to simplify resource management, access management and cost allocation
Tagging is one of the powerful strategies in organization to control Cost, Access control and Security Risk management
Tags for AWS Console Organization and Resource Groups
· Organize AWS resources in the AWS Management Console
· View resources and can search and filter by tag
By default, the AWS Management Console is organized by AWS service. However, the Resource Groups tool allows customers to create a custom console that organizes and consolidates AWS resources based on one or more tags or portions of tags. Using this tool, customers can consolidate and view data for applications that consist of multiple services and resources in one place
Tags for Cost Allocation
AWS Cost Explorer and Cost and Usage Report support the ability to break down AWS costs by tag. Typically, customers use business tags such as cost center, business unit, or project to associate AWS costs with traditional financial reporting dimensions within their organization. However, a cost allocation report can include any tag. This allows customers to easily associate costs with technical or security dimensions, such as specific applications, environments, or compliance programs.
customers can use the AWS Tag Editor to view and manage tags on their AWS resources, regardless of service or region. They can use the tag editor to search for resources by resource type, region, or tag, and then manage the tags applied to those resources.
Please refer the below article on Tagging Governance and Best practices
https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf
How to Tag the untagged resources
First, find untagged resources and tag the resources
Find resources to tag
With Tag Editor in AWS, you can build a query to find resources in one or more AWS Regions that are available for tagging. You can choose up to 20 individual resource types or build a query on All resource types. Your query can include resources that already have tags, or resources that have no tags
Automate the tagging governance and manage cost efficiently
AWS has efficient architecture for extracting and querying tagged resources based on AWS cloud-native features such as the Resource Tagging API and S3 Select.
By relying on AWS cloud-native features, customers can save time and reduce costs while still being able to do customizations.
Resource Tagging API
AWS customers can use the Resource Tagging API to programmatically access the same resource group operations that had been accessible only from the AWS Management Console by now using the AWS SDKs or the AWS Command Line Interface (CLI). By doing so, customers can build automation that fits their need, e.g., code that extract, export, and queries tagged resources.
For further details, please read Resource Groups Tagging — Reference
S3 Select
S3 Select enables applications to retrieve only a subset of data from an object by using simple SQL expressions. By using S3 Select to retrieve only the data needed by the application, customers can achieve drastic performance increases — in many cases you can get as much as a 400% improvement.
For further details, please read:
- S3 Select and Glacier Select — Retrieving Subsets of Objects
- Amazon S3 Update: New Storage Class and General Availability of S3 Select
Below is the one of the recommended solutions to manage efficiently on tagged resource and query the data using S3 Select
Please refer the overall solution https://aws.amazon.com/blogs/architecture/how-to-efficiently-extract-and-query-tagged-resources-using-the-aws-resource-tagging-api-and-s3-select-sql/
Other solutions are using AWS Application Cost Monitoring
AWS Application Cost Monitoring is a serverless application provides granular AWS spend tracking.
Using the AWS Cost and Usage report this system will parse the report and feed the line items to a place where you can perform analysis.
For deploying AWS cost and usage report, please refer –
Sample Python code to get all the tagged resources
Import boto3
rgta = boto3.client(‘resourcegroupstaggingapi’)
rgta.get_resources()
Sample python code to get all ( Tagged and untagged ) ec2 instances
Filter using tags
Please refer boto resourcegroupstaggingapi -https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/resourcegroupstaggingapi.html
