Cost Optimization on AWS
Introduction
In this post we will focus on how we can optimise our billing cost based on Trusted Advisor service provided by AWS.
We will provide mechanism to stop or terminate running instances, elastic beanstalk or RDS based on under-utilised event for given period of time.
Architecture
Below architecture can be useful in detecting and terminating underutilise ec2/rds instances
Testing Method:
Manually refreshed Trusted Advisor and it will trigger respective event. In spite of everything Trusted Advisor run automatically every day.
Service Required:
1. Trusted Advisor
2. Cloudwatch events 3. SQS
4. Lambda
5. EC2 AMI
6. RDS snapshot
Note: All the services are launched in N. Virginia(us-east-1) because CloudWatch trigger to Trusted Advisor supports only in this region.
Trusted Advisor
There is two possibility to reduce cost, first either change instance type and once work is done keep in stopped state and second we can take AMI and terminate instances.
OR
You can use lambda script (given in later section) which will get data from Trusted advisor and SQS for further operation.
CloudWatch Events
Create rule based on Trusted Advisor for low utilization for ec2 instances and Idle RDS instances. Event Pattern should be similar to below image for ec2. Similarly, it is created for RDS.
optional: Once rule is created, we need to add target as SQS. Since number of request is high therefore, we would required SQS for queue service instead of lambda in target directly.
As per event pattern whenever any changes detect in Trusted Advisor, CloudWatch Event trigger SQS with message in JSON format.
SQS
It gets input from CloudWatch event in JSON format and SQS keep messages in queue to further send to lambda function to perform operation. We are using standard queue in SQS.
Lambda
There must be Lambda function which will process SQS output and get instance ID, region and low utilization / idle state since x number of days. If low utilization is last more than 14 days , we will take AMI/Snapshot and terminate instance.
If someone wants to opt-out instance from termination, needs to tag it’s resource with {‘Retention’ : ‘Yes}
Conclusion
Once successful implementation is complete, there will be significant cost saving in Trusted Advisor. Idle and low utilized instances( EC2, RDS , Elastic Beanstalk) will get terminate.
In one of the AWS account I have saved upto $4000 using above solution.
Reference Link
https://docs.aws.amazon.com/awssupport/latest/user/trusted-advisor.html
