library
turbot/aws_thrifty
- Detect & correct DynamoDB table with stale data
- Detect & correct EBS snapshots exceeding max age
- Detect & correct EBS volumes attached to stopped instances
- Detect & correct EBS volumes exceeding max size
- Detect & correct EBS volumes if unattached
- Detect & correct EBS volumes using gp2
- Detect & correct EBS volumes using io1
- Detect & correct EBS volumes with low IOPS
- Detect & correct EBS volumes with low usage
- Detect & correct EC2 application load balancers if unused
- Detect & correct EC2 classic load balancers if unused
- Detect & correct EC2 gateway load balancers if unused
- Detect & correct EC2 instances exceeding max age
- Detect & correct EC2 instances large
- Detect & correct EC2 instances of older generation
- Detect & correct EC2 instances without graviton
- Detect & correct EC2 network load balancers if unused
- Detect & correct EKS node groups without graviton
- Detect & correct Elasticache clusters exceeding max age
- Detect & correct EMR Clusters idle 30 mins
- Detect & correct Lambda functions without graviton
- Detect & correct RDS DB instances exceeding max age
- Detect & correct RDS DB instances of older generation
- Detect & correct RDS DB instances with low connection count
- Detect & correct RDS DB instances without graviton processor
- Detect & correct Route53 health checks if unused
- Detect & correct Route53 records with lower TTL
- Detect & correct S3 buckets without lifecycle policy
- Detect & correct SecretsManager secrets if unused
- Detect & correct VPC EIPs if unattached
- Detect & correct VPC NAT gateways if unused
Get Involved
Version
Detect & correct EMR clusters idle for more than 30 mins
EMR clusters which are live but not currently running tasks should be reviewed and checked whether the cluster has been idle for more than 30 minutes. It is ideal to delete such clusters for cost optimization.
This query trigger identifies EMR clusters idle for more than 30 mins and then either sends a notification or attempts to perform a predefined corrective action.
Getting Started
By default, this trigger is disabled, but it can be configured by setting the following variables:
emr_clusters_idle_30_mins_trigger_enabled
should be set totrue
as the default isfalse
.emr_clusters_idle_30_mins_trigger_schedule
should be set to your preferred schedule.emr_clusters_idle_30_mins_default_action
should be set to the desired action (e.g.,"notify"
for notifications or"terminate_cluster"
to terminate the cluster).
Then starting the server:
flowpipe server
or if you've set the variables in a .fpvars
file:
flowpipe server --var-file=/path/to/your.fpvars
Query
with cluster_metrics as ( select id, maximum, date(timestamp) as timestamp from aws_emr_cluster_metric_is_idle where timestamp <= current_timestamp - interval '30 minutes'),emr_cluster_isidle as ( select id, count(maximum) as count, sum(maximum) / count(maximum) as avagsum from cluster_metrics group by id, timestamp)select concat(i.id, ' [', i.region, '/', i.account_id, ']') as title, i.id, i.region, i._ctx ->> 'connection_name' as credfrom aws_emr_cluster as i left join emr_cluster_isidle as u on u.id = i.idwhere u.id is null and avagsum = 1 and count >= 7;
Schedule
15m