Pipeline: Detect & correct EMR Clusters idle 30 mins

Overview

EMR clusters which are live but not currently running tasks should be reviewed and checked whether the cluster has been idle for more than 30 minutes. It is ideal to delete such clusters for cost optimization.

This pipeline detects EMR clusters idle for more than 30 mins and then either sends a notification or attempts to perform a predefined corrective action.

Getting Started

This control will work out-of-the-box with some sensible defaults (configurable via variables).

You should be able to simply run the following command in your terminal:

flowpipe pipeline run detect_and_correct_emr_clusters_idle_30_mins

By default, Flowpipe runs in wizard mode and prompts directly in the terminal for a decision on the action(s) to take for each detected resource.

However, you can run Flowpipe in server mode with external integrations, allowing it to prompt for input via http, slack, teams, etc.

Alternatively, you can choose to configure and run in other modes:

Notify: Provides detections without taking any corrective action.
Automatic: Performs corrective actions automatically without user intervention.

Run the pipeline

To run this pipeline from your terminal:

flowpipe pipeline run aws_thrifty.pipeline.detect_and_correct_emr_clusters_idle_30_mins

Use this pipeline

To call this pipeline from your pipeline, use a step:

step "pipeline" "step_name" {
  pipeline = aws_thrifty.pipeline.detect_and_correct_emr_clusters_idle_30_mins  
}

Params

Name	Type	Required	Description	Default
database	connection.steampipe	Yes	Database connection string.	`connection.steampipe.default`
notifier	notifier	Yes	The name of the notifier to use for sending notification messages.	`notifier.default`
notification_level	string `info`, `verbose`, `error`	Yes	The verbosity level of notification messages to send. Valid options are 'verbose', 'info', 'error'.	`info`
approvers	list(notifier)	Yes	List of notifiers to be used for obtaining action/approval decisions.	`notifier.default`
default_action	string `notify`, `skip`, `delete_cluster`	Yes	The default action to use for the detected item, used if no input is provided.	`notify`
enabled_actions	list(string) `skip`, `delete_cluster`	Yes	The list of enabled actions to provide to approvers for selection.	`[ "skip", "delete_cluster" ]`

Outputs

This pipeline has no outputs.