Pipeline: Detect & correct S3 buckets without lifecycle policy

Overview

S3 Buckets without a lifecycle policy will not move objects between storage layers or expire objects, causing them to remain in their initial tier perpetually, this is inefficient and can be costly.

This pipeline detects S3 buckets which do not have a lifecycle policy attached and then either sends a notification or attempts to perform a predefined corrective action.

Getting Started

This control will work out-of-the-box with some sensible defaults (configurable via variables).

Note: You should review the variable s3_buckets_without_lifecycle_policy_default_lifecycle_configuration to ensure this meets your requirements prior to using the apply_lifecycle_configuration action.

You should be able to simply run the following command in your terminal:

flowpipe pipeline run detect_and_correct_s3_buckets_without_lifecycle_policy

By default, Flowpipe runs in wizard mode and prompts directly in the terminal for a decision on the action(s) to take for each detected resource.

However, you can run Flowpipe in server mode with external integrations, allowing it to prompt for input via http, slack, teams, etc.

Alternatively, you can choose to configure and run in other modes:

Notify: Provides detections without taking any corrective action.
Automatic: Performs corrective actions automatically without user intervention.

Run the pipeline

To run this pipeline from your terminal:

flowpipe pipeline run aws_thrifty.pipeline.detect_and_correct_s3_buckets_without_lifecycle_policy

Use this pipeline

To call this pipeline from your pipeline, use a step:

step "pipeline" "step_name" {
  pipeline = aws_thrifty.pipeline.detect_and_correct_s3_buckets_without_lifecycle_policy  
}

Params

Name	Type	Required	Description	Default
database	connection.steampipe	Yes	Database connection string.	`connection.steampipe.default`
lifecycle_configuration	string	Yes	Lifecycle configuration to apply to the S3 bucket, if 'apply' is the chosen response.	`{ "Rules": [ { "ID": "Transition to STANDARD_IA after 90 days", "Status": "Enabled", "Filter": {}, "Transitions": [ { "Days": 90, "StorageClass": "STANDARD_IA" } ] }, { "ID": "Transition to GLACIER after 180 days", "Status": "Enabled", "Filter": {}, "Transitions": [ { "Days": 180, "StorageClass": "GLACIER" } ] }, { "ID": "Transition to DEEP_ARCHIVE after 365 days", "Status": "Enabled", "Filter": {}, "Transitions": [ { "Days": 365, "StorageClass": "DEEP_ARCHIVE" } ] } ] }`
notifier	notifier	Yes	The name of the notifier to use for sending notification messages.	`notifier.default`
notification_level	string `info`, `verbose`, `error`	Yes	The verbosity level of notification messages to send. Valid options are 'verbose', 'info', 'error'.	`info`
approvers	list(notifier)	Yes	List of notifiers to be used for obtaining action/approval decisions.	`notifier.default`
default_action	string `notify`, `skip`, `apply_lifecycle_configuration`	Yes	The default action to use for the detected item, used if no input is provided.	`notify`
enabled_actions	list(string) `skip`, `apply_lifecycle_configuration`	Yes	The list of enabled actions to provide to approvers for selection.	`[ "skip", "apply_lifecycle_configuration" ]`

Outputs

This pipeline has no outputs.