Detect & correct DynamoDB table with stale data

Overview

DynamoDB table data can be quite costly to retain, it is also likely that after a certain point in time they're no longer required and should be cleaned up to prevent further costs.

This query trigger detects unused health checks and then either sends a notification or attempts to perform a predefined corrective action.

Getting Started

By default, this trigger is disabled, however it can be configured by setting the below variables

dynamodb_tables_with_stale_data_trigger_enabled should be set to true as the default is false.
dynamodb_tables_with_stale_data_trigger_schedule should be set to your desired running schedule
dynamodb_tables_with_stale_data_default_action should be set to your desired action (i.e. "notify" for notifications or "delete_table" to delete the table).

Then starting the server:

flowpipe server

or if you've set the variables in a .fpvars file:

flowpipe server --var-file=/path/to/your.fpvars

Pipeline

Correct DynamoDB table with stale data

Query

select
  concat(name, ' [', region, '/', account_id, ']') as title,
  name,
  region,
  _ctx ->> 'connection_name' as cred
from
  aws_dynamodb_table
where
  (current_timestamp - (90 :: int || ' days') :: interval) > (latest_stream_label :: timestamp)

Schedule

15m