GCP Mod for Flowpipe

GCP pipeline library for Flowpipe, enabling seamless integration of GCP services into your workflows.

Documentation

Pipelines →

Getting started

Requirements

Docker daemon must be installed and running. Please see Install Docker Engine for more information.

Installation

Download and install Flowpipe (https://flowpipe.io/downloads). Or use Brew:

brew tap turbot/tap
brew install flowpipe

Connections

By default, the following environment variables will be used for authentication:

GOOGLE_APPLICATION_CREDENTIALS

You can also create connection resources in configuration files:

vi ~/.flowpipe/config/gcp.fpc

connection "gcp" "gcp_token" {
  credentials = "path/to/credentials.json"
}

If no environment variables or configuration files are found, the mod will attempt to use Application Default Credentials if configured.

For more information on connections in Flowpipe, please see Managing Connections.

Usage

Initialize a mod:

mkdir my_mod
cd my_mod
flowpipe mod init

Install the GCP mod as a dependency:

flowpipe mod install github.com/turbot/flowpipe-mod-gcp

Use the dependency in a pipeline step:

vi my_pipeline.fp

pipeline "my_pipeline" {

  step "pipeline" "list_storage_buckets" {
    pipeline = gcp.pipeline.list_storage_buckets
    args = {
      project_id = "my-project"
    }
  }
}

Run the pipeline:

flowpipe pipeline run my_pipeline

Developing

Clone:

git clone https://github.com/turbot/flowpipe-mod-gcp.git
cd flowpipe-mod-gcp

List pipelines:

flowpipe pipeline list

Run a pipeline:

flowpipe pipeline run create_compute_instance --arg project_id=my-project --arg instance_name=i-1234567890abcdef0 --arg machine_type=n1-standard-1 --arg zone=us-central1-a --arg boot_disk_size="10"

To use a specific connection, specify the conn pipeline argument:

flowpipe pipeline run create_compute_instance --arg project_id=my-project --arg instance_name=i-1234567890abcdef0 --arg conn=connection.gcp.gcp_token --arg machine_type=n1-standard-1 --arg zone=us-central1-a --arg boot_disk_size="10"

Open Source & Contributing

This repository is published under the Apache 2.0 license. Please see our code of conduct. We look forward to collaborating with you!

Flowpipe is a product produced from this open source software, exclusively by Turbot HQ, Inc. It is distributed under our commercial terms. Others are allowed to make their own distribution of the software, but cannot use any of the Turbot trademarks, cloud services, etc. You can learn more in our Open Source FAQ.

Get Involved

Join #flowpipe on Slack →

Want to help but not sure where to start? Pick up one of the help wanted issues: