cloudfoundry/bosh-gcscli: GCS for BOSH · GitHub

A Golang CLI for uploading, fetching and deleting content to/from Google Cloud Storage. This tool exists to work with the bosh-cli and director.

This is not an official Google Product.

go get github.com/cloudfoundry/bosh-gcscli

bosh-gcscli --help

bosh-gcscli -c config.json put <path/to/file> <remote-blob>

bosh-gcscli -c config.json get <remote-blob> <path/to/file>

bosh-gcscli -c config.json delete <remote-blob>

bosh-gcscli -c config.json exists <remote-blob>

If there is an encryption key present in the config, then an additional header is sent

bosh-gcscli -c config.json sign <remote-blob> <http action> <expiry>

Where:

• <http action> is GET, PUT, or DELETE
• <expiry> is a duration string less than 7 days (e.g. "6h")

The command line tool expects a JSON configuration file. Run bosh-gcscli --help for details.

• static: A service account key will be provided via the json_key field.
• none: No credentials are provided. The client is reading from a public bucket.
• <empty>: Application Default Credentials will be used if they exist (either through gcloud auth application-default login or a service account). If they don't exist the client will fall back to none behavior.

1. Ensure gcloud is installed and you have authenticated (gcloud auth login). These credentials will be used by the Makefile to create/destroy Google Cloud Storage buckets for testing.
2. Set the Google Cloud project: gcloud config set project <your project>
3. Generate a service account with the Storage Admin role for your project and set the contents as the environment variable GOOGLE_APPLICATION_CREDENTIALS, for example:

   export project_id=$(gcloud config get-value project)
   
   export service_account_name=bosh-gcscli-integration-tests
   export service_account_email=${service_account_name}@${project_id}.iam.gserviceaccount.com
   credentials_file=$(mktemp)
   
   gcloud config set project ${project_id}
   gcloud iam service-accounts create ${service_account_name} --display-name "Integration Test Access for bosh-gcscli"
   gcloud iam service-accounts keys create ${credentials_file} --iam-account ${service_account_email}
   gcloud project add-iam-policy-binding ${project_id} --member serviceAccount:${service_account_email} --role roles/storage.admin
   
   export GOOGLE_SERVICE_ACCOUNT="$(cat ${credentials_file})"
   export GOOGLE_APPLICATION_CREDENTIALS="$(cat ${credentials_file})"
   export LC_ALL=C # fix `tr` complaining about "illegal byte sequence" on OSX

4. Run the unit and fast integration tests: make test-fast-int
5. Clean up buckets: make clean-gcs

• A Makefile is provided that automates integration testing. Try make help to get started.
• gvt is used for vendoring.

For details on how to contribute to this project - including filing bug reports and contributing code changes - please see CONTRIBUTING.md.

This tool is licensed under Apache 2.0. Full license text is available in LICENSE.