Dataflow

A collection of 2 posts

Converting table schemas from any database to BigQuery for Dataflow cover image
Dataflow,  Data Engineering,  GCP,  BigQuery

Converting table schemas from any database to BigQuery for Dataflow

Why do this? When using Dataflow Google-provided templates for any RDBMS to BigQuery (such as PostgreSQL to BigQuery) the target table in…

KMS encrypted credentials with Dataflow on GCP cover image
Dataflow,  Data Engineering,  GCP,  KMS

KMS encrypted credentials with Dataflow on GCP

Encrypting credentials Google KMS Cloud Key Management Service (Cloud KMS) lets you create and manage encryption keys for use in compatible…