Software Development, Data Science, Python

Flask application with custom package deployed to GCP Artifact Registry and Cloud Run

Christopher Mortimer
Specialising in data
More posts by Christopher Mortimer.

Christopher Mortimer

18 Jul 2024•4 min read

Flask application with custom package deployed to GCP Artifact Registry and Cloud Run

What we are looking to do in this example is demonstrate how you might develop a Python based web application that uses modules from a custom Python package that is contained within the same mono repo as the source code for the application.

The source code (https://github.com/mortie23/ml/) is same as from my previous post about Python mono repos for Data Science with Poetry.

Some of the benefits are:

Module reuse across exploratory data analysis (EDA), and many applications
Changes can be tested across the complete code base
Changes and associated deployment can be orchestrated for both the custom modules and applications with one feature branch

Our custom Python package Fruitbowl

First we will start with the super useful custom package we have called Fruit bowl, which has a Fruit module and function that returns a random name of a fruit. Just for demonstration purposes.

Publish to a GCP Artifact Registry Python package repository

Credit goes to Rodrigo Agundez who wrote this blog post:

GCP Python packages via GCPS Artifact Registry

# Update poetry
poetry self update
# Add Google artifact registry keyring to poetry
poetry self add keyrings.google-artifactregistry-auth
# Check the python repository on GCP
gcloud artifacts print-settings python --project=prj-xyz-dev-fruit-0 --repository=rpo-xyz-dev-fruit-pyn-0 --location=australia-southeast1
# Configure a poetry repository to point to the python artifact registry
poetry config repositories.gcp https://australia-southeast1-python.pkg.dev/prj-xyz-dev-fruit-0/rpo-xyz-dev-fruit-pyn-0/
# publish the package to the repository
poetry publish --build --repository gcp

The publish step should look like this if successful.

Using python3.12 (3.12.3)
Building fruitbowl (0.1.0)
  - Building sdist
  - Built fruitbowl-0.1.0.tar.gz
  - Building wheel
  - Built fruitbowl-0.1.0-py3-none-any.whl

Publishing fruitbowl (0.1.0) to gcp
 - Uploading fruitbowl-0.1.0-py3-none-any.whl 100%
 - Uploading fruitbowl-0.1.0.tar.gz 100%

The Hello Fruit Flask application

Now for the simple example of a Flask application that uses this Fruit bowl function from the custom Python module.

Setup

Firstly let’s just test if we can run it locally in development mode. In this mode we could test changing code from either the module or the application and see the changes.

# Change to the root of the application
cd app/flask-hellofruit/
# Install application dependencies using poetry
poetry install

Running the application

poetry run flask run

 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on http://127.0.0.1:5000
Press CTRL+C to quit
127.0.0.1 - - [17/Jul/2024 21:03:36] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [17/Jul/2024 21:03:36] "GET /favicon.ico HTTP/1.1" 404 -

Hosting on GCP with Cloud Run

Hosting this on GCP has many options, but we are going to use Cloud Run to host this in a serverless way.

Build a Docker container image

We need the Docker build process to have access to the Artifact registry repository. For this we create a service account and associated key file that can be temporarily copied during the Docker build process.

# Create a service account
gcloud iam service-accounts create gsvc-xyz-dev-fruit-rpo --display-name "Artifact Registry Service Account"
# Give service account permissions to artifact registry in project
gcloud projects add-iam-policy-binding prj-xyz-dev-fruit-0 --member=serviceAccount:gsvc-xyz-dev-fruit-rpo@prj-xyz-dev-fruit-0.iam.gserviceaccount.com --role=roles/artifactregistry.reader
# Create application keys JSON file for service account
gcloud iam service-accounts keys create gsvc-xyz-dev-fruit-rpo.json --iam-account gsvc-xyz-dev-fruit-rpo@prj-xyz-dev-fruit-0.iam.gserviceaccount.com

Put the JSON keys file in this location on your developer client machine ~\.config\gcloud\gsvc-xyz-dev-fruit-rpo.json. In this case it is the home path on the WSL2 Windows client laptop. It will be used later in the hellofruit app Docker build step.

Now we can build the container image.

docker build . --build-arg SA_KEY_BASE64=`base64 -w0 ~/.config/gcloud/gsvc-xyz-dev-fruit-rpo.json` -t australia-southeast1-docker.pkg.dev/prj-xyz-dev-fruit-0/rpo-xyz-dev-fruit-dkr-0/app/hellofruit:0.1

Run the container locally.

docker run -p 5000:5000 australia-southeast1-docker.pkg.dev/prj-xyz-dev-fruit-0/rpo-xyz-dev-fruit-dkr-0/app/hellofruit:0.1

[2024-07-17 11:34:23 +0000] [1] [INFO] Starting gunicorn 22.0.0
[2024-07-17 11:34:23 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
[2024-07-17 11:34:23 +0000] [1] [INFO] Using worker: gthread
[2024-07-17 11:34:23 +0000] [7] [INFO] Booting worker with pid: 7
[2024-07-17 11:34:23 +0000] [8] [INFO] Booting worker with pid: 8

Once we have tested that it works locally we can push it to the Docker registry.

# First auth to the docker artifact registry
gcloud auth configure-docker australia-southeast1-docker.pkg.dev
# Push the local image to the artifact registry
docker push australia-southeast1-docker.pkg.dev/prj-xyz-dev-fruit-0/rpo-xyz-dev-fruit-dkr-0/app/hellofruit:0.1

Cloud Run the service

Using the GCP web console, it is simple to select the container from the Artifact Registry and set the Container port to 5000. The remaining settings are left as the default settings.