Cloud Infrastructure,  Data Science

Model scoring API as a Docker container

Model scoring API as a Docker container

Use Docker to host a Python FastAPI service

In my preview blog post (Model scoring API) I went through the technical basics of a model scoring API. I finished it off at the step of having it running on you local client machine (laptop in my case). However to host such a service we need a way to deploy this somewhere. We have lots of options. The first option that comes to mind would be to:

  1. spin up a cloud Linux virtual machine
  2. go through the process of installing Python and the packages required
  3. run the uvicorn service in the background

However, let’s go through the process of hosting it using Docker. The benefits are the portability of the resulting service.

Install Docker on your local computer

I’m using Windows now for this process (if you are you may need to install the Docker for Windows application). The default is for it to build Windows containers (which I wasn’t aware of at the time). This results in a Docker image that needs to be run on Windows. So if you are planning on hosting it on a Windows server, then go with this. Otherwise, like I did in the end, you will need to switch to Linux containers (assuming you have WSL2 on your machine).

docker wsl

Build the image

You’ll need to start by cloning the associated repo

git clone

Then lets build the Docker image.

docker build -t mortimerxyz/linear:1.0 .

Docker files define layers. In this Docker file we start from a base image (which has Python3.9 on it) and then go through steps to build up the image by copying files we have in the repo. We also need to install the Python packages from the requirements.txt files within the image and then define the command that will run the service when the container is run.

FROM python:3.9


COPY ./requirements.txt ./

RUN pip install --no-cache-dir --upgrade -r requirements.txt

COPY ./fastapiApp /app

CMD ["uvicorn", "app.model:app", "--host", "", "--port", "80"]

Windows using PowerShell

Since I first run this on Windows, I have the logs here. The Windows build took significantly longer than the Linux one below.

Sending build context to Docker daemon  3.322MB
Step 1/6 : FROM python:3.9
 ---> 05cae63731aa
Step 2/6 : WORKDIR /
 ---> Running in 1e478e292609
Removing intermediate container 1e478e292609
 ---> be030c7a42e0
Step 3/6 : COPY ./requirements.txt ./
 ---> 2487f2c14aec
Step 4/6 : RUN pip install --no-cache-dir --upgrade -r requirements.txt
 ---> Running in f8b71525170b
Collecting fastapi[all]
  Downloading fastapi-0.79.0-py3-none-any.whl (54 kB)
     ---------------------------------------- 54.6/54.6 KB 1.4 MB/s eta 0:00:00
Collecting uvicorn
  Downloading uvicorn-0.18.2-py3-none-any.whl (57 kB)
     ---------------------------------------- 57.0/57.0 KB ? eta 0:00:00
Successfully installed [...]

 ---> b901eee1947f
Step 5/6 : COPY ./fastapiApp /app
 ---> b65d2074826b
Step 6/6 : CMD ["uvicorn", "app.model:app", "--host", "", "--port", "80"]
 ---> Running in 8f20e0e45191
Removing intermediate container 8f20e0e45191
 ---> 24ba8c92979c
Successfully built 24ba8c92979c
Successfully tagged mortimerxyz/linear:1.0

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

WSL on bash

After switching Docker to Linux containers I ran the build again from WSL bash shell. I’m going to push this one so I can then pull it on my Chromebook (Debian) and test using it.

[+] Building 74.9s (10/10) FINISHED
 => [internal] load build definition from Dockerfile                     0.1s
 => => transferring dockerfile: 430B                                     0.0s
 => [internal] load .dockerignore                                        0.1s
 => => transferring context: 2B                                          0.0s
 => [internal] load metadata for           10.3s
 => [auth] library/python:pull token for            0.0s
 => [1/5] FROM[...]                51.3s
 => => resolve[...]f                0.0s
 => => sha256: [...] 2.22kB / 2.22kB                                     0.0s
 => [internal] load build context                                        0.1s
 => => transferring context: 832B                                        0.1s
 => [2/5] COPY ./requirements.txt ./                                     0.6s
 => [3/5] RUN pip install --no-cache-dir --upgrade -r requirements.txt  11.8s
 => [4/5] COPY ./fastapiApp /app                                         0.0s
 => exporting to image                                                   0.6s
 => => exporting layers                                                  0.6s
 => => writing image sha256:[...]                                        0.0s
 => => naming to                        0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

Test running it

Once it is built we need to test that it runs.

docker run -p 80:80 mortimerxyz/linear:1.0

You should expect to see something like this in the terminal if you are successful.

INFO:     Started server process [1324]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on (Press CTRL+C to quit)
INFO: - "GET /weight/150 HTTP/1.1" 200 OK

You should be able to call the service from your browser or programmatically now (on port 80).

mdlapi height weight api

Publish it

To follow this process, you don’t need this step, but if you do, you will want to sign up for Docker Hub.

  1. Sign up to Docker Hub
  2. Create an access token

docker access token

docker login -u mortimerxyz
docker push mortimerxyz/linear:1.0
The push refers to repository []
849958a0eb7c: Preparing
a7b3e42a4b41: Pushed
c00d727af360: Pushed
27a391641a6d: Pushed
6bf06ec166f0: Pushed
b2809f3ef892: Pushed
1a31622a1cf9: Pushed
b588d5e23fdf: Pushed
f22fec08a08a: Pushed
69c9ca396f64: Pushed
b55de1dfa670: Pushed
91896eadc954: Pushed
8b5e3ef66306: Pushed
63a0eaed8845: Pushed
e673d0986f90: Pushed
5e610cf6f254: Skipped foreign layer
d6fdd6832d95: Skipped foreign layer
1.0: digest: sha256:[...] size: 4232

docker published linux

Test it on another platform

Now we see the benefit of all of this (hopefully). To get the whole thing running on another platform (another local machine or even a cloud VM) we can just to the following:

docker pull mortimerxyz/liner:1.0
# run with the IMAGE ID or the REPOSITORY:TAG
docker run -p 80:80 mortimerxyz/liner:1.0

docker run chromeos

Success and testing calling it is all good!

curl localhost:80/weight/200
{ "weight": 90.275 }