Model scoring API as a Docker container
Use Docker to host a Python FastAPI service
In my preview blog post (Model scoring API) I went through the technical basics of a model scoring API. I finished it off at the step of having it running on you local client machine (laptop in my case). However to host such a service we need a way to deploy this somewhere. We have lots of options. The first option that comes to mind would be to:
- spin up a cloud Linux virtual machine
- go through the process of installing Python and the packages required
- run the uvicorn service in the background
However, let’s go through the process of hosting it using Docker. The benefits are the portability of the resulting service.
Install Docker on your local computer
I’m using Windows now for this process (if you are you may need to install the Docker for Windows application). The default is for it to build Windows containers (which I wasn’t aware of at the time). This results in a Docker image that needs to be run on Windows. So if you are planning on hosting it on a Windows server, then go with this. Otherwise, like I did in the end, you will need to switch to Linux containers (assuming you have WSL2 on your machine).
Build the image
You’ll need to start by cloning the associated repo https://github.com/mortie23/fastapi-linear-model-demo.
git clone https://github.com/mortie23/fastapi-linear-model-demo.gitThen lets build the Docker image.
docker build -t mortimerxyz/linear:1.0 .Docker files define layers. In this Docker file we start from a base image (which has Python3.9 on it) and then go through steps to build up the image by copying files we have in the repo. We also need to install the Python packages from the requirements.txt files within the image and then define the command that will run the service when the container is run.
FROM python:3.9
WORKDIR /
COPY ./requirements.txt ./
RUN pip install --no-cache-dir --upgrade -r requirements.txt
COPY ./fastapiApp /app
CMD ["uvicorn", "app.model:app", "--host", "0.0.0.0", "--port", "80"]Windows using PowerShell
Since I first run this on Windows, I have the logs here. The Windows build took significantly longer than the Linux one below.
Sending build context to Docker daemon 3.322MB
Step 1/6 : FROM python:3.9
---> 05cae63731aa
Step 2/6 : WORKDIR /
---> Running in 1e478e292609
Removing intermediate container 1e478e292609
---> be030c7a42e0
Step 3/6 : COPY ./requirements.txt ./
---> 2487f2c14aec
Step 4/6 : RUN pip install --no-cache-dir --upgrade -r requirements.txt
---> Running in f8b71525170b
Collecting fastapi[all]
Downloading fastapi-0.79.0-py3-none-any.whl (54 kB)
---------------------------------------- 54.6/54.6 KB 1.4 MB/s eta 0:00:00
Collecting uvicorn
Downloading uvicorn-0.18.2-py3-none-any.whl (57 kB)
---------------------------------------- 57.0/57.0 KB ? eta 0:00:00
[...]
Successfully installed [...]
---> b901eee1947f
Step 5/6 : COPY ./fastapiApp /app
---> b65d2074826b
Step 6/6 : CMD ["uvicorn", "app.model:app", "--host", "0.0.0.0", "--port", "80"]
---> Running in 8f20e0e45191
Removing intermediate container 8f20e0e45191
---> 24ba8c92979c
Successfully built 24ba8c92979c
Successfully tagged mortimerxyz/linear:1.0
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix themWSL on bash
After switching Docker to Linux containers I ran the build again from WSL bash shell. I’m going to push this one so I can then pull it on my Chromebook (Debian) and test using it.
[+] Building 74.9s (10/10) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 430B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/python:3.9 10.3s
=> [auth] library/python:pull token for registry-1.docker.io 0.0s
=> [1/5] FROM docker.io/library/python:3.9@sha256:[...] 51.3s
=> => resolve docker.io/library/python:3.9@sha256:[...]f 0.0s
=> => sha256: [...] 2.22kB / 2.22kB 0.0s
=> [internal] load build context 0.1s
=> => transferring context: 832B 0.1s
=> [2/5] COPY ./requirements.txt ./ 0.6s
=> [3/5] RUN pip install --no-cache-dir --upgrade -r requirements.txt 11.8s
=> [4/5] COPY ./fastapiApp /app 0.0s
=> exporting to image 0.6s
=> => exporting layers 0.6s
=> => writing image sha256:[...] 0.0s
=> => naming to docker.io/mortimerxyz/linear:1.0 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix themTest running it
Once it is built we need to test that it runs.
docker run -p 80:80 mortimerxyz/linear:1.0You should expect to see something like this in the terminal if you are successful.
INFO: Started server process [1324]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:80 (Press CTRL+C to quit)
INFO: 192.168.0.19:23413 - "GET /weight/150 HTTP/1.1" 200 OKYou should be able to call the service from your browser or programmatically now (on port 80).
Publish it
To follow this process, you don’t need this step, but if you do, you will want to sign up for Docker Hub.
- Sign up to Docker Hub
- Create an access token
docker login -u mortimerxyzdocker push mortimerxyz/linear:1.0The push refers to repository [docker.io/mortimerxyz/linear]
849958a0eb7c: Preparing
a7b3e42a4b41: Pushed
c00d727af360: Pushed
27a391641a6d: Pushed
6bf06ec166f0: Pushed
b2809f3ef892: Pushed
1a31622a1cf9: Pushed
b588d5e23fdf: Pushed
f22fec08a08a: Pushed
69c9ca396f64: Pushed
b55de1dfa670: Pushed
91896eadc954: Pushed
8b5e3ef66306: Pushed
63a0eaed8845: Pushed
e673d0986f90: Pushed
5e610cf6f254: Skipped foreign layer
d6fdd6832d95: Skipped foreign layer
1.0: digest: sha256:[...] size: 4232
Test it on another platform
Now we see the benefit of all of this (hopefully). To get the whole thing running on another platform (another local machine or even a cloud VM) we can just to the following:
docker pull mortimerxyz/liner:1.0
# run with the IMAGE ID or the REPOSITORY:TAG
docker run -p 80:80 mortimerxyz/liner:1.0
Success and testing calling it is all good!
curl localhost:80/weight/200{ "weight": 90.275 }