How to use Python and FastAPI to Deploy Machine Learning Models on Heroku

In this article, I am going to talk about deploying machine learning models in production environments, using APIs.

Why is Machine Learning deployment so important?

Based on my experience as a data scientist, I realized that many Machine Learning models remain in the research stage, so many hours of training are wasted since these models never see the production environment. Very often these models die in the Jupyter Notebook in which they were created. This scenario is in contrast to the traditional software development process which has a long time of improvements.

Then, unlike conventional software development, the Machine Learning field has little time and still doesn’t have elaborate techniques or specific process schedules to develop a data science project which involves the creation of complex models. Due to this, since few models have passed the training and testing stage successfully, it is important to get out of the Jupyter Notebook the models which are passed all the testing process and deploy them in production environments, so the models can be used to solve the problems for which they were created.

Using Python for Machine Learning deployment

One of the most remarkable problems that every data scientist has to tackle during the development of Machine Learning models is how to carry out the deployment phase. I might mention a variety of challenges in machine learning deployment such as the programming language used by data scientists, which in many cases are not compatible with web environments, for example, the R language. Also, I can mention the computer power required to run some machine learning models like deep neural networks that can make the deployment phase hard.

However, here we have another challenge, this refers to the data scientist programming skills that are not the most appropriate to develop web applications, so building an API to deploy Machine Learning models can be a headache for more than one data scientist. Despite this, today some tools facilitate the deployment phase. For example, cloud platforms such Azure, Amazon Web Service, etc.

Meanwhile, over time programming languages have improved functionalities that facilitate the deployment of Machine Learning models. Python, one of the most popular programming languages used for the data scientist community, has libraries like Flask or FastAPI that make it easier to write code for web environments.

Flask is one of the most popular web frameworks available in Python, is minimalist, and allows us to build web apps easily and fast. Using this web framework you can build both web apps and APIs. On the other hand, we have FastAPI, which is a modern, web framework that allows you to build APIs using Python. The key FastAPI features are:

• Fast: Very high performance, on par with Node.js and Go, so it is one of the fastest Python frameworks available.
• Intuitive: Great editor support.
• Easy: Designed to be easy and learn.
• Robust: Get production-ready code with automatic interactive documentation.
• Standard-based: Based on (and fully compatible with) the open standards for APIs: OpenAPI and JSON Schema.

FastAPI has some valuable features which make this web framework ideal for deploying Machine Learning models through API. In addition, FastAPI generates interactive documentation as the API develops. This feature is very important since documentation allows developers to use the API and contribute to the project development. This feature is missing in Flask, so you will need to use other ways to create the API documentation when you are using it.

FastAPI

Now let’s see how to build a simple Hello World API using the FastAPI framework, all the necessary code to build your first API is shown below.

That’s all, with seven lines of code you can build a simple API that returns a “Hello World” message. In the first line we import the FastAPI class from fastapi, after that in line 3 I create our application creating an instantiate of the FastAPI class, which receives as an argument the API title, in this case, is “My Hello World API”.

Line 5 defines the path (also called route or endpoint), in FastAPI terms, this is called Path Operation and it is formed by three components. The path, the operation, and the function.

• The path, also called endpoint or route, refers to the last part of the URL starting at the first / so, in this https://example.com/items/foo the path would be items/foo in our specific case, the path would be /
• The operation refers to one of the HTTP methods, such as: POST GET PUT DELETE etc. In this case, I am defining the GET method using a Python decorator.
• The function is below the decorator, it will be called by FastAPI whenever it receives a request for the / URL using the GET operation.

To run our API we need to use uvicorn (in this specific case) to serve our API, the command is shown below.

uvicorn main:app

Here main is the file with the code and app refer to the FastAPI instance name. After running our server, we can see it in http://127.0.0.1:8000/docs our API.

API documentation.

Build a simple machine learning model.

For this example, I am going to use a popular dataset to build a machine learning model, I am referring to the iris dataset, since it has been used to test a wide variety of classifiers I am not going to give a lengthy description of how to build this model. So basically with scikit learn I can build a model that classifies the iris dataset instances, the code to build this model is shown below.

Running the code above, we have created a machine learning model in a pickle file called iris_classifier.pkl So now we already have an iris classifier.

Iris classifier API.

We are ready to build our API to deploy a simple machine learning model, first, let’s start creating an object to make handling the classifier easy. The code to create a Python class that wraps the functionalities that we are going to use in our API is shown below.

The code shown allows you to organize the model and handle it more easily. In essence, I created a Python class that wraps two main methods, the get_model() method is in charge of reading the pickle file and returns the model, meanwhile, the make_prediction()is in charge of making the prediction, this method receives a dictionary with the Iris plant features. Now, let's create our API.

The entire API is written with only 26 lines of code. In line 7 I create the application using the FastAPI class, which receives arguments for the title, version, and a short description of our API. Line 13 instantiate the Iris_Classifier object which receives as an argument the path to the pickle file where our model is stored. Line 17 creates a model object which serves as a layout to the Iris plant.

Finally, we created the endpoint, I used a POST operation, and the function below the decorator receives an argument called features, which is Iris type. So this function receives a feature object of type Iris with the Iris plant features. Using the classifier object created previously, I can predict the species of the plant and return it as a JSON response.

Now let’s run our server a make some predictions. To run this application we only use the following command:

uvicorn iris_classifier_api:app

Now we can see in our localhost something similar to this.

Deployment on Heroku.

Now let’s deploy our API in a production environment, to do this I am going to use the Heroku platform which allows deploying web apps for free, with some limitations. The first step is to create a Procfile file (without extensions) this file specifies the commands that will be executed by the app. We only need to write the next line in the Procfile file.

web:uvicorn iris_classifier_api:app --host=0.0.0.0 --port=${PORT:5000}

iris_classifier_api is the file that contains the API code, meanwhile, the app refers to the FastAPI instantiation. Now let’s go to the Heroku page. You need to have an account on this platform. Then, we can create a new app, as I am showing below.

When you press on create a new app, you will see something like this:

In this section, we can create a name for our API. In this post I am going to deploy the API using GitHub, so you need to have the code in a GitHub repository, then we can connect the GitHub repository with Heroku.

When we connect the repository with Heroku, the process only requires deploying the branch where our code is stored. In this example, it will be the master branch. If all goes well, to view our API, you only need to press the View button.

To use our API on the web browser we need to use the swagger interface, then be sure to use /docs at the end of the path, you will see something like this.

That’s all, now everybody can use the API to predict the Iris plant species according to the shape. This is an extremely simple demonstration of how to use FastAPI to build APIs and deploy machine learning models. APIs generally need to implement user authentication and store the data that is generated in some kind of database. For now, I hope this article allows you to start with FastAPI and deploy machine learning models in a production environment.

You can see the complete code of this blog in my GitHub repository.

References.

• FastAPI website
• Iris dataset from scikit-learn.

I am passionate about data science and like to explain how these concepts can be used to solve problems in a simple way. If you have any questions or just want to connect, you can find me on Linkedin or email me at manuelgilsitio@gmail.com

More content at plainenglish.io