Multi-target regression with TensorFlow.

This problem was taken from the specialization course TensorFlow: Advanced Techniques on Coursera.

Table of Content

1. Introduction.
2. Dataset Overview.
3. Exploratory Data Analysis.
4. Data preparation.
5. Building the model.
6. Train the model.
7. Predictions.
8. Conclusions.

Introduction

Many times, it is necessary to predict the behavior of several variables using the same features. One possible approach involves creating as many models as variables we need to predict. However, this approach is not the most efficient.

Using the functional TensorFlow API we are able to create models with several inputs and outputs, at the same time, so we can create one model that is able to predict the behavior of more than one variable using the same dataset.

Dataset Overview.

Demonstrate the ability and flexibility of TensorFlow to build models with more than one output. We are going to use one dataset from UCI Machine Learning Repository. You can download the Energy Efficient Dataset using the following link. You can use pandas to download and open the data set which is in excel format or load from your local machine.

Let’s see what the dataset is about.

Exploratory Data Analysis.

We can see that our dataset is formed by eight feature labels, in other words, we have eight independent variables or predictor variables and two dependent variables, our goal is to model the behavior of these variables using these features. That means we are dealing with a regression problem.

From the website of the dataset we can see what our data means. The dataset is about how different building shapes (building parameters) are related to the heating load and cooling load requirements they have. The variables in the dataset are specifically:

• X1: Relative Compactness.
• X2: Surface Area.
• X3: Wall Area.
• X4: Roof Area.
• X5: Overall Height.
• X6: Orientation.
• X7: Glazing Area.
• X8: Glazing Area Distribution.
• Y1: Heating Load.
• Y2: Cooling Load.

Using the describe() method from pandas we can observe some statistics of our original data.

Descriptive Statistics

We can see that our data has features with different scales. The features X2, X3, X4 have high values compared with the rest of the features. Let’s see how the data distribution looks.

Data Box plot

Data preparation

In this dataset we do not have missing values, the main issue is the feature scale, so to improve the learning process, let’s normalize the dataset. First we split our data into training and testing datasets and store the target values in different variables. Once our data has been split we can normalize the features by applying some operations in the data sets.

Building the model.

Now that we have our dataset prepared, let’s create the architecture of our neural network, we can use the functional API of TensorFlow which gives us more flexibility to create the model. The code to create our neural network is shown below.

I used a simple MLP architecture for the main part of the neural network, this part has two layers of 128 neurons each one, at this point I split the model into two paths one for each output, the first output called y2_output has another dense layer with 64 neurons before to the last layer which has only one neuron and uses a linear function as activation function.

The second path only is formed by the output layer called y1_output which has only one neuron and uses a linear function as activation function. We always need to use this kind of layer to solve regression problems. Also, we need to use a specific metric design for regression problems, in this specific case I am using the Root Mean Squared Error as a metrics and loss function. This metric is common for regression problems. The final architecture of the model is shown below.

Neural Network Architecture

Train the model.

The training process is established for 2000 epochs, and we use the test data as validation data, it is important to highlight that during the training process the model will not use the validation data. The neural network weights will be calculated only using the train data set.

The RMSE obtained during the training process can be seen in the following figure.

We can see the RMSE metric start with fluctuations at the beginning of the training process then it tends to decrease and stabilize through the epochs, that is probably because our target values were not normalized. Finally, we can see the predictions on our test data set.

Predictions.

Let’s see how our model behavior is.

We can see that our model performance is quite good, of course this was an easy dataset, without missing values or other issues generally present in most datasets.

Conclusions.

In this post we have seen how to use the TensorFlow functional API to implement a neural network with multiple outputs to solve regression problems. following this approach with can build more complex architectures using TensorFlow.

The notebook used to develop this project can be found in my GitHub repository.

Thanks for reading my entry. If you have any questions or just want to connect, you can find me on Linkedin or email me at manuelgilsitio@gmail.com