Keras Functional API for Multiple Inputs and Outputs and Accessing Model Layers

Github Link

https://github.com/Natan-Asrat/tensorflow_keras_functional_api__multiple_inputs__accessing_layers__freezing_layers

Contact

The Setup

Introduction

In this project, i explore the keras functional api by accomplishing tasks concerned with customizing models:

Multiple Inputs and Outputs.
Accessing Model Layers.

Libraries Used

TensorFlow
Numpy
Pandas
scikit-learn

Imports

import tensorflow as tf
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from tensorflow.keras import Input, layers
from tensorflow.keras.models import load_model, Model, Sequential
from tensorflow.keras.applications.vgg19 import preprocess_input
from tensorflow.keras.preprocessing import image

Run this if you are on Jupyter Notebook:

%matplotlib inline

Dataset

pd_dat = pd.read_csv('data/diagnosis.csv')
dataset = pd_dat.values

Build train and test data splits:

X_train, X_test, Y_train, Y_test = train_test_split(dataset[:,:6], dataset[:,6:], test_size=0.33)

Assign training and testing inputs/outputs:

temp_train, nocc_train, lumbp_train, up_train, mict_train, bis_train = np.transpose(X_train)
temp_test, nocc_test, lumbp_test, up_test, mict_test, bis_test = np.transpose(X_test)

inflam_train, nephr_train = Y_train[:, 0], Y_train[:, 1]
inflam_test, nephr_test = Y_test[:, 0], Y_test[:, 1]

The Code

Build the Model

Build the input layers:

shape_inputs = (1,)

temperature = Input(shape=shape_inputs, name="temp")
nausea_occurence = Input(shape=shape_inputs, name="nocc")
lumbar_pain = Input(shape=shape_inputs, name="lump")
urine_pushing = Input(shape=shape_inputs, name="up")
micturition_pains = Input(shape=shape_inputs, name="mict")
bis = Input(shape=shape_inputs, name="bis")

Create a list of all the inputs:

list_inputs = [temperature, nausea_occurence, lumbar_pain, urine_pushing, 
               micturition_pains, bis]

Merge all input features into a single large vector:

x = layers.concatenate(list_inputs)

Use a logistic regression classifier for disease prediction:

inflammation_pred = layers.Dense(1, activation='sigmoid', name='inflam')(x)
nephritis_pred = layers.Dense(1, activation='sigmoid', name='nephr')(x)

Create a list of all the outputs:

list_outputs = [inflammation_pred, nephritis_pred]

Create the model object:

model = tf.keras.Model(inputs=list_inputs, outputs = list_outputs)

Plot the Model

tf.keras.utils.plot_model(model, 'multi_input_output_model.png', show_shapes=True)

Compile the Model

model.compile(optimizer=tf.keras.optimizers.RMSprop(1e-3),
             loss={'inflam': 'binary_crossentropy', 
                   'nephr': 'binary_crossentropy'},
              metrics=['acc'],
              loss_weights=[1., 0.2]
             )

Fit the Model

Define training inputs and outputs:

inputs_train = {'temp': temp_train, 'nocc': nocc_train, 'lumbp': lumbp_train,
                'up': up_train, 'mict': mict_train, 'bis': bis_train}

outputs_train = {'inflam': inflam_train, 'nephr': nephr_train}

Train the model:

history = model.fit(inputs_train, outputs_train,
                   epochs=1000,
                   batch_size=120,
                   verbose=False)

Accessing Model Layers

Load the Pre-Trained Model

Load the VGG19 model:

vgg_model = load_model('models/Vgg19.h5')

Get the inputs, layers and display the summary:

vgg_input = vgg_model.input
vgg_layers = vgg_model.layers

Build a model that returns the layer outputs:

layer_outputs = [layer.output for layer in vgg_layers]
features = Model(inputs= vgg_input, outputs = layer_outputs)

To plot the model, use:

tf.keras.utils.plot_model(features, 'vgg199.png', show_shapes=True)

Load the Image

Preprocess the image:

img_path = 'data/cool_cat.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)

Extract the features:

extracted_features = features(x)

Visualize Features from Different Layers

Visualise the input channels:

f1 = extracted_features[0]
print('f1.shape: ', f1.shape)
imgs = f1[0,:,:]
plt.figure(figsize=(15,15))
for n in range(3):
    ax = plt.subplot(1,3,n+1)
    plt.imshow(imgs[:,:,n])
    plt.axis('off')
    
plt.subplots_adjust(wspace=0.01, hspace=0.01)

Visualise some features in the first hidden layer:

f2 = extracted_features[1]
print('f1.shape: ', f2.shape)
imgs = f2[0,:,:]
plt.figure(figsize=(15,15))
for n in range(64):
    ax = plt.subplot(8,8,n+1)
    plt.imshow(imgs[:,:,n])
    plt.axis('off')
    
plt.subplots_adjust(wspace=0.01, hspace=0.01)

Build a model to extract features by layer name:

extracted_features_block3_pool = Model(inputs = features.input, outputs= features.get_layer('block3_pool').output)
block3_pool_features =extracted_features_block3_pool.predict(x)

Visualise some features from the extracted layer output:

imgs = block3_pool_features[0,:,:]
plt.figure(figsize=(15,15))
for n in range(64):
    ax = plt.subplot(8, 8, n+1)
    plt.imshow(imgs[:, :, n])
    plt.axis('off')
plt.subplots_adjust(wspace=0.01, hspace=0.01)

Natville