Convolutional Neural Networks in Keras (Cat vs. Dog example)

In this post I’ll demonstrate how to use convolutional neural networks to classify between a dog and a cat. In particular, we will look at

• how to read images
• how to design a simple convolutional neural network in Keras
• how to train and evaluate the model

We will use Keras and Tensorflow to make a deep neural network model.

First head over to https://www.kaggle.com/c/dogs-vs-cats/data. There you can download the training dataset called train.zip. You don’t need test1.zip as it is only for making predictions and submitting the results to Kaggle. We will create our own training and testing set.

First extract the archive. There are 25,000 images of cats and dogs. One thing we immediately notice is that the images have different sizes. Also, images have different contrast and brightness.

Code

First we’ll import the necessary libraries

import matplotlib.pyplot as plt
import keras
import numpy as np
import glob
import os
from PIL import Image
import tqdm

Now we will define some constants. IMG_DIR is the directory where our images are. IM_WIDTH and IM_HEIGHT are the width and height of images after we do some pre-processing.

IMG_DIR = "./train/"
IM_WIDTH = 128
IM_HEIGHT = 128

We will need to read the images and make corresponding labels. So if an image is a dog then its label will be 1 otherwise it will be 0.

We loop through all jpeg files in the directory, read them and resize so that every image is of same size. We will convert this to a numpy array and add it to images list. For labels we will check the file name and determine the label for that image. After we are done will all images, we will return images and labels as numpy arrays. Note that reading all the files at once like this will use a lot of memory! If you have memory issues, create a subset of the data (make sure it contains equal number of images for both cat and dog) and read them instead of the full dataset.

def read_images(directory, resize_to=(128, 128)):
    """
    Reads images and labels from the given directory
    :param directory directory from which to read the files
    :param resize_to a tuple of width, height to resize the images
    : returns a tuple of list of images and labels
    """
    files = glob.glob(directory + "*.jpg")
    images = []
    labels = []
    for f in tqdm.tqdm_notebook(files):
        im = Image.open(f)
        im = im.resize(resize_to)
        im = np.array(im) / 255.0
        im = im.astype("float32")
        images.append(im)
       
        label = 1 if "dog" in f.lower() else 0
        labels.append(label)
       
    return np.array(images), np.array(labels)
 
X, y = read_images(directory=IMG_DIR, resize_to=(IM_WIDTH, IM_HEIGHT))
# make sure we have 25000 images if we are reading the full data set.
# Change the number accordingly if you have created a subset
assert len(X) == len(y) == 25000

Now that we have read all the images, lets split it into training and testing set. We will use only the training set for training the model and testing set for evaluating the model. This is done to evaluate how the model will perform on unseen data. There is a function built into sklearn library called train_test_split which we will use to split our data.

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test =
train_test_split(X, y, test_size=0.3)
# remove X and y since we don't need them anymore
# otherwise it will just use the memory
del X
del y

After splitting, we have 17,500 images for training and 7,500 for testing.

X_train.shape, X_test.shape
> ((17500, 128, 128, 3), (7500, 128, 128, 3))

Lets check some images. Let’s define a couple of helper functions to visualize images in a grid and also to convert numeric label to string.

def plot_images(images, labels):
    n_cols = min(5, len(images))
    n_rows = len(images) // n_cols
    fig = plt.figure(figsize=(8, 8))
 
    for i in range(n_rows * n_cols):
        sp = fig.add_subplot(n_rows, n_cols, i+1)
        plt.axis("off")
        plt.imshow(images[i])
        sp.set_title(labels[i])
    plt.show()
   
def humanize_labels(labels):
    """
    Converts numeric labels to human friendly string labels
    :param labels numpy array of int
    :returns numpy array of human friendly labels
    """
    return np.where(labels == 1, "dog", "cat")
 
plot_images(X_train[:20], humanize_labels(y_train[:20]))

IMAGE HERE

Now we can build our model.
There are two things to note:

1. We are working with images
2. We need to classify from two categories (dog or cat) which is called binary classification

When working with images, we use convolutional neural networks. We will make a simple convolutional neural network with Keras using a functional API.

from keras.layers import Input, Dense, Conv2D, BatchNormalization, Activation, Flatten, MaxPool2D
from keras.models import Model
 
image_input = Input(shape=(IM_HEIGHT, IM_WIDTH, 3))
x = Conv2D(filters=32, kernel_size=7)(image_input)
x = Activation("relu")(x)
x = BatchNormalization()(x)
x = MaxPool2D()(x)
 
x = Conv2D(filters=64, kernel_size=3)(x)
x = Activation("relu")(x)
x = BatchNormalization()(x)
x = MaxPool2D()(x)
 
x = Conv2D(filters=128, kernel_size=3)(x)
x = Activation("relu")(x)
x = BatchNormalization()(x)
x = MaxPool2D()(x)
 
x = Flatten()(x)
x = Dense(units=64)(x)
x = Activation("relu")(x)
x = BatchNormalization()(x)
x = Dense(units=1)(x)
x = Activation("sigmoid")(x)
 
model = Model(inputs=image_input, outputs=x)
model.compile(optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"])
model.summary()

Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         (None, 128, 128, 3)       0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 122, 122, 32)      4736      
_________________________________________________________________
activation_11 (Activation)   (None, 122, 122, 32)      0         
_________________________________________________________________
batch_normalization_9 (Batch (None, 122, 122, 32)      128       
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 61, 61, 32)        0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 59, 59, 64)        18496     
_________________________________________________________________
activation_12 (Activation)   (None, 59, 59, 64)        0         
_________________________________________________________________
batch_normalization_10 (Batc (None, 59, 59, 64)        256       
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 29, 29, 64)        0         
_________________________________________________________________
conv2d_9 (Conv2D)            (None, 27, 27, 128)       73856     
_________________________________________________________________
activation_13 (Activation)   (None, 27, 27, 128)       0         
_________________________________________________________________
batch_normalization_11 (Batc (None, 27, 27, 128)       512       
_________________________________________________________________
max_pooling2d_9 (MaxPooling2 (None, 13, 13, 128)       0         
_________________________________________________________________
flatten_3 (Flatten)          (None, 21632)             0         
_________________________________________________________________
dense_5 (Dense)              (None, 64)                1384512   
_________________________________________________________________
activation_14 (Activation)   (None, 64)                0         
_________________________________________________________________
batch_normalization_12 (Batc (None, 64)                256       
_________________________________________________________________
dense_6 (Dense)              (None, 1)                 65        
_________________________________________________________________
activation_15 (Activation)   (None, 1)                 0         
=================================================================
Total params: 1,482,817
Trainable params: 1,482,241
Non-trainable params: 576

First we need to define an input layer. The shape of input is image height by width by channels. This is the format when you are using tensorflow backend. If you are using Theano then the shape should be channels by height by width. In our case the shape would be (IM_HEIGHT, IM_WIDTH, 3). “3” is the number of color channels since we are working with RGB images.

Next we define a convolutional layer with 64 filters with kernel size 3. We then feed the output of the image_input to this layer. Next we define a RELU activation layer. RELU is commonly used activation function because it allows the model to converge faster.
Then we define a batch normalization layer. This layer normalizes the output of previous activation layer by subtracting the batch mean by batch standard deviation. This improves stability of the neural network. Next we add max-pooling layer. This combination of convolutional layer, activation and batch normalization is usually called convolution block.
We repeat the convolution block 2 times with different filters. We can repeat this as many times as we want allowing us to make deep neural networks. But notice that we are changing the number of filters after each convolution block. Also, in the model summary we can see that after each convlolution the spatial dimension (width and height) is reduced. Similarly max pooling significantly reduces the spatial dimension.

To understand the concepts of convolution layers, pooling, normalizing etc. I highly recommend you read from http://cs231n.github.io/convolutional-networks/ . It has all the necessary details about how convolution works, what parameters should we use and such.

After we decide that we have enough convolution blocks, we need fully connected layer to make the final classification. Fully connected layer is known as Dense layer in Keras. A Dense layer expects input in a form of 1 dimensional array. So we need to flatten the 3 dimensional data to 1 D. In Keras we can do it by using a layer called Flatten. Next we add a dense layer with 64 units or neurons. Like before we will use RELU activation and batch normalization. Finally we add another Dense layer that will actually produce the output that we want. Since we are doing a binary classification we only need one neuron that will produce a value from 0 to 1. For this we need to use sigmoid activation since the output from sigmoid is always between 0 and 1.

Now we need to define a model. In Keras, we can do this by instantiating a Model object. We need to specify what are the inputs and outputs of the model. Then we compile the model. Here we should specify the optimizer to use. There are many choices for which optimization algorithm to use like SGD, Adam, RMSProp etc. We will use “adam” as the optimizer. Next we need to define loss. Since this is a binary classification, we need to use binary_crossentropy. Next we also specify what metrics we would like to see during training. Since we are doing classification, we are interested in accuracy. So we specify “accuracy”.

You can look into Keras documentation for details about different optimizers, loss, metrics.

Now let’s train the model for 3 epochs with batch size of 64. Out of 17,500 training images we will take 64 images at a time and give it to the model to train, then we again take next 64 images and give it. This is called an iteration. Once we give all of 17,500 images to the model, we consider 1epoch to be finished. If you increase the batch size then you’ll need more memory in the GPU. When building your own models, you will have to experiment with different values and see which performs better.

model.fit(X_train, y_train, batch_size=64, epochs=3)

Epoch 1/3
17500/17500 [==============================] - 77s - loss: 0.5352 - acc: 0.7402    
Epoch 2/3
17500/17500 [==============================] - 73s - loss: 0.4048 - acc: 0.8182    
Epoch 3/3
17500/17500 [==============================] - 73s - loss: 0.3339 - acc: 0.8555

After 3 epochs, we have a training accuracy of 85%. But this is not what we are actually interested in. The real evaluation of the model should be on the data that was not seen by the model during the training process. Since we have already created a testing set before, we’ll use that to evaluate our model’s performance. Keras models have a function called evaluate which will give us the performance of our model. Evaluate function returns its outputs in a list and it may not be clear the numbers mean. So we can print metrics_names property to see what each numbers mean.

print(model.metrics_names)
model.evaluate(X_test, y_test, batch_size=128)

['loss', 'acc']  
[0.56499939314524328, 0.73893333314259846]

So the first number is the loss and second number is accuracy. We achieved 73% accuracy in our test data which is lower than the accuracy reported during training. You can change the parameters in convolution layers, dense layers to see how it impacts the model’s performance.

Finally lets visualize some of the predictions made by our model.

predictions = model.predict(X_test)
predictions = np.where(predictions.flatten() > 0.5, 1, 0)
# plot random 20
p = np.random.permutation(len(predictions))
plot_images(X_test[p[:20]], humanize_labels(predictions[p[:20]]))

prediction image here