How to Make More Data
Using data augmentation to create more data for image classification problems
In a world where data is abundant, a time will come when you just don’t have enough.
No need to panic!
Data augmentation can be used for multiple different data types, including text by replacing words and phrases with their synonyms, and numerical data by using techniques like SMOTE(Synthetic Minority Oversampling Technique).
If you working on an image classification project and are finding that your results are less than exciting, it may be worthwhile to see if data augmentation can help.
Data augmentation creates more data by taking an image and altering it in some specified way. Not only does this make more data, but it can also decrease overfitting by making the patterns less obvious.
Let’s take, for example, the iconic Hollywood sign. This image will make it easy to see exactly what is happening when we adjust the photo.
First, let’s load in the picture and our libraries.
Next, we will need to turn the picture into an array and reshape it. These next two steps are necessary to transform the image without getting an error.
Our reshaped image shape follows the format [batch, height, width, channels]. We have placed a “1” at the beginning of the new shape because this “batch” only has one picture.
Next, we can use a function that computes the data (.fit), generates augmented data (.flow), changes the array back into an image (.imshow) and plots four augmented pictures in one row.
The hard part is done! Now we can see what happens when we change the data. To make this happen in one quick step, we can use Keras ImageDataGenerator. The Generator augments the image randomly as it is passed into the model and returns an altered array.
The most popular use for ImageDataGenerator may be the rescale feature, but can also be used for other shifts as well. Check out the different transformations that are available:
Rotation
We gave a rotation of 90 degrees. That tells the generator we are okay with the image being rotated either way within 90 degrees.
Horizontal Flip
Horizontal flip tells the generator we want to mirror the image across the horizontal axis. Since the generator works at random, the image may not always be flipped.
Vertical Flip
Vertical flip is similar to horizontal, but uses the vertical axis causing the image to be mirrored AND turned upside down.
Shear
Shear will distort an image along the x- OR y-axis. One axis will stay the same, while the other will be lengthened. This will cause the image to look stretched, rather than simply rotated.
Width Shift
Using width shift, you can see that the image has moved along the horizontal axis within the stated pixel range. You can see how pixels have been drawn out to the end of the image rather than leave an area blank.
Height Shift
With height shift, you can move the image up and down on the vertical axis, again using a number of pixels as the range of movement.
Zoom
Zoom can move in closer by having a value that is less than 1. When the value of the zoom range is greater than 1, the image will zoom out like the second image in the row.
Channel Shift
When talking about images, channels refer to colors. Here, the channel range goes from 0–255 where 0= black and 255= white. Our statement above states the image can be darkened to be completely black or lightened to almost completely white.
All Together Now!
Not only did we put some augmentations together that you have seen before, I added a new one too. The default “fill_mode” is called nearest and causes the last pixel to be drawn out, but there are other options. I chose to use “constant” and fill the empty space with black (cval= 0) so the transformations could be easily identified.
Putting multiple augmentations together can reduce the overfitting of your models by making the each image distinct. You can see how these images have similarities, but they are not exactly the same.
At this point, maybe you are wondering, “Okay, but how much new data am I making here?” It’s important to note two things:
First, the new data being generated to train your model is temporary. It is randomly created during the training process and not stored in your memory. A new batch of transformed data is fed into the model each epoch, thus decreasing overfitting by limiting the times the original data is seen.
Second, the amount of augmented data depends on the batch size and amount of original data. The easiest way to determine how much new data your model is seeing is to divide the size of your dataset by the batch size. Dividing the number of samples in your data by the batch size is known as steps per epoch.
You may have noticed the steps per epoch already when you are running your neural network.
The model above ran through all 40 epochs and had a steps per epoch size of 156. This particular data set had 4656 images and a batch size of 30.
steps_per_epoch = len(data) / batch_size
156 = 4656 / 30While it may feel as though the model is “making new data”, it is simply modifying the existing data into something it hasn’t seen before. Your epochs and batch sizes will stay the same.
What does it look like when it all comes together?
For the example below, I used chest radiographs to predict if pneumonia was present. Notice how ImageDataGenerator was used twice. The first time was to make sure all the images were the same size. The generator was used to permanently alter the images. Then the a different generator was created to temporarily adjust each image being sent in to the model for training.
If you would like to see how the model performed (I’ll give you a hint: it was the winner!), please check out my GitHub. You’ll find data augmentation as well as other fun tricks for image classification projects.
If you would like the code for that was used for the Hollywood sign demos, please visit this link.
