Avoiding a Keras Applications pitfall

An easy bug and an elegant solution to Keras Applications preprocessing

Photo by Wilhelm Gunkel on Unsplash

Motivation

Keras Applications is an excellent source of your standard image classification architectures: ResNets, VGGs, EfficientNets, etc. But… it is pretty easy to use them incorrectly if you aren’t careful!

Let’s take a peek at the fine print in the documentation. From the documentation on EfficientNet:

Note: each Keras Application expects a specific kind of input preprocessing. For EfficientNet, input preprocessing is included as part of the model (as a Rescaling layer), and thus tf.keras.applications.efficientnet.preprocess_input is actually a pass-through function. EfficientNet models expect their inputs to be float tensors of pixels with values in the [0-255] range.

In comparison, let’s look at a VGG:

Note: each Keras Application expects a specific kind of input preprocessing. For VGG16, call tf.keras.applications.vgg16.preprocess_input on your inputs before passing them to the model. vgg16.preprocess_input will convert the input images from RGB to BGR, then will zero-center each color channel with respect to the ImageNet dataset, without scaling.

If we don’t use the right preprocessing, training with pretrained weights will have unstable loss in early epochs. Let’s take a look at a simple fix!

Solution

All we need to do is prepend a layer with the correct preprocessing function. Give it a try with any of your favorite architectures and constructor kwargs. Here’s one as a sample:

python keras_constructor.py ResNet50 — model_constructor_kwargs “{include_top: False, classes: 5, input_shape: (224, 224, 3)}”

A quick overview of the helper functions:

• get_model: Loads the standard Keras Applications architecture.
• get_model_with_preprocessing: Prepends the appropriate preprocessing.
• elegant_keras_applications_constructor: Adds a CLI handler for this tutorial.

Conclusion

I hope this helped you avoid a bug! Rather unexpectedly, Keras Applications models don’t automatically include the appropriate preprocessing functions. Luckily, now we have a nice design pattern to solve the problem.