Deep Learning With Python

Simple and Effective Tips and Tricks to Learn Deep Learning with Python

© Copyright 2020 - All rights reserved.

The contents of this book may not be reproduced, duplicated, or transmitted without direct written permission from the author.

Under no circumstances will any legal responsibility or blame be held against the publisher for any reparation, damages, or monetary loss due to the information herein, either directly or indirectly.

Legal Notice:

You cannot amend, distribute, sell, use, quote, or paraphrase any part of the content within this book without the consent of the author.

Disclaimer Notice:

Please note the information contained within this document is for educational and entertainment purposes only. No warranties of any kind are expressed or implied. Readers acknowledge that the author is not engaging in the rendering of legal, financial, medical, or professional advice. Please consult a licensed professional before attempting any techniques outlined in this book.

By reading this document, the reader agrees that under no circumstances is the author responsible for any losses, direct or indirect, which are incurred as a result of the use of the information contained within this document, including, but not limited to, —errors, omissions, or inaccuracies.

Table of Contents

Introduction

Chapter One: Introducing Deep Learning

How Does Deep Learning Work?

What is a Deep Learning Neural Network?

Deep Learning Methods

Deep Learning Applications

Deep Learning Challenges and Limitations

Machine Learning vs. Deep Learning

A Brief History

Chapter Two: Python and Deep Learning

The Top Python Libraries for Deep Learning

Chapter Three: A Look At Neural Networks

Neural Network Applications

Examples of What Deep Learning Can Do

The Elements of a Neural Network

Feedforward Networks

Multiple Linear Regression

Gradient Descent

Logistic Regression

Chapter Four: A Deeper Look at Recurrent Neural Networks and LSTMs

More on Feedforward Networks

Recurrent Neural Networks

Backpropagation Through Time (BPTT)

Long Short-Term Memory Units (LSTMs)

Capturing Remote Dependencies and Diverse Time Scales

Chapter Five: Introduction To CNNs and Image Processing

Images as 4D Tensors

The Definition of Convolutional

How Do Convolutional Neural Networks Work?

Max Pooling/Downsampling

Chapter Six: An Introduction to Deep Reinforcement Learning

The Definitions of Reinforcement Learning

Domain Selection

Neural Networks and Deep Reinforcement Learning

Chapter Seven: Let’s Build a Deep Learning Model with Keras and TensorFlow

Step One – Data Pre-Processing

Step Two – Splitting Your Dataset

Step Three — Transform the Data

Step Four — Build the Neural Network

Step Five — Use the Test Set for Predictions

Step Six — Use a Confusion Matrix

Step Seven — Make One Prediction

Step Eight — Improve the Model Accuracy

Step Nine — Reduce Overfitting By Adding Dropout Regularization

Step Ten — Tune the Hyperparameters

Chapter Eight: Building a Neural Network With TensorFlow For Handwritten Digit Recognition

Step One — Configure Your Project

Step Two — Import the MNIST Dataset

Step Three — Define the Architecture of Your Neural Network

Step Four — Build the TensorFlow Graph

Step Five — Train and Test

Chapter Nine: How to Keep Your Deep Learning Models Bug-Free

Step One – Keep Things Simple

Step Two – Implement Your Model and Debug It

Step Three – Evaluate the Model Performance

Step Four – Improving Your Model and Data

Step Five - Tuning the Hyperparameters

Conclusion

Introduction

Deep learning is fast becoming one of the most used "buzz phrases" in the world, along with 'machine learning' and 'artificial intelligence.' In fact, all three are part of one of the biggest global phenomena, data science. Deep learning is a subset of machine learning and AI, and it is a subset that is growing rapidly, increasing year on year. In the next ten years or so, it is expected that deep learning will grow 71 times over in the USA, but globally, much more than that. Who doesn't want to be a part of that?  Now, more than ever, it is a great time to get involved in deep learning to assure yourselves a position in the future.

I wrote this book to give you a jump-start in the fundamentals of deep learning. To tell you how to use deep learning to your advantage and teach you the methods, the algorithms, the learning mechanisms, everything you need to know to get started in the field.

Here's what we cover in this book:

- Chapter One - an introduction to deep learning, how it works and what its limitations are

- Chapter Two – a discussion on why we use Python for deep learning and a look at some of the more popular Python libraries used for deep learning

- Chapter Three – an introduction to neural networks, what they are, and how they work. We look at examples such as regression, clustering, and classification, and we look at the elements a neural network is made from.

- Chapter Four – next, we turn to recurrent neural networks (RNNs) and LSTM (Long Short-Term Memory) units. This takes us into feedforward networks, RNNs, and Backpropagation Through Time. Then we look at vanishing/exploding gradients, hyperparameter tuning, and remote dependencies.

- Chapter Five – we look at deep convolutional neural networks and how to use them for image processing tasks. We look at how CNNs work and how to do downsampling and max-pooling with them.

- Chapter Six – in this chapter, you get an introduction to deep reinforcement learning.

- Chapter Seven – now, you get to put everything into practice by building a deep learning model for employee retention predictions, using two of the more popular libraries, Keras and TensorFlow

- Chapter Eight – here, we use TensorFlow to build a neural network for recognizing handwritten digits

- Chapter Nine – Finally, we look at ways to make sure your deep learning models remain free of bugs.

What I haven't done with this book is to use one example all the way through. Instead, I have divided the book into separate chapters, each with their own information. That way, you can pick and choose the subject you want to look at, without worrying that you've missed some vital information. Basically, read this book in any order you want to. However, I must stress that I will NOT be giving you a primer on the Python computer language and I will NOT be giving you an overview of machine learning – these are things you should already have experience of before you even think about deep learning.

Why You Should Learn Deep Learning

There are plenty of reasons why you should take that step and get on the deep learning bandwagon:

- Better Earnings – Data scientists already earn a decent salary, but, when you specialize in machine learning and deep learning, you can earn an even better salary. On average, machine and deep learning specialists earn around 15% more than a standard data scientist salary so, if you have skills in this area, your resume will certainly stand out head and shoulders above the rest, even if you are not quite at the level of specialist.

- A Growing Demand – it is one of the fastest-growing fields in the world, growing at a pace we can't keep up with up; the faster it grows, the more demand there is for it, it's as simple as that.

- It's Time-Saving – if you have any experience of machine learning, then you already know that its time-consuming to convert parameters for inputs into features that are easily read by your algorithm. And it's not the easiest job, either. With deep learning, you can use neural nets, the best way of ensuring the conversions are done automatically. Rather than having to take the time to pull histograms, color data, and lots of other metrics from an image dataset, simply give the raw images to the neural net, and it will do it all for you. Yes, that is a rather simplistic way of doing things; it is a bit more complicated than that! The primary challenge lies in getting the neural network to the stage where it can do it all for you, but this does mean you get more time to work on the algorithms, rather than having to waste time on features.

- A Flexible Specialty – while it's great to specialize in something specific, you do run the risk of becoming stuck in a rut; at the end of the day, you may find yourself in a field that is limited in where you can work; with deep learning, it's different. It's in demand across most industries and is used to solve all kinds of tasks, such as translation, image recognition, robotics, and so on.

- It's Fun – putting careers to one side, why wouldn't you want to learn something that is just so cool? Let's face it; you can use deep learning to make a machine do just about anything you want it to, without having to do much in the way of teaching.

So, are you ready to dive into deep learning?

Then let's go.

Chapter One: Introducing Deep Learning

So, what is deep learning? Well, it is a kind of artificial intelligence and machine learning that mimics human methods of gaining knowledge. It is one of the most important parts of data science and includes predictive modeling and statistics. For the data scientist whose job involved collecting large amounts of data, analyzing it and interpreting it, deep learning is incredibly beneficial, as it significantly decreases the amount of time taken to perform the process.

In simple terms, we can consider deep learning as a process that automates predictive analytics. You probably already used some machine learning algorithms, and you will know that most of these are linear. With deep learning, the algorithms are stacked up, increasing in abstraction and complexity as they go.

One of the easiest ways to understand deep learning is to think of a child speaking its first word. We’ll say, for argument’s sake, that word is ‘cat.’ The child learns what cats are and what they are not, simply by pointing at things and saying the word, ‘cat.’ The parents will say, ‘no, that isn’t a cat,’ or ‘yes, that is a cat’ and, along the way, the child will learn to identify a cat by becoming aware of the features that every cat has. Without realizing it, what this child is doing is learning a complex abstraction (the concept of a cat). He does this by building up a hierarchy, each level made from the knowledge that came from the preceding layer.

How Does Deep Learning Work?

A computer program that makes use of deep learning will go through a similar process to that of the child learning what a cat is. The hierarchy contains multiple algorithms, and each one will transform the input using a nonlinear transformation. It will take what it learns from that transformation and create the output in the form of a statistical model. It will continue to iterate until the output is acceptably accurate. The name, ‘deep,’ comes from the fact that the data has to pass through multiple processing layers.

Traditional machine learning uses supervised learning, with the programmer having to be very specific about what it tells the computer to look for. For example, if you want the model to predict images of cats, you have to be very specific about feeding the model information about cat features, so it can determine if a given image does or does not contain a cat. This is known as ‘feature extraction,’ and it is a very time-consuming process. How successful the computer is will be entirely dependent on your ability, as the programmer, to define an accurate feature set for a ‘cat.’ Where deep learning comes into its own is this - the program can construct that features set without your help and with no supervision at all.  Unsupervised learning is incredibly accurate and faster than supervised learning.

In the beginning, the programmer may provide the program with what is known as a ‘training dataset.’ In this case, this would be a series of images, each labeled by a human with meta tags containing the words, ‘cat’ or ‘not cat.’ This training data will provide the programmer with a certain amount of information, which can then be used to create the ‘cat’ feature set and the predictive model.    What happens here is that the first model created by the computer could label any image that it sees a tail and four legs in as a ‘cat,’ but the program doesn’t know the labels of ‘tail’ and ‘four legs.’ Instead, it will look for patterns in the image pixels. Each time the data is iterated over, the model increase in complexity and becomes more accurate.

The child could take several weeks, even months, to learn what a cat is. The computer program using deep learning algorithms is shown one training dataset. It can then process millions of images and make accurate predictions on which images contain cats within minutes.

However, for their accuracy level to be considered acceptable, these programs must be fed with massive amounts of data for training purposes, not to mention the processing power required. Until recently, until cloud computing and big data came along, programmers never had access to that much data or power.

Deep learning models can take their iterative output and use it to create statistical models that are incredibly complex. Because of this, massive quantities of unstructured and unlabeled data can be used to create predictive models with very high levels of accuracy.  That is very important; as IoT (Internet of Things) becomes increasingly more pervasive, much of the data created by machines and humans are not structured, nor is it labeled.

What is a Deep Learning Neural Network?

Artificial neural networks are the underlying networks for pretty much all deep learning models. These networks are more advanced ML (machine learning) algorithms and, as a result, you may often see deep learning termed as ‘deep neural networking’ or ‘deep neural learning.’

You will be learning more about neural networks later in the book but, for now, it’s enough to know that there are several forms, including|:

- CNNs (convolutional neural networks)

- RNNs (recurrent neural networks)

- ANNs (artificial neural networks)

- FNNs (feedforward neural networks).

Each one of those is beneficial in certain use cases, but all of them work in much the same way. Data is fed in, and each model will work out whether it has made the right decision about a specific data element.

Neural networks work on something of a trial-and-error basis, and, for that reason, they need to train on huge amounts of data. There is no coincidence in the fact that neural networks didn’t grow popular until after the majority of enterprises started storing massive amounts of data. The first few iterations of the model are nothing more than educated guesses on segments of speech or image contents. Because of that, the training data has to be labeled so the model can tell how accurate its guesses were. However, although lots of enterprises use massive amounts of data, unstructured data isn’t so helpful in this case. Unstructured data analysis can only be done by deep learning models that have been trained and are acceptably accurate. However, you cannot train a deep learning model on unstructured data.

Deep Learning Methods

Decent deep learning models can be created using several different methods. That includes transfer learning, learning rate decay, dropout, and training from scratch.

- Transfer Learning – a process involving the perfection of a model that was trained earlier, requiring an interface to a pre-existing network’s internals.  The existing network is fed with new data that contains classification not known before. The network is adjusted, allowing for new tasks to be performed, with categorization abilities that are far more specific. Transfer learning doesn’t require a lot of data, so it is much faster in computation time.

- Learning Rate Decay – sometimes known as adaptive learning or learning rate annealing. The learning rate is one of the hyperparameters. These define the system, or they set specific conditions for how the system works before going through the learning process. This particular hyperparameter controls the level of change experienced by a model as a response to the error estimation whenever the weights are changed.  If the learning rate is too high, the training process may not be very stable, and the model may learn a set of weights that are less than optimal. If a learning rate is too low, the training process may be too long and could get stuck.  The learning rate decay method is when the learning rate is adapted for better performance and reduced training time. The commonest adaptation includes the reduction over time of the learning rate.

- Dropout – a method whereby units are randomly dropped, along with their connections, to solve overfitting in a network that has a lot of hyperparameters. The units are dropped during the training in a method proven to improve network performance in supervised learning tasks, such as computational biology, document classification, and speech recognition.

- Training from Scratch – a developer must collect a sizeable dataset and create an architecture that can learn features and build a model. This is a good technique for new applications and for those that have a lot of output categories. However, it isn’t a very common approach because it needs so much data; that makes training a very long process.

Deep Learning Applications

Deep learning models are built to process information much the same way that the human brain does. Because of that, we can use these models with many human tasks. Currently, we can find deep learning used in image recognition, speech recognition, and natural language processing, in diverse applications such as translation and self-drive vehicles.

Today, deep learning use cases include anything that uses big data. Some of the more specific fields where the models are used include:

- Customer Experience – chatbots are something many of us have seen on websites and, while they are relatively new, they are continually improving. They are used to help improve customer satisfaction and experiences.

- Text Generation – a machine learns style and grammar from a piece of text, and the resulting model can create a new text that matches the correct style, grammar, and spelling seen in the original.

- Military and Aerospace – deep learning models are employed to detect objects from satellites that show areas of potential interest, and safe/unsafe troop zones.

- Industrial Automation – it is also being used to improve safety for workers in warehouses, factories, and other similar environments. It does this by providing automatic detection services that can tell when an object or person is moving too close to machinery.

- Add Color – deep learning models can add color to black and white videos and photos much quicker than it can be done manually.

- Medical Research – more specifically, cancer research where deep learning models are being used in the automatic detection of cancer cells.

- Computer Vision – deep learning has made computer vision much more accurate, especially in areas such as segmentation, restoration, image classification, and object detection.

Deep Learning Challenges and Limitations

The fact that deep learning models learn through observation is one of the biggest limitations. Why? Because the model can only know what was in the training data. If the data is from one or two sources that don’t represent a broader area, or there is only a small amount of training data, the models cannot possibly learn in ways that can be generalized.

Another major issue revolves around biases. If the data used for training contains biases, they will be reproduced in the model’s predictions. This has caused no end of consternation, given that a model uses small data variations to learn differentiation. More often than not, the programmer won’t explicitly see what the model considers to be important factors. So, for example, if we have a facial recognition model, it might use factors such as gender or race to make predictions about human characteristics without the programmer being aware of them.

One of the biggest challenges is the learning rate. A high rate means the model converges early, and the solution will be suboptimal. A low rate, on the other hand, could cause the process to grind to a halt, making it tough, if not impossible, to get to a solution.

Something else that can cause limitations for deep learning models is the hardware requirements. Deep learning models need multicore GPUs (graphics processing units) that are incredibly high-performance, along with similar units, to ensure a reduction in time and increased efficiency.  However, these are not cheap and use a great deal of power. A deep learning system will also require RAM and a hard drive or an SSD.

Other challenges and limitations include:

- Models need huge amounts of data to be effective. And the more accurate and more powerful a  model is, the more parameters it needs, and that requires a lot more data.

- Once a deep learning model has been trained, it is no longer flexible, and multi-tasking is not an option. While their solutions are accurate and efficient, they are only for one given problem; if we wanted to solve a different problem, even one that was similar, we would need to retrain the system.

- Currently, deep learning cannot do long-term planning, nor can it manipulate algorithmic-like data or apply reasoning, even with massive amounts of data.

Machine Learning vs. Deep Learning

Contrary to popular belief, machine learning and deep learning are not quite the same. Deep learning is a machine learning subset that solves problems differently. With machine learning, a domain expert is needed to identify the features applied the most, whereas, in deep learning, the model learns the features gradually, doing away with the need for the domain expert.

A deep learning algorithm takes longer to train than those used in machine learning; however, the converse is also true – testing a deep learning algorithm takes less time than testing a machine learning algorithm. With the latter, as the data size increases, so does the testing time.

Also, machine learning doesn’t need such expensive machines, nor does it need the same level of performance from a GPU that deep learning does.

However, data scientists tend to opt for machine learning because it is far superior in terms of interoperability, and they can make more sense of the solutions. Plus, we use machine learning when we don’t have so much data, leaving deep learning for the massive datasets. We also prefer deep learning when there is no domain expert to interpret the features,  or when a problem is more complex, such as natural language processing or speech recognition tasks.

A Brief History

We can trace deep learning as far back as 1943 when Walter Pitts and Warren McCullough built a computational model using algorithms and mathematics. However, it took until the middle of the 2000s to start using the term ‘deep learning.’ Ruslan Salakhutdinov and Geoffrey Hinton published a paper showing multi-layered neural networks being trained on a single layer at a time.

Google made its mark on deep learning in 2012, when an algorithm they built showed it could recognize cats in images. In 2014, Google acquired a British AI startup called DeepMind and, two years later, their algorithm called AlphaGo, built on DeepMind, beat the world champion GO player at his own game during a tournament held in Seoul.

In recent times, most of the advances made in artificial intelligence have come from deep learning models. We have also seen the emergence of deep reinforcement learning, which we’ll be discussing later, to integrate complex applications, like self-drive vehicles, video gaming, and robotics, with AI. There is a difference between reinforcement and deep learning, in that the latter uses a training set to learn and then applies it learning to an unseen data set. In contrast, the form uses dynamic learning, using continuous feedback to adjust its actions to get a better reward.

Reinforcement learning models can provide strong, fast control of GANs (Generative Adversarial Networks.  GANs and deep reinforcement learning are combined by ATNC (Adversarial Threshold Neural Computer) to come up with organic molecules that contain a specific pharmacological property set.

And GANS also get used for generating the artificial training data required by machine learning, useful for datasets where the data is not balanced or when sensitive information is contained within the data.

Chapter Two: Python and Deep Learning

For a long time now, Python has been the programming language of choice for data scientists, especially those working in machine learning, deep learning, and artificial intelligence. Why? Because it is one of the most flexible programming languages in existence, not to mention powerful. And it is packed with features that help developers and programmers increase productivity and to produce high-quality code. Add to that the extensive libraries the language offers, and it’s no surprise it’s the number one choice.

Some of the features that put Python at the top are:

- It’s free and open-source – which makes it user-friendly, community-friendly and with guaranteed improvements issued regularly

- Extensive libraries – that provide a solution for just about any problem you can think of

- Easy to implement and integrate – making it one of the most accessible languages for people of all skill levels

- Less time needed to code and debug – meaning a significant increase in productivity

- Easy to use – for natural language processing and soft computing

- Easy integration – with code modules from C and C++

The Top Python Libraries for Deep Learning

Sure, there are loads of computer programming languages to choose from, but none of them are as friendly to developers as Python is, especially when it comes to deep learning. Python has a set of libraries built-in that cater to just about everything you can think of, and these are the best ones for deep learning:

- TensorFlow

TensorFlow is one of the fastest open-source libraries, incredibly flexible and scalable, ideal for research and production. Owned by Google, the library makes it easy to build machine and deep learning models, no matter what your level of expertise is.

With the TensorFlow library, you can easily create deep learning models and train them. And not just on a computer either. If you use TensorFlow Lite or TensorFlow Serving, you can even create them on mobiles and servers, taking advantage of the same benefits as the computer version.

Some of the essential deep learning areas where TensorFlow is useful are:

- Deep neural networks

- Partial Differential Equations

- Natural Language Processing

- Abstraction

- Speech, text, and image recognition

- Easy to collaborate on code and ideas

You can learn more about TensorFlow at https://tensorflow.org

Keras

One of the more popular neural network libraries, it was a Google engineer who first designed Keras, for ONEIROS (Open-Ended Neuro Electronic Intelligent Robot Operating System). It didn’t take long before the open-source library was given full support in the core library for TensorFlow, ensuring that Keras could be used on top of TensorFlow.

Keras is packed with some of the best tools and building blocks required to create  neural networks, including:

- Activation functions

- Batch normalization

- Cost functions

- Dropout

- Neural layers

- Objectives

- Pooling

Keras uses these features to extend TensorFlow’s usability and, with a huge community backing it up, including a Slack channel, there is plenty of support available. That includes support for standard neural networks, recurrent neural networks, and convolutional neural networks.

You can get more information about Keras from https://keras.io

PyTorch

PyTorch is a Facebook development and, as well as Python, it also has support for C++ with a specific interface. It is considered to be one of the best deep learning frameworks, in competition with TensorFlow.

Some features set PyTorch apart, though:

- It includes tensor computing and accelerated processing through GPUs

- It’s an easy library to learn and can be integrated with the Python ecosystem very easily

- An auto-diff system, based on tapes, offers neural network support

PyTorch also comes with several modules designed to create neural networks and train them:

- torch.Tensor for tensors

- torch.optim for optimizers

- nn module for neural networks

- Autograd

While PyTorch is customizable and is used a lot in deep learning research, it isn’t speed-optimized, and it doesn’t have so many NLP abstractions.

Scikit-Learn

Another incredibly popular deep learning library, Scikit-learn, is easy to integrate with other similar libraries, such as Pandas and NumPy. Scikit-learn supports several algorithms, including:

- Classification

- Clustering

- Dimensionality Reduction

- Model Selection

- Preprocessing

- Regression

The library is flexible, easy to use, and is focused primarily on modeling data, not on loading data, handling, manipulating, or visualizing it. It is enough of a library to be used end-to-end, right from the research phase of a project, to the deployment phase.

For more information on Scikit-Learn, go to http://scikit-learn.org

Pandas

One of the most popular libraries for data analysis, Pandas is mostly used for manipulating and analyzing data. The time to use Pandas is before you prepare your dataset for training purposes.

You can easily work with multi-dimensional data (structured) and time-series using Pandas and, some of its best data handling features are:

- It can reshape and pivot data

- It can merge and join datasets

- It can handle data alignment and missing data

- It offers several options for indexing, including Fancy and Hierarchical Axis

- It offers several options for filtering data

When you use Pandas, you will come across the term ‘DataFrames’ – these are used by the library and are nothing more than two-dimensional data representations that use DataFrame objects.

For more information on Pandas, go to https://pandas.pydata.org

NLTK

NLTK is a Natural Language Toolkit offered by Python to make it easy to work with natural language processing. It is a popular library, especially for those working on human language projects. It has a simple interface and many lexical resources, including Word2Vec, WordNet, and FrameNet.

Some of its best features are:

- Looking through documents for specific keywords

- Classification and tokenization of text

- Handwriting and voice recognition

- Word stemming and word lemmatization

NLTK offers an entire package suite that is suitable for any industry that works with language.

For more information on NLTK, go to https://nltk.org

Spark MLlib

The Spark MLlib library is an Apache development that offers easy computation scaling. It is an easy library to use, simple to set up and integrates easily with other tools. It has become an instant favorite for developing applications and algorithms for deep learning models.

Some of the more popular APIs and algorithms that Spark MLlib can make use of are:

- Basic Statistics

- Classification

- Clustering

- Dimensional Reduction

- Feature Extraction

- Optimization

- Regression

For more information on Spark MLlib, go to https://spark.apache.org/mllib/

Theano

Theano is one of the more powerful of the learning libraries and is primarily used to make it easy to define, optimize, and evaluate mathematical expressions. It is one of the best libraries for large scale scientific calculations, offering features like:

- GPU support to enable better heavy-duty computations to be performed

- It has strong support for integration with NumPy

- It is faster and more stable at evaluating variables, even the trickiest ones

- Creating custom C code is easy for mathematical operations

Theano allows you to develop highly efficient algorithms rapidly, and several of the best-known libraries are built on Theano, such as Lasagne, Blocks, and Keras.

For more information on Theano, go to http://deeplearning.net/software/theano/

MXNet

Another Apache offering, MXNet is the ideal library for training deep neural nets and deploying them. It is one of the most scalable libraries,  with full support for rapid training, working with many other languages besides Python.

Because MxNet is so scalable and portable, you can easily take from a platform, scale it to your requirements, and put it on another platform. It is used by some big names, including Microsoft, Intel, and MIT, as well as Amazon AWS for deep learning.

For more information about MXNet, you can go to https://mxnet.apache.org/

NumPy

Last, but by no means least is NumPy, the top library to handle large multi-dimensional data as well as the complex mathematical operations that do the data operations. It offers fast computation and can execute complex functions that work on arrays.

Some of the features offered by NumPy include:

- It supports logical and mathematical operations

- Shape manipulation

- Capabilities to sort and select

- Discrete Fourier transformations

- Random simulation

- Linear algebra operations

- Statistical operations

- It supports n-dimensional arrays

NumPy is an object-oriented library that can also be integrated with Fortran, C, and C++ code.

For more information on NumPy, go to https://numpy.org/

Python is one of the very best languages for all things deep learning. It isn’t just a great language for general programming; it also supports specific niches. It offers plenty of libraries that expand its capabilities, making it the number one choice for data scientists the world over.

Chapter Three: A Look At Neural Networks

Neural networks are the backbone of deep learning, a series of algorithms loosely based on the human brain and designed to find patterns and relationships in data. They work in much the same way as the neurons in the human brain, but, in this case, the neurons are mathematical functions. They collect information, and they classify it based on the architecture of the learning model.

Neural networks are networks of nodes, all interconnected. Every node is called a perceptron, and they are very much like multiple linear regression. The signal that comes from the linear regression is fed to an activation function by the perceptron; that activation function may be non-linear.

Where you have an MLP, or multi-layered perceptron, all the perceptron’s are in layers, all interconnected. The input patterns are collected by the input layer, while the output layer has the output signals or classifications the input patterns map to. For example, you may have technical indicator quantities for securities in a list; potentially, the outputs could be one of three – buy, hold, sell.

Then we have the hidden layers. These are the input weights that are fine-tuned until the network has a minimum error margin. The hidden layers are also responsible for extraction features from the data. Those will be the features that have a direct effect on the predictions.

Neural Network Applications

Neural networks tend to be more used for financial applications, trading, product maintenance, enterprise planning, and business analytics. However, just recently, they have also started to gain traction in fraud detection solutions, risk assessment, marketing research, forecasting, and other business applications.

Neural networks are great for evaluating data, especially price data, and providing businesses with the opportunity to use that data analysis to make business decisions. The networks can tell the difference in patterns and interdependencies (non-linear) that other analytical methods can’t. Research shows that there is some difference in how accurate different networks are in predicting stock prices. While some will make the right prediction, approximately 50-60% of the time, others show an accuracy of 70% or higher, all the time.

Yes, there are always going to be task classes and datasets that an earlier developed algorithm will analyze much better, but it isn’t always about the algorithm. It’s more about how well you prepare the data for input on the indicator; that’s what determines how successful, or otherwise, a neural network is.

Neural networks are typically used for classification and clustering tasks and are often thought of as classification and clustering layers that sit on top of your stored data. They help place unlabeled data into groups based on similarities found in the training inputs, and, when they are working on a labeled dataset, they can classify the data. Plus, you can also train a neural network to extract features that have been given to another clustering or classification algorithm. In that way, deep neural networks could be considered as part of a machine learning model or application that involves regression, classification, and reinforcement learning algorithms.

Examples of What Deep Learning Can Do

We know that deep learning will map an input to an output, and we know that it will find relationships between them. These models are often called “universal approximators” because they can learn how to approximate unknown functions f(x) = y between input x and an output y. Of course, that assumes that there is a relationship of any kind between the input and output, be it causation or correlation.

As they learn, neural networks can locate the correct f or the right way of turning x to y, be that f(x) – 3x + 12 or f(x) = 9x – 0.1.

Let’s see some examples of what deep learning can be used for.

Classification

Every classification task, no matter what it is, is dependent on the dataset being labeled. That means humans have to be involved in the process, providing the dataset with their knowledge to that the network can learn the relationships between data and labels.  We call this supervised learning, and it is used for:

- Face detection, identifying faces in images, facial expression recognition, etc.

- Identifying specific objects in an image

- Recognizing different gestures from videos

- Voice recognition, speech-to-text translation, sentiment recognition, etc.

- Text classification – spam emails, fraudulent insurance claims, and sentiment recognition in text-based customer feedback

Basically, if a human can generate a label, if there is a specific outcome that you want that correlates to some data, you can use it to train your network.

Clustering

Clustering is sometimes known as grouping, and it is all about finding similarities in data. To do this, labels are not required, and this is known as unsupervised learning. The vast majority of the world’s data is unlabeled, and one of the most important things to remember is that the more data you give an algorithm for training purposes, the better it will learn, and the more accurate its predictions will be. As such, unsupervised learning is more likely to produce accurate models than supervised learning is. Some of the things clustering algorithms are used for include:

- Searching documents, sounds and images for similar items

- Detecting anomalies or any unusual behavior, one of the main reasons why clustering algorithms are used for fraud detection.

Regressions and Predictive Analytics

Using classification algorithms, we can find relationships between things, such as a person’s name and pixels from an image.  This is sometimes known as static predictions and, given sufficient data, deep learning can even establish if there is any correlation between present and future events, running regressions between the past and future. In a sense, the future event is similar to a label. Deep learning isn’t interested in time, and it doesn’t case whether an event has happened or not. Provided it has a time series, it can read a number string and predict what number occurs next. These deep learning algorithms are used for:

- Data centers, transport, manufacturing, and other hardware breakdowns

- Heart attacks, strokes, and other health breakdowns, based on data and statistics from wearable devices

- Predicting how likely a customer will leave a website or purchase a specific product, based on metadata and web activity

- Predicting how likely an employee is to leave

The better the predictions we can make,  the better we can pre-empt and prevent disasters and breakdowns. Neural networks are taking us closer to a world where there are fewer surprises, a world of smart agents that reach their goals by combining neural networks and algorithms, such as reinforcement learning.

The Elements of a Neural Network

Deep learning is another way of saying a “stacked neural network” or a network that has several layers. Each layer is made up of nodes, each of which is just somewhere for a computation to take place. These are loosely based on the neurons found in the human brain, neurons that fire up when enough stimuli are introduced to it.

These nodes take the input data and combine them with a set of weights or coefficients. These will do one of two things – dampen or amplify the input – and, in that way, they give the inputs a significance, based on what the algorithm is designed to learn.

The products of these input-weights are then summed; that sum is passed through the activation function in the node. This works out whether the signal should go any further through the network, whether it should have any say on the outcome, and if so, to what extent. Once the signal passes through the node, the neuron is said to have been activated.

Node layers are rows of switches, like the neurons, that switch off or on as the input goes through the net. The output from each layer becomes the input for the next layer, with the initial layer being the one that takes your data.

We assign the features with significance by pairing the adjustable weights with the input features, regarding the way inputs are clustered and clasified by the neural network.

The Key Concepts of a Deep Neural Network

We can tell the difference between the deep learning and the single hidden-layer networks simply by how deep they are. In a deep learning network, to achieve pattern recognition, the data has to pass through multiple layers of nodes.

The early neural networks were shallow, having just one input layer and one output layer, with no more than a single hidden layer tucked in between them. If there are more than three layers, it is a deep learning network.

In these, every node layer will train on a set of features based on the output of the previous layer. As you delve deeper into the neural net, the features recognized by the nodes become more complex, as they are aggregate the features from the last layer and recombine them.

We call this feature hierarchy as it is a hierarchy made up of abstraction and complexity that increases the deeper we go. It is this that ensures deep learning networks can handle datasets of high dimensions and with many millions of parameters that go through functions that

More than that, deep neural nets can also search through unstructured and unlabeled data to find latent structures. You will also see unstructured data termed as raw media, for example, audio recording, videos, text, images, and so on. As such, deep learning is often used to process and cluster unstructured raw media, find anomalies and similarities in data that has not been touched by human hand, neither labeled nor stored in a relational database.

As an example, we could feed a deep learning network with millions of images and have the network search them for similarities and cluster them accordingly. Cats, dogs, people, and so on. This what smart photo albums are based on.

Now we can apply the exact same principle to other types of data, such as emails, news articles, and so on. It could cluster complaint emails in one space, spam in another, and so on. This is what email filters are based on, commonly used in CRM, or customer relationship management.

When a network is fed time-series data, it may cluster the data into healthy/normal behavior and dangerous/anomalous behavior. If a smartphone generates the data, it could provide some insights into the habits or health of the device user.

Deep learning networks do not need human intervention to do automatic feature extraction, whereas a standard machine learning algorithm does.  Given that it can take large numbers of data scientists many years to accomplish feature extraction satisfactorily, deep learning is an excellent addition to the unscalable power experienced by smaller data science teams.

When a deep network is trained on unlabeled data, each of the node layers will automatically learn the features by trying, over and again,  to rebuild the input it got its samples from. It will try to keep the difference between the guesses from the network and input data’s probability distribution to a minimum.

During this process, the neural network will learn to spot any correlations between specific features and the best results.  They will pinpoint any connections existing between the feature signals and whatever it is the features are representing – this could be with labeled data or with a full reconstruction.

We can then apply a deep learning network that has been trained on labeled data to the unlabeled and unstructured data. This will provide it with far more information than any machine learning network has access to. This is where the high performance of a deep learning network comes in – the more data we can train the network on, the better the predictions will be. Even a bad algorithm can be better than a good one if it has been trained on a lot more data.

At the end of a deep learning network is an output layer. This is a SoftMax or logistic classifier, responsible for assigning a label or outcome with a likelihood. This is known as predictive but in a much broader sense. For example, if we provided a deep learning network with an image, it may determine that there is a 90% likelihood the final output will be representative of a person.

Feedforward Networks

When we use a neural network, we want to get the network to the least error as quickly as we can. Think of it as running on a round track, where you go past the same point time and time again. The starting point is the state at which we initialized the weights, and the endpoint is the state of the parameters at the point that they can produce accurate predictions and classifications.

Throughout this epic race, there are multiple steps. Each one resembles the one that came before and the one that comes after. Much like running a race, the task involves doing the same thing over and again to reach the finish. Every step involves three things – a guess, a measurement of error, and a weights update. The latter steps adjust the coefficients incrementally, as the network learns to look at the most crucial features.

Whether they are at the initial state or the finished state, the collection of weights is known as a model. This is because we are attempting to model the relationship between the data and the ground-truth label to learning the data structure.  Typically, these models begin as bad models and, over time, change in relation to the parameters as the network updates them; by the end, they may not be good models, but they won’t be as bad as when they started.

Neural nets do not know which biases and nets are going to be the best inputs for making accurate guesses. The model must begin by making a guess; it will then learn from the result to make better guesses.

Let’s see what goes on in the learning process with the simplest architecture, the feedforward network.

The input goes into the network; the weights map it to a set of guesses made by the network at the end.

input * weight = guess

The result of the weighted input is a guess about what the input could be. The neural net will then compare that guess to a ground-truth; in effect, it is asking, ‘was I correct?”

ground truth - guess = error

The difference between the guess and the truth becomes the error; this is measured by the network and then taken back over the model. The weights may be adjusted according to their contribution to the error.

error * weight's contribution to error = adjustment

The pseudo- formulas shown above indicate the key functions of any neural network – scoring the input, working out the loss, updating the model. These functions are repeated over and again, effectively making a neural network into a loop used for corrective feedback. The weights that support the right guesses are rewarded, and those leading to error are penalized.

Multiple Linear Regression

Although the name, artificial neural network, is inspired by the human brain, they are nothing more complicated than code and math, the same as a traditional machine learning algorithm. Linear regression is one of the first things you learn in statistics and, if you understand it, you understand the way neural networks work.

At its simplest, we can express linear regression as:

Y_hat = bX + a

Here:

- Y_hat – estimated output

- X – input

- b – the slope

- a – intercept of a line on a two-dimensional graph’s vertical axis.

To make this clearer, for example, X could be the exposure to radiation, while Y could be the risk factor for cancer.

Imagine that, whenever a unit is added to x, Y_hat, which is a dependent variable, will increase at a proportionate rate, regardless of position on the X-axis. That relationship, the variables that move together, is the starting point.

Next, we want to think about multiple regression. You would have multiple input variables, all producing a single output variable. The typical expression is:

Y_hat = b_1*X_1 + b_2*X_2 + b_3*X_3 + a

Now, multiple linear regression happens, in that form, at each node in your neural network. For every node on one layer, the input from the individual nodes on the layer before gets combined with input from all other nodes. In short, the inputs get mixed up in different proportions, as per their weights, which are different when they lead to the nodes on the next layer. The network will try to reduce the error by testing the input combinations to find the significant ones.

When the node inputs have been summed, they arrive at Y_hat after passing through a nonlinear function. Why? Because if each node did nothing more than multiple linear regression, the value of Y_hat would increase linearly, with no limits, each time X increased. That isn’t what we want.

We want a kind of switch at each node, a neuron if you like. These can be turned on or off, dependent on whether the input signal should be allowed to pass through. This means they have a direct effect on the decisions made by the network.

When there are switches, there are classification problems. Should the signal form the input indicate whether it should be classified by the node as on, off, enough, not_enough? We use 1 and 0 to represent binary decisions; logistic regression is a function (nonlinear) that squashes the input so it can be translated to a space somewhere between 1 and 0.

The nonlinear transformations that happen at every node are typically s-shaped and much like logistic regression. These transformations are named sigmoid, tanh, hard tanh, and so on, and they are used to shape each node’s output.

The output for all of the nodes, after each has been squashed to a space between o and 1, becomes the input for the next layer. This continues until the last layer is reached, and that is where the neural network makes its decisions.

Gradient Descent

One of the most common optimization functions is the gradient descent, used for adjusting the weights as per the error they are responsible for.

Gradient means “slope,” and, on an x-y graph, a slope typically represents the relationship between two variables. In our case, that is the relationship between one weight and network error, i.e., the way the error varies with each weight adjustment.

So, we want to know which weight produces the least error. Which reads and represents the input data signals correctly, translating them to the right classification?

While a neural network is learning, it is gradually adjusting lots of weights, enabling the correct map between the signal and the meaning. The network Error and the individual weights have a derivative relationship, dE/dw. This measures the degree to which even the slightest of changes in a weight can change the error.

Every weight is but one single factor in a network that has many transformations. The weight signal goes through the activation functions and sums in multiple layers. As such, the chain rule of calculus is used to go back through the activations and the outputs, reaching the relevant weight, and seeing how it relates to the network error.

According to the chain rule in calculus:

=

In feedforward networks, we might see the relationship between the error and one weight as:

= *

So, we have a variable called Error and a variable called weight. Another variable, called activation, mediates between them. The weight is passed through the activation variable, allowing you to work out how any change to the weight variable brings about changes in the Error variable. To do that, you calculate how changes in activation bring about changes in Error, and how changes in weight bring about changes in activation.

In essence, that is what deep learning is all about, adjusting weights in a model in response to the error until the error cannot be reduced anymore.

Optimization Algorithms

Some of the most common optimization algorithms include:

- SGD

- RMSPROP

- NONE

- NESTEROVS

- LINE GRADIENT DESCENT

- LBFGS

- HESSIAN FREE

- CONJUGATE GRADIENT

- ADAM

- ADAGRAD

- ADADELTA

Activation Functions

Activation functions determine what output will be generated by a node, based on the input. Some of the common examples include:

- TANH

- SOFTSIGN

- SOFTPLUS

- SOFTMAX

- SIGMOID

- RRELU

- RELU

- RATIONALTANH

- LEAKYRELU

- IDENTITY

- HARDTANH

- HARDSIGMOID

- ELU

- CUBE

Logistic Regression

When you have a deep neural network, with many, many layers, the last one plays a specific role. When you are working with labeled inputs, each example is classified by the output layer, with the likeliest label being applied. In the output layer, each node is representing a label. The node will switch on or off depending on the signal strength it gets from the parameters and input from the previous layer.

Every output node will have two possible outcomes – o or 1, which are binary output values. This is because an input variable will deserve to have a label, or it will not.  There is no in-between here.

When they work with labeled data, a neural network’s output will be binary. However, the input is, more often than not, continuous. That means the signals received by the network as an input can cover a wide value range. They may also have one or more metrics, depending on what type of problem it is looking to solve.

As an example, a recommendation engine must decide whether an ad should be shown or not – that’s binary, yes or no. But the input the engine uses to make its decision could also have information about the amount of money spent on Amazon over the last seven days, or how many times the customer went to the site.

So, in making its decision, the output layer has to take multiple signals, such as $50 spent on books and 19 visits to a specific website, and condense them down to a range somewhere between 0 and 1. That indicates whether the input should be given a label or not.

Logistic regression is the mechanism used for converting a continuous signal to a binary output. However, it is a bit of a misnomer, given that logistic regression tends to be more used on classification tasks than it is on regression. What it does is work out the probability that the inputs match the label:

F (X) =

Let’s delve into this formula.

Expressing a continuous input as a probability requires that the output be a positive result. Why? Because negative probabilities do not exist. This is why the input is expressed in the denominator as an exponent of e. Exponents force results to be more than zero.

Now think about the relationship between the exponent of e and the fraction of 1/1. As we already know, 1 is the upper limit of the probability, and the results cannot go above this without being too stupid even to consider.

Input x triggers a label and, as the input grows, the expression, e to the x, will shrink down close to zero. This leaves us with a fraction of 1/1 or, n other words, 100%. This means that we can be as close to absolute certainty as we can that the label is correct.  If your input has a negative correlation to the output, the value is flipped by the negative symbol on the exponent of e. As the negative signal increases, the quantity of e to the x grows larger, and the fraction gets even closer to zero.

Now imagine that we no longer have an exponent of x. Instead, we have the sum of the product of every weight, along with its input; this is the total signal that passes through the network. That is exactly what gets fed into the classifier network, in the logistic regression layer, which is the output layer.

Using this layer, we can set up a decision threshold. Above this threshold, the example is given a label of 1; below, and it gets a label of 0. It’s down to you what threshold you set; just be aware that lower thresholds give you more false positives while higher ones give you more false negatives.

Neural Networks and Artificial Intelligence

Neural networks are considered, in some circles, as a kind of brute force AI. This is because they begin with nothing and force their way through to produce as accurate a model as they can.  Yes, they are quite effective, but not everyone believes them to be an efficient method of modeling because they are unable to make an assumption about the functional dependencies that exist between the input and the output.

However, gradient descent does not find the best match by combining each weight with all the others. It works by reducing the weight space, which, in turn, reduces how many updates are done and how much computation is required.

In the next chapter, we are going to dive deeper into recurrent neural networks and LSTMs. Things are going to get a little heavier from hereon in so now might be a good time to take a break.

Chapter Four: A Deeper Look at Recurrent Neural Networks and LSTMs

A recurrent neural network, otherwise known as an RNN, is an artificial neural network (ANN) specifically designed to find patterns in data sequences, like time-series data (numerical) that comes from stock markets, sensors, government agencies, even down to handwriting, genomes and so on. Where the RNN, and its powerful subset, the LSTM, differ from all the other types of neural networks, is that they have something called a temporal dimension, taking sequences and time into account in their decisions.

Research has shown that RNNs are incredibly powerful and are the most useful neural network. We can even apply them to images, which can be broken down (decomposed) into patches and used as a sequence.

Because a recurrent neural network has a specific kind of memory and we all know that memory is something all humans possess, in varying degrees, you will note that there are analogies throughout this section to human brain memory.

More on Feedforward Networks

To get a good understanding of the recurrent neural network, you need to understand the feedforward network. Now, we talked about these in the last chapter, but we are going to go over them again here. Both RNNs and feedforward networks are named because of how information is channeled through mathematical operations that happen at the network nodes. The feedforward network sends information through, without it touching on any node more than once, while the recurrent networks cycle the information around a loop.

With the feedforward network, the network is given the input data, which then transforms them into an output. That output, using supervised learning, would be a label, a name the input is given. In basic terms, they take raw data, map it to categories, find patterns in the data and work out what the patterns are signaling, i.e., that an image should be given a label of ‘cat’ or ‘person,’ for example.

We train feedforward networks on labeled images, and the training continues until the network has the minimum error on the guesses it makes. The set of trained parameters or weights makes up the model, and the network uses this to look at unlabeled, unseen data and categorize it. We can give a trained feedforward network a random set of images; the first one it sees may not have any effect on how it will categorize the second one. For example, seeing an image of a dog will not automatically lead to the network seeing a cat in the next image.

You see, feedforward networks do not have a clue about the order in time; the only input these networks will consider is the data or example it sees at the time it sees it, nothing else. In terms of what they have seen before, the feedforward network will not remember it.

Recurrent Neural Networks

Conversely, the recurrent neural network doesn’t just consider the current input; it adds the previously learned information to the current input and works on that.

When a recurrent network reaches a decision at time step t-1, that decision will have an effect on the decision it makes next, in time step t. So, that means the network has two input sources – the present input and the recent past. These are combined, and the network will work out how it should react to new data, in much the same way as we humans do every day.

We can distinguish the recurrent network from the feedforward by the feedback loop that has a connection to previous decisions by the fact that the RNN takes its output and adds it to the input in the next layer. You often hear it said that a recurrent network has a memory, and giving a neural network memory serves a purpose -  the sequence contains information, and that is used by the recurrent network to do things that feedforward networks cannot do.

The sequential information is kept in the hidden state in the network, spanning several time steps as it moves forward to have an effect on each example’s processing. It is seeking out relationships between events that may have many minutes between them; these relationships or correlations are known as ‘long-term dependencies,’ because something that happened earlier in time is a function of and dependent on at least one event that happened before. If you want to, you can think about an RNN as a way of sharing weights across time.

Human memory circulates around the human body, but you can’t see it. As it roams invisibly, it affects how we behave without ever showing us its real shape. In the same way, information moves around a recurrent network’s hidden states. In the English language, there are many ways of describing memory feedback loops, and when a person is ‘haunted’ by something they did in the past, we are talking about the way things we did in the past impact on the present. The French also have a name for this, ‘Le passé qui ne passe pas ’ which translates to ‘The past that does not pass away.’

In mathematical terms, carrying memory forward can be described as:

Ht = ϕ (Wxt + Uht-1 )

At the time step t, the hidden state is h_t. This is a function that belongs to the input at x_t, the same time step, but a weight matrix, W, has modified it. Then it was added to the previous time step’s hidden state, h_t-1, and multiplied by U, which is its own matrix (hidden state to hidden state). This is called a transition matrix and is much like the Markov chain.

The weight matrices are actually filters, used to work out the level of importance to give to the current input and the previous hidden state. Backpropagation is used to return the generated error, and the weights are adjusted until the error is at its lowest.

The φ function squashes the sum of the hidden state and the weight input – this will be a tanh or a sigmoid function, standard tools for shrinking small or large values into a logistic space, and ensures that gradients work for backpropagation.

That feedback loop happens at every one of the time steps in the time series. Because of this, there is a trace of the previous hidden state and all of those that came before h_t-1 in every hidden state, and they will stay there for as long as the memory persists.

So, if you give a recurrent network a series of letters, it will take the first one to help it determine what it perceives what the next character will be. For example, if the first letter in the series were a q, it would be reasonable for the network to assume that the next letter would a u; the first letter of t could lead the network to assume that the next letter is an h.

Backpropagation Through Time (BPTT)

Ok, so we know that recurrent networks are used to provide accurate classification of sequential input. To do that, we need to rely on gradient descent and the backpropagation of error.

In a feedforward network, backpropagation goes backward from the last error, moving through outputs, weights, and inputs in every hidden layer. It will assign each weight with a percentage of responsibility for the error, and it does this by working out the relationship, or the partial derivative, ∂E/∂w, between the exchange rates. The learning rule, which is gradient descent, then takes those derivatives and uses them to adjust the weights in the direction that will decrease the error, either up or down.

The RNNs are reliant on backpropagation through time (BPTT), which is an extension of backpropagation. In this case, time is expressed by a series of calculations, ordered and well-defined, that link the time steps to one another – that is the only thing that backpropagation requires to work.

Whether a neural network is recurrent or otherwise, it is nothing more than a set of composite functions, such as f(g(h(x))), nested inside one another. When you add a time element to it, it simply extends that function series, which is used to work out the derivatives using the chain rule.

Truncated BPTT

This is an approximation of the full version, but it is the preferred version for long sequences. This is because the full version’s forward/backward cost for each update to the parameters rises steeply the more time steps it goes over. However, the truncated version does have a downside – it stops the gradient flowing back so far, and that means the network is unable to learn long dependencies like the full version can.

Vanishing and Exploding Gradients

Recurrent neural networks are not new. By the time the 1990s rolled in, recurrent neural networks were facing a major problem – vanishing gradients.  Straight lines express changes in x beside changes in y; gradients express changes in every weight concerning the error change.  If we have no way of knowing what the gradient is, we have no way of making adjustments to the weights in the right direction to reduce the error; at that point, the network will simply stop learning.

The result was that recurrent networks were hobbled, especially those that were looking for relationships between an output event and one or more events that happened multiple time steps earlier. The reason for this is that the networks cannot possibly know the level of importance to give to any remote input. Some of this is down to the fact that information that goes through a neural net goes through lots of different multiplication stages.

If you know anything about compound interest, you will know that, when you multiply a quantity frequently by a slightly larger amount than one, that quantity can become so large, it can’t be measured. But it’s also true the other way around, using a number smaller than one to do the multiplication. That’s why a gambler will soon lose all their money when they win back just 96 cents for every dollar they bet with.

The time steps and the layers in any deep neural network are all related to one another by multiplication. Because of that, the derivatives are highly susceptible to exploding or vanishing.

With an exploding gradient, every weight is treated as though it were a delicate butterfly but whose wings cause a hurricane in the distance. On the high end, those weights will become highly saturated, i.e., it is assumed that they are just too powerful. But the thing with exploding gradients is that they are quite easy to resolve because they can be squashed or truncated.  The vanishing gradients, on the other hand, are not so easy because they get too small for a computer to deal with, and networks cannot learn them.

Long Short-Term Memory Units (LSTMs)

Towards the middle of the 1990s, a new variation of the recurrent network was proposed to try and resolve the issue of the vanishing gradients. The proposal came from a pair of German researchers called Juergen Schmidhuber and Sepp Hochreiter; it was the RNN with the LSTM – Long Short-Term Memory.

The LSTM helps to preserve errors that may be backpropagated through time and layers.  By doing this, a constant error is maintained, allowing the RNN to carry on learning over more than a thousand time steps. As such, a channel is opened to link the causes and effects remotely. This happens to be a primary challenge to AI and machine learning because, more often than not, algorithms face environments containing delayed and sparse reward signals, kind of like life itself.

An LSTM has got information that falls outside the normal RNN flow inside a gated cell.  That information can be stored in a cell, written to it, or read from it, in the same way that we can read data in our computer’s memory. The cell will decide what it will store, and it will decide when that information can be read or written to, and when it can be erased, this is done through opening and closing gates.  However, where it differs from computer memory is that these are analog gates, not digital. They are implemented by sigmoids, using element-wise multiplication, and these sigmoids are all between 0 and 1. Analog has one advantage over digital - it is differentiable and that makes it appropriate for use with backpropagation.

The gates receive signals, and they act on them. In much the same way as the nodes in a neural network do, they block the information, or they pass it on. This is based on the import and strength of that information, which the gates filter using a set of weights. Like the weights used for modulating hidden states and input, these weights are adjusted by the learning process. In basic terms, the cells will learn when they can let the data in and when it can leave or when it can be erased, using the iterative process if guessing, backpropagation of the error, and using gradient descent to adjust the weights.

Have a look at the figure below; it shows you how the data goes through the memory cells and how the gates control it.

Courtesy of https://pathmind.com

There is quite a lot going on here, so if LSTMs are new to you, take your time over this diagram. Look at it carefully, and you will soon see it begin to give up its secrets.

Let’s start from the bottom. The triple arrow indicates the multiple points where information can flow into a cell. The combination of the current input and the past state of the cell is fed into the cell, and into each of the gates – there are three of them.  This determines the way the input will be handled.

The black dots indicate the gates. These determine whether the new input should be allowed in, whether the current cell state should be erased and/or whether the state should be allowed to impact the output from the network at the current time step. The memory cell’s current state is S_c, while the current input is g_y_in. Remember, these gates can be individually opened or closed, and, at every step, their current open state and closed state are combined. The cell may forget its state, or it may not; it may be written to, or it may not, and it may be read to or it may not, at every time step. Those flows are shown in the image. And the large letters in bold indicate each operation’s result.

The next figure shows you a recurrent neural network on the left compared to an LSTM cell on the right. Ignore the blue lines, but do take note of the legend at the side.

Courtesy of https://pathmind.com

What you must note is that the memory cells in an LSTM provide addition and multiplication with different roles during an input’s transformation. The plus sign in the center of the two diagrams is, in essence, the LSTM’s secret. It may seem very simple, but this change helps with the preservation of a constant error, especially when deep backpropagation is required.  It won’t determine what the subsequent state of the cell is through the multiplication of the current state and the new input. Instead, it adds them together, and that makes a whole heap of difference.

The input is filtered for input, output, and forgetting, and these are all done with different weight sets. A linear identity function is used to indicate the forget gate because, should the gate be open, the memory cell’s current state is multiplied by one, so it can propagate forward another time step.

So, why does an LSTM have a forget gate? Surely, their only purpose is linking a distant occurrence to the output? Well, on occasion, forgetting is a good thing.  If, for example, you are analyzing text and you reach the end of it, you might not have any reason to think that the subsequent document will have any relation to this one. As such, setting the memory cell to zero is the best option before the network takes in the first part of the next document.

The next figure shows the gates working; the closed gates are represented by straight lines and the open ones by blank circles. The forget gates are represented by the circles and lines that go horizontally through the hidden layer.

Courtesy of https://pathmind.com

You should be aware that the feedforward networks map inputs to outputs, one on one. However, the recurrent networks can map an input to many outputs, i.e., an image to multiple caption words. And they can map many inputs to many outputs, i.e., translation, or many inputs to one output, i.e., voice classification.

Capturing Remote Dependencies and Diverse Time Scales

You could also be asking yourself, what is the value of an input gate that provides protection to a memory cell, stopping new data from entering, or the value of the output gates that stop it having an effect on certain RNN outputs? Well, the LSTM could be thought of as letting the neural net operate on lots of different time scales at the same time.

Let’s consider a human being, their life. Let’s say that a time series is providing us with lots of data streams concerning that human life. At every time step, geolocation is an important feature for the next step, so that time scale must always be open to new information.

Let’s now say that our human is a good citizen, one who uses their vote every few years. So, when it comes around to election time, we would want to look closely at what they do before they go back to their normal life and leave the bigger issues behind. What we don’t want to do is let the noise created by geolocation have any effect on the political analysis.

Now let’s say that our human has a good daughter; we could use that to build a familial time series that looks for patterns in regular calls that happen every Saturday and more when the holidays arrive. This has nothing to do with geolocation or political cycles.

Lots of other data types are similar to this; music, for example, is polyrhythmic, while text often has recurring themes at different intervals. Economies and stock markets go up and down within certain times. All of these work on the time scales captured by an LSTM.

LSTM Hyperparameter Tuning

Some things you should bear in mind when you optimize your RNN parameters manually:

Chapter Five: Introduction To CNNs and Image Processing

A CNN is a convolutional neural network used mainly for image classification. Like the British TV game show, Catchphrase, the idea of a CNN is that it says what it sees. They also search images and cluster them in groups of similarity and also look at scenes and try to classify certain objects. As an example, we typically see CNNs, also called ConvNets, being used in facial recognition tasks, as well as those that identify cancerous tumors, street signs, and many other visual data tasks.

Image recognition is the type of task that has brought the CNN to the forefront of deep learning, the fact that they are so good at it. It could even be said that the CNN is responsible for the fame that deep learning is currently enjoying. The shot across the world’s bows came in 2012 when AlexNet trounced the competition in the ImageNet competition. AlexNet is now one of the most famous deep convolutional neural network architectures in the world. That led to a huge advancement in CNNs, and now we see them as the driving force behind robotics, self-drive vehicles, drones, medical diagnosis, security, and even treatment for those who have a visual impairment.

But the CNN is also good for more basic, ordinary tasks that have proven profitable over the years. These include OCR (optical character recognition), which digitizes text and makes it possible to use NLP (natural language processing) on both handwritten and analog documents where symbols are used in the images.

CNN isn’t just good for image recognition; they can also do text analyses and sound, particularly when the sound is represented as a spectrogram, and they work with graph convolutional networks to deal with graph data.

Images as 4D Tensors

A convolutional network will take in images and process them as tensors. A tensor is a number matrix with extra dimensions.

They are not terribly easy to visualize, though, so it’s best to use analogies to discuss them.

A scalar is a number, nothing more, nothing less. A vector is a list of several numbers, such as [4, 5, 6], while a matrix is a grid of numbers, rectangular, and covering multiple columns and rows.

In geometric terms, if a scalar is termed a zero-dimensional point, it follows that a vector is a 1D line. Then, a matric is a 2D plane, while a matrix stack is a 3D cube. When every element in the matrices is attached to a “stack of feature maps,” well, then we enter the 4D world.

Let’s see a 2 x 2 matrix for reference:

[2, 3]

[4, 7]

That is a 2D plane, and tensors will encompass all the dimensions that go beyond that. Picturing a 3D tensor is pretty simple; just imagine the numbers arranged in a cube. The tensor below is 2 x 3 x 2, flatly presented:

Courtesy of https://mathworld.wolfram.com/

That tensor would appear, in code form, as:

[[[2,3],[3,5],[4,7]],[[3,4],[4,6],[5,8]]]

And the figure below shows you a visual representation:

Courtesy of https://mathworld.wolfram.com/

So, what we are saying here is that a tensor is formed by nesting arrays within arrays, infinitely in some cases, That accounts for a random number of dimensions way more than we could ever spatially visualize.  Creating a 4D tensor simply means that each scaler is replaced with an array that has been nested another level deeper.

CNNs work with 4D tensors, much like the one in the next figure – note that the array is nested:

Courtesy of https://mathworld.wolfram.com/

Sometimes, you will see that multi-dimensional array, tensor, and NDArray are used synonymously – they all mean the same thing. The dimensionality of a tensor is known as its order – (2, 3, 4,…, n), so, for example, if you have a fifth-order tensor, it has five dimensions.

We can easily understand the height and width of an image. And depth is required because of the way colors are encoded. Take RGB (Red-Green-Blue); this produces a three-layer-deep image. Each of those layers is a channel, and convolution produces those feature map stacks we mentioned earlier. These maps come from the fourth dimension, not far down the road from time itself. A feature is nothing more than details about an image, such as curves or lines, and the CNN uses those to create maps.

So, rather than considering all images to be 2D areas, the convolutional network treats them as 4D volumes, and we’ll look at these ideas more as the chapter progresses.

The Definition of Convolutional

I guess it would help to know what it means, so you get a clearer idea of what a convolutional neural net is all about.  It comes from a Latin word, “convolvere,” which means “to convolve.” In turn, that means “to roll together.” In mathematical terms, convolutions are integral measurements on the extent to which functions overlap as they pass one another. Another way to think of this would be two functions mixed together through multiplication.

Courtesy of https://mathworld.wolfram.com/

In this, the green curve indicates the convolution of the red and blue as a function of t. That is the position the vertical green line indicates. The grey is the product, g(tau)f(t-tau) as another function of t, meaning that, as a function of t, the area shows the exact convolution.

In the middle of that graph, you can see the tall bell curve. The area beneath it is the integral. Another bell curve stands nearby, not so tall and narrow, and you can see that it wanders from the left to the right side. Where the product of the functions overlaps at points on the x-axis, this is the convolution; as you can see, they are rolled together.

Where image analysis is concerned, the underlying function, which is static, is the input image. This is what is being analyzed. The second function that moves is the filter because that is what detects the feature or signal in the input image. The functions are related via multiplication.

Next, you must understand that a convolutional neural net passes multiple filters over each image, and each filter will find a different feature. Early on in the layers, you could picture them as being a horizontal line, a vertical line, and a diagonal line, all creating a map of the edges found in the image.

The network will take the features, along with slices of the feature space for the image, and they are mapped individually. That is what creates the feature map, the places where the feature occurs. When they do this, they learn different parts of the feature space, and that makes the highly scalable and robust when it comes to feature engineering.

So, in effect, what a convolutional net is doing is searching. Imagine that you have a small magnifying glass, and you are moving it slowly across an image, left to right. When you come to the end of the first pass, you start again on the left. Only one thing is recognized in that window. For argument’s sake, we’ll say it’s a short vertical line, a stack of three dark pixels. That filter is moved over the image pixels, searching for a match.

Whenever it finds a match, it maps it to the correct feature space, where every vertical line’s location is recorded. CNNs will run multiple searches over a single image, looking for horizontal, vertical, and diagonal lines to ensure all the visual elements are found.

However, CNNs don’t just perform convolutions on input. Once the convolutional layer is done, the input is them put through a transform (nonlinear, usually rectified linear unit or tanh). This transform squashes the input values down to a range of 1-1.

How Do Convolutional Neural Networks Work?

Perhaps the most important thing to realize about a convolutional net is that it doesn’t see an image in the way we do.  So, you need to think differently about what these images mean as we feed them to the network for processing.

How do they see these images? As volumes, for example, a 3D object and not a flat one that can only be measured by height and width. Color digital images are encoded with RGB, mixing red, green, and blue to produce a color spectrum that we mere mortals can see. As such, these images are taken in and processed by the network as three different colors, one on top of the next.

So, when you feed a color image to a CNN, it sees a rectangular box with a height and width measured by how many pixels are registered on the two dimensions. But that box is also three layers deep, one each for red, blue, and green, all three of which are called channels.

We will refer to the images moving through the network in input and output terms. We will express them in mathematical terms, as 30x30x3 – a matrix with multiple dimensions. From one layer to the next, the dimensions will change, and we’ll discuss why this happens below.

Pay very close attention to each dimension’s precise measurements in the image volume. These are what the linear algebraic operations are based on for image processing.

For every individual pixel in an image, the R, G, and B intensity are expressed using a number. That number is an element in one of the stacked 2D matrices that make up the image volume; there are three of those.

The numbers are the initial features the CNN is fed, both sensory and raw features, and the CNN has to determine which numbers are significant in helping it work out the correct image classification.

Instead of looking at the pixels one at a time, a CNN will look at a square patch, passing it through a filter. The filter is another square matrix, not as big as the image but the same size as the patch. This is known as a kernel, and its job is to look for patterns within the pixels.

Let’s say that we have two matrixes, 30x30 and 3x3. The filter will cover an area of the channel surface area that is one-hundredth of the original image size.

What we want is the filter’s dot product. If both matrices show high values in identical positions, the output from the dot product will be equally high; if not, the dot product is low. By doing this, the dot product output indicates whether the pixel patterns in the filter and the underlying image match.

Let’s now say that we get an expression of a horizontal line from our filter. That line shows high values on the second row, and, in the first and third, we get low values. Now imagine that we started in the top left corner of the image, and the filter moves one step at a time across the image until it gets to the top right corner. The step size is called a stride, and you have the option of moving the filter a column at a time or in a larger step.

With every step, a new dot product is taken. The results from these go in an activation map, which is another matrix. The activation map’s width (number of columns) is the same as the number of steps taken for the filter to travel across the image. The larger the stride, the less number of steps, and that will result in the activation map being smaller. Now, that is an important point. The size of each matrix processed by the CNN and produced at every layer is in direct proportion to the computational cost and the time taken to train it. The smaller the stride, the more expensive and time-consuming it is, and vice versa.

If you superimpose a filter on the top three rows, it will slide over and then start anew, focusing on the next three rows. If the stride is three, a 10x10 dot product matrix is produced. The filter that represents the horizontal line may be applied to the R, G, and B layers of the original image. Then the activation maps, all 10x10, are added with the result that the aggregate map is the same size. That’s for the horizontal line.

Most, if not all, images contain lines that go off in all directions and contain lots of different pixel and shape patterns. Because of that, all the other filters should be slid over the image to try to find those other patterns. For example, you could search for  patterns in the image pixels. Those patterns would result in 100 activation maps stacked on top of one another and we would get a new volume of 10x10x100.

Look at the next figure, where the input image has been relabeled, as have the output activation maps and the kernels, just to make it all clear.

Courtesy of https://mathworld.wolfram.com/

That, my friends, is a convolution, nothing more than a fancy version of the multiplication used for signal processing. You could also consider the pair of matrices that produce the dot product as a pair of functions.

One of the biggest problems is that images are high dimensional, resulting in processing being time-consuming and computationally expensive. CNNs were designed as a way of reducing dimensionality in several ways, two of which are filter stride, which we discussed above, and downsampling, which we’ll look at next.

Max Pooling/Downsampling

Three different names are given to the next layer in the network – subsampling, downsampling, and max pooling. The downsampling map is fed with the activation maps. Like the convolutions, this is done a patch at a time.  Max pooling will take the biggest value from one of the patches in an image, puts it inside a new matrix, and adds in the largest values (max values) from the other patches. The remaining information in the activation maps is discarded.

The only image locations that get preserved are those that have a strong correlation to the maximum value or feature; those values are then combined, forming a low-dimensional space.

A great deal of information about the lower values gets lost here, and, right now, much research is going in to find better ways to do this. Downsampling has a much better advantage; because that information is lost, loss processing and storage space are needed.

Alternating Layers

In the figure below, you can see the transformation sequence in a standard convolutional neural network:

Courtesy of https://mathworld.wolfram.com/

Looking at it from left to right, you can see:

- The input image fed to the network, which looks for features. The small light rectangle you see is the filter that gets passed over the image.

- A stack of activation maps, one for every filter you use. The bigger rectangle is a patch for downsampling.

- The activation maps after downsampling has condensed them.

- New activation maps; these come from the filters being passed after the previous downsampled stack.

- Downsampling round two, resulting in the second lot of activation maps being condensed.

- The final one is a fully connected layer; this is used for classification of the output, with each node having one label.

As more information gets lost, the convolutional net is processing patterns that become more abstract and begin to look nothing like the patterns that a human being can recognize. So don’t be disheartened if, as your CNN gets deeper and deeper, it offers less easy intuitions – that’s just their nature.

In chapter six, we’re going to introduce you to deep reinforcement learning.

Chapter Six: An Introduction to Deep Reinforcement Learning

Deep reinforcement learning is a combination of an artificial neural network and reinforcement learning. The architecture enables agents, defined by software, to learn what the best actions are to get to their goals, and this is done in a virtual environment. In simple terms, it combines target optimization with function approximation and maps a state-action pair to an expected reward.

Phew! Yes, that is a bit of a mouthful, but all will become clear as you work through this chapter.

Neural networks have played a large part in AI breakthroughs for time-series prediction problems, machine translation, and computer vision. However, we can also combine reinforcement learning algorithms with neural networks and create mindblowing results, like AlphaGo from Google’s Deepmind, which went on to beat the human world champion of the popular board game, GO.

Reinforcement learning is all about totally goal-oriented algorithms. These algorithms learn to attain goals, which are usually complex objectives, or they learn how to maximize, using multiple steps, over a specific dimension. An example of that is maximizing how many points they win in a game over multiple steps.

A reinforcement learning algorithm can start from nothing; given the correct conditions, superhuman performance is perfectly attainable. When they make a mistake, the algorithm is penalized but, when the decision is right, they are rewarded, and that’s how they learn – through reinforcement.

Where deep neural networks are incorporated into a reinforcement learning algorithm, they can beat a human expert playing video games, and even board games. Sure, that might sound somewhat trivial, especially if you are not a gamer, but it’s a fantastic improvement over the previous accomplishments of reinforcement learning, and things are progressing fast from there.

We can use reinforcement learning to solve the tricky issues of putting immediate actions together with the delayed returns produced by them. Like human beings, these algorithms often have to wait to see if their decisions bear fruit. They are working in an environment of delayed return, and it isn’t always easy to work out how actions and outcomes correlate over multiple time steps.

It’s not unreasonable to assume that these algorithms will gradually increase in performance in some real-life environments when they have a series of different actions to choose from, and not the limited actions available in short, repeatable video games. We expect these algorithms, over time, to increase in value in what they can achieve in the real-world and, given enough data and computing power, they could even become one of the most promising and relevant paths to very strong artificial intelligence.

The Definitions of Reinforcement Learning

We can understand reinforcement learning by using five concepts:

- Agents

- Environments

- States

- Actions

- Rewards

We’ll explain these in a moment, but first, understand that we use capital letters to indicate a set of things, while lower case letters are used to indicate an instance of a thing. For example, A indicates a set of possible actions, and a is one action in that set.

- Agent – These take actions, i.e., a video game character navigating the game, a drone dropping off a parcel, and so on. This is the algorithm, and, for a real-world example, you would be the agent.

- Action (A) – This is the set of possible moves an agent may make. In all honesty, this is pretty self-explanatory, but we should make it clear that an agent will usually make a selection from a list containing possible, discrete actions. In the video game, for example, that could include jumping, running, crouching, sliding, and so on. With the delivery drones, it could be accelerations and velocities in 3D space.

- Discount Factor – This is multiplied to reduce the effect of the reward on the action the agent chooses. The multiplication is done with the rewards the agent discovers in the future, but why is this necessary? It’s done to ensure that future rewards don’t have a higher value than the current rewards. This factor is often expressed using γ (lower-case). This is the Greek letter for gamma, and, if γ is 8, then after three time steps, the reward of 10 points becomes 0.8^3 x 10. By applying a discount factor of one, future rewards would have the same value as the current ones. In effect, we are fighting delayed gratification.

- Environment – This is the world the agent moves through, the world that responds to the agent. The input the environment takes is the current state and action for the agent, and it returns an output of the reward and the subsequent state. If you were the agent, that environment could be societal rules or the law of physics that take your actions, processes them, and works out what the consequences are.

- State (S) – The state is an immediate, concrete situation the agent may find itself in. For example, it could be a certain place and time, an instant configuration that places the agent relative to prizes, enemies, obstacles, tools, or anything else significant to the situation. It could be what the environment returns as the current situation or even a future situation.

- Reward (R) – This is the feedback used to measure how successful, or otherwise, an agent’s actions are in a given state, For example, a video game character may earn points whenever they touch something specific. From the given state, the agent provides output to the environment. That output is an action, and the environment returns the new state for the agent, a result of acting on the previous one, along with any available rewards. The rewards can be immediate ones or delayed ones and are effectively used for evaluating the action.

- Policy (π) – This is the strategy employed by the agent, using the current state as the basis, to work out what the next action is. It will map a state to an action that promises to give the biggest reward.

- Value (V) -   This is the long-term return that is expected, along with a discount, and not the short-term reward (R), that was explained earlier. It is defined as “the expected long-term return of the current state under policy π.” Rewards are discounted, which means their estimated value is reduced, the further in the future they happen.

- Q-Value or Action-Value )Q) - This is much like Value but with an additional parameter, which is a, the current action. Qπ(s, a) references an action’s long-term return when action a is taken under policy π from s, which is the current state.  Q will map a state-action pair to a reward. Be aware that there is a difference between Q and policy.

- Trajectory – This is a sequence containing states and the actions that have an influence on those states. It comes from the Latin word that means “to throw across.”

- Key Distinctions – A reward is a signal given immediately in a given state; value is the sum of all the rewards that could come from that state. Value is defined as a long-term expectation, and reward is something good given immediately. It is possible to have a state where there is a divergence between value and reward. You could get an immediate reward that isn’t quite so good now but has the potential to be great over the long-term, or you could get an immediate reward that is fantastic now, but, over the long-term, it doesn’t hold its value. That is why reinforcement learning tries to predict the value function and control it, rather than focusing on immediate rewards.

So, an environment is a function that can transform an action in the current state to the next state, with a reward. An agent is a function that can transform that new state and its reward into yet another action. We can know the function of the agent, and we can set it but, in the bulk of situations where reinforcement learning is useful, we will not know what the environment’s function is.  Think of it as a large black box where all we can see is the input and the output. Think of it as the internet – most of us know what it is and what it does, but we don’t know how it works. With reinforcement learning, we have a representation of an agent trying to work out what the environment’s function is in such a way that actions can be sent to the environment to get the maximum reward.

Courtesy of http://incompleteideas.net/

Above, you can see a feedback loop. The subscripts tell you that we have two time steps, t, and t+1. Each of these refers to a different state, the current state at each step. You can only consider reinforcement learning in terms of sequences of state-action pairs that happen one after another.

In reinforcement learning, actions are judged by what results they produce. This kind of learning is entirely focused on goals, with just one aim – to learn a sequence of actions that let an agent achieve the goals or get the best out of the objective function. Let’s see how that works, using some examples:

- Video Games – the idea with any video game is to get to the end of the game with maximum points. Every point earned affects how the agent behaves next; for example, the agent might learn that to earn as many points as possible, it must dodge objects, touch things, or shoot the enemy.

- Real World – the goal could be that a robot must travel from point one to point two. Every step closer to point two that the robot gets could be counted as a point.

Have a look at this example; it shows an objective function in reinforcement learning, i.e., the way the goal is defined:

The reward function, r, is summed over t, which are the time steps. This function is calculating the total potential reward for going through, for example, a game. That state for a specified time step is x, and the action that happens in the state is a. The reward function, r, is for x and a. We haven’t forgotten about γ; we’re just ignoring it for now.

Where reinforcement learning differs from unsupervised and supervised learning is in the way the outputs are interpreted.  Let’s see that difference by talking about what each type of learning will learn about a thing:

- Unsupervised – this thing is like another thing. In other words, unsupervised algorithms will learn the similarities but without any names (labels). As a result, they see the inverse and anomaly detection can be performed by recognizing out-of-the-ordinary details.

- Supervised – this thing is a “fish taco.” In other words, the algorithm will learn correlations between data and its labels. They can only be trained on labeled datasets because the labels supervise the learning and correct algorithms that make incorrect predictions.

- Reinforcement – that thing tastes good and will help you live longer, so eat it. This type of learning has actions that are based on both short-term and long-term rewards. It is a type of supervised learning but in an environment where feedback is sparse.

Domain Selection

One of the easiest ways to visualize an agent for this reinforcement learning is to think of a blind person, with just a white cane and their ears to help them get around. An agent only has a small window through which to perceive the environment they are in, and that window may not even be that efficient.

The art of deciding the input types and the feedback the agent should be paying more attention to is not an easy one. It’s a process called domain selection, and this is pretty much ignored by algorithms tasked with learning a video game for one reason – they are working in a man-made and limited environment. As such, video games can be seen as a kind of laboratory, a sterile environment where we can test ideas about reinforcement learning.

With domain selection, we require human intervention in the form of decisions. These are typically based on real knowledge or solid theories regarding the problem that needs solving. For example, if you need to choose the input domain for an algorithm for a self-drive vehicle, you might opt to include GPS data, cameras, and radar sensors.

The State Action Pairs and the Complex Probability Reward Distributions

Reinforcement learning has one primary goal – to choose the best action for a specified state. The actions need to be ranked, and each given a value that is relative to the others. These actions are dependent on the state, and that means what we are looking at is the value for a state-action pair. Let’s see some examples to show that an action’s meaning and value are dependent upon the state the action is taken in:

- Marriage – the action is to marry a person. If you are 18 and you marry a 37-year-old, it will have a different meaning than a 90-year-old marrying a 37-year-old. Both will have a different motivation behind them, and both will have a different outcome – probably.

- Shouting, “Fire! ” – if you did this in a theatre crowded with people, it would have a different meaning than if you shouted it in front of a squadron of soldiers carrying rifles.  We need to know the context to predict the outcome of an action.

State-action pairs are mapped to the values we expect to see produced with the Q function that we talked about a while back. The input to a Q function is the state and action of the agent; this is then mapped to a probable reward.

In reinforcement learning, the agent is taken through a sequence of state-action pairs. It watches to see what rewards are given and then adapts the Q function predictions to the given rewards until it can make the right prediction for the best path the agent should take. We call this prediction a “policy.”

With reinforcement learning, we are trying to model complex probability reward distributions related to multiple state-action pairs. This is just one of the reasons why we pair reinforcement learning with something like the Markov decision process, which infers its properties by sampling from a complex distribution. This bears a close resemblance to a problem that led to the Monte Carlo method being created by Stan Ulam – attempting to infer the probability of a Solitaire hand being successful.

Statistical approaches are often considered to be confessions of ignorance.  Some phenomena, such as sociological, political, biological, even those related to a board game, are immensely complex, and that makes it nigh on impossible to use first principles to reason from. Statistics really is the only way to study these, taking one-dimensional events, measuring them, and trying to find correlations between them, even if we don’t really understand how they relate to one another. Reinforcement learning and deep neural nets are two ways to do this, using a sampling method to get the information we want from the data.

Reinforcement learning algorithms spend time with, say, the Markov decision process, learning to estimate the probability reward distribution over a series of state-action pairs. After that, it may then repeat those actions that result in rewards and not try out any alternatives. There is a measure of tension between continuing to exploit the known rewards and continuing to explore, to find new actions that also pay off. An oil company has two jobs – to pump oil from a known field while, simultaneously, drilling to find new fields. The reinforcement learning algorithms can, in the same way, be forced to explore and exploit, to make sure that they don’t favor a known winner over finding new actions that reward.

Reinforcement learning is iterative, and, in some of the more interesting applications of it, it will not know what the state-action pairs will produce in the form of rewards. Instead, it will iterate through the states repeatedly, learning the relationships to improve itself.

Machine Learning With Time

It would be fair to say that algorithms are methods that will help us aggregate time lessons more quickly. These algorithms don’t have the same relationship with time that we humans do. Instead, they can repeatedly go through the same states, using different actions to experiment with, until they learn the best actions form each state. In effect, it’s very much like Groundhog Day – they start out stupid and, as they repeat over and again, they get wiser.

Outside of the popular movie, humans cannot experience this Groundhog Day, so it follows that, because they do, the reinforcement learning algorithms can potentially learn far more, far better than humans do.

These algorithms have one primary advantage over a human being, and it doesn’t come from their inherent nature so much as it comes from the fact that they can exist on multiple chips at the same time, that they can keep going 24/7, without the need to rest. That is why they can learn so much more, so much faster and better than humans. Take the DeepMind algorithm, AlphaGo, that was trained to play the board game, GO. That algorithm has played far more games than any of us mere mortals could ever hope to play in 100 lifetimes.

Neural Networks and Deep Reinforcement Learning

So, where do our neural nets fit into all of this?

A neural network is a function estimator or approximator. These have proven incredibly useful where the state or action space in reinforcement learning is too big and cannot be completely known.

We use neural nets to estimate the value or policy function. In other words, they can learn how to map a state to a value, or a state-action pair to a Q value. Instead of storing, indexing, and updating all the potential states and values in a lookup table, which is not really possible when the problem is a large one, the neural net is trained on samples from the action or state space. It learns to predict the level of value those samples have relative to the target.

To estimate the functions relative to the inputs/outputs, they use coefficients. They learn to find the right ones to use by adjusting them iteratively, on each pass, along gradients that promise reduced errors.

We can use convolutional networks in reinforcement learning; these learn to recognize the state when the agent’s input is visual, i.e., a video screen. In short, they do the task they were designed for – image recognition.

However, a convolutional network will interpret an image differently in reinforcement learning than it does in supervised learning. In the latter, an image is given a label (names matched to pixels.)

Courtesy of https://pathmind.com

What it does is it looks at probabilities and ranks each label in order of best-fit to the image.  Show it an image of a horse; it might determine it is 75% likely to be a horse, 55% likely to be a donkey, and 25% likely to be a wolf.

With reinforcement learning, if you give the convolutional network an image representing a state, it will give the actions a rank based on their potential to perform in the given state. It could, for example, predict that sprinting left will return 10 points, jumping 6, and sprinting right, no points at all.

Courtesy of https://pathmind.com

In the image above, you can see what policy agents do. They map the state to the best possible action. As an equation, this can be defined as:

If you remember from earlier, this is not the same as Q, which will map a state-action pair to a reward. Specifically, it will map that state-action pair to the best possible combination of an immediate reward and a future reward that could be received later on by another action. We can see that defined below:

Once the Q function has assigned the values to the rewards, it will choose the state-action pair that has the highest value.

At the start of the reinforcement learning, the coefficients may be randomly or stochastically initialized. Neural nets can take feedback from the environment and use it, along with the difference between the expected and ground-truth rewards, to adjust the coefficients and make improvements to how it interprets the state-action pairs.

We can see this feedback loop as being similar to the backpropagation of errors used in supervised learning. However, in supervised learning, the ground-truth labels it wants the network to predict are known right from the start. It differs in that it creates the model for mapping the images to their correct names.

Reinforcement learning is dependent upon the environment to give it a scalar number as the response to every new action. The environment returns rewards, and these can be delayed, varied, and may not even be affected by an unknown variable. This brings a certain level of noise to the loop, which leads us to an even more complete Q function expression. This one will take the immediate rewards an action generates into account, as well as the delayed rewards that could come multiple steps down the line.

The Q function is recursive, the same as a human being is. When you call the function on a specified state-action pair, predicting the next state’s value requires a nested Q function; in turn, that depends on the next state’s Q function, and so it goes on.

Hopefully, you understood all that. If not, don’t be afraid to go over it again as many times as you need to. In the next couple of chapters, we’re getting down and dirty to build our own neural networks, starting with using Keras and TensorFlow to build a deep learning network that predicts employee retention.

Chapter Seven: Let’s Build a Deep Learning Model with Keras and TensorFlow

Keras is one of the most popular Python neural network APIs, running on top of Theano, CNTK, and TensorFlow. Being a high-level abstraction of all those frameworks, it ensures that we can experiment much easier and much faster. Keras is also modular, meaning it is much easier to implement – models can easily be extended simply by adding more modules.

TensorFlow is a machine learning library, open-source, perfect with computations that have arrays. That makes it one of the best choices for the deep learning model we’re going to build here. TensorFlow allows you to execute code on a GPU or a CPU, a great feature when you are working with huge datasets.

What we’re going to build in this chapter is a deep learning model that predicts the probability of a given employee leaving their employment; most companies consider employee retention to be one of the most important factors.

You will need to download this Kaggle dataset for this; its features are used for measuring employee satisfaction. You will also use the sequential layer in Keras to build all the different layers.

Here’s what you need:

Before you start, you will need:

- The Anaconda development environment – once installed, set up your environment, using the following command - conda create --name my_env python=3

- Jupyter Notebook installed – Anaconda typically installs this

- Some experience with machine learning and with Python programming language

Step One – Data Pre-Processing

Pre-processing your data is a vital step as it prepares the data so the deep learning model can take it and work with it. If your data contains categorical variables, they must be converted to numbers – the algorithm can only take numerical figures. Categorical variables contain quantitative data that is represented by names. In this first step, we’ll use the best data manipulation library Python has to offer to load the dataset. That library is pandas.

Before starting the pre-processing, your environment must be activated and you must make sure you have all the right packages installed.  The best way is to install Keras and TensorFlow using conda because this will ensure all the necessary dependencies for these are installed.

Go to the environment you should have created once you installed Anaconda by typing the following command:

conda activate my_env

Now you can install TensorFlow and Keras using this command:

conda install tensorflow keras

Open the Jupyter Notebook by using the following command:

jupyter notebook

When you get into the Notebook, choose the file called anaconda3 and go to the top of the screen; click New and then start a new notebook by clicking Python 3.

Next, we need the modules for this particular project, and the dataset needs to be loaded into a cell in your notebook. We need the pandas module so we can manipulate our data, and we need the numpy module so we can convert the data to numpy arrays. We also need all the string-format columns into numerical values so the computer can process them.

Type the following into a cell in the notebook and click on Run:

import pandas as pd

import numpy as np

df = pd.read_csv("https://xxxxxxx/HR_comma_sep.csv")

What you have done here is imported pandas and numpy, then pandas was used to load in the dataset. Where I have placed the row of x’s, insert the location on your computer where the dataset is saved

If you use the head() command, you can get a quick look at the dataset; it will show you the first five entries. Type the command into a cell in the notebook:

df.head()

Courtesy of https://digitalocean.com

The next step is converting those categorical columns into numbers. How? By converting each one to a dummy variable, which is typically made up of zeros and ones, indicating the respective absence or presence of a categorical feature. To avoid the “dummy variable trap,” you need to drop the first dummy.

Note

This trap happens when you have at least two variables with a high correlation, leading to a poor-performing model. By dropping one of the dummies, you always have N-1 dummy variables. You can drop any of them; it doesn’t matter which one so long as you have N-1 dummy variables. A good example of this is a switch that turns on and off. When the dummy variable is created, you will get a pair of columns – one for on and one for off. Either can be dropped; if the switch is not on, then it must be off and vice versa.

So, in the next cell, type in this code and execute it:

feats = ['department','salary']

df_final = pd.get_dummies(df,columns=feats,drop_first=True)

The first line, feats = [‘department’, ‘salary’] is defining the pair of columns we want dummy variables created for, while the second line, pd.get_dummies(df,columns=feates,drop_first=True) generates the required numerical variables. The defined feats are converted from categorical values to numerical.

Okay, the dataset is loaded, the department and the salary columns are properly converted so the model can accept them; now, we need to split our dataset into two more – a training dataset and one for testing.

Step Two – Splitting Your Dataset

Splitting the dataset is a requirement as it allows us to use part of the dataset for training and some for testing the model’s performance. This is common with deep learning models, and we use Scikit-learn to do it.

The split must be implemented in the dataset, thus ensuring your model cannot access the test data while it is training. That way, the model can only learn from the training dataset, and its performance then tested with the remainder of the data, which is essentially unseen. If your model saw the test data while you were training it, all it would do is memorize the outcomes we expect to get. As a result, the model’s predictions would not be accurate on unseen data.

First, we need a new module imported, the train_test_split module found in Scikit-learn. Go to the next cell in your notebook and run this code:

from sklearn.model_selection import train_test_split

Once the module has been imported, we can start predicting the probability of an employee leaving; this is done in your dataset’s left column, so you must make sure your model cannot have any contact with that column.

Put the following code into a notebook cell, ensuring the left column is dropped:

X = df_final.drop(['left'],axis=1).values

y = df_final['left'].values

Because the model will expect the data to be in arrays, the second line above converts it to numpy arrays, using the attribute of .values.

Now it’s time to split the dataset. We want 70% of the data for training purposes and the remaining 30% for testing purposes. You can also use an 80/20 split, so long as the bulk of the data is for training.

In the next cell, type and run the code below:

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)

In this step, our data was converted to a numpy array, so that Keras can work with it. We also split the data into two separate datasets, one for training, one for testing. Later on, this data will be given to the model, but, before that, the data needs to be transformed.

Step Three — Transform the Data

When you build a deep learning model, good practice says that you should scale the dataset; this is done to ensure more efficient computations. The StandardScaler is used to ensure the values in your dataset have a unit variable and a mean of zero. Doing this means the dataset can be normally distributed.

The features are scaled to make sure they are in the same range, using the StandardScaler from Scikit-learn. The values will feature a standard deviation of 1 and a mean of zero. This is one of the most important steps because the features you are comparing have all got different measurements.

To do all this, put the following code into a notebook cell and execute it:

from sklearn.preprocessing import StandardScaler

sc = StandardScaler()

X_train = sc.fit_transform(X_train)

X_test = sc.transform(X_test)

The first thing we did was imported the StandardScaler; then, we called an instance of it and finally scaled both the training dataset and the test dataset using the fit_transform method from StandardScaler.

All your dataset features are scaled, so now we can begin to build our network.

Step Four — Build the Neural Network

To build our model, we need to import Keras; by default, this will run on top of TensorFlow. Then we’ll want a Keras module named Sequential; this is to initialize our new network. Lastly, to give our network some layers, we’ll need the module called Dense, too.

With deep learning models, we would typically specify three types of layers. These are:

- The Input Layer – all the dataset features are passed to this layer. No computation will happen in the input layer; it is purely to pass those features onto the hidden layers.

- The Hidden Layers – these fall between the input and the output layers. There is no limit on how many there are, and these are the layers that do all the computation. The information is then passed to the output layer.

- The Output Layer – this layer is where you get all your results from once the model has been trained. It is where the output variables are produced.

Input the following code into a notebook cell and run it to import the three modules you need:

import keras

from keras.models import Sequential

from keras.layers import Dense

The Sequential module is used for initializing linear stacks of layers. Now, because we are dealing with a classification type of problem, you will need a classifier variable. Classification problems involve using labeled data to make predictions on unseen data.

To create the classifier variable,  the following code must be added to your notebook:

classifier = Sequential()

Now that the classifier has been initialized, we can begin to add some layers to our network. The code below should be run un your notebook:

classifier.add(Dense(9, kernel_initializer = "uniform",activation = "relu", input_dim=18))

The .add() function is used for adding layers and specifying the necessary parameters, of which we have four:

- Parameter One – this indicates how many nodes should be in the network. The neural network is formed through the connections between the different nodes. A primary strategy for determining how many nodes are needed is to average the number in the input and output layers.

- Parameter Two – this is kernel_initializer. When these models are fit, the weights or coefficients are initialized – not to zero but close to it. You need a uniform distribution initializer to do this,  and the one that deals with initializing weights is kernel_initializer.

- Parameter Three – this is an activation function, and this is what the model learns through. Typically, there are both non-linear and linear activation functions, and, in this case, we are using relu. It generalizes on data very well and is a non-linear activation function – we don’t tend to use activation functions on these problems because they are straight-line functions, and that doesn’t work for this.

- Parameter Four – input_dim is used to represent how many features your dataset has.

The next step is adding the output layer so we can get some predictions:

classifier.add(Dense(1, kernel_initializer = "uniform",activation = "sigmoid"))

In the output layer, we have these parameters:

- Parameter One – this indicates how many output nodes there are. You would expect a single output in this case, if an employee were to leave the organization. As such, only one output node is specified.

- Parameter Two – the kernel_initializer requires a different activation function, the sigmoid. This will allow us to obtain the probability of an employee leaving. If by some chance, you had two categories that needed dealing with, the SoftMax activation function would be a better bet; this is a sigmoid variant.

The next thing the network needs is gradient descent, a type of optimization strategy that helps cut down on errors throughout the training. Gradient descent is how a neural network’s weights, randomly assigned, are adjusted; the cost function is reduced to achieve this. This is a measure of the efficiency and accuracy a network performs at, based on what output we expect from it.

Gradient descent is designed to reach the point of least error. We do that by finding the location the cost function is at when at its lowest. This is called the local minimum. Gradient descent allows us to specify a point and find the slope at it; we also want to know if it is a positive or negative slope – the descent part takes you down into the cost function’s minimum. There are lots of different optimization strategy types, but the most popular one, which we will be using, is called adam.

Type the following code into a cell in your notebook and run it:

classifier.compile(optimizer= "adam",loss = "binary_crossentropy",metrics = ["accuracy"])

The compile function is used to apply gradient descent, and that function takes these parameters:

- Parameter One - optimizer – the gradient descent

- Parameter Two - loss – a function used in gradient descent. Because we are dealing with a binary classification task, the loss function we’ll use is binary_crossentropy.

- Parameter Three – metric – used for model evaluation. We want our evaluation to be on the prediction accuracy.

Now, we can fit our dataset with the classifier, and we use a method called fit() from Keras, Put the code below into a notebook cell and execute it:

classifier.fit(X_train, y_train, batch_size = 10, epochs = 1)

We provide the fit() method with four parameters:

- Parameter One – the training set with all of its features

- Parameter Two – the column the predictions are being made on

- Parameter Three – batch_size, representing how many samples will be put through the network for each training pass

- Parameter Four – epochs, representing how many times the dataset goes through the network. The higher the number of epochs, the longer it takes for the model to run, but this provides much better results.

Our model has been created and compiled, and it’s been fitted to the dataset. Now, we can use our model to start making some predictions on unseen data – that’s the test set we created earlier when the dataset was split.

Step Five — Use the Test Set for Predictions

We want to put our model to the test, so we use the test dataset we created earlier. Keras has a useful function, called .predict(), that helps us make some predictions. Put the code below into a cell in your notebook and execute it to start making some predictions.

y_pred = classifier.predict(X_test)

The classifier has already been trained on your training set, so the code will allow the model to use what it learned to make the predictions on the test dataset. The predictions will tell you the probability that a given employee will leave, using a 50% or higher probability to indicate a higher chance of that happening.

Let’s set the threshold by adding this code to the notebook:

y_pred = (y_pred > 0.5)

Here, you have used the .predict() method to make some predictions, setting the threshold that determines the probabilities. Now we want to see how well the model did on those predictions; we need a confusion matrix to do that.

Step Six — Use a Confusion Matrix

The confusion matrix will help us determine how many right and wrong predictions the model has made. These are often termed as an error matrix and is a square matrix indicating how many true and false positives (tp and fp) and how many true negatives and false negatives (tn and fn) the classifier made.

True positives indicate the model has made the correct predictions on the positive class, while false positives indicate correct predictions on negative classes. False positives and false negatives indicate incorrect predictions on positive and negative classes, respectively.

Scikit-learn has the confusion matrix that we want to use, and we import it by adding the following code to the notebook cell:

from sklearn.metrics import confusion_matrix

cm = confusion_matrix(y_test, y_pred)

cm

The output is

array([[3305,  106],

[ 714,  375]])

The output you see from the confusion matrix should indicate that your model made 106 + 714 incorrect predictions and 3305 + 375 correct ones. You can calculate the model’s accuracy with this sum – (3305 + 375) / 4500. The dataset has 4500 observations in it, and the accuracy from this is 81.7%. That’s not a bad rate for a first go-around

The confusion matrix output means that your deep learning model made 3305 + 375  correct predictions and 106 + 714  wrong predictions. You can calculate the accuracy with: (3305 + 375) / 4500 . The total number of observations in your dataset is 4500. This gives you an accuracy of 81.7%. This is a very good accuracy rate since you can achieve at least 81% correct predictions from your model.

Your model has been evaluated with a confusion matrix; next, we look at how to make a single prediction.

Step Seven — Make One Prediction

Now we are going to make one prediction using details of a specific employee. This is done with a prediction of the probability that one employee will leave the organization. We give the .predict() method the specific details, scaling them and converting them to a numpy array.

The following code should go in a notebook cell:

new_pred = classifier.predict(sc.transform(np.array([[0.26,0.7 ,3., 238., 6., 0.,0.,0.,0., 0.,0.,0.,0.,0.,1.,0., 0.,1.]])))

Those are the features of one employee. As you saw in step one, the features are employee satisfaction, last evaluation, how many projects the employee has been involved in, and so on. And, as we did in step three, the features must be transformed so the model can accept them.

Use the code below to put in a threshold of 50%

new_pred = (new_pred > 0.5)

new_pred

Now we know that any probability over 50% is an indication the employee will leave. Looking at the output, it’s clear this is one happy employee who has no plans to leave:

array([[False]])

You can set whatever threshold you want. If you set it to 60%, for example, you would add this code:

new_pred = (new_pred > 0.6)

new_pred

We still get an indication that the employee isn’t going anywhere:

array([[False]])

This step has shown you how to provide features of an employee to make one prediction. Next, we want to know how we can make our model more accurate.

Step Eight — Improve the Model Accuracy

The more you train your model, the wider the variety of results you will get. And each training session will provide a high variance of accuracies. We can solve this issue by using K-fold cross-validation. Typically, K is set as 10; the first nine are for training and the tenth for testing. This continues until every fold has been used, and every iteration provides an accuracy. The overall model accuracy is the average of all of them.

K-fold cross-validation is implemented using the KerasClassifier from Keras. First, we import a cross-validation function called cross_val_score and KerasClassifier. Input the code below into the notebook cell and then run it:

from keras.wrappers.scikit_learn import KerasClassifier

from sklearn.model_selection import cross_val_score

Next, add the code below; this creates the function that gets passed to the KerasClassifier:

def make_classifier():

classifier = Sequential()

classifier.add(Dense(9, kernel_initializer = "uniform", activation = "relu", input_dim=18))

classifier.add(Dense(1, kernel_initializer = "uniform", activation = "sigmoid"))

classifier.compile(optimizer= "adam",loss = "binary_crossentropy",metrics = ["accuracy"])

return classifier

We created the function, which is an argument expected by the classifier.  It is a rapper similar to the one we used earlier and the parameters passed to it are also similar. The classifier is initialized in the function with the Sequential() method; then, the input layer and output layer are added using Dense. Lastly, the classifier is compiled and returned.

The code below passes the function to the KerasClassifier, so add it to your notebook cell:

classifier = KerasClassifier(build_fn = make_classifier, batch_size=10, nb_epoch=1)

KerasClassifier will take three parameters or arguments:

- build_fn – this si the neural network design function, similar to what we used earlier

- batch_size – indicates how many samples are passed in every network iteration

- nb_epoch – indicates how many epochs will be run by the network

Now the cross-validation must be applied, and this is done with cross_val_score from Scikit-learn. Put the code below into your notebook and execute it:

accuracies = cross_val_score(estimator = classifier,X = X_train,y = y_train,cv = 10,n_jobs = -1)

Because we have specified ten folds, we get ten accuracies from the function. So, the accuracy variable is assigned to the function, and it gets used later to work out the mean accuracy. The following parameters are passed to it:

- estimator – this is the newly-defined classifier

- X – the features for the training set

- y – the value the training set will predict

- cv – indicates how many folds there are

- n_jobs – indicates how many CPUs are to be used. If you specify -1, all available CPUs would be used.

The cross-validation has been applied; now, we can work out the mean accuracy and the variance. The code below goes into your notebook and is run:

mean = accuracies.mean()

mean

The output should be

0.8343617910685696

This indicates an accuracy of 83%

To do the variance, use this code:

variance = accuracies.var()

variance

The output is

0.0010935021002275425

As you can see, we have a variance of 0.00109; this is low, which indicates very good performance from our model.

We used K-Fold cross-validation to improve model accuracy. Next, we want to look at how to reduce overfitting.

Step Nine — Reduce Overfitting By Adding Dropout Regularization

Predictive deep learning models are incredibly prone to overfitting, which means the model will memorize the training set results. As a result of this, the model cannot generalize on unseen data. Typically, overfitting occurs when the accuracies show a high variance, and, to help reduce this, we can add another layer to our model.

The best technique to prevent, or at least reduce, overfitting, in neural networks, is dropout regularization. We use this to add a Dropout layer to the network; the Rate parameter indicates how many neurons are going to deactivate on every iteration. Neuron deactivation ends to be quite random, and, in our case, we are going to specify the rate as 0.1; this means that we want to deactivate 1% of the neurons without changing the network design.

The next code will add the Dropout layer so put it in the next notebook cell:

from keras.layers import Dropout

classifier = Sequential()

classifier.add(Dense(9, kernel_initializer = "uniform", activation = "relu", input_dim=18))

classifier.add(Dropout(rate = 0.1))

classifier.add(Dense(1, kernel_initializer = "uniform", activation = "sigmoid"))

classifier.compile(optimizer= "adam",loss = "binary_crossentropy",metrics = ["accuracy"])

So, our Dropout layer now lies between the input layer and the output layer. Because we have a rate of 0.1, 15 neurons will be deactivated during the training; that way, the classifier cannot overfit. Once the Dropout layer and the output layer were added, the classifier was compiled.

We added the Dropout layer to try to reduce overfitting and, next, we’ll tune our parameters.

Step Ten — Tune the Hyperparameters

One useful technique for experimenting with parameters is grid search. It will try all different parameters and will return the ones that give the best results. We’ll use grid search to find the best ones for our model to try to improve the model’s accuracy. We use the GridSearchCV function from Scikit-learn for this so use the following code to modify the function called make_classifer; this will allow us to test other optimizer functions:

from sklearn.model_selection import GridSearchCV

def make_classifier(optimizer):

classifier = Sequential()

classifier.add(Dense(9, kernel_initializer = "uniform", activation = "relu", input_dim=18))

classifier.add(Dense(1, kernel_initializer = "uniform", activation = "sigmoid"))

classifier.compile(optimizer= optimizer,loss = "binary_crossentropy",metrics = ["accuracy"])

return classifier

Here, we imported GridSearchCV and then modified make_classifier to give us a way of trying optimizer functions. The classifier has been initialized, the input layer and output layers added, and the classifier compiled and returned.

As you did in step four, define the classifier using this code:

classifier = KerasClassifier(build_fn = make_classifier)

KerasClassifier was used to define the classifier. This will be looking for a parameter to come to from the build_fn parameter. KerasClassifier has been called, and it was passed to make_classifier.

Now we want some parameters to play about with. Put the code below into your notebook cell and run it:

params = {

'batch_size':[20,35],

'epochs':[2,3],

'optimizer':['adam','rmsprop']

}

We added some different batch sizes, indicated how many epochs there will be, and the different optimizer functions to use.

We only have a little dataset, so 20-35 is good for the batch size. If you have a much larger dataset, obviously that batch size should be larger, If your epoch number is low, you are more likely to get results pretty quickly. You can, if you want, use larger numbers to experiment with; that will depend on how fast your server is, though. And the two optimizers, rmsprop, and adam are best for this type of network.

Next, we use our defined parameters to find the best ones and that’s done using GridSearchCV. Use the code below in your notebook:

grid_search = GridSearchCV(estimator=classifier,

param_grid=params,

scoring="accuracy",

cv=2)

GridSearch wants these parameters:

- estimators – the classifier in use

- param_grid – the parameters for testing

- scoring – the metric in use

- cv – how many folds are used for testing

Next, we need to fit grid_search to the training set:

grid_search = grid_search.fit(X_train,y_train)

You should get something similar to the following output but it may take a minute or two to finish:

Epoch 1/2

5249/5249 [==============================] - 1s 228us/step - loss: 0.5958 - acc: 0.7645

Epoch 2/2

5249/5249 [==============================] - 0s 82us/step - loss: 0.3962 - acc: 0.8510

Epoch 1/2

5250/5250 [==============================] - 1s 222us/step - loss: 0.5935 - acc: 0.7596

Epoch 2/2

5250/5250 [==============================] - 0s 85us/step - loss: 0.4080 - acc: 0.8029

Epoch 1/2

5249/5249 [==============================] - 1s 214us/step - loss: 0.5929 - acc: 0.7676

Epoch 2/2

5249/5249 [==============================] - 0s 82us/step - loss: 0.4261 - acc: 0.7864

To find the best parameters, use the code below, with the best_params attribute:

best_param = grid_search.best_params_

best_accuracy = grid_search.best_score_

The code below will find the best parameters, so add it to your notebook:

best_param

The output is:

{'batch_size': 20, 'epochs': 2, 'optimizer': 'adam'}

As you can see, 20 is the optimal batch size, 2 epochs, and adam is the best optimizer.

You can also find out what the best accuracy is for your model, using best_accuracy number:

best_accuracy

You should see something similar to this as your output:

0.8533193637489285

Here, GridSearch helped us work out the best parameters to use, and we also worked out the best accuracy too, which came in as 85%, slightly higher than before optimization.

So, what did you think of that? You’ve built an employee retention deep learning model to predict how likely it is that an employee will leave, trained it, tested it, improved it, tuned the parameters, and more. In the next chapter, we’ll use TensorFlow to build a model to recognize handwritten digits.

Chapter Eight: Building a Neural Network With TensorFlow For Handwritten Digit Recognition

Neural networks have been around for some time, continually improving to the extent where they are better at some tasks than humans are. One of those tasks is object recognition. Historically, computers and machines have not been able to match human vision. Still, now we can build a neural network that recognizes all sorts of things, including emotions, text, faces, and objects.

In our final hands-on tutorial, we are going to build a neural network that can recognize handwritten digits. We’ll use TensorFlow to do this, a Python library developed specifically for dep learning by the Google Brain laboratory. We’ll be using a set of handwritten digits, from 0 to 9 inclusive; we’ll build the network, and we’ll train it to recognize digits and make correct predictions on unseen data.

Before You Start

To do this, you will need:

- Python 3

- pip – to install the Python packages

- venv – to create a virtual environment

Step One — Configure Your Project

Before you can even think about building the network, you need some dependencies, and you need a workspace where your files will be stored. The dependencies will be stored in a Python 3 virtual environment so, the first thing to do is create a new project directory and then go to it:

mkdir tensorflow-demo

cd tensorflow-demo

Next, to set the virtual environment up, execute these commands:

python3 -m venv tensorflow-demo

source tensorflow-demo/bin/activate

Last, we need some libraries. We’ll make a file called requirements.txt to use specific library versions. This file will be in your project directory, and it will specify the version and the requirement. To create the file:

touch requirements.txt

Using your text editor, launch the file and add these lines of code to it. These will specify the libraries and the versions that we want – Image, TensorFlow, NumPy.

requirements.txt

image==1.5.20

numpy==1.14.3

tensorflow==1.4.0

Save it and come out of the editor and then use the following command to install the three libraries:

pip install -r requirements.txt

Now we can start to build our network.

Step Two — Import the MNIST Dataset

We are going to use a popular and well-known dataset called MNIST. It consists of images, each containing one handwritten digit, and each image is 28 pixels by 28 pixels.

To use this dataset, we first need a Python program. We’re going to create a single file for everything in this project so create one named main.py:

touch main.py

Using your text editor, open the file and add the following code line; this imports TensorFlow:

main.py

import tensorflow as tf

Now import the MNIST dataset by adding the next code line to the file and store it in a  variable named mnist:

main.py

...

from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)  # y labels are oh-encoded

We are using one-hot-encoding to read the data in. This will represent all the labels that go with the images, i.e., the number on the image. One-hot-encoding makes use of a vector of binary values for representing the categorical or numerical values. Our labels are representing digits between 0 and 9 inclusive,  so one for each, a total of ten values in the vector. Here’s where things get a bit complicated; one value is set as a binary 1, representing the digit at the vector index, while all the others are set as a binary 0. For example, to represent the digit 4, the vector would be [0, 0, 0, 0, 1, 0, 0, 0, 0, 0] – the values for index 4 is 1 so the vector represents  digit 4.

If we wanted the images to be represented, we would flatten the pixels, which are 28x28 pixels, to one-dimensional vector, which would be 784 pixels. Each one of those pixels is stored as a separate value from 0 to 255 inclusive. This will determine the pixel’s grayscale; all the images are black and white, not in color. So, black pixels are given a value of 255 while the white ones are 0; all the different grade shays in between take up the rest of the values.

The mnist variable is used to determine what size dataset we imported. Each subset, there are three of them, has a num_examples, which tells us that we have a dataset split of 55,000 images for training, 5000 images for validation, and 10,000 for the testing dataset.

The following code should be added to your file:

main.py

...

n_train = mnist.train.num_examples  # 55,000

n_validation = mnist.validation.num_examples  # 5000

n_test = mnist.test.num_examples  # 10,000

Our data is successfully imported, so we need to turn our attention to the network.

Step Three — Define the Architecture of Your Neural Network

The neural network architecture is all about the elements. This includes how many layers there are, how many units each layer has, and the way the units are connected in between.

Because a neural network is based, somewhat loosely, on how the human brain works, we use the term, ‘unit’ to represent a neuron in biological terms. In the same way, that neurons pass signals around a human brain, a unit in a network takes, as its input, value from the previous unit. A computation is performed, and the new value is passed, as the output, to another unit. The network is formed by layering these units, starting with an input layer and an output layer as a minimum. Any layer sandwiched between these two is known as a hidden layer.

There are lots of different architectures, and each can have very different results; performance is seen as one of the functions of the architectures, among other things like data, parameters, and training duration.

The lines of code below should be added to your file. This ensures the number of units for each layer is stored in a series of global variables.  The network architecture can be altered from one place,  and when this chapter is done, you can run your own tests to see how changing the number of layers and the number of units will change the model results.

main.py

...

n_input = 784  # input layer (28x28 pixels)

n_hidden1 = 512  # 1st hidden layer

n_hidden2 = 256  # 2nd hidden layer

n_hidden3 = 128  # 3rd hidden layer

n_output = 10  # output layer (0-9 digits)

So, what’s the difference between a shallow neural network and a deep neural network? The only difference lies in how many hidden layers there are. If there is just one, it’s called a shallow net, while, if there are several, it’s a deep net.

If you provided a shallow net with enough data for training and provided the network has enough units, it should be able, in theory, to represent the same functions that deep nets can. However, more often than not, it is more efficient, in computational terms, at any rate,   to use a small deep neural net to do the same job as a shallow network with a lot more hidden units.

Shallow nets very often struggle with overfitting, too, another reason for deep nets being used instead. Because there are so many layers hidden between the input layer and the output layer, the network can learn the features at different abstraction levels, ensuring the network can generalize on unseen data much better.

We also need to define the hyperparameters in the neural net. These are not updated during training like other parameters are; instead, we set the values at the start, and they stay the same throughout.

The variables and values below should be set in your file:

main.py

...

learning_rate = 1e-4

n_iterations = 1000

batch_size = 128

dropout = 0.5

Here, learning_rate indicates the level at which the parameters are adjusted at each step. This is an important part of the training. After each iteration over the network, the weights are tuned a little in a bid to bring the loss down. With a  larger learning rate, convergence is much faster, but they can also go way past their optimal values when they are updated.

Next, n_iterations indicates the number of times the training step is gone through. And batcha-size indicates the number of training samples used at each step. Lastly, dropout is a variable that indicates the threshold at which some units are eliminated at random.  We are using that in the last hidden layer, providing each unit with a 50% probability they will be eliminated at each training step; this is useful to reduce the chances of overfitting.

We’ve defined our network architecture, and we added the hyperparameters. Next, we are going to start building a TensorFlow graph as our network.

Step Four — Build the TensorFlow Graph

Now we come to building the network, and, for digit recognition, we are going to set it up as a computational graph; TensorFlow will then be able to execute this. TensorFlow has one primary core concept, and the clue is in the name. That’s right, it is the tensor, and that is nothing more than a data structure that looks much like a list or an array. As it goes through the graph, the tensor is initialized and manipulated; as the learning process continues, it gets updated.

We’ll begin with three placeholder tensors. These allow us to add value to them later on. To define those, add the code below to your file:

main.py

...

X = tf.placeholder("float", [None, n_input])

Y = tf.placeholder("float", [None, n_output])

keep_prob = tf.placeholder(tf.float32)

We only need to specify one parameter when it is declared, and that is the size of the data being given to the network. X has been given a shape of [None, 784]; None indicates an unspecified amount because we will be feeding the network and an undefined number of images, each of which is 784 pixels. Y has a shape of [None, 10]; this will be used for a number of label outputs, as yet undefined, with a possible 10 classes.

Lastly, the dropout rate is controlled with the keep_prob tensor. It has been initialized but not as an immutable variable. Instead, it is a placeholder; that is because we want that tensor used for training when the dropout is defined as 0.5, and testing, when its set as 1.0.

The network will update two parameters during training – the values for weight and bias. Each parameter needs an initial value and should not be set as placeholders. These values are used in the neuron’s activation functions as a representation of the connection strength between units. They are, pretty much, where all the learning is done.

We could initialize them as zero if we wanted because the will be optimized throughout the training process. But it’s worth bearing in mind that the value you initialize it with will impact the model accuracy quite significantly. Our values are random, and they come from a truncated normal weight distribution. Ideally, the should be as close to zero as we can get them; that way, it’s easy to adjust them negatively or positively. We also want the values to be a little different; that way, the generated errors are a little different. Doing this makes sure our model can learn useful information.

Add the following code to your file:

main.py

...

weights = {

'w1': tf.Variable(tf.truncated_normal([n_input, n_hidden1], stddev=0.1)),

'w2': tf.Variable(tf.truncated_normal([n_hidden1, n_hidden2], stddev=0.1)),

'w3': tf.Variable(tf.truncated_normal([n_hidden2, n_hidden3], stddev=0.1)),

'out': tf.Variable(tf.truncated_normal([n_hidden3, n_output], stddev=0.1)),

}

We gave the bias a constant value, just a small one, to make sure that, in the initial stages, our tensors activate. That way, they will contribute something to the propagation. We store both the bias and weight tensors in dictionary objects, just to make them easier to access.

Define your biases with this code:

For the bias, we use a small constant value to ensure that the tensors activate in the initial stages and therefore contribute to the propagation. The weights and bias tensors are stored in dictionary objects for ease of access. Add this code to your file to define the biases:

main.py

...

biases = {

'b1': tf.Variable(tf.constant(0.1, shape=[n_hidden1])),

'b2': tf.Variable(tf.constant(0.1, shape=[n_hidden2])),

'b3': tf.Variable(tf.constant(0.1, shape=[n_hidden3])),

'out': tf.Variable(tf.constant(0.1, shape=[n_output]))

}

Now we can set up the network layers. This requires us to define the operations used for tensor manipulation. Add the code below to your file:

main.py

...

layer_1 = tf.add(tf.matmul(X, weights['w1']), biases['b1'])

layer_2 = tf.add(tf.matmul(layer_1, weights['w2']), biases['b2'])

layer_3 = tf.add(tf.matmul(layer_2, weights['w3']), biases['b3'])

layer_drop = tf.nn.dropout(layer_3, keep_prob)

output_layer = tf.matmul(layer_3, weights['out']) + biases['out']

All hidden layers execute matrix multiplication on the outputs from the previous layer and the weights from the current layer. Then the bias is added to the values. When we get to the final hidden layer, a dropout operation is applied, using the value of 0.5 from keep_prob.

Now we come to the last step. We need a loss function defined for optimization, and perhaps the most common one, in this case, is the cross-entropy loss function, also called log-loss. This will quantify the difference between the predictions and labels, which are the two probability distributions. The absolute best classification would be a cross-entropy of 0, and the loss minimized completely.

We will also need to minimize the loss function, and this is done with an optimization algorithm. The commonest method is the gradient descent optimization, typically used for finding a function’s minimum (local). It does this by iteratively stepping in a descending, or negative, direction down the gradient. There are quite a few of these algorithms already in TensorFlow, but we are going to use the Adam optimizer.

The Adam optimizer uses momentum to extend gradient descent, speeding the process up by computing a weighted average (exponentially) of the gradients and then applying it when the adjustments are made.

Add this code into your file:

main.py

...

cross_entropy = tf.reduce_mean(

tf.nn.softmax_cross_entropy_with_logits(

labels=Y, logits=output_layer

))

train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

The network has been defined, and we used TensorFlow to build it. Next, we want to train the graph by giving it some data and then test it to see if it learned anything.

Step Five — Train and Test

Training our network requires that the training dataset is fed into the graph and the loss function optimized. On each iteration of the training image batches, the parameters are updated by the network, reducing the loss to get a more accurate prediction.

Testing the network requires that the testing data is passed through our newly trained graph, tracking how many correct predictions are made so the final accuracy can be calculated.

Before we begin training, we need to define the method we want to use to evaluate the accuracy. Doing that will enable the accuracy to be printed on mini-batches of images throughout the training process. This will let us track the loss reduction and increase in accuracy from the first to the last iteration. And we can also see whether sufficient iterations have been run to give us the most consistent, optimal accuracy result.

Here’s the code to do that:

main.py

...

correct_pred = tf.equal(tf.argmax(output_layer, 1), tf.argmax(Y, 1))

accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

Here, correct_pred takes the arg_max function and uses it to compare the correctly predicted images. It does this by observing the predictions (output_layer) and the labels (Y) with an = function returning the result as a Boolean list. Then, the list is cast to floats and we get the accuracy score by calculating the mean.

Now a session can be initialized to run the graph. Here, we feed the training examples to our network, wait for the training to complete, and then pass the test samples through the (hopefully) trained graph.

The code below does this for you:

main.py

...

init = tf.global_variables_initializer()

sess = tf.Session()

sess.run(init)

The primary function of training a deep learning net is loss function optimization. We want the difference between the predicted and the true image labels minimized. There are four steps to this process, repeated over a defined series of iterations. Those steps are:

1. The values are forward-propagated through the network.

2. The loss is computed

3. The values are backpropagated through the network

4. The parameters are updated

At every step in the training process, slight adjustments are made to the parameters to reduce the loss for the following step. As the network learns, the loss should reduce visibly, and, eventually, we will be able to stop the training and start testing the model in the test data.

The code below should be added to your file:

main.py

...

# train on the mini batches

for i in range(n_iterations):

batch_x, batch_y = mnist.train.next_batch(batch_size)

sess.run(train_step, feed_dict={

X: batch_x, Y: batch_y, keep_prob: dropout

})

# print loss and accuracy (per minibatch)

if i % 100 == 0:

minibatch_loss, minibatch_accuracy = sess.run(

[cross_entropy, accuracy],

feed_dict={X: batch_x, Y: batch_y, keep_prob: 1.0}

)

print(

"Iteration",

str(i),

"\t| Loss =",

str(minibatch_loss),

"\t| Accuracy =",

str(minibatch_accuracy)

)

Mini-batches are fed to the network, and after 100 iterations of each step, the loss and accuracy are printed for each mini-batch.  We don’t expect to see a reduction in loss and an increase in accuracy at this stage because the values are for each batch and not the whole model. Mini-batches are used as it is quicker than feeding the network with each individual image. We want the training process to be as quick as possible, with the network seeing as many examples as possible before the parameters are updated.

When the training is finished, the session can then be run using the test data. We use a dropout rate of 1.0 for keep_prob, as this ensures every test unit is active.

Here’s the code:

main.py

...

test_accuracy = sess.run(accuracy, feed_dict={X: mnist.test.images, Y: mnist.test.labels, keep_prob: 1.0})

print("\nAccuracy on test set:", test_accuracy)

Now we can run the program to see how accurate the neural net is in recognizing the handwritten digits in the images. Save your main.py file and then run your script by running the command below in the terminal:

python main.py

You should see something much like the output below, although the results may be slightly varied:

Iteration 0     | Loss = 3.67079    | Accuracy = 0.140625

Iteration 100   | Loss = 0.492122   | Accuracy = 0.84375

Iteration 200   | Loss = 0.421595   | Accuracy = 0.882812

Iteration 300   | Loss = 0.307726   | Accuracy = 0.921875

Iteration 400   | Loss = 0.392948   | Accuracy = 0.882812

Iteration 500   | Loss = 0.371461   | Accuracy = 0.90625

Iteration 600   | Loss = 0.378425   | Accuracy = 0.882812

Iteration 700   | Loss = 0.338605   | Accuracy = 0.914062

Iteration 800   | Loss = 0.379697   | Accuracy = 0.875

Iteration 900   | Loss = 0.444303   | Accuracy = 0.90625

Accuracy on test set: 0.9206

The accuracy of our model is 92%. If we wanted to improve that or if we wanted to learn more about how hyperparameter tuning affects the model, we could change the learning rate, the batch size, dropout threshold, and how many iterations are done. We could change how may units the hidden layers have to see the effect on the model accuracy.

So, let’s see if the model really does work by using an image of our own. You have two choices here – download a test image using curl (see command below) or create a handwritten digit image of your own, using a graphics editor. If you choose that option, the image must be 28x28 pixels. If you want to download the image, use curl:

curl -O https://raw.githubusercontent.com/do-community/tensorflow-digit-recognition/master/test_img.png

Go to your editor and open main.py. Add the code below at the top of the file so you can import the libraries you need to manipulate the image:

main.py

import numpy as np

from PIL import Image

...

Go to the end of the file and add the code below to bring in your test image:

main.py

...

img = np.invert(Image.open("test_img.png").convert('L')).ravel()

The Image library has an open function that loads in your image as a 4D array. In that array are the Alpha transparency and three color channels (RGB). This is different from the representation we used earlier when the dataset was read in with TensorFlow, so a bit more work is needed.

The convert function has been given an L parameter; we use this function to reduce the representation to a single grayscale channel. This is then stored as a numpy array and np.invert used to invert it. We do this because our current matrix uses 0 for black and 1 for white. We need it to be the other way around. Lastly, the array is flattened by calling ravel.

We now have our data in the right structure, so let’s run a session and feed in a single image to test the model. The code below tests the image and prints the resulting label:

main.py

...

prediction = sess.run(tf.argmax(output_layer, 1), feed_dict={X: [img]})

print ("Prediction for test image:", np.squeeze(prediction))

We called np.squeeze on the prediction to return one integer from our array, for example, go from [2] to 2. The output shows us that the model predicts the image as 2.

Have a play about with the network, test it on poorly written digits, those that closely resemble others, and see how well, or otherwise, your model performs.

Well done! If you followed this along practically, you have built a neural network and trained it on the MNIST dataset to classify images. It achieved an accuracy of 92%, so you tested it on your own image. Now you know how to do this, try it on your own data or find and download other popular datasets from the internet.

In our final chapter, we look at how to keep your deep learning models as bug-free as possible.

Chapter Nine: How to Keep Your Deep Learning Models Bug-Free

Traditionally, in software engineering, if your program gets a bug, it will crash. Yes, users find this incredibly irritating, but developers welcome it. Why? So that they can take a deeper look at the error and see what went wrong.

In deep learning, we do get to see the errors occasionally, but, in most cases, the program will just crash, and there won’t be any clear reason as to why. We can do manual debugging but, most of the time, when a deep learning model crashes, it is down to bad output predictions. What makes it worse is, when our model is showing low performance, we often can’t see what the problem is, why it failed, or even when the model failed.

One of the biggest bugbears of most deep learning practitioners is the amount of time they spend debugging their models and tuning them – around 80 to 90% of their time, believe it or not. The remaining 10 to 20% of their time is spent deriving the equation, implementing stuff, and building their models.

In fact, a common sentiment among practitioners is that they spend 80–90% of time debugging and tuning the models, and only 10–20% of time deriving math equations and implementing things.

Let’s say that you have a research paper, and you want to reproduce a result. However, your results are coming out far worse so you could be wondering why your model is performing so much worse than the paper you want to be reproduced.

Several factors could be at play here:

- Implementation Bugs – many of the bugs in deep learning models are not visible

- Hyperparameter choices – these can also cause a degradation in performance because deep leaning models are sensitive to them. Even a slight change in weight initialization and learning rate can make a huge difference.

- Data/Model Fit – if you pre-trained your model using, for example, ImageNet data and then tried to fit it using images of self-driving cars, you could have problems because the latter is not so easy to learn.

- Dataset Construction – it could even be down to how your dataset has been constructed. If you didn’t use sufficient examples or your labels are noisy, classes imbalanced, even using a different distribution to split your dataset into training and testing sets, can all have a negative effect on how your model performs.

So, how do you stop your models from getting to this stage? We’re going to look at five steps that will help you keep your models as bug-free as you possibly can.

Step One – Keep Things Simple

Especially at the beginning. If your workflow is too complicated, it will be much harder to troubleshoot it.

Starting simple requires you to consider a few things. First, you need to know how to choose a good but simple architecture. These are simple to implement and will get you at least some of the way towards solving the task without too many bugs being introduced.

Choosing the right architecture is one of the hardest yet most important parts of deep learning, at least for beginners. Papers are constantly being published, and all of them claim they are state-of-the-art on certain problems. These tend to get complicated – very quickly.

If you want maximum performance from your model, choosing the right architecture is something of a challenge. But, when you begin a new problem, it’s very simple to solve some simple rules that let you choose the right architecture for the job:

- If you are using image data, begin by using an architecture similar to LeNet. And, as your codebase matures, think about using something similar to ResNet.

- If you are using sequence data, use an LSTM that has a single hidden layer, classical or temporal convolutions, or a combination of layers and convolutions. Once your problem matures, then you can think about using a model that is attention-based or similar to WaveNet.

- If your problem is anything else, begin by using a fully-connected neural net with a single hidden layer. As your problem matures, you could move onto a more advanced network.

The reality is this. Most of the time,  your input data will be a combination of the above. So, how do we work with multiple input modalities? How do we deal with feeding them to the network? You could try these three steps:

1. The first step is to map the individual modalities to a feature space that is lower-dimension. For example, you could pass images through a convolutional neural network and words through an LSTM.

2. Next, the network outputs need to be flattened. That way, we have one vector for every input into the model. Those inputs are then concatenated.

3. Last, pass the inputs to the output layer through one or more fully connected layers.

Choosing the right, simple architecture is just one step; you also need the right hyperparameter defaults at the beginning. Here are a few ideas:

- Adam optimizer – 3e-4 is the ‘magic’ value for the learning rate

- ReLU Activation – best for convolutional and fully connected models, or Tanh activation if you use an LSTM model

- HE initialization – if you use a ReLU activation function or, if you use the Tanh function, use the Glorot initialization

Next, the input data must be normalized. This means the mean is subtracted and the variance used for division. For images, values can be scaled to [0, 1] or [-0.5, 0.5] through a division with 255, for example.

Last, try to simplify the actual problem. If you are working with a complex problem that has huge amounts of data and classes, think about:

- Using just a small training set, say 10,000 examples

- Fixing the input size, object number, classes, etc.

- Create a synthetic training set that is simpler, much like they do in research labs.

Do this, and you can be reasonably confident your model can solve the problem, and you will have a much faster iteration speed.

Step Two – Implement Your Model and Debug It

There are five very common bugs found in deep learning models, and we’ll look at those before we tackle implementing the model:

- Network Tensor Shapes are Not Correct – it is one of the most common bugs and it can fail in silence. Much of the time,  this is down to silent broadcasting by automatic differentiation systems in the framework. The tensors change shape in the network and go on to cause a lot of strife.

- Incorrect Input Pre-Processing – you may have forgotten to normalize the inputs, or you may have over-normalized it, putting too much pre-processing into them and augmented the data too much.

- The Loss function has the Wrong Input – if, for example, you used a SoftMax output to a loss that is looking for logits.

- You Didn’t Set up Train Mode Properly for the Network – you may, for example, have toggled training and evaluation mode or you are controlling the nom dependencies for the batch

- Numerical Instability – if you get an output of NaN or inf. More often than not, this is because an exponent, division, or log operation has been used in the code.

Implement your model using this advice:

- Begin by using a lightweight implementation. The first version of the model should have the least amount of new code lines that you can get away with. As a rule, stick to 200 or fewer lines of code, not counting PyTorch/TensorFlow code or tested infrastructure components.

- Make use of off-the-shelf components like Keras as much as you can. Keras works really well right off the bat, but if you need to use TensorFlow, then make sure you use the functions built into it. Do NOT attempt to do the math yourself, avoid making the common numerical instability problems.

- Leave your complicated pipelines until later. While these are important when you are working with large machine learning systems, you don’t need them at the start. Data pipelines are a common source of deep learning bugs, so just use a dataset that can easily be loaded into memory.

The first step in implementing a deep learning model that is as bug-free as it can be is getting that model to run. Some things can stop this happening:

- Mismatched shapes/casting issues – address this by stepping through the creation and the inference of your model, one step at a time, using a debugger. Look out for incorrect shapes and make sure the tensors have the right data types.

- Out-Of-Memory Issue – this isn’t easy to debug. Try scaling the operations that use a lot of memory back one at a time. For example, if your code contains large matrices, reduce their dimensions, or halve your batch size.

- Any other issue – use Google, look at Stack Overflow, and any other source that can help you with this.

Let’s take a look at how to step through your model in a debugger, and discuss using debuggers in the code for deep learning models.

- If you use PyTorch, use ipbd, which will export your functions to the IPYthon debugger, which, incidentally, is interactive.

- If you use TensorFlow, it’s a little harder. TensorFlow will split the graph creation process from operation execution, and you can try these three options:

- Step through the creation (graph) and look at each tensor layer

- Step into the loop (training) and evaluate those layers

- Use tfdb, the TensorFlow Debugger, which will automatically take care of the first two options for you.

Once your model is running, you need to try overfitting one batch of your data. |This heuristic can catch an astonishing amount of bugs, and what you are looking for here is getting your training error as close to 0 as you can.

When you try overfitting one batch and it doesn’t work, there are a few things that could happen:

- The Error Rises – this is commonly down to the gradient or loss function containing a flip sign somewhere

- The Error Explodes – usually, this is down to a numerical issue but a high learning rate can also cause it

- The Error Oscillates – try lowering the learning rate and looking at the data for the wrong data augmentation or shuffled labels

- The Error Plateaus – try increasing the learning rate and remove regulation. Then look at the data pipeline and loss function to make sure it is all correct.

Even when you can get your model to overfit on one batch, bugs can still be caused by a few things. So, the best thing to do here is to compare your own results to another known result. But which one? Which are the most useful?

- You will find the most useful results in an official implementation that was evaluated using a dataset similar to what you used. Line up the models and step through them, one line at a  time, and make sure your model’s output is the same. You are looking to make sure your model performs at a level on par with expectations.

- If there is not official implementation evaluated on a similar dataset, you can try one that used a benchmark dataset for evaluation. In this case, you must walk the code, one line at a time, to ensure the output is the same.

- If you can’t find either of these implementations, use an unofficial implementation instead. However, while you can step through the code, you shouldn’t have such high expectations; most of these types of implementation already contain bugs.

- Lastly, you can try comparing your results to a paper that hasn’t any code. This will ensure that your model is performing on par with expectations. Compare the results of your model using a benchmark dataset to ensure your model can perform properly in simple settings and those results from similar models on similar datasets – that will help understand the performance you should expect.

- Simple baselines produce one of the most under-rated result sources and could be linear regression, output averages, and so on. These can help you to ensure that your model actually learns something.

Step Three – Evaluate the Model Performance

The third step requires that you evaluate how your model is performing. You can then use the result to work out how to improve it. For this, you want a concept known as bias-variance decomposition:

Courtesy of https://fullstackdeeplearning

- The blue line on the left-hand plot indicates human-level performance, while the green is the error curve for training. The red is the error curve for validation, usually higher than the training curve, while purple is the error curve for testing, usually higher than that of validation

- The right-hand plot shows the bias-variance decomposition decomposing the test error into all its separate components. Those components are:

- Irreducible error – results from baseline performance

- Avoidable bias – underfitting, in other words. The gap between training and irreducible errors is used to measure this.

- Variance – overfitting, in other words. The gap between the training and validation errors measures this.

- Validation set over- fitting –measured by the gap between the validation and test errors and is an indication of how much the model overfits on the validation set.

Of course, this will assume that the datasets for training, validation, and testing are all from the same place. What if they are not? You might be training a model for object detection for self-drive vehicles, but you use training images taken during the day and testing images taken at night.

What you should do here is have two validation sets, one taken from the training data and the other from the testing data distributions. Look at the figure below. In the left-hand plot, we have the training and test errors, but we also have two validation set errors as well. These are for the training dataset and the testing dataset.

Now the formula for the bias-variance decomposition is given another term, this one being the measure of the distribution shift; that is the difference between the training and testing validation errors.

Courtesy of https://fullstackdeeplearning

To summarize, there is a very simple strategy to evaluate your model performance:

Test Error – Irreducible Error + Bias + Variance + Distribution Shift + Validation Overfitting.

Step Four – Improving Your Model and Data

When it comes to improving your model, you need to prioritize how you are going to do it. The first thing to tackle is underfitting, i.e., reduce the bias of your model. There are a few things you can do to achieve this:

- Often, the easiest way is to enlarge your neural network, and you do that by adding in some extra layers or having more units in use in each layer.

- You could have a look at reducing regularization

- Carry out an error analysis

- Try a more state-of-the-art neural network architecture

- Tune the hyperparameters

- Add a few more features

Next, you tackle overfitting, i.e., reduce the variance of your model. Again, there are a few things you can try:

- The easiest way to do this is to add in some more training data.

- If that isn’t possible, try adding normalization – choose layer or batch norm

- Try data augmentation

- Increase the regularization – weight decay, L2, dropout, etc.

- Carry out an error analysis

- Try a more state-of-the-art neural network architecture

- Tune those hyperparameters

You could also try removing some features, reducing the size of your model, or use early stopping.

When you get the training error and the training validation error to where you think they should be, you can look at the distribution shift in the data. A few things you can try are:

- Manually check the errors made by your model on the test-validation set. You are looking for generalizable errors; get more training data into your model to take care of this.

- Similar to the above but synthesize the training data instead of collecting more

- You can also try applying some domain adaptation techniques, the training distribution, and the test distribution. These techniques tend to be research-aimed rather than production-ready. You can train them on the source distribution and generalize them to a different target, using a limited amount of labeled data or only unlabeled data.  Think about using this where there is only limited access to the labeled data from the test distribution, or you have full access to data that is somewhat similar. There are two domain adaptations to choose from:

- Supervised – limited data is available from the target domain. Includes adding more target data to a training set and fine-tuning your already trained model.

- Unsupervised – loads of unlabeled data is available from the target domain. Includes CycleGAN, domain confusion, and correlation alignment.

Lastly, if it applies, you can rebalance the datasets. During the training process, you should make periodic checks on the error for the hold-out test set.  If the model is performing better in the test-validation set than it is on the test set, the validation set has been overfitted. This typically happens when you have a small validation set, or you’ve done a lot of hyperparameter tuning. In this situation, simply recollect all the validation data. You do this by reshuffling your ratio for the test and validation split.

Step Five - Tuning the Hyperparameters

Deep learning networks are packed with hyperparameters, something you don’t tend to find in standard machine learning models. A hyperparameter is a training variable that you manually set with a value, pre-determined, and set before you start training the model. It isn’t easy to work out which hyperparameters need optimizing; some are sensitive, more so than others, and they are dependent on the model. Here, you can see the sensitivity to default values for these hyperparameters:

Hyperparameter

Approximate Sensitivity

Learning rate

High

Learning rate schedule

High

Optimizer choice

Low

Other parameters for the optimizer, i.e., beta 1, Adam, etc.

Low

Batch size

Low

Weight initialization

Medium

Loss function

High

Model depth

Medium

Layer size

High

Layer parameters, i.e., kernel size

Medium

Regularization weight

Medium

Nonlinearity

Low

Conclusion

I want to thank you for taking the time to read "Deep learning with Python: Simple and Effective Tips and Tricks to learn Deep learning with Python." I trust that you found it useful.

Deep learning is a varied subject, and we couldn't possibly cover it all in this book. However, I have tried to cover the fundamentals, to give you a good grounding in what deep learning is all about and how to use it. I also gave you a complete deep learning model to build from start to finish, as a way of pulling everything together.

Deep learning is the way forward; it is the future and, if you don't want to get left behind, it's a subject you seriously need to consider learning. And, if you are already into machine learning, then it's the next logical step for you.

From here, if this has whetted your appetite, then there are plenty of online courses you can take to further your deep learning knowledge.

Wherever you go from here, I want to wish you luck on your journey and every success for the future.

References

https://www.dataquest.io/

https://pythonprogramming.net/

https://searchenterpriseai.techtarget.com/

https://machinelearningmastery.com/

https://towardsdatascience.com/

https://pathmind.com/

https://www.analyticsvidhya.com/

https://becominghuman.ai/

https://www.explainthatstuff.com/

https://www.digitalocean.com/

Table of Contents

Introduction

Chapter One: Introducing Deep Learning

How Does Deep Learning Work?

What is a Deep Learning Neural Network?

Deep Learning Methods

Deep Learning Applications

Deep Learning Challenges and Limitations

Machine Learning vs. Deep Learning

A Brief History

Chapter Two: Python and Deep Learning

The Top Python Libraries for Deep Learning

Chapter Three: A Look At Neural Networks

Neural Network Applications

Examples of What Deep Learning Can Do

The Elements of a Neural Network

Feedforward Networks

Multiple Linear Regression

Gradient Descent

Logistic Regression

Chapter Four: A Deeper Look at Recurrent Neural Networks and LSTMs

More on Feedforward Networks

Recurrent Neural Networks

Backpropagation Through Time (BPTT)

Long Short-Term Memory Units (LSTMs)

Capturing Remote Dependencies and Diverse Time Scales

Chapter Five: Introduction To CNNs and Image Processing

Images as 4D Tensors

The Definition of Convolutional

How Do Convolutional Neural Networks Work?

Max Pooling/Downsampling

Chapter Six: An Introduction to Deep Reinforcement Learning

The Definitions of Reinforcement Learning

Domain Selection

Neural Networks and Deep Reinforcement Learning

Chapter Seven: Let’s Build a Deep Learning Model with Keras and TensorFlow

Step One – Data Pre-Processing

Step Two – Splitting Your Dataset

Step Three — Transform the Data

Step Four — Build the Neural Network

Step Five — Use the Test Set for Predictions

Step Six — Use a Confusion Matrix

Step Seven — Make One Prediction

Step Eight — Improve the Model Accuracy

Step Nine — Reduce Overfitting By Adding Dropout Regularization

Step Ten — Tune the Hyperparameters

Chapter Eight: Building a Neural Network With TensorFlow For Handwritten Digit Recognition

Step One — Configure Your Project

Step Two — Import the MNIST Dataset

Step Three — Define the Architecture of Your Neural Network

Step Four — Build the TensorFlow Graph

Step Five — Train and Test

Chapter Nine: How to Keep Your Deep Learning Models Bug-Free

Step One – Keep Things Simple

Step Two – Implement Your Model and Debug It

Step Three – Evaluate the Model Performance

Step Four – Improving Your Model and Data

Step Five - Tuning the Hyperparameters

Conclusion