Artificial neural networks (ANNs) are one of the most powerful machine learning algorithms out there. They have become very popular in recent years, with applications ranging from predicting house prices to helping companies make better decisions.

In this article we will look at how to draw an ANN diagrammatically. We’ll also take a closer look at some common types of neural network architectures and what benefits they offer.

Once you have drawn your diagrams, you can use them as a basis for creating your own ANNs!

If you want to learn more about how to apply artificial intelligence in business, check out our other articles here on StatPlanet.com.

Let's get started.

Types of neural networks

There are several types of neural network architectures. Which one is best for your project depends mostly on what you want to accomplish.

Traditional feed-forward artificial neural networks (FFANNs) are perhaps the most well known type of NN. FFNNs have nodes that connect to each other in a sequential pattern, usually starting with an input node and ending with an output node.

Recurrent neural networks (RNNs) work similarly to FFANNs but instead of connecting sequentially, they form feedback or circular connections between themselves. This feature allows RNNs to learn patterns that repeat itself like conversations do.

Convolutional neural networks (CNNs) focus on localizing specific features of an image such as lines, shapes, and textures. CNNs use convolutions to manipulate small chunks of information from the source material.

Hybrid models combine characteristics of different types of ANNs into one. For instance, some hybrid models mix together elements of FFANNs and RNNs.

Feed-forward neural networks

Feed-forward neural networks are one of the most common types of neural network architectures you will learn about in your studies of neural networks. They contain an input layer, hidden layers, and an output layer just like regular neurons do!

The key difference is that there are no outputs from each neuron other than through the final output layer. The internal nodes (neurons) of a feed- forward neural network instead perform computation over time. This process is referred to as “training” or “learning” the net.

By having more than one node in the hidden layer, we can create something called a "fully connected" network. Each node in this layer receives inputs directly from at least one node in the previous layer, and these two nodes are connected together. In our example here, both the input layer and the preceding hidden layer have a connection with every node in the next hidden layer.

Recurrent neural networks

Another type of deep learning network is recurrent or time dependent. These are also called sequential pattern recognition models because they can learn patterns that exist over time.

Recurrent neural networks were first proposed in 1989 by Chris Bishop at Stanford University. Since then, they have become one of the most popular types of architectures for natural language processing (NLP) and other applications such as speech recognition and video analysis.

Some examples of applications include chatbots, computer vision tasks (such as object detection and image captioning), and sequence generation (writing paragraphs). Sequence generation applies to things like text summarization, product descriptions, and informal essays.

Because these networks process information sequentially, there’s an extra term added to the word “deep” — depth. A recurrent layer connects all previous layers with new inputs, creating a feedback loop which allows it to remember past states and use those knowledge to make predictions for the future.

There are several variants of RNNs, but the ones we will focus on here are gated recursive units (GRUs) and long short-term memory cells (LSTMs).

Convolutional neural networks

CNNs are one of the most influential architectures in modern computer vision. They have seen dramatic success across many applications, particularly in the field of image recognition.

CNNs work by taking large input images and breaking them into smaller overlapping sections called kernels or filters. These filters get trained as individual units and then combined together into larger convlayers that learn more about the whole image.

The layers between each filter become thinner and thicker depending on the size of the kernel used. The last layer is an output softmax classifier where it learns which classes the network predicts.

By using such small sized filters, the net can pick up on fine level features of the image. This helps when there are not clear defining lines for categories like humans recognize shapes of animals and plants.

Using these feature extraction concepts along with the learning algorithm mentioned above, CNNs are able to perform very well on a wide range of tasks.

Deep convolutional neural networks

Recent developments in computer vision and other fields have led to deep learning, or deep neural network (DNN) architectures. DNNs are inspired by how neurons work in our brains!

In DNNs, there is an input layer, one or more hidden layers, and then an output layer. The number of layers varies depending on the problem being solved.

The hidden layers contain many individual nodes that connect to each other. These connections either activate quickly (positive feedback), deactivate slowly (negative feedback), or do neither (no feedback).

It is this repeated process — activation, no activation, activation — that allows your brain to understand patterns and concepts.

Recurrent convolutional neural networks

In recurrent architectures, instead of having fully connected layers that connect all nodes together, there are time sequences or loops where information can be shared. For example, say you wanted to recognize cats based off pictures. You could have one node that is just a feature extractor that gets trained as normal, but then there would need to be another layer or sequence where it connects this feature vector to identify if it is a cat or not. This second layer is referred to as a temporal connection because it has an internal loop or structure where information can be passed along.

Now let’s look at some examples! If you take a look at the picture below, I want you to pay close attention to the shape of the nose and the color of the tail. These two features go into a prediction for whether the animal is a dog or not.

In order to understand how these features influence the prediction, we will have to re-examine our previous diagram. The longer white line coming out of the nose goes in the prediction, and the shorter white line going down from the tail goes in the classification of either being a pure breed or mixed breed.

By looking at the length difference of both lines, we can determine what kind of dog it is. A longer nose means it is more likely to be a large, strong type of dog such as a Labrador Retriever.

LSTM neural networks

While feed-forward neural networks have become very popular in recent years, there are still some applications where they do not work as well. One such application is time series prediction with sequential data sets.

Traditional neural network architectures can sometimes struggle when trained using large amounts of training data. This is because these networks learn how to combine input features directly into an output layer, but often times you will not have enough data for each feature individually.

Sequence models like recurrent neural networks (RNNs) or long short term memory (LSTMs) mitigate this by keeping track of past information within the model itself. These internal states help the RNNs or LSTMs determine what element in the sequence comes next.

There are many types of RNNs and LSTMs, so it is important to know which one is best suited for your task. In this article we will focus on learning how to draw simple LSTM networks. You will also learn about different initializations for the weights and biases of the layers.

Bi-directional LSTM neural networks

The second most popular architecture for NN is the bi-directional long short term memory (biLSTM) network. This type of NN works by having an additional input that feeds into both sides of the layer, just like how we have our mouth which helps us process oral information.

The difference here is that this extra input allows you to incorporate temporal information in addition to spatial information. So as the name suggests, it can remember past events as well as future ones!

This seems intuitive because we learn from experience that things that happened earlier are important in helping us understand what’s happening now. For example, if you were reading this article right now then that word you see before this one would be something you have read before.

A lot of times, people get stuck thinking about the hard parts of AI as being either scary or optimistic. But I believe there is an overlooked middle ground: boring. Because AIs that do simple tasks seem almost magical at times.