Deep Learning Basics: What are Neural Networks?

The human brain is a fantastic pattern-recognizing machine. It processes external “inputs,” categorizes them, and generates an “output,” with minimal conscious effort. At its core, deep learning (and more broadly artificial intelligence) tries to mimic this brain process. The actual mapping, however, happens with something called a neural network.

Neural networks are one of the most popular methods of solving problems in machine learning. As data science jobs rise in demand in tech hubs like Toronto and New York, understanding how to apply these concepts will be critical. BrainStation’s Data Science Bootcamp will teach you these skills and prepare you for a career in data. But let’s go back to the beginning:

What is a Neural Network and How Does it Work?

A neural network is a system of hardware and code patterned on the way neurons work in the human brain. It helps computers think, understand, and learn like humans. 

As an example, think about a child touching something hot (say a cup of coffee), which causes a burn. In most cases, this would prevent the child from touching a hot cup of coffee again. It’s safe to say, though, that the child did not have any conscious understanding of this kind of pain before touching the mug.

This modification of the person’s knowledge and understanding of the external world is based on recognizing and understanding patterns. Similar to humans, computers also learn through the same method of pattern recognition. This forms the basis of the way a neural network works.

Earlier, traditional computer programs worked on logic trees, which meant if A happens then B happens. All potential outcomes for each system could be preprogrammed. This, however, eliminated any freedom for flexibility.

Neural networks, on the other hand, are built without any pre-defined logic; they are merely a system trained to search for and adapt to patterns contained within data. This is modeled on how the human brain works, where each neuron or idea is connected via synapses. Synapse include a value that is a representation of the probability of a connection to occur between two neurons.

A neuron is a singular concept. The mug, the color white, the tea, the burning sensation of touching a hot cup — all of these can be taken as possible neurons, and each of these can be connected. The strength of the connection is determined by the value of their respective synapse. The higher the cost, the better the bond.

Here is an example of an essential neural network connection helping form a better understanding:

In the diagram above, neurons are represented by nodes, with the lines connecting them representing synapses. The value of the synapse denotes the possibility of that one neuron being found alongside another. So, in this example, the diagram represents a mug that contains coffee, which is white and extremely hot.

All cups would not have the properties like the one in this example, and we can connect different neurons to the cup (for example, tea instead of coffee). The possibility of two neurons being connected is determined by the strength of the corresponding synapse connecting them.

That said, in a scenario where mugs are not regularly used to carry hot drinks, the number of hot cups would decrease substantially, which would also reduce the strength of the synapses connecting mugs to heat.

So:

Becomes:

What is a Perceptron?

Perceptrons are the foundational model of a neural network. It uses multiple binary inputs (x1, x2, etc.) to produce a single binary output. Like this:

To better understand this neural network, let’s use an analogy.

Assume you walk to work. This decision of going to work can be based on two major factors:  the weather, and whether or not it is a weekday. While the weather factor is manageable, working on weekends is (often) a deal-breaker. Since we’re working with binary inputs here, let us propose the conditions in the form of “yes or no” questions.

Is the weather fine? One for yes, zero for no. Is it a weekday? One yes, zero for no.

Keep in mind that we can’t inform the neural network of these conditions at the outset. The network will need to learn them for itself. How will the network decide the priority of these factors when making a decision? By using what is known as “weights.” Weights are numerical representations of preferences. A higher weight will make the neural network assume that input is a higher priority than the rest. This is represented by the w1, w2…in the flowchart shown above.

The Value of Neural Networks

Any system that needs machine learning references a neural network for assistance, and there are many reasons for this:

1. With the help of neural networks, users can solve problems for which a traditional-algorithmic method either does not exist or is too expensive to implement.
2. Neural networks learn by example, reducing the need for additional programs.
3. Neural networks are significantly faster and more accurate than conventional methods.

Real-life Applications of Neural Networks

Deep learning, with the help of neural networks, has found extensive use in the following areas:

Speech Recognition

For a good example of this, look no further than the Amazon Echo Dot, which lets users get news reports and weather updates, order food, or complete a purchase online just by speaking.

Handwriting Recognition

Neural networks are trained to understand patterns in a person’s handwriting, and Google’s Handwriting Input application makes use of this to convert scribbles into meaningful texts.

Face Recognition

From improving the security of handheld devices to various Snapchat filters, face recognition is everywhere. A good example is the technology Facebook uses to suggest people to tag when a photo is uploaded to the site.

To sum up, neural networks make up the spine of a wide variety of innovative technologies in use today. In fact, imagining a deep/machine learning initiative without them is almost impossible, and that will only increase with time.  

Find out more about BrainStation’s Machine Learning Certificate course and Data Science Diploma.