Diving into deep learning

Every time you search google or ask Amazon’s Alexa a question you’re benefiting from a type of artificial intelligence called deep learning. It’s the artificial brain power behind much of the AI in all of our lives. Sarah Harrop explains what deep learning is and how it works. 

Neural networks in computer science

To understand deep learning, you really have to begin with artificial neural networks (also known as neural nets, or ANNs).

ANNs are a form of artificial intelligence (AI), specifically a type of AI called machine learning. In essence, ANNs are computing systems that try to understand things and make decisions in a human-like manner.

Artificial neural networks get their name from brain cells (neurons) because how they work is somewhat similar to the way in which the wiring works in the living human brain and nervous system.

They’re a large collection of units (also known as nodes, or neurons) which are interlinked in a pattern that allows the units to communicate with each other. Each artificial neuron receives signals, then processes them and can signal to neurons connected to it.

Every connection link between neurons is associated with a ‘weight’ containing information about the input signal. As data is fed into it and the ANN works on solving a problem, the weight usually either boosts or dampens the signal.

Neurons also have a so-called activation signal. When input signals and the activation rule are combined and a specified threshold value (or ‘bias’) within the node is reached, an output signal is produced that can then be sent to other neurons. If the threshold value isn’t reached, there is no signal.

Layers of complexity

Neurons are usually aggregated into layers. Different layers may perform different transformations on their inputs. Signals go from the input layer — the first layer — through the next layer and usually more hidden layers to the final layer, known as the output layer. Sometimes the signals go back and forth in a nonlinear fashion through all the layers multiple times.

When human brains learn, connections are made between neurons, or changed, or they are removed altogether. Likewise, an ANN adjusts its weights to account for each new set of input data that are used to ‘train’ it. Every new data set helps the ANN to learn and become ‘cleverer’. Once their accuracy has been honed, these machine learning algorithms become powerful tools for rapid classifying and clustering of data.

ANNs are a central part of a subset of machine learning: namely, deep learning. By processing data through a variety of neural networks, it is possible to do some amazing things: everything from Google searches to human speech recognition to creating self-driving cars.

Types of artificial neural networks

There are a number of types of ANN, which include:

• Recurrent neural networks (RNNs) – these are neural networks with feedback loops. These learning algorithms can be fed with time series data to make predictions about future outcomes, such as sales forecasts.
• The perceptron – the oldest and simplest form of a neural network, with a single artificial neuron. These were inspired by the neurons in human brains and consist of one or more inputs, a processor and only one output.
• Feedforward neural networks – these have an input layer, a hidden layer or layers, and an output layer. Data is usually fed into these models to train them, and they are the foundation for computer vision, natural language processing, and other neural networks.
• Convolutional neural networks (CNNs) –  usually used for image recognition, pattern recognition and other types of computer vision. They take principles from a type of linear algebra called matrix multiplication to spot patterns inside images.

So what is deep learning?

“We live in a world where, for better and for worse, we are constantly surrounded by deep learning algorithms,” says data scientist Anne Bonner from Towards Data Science. “From social network filtering to driverless cars to movie recommendations, and from financial fraud detection to drug discovery to medical image processing, the field of deep learning influences our lives and our decisions every single day.”

Deep learning is a subcategory of machine learning, and it simulates the complex decision-making power of the human brain. It uses many layers of artificial neural networks, known as deep neural networks (DNNs). Typically, DNNs are examples of the feedforward networks or recurrent neural networks mentioned above.  

DNNs have at least three or more layers of neurons, however most DNNs have lots more than this. During the learning process, DNNs are trained on large amounts of data use cases to spot and categorise phenomena; recognise patterns, links and relationships; evaluate possibilities; and make predictions and decisions. While a single-layer neural network can make useful, approximate predictions and decisions, the extra layers in a DNN help refine and optimise those outcomes for greater accuracy.

Important in deep learning is the backpropagation algorithm, which is the method for fine-tuning the weights of a neural network according to the error rate obtained in the previous iteration; a standard method of training ANNs.

“Deep learning drives many applications and services that improve automation, performing analytical and physical tasks without human intervention. It lies behind everyday products and services—e.g., digital assistants, voice-enabled TV remotes,  credit card fraud detection—as well as still emerging technologies such as self-driving cars and generative AI,” explains IBM.

Deep learning is also big business. According to Codecademy, deep learning is also now “one of the hottest up-and-coming job sectors in the world, with a market currently ranging between $3.5 and $5.8 trillion”.

Some real-world deep learning use cases

Industrial automation and driverless car technologies aside, deep learning is used in an increasingly broad variety of ways in our modern world, including:

• Defence systems – by picking out objects and areas of interest, deep learning can determine whether military personnel can land safely in war zones.
• Medicine and healthcare: Neural networks have been used to diagnose cancer  by training algorithms that can identify cancer cells at the microscopic level at the same level of accuracy as specialist doctors. ANNs have been trained to spot the early stages of eye diseases by training algorithms on thousands of patient eye scans, and facial analysis of patient photos has been used to diagnose rare diseases.
• Automatic speech and speech recognition: Deep learning can be harnessed to listen to and study speech, as well as to produce human-like speech patterns. For example, Amazon’s Alexa and other digital personal assistants can recognise speech using a type of deep learning called Natural Language Processing (NLP) to interact with users and come up with a response.  
• Cybersecurity. Computer software incorporating deep learning is capable of detecting cyber threats based on their behaviour within a network.
• Image recognition and facial recognition. Deep learning can be trained to identify categories of images, such as people’s faces, equipment and enemy weapons or vehicles in combat situations.
• Text generation. ChatGPT is an example of AI incorporating deep learning algorithms and natural language processing that can learn the grammar and punctuation rules of a language and generate text that reads as if it has been written by a real person.
• Colouring in videos and pictures. Deep learning has saved hours of time for people who once had to manually add colour to old black and white images and footage. With deep learning, systems can automatically add a full spectrum of colour to images.
• Social media. For example, Facebook’s ‘people you may know’ feature which suggests people you might know in real life so you can connect with them, is the result of neural networks analysing your profile data behind the scenes to work out who you might know. 
• Big data analytics (BDA), where raw data is often unlabeled or uncategorized, can greatly benefit from deep learning because it’s able to analyse and learn from enormous amounts of these ‘unstructured’, or non-labeled datasets. BDA can gain powerful insights for business, public administration, national security, scientific research, healthcare, Internet of Things (IoT), commercial recommendations, stock exchanges and much more.

 

Secure the skills for success in a fast-moving field

Want to get a deeper insight into deep learning? The University of Wolverhampton’s 100% online MSc Computer Science is designed for ambitious people who may not have a background in the field but who want the skills for a career in this fast-paced, rapidly growing sector. Study on this Master’s course and you’ll be equipped with skills that are fiercely sought-after by employers across sectors and industries, from virtualisation and cloud computing to data mining, informatics and AI technologies.

Designed in close consultation with computer science experts and in alignment with industry accreditation requirements, this Master’s degree will open doors to computer science roles in the creative industries, product design, the games industry, education, public bodies, environmental monitoring and more. Furthermore, you will never need to visit campus because the course is delivered entirely online and at your own pace. Learn more about this Master’s degree in computer science.