Computer vision and the algorithms that help computers understand visual information

With rapid advancements in computers and technology, there’s a lot we’ve learned to take for granted or that may seem simple or straightforward.

A good example of this is the facial recognition functionality on most smartphones. It’s a common feature today, but sitting behind it is a complex subset of artificial intelligence (AI) known as computer vision, which is the product of a long history of innovation and a whole host of AI technology and algorithms.

In short, this technology enables computers to ‘see’ – and it’s anything but simple. Machine learning algorithms work to interpret and understand visual data, and through this technology, computers can mimic human vision and pattern recognition, identify and process objects in digital images and video, and support everything from automation to the automotive industry.

How computers use algorithms to ‘see’ and understand visual data

Computer vision algorithms – the step-by-step instructions given to computers to help them perform specific tasks, such as image analysis or identifying objects within digital imagery – translate visual datasets from photos and videos into a format that machines can understand. 

These algorithms essentially train computers to interpret and understand the visual world. They use existing digital images and deep learning models or artificial neural networks to teach computers how to accurately identify and classify objects, or make decisions based on what they ‘see’ during image processing.

“Computer vision needs lots of data,” IBM explains in an article about computer vision. “It runs analyses of data over and over until it discerns distinctions and ultimately recognises images. For example, to train a computer to recognise automobile tires, it needs to be fed vast quantities of tire images and tire-related items to learn the differences and recognise a tire, especially one with no defects.”

And if enough data is fed through the model, IBM adds, the computer can ‘look’ at the data and teach itself to tell one image from another: “Algorithms enable the machine to learn by itself, rather than someone programming it to recognize an image.”

Common algorithms and models used for computer vision

As we know, the field of computer vision has evolved to handle complex tasks like facial recognition, object tracking, and even autonomous vehicle navigation. Some of the key algorithms or learning models that make these technologies possible include:

• Convolutional Neural Networks (CNNs). Commonly used for classification and image recognition tasks, CNNs can recognise patterns in images. In fact, CNNs are at the heart of many advanced computer vision systems. These deep neural networks are particularly powerful for analysing visual imagery. They use a mathematical operation called convolution that processes data through a grid of stored values, effectively enabling the network to ‘see’ various aspects of an image and learn from them. This capability makes CNNs exceptionally useful for computer vision tasks such as image classification and object recognition.
• Generative Adversarial Networks (GANs). Often used in augmented reality and artistic content generation, GANs can use training images or data to generate new content.
• You Only Look Once (YOLO). Particularly useful for real-time object detection, YOLO divides images into regions and predicts the likelihood of an object’s presence in each region.

Other computer vision techniques and processes

There are a lot of processes and machine learning techniques that make computer vision possible or more efficient. Examples include:

• Image segmentation. By dividing a digital image into multiple segments, this process  can simplify the image for the machine and make it easier to analyse. 
• Semantic segmentation. This area of image segmentation assigns class labels to different groups of pixels to help machines process the visual data.
• Feature extraction. Before a computer can recognise an object in an image, it must process that image into a form it can understand. This involves converting an image into a set of digital data points or pixels. Algorithms then perform feature extraction to identify distinctive attributes – like edges, textures, or specific shapes – in the images. This feature extraction process identifies and extracts the most important or relevant patterns in visual data.

Real-world applications of computer vision technology

The practical applications of computer vision in various industries are growing alongside advancements in AI and computing power. From social media filters to self-driving cars, machine vision technology underpins many of our modern digital conveniences. 

But it’s also helping in life-changing ways, too.

In healthcare, for example, computer vision can assist in reading medical imaging from MRIs and X-rays to enhance diagnostic accuracy. 

“Rare or early-onset disease can be hard to detect for even the top medical experts,” says the World Economic Forum in its 2022 article, How ‘computer vision’ could change healthcare, retail and more. “However, when machines are comparing hundreds of thousands of images, anomalies that are hard for the human eye to detect can be brought to the attention of a doctor. Such insights can bring dramatic positive outcomes to patients and the health care organisation’s that support them. Disease identified early can save time, money, and lives.”

Other applications include:

• Autonomous vehicles and robotics. Through cameras and sensors, self-driving cars use computer vision systems to navigate and understand their environment, from detecting pedestrians to recognising traffic signals. Similarly, robotics uses computer vision for various tasks such as object manipulation, navigation, and complex decision-making.
• Social media and facial recognition. Platforms like Facebook and Instagram can use face recognition technologies to tag people in photos automatically. This technology uses deep learning algorithms to analyse facial features and match them with identities in their database. However, due to privacy concerns, Meta discontinued this feature in recent years.
• Augmented reality. Augmented reality systems use computer vision to integrate digital information into the real world in real time. This technology has applications ranging from smartphone AR games to head-up displays in cars, where digital content is overlaid on the physical world to enhance the user’s interactions with their environment.

Launch your career in computer science

Learn about the algorithms that make computer vision – and the wider field of artificial intelligence – possible with the 100%-online MSc Computer Science at the University of Wolverhampton. This flexible Master’s degree has been developed for ambitious individuals who may not have a background in computer science.

You’ll develop comprehensive knowledge and skills across specialised and applied areas of computer science, including:

• artificial intelligence technologies
• Internet of things (IoT) security
• virtualisation and cloud computing
• data mining
• networking and the internet