Home General Facial Recognition: All you Need to Know

Facial Recognition: All you Need to Know

Facial Recognition Technology

Facial recognition system is fast becoming one of the most important and controversial technology.

Facial recognition is a method of recognizing a human face with technology. A facial recognition system uses biometrics to chart facial features from an image or video. It compares the data with a database of known faces to confirm a match.

It is already present in smartphones and similar intelligent technology.

Increasingly efforts are developing and improving the applications.

From personalising the customer experience to optimising business performance facial recognition software has a number of exciting applications.

As well as explaining exactly what facial recognition software is this article will explore exactly how it works.

We will also highlight a number of current use cases, illustrating exactly how facial recognition software is already transforming a number of industries from banking to consumer goods.

What is Facial Recognition?

Facial recognition is the process of verifying or identifying the identity of a person with their face. Facial recognition software captures and analyzes the person’s facial features.

From Facebook’s ability to tag photographs to security surveillance and biometric tools to unlock your phone.

Facial recognition is already being used in a number of real-world scenarios.

Facial recognition software uses algorithms to map a face identifying key features.

This is then compared to known images, stored in a database.

By matching the image with one already in a database we are able to identify people.

This application can be used in everything from tagging photographs to secure entry into accounts and identifying people of interest.

How Does Facial Recognition Work?

Facial recognition software is powered by algorithms. These algorithms are usually, deep neural networks.

They are trained on large data sets of photographs of human faces in a number of positions, displaying a range of emotions.

This training means that the algorithm learns, over time, to identify faces.

Once the algorithm is able to detect a face it must learn to recognize faces.

This is done with a second neural network. Here the algorithm is again fed images of faces.

Different algorithms use different methods to distinguish between faces.

Some use biometrics. Here the face is mapped.

Unique features such as eye shape or the distance from the forehead to the chin are identified.

By mapping facial features in this way the algorithm learns to tell one face from another. Other algorithms use more abstract methods or features.

While the methods change, the aim of each approach is the same, to identify features that make a face unique.

In other words, this system is able to read the geometry of a face.

Mapping key features to create a map of facial landmarks is compiled.

This creates a facial signature. Your facial signature is a mathematical formula that the algorithms work out following the mapping of the face.

This can be used to accurately compare your face to others stored in a database.

Facial recognition can also be used to analyse individual portraits, crowd shots or videos.

SEE MORE: Sensetime: Inside the World’s Biggest Facial Recognition Startup

Facial Recognition Algorithm: LBPH Algorithm

One of the most established facial recognition algorithms is LBPH (Local Binary Patterns Histograms). The LBPH algorithm is one of the simplest, using 4 parameters to read a facial image. The algorithm is then trained on a data set, using the set parameters to identify one image from another.

Once trained the algorithm is reliably able to identify are recognise images.

It is particularly reliable in controlled environments and on grayscale images.

Description of facial expressions with local binary patterns. LBPH algorithm is one of the simplest facial recognition algorithm using 4 parameters
Description of facial expressions with local binary patterns. LBPH algorithm is one of the simplest facial recognition algorithm using 4 parameters

 

Other Useful Algorithms

Other facial recognition algorithms are more complex than LBPH.

Linear Discriminant Analysis, or LDA, is an abstract approach.

It is based on Fisher’s Linear Discriminant Analysis and Support Vector Machine (SVM).

LDA works by projecting high dimensional image space into low dimensional space.

This allows the algorithm to identify discriminant features.

Another adaptive approach is Evolutionary Pursuit.

This uses Eigenspace to compare facial structures, rendering 2D images of the human head. Symmetrisation techniques allow for problems such as poor lighting to be rectified.

This approach allows new images to be represented as linear outputs.

Linear outputs make it easier to match images and identify similarities.

Elastic Graph Matching is a facial recognition algorithm inspired by biology.

The EGM method of analysing visual features is based on Gabor wavelets.

This means that EGM provides a reliable way of modelling the brains early visual processing.

In simple terms EGM analyses the topological structure of the face, creating graphs with nodes at fiducial points such as the nose.

EGM is a robust algorithm which regularly returns high recognition rates.

This approach is also able to identify faces from images taken at different angles and with different lighting and backgrounds.

SEE MORE: World’s Largest Facial Dataset for Studying Bias Released by IBM

3-D Modelling in Facial Recognition

3-D Face Recognition is a non-invasive biometric technique that works by allowing the algorithm to compare surfaces.

This is done independently of natural deformations such as changes in facial expressions.

Replicating a natural recognition process in this way has a number of advantages of more conventional 2D facial recognition systems.

The versatility of 3D recognition algorithms has a wider range of applications.

It can also be used in a wider range of less than ideal scenarios.

For example, 3-D Face Recognition can recognise faces in poor lighting.

It also returns reliable matches regardless of the expression or position of the face.

Facial recognition algorithms such as LBPH, Eigenfaces or Fisherfaces all aim to read the face and learn its identifying features.

How these algorithms approach this task varies.

This means that one algorithm may be more useful in a certain situation or with a certain data set than another.

Additionally, as facial recognition technology is still developing, algorithms are also constantly evolving and developing.

Complex algorithms such as 3-D Face Recognition are more reliable and capable of being used in a range of situations.

As reliability and versatility improve so will the use and reliability of facial recognition software.

Benefits and Applications of Facial Recognition Technology

Properly applied facial recognition has a number of benefits.

Firstly it can help to make us safer.

From identifying missing people to quickly arresting people accused of serious crimes, recognition technology has an important role to play.

Facial recognition systems are already helping to protect national borders and buildings.

Beijing's New Airport Daxing will use advanced facial recognition software. Image Credit -China Daily
Beijing’s New Airport Daxing will use advanced facial recognition software. Image Credit -China Daily

Airports, in particular, are increasingly making use of technology.

The Department of Homeland Security in the U.S. predicts that 97% of travellers using American airports will be scanned by facial recognition software by the year 2023.

In smaller applications, the technology can help small business owners to deter petty criminals.

The technology also allows for seamless identity verification.

Facial recognition software has a number of potential uses ranging across a number of industries.

Here are some of the most interesting current applications.

Facial Recognition in Banking

Banks need to be secure and safe.

With a focus on constantly improving safety and security banks are constantly looking at new applications and technologies.

A number of banks are already using Apple’s Face ID to allow customers to safely access their account on a mobile banking app.

HSBC's facial recognition software creates a detailed “depth map” of 30,000 points to recognise someone's face
HSBC’s facial recognition software creates a detailed “depth map” of 30,000 points to recognise someone’s face

These banks include USAA and HSBC.

HSBC’s application requires customers to look at a screen for less than a second before their image is verified.

The facial recognition software behind this creates a detailed “depth map” of 30,000 points.

Impressively this map is able to adapt and learn when a person’s appearance changes, for example as they age or begin to wear glasses.

Niall Cameron is HSBC’s Global Head of Corporate and Institutional Digital.

He revealed that HSBC is “pioneering this technology within the financial services sector for businesses and corporates.

Our unparalleled global digital footprint allows us to deploy new technology like this quickly and at scale to make a real difference to our customers.”

SEE MORE: Brazil’s Largest Bank Launches AI-Powered Cameras

Despite Limitations, This is a Popular Application

Currently, customers in 24 countries, including China, the US and the UK are able to use facial recognition application to access HSBCnet.

However, it is currently limited to those customers who have an iPhone X because of its reliance on Apple’s Face ID technology.

Despite this limitation, the application is growing in popularity.

Diane Reyes is Global Head of Liquidity and Cash Management at HSBC.

Reyes revealed that “HSBCnet mobile use has grown by 60 per cent in the last year alone, with an equivalent growth in value.”

Lloyds Bank is experimenting with Microsoft's biometric authentication
Lloyds Bank is experimenting with Microsoft’s biometric authentication

Meanwhile, Lloyds Bank is experimenting with Microsoft’s AI biometric authentication technology to provide secure banking.

With a focus on facial recognition software, MasterCard is experimenting with their “selfie pay” feature.

This allows customers to use their smartphone camera to approve online purchases.

MORE: 10 Applications of Machine Learning in Finance

Using your Face to Personalise the Driving Experience

Facial recognition software is being developed to act as a method of driver authentication.

Automotive manufacturers believe that once proved to be reliable this can help to greatly reduce car theft.

Intel and Ford partner on facial recognition software to identify the known primary driver as well as other authorised driver
Intel and Ford partner on facial recognition software to identify the known primary driver as well as other authorised driver

Intel and Ford are working on this application under the name Project Mobil.

This sees dashboard cameras using recognition software to identify the known primary driver as well as other authorised drivers.

Eventually, this application will see the car refusing to start should an unknown driver sit in the driver’s seat.

This application of facial recognition software will also help to personalise the driving experience.

By recognising the driver the car’s settings on items such as seat position or music volume will automatically calibrate.

Ford is also exploring facial recognition and biometric device applications that could replace the need for a car.

Chrysler's Portal minivan with use facial recognition software to recognise primary drivers.
Chrysler’s Portal minivan using facial recognition software to recognise primary drivers.

Similarly, Chrysler is developing the use of cameras in their Portal minivan.

The company is also aiming to develop vehicles that recognise the primary drivers and automatically adapt.

Like Ford and Intels developments, this will allow for the personalisation of the driving experience.

Applications to Improve Patient Care

London’s famous Great Ormond Street Hospital is just one healthcare provider embracing the possibilities of recognition software.

One exciting application has seen them team up with the London Science Museum.

The Me in 3D exhibition at the Science Museum utilises a series of cameras to build a virtual image of visitors.

These images can then be viewed and manipulated.

The data collected by this exhibition is being used by researchers at Great Ormond Street Hospital alongside researchers at Eastman Dental Hospital and Institute and the University College Hospital.

Great Ormond Street Hospital Embracing Facial recognition technology
Great Ormond Street Hospital Embracing Facial recognition technology

This data is helping researchers develop better treatment plans and surgery options for disfigured patients.

It is also helping to improve the care of patients suffering from congenital conditions.

Dr Chris Abela is a senior craniofacial fellow at Great Ormond Street.

He explained that “We know a lot about the bones in our faces but little is known about what makes our face the shape it is and about the skin and muscles that make up our face.”

The Me in 3D exhibition is allowing the institutions to gather more 3D facial images.

A wide data set will allow them to develop “a greater understanding of our complex faces, and have greater knowledge to plan and perform the best facial surgery in the future.”

Recognition Software is at the Forefront of Patient Care

Arm is aiming to improve hospital care.

With this goal in mind, they have partnered with Great Ormond Street Hospitals Digital Research, Informatics and Virtual Environments (DRIVE) initiative.

Together they aim to use innovative, smart technologies and implement them within Great Ormond Street Hospital and eventually across the NHS.

Much of Arm’s focus will be on artificial intelligence applications and processes.

Recognition software is currently being developed to allow known visitors, as well as permanent staff, to automatically check-in.

This system will also automatically log entry and exit times. By allowing security to know exactly who is in the hospital at any given time it can make the buildings safer.

Taking this application a step further the movement and location of any individual can also be tracked.

This makes it easier to locate staff in an emergency, possibly helping to save lives.

This is an example of frictionless technology.

By relying on cameras visitors do not have to scan their irises or log in.

A person can be tracked from the moment they walk through the door until the time they leave.

Improving ICU Patient Care

Meanwhile, in Japan, facial recognition software is being developed that can predict if an intensive care unit patient requires help.

This application is currently returning impressibly high accuracy rates of around 75%.

This application means that staff shortages need not endanger patient safety.

Dr Akane Sato from Yokohama City University Hospital heads the research team behind this application.

She revealed that they were “surprised about the high degree of accuracy that we achieved, which shows that this new technology has the potential to be a useful tool for improving patient safety, and is the first step for a smart ICU which is planned in our hospital.”

As we can see applications of facial recognition technologies, while only in their infancy, have the potential to improve patient care and safety.

Facing up to Insurance Fraud

The Association of British Insurers claims that in 2016 fraudulent claims can cost the insurance industry in the U.K. around £1.3 billion.

It is thought that an additional 125,000 fraudulent claims go undetected each year.

With so much money at stake, it is unsurprising that insurance companies are frequently turning to smart technology to increase security and detection.

Facial Recognition to act as a Lie Detector

The next generation of facial reading software will see applications able to detect emotions.

These tools can also highlight small indicators of suspicious behaviour such as pupil dilation or, changes in vocal tones or speech patterns.

All of this can indicate that a person is lying.

For insurers using this technology to record people making a claim may help to tell if the claimant is honest or attempting to make a false claim.

One such application has been developed by WeSee.

Their recognition systems work in harness with the claim handlers smartphone camera.

The claim handler would simply film the claimant with their smartphone and the system would analyse their statement and face.

Ping An Insurance is using facial recognition software to determine whether claims are fraudulent.
Ping An Insurance is using facial recognition software to determine whether claims are fraudulent.

Similarly, Ping An Insurance, a Chinese insurer, is also using facial recognition software to determine whether claimants are fraudulent.

One of the worlds largest financial service providers, Ping An have been working on this application since 2016.

In the same year, the company also announced plans for facial recognition-based loan products.

Ping An has been able to make large strides in this area.

The company sees this application of facial recognition software as a means of gaining an edge over competitors.

Their fintech subsidiary OneConnect is pivotal in this drive.

By using facial recognition software to identify fraudulent claims insurance companies will be able to save money and cut payments.

In turn, this will lead to the cost of premiums also falling.

Leading the Fight Against Crime

While there are privacy concerns surrounding facial recognition software this hasn’t stopped its use from steadily increasing.

Research by Component suggests that, in the United States alone, the facial recognition industry will grow to a value of $7 billion by 2021.

One of the key areas where facial recognition software is making an impact is in the field of law enforcement.

Many U.S. police departments, including Chicago and New York, use the software to monitor crowds and identify people of interest.

Body camera, frequently worn by law enforcement officers, can also be fitted with facial recognition software.

However, some states such as California have outlawed this application.

In 2018 a report revealed that the FBI also uses the technology and is thought to have a database of over 412 million facial images.

The UK's Metropolitan Police have experimenting with facial recognition technology.
The UK’s Metropolitan Police have experimented with facial recognition technology.

Policing Concerns in the UK

In the UK, both South Wales Police and the Met Police have experimented with recognition technology.

The Met Police has been using Live Facial Recognition, which is powered by the Japanese company’s NEC’s NeoFace technology.

This application has seen the system analysing and learning the images of people on a police watch list.

Body cameras worn by police officers are also equipped with the software.

This means that each time a police officer interacts with a member of the public their image is analysed and compared to those on the watch list.

Aware of safety and privacy concerns this facial scan will only take place with the permission of the member of the public.

Additionally, the scanned data can only legally be kept for 30 days.

After this time it is destroyed. So far these trials have not been a complete success.

University of Essex academics were able to access 6 of the Mets live trials in areas such as Soho and Romford.

Analysis by the academics revealed that people were often misidentified and also warned of “surveillance creep”.

Former UK shadow home secretary David Davis warned that unless these issues were rectified facial recognition software could cause “miscarriages of justice and wrongful arrests.”

Researchers also raised concerns about intentional, or unintentional bias.

Supported by research conducted in America researchers warned that current recognition software is prone to misidentifying people with dark skin tones.

Delta Airline launches the first biometric terminal at Atlanta airport in the US

Border Control

The U.S. Customs and Border Protection agency have partnered with Delta.

This will see facial recognition software in the form of biometric scanning used at various airports in the United States.

Recently Los Angeles International Airport became the latest airport to implement the technology.

Rimmel's new Get the Look app lets you snap any look and try it on live
Rimmel’s new Get the Look app lets you snap any look and try it on live

Experimenting With Virtual Makeup

Interestingly many beauty brands and product manufacturers are also excited by the possibilities of facial recognition software.

Here facial software and VR applications can help a customer to try out new looks and styles without going to the expense of a time-consuming makeover.

This application may prove especially attractive for nervous customers, or those apprehensive of wasting money and being stuck with a new style that really doesn’t suit them.

Rimmel is just one brand using augmented reality and facial recognition software in this way.

Covergirl is using similar software in its Custom Blend App.

This helps the user to find a shade of foundation that complements their skin tone.

Essentially this allows the customer to virtually try on makeup.

L’Oréal is experimenting with facial recognition software.
L’Oréal is experimenting with facial recognition software. The company’s virtual try-on feature is on their website which allows you to try on hair colour, makeup and lipstick before you buy.

L’Oréal Paris is also experimenting with facial recognition software.

The company’s virtual try-on feature is located on their website.

Utilising ModiFace technology users can experiment with cosmetics and hair colors.

By giving the customer a realistic expectation of how they can look the company hopes to increase sales and brand loyalty.

The more satisfied a customer is with their purchase then more likely they are to repeat buy.

Driving Safety Improvements in the Energy Industry

For the energy industry, facial recognition software has a range of potential applications.

Here the software may be able to improve safety, security and the health of drivers.

It has been reported that one in six crashes that cause death or injury is caused by driver fatigue.

In heavy goods vehicles, around 40% of crashes are caused by driver fatigue.

With this in mind, Subaru and Chevron are already experimenting with facial recognition technology.

The idea is that cameras installed in the vehicle will identify when a driver is becoming fatigued.

By identifying when a motorist is fatigued they can be encouraged to break their journey for a rest period.

This will improve safety and productivity, optimising long-distance deliveries such as fuel transportation.

MORE: World’s Top 33 Companies Working on Self Driving Cars

Subaru using facial recognition technology to detect driver fatigue
Subaru using facial recognition technology to detect driver fatigue

Making Energy Plants Safer

Digital Barriers is aiming to deliver facial recognition software that will improve plant security.

Their systems ensure that only recognised personnel with the right level of clearance are able to access restricted areas.

A similar application has been developed by AllGoVision.

Like Digital Barriers they are using facial recognition and video analytics software to increase on-site safety.

AllGoVision solutions are tailored to the Power and Oil and Gas industries.

It has already been adopted by Bharat Oman Refineries Limited, one of the largest oil refineries in India.

MORE: 10 Applications of Machine Learning in Oil & Gas

Personalising the Consumer Experience

Facial recognition software is being used to personalise the customer experience.

Fast-food giant KFC has teamed up with Baidu, the Chinese search engine, to develop software that can predict your order.

A KFC spokesperson explained that the system works by recommending “menu items based on a customer’s estimated age and mood.”

Fast-food giant KFC has partnered with Baidu, the Chinese search engine, to develop software that can predict what your order based on your mood.

So a young male would be offered a different meal to an older woman, for example.

This application of facial recognition software and AI tools is primarily for convenience.

Zhao Li, general manager of KFC in Beijing explained that the “digitalisation of the restaurant will also help to provide faster and easier services.”

Other retailers, such as CaliBurger a California based food chain, is linking facial recognition software to their loyalty program.

As a registered customer engages with an ordering kiosk, they are recognised and their loyalty account is activated.

This means that a customer is instantly presented with meal suggestions, based on previous purchases.

Matching Adverts to Your Face

This will not only help to speed up ordering and transaction time it may also help to improve customer satisfaction.

Meanwhile, Walmart is developing applications that can read a customer’s facial expression and mood.

By gauging the level of satisfaction of customers in-store the company can alter in-store displays and real-time promotions.

UK retail giant Tesco has installed facial recognition at its gas stations.

Here the application is not to improve security but to personalise advertising.

As customers wait in line digital screens will display adverts tailored to the user’s age, gender and appearance.

This is made possible by the use of Optimeyes, an Amscreen development.

Simon Sugar, Amscreen’s CEO is excited by this application.

MORE: Artificial Intelligence in Marketing – 6 Examples Making an Impact

UK retail giant Tesco has installed facial recognition at its gas stations to personalise advertising.
UK retail giant Tesco has installed facial recognition at its gas stations to personalise advertising.

He said that “this could change the face of British retail and our plans are to expand the screens into as many supermarkets as possible.

Brands deserve to know not just an estimation of how many eyeballs are viewing their targets, but who they are too.”

Helping Retail Outlets Identify Petty Criminals

This application of facial recognition software may make a major impact.

A 2017 report by the BRC revealed that retailers lost £700 million because of low-level crimes such as shoplifting.

With police resources increasingly strained it is often left to the stores themselves to provide and increase security.

Retail companies are also recognising the possible benefits of using facial recognition software in their stores.

The main problem that this software will be able to counteract is theft and shoplifting.

Aylesbury based supermarket Budgens is managed by Paul Wilks.

The store has noticed that thefts are carried out by a small number of repeat offenders.

In an attempt to counteract their problem Wilks installed Facewatch.

Facewatch provides facial recognition software that monitors people entering the store, comparing their images to a database of a known or suspected shoplifter.

If a match is identified the system alerts the store manager and the suspect can be removed from the store before they are able to steal anything.

The company claims that its system cuts theft by 35%.

MORE: 10 Powerful Applications of AI in Retail

Applications of Facial Recognition Systems in the Gaming Industry

Earlier this year, the Macau Daily Times reported that casinos in the region were introducing facial recognition software.

Paul Martins Chan, director of the Gaming Inspection and Coordination Bureau, explained that casinos “have to strictly obey the personal data protection law.”

Casinos in Macau have introduced facial recognition software
Casinos in Macau have introduced facial recognition software

Chan also explained that any equipment installed by these casinos must first be approved by his bureau.

As with other applications of facial recognition software some people have voiced privacy concerns.

Here in Macau, the possibility of the governing party using the software to enforce bans has been raised.

Chan, in the same interview, responded to these concerns.

In Chan’s view “it will take time for the parties involved to comprehend the personal data protection law.”

Chan believes that by trailing the software in a few, small scenarios first will allow them to work out any issues before a large scale roll-out is implemented.

Lawmakers such as Lam Lon Wai believe that privacy is too large a concern.

Instead, they are campaigning for an increase in the use of recognition software, and surveillance cameras.

The view of Wai and others is that increased surveillance, targeting known criminals is the best way to help cut the areas rising crime levels.

League of Legends, the American version of Honor of Kings uses facial recognition to monitor players.
League of Legends, the American version of Honor of Kings uses facial recognition to monitor players.

Facial Recognition to Safeguard Gamers Eyesight

It is not just gamers in casinos that will soon be monitored as they play.

Tencent, a large Chinese technology company, has announced its intention to use facial recognition to monitor players of the Kings of Glory video game.

An online, multiplayer battle game it is also known as Arena of Valor.

In China, the game is incredibly popular, reportedly one of the highest-grossing online games.

Tencent unveiled their plans with the announcement that 1000 randomly selected players will have their faces mapped and matched to their ID photo held in China’s citizen database.

This is made possible because since 2017 gamers have had to provide their real name before being allowed to access the game.

This application of recognition software means that minors are only allowed access to the game for a certain amount of time each day.

The software is also capable of blurring the screen if the players face gets too close to the screen.

This application of recognition technology is in response to criticism that screen time can be addictive and damaging to the health of the gamers.

A 2015 report revealed that almost half of China’s population suffer from some form of visual impairment.

Additionally, since 2008 Chinese health officials have recognised internet and gaming addiction as a serious disorder.

By applying facial recognition software in this way companies can protect the health of gamers and users.

Facial Recognition Software is Changing the Face of Technology

Facial recognition software has a number of potential applications. Many of these are already being trailed and applied in real-world situations.

While privacy concerns still remain, for many facial recognition software will only improve the modern world.

Images: Flickr Unsplash Pixabay Wiki & Others

Subscribe to our newsletter

Signup today for free and be the first to get notified on the latest news and insights on artificial intelligence

KC Cheung
KC Cheung has over 18 years experience in the technology industry including media, payments, and software and has a keen interest in artificial intelligence, machine learning, deep learning, neural networks and its applications in business. Over the years he has worked with some of the leading technology companies, building and growing dynamic teams in a fast moving international environment.
- Advertisment -

MOST POPULAR

AI Model Development isn’t the End; it’s the Beginning

AI model development isn’t the end; it’s the beginning. Like children, successful models need continuous nurturing and monitoring...