The Evolution Of Facial Recognition: From Bodycams To Video Surveillance

Download PDF version

Facial recognition has a long history dating back to the 1800s. To track down criminals, such as infamous bandits Jesse Woodson James and Billy the Kid, law enforcement would place “Wanted Alive or Dead” posters advertising bounties and soliciting public cooperation to help locate and even apprehend the alleged criminals.

In addition to the bounty, these posters would include a photo and brief description of the crime, which would then be circulated to law enforcement agencies around the country and displayed in every US Post Office to speed up apprehension.

Facial Recognition

Today, technology such as social media, television and other more specialized communication networks play a more influential role in the recognition process. Advancements in artificial intelligence and biometric technology, including the development of Machine Learning capabilities, have led to increased accuracy, accessibility and the widespread use of computerized facial recognition. The significance of this means that facial recognition can occur on an even larger scale and in more challenging environments.

Advancements in artificial intelligence and biometric technology have led to the widespread use of computerised facial recognition

This article will explore key milestones and technological advances that have resulted in the modern incarnation of facial recognition, before discussing the capabilities of cutting-edge “one-to-many” technology which is increasingly being used by counter-terror defense, police and security forces around the world.

Technology Inception And Developments

The 1960s marked the start of computerized facial recognition, when Woodrow Wilson (Woody) Bledsoe developed a way to classify faces using gridlines. Bledsoe’s facial recognition still required a large amount of human involvement because a person had to extract the co-ordinates of the face’s features from a photograph and enter this information into a computer. The technology was able to match 40 faces an hour (each face took approximately 90 seconds to be matched) which was considered very impressive at the time.

The technology was able to match 40 faces an hour, which was considered very impressive at the time

By the end of the 1960s, facial recognition had seen further development at the Stanford Research Institute where the technology proved to outperform humans in terms of accuracy of recognition (humans are notoriously bad at recognizing people they don’t know). By the end of the century, the leading player in the field was a solution that came out of the University of Bochum in Germany – and the accuracy of this technology was such that it was even sold on to bank and airport customers.

From this stage on, the facial recognition market began to blossom, with error rates of automatic facial recognition systems decreasing by a factor of 272 from 1993 to 2010 according to US Government-sponsored evaluations.

The safety and security market requires near instant feedback on who a person matched against a watchlist is

The aim for facial technology is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware

Modern Usage Of Facial Recognition

Fast-forward to the modern day and facial recognition has become a familiar technology when using applications such as the iPhone X’s Face ID capability or MasterCard Identity Check, passport e-gates at airports and other security and access control points. These solutions implement a consensual form of identity verification, as the user has a vested interest in being identified.

This is a “one-to-one” facial recognition event, one person in front of the camera being compared to one identity either on a passport or the app. In these scenarios, the hardware is specifically developed for the application at hand, therefore technically much easier to accomplish.

Facial recognition can now be used in a variety of governmental and commercial environments The safety and security world brings a much more complex problem to solve – how to pick out a face in a moving and changing environment and compare it to several faces of interest. “One-to-many” facial recognition is a much harder problem to solve.

It’s even more challenging when the aim is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware. And unlike in the 1960’s where identifying a face every 90 seconds was acceptable; the safety and security market requires near instant feedback on who a person matched against a watchlist is.

Security And Safety Applications

The idea behind all facial recognition technologies is broadly the same: you start with an image of a person’s face (ideally a high quality one, although machine learning means that to a point we can now even use video without reducing accuracy). A fully front facing image is best, think a passport photo, but machine learning and new software has made this more flexible.

An algorithm converts this image into a numeric template, which cannot be converted back to an image and so represents a secure one way system. Every numeric template is different, even if it started out as an image of the same person, although templates from the same person are more similar than templates from different people.

The accuracy of facial recognition continues to increase alongside deployments in more challenging and complex environments What happens next sounds simple although the technology is extremely complex: templates of people’s faces are taken in real time and compared to those in the database. The technology identifies individuals by matching the numeric template of their face with all the templates saved in a database in a matter of seconds or milliseconds. To put this into perspective, imagine you are at the turnstiles of a busy train station looking for a person on the run.

Today’s facial recognition technology would be able to identify that person should they pass in view of a CCTV camera, as well as notify the police of any additional persons of interest, whether they are a known terrorist or missing vulnerable person on an entirely separate watch list.

Because of technical progression, facial recognition can now be used in a variety of governmental and commercial environments, from identifying barred hooligans attempting entry at a football stadium or helping self-excluded gamblers at casino to overcome addiction.

Real-Time Assessments

The latest evolution of facial recognition pits the technology against an even more challenging application – directly matching individuals from body worn cameras for real time recognition for police officers on the beat. This capability equips first responders with the ability to detect a person from a photo and verify their identity with assurance.

The broader implication for this means that every interaction, such as stop and search or arrest, can be supported by real-time facial recognition which will see cases of mistaken identity driven down on the streets. First responders can now for the first time be deployed and furnished with the ability to identify wider groups of people of interest with a degree of accuracy that previously relied only on the fallible human memory.

As the accuracy of the technology continues to increase alongside deployments in more challenging and complex environments, its ability to support government initiatives and law enforcement means the debate about the lawful and appropriate use of facial recognition must be addressed. Facial recognition should not be everywhere looking for everyone, but when used properly it has the potential to improve public safety and we should make the most of its potential.

Download PDF version Download PDF version

Author profile

Fernande van Schelle Product Manager, Digital Barriers

Fernande van Schelle is a product manager for facial recognition technology with Digital Barriers. With over a decade of experience, she leads the product management and development activities to deliver facial recognition capability using deep neural networks. Fernande holds a master’s degree in Business Administration from INSEAD and a master’s in history of International Relations from LSE.

Related companies
Digital Barriers

Related links
Face Recognition Software Access control software
Articles by Fernande van Schelle

View all news from
Digital Barriers

In case you missed it

How Businesses Can Prepare Their Communications Infrastructure And Critical Event Management Plans For The Next Chapter Of The Pandemic

The global pandemic has created a working environment filled with uncertainty and, at times, fear, as COVID-19 cases surge yet again and businesses continue to navigate a complex web of infectious disease mitigation protocols and managing the distribution of a potential vaccine. Organizations are operating in an environment where a critical event, posing significant risk to its employees and daily operations, could occur at any moment. Even with a vaccine showing light at the end of a very dark tunnel, the pandemic unfortunately may be far from over, and the communication of accurate public health information to a widely distributed, often remote workforce is vital to keeping employees safe and businesses running. Organizations that plan ahead, invest in an emergency management system and share key updates quickly, reliably and securely, can keep employees safe while ensuring business continuity when it matters most. Taking time to plan and prepare Throughout the pandemic, U.S. offices have gone through alternating stages of reopening and re-closing Throughout the pandemic, U.S. offices have gone through alternating stages of reopening and re-closing. However, whether businesses are operating at a limited or full capacity, medical experts are expecting continuous waves of COVID-19 cases, as community transmission continues to hit record highs. The only way for businesses to keep their employees and customers safe, protect their operations, and retain trust with their key stakeholders during these tumultuous times is to be proactive in nature. Organizations need to put a business resiliency plan in place now that outlines key actions to take if (or when) an issue relating to local spread of the novel coronavirus arises. By having a plan in place and practicing it regularly, organizations can minimize risks and maximize employee safety surrounding critical events, such as suspected or confirmed exposure to COVID-19 in the workplace. Context of emergency management Ensuring the safety of employees (and others within the company’s facilities) needs to be the number one priority for organizations; and in any crisis scenario, a prepared and practiced plan maximizes a company’s chances of success. In PwC's 2019 Global Crisis Survey, business leaders across a range of industries shared their experiences, expectations, and top strengths and weaknesses in the context of emergency management. By a wide margin (54% vs. 30%), organizations that had a crisis response plan in place fared better post-crisis than those who didn’t. When it comes to ensuring the wellbeing of staff, businesses need to think through a comprehensive, iterative infectious disease mitigation and operational continuity strategy and practice it as often as possible. Investing in proper technology A vital step in adequate critical event management planning is investing in the proper technology infrastructure For today’s modern organization, a vital step in adequate critical event management planning is investing in the proper technology infrastructure to streamline the communication of vital information. Organizations should explore risk intelligence, critical communication and incident management software to keep their people safe, informed, and connected during critical events - and, thankfully, decision-makers are starting to take note. The Business Continuity Institute 2020 Emergency Communications Report found that 67% of organizations at least use emergency notification and/or crisis management tools. Reliable risk intelligence system Building upon that trend, a reliable risk intelligence system can anticipate and analyze the potential impact of incidents, such as increases in local cases of COVID-19, send vital updates to a distributed workforce of any size on multiple devices regarding infectious disease mitigation protocols and public health directives, and then help incident response teams virtually collaborate while maintaining compliance standards. Automating as much of this process as possible through technology allows human decision makers to efficiently and effectively focus their time, effort and expertise on what matters most in a crisis situation - implementing sound operational continuity strategies and, more importantly, ensuring employees’ safety and well-being are prioritized and appropriately considered when stress rises. Communicating vital updates This is the cultural component of incident management based on emotional intelligence, empathy, effective employee engagement, and authentic listening that makes or breaks an organization’s response to challenging situations. Employees must be aware at a moment’s notice to stay away from or exit contaminated areas If employees are exposed in the workplace to a confirmed or suspected case of COVID-19, employers must be prepared to quickly update staff on vital next steps, as outlined by the CDC and other public health authorities, and arm key functions - such as security operations, HR, facility management, legal and compliance - with the information they need to mitigate potential spread of the virus, including: Closing/cleaning the office: Employees must be aware at a moment’s notice to stay away from or exit contaminated areas. From there, it is critical that businesses communicate clearly with cleaning staff to follow procedure, use the right disinfecting products and sanitise high-touch surfaces. Alerting key groups that may have been exposed: Employers have a duty to rapidly notify workers of potential exposure to COVID-19. Having the proper communication infrastructure in place can streamline contact-tracing as well as the subsequent testing process, and save vital time. Ensuring work-from-home continuity or diverting workflows to alternative physical environments: Every work environment looks different today. Whether an organization is managing a distributed workforce, full-capacity essential workers or something in between, there needs be a communication system in place to ensure business continuity. Outlining next steps for reopening: After a potential exposure, employees require the proper reassurance that they will be returning to a safe working environment in an organized, thoughtful manner, which is aligned to public health best practices. Whether it is coordinating a limited capacity return to the office or outlining new infectious disease mitigation protocols - such as steps for receiving a vaccine in the coming months - employees must continue to be updated quickly, comprehensively, and often. Incident management technology There is no doubt that organizations will continue to face a myriad of challenges as they navigate business operations during the pandemic into 2021, as the general public awaits the broad deployment of a vaccine. Public and private sector leaders still have months ahead of them before daily operations even begin to resemble “business as usual.” To best prepare for the next chapter of the global pandemic, organizations should outline a plan tailored to infectious disease mitigation protocols; explore augmenting their crisis management policies with risk intelligence, crisis management and incident management technology; and focus employee communications on containing and rapidly resolving events associated with COVID-19 exposure. Keeping employees safe, informed, and connected during critical events are mandatory considerations for leaders as they analyze existential threats to their business in 2021 and beyond.

Unlocking human-like perception in sensor-based technology deployments

Like most industries, the fields of security, access, and safety have been transformed by technology, with AI-driven automation presenting a clear opportunity for players seeking growth and leadership when it comes to innovation. In this respect, these markets know exactly what they want. They require solutions that accurately (without false or negative positives) classify and track people and/or vehicles as well as the precise location and interactions between those objects. They want to have access to accurate data generated by best-of-class solutions irrespective of the sensor modality. And, they need to be able to easily deploy such solutions, at the lowest capex and opex, with the knowledge that they can be integrated with preferred VMSs and PSIMs, be highly reliable, have low install and maintenance overheads and be well supported. With these needs in mind, camera and computer vision technology providers, solutions providers, and systems integrators are forging ahead and have created exemplary ecosystems with established partnerships helping to accelerate adoption. At the heart of this are AI and applications of Convolutional neural networks (CNN), an architecture often used in computer vision deep learning algorithms, which are accomplishing tasks that were extremely difficult with traditional software. But what about 3D sensing technologies and perception? The security, safety, and access market have an additional crucial need: they must mitigate risk and make investments that deliver for the long-term. This means that if a systems integrator invests in a 3D sensing data perception platform today, it will support their choice of sensors, perception strategies, applications, and use cases over time without having to constantly reinvest in alternative computer hardware and perception software each time they adopt new technology or systems. This begs the question - if the security industry knows what it needs, why is it yet to fully embrace 3D sensing modalities? Perception strategy Intelligent perception strategies are yet to evolve which sees designers lock everything down at the design phaseWell, one problem facing security, safety, and access solutions providers, systems integrators, and end-users when deploying first-generation 3D sensing-based solutions is the current approach. Today, intelligent perception strategies have yet to evolve beyond the status quo which sees designers lock everything down at the design phase, including the choice of the sensor(s), off-the-shelf computer hardware, and any vendor-specific or 3rd party perception software algorithms and deep learning or artificial intelligence. This approach not only builds in constraints for future use-cases and developments, it hampers the level of perception developed by the machine. Indeed, the data used to develop or train the perception algorithms for security, access, and safety use cases at design time is typically captured for a narrow and specific set of scenarios or contexts and are subsequently developed or trained in the lab. Technology gaps As those in this industry know too well, siloed solutions and technology gaps typically block the creation of productive ecosystems and partnerships while lack of commercial whole products can delay market adoption of new innovation. Perception systems architectures today do not support the real-time adaptation of software and computing engines in the field. They remain the same as those selected during the design phase and are fixed for the entire development and the deployment stage. Crucially, this means that the system cannot deal with the unknowns of contextually varying real-time situations where contexts are changing (e.g being able to reflex to security situations they haven’t been trained for) and where the autonomous system’s perception strategies need to dynamically adjust accordingly. Ultimately, traditional strategies have non-scalable and non-adaptable competing computing architectures that were not designed to process the next generation of algorithms, deep learning, and artificial intelligence required for 3D sensor mixed workloads. What this means for industries seeking to develop or deploy perception systems, like security, access, and safety, is that the available computing architectures are generic and designed for either graphic rendering or data processing. Solutions providers, therefore, have little choice but to promote these architectures heavily into the market. Consequently, the resulting computing techniques are defined by the computing providers and not by the software developers working on behalf of the customer deploying the security solution. Context…. we don’t know what we don’t know Perception platform must have the ability to adjust to changes in context, thereby improving the performance post-deployment To be useful and useable in the security context and others, a perception platform must have the ability to adjust to changes in context, can self-optimize, and crucially, can self-learn, thereby improving the performance post-deployment. The combinations of potential contextual changes in a real-life environment, such as an airport or military base, are innumerable, non-deterministic, real-time, often analog, and unpredictable. The moment sensors, edge computing hardware, and perception software are deployed in the field, myriad variables such as weather, terrain as well as sensor mounting location and orientation all represent a context shift where the perception systems’ solution is no longer optimal. For example, it might be that a particular sensor system is deployed in an outdoor scenario with heavy foliage. Because the algorithm development or training was completed in the lab, the moving foliage, bushes, or low trees and branches are classified as humans or some other false-positive result. Typically, heavy software customization and onsite support then ensue, requiring on-site support by solutions vendors where each and every sensor configuration needs to be hand-cranked to deliver something that is acceptable to the end customer. A new approach for effective perception strategies Cron AI is building senseEDGE, which represents a significant evolution in the development of sensing to information strategy. It is a 3D sensing perception and computer vision platform built from the ground up to address and remove the traditional deployment and performance bottlenecks we’ve just described. senseEDGE is aware of the user application reaction plan indication to trigger an alarm or turning on a CCTV camera The entire edge platform is built around a real-time scalable and adaptable computing architecture that’s flexible enough for algorithms and software to scale and adapt to different workloads and contexts. What’s more, it has real-time contextual awareness, which means that the entire edge platform is, at any time, aware of the external context, the sensor and sensor architecture, and the requirements of the user application. Furthermore, when it produces the object output data, it also aware of the user application reaction plan indication, which could be triggering an alarm or turning on a CCTV camera when a specific action is detected. This approach turns traditional perception strategies on their head: it is software-defined programmable perception and computing architecture, not hardware-defined. It is free from the constraints imposed by traditional CPU or GPU compute dictated by hardware architecture providers and not limited to the perception built defined during design time. And, being fully configurable, it can be moved from one solution to another, providing computation for different modalities of sensors designed for different use cases or environments, and lower risk of adoption and migration for those developing the security solution. Future perception requirements senseEDGE is also able to scale to future perception requirements, such as algorithms and workloads produced by future sensors as well as computational techniques and neural networks that have yet to be invented. Meanwhile, latency versus throughput is totally software-defined and not limited by providers of computing architecture. Finally, contextually aware, it is fully connected to the real world where the reflexes adapt to even the subtlest changes in context, which makes all the difference in time and accuracy in critical security situations. This is how CronAI sees the future of perception. It means that security and safety innovators can now access and invest with low risk in a useable and scalable perception solution that can truly take advantage of current and future 3D sensor modalities.

Maximizing Effectiveness Of Thermal Cameras For Temperature Screening

Thermal cameras can be used for rapid and safe initial temperature screening of staff, visitors and customers. Used the right way, the cameras can help prevent unnecessary spread of viruses like the novel coronavirus. During the global pandemic, use of thermal cameras has increased, but they have not always been used correctly, and therefore, not effectively. Hikvision’s temperature screening thermal products are currently assisting users in initial temperature screening across the global market. During 2020, demand increased in most markets, and the company highly recommends that Hikvision’s thermographic cameras be used in accordance with local laws and regulations. Limitations of the technology include throughput and the impact of ambient conditions. Detect viruses and fever Hikvision releases a video that illustrates how skin temperature measurements are normalized within minutes Thermal cameras cannot detect viruses and fever and should only be used as a first line of screening before using secondary measures to confirm, says Stefan Li, Thermal Product Director at Hikvision. “We also believe it is important for businesses and authorities to use [thermal cameras] alongside a full program of additional health and safety procedures, which includes handwashing, regular disinfection of surfaces, wearing protective clothing such as masks, and social distancing.” Hikvision has released a video that illustrates how skin temperature measurements are normalized within minutes after someone emerges from the cold. Mr. Li says the video demonstrates the accuracy of forehead measurement under difficult circumstances when people come inside from a cold outdoor environment. Temperature screening facilities “There have been some claims that measuring the forehead temperature is not as accurate as measuring the inner canthus, and we believe this video demonstrates the accuracy of forehead measurement very well,” he says. “We also illustrate how the skin temperature will experience a process of recovery (warming up), no matter if it is measured by a thermal camera or a thermometer.” Mr. Li adds that people should wait five minutes in such circumstances before starting a temperature measurement. “We hope that stakeholders who are involved in the design of temperature screening facilities and associated health and safety procedures will recognize how important it is to consider the skin temperature recovery time, and that forehead measurement can provide accurate test results,” says Mr. Li. Thermal imaging manufacturers The algorithm is based on a large number of test results to obtain a value that tends to be dynamically balanced The temperature measurement principle of thermal imaging is to detect the heat radiation emitted by the human body. The detected heat value often does not reflect the true internal body temperature of an individual. Furthermore, the temperature varies among different parts of the human, such as the forehead, ears, underarms, etc. A temperature compensation algorithm can be used to adjust the measured skin temperature to align with the internal body temperature. The algorithm is based on a large number of test results to obtain a value that tends to be dynamically balanced. At present, thermal imaging manufacturers in the market, and even forehead thermometer manufacturers, have developed their own algorithms to map the skin temperature measured by the camera to the internal body temperature, so as to compensate the skin temperature to the internal body temperature. Thermal cameras This is also why Hikvision recommends that the "actual body temperature" should be checked with a secondary device for confirmation. The calibration work for a thermal camera is completed in the production process at the factory, including calibration of reference values and detection point and so on. At the same time, the equipment parameters should be adjusted before on-site use to ensure accurate temperature reads. Hikvision does not deny the accuracy of temperature measurement at the inner canthus but prefers forehead temperature measurement and algorithms based on actual use scenarios, says Mr. Li. A large amount of test data and practical results indicates that the forehead is a correct and easy-to-use temperature measurement area, says the company. There are advantages and disadvantages of choosing different facial areas for temperature measurement. Default compensation temperature Two main approaches direct the measurement area and how compensation algorithms are applied: Forehead area + default forehead compensation algorithm value Upper half face (forehead + canthus) + default inner canthus compensation algorithm value. Both methods deploy compensation algorithms, but the default compensation temperature of the inner canthus will be less than the default compensation temperature of the forehead, generally speaking. The reason is that the temperature of the inner canthus of most people is higher than their forehead, so the temperature compensation is relatively low (i.e., closer to the actual temperature inside the body.) Upper face area Hikvision found that selecting the upper face area plus the default compensation value for the inner canthus resulted in situations when the calculated temperature is lower than the actual temperature. For the Hikvision solution, the forehead is a relatively obvious and easy-to-capture area on an entire face Mr. Li explains: “The reason is that when the camera cannot capture the position of the inner canthus (for example, when a person is walking, or the face is not facing the camera), the camera will automatically capture the temperature of the forehead. Then the result that appears is the sum of the forehead temperature plus the default compensation temperature of the inner canthus, which is lower than the actual temperature of the person being measured. Therefore, errors are prone to occur.” Thermal imaging products But for the Hikvision solution, the forehead is a relatively obvious and easy-to-capture area on an entire face. Also, the default forehead compensation temperature is based on rigorous testing and can also correctly mimic the actual temperature of the person being measured, says Mr. Li. After many test comparisons, considering that the results of forehead temperature measurement are relatively more stable, and in order to avoid the false results from inner canthus temperature measurement, Hikvision chose the forehead temperature measurement approach. “We look forward to bringing thermal imaging products from a niche market where there is a relatively high-end industry application to a mass market and serving more users,” says Mr. Li. Facial recognition terminals Additional application parameters can maximize effectiveness of thermal cameras for measuring body temperature: Positioning and height - All cameras must be mounted appropriately to avoid loss of accuracy and performance. The installation height of each camera must be adjusted according to camera resolution and focal length, and stable installation is needed to avoid errors caused by shaking. Ensuring a ‘one-direction path’ - The detection area must ensure that cameras capture the full faces of all those passing by or stopping, and obstacles should be avoided in the field of view, such as glass doors that block the camera. Adequate start-up and usage - A waiting time of more than 90 minutes is required for preheating, after the initial start-up. Before conducting a thermal scan, people should be given three to five minutes to allow their body temperature to stabilize. When Hikvision MinMoe facial recognition terminals are used, people must stand at a fixed distance, pass one by one, make a short stop, and face the camera directly. Hikvision cameras support efficient group screening, but one-by-one screening is suggested for more accurate results, says Mr. Li. Unstable environmental condition An unstable environmental condition may affect the accuracy of thermal camera systems Environmental factors can impact the accuracy of thermal cameras, and the idea of using a black body is to provide the camera with a reference point that has a stable temperature. The black body is heated to a specific temperature and helps the thermal camera to know how much error is caused by environmental factors in the room, and how the camera should calibrate itself in real time to improve its accuracy. A black body can help increase the temperature measurement accuracy, and the most common improvement is from ±0.5 degrees to ±0.3 degrees. However, it also increases the cost of the installation. In some markets, customers may require black bodies in order to comply with regulatory accuracy requirements. An unstable environmental condition may affect the accuracy of thermal camera systems for measuring temperature. Medical temperature measurement Therefore, Hikvision suggests that the ambient conditions should be met for installation and use. First of all, users should avoid installing devices in hot or changeable environments. All cameras require indoor environments with calm air, consistent temperature and no direct sunlight. Installation should also be avoided in semi-open locations that may be prone to changes in ambient conditions, such as doorways, and there should be enough stable, visible light. All devices should be installed to avoid backlighting, high temperature targets, and reflections in the field of view as far as possible. “We often see the misconception that thermal cameras can replace medical temperature measurement equipment, which is not the case,” says Mr. Li. Rapid preliminary screening “Temperature screening thermographic cameras are designed for the detection of skin-surface temperatures, and the measurement should be conducted to achieve rapid preliminary screening in public areas. It is really important that actual core body temperatures are measured subsequently with clinical measurement devices.”