The evolution of facial recognition: from bodycams to video surveillance

Download PDF version

Facial recognition has a long history dating back to the 1800s. To track down criminals, such as infamous bandits Jesse Woodson James and Billy the Kid, law enforcement would place “Wanted Alive or Dead” posters advertising bounties and soliciting public cooperation to help locate and even apprehend the alleged criminals. In addition to the bounty, these posters would include a photo and brief description of the crime, which would then be circulated to law enforcement agencies around the country and displayed in every US Post Office to speed up apprehension.

Facial Recognition

Advancements in artificial intelligence and biometric technology have led to the widespread use of computerised facial recognitionToday, technology such as social media, television and other more specialized communication networks play a more influential role in the recognition process. Advancements in artificial intelligence and biometric technology, including the development of Machine Learning capabilities, have led to increased accuracy, accessibility and the widespread use of computerized facial recognition. The significance of this means that facial recognition can occur on an even larger scale and in more challenging environments.

This article will explore key milestones and technological advances that have resulted in the modern incarnation of facial recognition, before discussing the capabilities of cutting-edge “one-to-many” technology which is increasingly being used by counter-terror defense, police and security forces around the world.

Technology Inception And Developments

The technology was able to match 40 faces an hour, which was considered very impressive at the time

The 1960s marked the start of computerized facial recognition, when Woodrow Wilson (Woody) Bledsoe developed a way to classify faces using gridlines. Bledsoe’s facial recognition still required a large amount of human involvement because a person had to extract the co-ordinates of the face’s features from a photograph and enter this information into a computer. The technology was able to match 40 faces an hour (each face took approximately 90 seconds to be matched) which was considered very impressive at the time.

By the end of the 1960s, facial recognition had seen further development at the Stanford Research Institute where the technology proved to outperform humans in terms of accuracy of recognition (humans are notoriously bad at recognizing people they don’t know). By the end of the century, the leading player in the field was a solution that came out of the University of Bochum in Germany – and the accuracy of this technology was such that it was even sold on to bank and airport customers.

From this stage on, the facial recognition market began to blossom, with error rates of automatic facial recognition systems decreasing by a factor of 272 from 1993 to 2010 according to US Government-sponsored evaluations.

The safety and security market requires near instant feedback on who a person matched against a watchlist is

The aim for facial technology is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware

Modern Usage Of Facial Recognition

Fast-forward to the modern day and facial recognition has become a familiar technology when using applications such as the iPhone X’s Face ID capability or MasterCard Identity Check, passport e-gates at airports and other security and access control points. These solutions implement a consensual form of identity verification, as the user has a vested interest in being identified.

This is a “one-to-one” facial recognition event, one person in front of the camera being compared to one identity either on a passport or the app. In these scenarios, the hardware is specifically developed for the application at hand, therefore technically much easier to accomplish.

Facial recognition can now be used in a variety of governmental and commercial environments The safety and security world brings a much more complex problem to solve – how to pick out a face in a moving and changing environment and compare it to several faces of interest. “One-to-many” facial recognition is a much harder problem to solve.

It’s even more challenging when the aim is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware. And unlike in the 1960’s where identifying a face every 90 seconds was acceptable; the safety and security market requires near instant feedback on who a person matched against a watchlist is.

Security And Safety Applications

The idea behind all facial recognition technologies is broadly the same: you start with an image of a person’s face (ideally a high quality one, although machine learning means that to a point we can now even use video without reducing accuracy). A fully front facing image is best, think a passport photo, but machine learning and new software has made this more flexible.

An algorithm converts this image into a numeric template, which cannot be converted back to an image and so represents a secure one way system. Every numeric template is different, even if it started out as an image of the same person, although templates from the same person are more similar than templates from different people.

The accuracy of facial recognition continues to increase alongside deployments in more challenging and complex environments What happens next sounds simple although the technology is extremely complex: templates of people’s faces are taken in real time and compared to those in the database. The technology identifies individuals by matching the numeric template of their face with all the templates saved in a database in a matter of seconds or milliseconds. To put this into perspective, imagine you are at the turnstiles of a busy train station looking for a person on the run.

Today’s facial recognition technology would be able to identify that person should they pass in view of a CCTV camera, as well as notify the police of any additional persons of interest, whether they are a known terrorist or missing vulnerable person on an entirely separate watch list.

Because of technical progression, facial recognition can now be used in a variety of governmental and commercial environments, from identifying barred hooligans attempting entry at a football stadium or helping self-excluded gamblers at casino to overcome addiction.

Real-Time Assessments

The latest evolution of facial recognition pits the technology against an even more challenging application – directly matching individuals from body worn cameras for real time recognition for police officers on the beat. This capability equips first responders with the ability to detect a person from a photo and verify their identity with assurance.

The broader implication for this means that every interaction, such as stop and search or arrest, can be supported by real-time facial recognition which will see cases of mistaken identity driven down on the streets. First responders can now for the first time be deployed and furnished with the ability to identify wider groups of people of interest with a degree of accuracy that previously relied only on the fallible human memory.

As the accuracy of the technology continues to increase alongside deployments in more challenging and complex environments, its ability to support government initiatives and law enforcement means the debate about the lawful and appropriate use of facial recognition must be addressed. Facial recognition should not be everywhere looking for everyone, but when used properly it has the potential to improve public safety and we should make the most of its potential.

Download PDF version

Author profile

Fernande van Schelle Product Manager, Digital Barriers

Related companies
Digital Barriers

Related links
Face Recognition Software Access control software
Articles by Fernande van Schelle

View all news from
Digital Barriers

In case you missed it

What Do You Need To Know About Thermal Imaging Cameras?

As businesses, schools, hospitals and sporting venues look to safely reopen in a COVID-19 world, thermal imaging systems will play a critical role in helping to detect and distinguish skin temperature variations in people. Thermal surveillance, a mainstay of traditional physical security and outdoor perimeter detection, is now being deployed to quickly scan employees, contractors and visitors as part of a first line of defense to detect COVID-19 symptoms. In the coming weeks and months, the security industry will look to implement thermal camera solutions for customers, yet many questions remain as to the differences between different system types and how to properly install thermal imaging cameras. In this Q&A, Jason Ouellette, Head of Technology Business Development for Johnson Controls, answers several of these questions. Q: What are some of the different thermal imaging solutions available in the market to detect an elevated temperature in a person? For the general market, there are three types of these thermographic screenings. There is the handheld device, which is typically lower cost, very portable, and very easy to use. Typically, this is a point and shoot type of device, but it requires you to be three feet or less from the person that you're screening, which, in today's world, means the user needs to wear protective personal equipment. For the general market, there are three types of these thermographic screenings The second type of solution would best be described as a thermal camera and kiosk. The advantage of this system over a handheld device is this can be self-service. An individual would go up to and engage with the kiosk on their own. But many of these kiosk type solutions have some integration capability, so they can provide some type of output, for either turnstiles, or physical access control, but not video management systems (VMS). Some of the downside of this type of system is that it’s less accurate than a thermographic solution because it does not have a blackbody temperature calibration device and the readings are influenced by the surrounding ambient temperature, called thermal drift. So instead of being able to achieve a ±0.3ºC accuracy rating, this system probably provides closer to ±0.5ºC at best. Some of these devices may be classed as a clinical thermometer with a higher degree of one time accuracy, but do not offer the speed and endurance of the thermographic solution for adjunctive use. And then there are thermal imaging camera systems with a blackbody temperature calibration device. These types of systems include a dual sensor camera, that has a visual sensor and a thermal sensor built right into the camera, along with a separate blackbody device. This provides the highest degree of ongoing accuracy, because of the blackbody and its ability to provide continuous calibration. These systems can provide much more flexibility and can offer integrations with multiple VMS platforms and access control devices. Q: When installing a thermal imaging camera system what is the most important element to consider? Camera placement is critical to ensure the system works as expected, however the placement of the blackbody device which verifies the correct calibration is in place is equally as important. If the customer wants to follow FDA medical device recommendations for camera placement, both the height of the camera and the blackbody as well as the distance between these devices should comply with the product installation instructions. This takes into account the device focal range and calibration parameters in addressing the distance from the person undergoing the scan. Also, integrators should minimize camera detection angles to ensure optimal accuracy and install cameras parallel with the face as much as possible, and again in compliance with installation instructions. Integrators should minimize camera detection angles to ensure optimal accuracy The blackbody should be placed outside of the area where people could block the device and located more towards the edges of the field-of-view of the camera. You need to keep in mind the minimum resolution for effective thermographic readings which is 320 by 240 pixels as defined by the standards. To achieve this, you would need to follow medical electrical equipment performance standards driven by IEC 80601-2-59:2017 for human temperature scanning and FDA guidelines. Within that measurement, the face needs to fill 240 x 180 pixels of the thermal sensor resolution, which is close to or just over 50 percent of the sensor’s viewing area typically, meaning a single person scanned at a time in compliance with the standards for accuracy. Along with height and distance placement considerations, the actual placement in terms of the location of the system is key. For example, an expansive glass entryway may impact accuracy due to sunlight exposure. Installations should be focused on ensuring that they are away from airflow, heating and cooling sources, located approximately 16 feet from entry ways and in as consistent of an ambient temperature as possible between 50°F and 95°F. Q: Once a thermal imaging camera system is installed, how do you monitor the device? There are several choices for system monitoring, depending on whether the solution is used as standalone or integrated with other technologies, such as intrusion detection, access control or video systems. For standalone systems, the ability to receive system alerts is typically configured through the camera’s webpage interface, and the cameras include abilities such as the live web page, LED display for alerting, audio alerts and physical relay outputs. When done right, these features will all follow cybersecurity best practices which is important for any network solution today, including changing default passwords and establishing authentication methods. The ability to receive system alerts is typically configured through the camera’s webpage interface These types of thermal cameras can also integrate with turnstile systems, VMS platforms and access control systems. This is typically done through the integration of a relay output, activated by a triggered temperature anomaly event on a thermal imaging camera which can then be used for activities such as locking a turnstile, or through access control and video systems to send an email or provide an automated contagion report for contact tracing. These capabilities and integrations extend the monitoring capability above that of the standalone solution. The camera can be configured to monitor a specific range of low and high alerts. Users can determine the actions that should be taken when that alert exceeds the preset low or high threshold. These actions include things like a bright and easy-to-see LED can provide visual notification through pulsing and flashing lights as an example. Q: What about system maintenance? Does a thermal imaging camera require regular service in order to operate accurately? First it’s important to make sure the system is calibrated. This can be done after the unit stabilises for at least 30 minutes to establish the initial reference temperature source known as the blackbody. Calibrations conducted before this warm up and stability time period can throw off accuracy. Also, as part of your system maintenance schedule you will want to perform a calibration check of the blackbody device every 12 months, along with following recommendations of the FDA and IEC. If you install the solution and don’t perform maintenance and the blackbody calibration certificate expires, over time there’s a risk that the device will experience drift and a less accurate reading will result. There’s a risk that the device will experience drift and a less accurate reading will result Q: What final pieces of advice do you have for either an integrator who plans to install a thermal imaging camera system or an end user who plans to invest in this solution? Before you buy a thermal imaging camera check to see if the manufacturer ships the camera with a calibration certificate. Also, become familiar with FDA’s guidance released in April 2020, Enforcement Policy for Telethermographic Systems During the Coronavirus Disease 2019 (COVID-19) Public Health Emergency. This document places thermal/fever products for adjunctive use under the category of a Class I medical devices and subject to its regulatory control. Driven by these regulations and categorisation, users need to understand specifically what is required to meet the required level of accuracy for successful detection. While thermal imaging camera systems are more complex than traditional surveillance cameras, they can prove to be a valuable resource when set up, configured and maintained properly.

Functionality Beyond Security: The Advent of Open Platform Cameras

The coronavirus (COVID-19) pandemic marks the biggest global disruption since World War II. While the ‘new normal’ after the crisis is still taking shape, consumers are apprehensive about the future. According to a recent survey, 60% of shoppers are afraid of going grocery shopping, with 73% making fewer trips to physical stores. Returning to the workplace is also causing unease, as 66% of employees report feeling uncomfortable about returning to work after COVID-19. Businesses and employers are doing their best to alleviate these fears and create safe environments in and around their buildings. This also comes at tremendous costs for new safety measures and technologies – including updates to sanitation protocols and interior architecture – that protect against COVID-19. Costs in the billions that most businesses will face alone, without support from insurance and amidst larger, macroeconomic challenges. Saving costs and increasing security But what if building operators, retail shop owners, and other stakeholders could save costs by leveraging new functionality from their existing security infrastructure? More specifically, expanding the use of current-generation security cameras – equipped with AI-driven image analysis capabilities – beyond the realm of security and into meeting new health regulations. This is exactly where video analytics algorithms come into play. And in the next step, a new evolutionary approach towards open security camera platforms promises new opportunities. Security cameras have evolved from mere image capturing devices into complex data sensors Over the past decade, security cameras have evolved from mere image capturing devices into complex data sensors. They provide valuable data that can be analyzed and used in beneficial ways that are becoming the norm. Since 2016, Bosch has offered built-in Video Analytics as standard on all its IP cameras. On one hand, this enables automated detection of security threats more reliably than human operators. And on the other hand, video analytics collect rich metadata to help businesses improve safety, increase efficiency, reduce costs, and create new value beyond security. Expanding Camera Functionality Beyond Security Today, we have ‘smart’ security cameras with built-in video analytics to automatically warn operators of intruders, suspicious objects and dangerous behaviors. The rich metadata from several cameras on the same network can also be consolidated by making use of an intelligent software solution. It offers so-called pre-defined widgets to provide business intelligence by measuring area fill levels, counting building occupancy and detecting the formation of crowds. In combination with live video stream data, these insights enable heightened situational awareness to security operators. What’s more, operators are free to set their own parameters – like maximum number of occupants in a space and ‘off limit’ areas – to suit their needs. These user-centric widgets also come in handy in dealing with the coronavirus pandemic. Specific widgets can trigger an alarm, public announcement or trigger a 'traffic light' when the maximum number of people in a space is exceeded. Building operators can also use available intelligence such as foot traffic ‘heat maps’ to identify problem areas that tend to become congested and place hand sanitizer stations at heavily frequented hotspots. At the same time, the option to perform remote maintenance on these systems limits the exposure of technicians in the field during the pandemic. Again, the underlying camera hardware and software already exist. Cameras will be able to ‘learn’ future functionality to curb the spread of the coronavirus Looking ahead, cameras with video analytic and neural network-based analytic capabilities will be able to ‘learn’ future functionality to curb the spread of the coronavirus. For instance, cameras could monitor distances between individuals and trigger voice announcements when social distancing guidelines are violated. Facial recognition software can be trained to monitor personal protective equipment (PPE) compliance and sound alerts for persons entering buildings without masks. The technical requirements are already in place. The task at hand is to deliver these new functionalities to cameras at scale, which is where open camera platforms hold the key. Why Open Camera Operating Systems? When it comes to innovating future camera applications that extend beyond security, no hardware manufacturer should go at it alone. Instead, an open platform approach provides the environment for third-party developers to innovate and market new functions. In essence, an open platform principle allows customers and users to change the behavior of devices by adding software afterwards. This software can either be found in an app store or can be self-developed. For a precedent, we can look at the mobile phone industry. This is where software ecosystems like Android and Apple’s iOS have become the norm. They have also become major marketplaces, with the Apple App Store generating $519 billion in billings on 2019, as users use their phones for far more than just making phone calls. In the same way, intelligent cameras will be used far beyond classic video applications in the future. To get there, adopting an open platform principle is essential for a genuine transformation on an industry level. But establishing an open platform principle in the fragmented video security industry demands a cooperative approach. In 2018 Bosch started a fully owned start-up company, Security & Safety Things, and became one of five founding members of OSSA (Open Security & Safety Alliance). With more than 40 members, the Alliance has collectively created the first Technology Stack for “open” video security devices. This includes the OSSA Application Interface Specification and Compliant Device Definition Specification. An open camera platform for innovating future functionality Based on OSSA’s common APIs, collective approach on data security and core system requirements for video security cameras, the first camera manufacturers were able to build video security cameras that adopt an open platform principle. Further fueling innovation, OSSA focused on driving the creation of one centralized marketplace to unite demand and supply in the market. Camera devices that are built in accordance with OSSA’s Technology Stack, so-called “Driven by OSSA” devices, can benefit from this marketplace which consists of three pillars: a development environment, an application store, and a device management portal. Security & Safety Things has advanced OSSA’s open camera platform concept, built this marketplace for the security and safety industry and has developed the open OS that powers the first “Driven by OSSA” devices. Making it quick and simple to customize security solutions by installing and executing multiple apps This year, Bosch, as one of the first camera manufacturers, introduces the new INTEOX generation of open platform cameras. To innovate a future beyond security functionality, INTEOX combines built-in Intelligent Video Analytics from Bosch, an open Operating System (OS), and the ability to securely add software apps as needed. Thanks to the fully open principle, system integrators are free to add apps available in the application store, making it quick and simple to customize security solutions by installing and executing multiple apps on the INTEOX platform. In turn, app developers can now focus on leveraging the intelligence and valuable data collected by analytics-equipped cameras for their own software developments to introduce new exciting possibilities of applying cameras. These possibilities are needed as smart buildings and IoT-connected technology platforms continue to evolve. And they will provide new answers to dealing with COVID-19. The aforementioned detection of face masks and PPE via facial detection algorithms is just one of manifold scenarios in which new apps could provide valuable functionality. Contact tracing is another field where a combination of access control and video analytics with rich metadata can make all the difference. Overall, open camera platforms open a future where new, complex functionality that can save lives, ensure business continuity and open new business opportunities will arrive via something as simple as a software update. And this is just the beginning.

Recognizing The Importance Of Security Officers To Promote Safety

The general public doesn’t give much thought to the important role of security officers in creating and promoting safer environments. The low-profile work of security officers is vital to protecting people, places and property. During the pandemic, newer aspects to that role have emerged. Security personnel have been called on to perform diverse tasks such as managing queues at the supermarket, safeguarding testing centers and hospitals, ensuring food deliveries, and supporting police patrols. The British Security Industry Association (BSIA) and two other organizations in the United Kingdom are joining forces to raise awareness of the work of security officers and to recognize the vital importance of the duties they perform. BSIA, a trade association, includes members who are responsible for 70% of privately provided UK security products and services, including security guarding, consultancy services, and distribution and installation of electronic and physical security equipment. BSIA, the Security Institute and the Security Commonwealth Joining BSIA in the awareness campaign are the Security Institute, a professional security membership body; and the Security Commonwealth, which is comprised of 40 organizations from across the security landscape with common objectives to build professionalism, raise standards and share best practices. “The recognition of security officers as key workers is the start of a re-appraisal of what service they provide to the community in keeping the public safe and secure,” says Mike Reddington, BSIA Chief Executive. “As we exit lockdown and have to navigate public spaces again, [security officers] will have a crucial role in supporting public confidence. We are working closely with the Police and all other public bodies to find the best way to achieve this.” Security officers acknowledged as key workers The campaign will showcase security professionals as a respected, valued, professional service provider and a key worker that is acknowledged and embedded in daily lives. The British Security Industry Association (BSIA) and two other organizations in the United Kingdom are joining forces to raise awareness of the work of security officers “Great effort has been invested in the professional standards and capabilities of frontline [security] officers, and they have proven their worth during the coronavirus crisis in the UK,” says Rick Mounfield, Chief Executive, the Security Institute. “They, along with the wider security sector, deserve to be recognized, respected and appreciated for the safety and security they provide across the United Kingdom.” “[We are working to] build professionalism, raise standards and share best practices, and I hope this campaign can make more people recognize the changes we have all made and continue to make,” says Guy Matthias, Chairman of the Security Commonwealth (SyCom). The industry will be reaching out to companies, professionals, and organizations in the sector to participate in the campaign. The hope is that, over the coming weeks as lockdown is eased, the industry can play its part to ensure that the country emerges with confidence to start to recover and build for the future. Private security more important than ever The campaign will showcase security professionals as a respected, valued, professional service provider Across the pond in the United States, law enforcement professionals are facing a crisis of confidence during a time of civil unrest as protestors call to “defund the police” and to otherwise undermine and/or recast law enforcement’s role in preserving the peace and ensuring public safety. If an upshot is that public policing is starved of resources, the role of private security to supplement their mission is likely to increase. In short, the role of private security is more important than ever on both sides of the Atlantic. Public recognition of that role is welcome, obviously. In any case, the importance of their role protecting people, places and property has never been greater.