Facial recognition has a long history dating back to the 1800s. To track down criminals, such as infamous bandits Jesse Woodson James and Billy the Kid, law enforcement would place “Wanted Alive or Dead” posters advertising bounties and soliciting public cooperation to help locate and even apprehend the alleged criminals.

In addition to the bounty, these posters would include a photo and brief description of the crime, which would then be circulated to law enforcement agencies around the country and displayed in every US Post Office to speed up apprehension.

 

Facial Recognition

Today, technology such as social media, television and other more specialized communication networks play a more influential role in the recognition process. Advancements in artificial intelligence and biometric technology, including the development of Machine Learning capabilities, have led to increased accuracy, accessibility and the widespread use of computerized facial recognition. The significance of this means that facial recognition can occur on an even larger scale and in more challenging environments.

Advancements in artificial intelligence and biometric technology have led to the widespread use of computerised facial recognition

This article will explore key milestones and technological advances that have resulted in the modern incarnation of facial recognition, before discussing the capabilities of cutting-edge “one-to-many” technology which is increasingly being used by counter-terror defense, police and security forces around the world.

Technology Inception And Developments

The 1960s marked the start of computerized facial recognition, when Woodrow Wilson (Woody) Bledsoe developed a way to classify faces using gridlines. Bledsoe’s facial recognition still required a large amount of human involvement because a person had to extract the co-ordinates of the face’s features from a photograph and enter this information into a computer. The technology was able to match 40 faces an hour (each face took approximately 90 seconds to be matched) which was considered very impressive at the time.

The technology was able to match 40 faces an hour, which was considered very impressive at the time

By the end of the 1960s, facial recognition had seen further development at the Stanford Research Institute where the technology proved to outperform humans in terms of accuracy of recognition (humans are notoriously bad at recognizing people they don’t know). By the end of the century, the leading player in the field was a solution that came out of the University of Bochum in Germany – and the accuracy of this technology was such that it was even sold on to bank and airport customers.

From this stage on, the facial recognition market began to blossom, with error rates of automatic facial recognition systems decreasing by a factor of 272 from 1993 to 2010 according to US Government-sponsored evaluations.

The safety and security market requires near instant feedback on who a person matched against a watchlist is

The aim for facial technology is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware

Modern Usage Of Facial Recognition

Fast-forward to the modern day and facial recognition has become a familiar technology when using applications such as the iPhone X’s Face ID capability or MasterCard Identity Check, passport e-gates at airports and other security and access control points. These solutions implement a consensual form of identity verification, as the user has a vested interest in being identified.

This is a “one-to-one” facial recognition event, one person in front of the camera being compared to one identity either on a passport or the app. In these scenarios, the hardware is specifically developed for the application at hand, therefore technically much easier to accomplish.

Facial recognition can now be used in a variety of governmental and commercial environments The safety and security world brings a much more complex problem to solve – how to pick out a face in a moving and changing environment and compare it to several faces of interest. “One-to-many” facial recognition is a much harder problem to solve.

 

It’s even more challenging when the aim is to achieve successful and accurate recognition on commonly available hardware like live CCTV feeds and standard computing hardware. And unlike in the 1960’s where identifying a face every 90 seconds was acceptable; the safety and security market requires near instant feedback on who a person matched against a watchlist is.

Security And Safety Applications

The idea behind all facial recognition technologies is broadly the same: you start with an image of a person’s face (ideally a high quality one, although machine learning means that to a point we can now even use video without reducing accuracy). A fully front facing image is best, think a passport photo, but machine learning and new software has made this more flexible.

An algorithm converts this image into a numeric template, which cannot be converted back to an image and so represents a secure one way system. Every numeric template is different, even if it started out as an image of the same person, although templates from the same person are more similar than templates from different people.

The accuracy of facial recognition continues to increase alongside deployments in more challenging and complex environments What happens next sounds simple although the technology is extremely complex: templates of people’s faces are taken in real time and compared to those in the database. The technology identifies individuals by matching the numeric template of their face with all the templates saved in a database in a matter of seconds or milliseconds. To put this into perspective, imagine you are at the turnstiles of a busy train station looking for a person on the run.

Today’s facial recognition technology would be able to identify that person should they pass in view of a CCTV camera, as well as notify the police of any additional persons of interest, whether they are a known terrorist or missing vulnerable person on an entirely separate watch list.

Because of technical progression, facial recognition can now be used in a variety of governmental and commercial environments, from identifying barred hooligans attempting entry at a football stadium or helping self-excluded gamblers at casino to overcome addiction.

Real-Time Assessments

The latest evolution of facial recognition pits the technology against an even more challenging application – directly matching individuals from body worn cameras for real time recognition for police officers on the beat. This capability equips first responders with the ability to detect a person from a photo and verify their identity with assurance.

The broader implication for this means that every interaction, such as stop and search or arrest, can be supported by real-time facial recognition which will see cases of mistaken identity driven down on the streets. First responders can now for the first time be deployed and furnished with the ability to identify wider groups of people of interest with a degree of accuracy that previously relied only on the fallible human memory.

As the accuracy of the technology continues to increase alongside deployments in more challenging and complex environments, its ability to support government initiatives and law enforcement means the debate about the lawful and appropriate use of facial recognition must be addressed. Facial recognition should not be everywhere looking for everyone, but when used properly it has the potential to improve public safety and we should make the most of its potential.

Download PDF version Download PDF version

Author profile

Fernande van Schelle Product Manager, Digital Barriers

Fernande van Schelle is a product manager for facial recognition technology with Digital Barriers. With over a decade of experience, she leads the product management and development activities to deliver facial recognition capability using deep neural networks. Fernande holds a master’s degree in Business Administration from INSEAD and a master’s in history of International Relations from LSE.

In case you missed it

5 Intelligence-Based Capabilities For Boosting Executive Protection
5 Intelligence-Based Capabilities For Boosting Executive Protection

Corporate global security operations centers (GSOCs) bear the responsibility for protecting C-suite executives from physical and reputational harm amid social upheaval, extreme weather events, and escalating cyber threats. That mission, daunting as it is, becomes more difficult when GSOCs lack the data necessary for conducting a comprehensive threat assessment. Threat intelligence Unfortunately, many centers never move past basic threat intelligence which focuses on scenario-driven queries such as “what” could go wrong and “where” the event could occur. They fail to capture information that could help them identify threat actors, the critical “who” and “who else” is involved, and uncover their plans before a plot unfolds. Addressing threat environment GSOC's approaches should span all aspects of executive protection, from pre-mission planning to real-time operations GSOC executive protection teams must expand their narrow-focus intelligence capabilities to deal with an increasingly complicated threat environment. They need to obtain a wide-angle view that strategically places individual events into a broader context, while also uncovering the bits of data that, when correlated, can make all the difference in diffusing a potential crisis. In addition, these intelligence approaches should span all aspects of executive protection, from pre-mission planning to real-time, location-based operations. Here then are five intelligence-based capabilities all executive protection teams need today: 1) Advanced scouting intelligence GSOCs often rely on threat intelligence companies to alert them to dangerous situations or compromises that could impact the executives under their protection. While threat intelligence is important, it stops well short of providing the complete picture. Executive protection teams need more than superficial information gleaned from news feeds or international weather reports. The centralized GSOC, serving as the risk HQ, must take a proactive approach to mine intelligence data. To that end, GSOCs should provide advanced scouting intelligence. Data-driven insights If an executive aims to attend an event in another country, the center should provide data-driven insights for pre-mission planning. That means digging into local news, international news, the geopolitical structure, and weather conditions among other factors. GSOCs should also probe the deep and dark layers of the web in addition to the commonly used and well-traveled websites, forums, and online platforms. A thorough understanding of the pending event and its context lends executive protection teams an intelligence edge when it's time to deploy. 2) An understanding of threat actors' intelligence capabilities GSOCs should use "red teaming" to get inside the head of a threat actor and uncover an executive's vulnerabilities But advanced scouting doesn't end there. GSOCs should also use "red teaming" to get inside the head of a threat actor and uncover an executive's vulnerabilities. Red teaming determines what an adversary might be able to find out about executives, their family members, and entourages. What's their level of exposure? Publicly available photos posted online can prove particularly harmful. For example, a photo of an executive's private jet could reveal a tail number that a threat actor could track, via open-source intelligence, to determine where the plane is heading. Images of vehicle license plates can be similarly exploited. Ways to strengthen vulnerability assessment A red team may also find images of the executive's family members, tagged in online photos and readily identifiable. Online sources may also reveal where family members work and mention where their children attend school. This investigative intelligence should explore three or four degrees of separation, not just immediate relationships. The GSOC's red team can also check for data leaks, breaches, and evidence of doxing to strengthen the vulnerability assessment well before the executive travels to the event. 3) Deep, dark web intelligence Open-source intelligence often relies on information gathered from the surface web of everyday use. GSOCs, however, must also look for threats lurking beneath the familiar online world. The deep web, for example, houses myriad sites that aren't indexed and, therefore, can't be searched using standard web browsers. The dark web, a subset of the deep web, is even less accessible, requiring specialized software such as an anonymizing browser. Situational awareness The dark web, in particular, demands a GSOC investigator's attention. Threat actors use this web layer to communicate, collaborate and plan operations with a relatively high degree of secrecy. Failure to tap this information resource can dramatically reduce threat visibility and limit an executive protection team's situational awareness. GSOCs, however, can conduct deep and dark web investigations, provided they have the know-how and technology tools to do so. Exploiting the dark web Acquiring such skills requires time, commitment, and, potentially, new investigative policies Acquiring such skills requires time, commitment, and, potentially, new investigative policies. Existing guidelines, for example, may prohibit GSOC personnel from downloading an anonymizing browser. In addition, investigators could stumble upon an exploit in the dark web, which could compromise their computers and networks. A computer used to explore the dark web should be isolated from the GSOC's production network as a matter of policy. With the proper tools, investigative techniques, and procedural guardrails in place, a GSOC can tap a valuable intelligence source. 4) Deanonymisation of threat actors Typical threat intelligence offerings help a GSOC answer the "what" question as they prepare to protect executives on the move. That is, they provide information on a particular incident in the executive's vicinity. But it's critically important to address the "who" question as well. That task calls for savvy investigators augmented by artificial intelligence. GSOCs can use AI to craft custom searches spanning the surface, deep and dark webs. Casting such a wide net lets investigators pursue threat actors, who can move rapidly between commonplace public networks and more obscure platforms. Artificial intelligence A crackdown on surface web activities, for instance, will drive threat actors to a dark web hideout. An investigator confined to the surface web will soon lose the trail. A scouring of all the web's layers will generate loads of data, which GSOCs will need to parse. AI can also prove an asset here, accelerating the process of extracting actionable intelligence from, potentially, terabytes of data. Manual data analysis can take days, if not longer, bogging down investigations and delaying the flow of crucial intelligence to teams on the front lines of executive protection. Automated approaches and network monitoring Learning about networks becomes important when a security team needs to exfiltrate an executive Automated investigative approaches can quickly correlate the bits of data collected in a web search – threat actors' online handles, IP addresses, phone numbers, and photos, for example. String together enough informational breadcrumbs can help deanonymize threat actors. Once an identity is unmasked, platform analysis lets investigators uncover additional relationships and entire networks of threat actors. Learning about networks becomes important when a security team needs to exfiltrate an executive. A threat actor network could operate in several geographic locations, not just the area in which an incident occurred. It makes no sense to evacuate an executive from one dangerous location only to relocate him or her to another trouble spot.  5) Real-time situational intelligence The traditional intelligence approach still in use is reliant on historical data to help organizations make executive protection decisions. GSOCs have to move past this and incorporate real-time situational intelligence as a core component of their executive protection programs. On the ground security teams will then be able to ensure that the strategies and protocols used to safeguard executives are based on current, actionable and trustworthy intelligence gleaned from multiple verified sources. Perimeter protection Real-time situational intelligence requires analyzing data that is relevant to a specific event within a geographical area and time. This strategic approach will require a virtual perimeter to be placed around the geographic area in which an executive will be active – a conference venue and the surrounding city, for instance. GSOC analysts can then evaluate all online activity emanating within the predetermined boundary. This examination can take place before and during an event and the resulting intelligence can be fed through to ‘on-the-ground’ protection teams in real-time while an incident unfolds, or before an event. Unstructured data analysis GSOCs must be able to analyze unstructured data in various formats including online platforms and remote sensors Additionally, analysis of publicly available information can yield important locational clues such as photos of well-known landmarks, images containing signage, or other objects with text that can offer additional geographic markers. But to take advantage of such intelligence sources, GSOCs must be able to analyze unstructured data in various formats including online platforms and remote sensors. Interpreted correctly this will provide real-time data that can be used as a basis for actionable and trustworthy intelligence. This ability will also bolster the on-the-ground situational capabilities of the security team. Alert notifications Forward-deployed executive protection teams can also contribute their real-time situational intelligence as events unfold on the ground. Teams receiving alerts from the central GSOC can vet that information based on what they are seeing locally. They can disseminate that intelligence back to the GSOC, which in turn can incorporate those insights into their data and adjust an alert level accordingly. Direct communication between the GSOC and on-the-ground executive protection teams is crucial. This flow of intelligence completes a virtuous circle:  The pre-mission planning intelligence supports the executive protection team and the team's real-time intelligence informs and refines the GSOC's guidance. Restricting data limits visibility The current threat landscape, more varied and complex than ever, requires a comprehensive response. GSOCs and their executive protection teams need to take in as much data from as many intelligence sources as possible. Restricting data gathering to a few conventional websites severely limits visibility. The multitude of online platforms, and the staggering amounts of structured and unstructured data they generate, make technology an important asset for getting the intelligence job done within a reasonable timeframe. Toward a bigger intelligence picture It's not all about automation, however. Highly skilled personnel, including red teams, are a major part of the intelligence operation. Policies and best practices for intelligence gathering round out the list of requirements. GSOCs must pursue the big picture when it comes to executive protection. A narrow view of data sources, techniques, and technologies won't suffice. It's no time for tunnel vision when lives and reputations are at stake.

What Are The Security Challenges Of The Transportation Industry?
What Are The Security Challenges Of The Transportation Industry?

The transportation industry is unique because it plays a role in both the minutiae of everyday life and the overall economy's well-being. Providing uninterrupted and efficient transportation services depends heavily on the safety and security of transportation entities. Due to the vast amount of people frequenting transit stations and the importance of the items on ships and ferries, the transportation sector faces multiple challenges that evolve as the industry advances. We asked this week’s Expert Panel Roundtable: What are the security challenges of the transportation industry?

How Security Pros Can Leverage Clients’ Trust Into Bigger Jobs And New Revenue Streams
How Security Pros Can Leverage Clients’ Trust Into Bigger Jobs And New Revenue Streams

Business owners know the security of their properties is one of their most important daily concerns, and that leads many of them to work with professional security integrators who can design and deliver robust systems that provide protection and peace-of-mind through modern technologies and services. That’s why security integrators, by their very nature, have an invaluable tool that can be leveraged into bigger jobs: clients’ trust. As highly-trusted partners with extensive experience installing low voltage systems, security integration firms have immediate opportunity to expand into the related field of commercial audio and video installations that often accompany traditional security projects. Effective security solutions This is a significant market, considering audio and video systems are commonly used in all manner of business environments, from offices, retail shops and restaurants to warehouses, factories and public venues. Racks, power protection, wall and ceiling mounts are all common components in both security and AV systems Any time a trusted partner is asked if they can provide a new or additional service, the decision ultimately comes down to their comfort level. As it turns out, many of the skills required to design and implement effective security solutions overlap with the skills needed for custom audio and video installations. For starters, much of the technology infrastructure is the same: racks, power protection, wall and ceiling mounts, networking, digital displays and speakers are all common components in both security and AV systems. Single-Focus integrators Many security professionals already possess a solid base of low voltage and networking knowledge, so adding commercial AV services is a natural progression to grow their business. As such, manufacturers and distributors often provide training programs, courses and materials to help single-focus integrators become proficient in AV installations and maintenance, which can include ongoing service contracts similar to security monitoring contracts. While security firms are comfortable tackling even the most complex and critical security needs, there is a tendency for AV to be perceived as more involved and more complicated, and therefore not worth the effort or training. In the past, security and AV systems may indeed have been separate and disparate, but today they share many similar characteristics and often are designed to work together seamlessly, adding even more power and capability to the security functions. Standard security components Firms that offer digital signage, for instance, are currently in high demand Clients who want surveillance, for instance, may want a room on-site to view live and recorded footage that may involve multiple screens and an audio system. These standard security components form the core of many AV solutions, with the main differences being content delivery, programming and control for multi-zone audio and video. There are, of course, many different AV needs and possibilities, from conference room displays and video conferencing cameras to warehouse public address systems and corporate lobby welcome screens. As product knowledge grows and employees become more confident installing things like DVLED video walls, 70-volt audio systems and MoIP (media over IP) content distribution, the opportunities to bid on larger, more profitable jobs can skyrocket. Firms that offer digital signage, for instance, are currently in high demand as the product category experiences aggressive expansion in virtually every sector of the economy. Rapid issue resolution One of the key aspects of the latest AV systems, control systems and other related components is their ability to be managed and controlled remotely by both the owner and the integrator. This has drastically lowered the cost to serve for integrators and presents the chance to enroll clients in highly profitable service contracts that provide rapid issue resolution and user training. Many problems can be resolved remotely without needing to send a truck to the site Since most leading solutions offer web-based or mobile app-based access to systems from anywhere, integrators can create streams of reliable monthly revenue that require far less time and effort than they did a decade ago. Many problems can be resolved remotely without needing to send a truck to the site, providing clients with enhanced responsiveness and peace-of-mind while streamlining operations so the integrator can tackle even more jobs. Profitable client relationships Security firms looking to grow revenue, capabilities or industry reputation must remain vigilant in terms of offering the latest solutions and utilizing industry best practices. As one of the most trusted partners in any project, security firms have a unique opportunity to expand their service offerings and develop more consolidated and profitable client relationships. In light of this opportunity and the high chance that security clients may ask about AV systems, it’s vital for security firms to consider training and education on the latest AV products and solutions so they are prepared to say yes when the moment comes. With programs available through industry associations, distributors and manufacturers, the barrier to entry is lower than ever for owners to expand their integration capabilities and grow their business. Clint Choate is a Senior Director at Snap One. Based in Charlotte, North Carolina. Snap One is a manufacturer and exclusive source of A/V, surveillance, control, networking and remote management products for professional integrators.