Artificial Intelligence (AI) Trends In Physical Security Systems

Download PDF version

If you’ve been paying attention over the last twelve months, you will have noticed that deep learning techniques and artificial intelligence (AI) are making waves in the physical security market, with manufacturers eagerly adopting these buzzwords at the industry's biggest trade shows. With all the hype, security professionals are curious to know what these terms really mean, and how these technologies can boost real-world security system performance.

The growing number of applications of deep learning technology and AI in physical security is a clear indication that these are more than a passing fad. This review of some of our most comprehensive articles on these topics shows that AI is an all-pervasive trend that the physical security industry will do well to embrace quickly. Here, we examine the opportunities that artificial intelligence presents for smart security applications, and look back at how some of the leading security companies are adapting to respond to rapidly-changing expectations:

What Is Deep Learning Technology?

Machine Learning involves collecting large amounts of data related to a problem, training a model using this data and employing this model to process new data. Recently, there have been huge advances in a branch of Machine Learning called Deep Learning. This describes a family of algorithms based on neural networks. These algorithms are able to learn efficiently from example, and subsequently apply this learning to new data. Here, Zvika Ashani explains how deep learning technology can boost video surveillance systems.

Relationship Between Deep Learning And Artificial Intelligence

There have been huge advances in a branch of Machine Learning called Deep Learning

With deep learning, you can show a computer many different images and it will "learn" to distinguish the differences. This is the "training" phase. After the neural network learns about the data, it can then use "inference" to interpret new data based on what it has learned. For example, if it has seen enough cats before, the system will know when a new image is a cat. In effect, the system “learns” by looking at lots of data to achieve artificial intelligence (AI). Larry Anderson explores how new computer hardware - the Graphic Processing Unit (GPU) – is making artificial intelligence accessible to the security industry.

Improving Surveillance Efficiency And Accuracy With AI

The Graphic Processing Unit (GPU) – is making artificial intelligence accessible to the security industry

Larry Anderson explains how the latest technologies from Neurala and Motorola will enable the addition of AI to existing products, changing an existing solution from a passive sensor to a device that is “active in its thinking.” The technology is already being added to existing Motorola body-worn-cameras to enable police officers to more efficiently search for objects or persons of interest. In surveillance applications, AI could eliminate the need for humans to do repetitive or boring work, such as look at hours of video footage.

Intelligent Security Systems Overcome Smart City Surveillance Challenges

In surveillance applications, AI could eliminate the need for humans to do repetitive or boring work, such as look at hours of video footage

AI technology is expected to answer the pressing industry questions of how to use Big Data effectively and make a return on the investment in expensive storage, while maintaining (or even lowering) human capital costs. However, until recently, these expectations have been limited by factors such as a limited ability to learn, and high ongoing costs. Zvika Ashani examines how these challenges are being met and overcome, making artificial intelligence the standard in Smart City surveillance deployments.

Combining AI And Robotics To Enhance Security Operations

Artificial intelligence is becoming the standard in Smart City surveillance deployments

With the abilities afforded by AI, robots can navigate any designated area autonomously to keep an eye out for suspicious behavior or alert first responders to those who may need aid. This also means that fewer law enforcement and/or security personnel will have be pulled from surrounding areas. While drones still require a human operator to chart their flight paths, the evolution of artificial intelligence (AI) is increasing the capabilities of these machines to work autonomously, says Steve Reinharz.

Future Of Artificial Intelligence In The Security Industry

While drones still require a human operator to chart their flight paths, the evolution of artificial intelligence (AI) is increasing the capabilities of these machines to work autonomously

Contributors to SourceSecurity.com have been eager to embrace artificial intelligence and its ability to make video analytics more accurate and effective. Manufacturers predicted that deep learning technology could provide unprecedented insight into human behaviour, allowing video systems to more accurately monitor and predict crime. They also noted how cloud-based systems hold an advantage for deep learning video analytics. All in all, manufacturers are hoping that AI will provide scalable solutions across a range of vertical markets.

Download PDF version

People mentioned in this article

Larry Anderson

Zvika Ashani Agent Vi (Agent Video Intelligence)

Steve Reinharz Robotic Assistance Devices, LLC.

In case you missed it

What Do You Need To Know About Thermal Imaging Cameras?

As businesses, schools, hospitals and sporting venues look to safely reopen in a COVID-19 world, thermal imaging systems will play a critical role in helping to detect and distinguish skin temperature variations in people. Thermal surveillance, a mainstay of traditional physical security and outdoor perimeter detection, is now being deployed to quickly scan employees, contractors and visitors as part of a first line of defense to detect COVID-19 symptoms. In the coming weeks and months, the security industry will look to implement thermal camera solutions for customers, yet many questions remain as to the differences between different system types and how to properly install thermal imaging cameras. In this Q&A, Jason Ouellette, Head of Technology Business Development for Johnson Controls, answers several of these questions. Q: What are some of the different thermal imaging solutions available in the market to detect an elevated temperature in a person? For the general market, there are three types of these thermographic screenings. There is the handheld device, which is typically lower cost, very portable, and very easy to use. Typically, this is a point and shoot type of device, but it requires you to be three feet or less from the person that you're screening, which, in today's world, means the user needs to wear protective personal equipment. For the general market, there are three types of these thermographic screenings The second type of solution would best be described as a thermal camera and kiosk. The advantage of this system over a handheld device is this can be self-service. An individual would go up to and engage with the kiosk on their own. But many of these kiosk type solutions have some integration capability, so they can provide some type of output, for either turnstiles, or physical access control, but not video management systems (VMS). Some of the downside of this type of system is that it’s less accurate than a thermographic solution because it does not have a blackbody temperature calibration device and the readings are influenced by the surrounding ambient temperature, called thermal drift. So instead of being able to achieve a ±0.3ºC accuracy rating, this system probably provides closer to ±0.5ºC at best. Some of these devices may be classed as a clinical thermometer with a higher degree of one time accuracy, but do not offer the speed and endurance of the thermographic solution for adjunctive use. And then there are thermal imaging camera systems with a blackbody temperature calibration device. These types of systems include a dual sensor camera, that has a visual sensor and a thermal sensor built right into the camera, along with a separate blackbody device. This provides the highest degree of ongoing accuracy, because of the blackbody and its ability to provide continuous calibration. These systems can provide much more flexibility and can offer integrations with multiple VMS platforms and access control devices. Q: When installing a thermal imaging camera system what is the most important element to consider? Camera placement is critical to ensure the system works as expected, however the placement of the blackbody device which verifies the correct calibration is in place is equally as important. If the customer wants to follow FDA medical device recommendations for camera placement, both the height of the camera and the blackbody as well as the distance between these devices should comply with the product installation instructions. This takes into account the device focal range and calibration parameters in addressing the distance from the person undergoing the scan. Also, integrators should minimize camera detection angles to ensure optimal accuracy and install cameras parallel with the face as much as possible, and again in compliance with installation instructions. Integrators should minimize camera detection angles to ensure optimal accuracy The blackbody should be placed outside of the area where people could block the device and located more towards the edges of the field-of-view of the camera. You need to keep in mind the minimum resolution for effective thermographic readings which is 320 by 240 pixels as defined by the standards. To achieve this, you would need to follow medical electrical equipment performance standards driven by IEC 80601-2-59:2017 for human temperature scanning and FDA guidelines. Within that measurement, the face needs to fill 240 x 180 pixels of the thermal sensor resolution, which is close to or just over 50 percent of the sensor’s viewing area typically, meaning a single person scanned at a time in compliance with the standards for accuracy. Along with height and distance placement considerations, the actual placement in terms of the location of the system is key. For example, an expansive glass entryway may impact accuracy due to sunlight exposure. Installations should be focused on ensuring that they are away from airflow, heating and cooling sources, located approximately 16 feet from entry ways and in as consistent of an ambient temperature as possible between 50°F and 95°F. Q: Once a thermal imaging camera system is installed, how do you monitor the device? There are several choices for system monitoring, depending on whether the solution is used as standalone or integrated with other technologies, such as intrusion detection, access control or video systems. For standalone systems, the ability to receive system alerts is typically configured through the camera’s webpage interface, and the cameras include abilities such as the live web page, LED display for alerting, audio alerts and physical relay outputs. When done right, these features will all follow cybersecurity best practices which is important for any network solution today, including changing default passwords and establishing authentication methods. The ability to receive system alerts is typically configured through the camera’s webpage interface These types of thermal cameras can also integrate with turnstile systems, VMS platforms and access control systems. This is typically done through the integration of a relay output, activated by a triggered temperature anomaly event on a thermal imaging camera which can then be used for activities such as locking a turnstile, or through access control and video systems to send an email or provide an automated contagion report for contact tracing. These capabilities and integrations extend the monitoring capability above that of the standalone solution. The camera can be configured to monitor a specific range of low and high alerts. Users can determine the actions that should be taken when that alert exceeds the preset low or high threshold. These actions include things like a bright and easy-to-see LED can provide visual notification through pulsing and flashing lights as an example. Q: What about system maintenance? Does a thermal imaging camera require regular service in order to operate accurately? First it’s important to make sure the system is calibrated. This can be done after the unit stabilises for at least 30 minutes to establish the initial reference temperature source known as the blackbody. Calibrations conducted before this warm up and stability time period can throw off accuracy. Also, as part of your system maintenance schedule you will want to perform a calibration check of the blackbody device every 12 months, along with following recommendations of the FDA and IEC. If you install the solution and don’t perform maintenance and the blackbody calibration certificate expires, over time there’s a risk that the device will experience drift and a less accurate reading will result. There’s a risk that the device will experience drift and a less accurate reading will result Q: What final pieces of advice do you have for either an integrator who plans to install a thermal imaging camera system or an end user who plans to invest in this solution? Before you buy a thermal imaging camera check to see if the manufacturer ships the camera with a calibration certificate. Also, become familiar with FDA’s guidance released in April 2020, Enforcement Policy for Telethermographic Systems During the Coronavirus Disease 2019 (COVID-19) Public Health Emergency. This document places thermal/fever products for adjunctive use under the category of a Class I medical devices and subject to its regulatory control. Driven by these regulations and categorisation, users need to understand specifically what is required to meet the required level of accuracy for successful detection. While thermal imaging camera systems are more complex than traditional surveillance cameras, they can prove to be a valuable resource when set up, configured and maintained properly.

Functionality Beyond Security: The Advent of Open Platform Cameras

The coronavirus (COVID-19) pandemic marks the biggest global disruption since World War II. While the ‘new normal’ after the crisis is still taking shape, consumers are apprehensive about the future. According to a recent survey, 60% of shoppers are afraid of going grocery shopping, with 73% making fewer trips to physical stores. Returning to the workplace is also causing unease, as 66% of employees report feeling uncomfortable about returning to work after COVID-19. Businesses and employers are doing their best to alleviate these fears and create safe environments in and around their buildings. This also comes at tremendous costs for new safety measures and technologies – including updates to sanitation protocols and interior architecture – that protect against COVID-19. Costs in the billions that most businesses will face alone, without support from insurance and amidst larger, macroeconomic challenges. Saving costs and increasing security But what if building operators, retail shop owners, and other stakeholders could save costs by leveraging new functionality from their existing security infrastructure? More specifically, expanding the use of current-generation security cameras – equipped with AI-driven image analysis capabilities – beyond the realm of security and into meeting new health regulations. This is exactly where video analytics algorithms come into play. And in the next step, a new evolutionary approach towards open security camera platforms promises new opportunities. Security cameras have evolved from mere image capturing devices into complex data sensors Over the past decade, security cameras have evolved from mere image capturing devices into complex data sensors. They provide valuable data that can be analyzed and used in beneficial ways that are becoming the norm. Since 2016, Bosch has offered built-in Video Analytics as standard on all its IP cameras. On one hand, this enables automated detection of security threats more reliably than human operators. And on the other hand, video analytics collect rich metadata to help businesses improve safety, increase efficiency, reduce costs, and create new value beyond security. Expanding Camera Functionality Beyond Security Today, we have ‘smart’ security cameras with built-in video analytics to automatically warn operators of intruders, suspicious objects and dangerous behaviors. The rich metadata from several cameras on the same network can also be consolidated by making use of an intelligent software solution. It offers so-called pre-defined widgets to provide business intelligence by measuring area fill levels, counting building occupancy and detecting the formation of crowds. In combination with live video stream data, these insights enable heightened situational awareness to security operators. What’s more, operators are free to set their own parameters – like maximum number of occupants in a space and ‘off limit’ areas – to suit their needs. These user-centric widgets also come in handy in dealing with the coronavirus pandemic. Specific widgets can trigger an alarm, public announcement or trigger a 'traffic light' when the maximum number of people in a space is exceeded. Building operators can also use available intelligence such as foot traffic ‘heat maps’ to identify problem areas that tend to become congested and place hand sanitizer stations at heavily frequented hotspots. At the same time, the option to perform remote maintenance on these systems limits the exposure of technicians in the field during the pandemic. Again, the underlying camera hardware and software already exist. Cameras will be able to ‘learn’ future functionality to curb the spread of the coronavirus Looking ahead, cameras with video analytic and neural network-based analytic capabilities will be able to ‘learn’ future functionality to curb the spread of the coronavirus. For instance, cameras could monitor distances between individuals and trigger voice announcements when social distancing guidelines are violated. Facial recognition software can be trained to monitor personal protective equipment (PPE) compliance and sound alerts for persons entering buildings without masks. The technical requirements are already in place. The task at hand is to deliver these new functionalities to cameras at scale, which is where open camera platforms hold the key. Why Open Camera Operating Systems? When it comes to innovating future camera applications that extend beyond security, no hardware manufacturer should go at it alone. Instead, an open platform approach provides the environment for third-party developers to innovate and market new functions. In essence, an open platform principle allows customers and users to change the behavior of devices by adding software afterwards. This software can either be found in an app store or can be self-developed. For a precedent, we can look at the mobile phone industry. This is where software ecosystems like Android and Apple’s iOS have become the norm. They have also become major marketplaces, with the Apple App Store generating $519 billion in billings on 2019, as users use their phones for far more than just making phone calls. In the same way, intelligent cameras will be used far beyond classic video applications in the future. To get there, adopting an open platform principle is essential for a genuine transformation on an industry level. But establishing an open platform principle in the fragmented video security industry demands a cooperative approach. In 2018 Bosch started a fully owned start-up company, Security & Safety Things, and became one of five founding members of OSSA (Open Security & Safety Alliance). With more than 40 members, the Alliance has collectively created the first Technology Stack for “open” video security devices. This includes the OSSA Application Interface Specification and Compliant Device Definition Specification. An open camera platform for innovating future functionality Based on OSSA’s common APIs, collective approach on data security and core system requirements for video security cameras, the first camera manufacturers were able to build video security cameras that adopt an open platform principle. Further fueling innovation, OSSA focused on driving the creation of one centralized marketplace to unite demand and supply in the market. Camera devices that are built in accordance with OSSA’s Technology Stack, so-called “Driven by OSSA” devices, can benefit from this marketplace which consists of three pillars: a development environment, an application store, and a device management portal. Security & Safety Things has advanced OSSA’s open camera platform concept, built this marketplace for the security and safety industry and has developed the open OS that powers the first “Driven by OSSA” devices. Making it quick and simple to customize security solutions by installing and executing multiple apps This year, Bosch, as one of the first camera manufacturers, introduces the new INTEOX generation of open platform cameras. To innovate a future beyond security functionality, INTEOX combines built-in Intelligent Video Analytics from Bosch, an open Operating System (OS), and the ability to securely add software apps as needed. Thanks to the fully open principle, system integrators are free to add apps available in the application store, making it quick and simple to customize security solutions by installing and executing multiple apps on the INTEOX platform. In turn, app developers can now focus on leveraging the intelligence and valuable data collected by analytics-equipped cameras for their own software developments to introduce new exciting possibilities of applying cameras. These possibilities are needed as smart buildings and IoT-connected technology platforms continue to evolve. And they will provide new answers to dealing with COVID-19. The aforementioned detection of face masks and PPE via facial detection algorithms is just one of manifold scenarios in which new apps could provide valuable functionality. Contact tracing is another field where a combination of access control and video analytics with rich metadata can make all the difference. Overall, open camera platforms open a future where new, complex functionality that can save lives, ensure business continuity and open new business opportunities will arrive via something as simple as a software update. And this is just the beginning.

Recognizing The Importance Of Security Officers To Promote Safety

The general public doesn’t give much thought to the important role of security officers in creating and promoting safer environments. The low-profile work of security officers is vital to protecting people, places and property. During the pandemic, newer aspects to that role have emerged. Security personnel have been called on to perform diverse tasks such as managing queues at the supermarket, safeguarding testing centers and hospitals, ensuring food deliveries, and supporting police patrols. The British Security Industry Association (BSIA) and two other organizations in the United Kingdom are joining forces to raise awareness of the work of security officers and to recognize the vital importance of the duties they perform. BSIA, a trade association, includes members who are responsible for 70% of privately provided UK security products and services, including security guarding, consultancy services, and distribution and installation of electronic and physical security equipment. BSIA, the Security Institute and the Security Commonwealth Joining BSIA in the awareness campaign are the Security Institute, a professional security membership body; and the Security Commonwealth, which is comprised of 40 organizations from across the security landscape with common objectives to build professionalism, raise standards and share best practices. “The recognition of security officers as key workers is the start of a re-appraisal of what service they provide to the community in keeping the public safe and secure,” says Mike Reddington, BSIA Chief Executive. “As we exit lockdown and have to navigate public spaces again, [security officers] will have a crucial role in supporting public confidence. We are working closely with the Police and all other public bodies to find the best way to achieve this.” Security officers acknowledged as key workers The campaign will showcase security professionals as a respected, valued, professional service provider and a key worker that is acknowledged and embedded in daily lives. The British Security Industry Association (BSIA) and two other organizations in the United Kingdom are joining forces to raise awareness of the work of security officers “Great effort has been invested in the professional standards and capabilities of frontline [security] officers, and they have proven their worth during the coronavirus crisis in the UK,” says Rick Mounfield, Chief Executive, the Security Institute. “They, along with the wider security sector, deserve to be recognized, respected and appreciated for the safety and security they provide across the United Kingdom.” “[We are working to] build professionalism, raise standards and share best practices, and I hope this campaign can make more people recognize the changes we have all made and continue to make,” says Guy Matthias, Chairman of the Security Commonwealth (SyCom). The industry will be reaching out to companies, professionals, and organizations in the sector to participate in the campaign. The hope is that, over the coming weeks as lockdown is eased, the industry can play its part to ensure that the country emerges with confidence to start to recover and build for the future. Private security more important than ever The campaign will showcase security professionals as a respected, valued, professional service provider Across the pond in the United States, law enforcement professionals are facing a crisis of confidence during a time of civil unrest as protestors call to “defund the police” and to otherwise undermine and/or recast law enforcement’s role in preserving the peace and ensuring public safety. If an upshot is that public policing is starved of resources, the role of private security to supplement their mission is likely to increase. In short, the role of private security is more important than ever on both sides of the Atlantic. Public recognition of that role is welcome, obviously. In any case, the importance of their role protecting people, places and property has never been greater.