Over the years, video analytics has gained an unfavourable reputation for over-promising and under-delivering in terms of performance. One of the biggest complaints regarding video analytics has been its inability to correctly identify objects in situations which appear trivial to the human observer. In many cases, this has resulted in a tendency to generate substantial numbers of false alarms, while not detecting actual events accurately. This, together with a propensity for complex set-up procedures and much need for manual fine-tuning, has prevented video analytics from becoming a mainstream application deployed on large numbers of cameras.

Machine Learning And Video Analytics

Machine Learning is a well-established field of research that has existed for decades, which is already present in many products and applications. Machine Learning is based on collecting large amounts of data specific to a particular problem, training a model using this data and then employing this model to process new data. With regard to video analytics, one of the most critical problems impacting accuracy is object classification. Fundamental to improving performance is the capability to teach the algorithm to distinguish between people, animals, different types of vehicles and sources of noise at an extremely high level of accuracy.

Deep Learning Algorithms

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units

Until recently there have been minimal applications of Machine Learning used in video analytics products, largely due to high complexity and high resource usage, which made such products too costly for mainstream deployment. However, the last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning. Deep Learning is a name used to describe a family of algorithms based on the concept of neural networks. Very loosely speaking, these algorithms try to emulate the functionality of the brain’s neurons, enabling them to learn efficiently from example, and subsequently apply this learning to new data.

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units (GPUs). GPUs can efficiently train and run Deep Learning algorithms, and have allowed the scientific community to accelerate their research and application, bringing them to the point where they exceed the performance of most traditional Machine Learning algorithms across several categories.

Solving Object Classification With Deep Learning

This means that Deep Learning can now be used to solve the most crucial problem facing video analytics – object classification – by collecting many thousands of images from hundreds of surveillance cameras, which must first be manually labelled and classified by a human, into a range of categories that include: person, car, bus, truck, bird, vegetation, dog and many more. To achieve the required accuracy rates, such a vast database must be collected and identified from actual surveillance footage.

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications is the ability to continuously update the models as more data is collected
Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics

A Deep Learning algorithm trained on images collected from YouTube, Google Search and elsewhere on the Internet will completely fail in analysing images from surveillance cameras, due to the difference in viewing angles, resolution and image quality. Once enough images are collected, a Deep Learning classifier algorithm can be trained and deployed as part of a video analytics solution, enabling it to practically eliminate most of the existing causes for false alarms.

Due to GPU requirements in order for the algorithms to run efficiently, video analytics solutions using Deep Learning will initially need to run on a server. A few solutions of this nature are already available and are showing a dramatic leap in performance in comparison to traditional video analytics, with a drastic reduction in false alarm rates and a significant increase in detection accuracy. Concurrently, these new solutions do not require manual tweaking by the user and are essentially plug-and-play, making mass deployment a realistic premise.

Surveillance Applications Of Deep Learning

Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics, but they are by no means the only ones. In the not-too-distant future, we will see Deep Learning enabling as yet not possible video analytics applications, such as identifying objects carried by people, such as a gun, handbag, or a knife, or being able to quickly find people and vehicles with similar appearances across multiple cameras and more.

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers to running inside cameras

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers, to running inside cameras, as powerful, low-cost hardware capable of running Deep Learning becomes more available and a basic function of newer surveillance camera models. This will push the acceptance of video analytics even further, eventually making it a fundamental element of every surveillance camera deployed.

Increasing Accuracy With Updates

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications, is the ability to continuously update the models as more data is collected so that the models increase in accuracy. This will give an advantage to cloud-based video analytics services, since they can collect vast amounts of data from cameras connected to the service, train new models in the cloud based on this data and then push these new models to cameras at the edge. This continuous improvement cycle will be instrumental in helping video analytics fulfil the promise of improving peoples’ safety and security, by giving surveillance cameras human-level accuracy and a comprehensive understanding of the environment.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

In case you missed it

Are Privacy Concerns Stifling Innovation in Security?
Are Privacy Concerns Stifling Innovation in Security?

Facial recognition is the latest technology to be targeted because of concerns about privacy. If such concerns cloud the public perception, they can be harmful to technology markets. Whether the concerns are genuine or based on misinformation is often beside the point; the practical damage has already been done. But beyond market demand, what is the impact of privacy concerns on technology innovation? We asked this week’s Expert Panel Roundtable: Are privacy concerns stifling innovation in security and related markets? 

Building Security: How Audio Tells the Whole Story
Building Security: How Audio Tells the Whole Story

Every building starts with the entrance. A solid enterprise risk mitigation and security strategy include protecting that entrance. Often, risk mitigation strategies protecting the entrance have included high-resolution video surveillance cameras, video management systems, and access control solutions. But that strategy and set of security solutions only tells part of the story. Imagine a security guard who is protecting a facility after hours, when an individual approaches the entrance and seeks to gain access. The security guard can pull up the video surveillance feed and see the individual and his movements, which appear to be suspicious. But he also needs to hear him in order to decide the next decisions and actions. Does he escalate the situation, calling for backup and for first responders’ response, or does he allow the individual access to the building because he works there and is authorized to enter?   Meet high-definition voice What the security guard needs is to be able to hear and to communicate with that individual. All enterprise security systems need three primary components in order to successfully protect the entrance and to mitigate risk – access control, video surveillance, and the ability to hear and communicate. Each component plays an integral role in supporting a unified security system, and without all three, the security system is not complete. Access control can be thought of as the brains of a security system by holding data and permissions. It serves as the arms and hands of the system; it can either keep someone out or invite them in. IP video allows a security team to remotely position a set of eyes anywhere an IP camera can be placed on a network. With a video management system, security teams can see what is happening and decide how to respond. However, with remote viewing, the event may be over by the time security physically responds.  Audio adds interactivity That three-component enterprise security system – comprising IP video, access control, and high-definition voice working together mitigates risks and provides value. It also means that security is interactive. Security teams talk and listen to the person that’s seen on a video surveillance system, no matter where the location or how remote. If the person is lost or simply needs assistance, security personnel can talk to them and provide direction and reassurance. Even more, in an emergency, an interactive solution becomes a critical life-saving tool, as it provides data that can be shared between security, police, emergency services, and more. Audio can also detect voices, noises, breaking glass, or other sounds that are not within direct view of a video camera. An interactive security system creates an informed response, by providing real-time situation awareness management. Post-event, it supports forensics and investigations to mitigate future security incidents.  Audio and COVID-19 We are living in extraordinary times. As businesses begin to reopen and stay open, they are looking for any tools that can help them overcome the enormous challenges they face. In buildings and facilities, the COVID-19 pandemic has created a new security perimeter, one that demands contactless access with entry and exit, and that has also created a new duty of care for security professionals. Now more than ever is the need to interact and communicate with individuals moving in and out of doors and spaces without physical intervention. Intelligent communications, integrated with contactless access control, can help a business to comply with pandemic safety guidelines and ultimately, reopen for business and stay open.  COVID-19 has also increased the need for clean-room isolation and quarantine spaces, sometimes in areas not originally intended for that use, where risk of infection is high, and equipment must be easily disinfected between patients. Here, purpose-built cleanroom intercoms, providing clear touchless communications despite the noisy environment, have emerged as critical tools for enabling patient care while reducing the need to enter the contaminated space. For example, voice communication can enable hospital staff to verify identity and to communicate with patients without entering the isolated and infectious environment, which can save on personal protective equipment (PPE) and reduce the amount of exposure to the virus. In non-emergency healthcare facilities, such as medical centers, voice can effectively relay information to building occupants and visitors for screening purposes. Visitors can be seen and heard. For example, a patient who seeks access to a medical center for an appointment can hear important instructions from a nurse via the intercom solution. Seeing the person that you talk to is one thing but hearing them conveys a much better sense of closeness, making it possible to maintain a high level of security and customer service.  The whole story Today’s security systems should no longer simply involve video surveillance cameras generating feedback and images to a security guard. Instead, a new ecosystem for enterprise security and risk mitigation has emerged, and it’s one that involves video surveillance, access control, and high-definition voice. That ecosystem can ensure well-rounded and responsive information management and security platform, all communicating with each other and offering actionable insight into risks and potential physical breaches. Audio is the new value hub of the connected and intelligent school, campus, building, correctional facility, and more. Simply put, a silent security system cannot be an effective security system. In every situation, it is crucial for all security professionals to mitigate risk, no matter what they are protecting. This emphasizes the need to hear, be heard, and be understood in virtually any environment.  

Inclusion and Diversity in the Security Industry: ‘One Step at a Time’
Inclusion and Diversity in the Security Industry: ‘One Step at a Time’

Historically, concerns about inclusion and diversity have not been widely discussed in the security market. In the last couple of years, however, the Security Industry Association (SIA) and other groups have worked to raise awareness around issues of diversity and inclusion. Specifically, SIA’s Women in Security Forum has focused on the growing role of women in all aspects of security, and SIA’s RISE community has focused on “rising stars” in an industry previously dominated by Baby Boomers. The next generation of security leaders There is a business case to be made for diversity and inclusion, says a report by McKinsey & Company. According to the management consulting company, gender-diverse companies are 24% more likely to outperform less diverse companies, and ethnically diverse companies are 33% more likely to outperform their less diverse counterparts. Furthermore, the “next generation of security leaders” – employees under 30 – are particularly focused on diversity and inclusion. Diversity refers to the traits and characteristics that make people unique A panel discussion at ISC West’s Virtual Event highlighted aspects of inclusion and diversity, starting with a definition of each. Diversity refers to the traits and characteristics that make people unique. On the other hand, inclusion refers to the behavior and social norms that ensure people feel welcome. “We are all on a journey, and our journey takes different paths,” said Willem Ryan of AlertEnterprise, one of the SIA panelists. “There are opportunities to improve over time. We can all change and increase our ability to have a positive impact.” Industry responsibility The industry has a responsibility to the next generation of industry leaders to address issues of inclusion and diversity. Forbes magazine says that millennials are more engaged at work when they believe their company fosters an inclusive culture. So the question becomes: How do we unify and create opportunities to work with and champion tomorrow’s leaders? SIA is driving change in our industry to achieve that goal. More women are active in SIA than ever before. The SIA Women in Security Forum now has 520 members, said Maureen Carlo of BCD International, the SIA Women in Security Forum Chair and another panelist. Also, more women than ever are chairing SIA committees and serving on the SIA Board of Directors. More women than ever are chairing SIA committees Overcoming unconscious bias Former SIA Chairman Scott Shafer of SMS Advisors, another of the panelists, noted that SIA awarded the Chairman’s Award to the Women in Security Forum in 2019, and to the RISE community steering committee in 2020. “There are lots of ways we are seeing the elevation of women and ethnic groups in the security industry,” said Shafer. One topic of interest is the problem of “unconscious bias,” which can be overcome by looking at something through some else’s lens. Ryan suggested use of the acronym SELF –  Slow Down, Empathize, Learn, and Find commonalities. Ryan recalled the value of being mentored and having someone shepherd him around the industry. “Now I want to give back,” he said. “We need to look at the things we can change in ourselves, in our company, in our communities, and in our industry. Change comes from the bottom and the top.” Increasing representation “It takes all of us to increase representation everywhere,” said Kasia Hanson of Intel Corp., another panelist. “We have in common that we are all human beings. Let’s make sure the next generation all have opportunities.” Diverse companies can attract better talent Moving forward, the panelists urged the industry to get involved and create opportunities because inclusion drives diversity. Diverse companies can attract better talent and attain a competitive advantage. Awareness of unconscious bias, and working to eliminate it, is an important element of change. Despite the progress the security industry is making, change continues to be incremental. As Ruth Bader Ginsburg has said, “Real change, enduring change, happens one step at a time.”