Video surveillance is commonly associated with security. But in most cases, it's used to record incidents and assist in investigations after the fact rather than prevent undesirable events. Artificial intelligence–powered video analytics is a highly promising trend that fundamentally changes the way things work. Extracting manageable data from a video stream can help recognize risky situations early on, minimizing damage and, ideally, completely avoid emergencies. At the same time, AI significantly expands the areas of application of video surveillance beyond security systems.

AI significantly expands the areas of application of video surveillance beyond security systems

However, the hype around this new, trendy technology prevents the potential user from choosing quality solutions in a wide variety of products. This often leads to over-expectation, followed by a complete let-down. Can AI-powered video analytics really be the key to a technological breakthrough in video surveillance? We'll take a look at what the technology can do, what it can't, and where it can go from here.

Technological breakthrough or just another bubble?

It's often said that the video management software (VMS) market is becoming increasingly commoditized and widely available. A lot of products with similar features (or, at least, similar promises from the manufacturer) make it hard to choose. As a result, vendor names and reputations are turning into one of their primary selling points. Manufacturers have two choices available: get wrapped up in a price war and rely on cutting expenses, or offer a product that's truly innovative and revolutionary.

Manufacturers have two choices available: get wrapped up in a price war, or offer a product that's truly innovative and revolutionary

VMS developers who choose the second route are gravitating towards creating products that use artificial intelligence based on neural networks and deep learning. Emerging two or three years ago, the AI video analytics market is experiencing a boom in growth. This new tech wave has stirred the still, stagnant backwaters of the VMS world and gave small, ambitious developers something to be optimistic about. It seems they now have a chance to emerge as market leaders in the next few years.

However, the hype around this popular trend is raising reasonable concerns among experienced security industry professionals. These concerns come from clients looking for a solution to their problems, and from suppliers building a long-term development strategy. This largely resembles another tech bubble, like the one built up around pre-AI video analytics and burst when it became clear that the sensational promises around it were pure marketing hype (and rather unscrupulously so). However, there are a lot of factors that indicate that AI-powered video surveillance systems aren't another bubble.

The three factors

The first — and the main one — comes from systems already in place on customers' sites. They fulfill the same promises made during the previous bubble by hotheads in a rush to teach the computer to analyze events in real time using a classical algorithmic approach.

The second is the fact that this new technology has seen investment from not only software and cloud startups, but also established VMS developers. Even giants like Intel, which has presented a full line of neural network accelerator hardware and a set of software tools that streamlines working with them, specifically in the field of computer vision.

This new technology has seen investment from not only software and cloud startups, but also established VMS developers

The third factor lies in artificial intelligence's abilities. AI plays chess, drives cars, and works wonders in many other fields. Why shouldn't it be applied to video monitoring and analysis?

What AI can do

Just what can artificial intelligence do in video surveillance systems at this stage of development? It can't quite analyze a sequence of events and understand the "logic" of what's happening in the cameras' field of view. At least not yet. But it's probable that AI will learn to do this in the next few years. But neural network analytics can already detect, classify, and track objects very well, providing high accuracy even in busy scenes.

Artificial intelligence can be used in the real world to:

  • detect smoke and flames for early fire warning at open areas (forest, open warehouse, parking lot, etc.);
  • distinguish people/vehicles from animals and other moving objects, e.g. to protect the perimeter of a nature park from poachers;
  • distinguish a person in a helmet and protective clothing from a person without them to prevent accidents at a dangerous production facility or construction site;
  • count objects of a specific type, e.g. cars in a parking lot, people in the sales floor, wares moving on a conveyor belt, etc. in non-security-related solutions.

Those are just a few examples. After training a neural network, it can tackle other, similar tasks, too. Generally, a neural network trained in specific conditions isn't replicable. In other words, it won't work as well under different conditions. On the other hand, developers have learned how to quickly train AI for the needs of a specific project. The most important requirement is having enough video footage.

Somewhat apart from that is the use of neural networks in facial and automatic number-plate recognition. This is an example of reproducible neural networks (train once, deploy everywhere), which makes them more appealing commercially. If non-reproducible neural networks have only recently become economically feasible due to the rapid evolution of specialized hardware (aforementioned Intel's product, for example), then the use of AI in facial recognition and ANPR has been well established for a long time.

The use of AI in facial recognition and ANPR has been well established for a long time

Another kind of AI analytics that we'll explore is behavior analytics. This function, probably more than any other, is bringing video surveillance systems closer to understanding what's happening on camera. Its potential is vast.

How Behavior Analytics Works

From a technical point of view, behavior analytics combines artificial intelligence with a classic algorithmic approach. A neural network trained on a multitude of scenarios can determine the position of the bodies, heads, and limbs of humans in the camera's field of view. The algorithm outputs an array of data containing descriptions of their poses.

Conditions can be set for data to detect a specific pose, such as raised hands, prostrated or crouching persons. Developers can use this to quickly create new detection tools to identify potentially dangerous behavior specified by a government or business client. There's no need for additional training of the neural network.

How Behavior Analytics Can Be Deployed

Someone crouched down next to an ATM could be a technician, CIT guard, or burglar. Bank security should be notified in any of the cases.

A person in shooter position, together with a bank employee or cashier with their hands raised could indicate a robbery. The system can be configured to automatically send alerts with a surveillance snapshot to the police so they can assess the threat and take action if needed. It's vital that the police receive the alert, even if the employee is unable to activate the alarm.

In many cases, attention should be directed to a prostrate individual. This could be somebody who needs immediate help, or it could be someone sleeping in an inappropriate public place, for example, a 24/7 ATM space.

Behavioral analytics can also be used to ensure workplace safety. For example, tracking whether employees are holding the handrails when using the stairs at a manufacturing facility or a construction site.

What Now?

Behavior analytics can be deployed wherever your clients' imagination takes them. With this feature, practically any pose that indicates potentially dangerous behavior can be detected. Timely response to an alarm helps avoid material damages or, in other situations, casualties.

Practically any pose that indicates potentially dangerous behavior can be detected

An area of potential development for behavior analytics is the ability to analyze a sequence of poses by the same person or a combination of poses and relative positions of several individuals. That will be the next level of evolution in AI's use in video surveillance: moving from "detecting" to "understanding" behavior in real time.

In its most basic form, this type of analytics can be deployed to detect deviations from the search procedure in correctional facilities when a person being inspected must assume a pre-defined sequence of poses. A more advanced form allows it to detect any kind of abnormal behavior, such as a brawl breaking out in a public space. Ideally, behavior analytics can predict dangerous situations based on nearly imperceptible cues gleaned from collected statistics and a Big Data analysis.

At the moment, this sounds like pure fantasy, but what seemed like whimsy not too long ago is now a reality with AI. It's already beaten humans in chess and the game of Go (Weiqi). Will artificial intelligence be able to outplay humans at charades one day? It's entirely possible that we'll soon see for ourselves.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

Alan Ataev Chief Sales and Marketing Officer, AxxonSoft

In case you missed it

Sanitization, Safety And Getting Back To Business
Sanitization, Safety And Getting Back To Business

You are not alone: operators everywhere are asking themselves what are they going to do? How are they going to get back to business, and fast? How are they going to cost-effectively operate with all the new safety requirements that have arisen as a result of COVID? How are they going to ensure it all gets done for the safety of customers and staff? How are they going to protect their brand from the negative exposure of being identified as a property with a reputation for COVID? The economic impact of COVID is expected to hit brick and mortar businesses the worst, as their businesses are dependent on people being physically present. According to a recent report by RBC, it is estimated that 70% of Americans expect to avoid public spaces, 57% of Canadians will be unwilling to attend conferences without a vaccine and 63% of people will prefer to drive vs fly.  This means, that for those of you in the business of travel, conferences, co-working spaces, retail stores, museums, art galleries, restaurants, sports arenas, hotels, cruises, airlines, resorts, theme parks, long-term care, education, etc. in the blink of an eye your approach to on-site safety just changed. To ensure your property is safe and secure, it is no longer just about access control, video surveillance and intruder alarms; it is also about sanitisation To get back to business and operating at full capacity after COVID, operations must find a way to eliminate the fear, uncertainty and doubt in the minds of their customers and employees. The affect of COVID-19 on safety and security To safely get back to business, the Centers of Disease Control and Prevention (CDC) emphasis that all operations need a pandemic response planJust like cybersecurity has had a direct impact on the IT strategy and budget, COVID will have a direct hit on the operations strategy and budget. To ensure your property is safe and secure, it is no longer just about access control, video surveillance and intruder alarms; it is also about sanitization, the lines between the security and maintenance just blurred. From customers, to employees, to government regulators, to management, the focus is now on operations and the sanitization policies, procedures and actions of the team. To put this change of priority into perspective, six months ago, sanitisation was not top of mind for people. Why, because it was not a life or death issue, we had other first world problems to garner our attention. From an operations perspective if we enabled a sanitization issue to become significant enough to impact the safety of customers and staff and therefore the brand, then that was an operational choice versus a mistake. Standards for sanitisation  Just like cybersecurity has had a direct impact on the IT strategy and budget, COVID will have a direct hit on the operations strategy and budgetThe issue is, today while the operating priority of sanitization has significantly increased, it is not measured and managed to the same standard as the other safety and security concerns across a business. Also, important to consider, while people may not hold an operation liable during this first wave, we can guarantee they are not going to be as understanding during the second wave or a future pandemic. To safely get back to business, the Centers of Disease Control and Prevention (CDC) and the Occupational Health and Safety regulators emphasis that all operations need a pandemic response plan and should follow these simple guidelines: Develop your plan Implement your plan Maintain and revise your plan While this sounds simple enough, keep in mind that requirements are constantly evolving and will continue to do so for the foreseeable future, or at least until all the research is in. To create an emergency response plan for a pandemic, properties must first determine what needs to be sanitized. The current requirements dictate that most surfaces and objects will just need a normal routine cleaning, it is only the frequently touched surfaces and objects like light switches and COVID has changed the game and made the digital transformation of operating procedures not a ‘nice-to-have’ but a must-havedoorknobs that will need to be cleaned and then disinfected to further reduce the risk of germs on surfaces and objects. The challenge is when you step back and consider what people touch in a day; the list quickly grows. After only 30 minutes, I easily came up with a list of over 60 items that one could call ‘high touch’! If you think about it, the list is extensive; telephones, doorknobs, drawer handles, counters, pens, keypads, computers, etc. and the list is only going to get longer as the research comes in.   The challenge is when you step back and consider what people touch in a day; the list quickly grows Operating efficiency  If we don’t change our ways, not only will we be doomed to continue making the same mistakes, but we will continue to be lost in paper and filing cabinetsTo scope the impact on operations as part of the plan, we must then find and identify all of those high touch things across the property. If we then combine that with the fact that CDC requires that all high touch locations must not only be cleaned more often, but that they also require that each location is first cleaned with soap and water, and then disinfected for one minute before finally being wiped down. This means a one-minute task just turned into a 4-minute task, that must now be completed multiple times a day. From a resourcing perspective this adds up quickly, and operating efficiency must be a priority. Not to mention it is going to get very complicated to measure and manage especially. Post COVID rules Getting back to business is going to be complicated; lots to do, lots of moving parts and no technology to help. The fundamental challenge to keep in mind is not that the sanitization requirements have evolved, the real issue is that for most businesses this area has been left unchanged for generations. Still today most rely on checklists, logbooks and inspections to manage the responsibilities of our front-line workers, which might have been fine before COVID. Post-COVID the rules have changed and so should the approach to managing physical operating compliance on the front lines. COVID like most physical operating requirements is tactical, detailed and specific; broad strokes, the honor system and inspections are not going to cut it. The digital transformation  COVID has changed the game and made the digital transformation of operating procedures not a ‘nice-to-have’ but a must-have. If we don’t change our ways, not only will we be doomed to continue making the same mistakes, but we will continue to be lost in paper, filing cabinets filled with checklists, never to be seen again. Only with the right data can we significantly improve the operational decisions necessary to accelerate our return to full operating capacity. At the end of the day, to fully recover, operations must eliminate the fear, uncertainty and doubt in the minds of customers and employees, only then can we really get back to business.

Which Security Technology Is Most Misunderstood, And Why?
Which Security Technology Is Most Misunderstood, And Why?

The general public gets much of its understanding of security industry technology from watching movies and TV. However, there is a gap between reality and the fantasy world. Understanding of security technologies may also be shaped by news coverage, including expression of extreme or even exaggerated concerns about privacy. The first step in addressing any challenge is greater awareness, so we asked this week’s Expert Panel Roundtable: Which security industry technology is most misunderstood by the general public and why?

Lessons Learned With Vanderbilt: How Have You Adapted To The COVID-19 Pandemic?
Lessons Learned With Vanderbilt: How Have You Adapted To The COVID-19 Pandemic?

With the postponement of tradeshows and events due to the effects of COVID-19, Vanderbilt and ComNet have taken their high quality, innovative solutions online, directly to their customer base. Through an Online Events and Training resource, you can stay connected with the brands’ top resources and products, as well as join upcoming product webinars hosted by their in-house experts. With a majority of the world currently working from home, businesses must respond to this changing landscape. As such, Vanderbilt and ComNet have turned to online resources to share new product demonstrations and other company news. One cornerstone of the ACRE brands approach was the launch of their Online Events and Training resource page. Ross Wilks, Head of Marketing Communications at Vanderbilt, credits this online resource as the anchor to their communicative success with customers at present. “Through weekly webinars delivered by our in-house experts, Vanderbilt and ComNet have embraced more virtual opportunities to continuously communicate to our customers regarding our latest and most relevant products,” he says. “To date, our webinars have covered a wide range of industry topics such as Why Physical Security and Cloud go together, and The most recent developments in card cloning and reader hacking. Attendance to these online events has proved popular and effective in keeping communication with our customer base open and engaging.” Each webinar ends with a Q&A section, as well as follow-up articles on the most asked questions, plus recordings of the webinars being made available to attendees. As such, the webinar approach has proven a receptive approach for Vanderbilt and ComNet. The Online Events and Training resource acts as a one-stop-shop for all virtual information. Overall, the page outlines the brands’ value-added resources for customers, including the ability to request a remote product demonstration, the availability of free online training, 24/7 access to the Vanderbilt webshop, plus the aforementioned weekly webinars. Vanderbilt and ComNet’s business mantra is built on a foundation of customer-focused core values such as empowerment, collaboration, and high performance and Wilks credits this mentality with their ability to keep information flowing to their base during the present pandemic. “The ACRE brands moved early to kick-start online webinars and ramp up awareness of their already existing online training and shopping options. Now more than ever, it is important to keep customers up to date on the latest offerings,” Wilks explains. “Our commitment has always been to make their customer’s security journey the best possible experience, and that is what this Online Events and Learning page primarily focuses on,” he concludes.