Video surveillance is commonly associated with security. But in most cases, it's used to record incidents and assist in investigations after the fact rather than prevent undesirable events.

Artificial intelligence–powered video analytics is a highly promising trend that fundamentally changes the way things work. Extracting manageable data from a video stream can help recognize risky situations early on, minimizing damage and, ideally, completely avoid emergencies. At the same time, AI significantly expands the areas of application of video surveillance beyond security systems.

AI significantly expands the areas of application of video surveillance beyond security system


However, the hype around this new, trendy technology prevents the potential user from choosing quality solutions in a wide variety of products. This often leads to over-expectation, followed by a complete let-down. Can AI-powered video analytics really be the key to a technological breakthrough in video surveillance? We'll take a look at what the technology can do, what it can't, and where it can go from here.

Technological Breakthrough Or Just Another Bubble?

It's often said that the video management software (VMS) market is becoming increasingly commoditized and widely available. A lot of products with similar features (or, at least, similar promises from the manufacturer) make it hard to choose. As a result, vendor names and reputations are turning into one of their primary selling points. Manufacturers have two choices available: get wrapped up in a price war and rely on cutting expenses, or offer a product that's truly innovative and revolutionary.

Manufacturers have two choices available: get wrapped up in a price war, or offer a product that's truly innovative and revolutionary

VMS developers who choose the second route are gravitating towards creating products that use artificial intelligence based on neural networks and deep learning. Emerging two or three years ago, the AI video analytics market is experiencing a boom in growth. This new tech wave has stirred the still, stagnant backwaters of the VMS world and gave small, ambitious developers something to be optimistic about. It seems they now have a chance to emerge as market leaders in the next few years.

However, the hype around this popular trend is raising reasonable concerns among experienced security industry professionals. These concerns come from clients looking for a solution to their problems, and from suppliers building a long-term development strategy.

This largely resembles another tech bubble, like the one built up around pre-AI video analytics and burst when it became clear that the promises around it were pure marketing hype. However, there are factors that indicate that AI-powered video surveillance systems aren't another bubble.

The Three Factors

The first — and the main one — comes from systems already in place on customers' sites. They fulfill the same promises made during the previous bubble by hotheads in a rush to teach the computer to analyze events in real time using a classical algorithmic approach.

The second is the fact that this new technology has seen investment from not only software and cloud startups, but also established VMS developers. Even giants like Intel, which has presented a full line of neural network accelerator hardware and a set of software tools that streamlines working with them, specifically in the field of computer vision.

This new technology has seen investment from not only software and cloud startups, but also established VMS developers

The third factor lies in artificial intelligence's abilities. AI plays chess, drives cars, and works wonders in many other fields. Why shouldn't it be applied to video monitoring and analysis?

What AI Can Do

Just what can artificial intelligence do in video surveillance systems at this stage of development? It can't quite analyze a sequence of events and understand the "logic" of what's happening in the cameras' field of view. At least not yet. But it's probable that AI will learn to do this in the next few years. But neural network analytics can already detect, classify, and track objects very well, providing high accuracy even in busy scenes.


Artificial intelligence can be used in the real world to:

  • Detect smoke and flames for early fire warning at open areas (forest, open warehouse, parking lot, etc.);
  • Distinguish people/vehicles from animals and other moving objects, e.g. to protect the perimeter of a nature park from poachers;
  • Distinguish a person in a helmet and protective clothing from a person without them to prevent accidents at a dangerous production facility or construction site;
  • Count objects of a specific type, e.g. cars in a parking lot, people in the sales floor, wares moving on a conveyor belt, etc. in non-security-related solutions.

Those are just a few examples. After training a neural network, it can tackle other, similar tasks, too. Generally, a neural network trained in specific conditions isn't replicable. In other words, it won't work as well under different conditions. On the other hand, developers have learned how to quickly train AI for the needs of a specific project. The most important requirement is having enough video footage.

Somewhat apart from that is the use of neural networks in facial and automatic number-plate recognition. This is an example of reproducible neural networks (train once, deploy everywhere), which makes them more appealing commercially. If non-reproducible neural networks have only recently become economically feasible due to the rapid evolution of specialized hardware (aforementioned Intel's product, for example), then the use of AI in facial recognition and ANPR has been well established for a long time.

The use of AI in facial recognition and ANPR has been well established for a long time

Another kind of AI analytics that we'll explore is behavior analytics. This function, probably more than any other, is bringing video surveillance systems closer to understanding what's happening on camera. Its potential is vast.

How Behavior Analytics Works

From a technical point of view, behavior analytics combines artificial intelligence with a classic algorithmic approach. A neural network trained on a multitude of scenarios can determine the position of the bodies, heads, and limbs of humans in the camera's field of view. The algorithm outputs an array of data containing descriptions of their poses.

Conditions can be set for data to detect a specific pose, such as raised hands, prostrated or crouching persons. Developers can use this to quickly create new detection tools to identify potentially dangerous behavior specified by a government or business client. There's no need for additional training of the neural network.

How Behavior Analytics Can Be Deployed

Someone crouched down next to an ATM could be a technician, CIT guard, or burglar. Bank security should be notified in any of the cases.

A person in shooter position, together with a bank employee or cashier with their hands raised could indicate a robbery. The system can be configured to automatically send alerts with a surveillance snapshot to the police so they can assess the threat and take action if needed. It's vital that the police receive the alert, even if the employee is unable to activate the alarm.

In many cases, attention should be directed to a prostrate individual. This could be somebody who needs immediate help, or it could be someone sleeping in an inappropriate public place, for example, a 24/7 ATM space.

Behavioral analytics can also be used to ensure workplace safety. For example, tracking whether employees are holding the handrails when using the stairs at a manufacturing facility or a construction site.

What Now?

Behavior analytics can be deployed wherever your clients' imagination takes them. With this feature, practically any pose that indicates potentially dangerous behavior can be detected. Timely response to an alarm helps avoid material damages or, in other situations, casualties.

Practically any pose that indicates potentially dangerous behavior can be detected

An area of potential development for behavior analytics is the ability to analyze a sequence of poses by the same person or a combination of poses and relative positions of several individuals. That will be the next level of evolution in AI's use in video surveillance: moving from "detecting" to "understanding" behavior in real time.

In its most basic form, this type of analytics can be deployed to detect deviations from the search procedure in correctional facilities when a person being inspected must assume a pre-defined sequence of poses. A more advanced form allows it to detect any kind of abnormal behavior, such as a brawl breaking out in a public space. Ideally, behavior analytics can predict dangerous situations based on nearly imperceptible cues gleaned from collected statistics and a Big Data analysis.

At the moment, this sounds like pure fantasy, but what seemed like whimsy not too long ago is now a reality with AI. It's already beaten humans in chess and the game of Go (Weiqi). Will artificial intelligence be able to outplay humans at charades one day? It's entirely possible that we'll soon see for ourselves.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

Alan Ataev Chief Sales and Marketing Officer, AxxonSoft

In case you missed it

What Will Be The Security Industry’s Biggest Challenge In 2021?
What Will Be The Security Industry’s Biggest Challenge In 2021?

What a year 2020 was for the security industry! There were vast challenges that could not have been foreseen at the beginning of the year. It is safe to say that the events of 2020 defied all industry prognosticators. However, is that any reason not to hope our expectations looking ahead to 2021 will be much closer to reality? Can we possibly benefit from looking ahead and trying to plan for the challenges of the year ahead? We asked this week’s Expert Panel Roundtable: What will be the security industry’s biggest challenge in 2021?

Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool
Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool

Artificial intelligence (AI) is more than a buzzword. AI is increasingly becoming part of our everyday lives, and a vital tool in the physical security industry. In 2020, AI received more attention than ever, and expanded the ways it can contribute value to physical security systems. This article will revisit some of those development at year-end, including links back to the originally published content. In the security market today, AI is expanding the use cases, making technologies more powerful and saving money on manpower costs - and today represents just the beginning of what AI can do for the industry. What it will never do, however, is completely take the place of humans in operating security systems. There is a limit to how much we are willing to turn over to machines - even the smartest ones. Beyond video analytics "Apply AI to security and now you have an incredibly powerful tool that allows you to operate proactively rather than reactively," said Jody Ross of AMAG Technology, one of our Expert Roundtable Panelists. AI made its initial splash in the physical security market by transforming the effectiveness of video analytics AI made its initial splash in the physical security market by transforming the effectiveness of video analytics. However, now there are many other applications, too, as addressed by our Expert Panel Roundtable in another article. Artificial intelligence (AI) and machine learning provide useful tools to make sense of massive amounts of Internet of Things (IoT) data. By helping to automate low-level decision-making, the technologies can make security operators more efficient. Biometrics with access control Intelligent capabilities can expand integration options such as increasing the use of biometrics with access control. AI can also help to monitor mechanics and processes. Intelligent systems can help end users understand building occupancy and traffic patterns and even to help enforce physical distancing. These are just a few of the possible uses of the technologies - in the end, the sky is the limit. AI is undoubtedly one of the bigger disrupters in the physical security industry, and adoption is growing at a rapid rate. And it’s not just about video analytics. Rather, it is data AI, which is completely untapped by the security industry. Bottom line: AI can change up your security game by automatically deciphering information to predict the future using a wide range of sources and data that have been collected, whether past, present, and future. That’s right. You can look into the future. Smarter perimeter protection Now, Intrusion Detection (Perimeter Protection) systems with cutting-edge, built-in AI algorithms to recognize a plethora of different object types, can distinguish objects of interest, thus significantly decreasing the false-positive intrusion rate. The more advanced AI-based systems enable the users to draw ROIs based on break-in points, areas of high-valuables, and any other preference to where alerts may be beneficial. AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store Similarly, AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store. The loitering time and region of interest are customizable in particular systems, which allows for a range of detection options. Smart security is advancing rapidly. As AI and 4K rise in adoption on smart video cameras, these higher video resolutions are driving the demand for more data to be stored on-camera. AI and smart video promise to extract greater insights from security video. Meeting urban needs Complex, extensive camera networks will already require a large amount of data storage, particularly if this is 24/7 monitoring from smart video-enabled devices. Newer edge computing will play an important role in capturing, collecting, and analyzing data. There are many more types of cameras being used today, such as body cameras, dashboard cameras, and new Internet of Things (IoT) devices and sensors. Video data is so rich nowadays, you can analyze it and deduce a lot of valuable information in real-time, instead of post-event. In smart cities applications, the challenge of identifying both physical and invisible threats to meet urban citizens’ needs will demand a security response that is proactive, adaptable and dynamic. Optimize security solutions As we look ahead to the future of public safety, it’s clear that new technologies, driven by artificial intelligence (AI), can dramatically improve the effectiveness of today’s physical security space. For smart cities, the use of innovative AI and machine learning technologies have already started to help optimize security solutions. In sports stadium applications, AI’s role in getting fans and spectators back after the COVID pandemic is huge, through capabilities such as social distance monitoring, crowd scanning/metrics, facial recognition, fever detection, track and trace and providing behavioral analytics. Technologies such as AI-powered collaboration platforms now work alongside National Leagues, Franchises and Governing Bodies to implement AI surveillance software into their CCTV/surveillance cameras. AI surveillance software In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI This is now creating a more collaborative effort from the operations team in stadiums, rather than purely security. AI surveillance software, when implemented into the surveillance cameras can be accessed by designated users on any device and on any browser platform. One of the biggest advantages of using AI technology is that it’s possible to integrate this intelligent software into building smarter, safer communities and cities. Essentially, this means developing a layered system that connects multiple sensors for the detection of visible and invisible threats. Integrated systems mean that threats can be detected and tracked, with onsite and law enforcement notified faster, and possibly before an assault begins to take place. In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI. Fighting illicit trade Using technology in this way means that thousands of people can be screened seamlessly and quickly, without invading their civil liberties or privacy. AI’s ability to detect visible or invisible threats or behavioral anomalies will prove enormously valuable to many sectors across our global economy. Revolutionary AI-driven technologies can help to fight illicit trade across markets. AI technologies in this specific application promise to help build safer and more secure communities in the future. AI can support the ongoing fight against illicit trade on a global scale in a tangible way. For financial transactions at risk of fraud and money laundering, for example, tracking has become an increasing headache if done manually. As a solution to this labor-intensive process, AI technology can be trained to follow all the compliance rules and process a large number of documents - often billions of pages of documents - in a short period of time.

Retail Security In 2021 And Beyond
Retail Security In 2021 And Beyond

This year has brought about changes in virtually every sector. As with other frontline industries, the security sector has been tested more than those able to move entirely to remote working. While the promise of a vaccine means an end is in sight, the post-COVID era will not bring with it a return to the ‘normal’ we knew before the pandemic. Organizations have adapted, becoming more resilient and agile and this will have lasting effects. The coming months will continue to be testing. The tiered system will see the precautions in place fluctuate with the situation. Initial lockdown period At the same time, a gradual return to normal as the vaccine is rolled out will require adaptive measures. The security sector will be at the heart of keeping people safe throughout this process. The initial lockdown period and the first wave of panic buying might seem like a lifetime ago. However, the introduction of the second lockdown in November was accompanied by another wave of stockpiling despite organizations trying to reassure their customers. It is uncertainty that breeds anxiety, and we continue to see this as the restrictions fluctuate across the country. The tier system depends upon a number of factors: case detection rate, how quickly case numbers are rising or falling, positive COVID-19 test numbers in the general population, pressure on the NHS in that region, and local context and exceptional circumstances. Social distancing measures For the sector to meet demand, technology will be needed to work alongside the manned guarding role While travel is allowed in all tiers if necessary for work, government advice still recommends that those able to work from home should do so. This means that throughout the country, many buildings will remain empty or at minimal capacity for some time to come. Security risks vary with the restrictions in each area. Although shops are largely open, tier three still requires the closure of many premises. Vacant premises are more vulnerable to theft and damage, meaning officers and security technology remain in higher demand than usual. As more premises are allowed to open, the need for officers to implement social distancing measures increases, stretching the sector like never before. For the sector to meet demand, technology will be needed to work alongside the manned guarding role. Temperature checking devices It will continue to be important in providing security when officers cannot be present in person through CCTV and sensors. But it will also be integrated into the manned guarding role to streamline processes. We are already seeing the start of this as many officers are using handheld temperature checking devices to reduce the spread of COVID-19. We’ll also see temperature scanners installed into buildings to allow security guards to focus on other priorities. Those businesses that are open will need to continue to adapt to the changing regulations in the coming months. Christmas is a busy period that stretches the retail sector. Unpredictability results in heightened stress levels and makes it more difficult for people to reliably take in and recall information. Security officers are a key first point of contact both to enforce measures and reassure anxious staff and members of the public. Enforcing one-Way systems Security staff will need to keep members of the public safe and prevent disruption Some shops are enforcing one-way systems and limiting the number of customers allowed inside. Over Christmas, many more may choose to do so. Security officers will be responsible for ensuring these precautions are followed. With the heightened pressure of the festive period, it can be hard to predict how members of the public will respond to officers enforcing measures. Security staff will need to keep members of the public safe and prevent disruption. Doing so will require tact and empathy in dealing with customers. Within shops, too, officers will be tasked with ensuring social distancing and other measures are followed effectively. Doing so, they must work closely with clients to understand what protocols are in place and how to handle a breach. They must also be able to enact discretion. For example, clients may not take issue with protocols being broken momentarily or accidentally. Extensive government guidance There is extensive government guidance on the precautions that should be taken on various premises. They include the introduction of one-way systems and limiting building capacity. Measures such as one-way systems may be broken by those that don’t notice or don’t care. Officers must be able to judge what responses are appropriate while maintaining a calm and reassuring presence. The security officer role has long been moving toward a more front of house position as, for many visitors to a building, they are the first point of contact. The pandemic has accelerated this trend. Working on the frontline of the pandemic, officers have had to play a more multifaceted role than ever before. Officers still act as deterrents and manage security issues, but they must also use empathy and strong communication skills to inform and reassure customers and staff onsite. Adapting to new technology They will need to be able to learn quickly on the job and adapt to new technology and practices Being able to demonstrate this flexibility and to read a situation and react appropriately will be some of the skills most in demand in the industry in the future. Officers will need to build close working relationships with clients. In addition, many will have new roles, such as taking temperatures with handheld devices. They will need to be able to learn quickly on the job and adapt to new technology and practices. Technology, too, will be more important. If the global pandemic has driven any point home, it is that we cannot always see or sense threats. Data-Driven insights Temperature checks and occupancy sensors will be the norm in protecting from COVID, while security technology and data-driven insights will continue to grow in popularity. The security sector specializes in adapting to the unexpected and the threatening. It continues to demonstrate incredible value through the pandemic. While the coming months will undoubtedly be trying, the sector is adapting. Lessons have been learned from the pandemic that will affect business globally. Security specialists are taking these on and creating a stronger and more effective industry.