Over the years, video analytics has gained an unfavourable reputation for over-promising and under-delivering in terms of performance. One of the biggest complaints regarding video analytics has been its inability to correctly identify objects in situations which appear trivial to the human observer. In many cases, this has resulted in a tendency to generate substantial numbers of false alarms, while not detecting actual events accurately. This, together with a propensity for complex set-up procedures and much need for manual fine-tuning, has prevented video analytics from becoming a mainstream application deployed on large numbers of cameras.

Machine Learning And Video Analytics

Machine Learning is a well-established field of research that has existed for decades, which is already present in many products and applications. Machine Learning is based on collecting large amounts of data specific to a particular problem, training a model using this data and then employing this model to process new data. With regard to video analytics, one of the most critical problems impacting accuracy is object classification. Fundamental to improving performance is the capability to teach the algorithm to distinguish between people, animals, different types of vehicles and sources of noise at an extremely high level of accuracy.

Deep Learning Algorithms

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units

Until recently there have been minimal applications of Machine Learning used in video analytics products, largely due to high complexity and high resource usage, which made such products too costly for mainstream deployment. However, the last couple of years have seen a tremendous surge in research and advances surrounding a branch of Machine Learning called Deep Learning. Deep Learning is a name used to describe a family of algorithms based on the concept of neural networks. Very loosely speaking, these algorithms try to emulate the functionality of the brain’s neurons, enabling them to learn efficiently from example, and subsequently apply this learning to new data.

The recent increased interest in Deep Learning is largely due to the availability of graphical processing units (GPUs). GPUs can efficiently train and run Deep Learning algorithms, and have allowed the scientific community to accelerate their research and application, bringing them to the point where they exceed the performance of most traditional Machine Learning algorithms across several categories.

Solving Object Classification With Deep Learning

This means that Deep Learning can now be used to solve the most crucial problem facing video analytics – object classification – by collecting many thousands of images from hundreds of surveillance cameras, which must first be manually labelled and classified by a human, into a range of categories that include: person, car, bus, truck, bird, vegetation, dog and many more. To achieve the required accuracy rates, such a vast database must be collected and identified from actual surveillance footage.

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications is the ability to continuously update the models as more data is collected
Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics

A Deep Learning algorithm trained on images collected from YouTube, Google Search and elsewhere on the Internet will completely fail in analysing images from surveillance cameras, due to the difference in viewing angles, resolution and image quality. Once enough images are collected, a Deep Learning classifier algorithm can be trained and deployed as part of a video analytics solution, enabling it to practically eliminate most of the existing causes for false alarms.

Due to GPU requirements in order for the algorithms to run efficiently, video analytics solutions using Deep Learning will initially need to run on a server. A few solutions of this nature are already available and are showing a dramatic leap in performance in comparison to traditional video analytics, with a drastic reduction in false alarm rates and a significant increase in detection accuracy. Concurrently, these new solutions do not require manual tweaking by the user and are essentially plug-and-play, making mass deployment a realistic premise.

Surveillance Applications Of Deep Learning

Basic classification and false alarm reduction are the first applications of Deep Learning for video analytics, but they are by no means the only ones. In the not-too-distant future, we will see Deep Learning enabling as yet not possible video analytics applications, such as identifying objects carried by people, such as a gun, handbag, or a knife, or being able to quickly find people and vehicles with similar appearances across multiple cameras and more.

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers to running inside cameras

Over the next few years, we will see a transition of video analytics using Deep Learning running on servers, to running inside cameras, as powerful, low-cost hardware capable of running Deep Learning becomes more available and a basic function of newer surveillance camera models. This will push the acceptance of video analytics even further, eventually making it a fundamental element of every surveillance camera deployed.

Increasing Accuracy With Updates

A crucial component in achieving and maintaining the high performance of Deep Learning-based applications, is the ability to continuously update the models as more data is collected so that the models increase in accuracy. This will give an advantage to cloud-based video analytics services, since they can collect vast amounts of data from cameras connected to the service, train new models in the cloud based on this data and then push these new models to cameras at the edge. This continuous improvement cycle will be instrumental in helping video analytics fulfil the promise of improving peoples’ safety and security, by giving surveillance cameras human-level accuracy and a comprehensive understanding of the environment.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

In case you missed it

Safety In Smart Cities: How Video Surveillance Keeps Security Front And Center
Safety In Smart Cities: How Video Surveillance Keeps Security Front And Center

Urban populations are expanding rapidly around the globe, with an expected growth of 1.56 billion by 2040. As the number of people living and working in cities continues to grow, the ability to keep everyone safe is an increasing challenge. However, technology companies are developing products and solutions with these futuristic cities in mind, as the reality is closer than you may think. Solutions that can help to watch over public places and share data insights with city workers and officials are increasingly enabling smart cities to improve the experience and safety of the people who reside there. Rising scope of 5G, AI, IoT and the Cloud The main foundations that underpin smart cities are 5G, Artificial Intelligence (AI), and the Internet of Things (IoT) and the Cloud. Each is equally important, and together, these technologies enable city officials to gather and analyze more detailed insights than ever before. For public safety in particular, having IoT and cloud systems in place will be one of the biggest factors to improving the quality of life for citizens. Smart cities have come a long way in the last few decades, but to truly make a smart city safe, real-time situational awareness and cross-agency collaboration are key areas which must be developed as a priority. Innovative surveillance cameras with integrated IoT Public places need to be safe, whether that is an open park, shopping center, or the main roads through towns Public places need to be safe, whether that is an open park, shopping center, or the main roads through towns. From dangerous drivers to terrorist attacks, petty crime on the streets to high profile bank robberies, innovative surveillance cameras with integrated IoT and cloud technologies can go some way to helping respond quickly to, and in some cases even prevent, the most serious incidents. Many existing safety systems in cities rely on aging and in some places legacy technology, such as video surveillance cameras. Many of these also use on-premises systems rather than utilising the benefits of the cloud. Smart programming to deliver greater insights These issues, though not creating a major problem today, do make it more challenging for governments and councils to update their security. Changing every camera in a city is a huge undertaking, but in turn, doing so would enable all cameras to be connected to the cloud, and provide more detailed information which can be analyzed by smart programming to deliver greater insights. The physical technologies that are currently present in most urban areas lack the intelligent connectivity, interoperability and integration interfaces that smart cities need. Adopting digital technologies isn’t a luxury, but a necessity. Smart surveillance systems It enables teams to gather data from multiple sources throughout the city in real-time, and be alerted to incidents as soon as they occur. Increased connectivity and collaboration ensures that all teams that need to be aware of a situation are informed instantly. For example, a smart surveillance system can identify when a road accident has occurred. It can not only alert the nearest ambulance to attend the scene, but also the local police force to dispatch officers. An advanced system that can implement road diversions could also close roads around the incident immediately and divert traffic to other routes, keeping everyone moving and avoiding a build-up of vehicles. This is just one example: without digital systems, analyzing patterns of vehicle movements to address congestion issues could be compromised, as would the ability to build real-time crime maps and deploy data analytics which make predictive policing and more effective crowd management possible. Cloud-based technologies Cloud-based technologies provide the interoperability, scalability and automation Cloud-based technologies provide the interoperability, scalability and automation that is needed to overcome the limitations of traditional security systems. Using these, smart cities can develop a fully open systems architecture that delivers interoperation with both local and other remote open systems. The intelligence of cloud systems can not only continue to allow for greater insights as technology develops over time, but it can do so with minimal additional infrastructure investment. Smart surveillance in the real world Mexico City has a population of almost 9 million people, but if you include the whole metropolitan area, this number rises sharply to over 21 million in total, making it one of the largest cities on the planet. Seven years ago, the city first introduced its Safe City initiative, and ever since has been developing newer and smarter ways to keep its citizens safe. In particular, its cloud-based security initiative is making a huge impact. Over the past three years, Mexico City has installed 58,000 new video surveillance cameras throughout the city, in public spaces and on transport, all of which are connected to the City’s C5 (Command, Control, Computers, Communications and Citizen Contact) facility. Smart Cities operations The solution enables officers as well as the general public to upload videos via a mobile app to share information quickly, fixed, body-worn and vehicle cameras can also be integrated to provide exceptional insight into the city’s operations. The cloud-based platform can easily be upgraded to include the latest technology innovations such as license plate reading, behavioral analysis software, video analytics and facial recognition software, which will all continue to bring down crime rates and boost response times to incidents. The right cloud approach Making the shift to cloud-based systems enables smart cities to eliminate dependence on fiber-optic connectivity and take advantage of a variety of Internet and wireless connectivity options that can significantly reduce application and communication infrastructure costs. Smart cities need to be effective in years to come, not just in the present day, or else officials have missed one of the key aspects of a truly smart city. System designers must build technology foundations now that can be easily adapted in the future to support new infrastructure as it becomes available. Open system architecture An open system architecture will also be vital for smart cities to enhance their operations For example, this could include opting for a true cloud application that can support cloud-managed local devices and automate their management. An open system architecture will also be vital for smart cities to enhance their operations and deliver additional value-add services to citizens as greater capabilities become possible in the years to come. The advances today in cloud and IoT technologies are rapid, and city officials and authorities have more options now to develop their smart cities than ever before and crucially, to use these innovations to improve public safety. New safety features Though implementing these cloud-based systems now requires investment, as new safety features are designed, there will be lower costs and challenges associated with introducing these because the basic infrastructure will already exist. Whether that’s gunshot detection or enabling the sharing of video infrastructure and data across multiple agencies in real time, smart video surveillance on cloud-based systems can bring a wealth of the new opportunities.

What Will Be The Security Industry’s Biggest Challenge In 2021?
What Will Be The Security Industry’s Biggest Challenge In 2021?

What a year 2020 was for the security industry! There were vast challenges that could not have been foreseen at the beginning of the year. It is safe to say that the events of 2020 defied all industry prognosticators. However, is that any reason not to hope our expectations looking ahead to 2021 will be much closer to reality? Can we possibly benefit from looking ahead and trying to plan for the challenges of the year ahead? We asked this week’s Expert Panel Roundtable: What will be the security industry’s biggest challenge in 2021?

Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool
Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool

Artificial intelligence (AI) is more than a buzzword. AI is increasingly becoming part of our everyday lives, and a vital tool in the physical security industry. In 2020, AI received more attention than ever, and expanded the ways it can contribute value to physical security systems. This article will revisit some of those development at year-end, including links back to the originally published content. In the security market today, AI is expanding the use cases, making technologies more powerful and saving money on manpower costs - and today represents just the beginning of what AI can do for the industry. What it will never do, however, is completely take the place of humans in operating security systems. There is a limit to how much we are willing to turn over to machines - even the smartest ones. Beyond video analytics "Apply AI to security and now you have an incredibly powerful tool that allows you to operate proactively rather than reactively," said Jody Ross of AMAG Technology, one of our Expert Roundtable Panelists. AI made its initial splash in the physical security market by transforming the effectiveness of video analytics AI made its initial splash in the physical security market by transforming the effectiveness of video analytics. However, now there are many other applications, too, as addressed by our Expert Panel Roundtable in another article. Artificial intelligence (AI) and machine learning provide useful tools to make sense of massive amounts of Internet of Things (IoT) data. By helping to automate low-level decision-making, the technologies can make security operators more efficient. Biometrics with access control Intelligent capabilities can expand integration options such as increasing the use of biometrics with access control. AI can also help to monitor mechanics and processes. Intelligent systems can help end users understand building occupancy and traffic patterns and even to help enforce physical distancing. These are just a few of the possible uses of the technologies - in the end, the sky is the limit. AI is undoubtedly one of the bigger disrupters in the physical security industry, and adoption is growing at a rapid rate. And it’s not just about video analytics. Rather, it is data AI, which is completely untapped by the security industry. Bottom line: AI can change up your security game by automatically deciphering information to predict the future using a wide range of sources and data that have been collected, whether past, present, and future. That’s right. You can look into the future. Smarter perimeter protection Now, Intrusion Detection (Perimeter Protection) systems with cutting-edge, built-in AI algorithms to recognize a plethora of different object types, can distinguish objects of interest, thus significantly decreasing the false-positive intrusion rate. The more advanced AI-based systems enable the users to draw ROIs based on break-in points, areas of high-valuables, and any other preference to where alerts may be beneficial. AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store Similarly, AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store. The loitering time and region of interest are customizable in particular systems, which allows for a range of detection options. Smart security is advancing rapidly. As AI and 4K rise in adoption on smart video cameras, these higher video resolutions are driving the demand for more data to be stored on-camera. AI and smart video promise to extract greater insights from security video. Meeting urban needs Complex, extensive camera networks will already require a large amount of data storage, particularly if this is 24/7 monitoring from smart video-enabled devices. Newer edge computing will play an important role in capturing, collecting, and analyzing data. There are many more types of cameras being used today, such as body cameras, dashboard cameras, and new Internet of Things (IoT) devices and sensors. Video data is so rich nowadays, you can analyze it and deduce a lot of valuable information in real-time, instead of post-event. In smart cities applications, the challenge of identifying both physical and invisible threats to meet urban citizens’ needs will demand a security response that is proactive, adaptable and dynamic. Optimize security solutions As we look ahead to the future of public safety, it’s clear that new technologies, driven by artificial intelligence (AI), can dramatically improve the effectiveness of today’s physical security space. For smart cities, the use of innovative AI and machine learning technologies have already started to help optimize security solutions. In sports stadium applications, AI’s role in getting fans and spectators back after the COVID pandemic is huge, through capabilities such as social distance monitoring, crowd scanning/metrics, facial recognition, fever detection, track and trace and providing behavioral analytics. Technologies such as AI-powered collaboration platforms now work alongside National Leagues, Franchises and Governing Bodies to implement AI surveillance software into their CCTV/surveillance cameras. AI surveillance software In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI This is now creating a more collaborative effort from the operations team in stadiums, rather than purely security. AI surveillance software, when implemented into the surveillance cameras can be accessed by designated users on any device and on any browser platform. One of the biggest advantages of using AI technology is that it’s possible to integrate this intelligent software into building smarter, safer communities and cities. Essentially, this means developing a layered system that connects multiple sensors for the detection of visible and invisible threats. Integrated systems mean that threats can be detected and tracked, with onsite and law enforcement notified faster, and possibly before an assault begins to take place. In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI. Fighting illicit trade Using technology in this way means that thousands of people can be screened seamlessly and quickly, without invading their civil liberties or privacy. AI’s ability to detect visible or invisible threats or behavioral anomalies will prove enormously valuable to many sectors across our global economy. Revolutionary AI-driven technologies can help to fight illicit trade across markets. AI technologies in this specific application promise to help build safer and more secure communities in the future. AI can support the ongoing fight against illicit trade on a global scale in a tangible way. For financial transactions at risk of fraud and money laundering, for example, tracking has become an increasing headache if done manually. As a solution to this labor-intensive process, AI technology can be trained to follow all the compliance rules and process a large number of documents - often billions of pages of documents - in a short period of time.