A tipping point is defined as: “The point at which a series of changes becomes significant enough to cause a larger, more important change”. In the same way that IP video changed surveillance a decade ago, our industry is now feeling the impact of recent developments in Artificial Intelligence, Machine Learning, Deep Learning, Big Data, and Intelligent Video Analysis.

Keyword Definitions

Let’s start with a few more definitions. Artificial Intelligence (AI) deals with the simulation of intelligent behavior in computers. Machine Learning (ML) deals with developing computer algorithms that access data and use it to learn for themselves. Neural networks are computer systems that loosely mimic human brain operation.

Deep Learning is a subset of ML based on neural networks that has been proven to provide breakthrough capabilities in many problems that were previously unsolvable, and Big Data, or metadata, refers to huge amounts of structured and/or unstructured data -- in our case, the immense quantities of video information being generated daily by security cameras deployed in cities around the world. Deep Learning is tipped to change Intelligent Video Analysis (IVA), the digital video technology integrated with analytical software that is a basic tool for our industry.

AI In Surveillance

Traditionally, the main benefit of surveillance cameras is the ability to collect evidence for debriefing or investigation, as well as the ability to view events remotely in real-time. A decade ago, video analytics technologies were introduced to solve the problem of human inattention -- computers don’t get tired, bored or distracted, and can monitor a camera continuously.

And then, camera costs dropped, deployment skyrocketed, and video management systems began collecting reams of useless, costly unstructured data. AI technology seemed to answer the pressing new industry needs of how to use this Big Data effectively, make a return on the investment in expensive storage, while maintaining (or even lowering) human capital costs.

Three Limiting Factors

All this was theoretical, however, as multiple technological barriers prevented AI solutions from real-world utilization. Despite decades of research on how to cause a computer to accurately recognize different objects in a video stream, the quality of the results, especially in urban environments, was, to put it mildly, underwhelming.

Deep Learning software must be able to differentiate between different objects and under various circumstances
Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video

AI was limited primarily by these three factors: 

  • Lack of understanding -- The software must be able to differentiate between different objects (person, vehicle, animal, etc.), and under various circumstances (day, night, seasonal weather conditions, etc.).

  • Inability to learn -- Traditional IVA applications relied on a rule-based approach that required software configuration -- by a human operator -- for each monitoring camera and each type of alert. Although effective in some scenarios, the exponential growth in camera counts rendered this approach impractical, given the amount of manual labor required to configure, reconfigure, and maintain rules.

  • High cost -- The hard truth is that budgets for security and safety will always be constrained. Until recently, implementing real-time AI was extremely cost-prohibitive, sometimes requiring a 1:1 server to camera ratio.

Meeting Challenges

That was yesterday. Today, the application of AI in security applications has reached its tipping point, meeting the above-mentioned challenges. 

  • Understanding -- Deep Learning has matured to the point where it can accurately detect and classify objects both in still images and in video. DL technology is fast becoming the basic building block for IVA.

  • Ability to Learn -- As an AI solution collects and analyses data over time, it creates metadata that describes all objects in each video stream. Machine Learning techniques process this metadata to generate models for “normally observed” behavior. These models are applied in real-time to detect behaviors deviating from the norm. Only those flagged as suspicious events require review by a human operator. This technique allows the solution to scale to an unlimited number of cameras, with no need for a human to configure each new device.

  • Lower cost -- The rapid increase in GPU computational capacity, coupled with mass market adoption, has lowered server costs to a reasonable level. Today, with the correct implementation, a single server can be deployed across hundreds and even thousands of cameras.

The convergence of Deep Learning for video analysis, advances in AI for fully automated event detection, plus the significant reduction in cost to implement these techniques – including cloud-based software as a service (SaaS) models -- means that the fully automated video surveillance solution for cities is fast becoming a reality. We’ll see more of this type of solution being deployed over the coming months, and within the next few years, it will be standard in any Smart City deployment.

Download PDF version

Author Profile

In case you missed it

Intelligent Video Surveillance And Deep Learning Dominate MIPS 2018 Agenda
Intelligent Video Surveillance And Deep Learning Dominate MIPS 2018 Agenda

Milestone Systems is embracing artificial intelligence and deep learning in a big way at this week's yearly Milestone Community Days (MIPS 2018) in Las Vegas. The Danish company's theme is "Creating an Intelligent World," and Milestone's stated goal is to make "the Milestone community part of every surveillance installation in the world."   Science Fiction Becomes Reality In a presentation on opening day, Milestone CSMO Kenneth Hune Petersen pointed to the 2002 movie The Minority Report as highlighting a variety of gadgets and systems that seemed futuristic at the time but are now perfectly possible, and in some cases outdated. Movies have previously highlighted gadgets and systems that were futuristic, but are now perfectly possible, or outdated "If we dare to dream together we can make this a better world," says Petersen. "Through AI and machine learning, we can help define tomorrow. There's no doubt about it: There is a massive business opportunity for us in artificial intelligence." Despite all the talk about artificial intelligence, only about 0.5 percent of all the data in the world has currently been analyzed or used, says Peterson. "Our open platform technology is the foundation for intelligent video systems and our partners have the expertise and infrastructure needed to reach the next frontier in intelligent video solutions," said Bjørn Skou Eilertsen, Milestone Systems CTO. "Together, we can provide unlimited solutions for our customers." Deeper Integration And Broader Coverage Expanding the Milestone community this year has included the addition of 1,000 new models of supported hardware devices; there are currently more than 7,000 models supported. Milestone is also pursuing broader coverage of installations through their partners, with deeper integration of functionality, and by deepening existing relationships with customers. ‘Creating an intelligent world’ includes deep learning and lots of video systems, says Milestone at their annual conference Under new agreements, hardware partners such as Dell EMC and BCDVideo now provide XProtect Essential+ software pre-loaded on servers they sell. The focus at MIPS 2018 on AI included a presentation by Tanmay Bakshi, the "world's youngest IBM coder" and TED Talk speaker, at 14 years old. The prodigy, who has been coding since the age of 4, has worked with IBM and other companies on a variety of AI-related projects. Using deep learning with video is currently limited because so much video is unlabelled and unstructured In his MIPS 2018 keynote speech, Bakshi traced the development of AI through high-profile events, such as IBM's development of the "Watson" computer, which successfully competed on Jeopardy!, and Google's development of AlphaGo, a program that successfully plays the complex ancient board game, Go. Data Demands Deep-Learning Bakshi focused on security and healthcare as two disciplines where deep learning can potentially have a big impact. Using deep learning with video is currently limited because so much video is unlabeled and unstructured. Still, projections are that there will be a billion cameras worldwide by 2020, providing an over-abundance of data that demands the use of deep learning to make sense of it all. "There is a misconception that AI is meant to replace us, to make humans obsolete. AI is not replacing us. It is created by humans to amplify human skills. AI can reduce information overload to enable humans to work with the data more efficiently," said Bakshi. He suggested that AI is equivalent to IA; Bakshi's abbreviation meaning "intelligence augmented." Johnmichael O’Hare (left) of the Hartford Police Department, and Tanmay Bakshi (right) discusses key security issues of the modern day The ability to scale AI applications using "distributed deep learning" and graphics processing unit (GPU) hardware is paving the way for greater use of deep learning in video applications. Adam Scraba, Global Business Development Lead at NVIDIA, outlined the trends that are making the current "Big Bang" of deep learning possible. He said it is "the most exciting time in tech history," with "software that can write its own software" now among the tools that make previously unsolvable problems now solvable. AI-driven intelligent video analytics can now achieve "super-human" results, he said. An Intelligent World To Combat Crime Instead of sitting for hours staking out a suspected drug dealer alone, entire investigations now take hours instead of days A success story about the game-changing capabilities of video data was supplied by Hartford, Conn.'s Capital City Crime Center (C4). The Hartford police department uses video data in a "predictive policing" approach. They have created an "intelligent world with smart policing to combat drug trafficking," according to C4 Supervisor Johnmichael O'Hare of the Hartford Police Department. Instead of sitting for hours staking out a suspected drug dealer, for example, video of a site can be analyzed to determine areas with higher levels of foot traffic that indicate drug buys. The result is investigations that take hours instead of days. Hartford incorporates several technologies, including ShotSpotter gunshot detection, Briefcam video synopsis and other systems, all tied together using the Milestone platform. More than 700 attendees make MIPS 2018 the largest such event ever, and exhibits by around 60 Milestone partner companies attest to the continuing expansion of the Milestone community. [Main image:Tanmay Bakshi (left) and Johnmichael O’Hare of the Hartford Police Department (right) discuss key security issues of the modern day]

Has The Gap Closed Between Security Fiction And Security Reality?
Has The Gap Closed Between Security Fiction And Security Reality?

Among its many uses and benefits, technology is a handy tool in the fantasy world of movie and television thrillers. We all know the scene: a vital plot point depends on having just the right super-duper gadget to locate a suspect or to get past a locked door. In movies and TV, face recognition is more a super power than a technical function. Video footage can be magically enhanced to provide a perfect image of a license plate number. We have all shaken our heads in disbelief, and yet, our industry’s technical capabilities are improving every day. Are we approaching a day when the “enhanced” view of technology in movies and TV is closer to the truth? We asked this week’s Expert Panel Roundtable: How much has the gap closed between the reality of security system capabilities and what you see on TV (or at the movies)?

How Moving To Security As A Service Benefits Both Providers And End Users
How Moving To Security As A Service Benefits Both Providers And End Users

The way we purchase services and products is changing. The traditional concept of buying and owning a product is giving way to the idea that it is possible to purchase the services it offers instead. This approach has come from the consumer realisation that it is the outcome that is important rather than the tools to achieve it. For example, this approach is evident with the rise of music streaming services as opposed to downloads or physical products.   With the physical security industry becoming ever more integrated – and truly open systems now a reality – there is every reason to assume this service-lead trend will come to dominate the way our industry interacts with its clients as well. Interest In Service-Based Security There is a significant change of mindset that the security industry needs to embrace before a large-scale move to Security as a Service can take place. Like many technology sectors in the past, security providers have focussed on ‘shifting boxes’ as their definitive sales model. This approach was especially prevalent when proprietary systems were the mainstay of the security industry. Essentially, if the customer wanted more services they simply bought a new product. This was a straightforward and economic sales approach for manufacturers and installers alike.The security industry needs to embrace a change of mindset before a move to SaaS can take place The flexibility of integrated and open technology has changed the way consumers view their purchase, so it shouldn’t be any surprise that there is increased interest in a service-based approach. Customer choice equates to a change of focus and interest, with physical products being eclipsed by the benefits of the overall solution. We have already seen these changes in other technology areas, notably with smart devices and general IT systems. Cloud-based services put the onus on the result rather than which device the user chooses. This approach is even starting to manifest in areas that couldn’t have been predicted in the past, such as the car industry for example. Consumers are focusing more on the overall costs and convenience of buying a car over the specific specification of the vehicle. Equally, urban dwellers don’t necessarily want the hassle and expense of owning and parking their own vehicle anymore. If you don’t use it every day, it can make more sense to rent a vehicle only when you travel beyond public transport. For these consumers the car has become a service item for a specific journey. Benefits For End Users At the heart of this approach is the simple equation that consumers have a need and suppliers need to provide the most cost-effective, and easiest, solution. At the same time, the security operator may not necessarily want to know (or care) what specification the system has, they just want it to perform the task as required.   By discussing with consumers, we can ensure we work even more closely with them to provide the expert support they need and deserve Most security buyers will identify the specific business needs and their budget to achieve this. This is where a service approach really comes into its own. Customers need expert advice on a solution for their requirements which takes away the stress of finding the right products/systems. In the past there was always a risk of purchasing an unsuitable solution, which could potentially be disastrous. The other issue was having to budget for a big capital expenditure for a large installation and then having to find further resources once an upgrade was due when systems went end of life. Most businesses find it far easier to pay a sensible monthly or annual fee that is predictable and can easily be budgeted for. A service model makes this far easier to achieve. Great Opportunities As well as the benefits for end users, there are considerable benefits for security providers too. Rather than simply ‘shifting boxes’ and enduring the inevitable sales peaks and toughs this creates; a service sales model allows manufacturers and installers to enjoy a more stable business model. You don’t have to win new business with every product, but rather sell ongoing services for a set period. Its highly likely that the whole security industry will start to take this approach over the next few years. Manufacturers are already well aware of this shift in customer expectations and are changing their approach to meet demands.There are major opportunities on offer in return for a change of perspective in the security industry With the service and leasing approach already firmly entrenched in other industries, this is well proven in a consumer market. The airline industry is a great example. Manufacturers understand that airlines need flexibility to upscale and downscale operations and therefore whole aircraft and even individual key components (such as engines or seating) can be leased as required. Using this approach, airlines can concentrate on what customers demand and not worry about the logistics of doing this. Manufacturers and leasing businesses provide assurances and guarantees of service time for aircraft and engines, taking care of the servicing and maintenance to ensure this delivery. This approach is just as well suited for the provision of security systems. Servicing The Future Security Market Undoubtedly there are major opportunities on offer in return for a change of perspective in the security industry. However, this will involve substantial changes in some quarters to ensure the business model is aligned with the market. Overall, the security industry needs to not only develop the right systems for the market, but also to deliver them in the right way as well. This will ensure we work even more closely with customers to provide the expert support they need and deserve.