Imagine a home surveillance camera monitoring an elderly parent and anticipating potential concerns while respecting their privacy. Imagine another camera predicting a home burglary based on suspicious behaviors, allowing time to notify the homeowner who can in turn notify the police before the event occurs—or an entire network of cameras working together to keep an eye on neighborhood safety.

Artificial Intelligence vision chips

A new gen of AI vision chips are pushing advanced capabilities such as behavior analysis and higher-level security

There's a new generation of artificial intelligence (AI) vision chips that are pushing advanced capabilities such as behavior analysis and higher-level security to the edge (directly on devices) for a customizable user experience—one that rivals the abilities of the consumer electronics devices we use every day.

Once considered nothing more than “the eyes” of a security system, home monitoring cameras of 2020 will leverage AI-vision processors for high-performance computer vision at low power consumption and affordable cost—at the edge—for greater privacy and ease of use as well as to enable behavior analysis for predictive and preemptive monitoring.

Advanced home monitoring cameras

With this shift, camera makers and home monitoring service providers alike will be able to develop new edge-based use cases for home monitoring and enable consumers to customize devices to meet their individual needs. The result will be increased user engagement with home monitoring devices—mirroring that of cellphones and smart watches and creating an overlap between the home monitoring and consumer electronics markets.

A quick step back reminds us that accomplishing these goals would have been cost prohibitive just a couple of years ago. Face recognition, behavior analysis, intelligent analytics, and decision-making at this level were extremely expensive to perform in the cloud. Additionally, the lag time associated with sending data to faraway servers for decoding and then processing made it impossible to achieve real-time results.

Cloud-based home security devices

The constraints of cloud processing certainly have not held the industry back, however. Home monitoring, a market just seven years young, has become a ubiquitous category of home security and home monitoring devices. Consumers can choose to install a single camera or doorbell that sends alerts to their phone, a family of devices and a monthly manufacturer’s plan, or a high-end professional monitoring solution.

While the majority of these devices do indeed rely on the cloud for processing, camera makers have been pushing for edge-based processing since around 2016. For them, the benefit has always been clear: the opportunity to perform intelligent analytics processing in real-time on the device. But until now, the balance between computer vision performance and power consumption was lacking and camera companies weren’t able to make the leap. So instead, they have focused on improving designs and the cloud-centric model has prevailed.

Hybrid security systems

Even with improvements, false alerts result in unnecessary notifications and video recording

Even with improvements, false alerts (like tree branches swaying in the wind or cats walking past a front door) result in unnecessary notifications and video recording— cameras remain active which, in the case of battery powered cameras, means using up valuable battery life.

Hybrid models do exist. Typically, they provide rudimentary motion detection on the camera itself and then send video to the cloud for decoding and analysis to suppress false alerts. Hybrids provide higher-level results for things like people and cars, but their approach comes at a cost for both the consumer and the manufacturer.

Advanced cloud analytics

Advanced cloud analytics are more expensive than newly possible edge-based alternatives, and consumers have to pay for subscriptions. In addition, because of processing delays and other issues, things like rain or lighting changes (or even bugs on the camera) can still trigger unnecessary alerts.

And the more alerts a user receives, the more they tend to ignore them—there are simply too many. In fact, it is estimated that users only pay attention to 5% of their notifications. This means that when a package is stolen or a car is burglarized, users often miss the real-time notification—only to find out about the incident after the fact. All of this will soon change with AI-based behavior analysis, predictive security, and real-time meaningful alerts.

Predictive monitoring while safeguarding user privacy

These days, consumers are putting more emphasis on privacy and have legitimate concerns about being recorded while in their homes. Soon, with AI advancements at the chip level, families will be able to select user apps that provide monitoring without the need to stream video to a company server, or they’ll have access to apps that record activity but obscure faces.

Devices will have the ability to only send alerts according to specific criteria. If, for example, an elderly parent being monitored seems particularly unsteady one day or seems especially inactive, an application could alert the responsible family member and suggest that they check in. By analyzing the elderly parent’s behavior, the application could also predict a potential fall and trigger an audio alert for the person and also the family.

AI-based behavior analysis

Ability to analyze massive amounts of data locally and identify trends is a key advantage of AI at the edge

The ability to analyze massive amounts of data locally and identify trends or perform searches is a key advantage of AI at the edge, for both individuals and neighborhoods. For example, an individual might be curious as to what animal is wreaking havoc in their backyard every night.

In this case, they could download a “small animal detector” app to their camera which would trigger an alert when a critter enters their yard. The animal could be scared off via an alarm and—armed with video proof—animal control would have useful data for setting a trap.

Edge cameras

A newly emerging category of “neighborhood watch” applications is already connecting neighbors for significantly improved monitoring and safety. As edge cameras become more commonplace, this category will become increasingly effective.

The idea is that if, for example, one neighbor captures a package thief, and then the entire network of neighbors will receive a notification and a synopsis video showing the theft. Or if, say, there is a rash of car break-ins and one neighbor captures video of a red sedan casing their home around the time of a recent incident, an AI vision-based camera could be queried for helpful information:

Residential monitoring and security

The camera could be asked for a summary of the dates and times that it has recorded that particular red car. A case could be made if incident times match those of the vehicle’s recent appearances in the neighborhood. Even better, if that particular red car was to reappear and seems (by AI behavior analysis) to be suspicious, alerts could be sent proactively to networked residents and police could be notified immediately.

Home monitoring in 2020 will bring positive change for users when it comes to monitoring and security, but it will also bring some fun. Consumers will, for example, be able to download apps that do things like monitor pet activity. They might query their device for a summary of their pet’s “unusual activity” and then use those clips to create cute, shareable videos. Who doesn’t love a video of a dog dragging a toilet paper roll around the house?

AI at the Edge for home access control

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring, and it’s an application that is expected to take off soon. With smart biometrics, cameras will be able to recognize residents and then unlock their smart front door locks automatically if desired, eliminating the need for keys.

And if, for example, an unauthorized person tries to trick the system by presenting a photograph of a registered family member’s face, the camera could use “3D liveness detection” to spot the fake and deny access. With these and other advances, professional monitoring service providers will have the opportunity to bring a new generation of access control panels to market.

Leveraging computer vision and deep neural networks

Ultimately, what camera makers strive for is customer engagement and customer loyalty. These new use cases—thanks to AI at the edge—will make home monitoring devices more useful and more engaging to consumers. Leveraging computer vision and deep neural networks, new cameras will be able to filter out and block false alerts, predict incidents, and send real-time notifications only when there is something that the consumer is truly interested in seeing.

AI and computer vision at the edge will enable a new generation of cameras that provide not only a higher level of security but that will fundamentally change the way consumers rely on and interact with their home monitoring devices.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

William Xu Senior Product Marketing Manager, Ambarella, Inc.

In case you missed it

Safety In Smart Cities: How Video Surveillance Keeps Security Front And Center
Safety In Smart Cities: How Video Surveillance Keeps Security Front And Center

Urban populations are expanding rapidly around the globe, with an expected growth of 1.56 billion by 2040. As the number of people living and working in cities continues to grow, the ability to keep everyone safe is an increasing challenge. However, technology companies are developing products and solutions with these futuristic cities in mind, as the reality is closer than you may think. Solutions that can help to watch over public places and share data insights with city workers and officials are increasingly enabling smart cities to improve the experience and safety of the people who reside there. Rising scope of 5G, AI, IoT and the Cloud The main foundations that underpin smart cities are 5G, Artificial Intelligence (AI), and the Internet of Things (IoT) and the Cloud. Each is equally important, and together, these technologies enable city officials to gather and analyze more detailed insights than ever before. For public safety in particular, having IoT and cloud systems in place will be one of the biggest factors to improving the quality of life for citizens. Smart cities have come a long way in the last few decades, but to truly make a smart city safe, real-time situational awareness and cross-agency collaboration are key areas which must be developed as a priority. Innovative surveillance cameras with integrated IoT Public places need to be safe, whether that is an open park, shopping center, or the main roads through towns Public places need to be safe, whether that is an open park, shopping center, or the main roads through towns. From dangerous drivers to terrorist attacks, petty crime on the streets to high profile bank robberies, innovative surveillance cameras with integrated IoT and cloud technologies can go some way to helping respond quickly to, and in some cases even prevent, the most serious incidents. Many existing safety systems in cities rely on aging and in some places legacy technology, such as video surveillance cameras. Many of these also use on-premises systems rather than utilising the benefits of the cloud. Smart programming to deliver greater insights These issues, though not creating a major problem today, do make it more challenging for governments and councils to update their security. Changing every camera in a city is a huge undertaking, but in turn, doing so would enable all cameras to be connected to the cloud, and provide more detailed information which can be analyzed by smart programming to deliver greater insights. The physical technologies that are currently present in most urban areas lack the intelligent connectivity, interoperability and integration interfaces that smart cities need. Adopting digital technologies isn’t a luxury, but a necessity. Smart surveillance systems It enables teams to gather data from multiple sources throughout the city in real-time, and be alerted to incidents as soon as they occur. Increased connectivity and collaboration ensures that all teams that need to be aware of a situation are informed instantly. For example, a smart surveillance system can identify when a road accident has occurred. It can not only alert the nearest ambulance to attend the scene, but also the local police force to dispatch officers. An advanced system that can implement road diversions could also close roads around the incident immediately and divert traffic to other routes, keeping everyone moving and avoiding a build-up of vehicles. This is just one example: without digital systems, analyzing patterns of vehicle movements to address congestion issues could be compromised, as would the ability to build real-time crime maps and deploy data analytics which make predictive policing and more effective crowd management possible. Cloud-based technologies Cloud-based technologies provide the interoperability, scalability and automation Cloud-based technologies provide the interoperability, scalability and automation that is needed to overcome the limitations of traditional security systems. Using these, smart cities can develop a fully open systems architecture that delivers interoperation with both local and other remote open systems. The intelligence of cloud systems can not only continue to allow for greater insights as technology develops over time, but it can do so with minimal additional infrastructure investment. Smart surveillance in the real world Mexico City has a population of almost 9 million people, but if you include the whole metropolitan area, this number rises sharply to over 21 million in total, making it one of the largest cities on the planet. Seven years ago, the city first introduced its Safe City initiative, and ever since has been developing newer and smarter ways to keep its citizens safe. In particular, its cloud-based security initiative is making a huge impact. Over the past three years, Mexico City has installed 58,000 new video surveillance cameras throughout the city, in public spaces and on transport, all of which are connected to the City’s C5 (Command, Control, Computers, Communications and Citizen Contact) facility. Smart Cities operations The solution enables officers as well as the general public to upload videos via a mobile app to share information quickly, fixed, body-worn and vehicle cameras can also be integrated to provide exceptional insight into the city’s operations. The cloud-based platform can easily be upgraded to include the latest technology innovations such as license plate reading, behavioral analysis software, video analytics and facial recognition software, which will all continue to bring down crime rates and boost response times to incidents. The right cloud approach Making the shift to cloud-based systems enables smart cities to eliminate dependence on fiber-optic connectivity and take advantage of a variety of Internet and wireless connectivity options that can significantly reduce application and communication infrastructure costs. Smart cities need to be effective in years to come, not just in the present day, or else officials have missed one of the key aspects of a truly smart city. System designers must build technology foundations now that can be easily adapted in the future to support new infrastructure as it becomes available. Open system architecture An open system architecture will also be vital for smart cities to enhance their operations For example, this could include opting for a true cloud application that can support cloud-managed local devices and automate their management. An open system architecture will also be vital for smart cities to enhance their operations and deliver additional value-add services to citizens as greater capabilities become possible in the years to come. The advances today in cloud and IoT technologies are rapid, and city officials and authorities have more options now to develop their smart cities than ever before and crucially, to use these innovations to improve public safety. New safety features Though implementing these cloud-based systems now requires investment, as new safety features are designed, there will be lower costs and challenges associated with introducing these because the basic infrastructure will already exist. Whether that’s gunshot detection or enabling the sharing of video infrastructure and data across multiple agencies in real time, smart video surveillance on cloud-based systems can bring a wealth of the new opportunities.

What Will Be The Security Industry’s Biggest Challenge In 2021?
What Will Be The Security Industry’s Biggest Challenge In 2021?

What a year 2020 was for the security industry! There were vast challenges that could not have been foreseen at the beginning of the year. It is safe to say that the events of 2020 defied all industry prognosticators. However, is that any reason not to hope our expectations looking ahead to 2021 will be much closer to reality? Can we possibly benefit from looking ahead and trying to plan for the challenges of the year ahead? We asked this week’s Expert Panel Roundtable: What will be the security industry’s biggest challenge in 2021?

Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool
Looking Back At 2020: AI Transitions From Buzzword To A Vital Tool

Artificial intelligence (AI) is more than a buzzword. AI is increasingly becoming part of our everyday lives, and a vital tool in the physical security industry. In 2020, AI received more attention than ever, and expanded the ways it can contribute value to physical security systems. This article will revisit some of those development at year-end, including links back to the originally published content. In the security market today, AI is expanding the use cases, making technologies more powerful and saving money on manpower costs - and today represents just the beginning of what AI can do for the industry. What it will never do, however, is completely take the place of humans in operating security systems. There is a limit to how much we are willing to turn over to machines - even the smartest ones. Beyond video analytics "Apply AI to security and now you have an incredibly powerful tool that allows you to operate proactively rather than reactively," said Jody Ross of AMAG Technology, one of our Expert Roundtable Panelists. AI made its initial splash in the physical security market by transforming the effectiveness of video analytics AI made its initial splash in the physical security market by transforming the effectiveness of video analytics. However, now there are many other applications, too, as addressed by our Expert Panel Roundtable in another article. Artificial intelligence (AI) and machine learning provide useful tools to make sense of massive amounts of Internet of Things (IoT) data. By helping to automate low-level decision-making, the technologies can make security operators more efficient. Biometrics with access control Intelligent capabilities can expand integration options such as increasing the use of biometrics with access control. AI can also help to monitor mechanics and processes. Intelligent systems can help end users understand building occupancy and traffic patterns and even to help enforce physical distancing. These are just a few of the possible uses of the technologies - in the end, the sky is the limit. AI is undoubtedly one of the bigger disrupters in the physical security industry, and adoption is growing at a rapid rate. And it’s not just about video analytics. Rather, it is data AI, which is completely untapped by the security industry. Bottom line: AI can change up your security game by automatically deciphering information to predict the future using a wide range of sources and data that have been collected, whether past, present, and future. That’s right. You can look into the future. Smarter perimeter protection Now, Intrusion Detection (Perimeter Protection) systems with cutting-edge, built-in AI algorithms to recognize a plethora of different object types, can distinguish objects of interest, thus significantly decreasing the false-positive intrusion rate. The more advanced AI-based systems enable the users to draw ROIs based on break-in points, areas of high-valuables, and any other preference to where alerts may be beneficial. AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store Similarly, AI Loitering Detection can be used to receive alerts on suspicious activity outside any given store. The loitering time and region of interest are customizable in particular systems, which allows for a range of detection options. Smart security is advancing rapidly. As AI and 4K rise in adoption on smart video cameras, these higher video resolutions are driving the demand for more data to be stored on-camera. AI and smart video promise to extract greater insights from security video. Meeting urban needs Complex, extensive camera networks will already require a large amount of data storage, particularly if this is 24/7 monitoring from smart video-enabled devices. Newer edge computing will play an important role in capturing, collecting, and analyzing data. There are many more types of cameras being used today, such as body cameras, dashboard cameras, and new Internet of Things (IoT) devices and sensors. Video data is so rich nowadays, you can analyze it and deduce a lot of valuable information in real-time, instead of post-event. In smart cities applications, the challenge of identifying both physical and invisible threats to meet urban citizens’ needs will demand a security response that is proactive, adaptable and dynamic. Optimize security solutions As we look ahead to the future of public safety, it’s clear that new technologies, driven by artificial intelligence (AI), can dramatically improve the effectiveness of today’s physical security space. For smart cities, the use of innovative AI and machine learning technologies have already started to help optimize security solutions. In sports stadium applications, AI’s role in getting fans and spectators back after the COVID pandemic is huge, through capabilities such as social distance monitoring, crowd scanning/metrics, facial recognition, fever detection, track and trace and providing behavioral analytics. Technologies such as AI-powered collaboration platforms now work alongside National Leagues, Franchises and Governing Bodies to implement AI surveillance software into their CCTV/surveillance cameras. AI surveillance software In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI This is now creating a more collaborative effort from the operations team in stadiums, rather than purely security. AI surveillance software, when implemented into the surveillance cameras can be accessed by designated users on any device and on any browser platform. One of the biggest advantages of using AI technology is that it’s possible to integrate this intelligent software into building smarter, safer communities and cities. Essentially, this means developing a layered system that connects multiple sensors for the detection of visible and invisible threats. Integrated systems mean that threats can be detected and tracked, with onsite and law enforcement notified faster, and possibly before an assault begins to take place. In many ways, it’s the equivalent of a neighborhood watch program made far more intelligent through the use of AI. Fighting illicit trade Using technology in this way means that thousands of people can be screened seamlessly and quickly, without invading their civil liberties or privacy. AI’s ability to detect visible or invisible threats or behavioral anomalies will prove enormously valuable to many sectors across our global economy. Revolutionary AI-driven technologies can help to fight illicit trade across markets. AI technologies in this specific application promise to help build safer and more secure communities in the future. AI can support the ongoing fight against illicit trade on a global scale in a tangible way. For financial transactions at risk of fraud and money laundering, for example, tracking has become an increasing headache if done manually. As a solution to this labor-intensive process, AI technology can be trained to follow all the compliance rules and process a large number of documents - often billions of pages of documents - in a short period of time.