Imagine a home surveillance camera monitoring an elderly parent and anticipating potential concerns while respecting their privacy. Imagine another camera predicting a home burglary based on suspicious behaviors, allowing time to notify the homeowner who can in turn notify the police before the event occurs—or an entire network of cameras working together to keep an eye on neighborhood safety.

Artificial Intelligence vision chips

A new gen of AI vision chips are pushing advanced capabilities such as behavior analysis and higher-level security

There's a new generation of artificial intelligence (AI) vision chips that are pushing advanced capabilities such as behavior analysis and higher-level security to the edge (directly on devices) for a customizable user experience—one that rivals the abilities of the consumer electronics devices we use every day.

Once considered nothing more than “the eyes” of a security system, home monitoring cameras of 2020 will leverage AI-vision processors for high-performance computer vision at low power consumption and affordable cost—at the edge—for greater privacy and ease of use as well as to enable behavior analysis for predictive and preemptive monitoring.

Advanced home monitoring cameras

With this shift, camera makers and home monitoring service providers alike will be able to develop new edge-based use cases for home monitoring and enable consumers to customize devices to meet their individual needs. The result will be increased user engagement with home monitoring devices—mirroring that of cellphones and smart watches and creating an overlap between the home monitoring and consumer electronics markets.

A quick step back reminds us that accomplishing these goals would have been cost prohibitive just a couple of years ago. Face recognition, behavior analysis, intelligent analytics, and decision-making at this level were extremely expensive to perform in the cloud. Additionally, the lag time associated with sending data to faraway servers for decoding and then processing made it impossible to achieve real-time results.

Cloud-based home security devices

The constraints of cloud processing certainly have not held the industry back, however. Home monitoring, a market just seven years young, has become a ubiquitous category of home security and home monitoring devices. Consumers can choose to install a single camera or doorbell that sends alerts to their phone, a family of devices and a monthly manufacturer’s plan, or a high-end professional monitoring solution.

While the majority of these devices do indeed rely on the cloud for processing, camera makers have been pushing for edge-based processing since around 2016. For them, the benefit has always been clear: the opportunity to perform intelligent analytics processing in real-time on the device. But until now, the balance between computer vision performance and power consumption was lacking and camera companies weren’t able to make the leap. So instead, they have focused on improving designs and the cloud-centric model has prevailed.

Hybrid security systems

Even with improvements, false alerts result in unnecessary notifications and video recording

Even with improvements, false alerts (like tree branches swaying in the wind or cats walking past a front door) result in unnecessary notifications and video recording— cameras remain active which, in the case of battery powered cameras, means using up valuable battery life.

Hybrid models do exist. Typically, they provide rudimentary motion detection on the camera itself and then send video to the cloud for decoding and analysis to suppress false alerts. Hybrids provide higher-level results for things like people and cars, but their approach comes at a cost for both the consumer and the manufacturer.

Advanced cloud analytics

Advanced cloud analytics are more expensive than newly possible edge-based alternatives, and consumers have to pay for subscriptions. In addition, because of processing delays and other issues, things like rain or lighting changes (or even bugs on the camera) can still trigger unnecessary alerts.

And the more alerts a user receives, the more they tend to ignore them—there are simply too many. In fact, it is estimated that users only pay attention to 5% of their notifications. This means that when a package is stolen or a car is burglarized, users often miss the real-time notification—only to find out about the incident after the fact. All of this will soon change with AI-based behavior analysis, predictive security, and real-time meaningful alerts.

Predictive monitoring while safeguarding user privacy

These days, consumers are putting more emphasis on privacy and have legitimate concerns about being recorded while in their homes. Soon, with AI advancements at the chip level, families will be able to select user apps that provide monitoring without the need to stream video to a company server, or they’ll have access to apps that record activity but obscure faces.

Devices will have the ability to only send alerts according to specific criteria. If, for example, an elderly parent being monitored seems particularly unsteady one day or seems especially inactive, an application could alert the responsible family member and suggest that they check in. By analyzing the elderly parent’s behavior, the application could also predict a potential fall and trigger an audio alert for the person and also the family.

AI-based behavior analysis

Ability to analyze massive amounts of data locally and identify trends is a key advantage of AI at the edge

The ability to analyze massive amounts of data locally and identify trends or perform searches is a key advantage of AI at the edge, for both individuals and neighborhoods. For example, an individual might be curious as to what animal is wreaking havoc in their backyard every night.

In this case, they could download a “small animal detector” app to their camera which would trigger an alert when a critter enters their yard. The animal could be scared off via an alarm and—armed with video proof—animal control would have useful data for setting a trap.

Edge cameras

A newly emerging category of “neighborhood watch” applications is already connecting neighbors for significantly improved monitoring and safety. As edge cameras become more commonplace, this category will become increasingly effective.

The idea is that if, for example, one neighbor captures a package thief, and then the entire network of neighbors will receive a notification and a synopsis video showing the theft. Or if, say, there is a rash of car break-ins and one neighbor captures video of a red sedan casing their home around the time of a recent incident, an AI vision-based camera could be queried for helpful information:

Residential monitoring and security

The camera could be asked for a summary of the dates and times that it has recorded that particular red car. A case could be made if incident times match those of the vehicle’s recent appearances in the neighborhood. Even better, if that particular red car was to reappear and seems (by AI behavior analysis) to be suspicious, alerts could be sent proactively to networked residents and police could be notified immediately.

Home monitoring in 2020 will bring positive change for users when it comes to monitoring and security, but it will also bring some fun. Consumers will, for example, be able to download apps that do things like monitor pet activity. They might query their device for a summary of their pet’s “unusual activity” and then use those clips to create cute, shareable videos. Who doesn’t love a video of a dog dragging a toilet paper roll around the house?

AI at the Edge for home access control

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring, and it’s an application that is expected to take off soon. With smart biometrics, cameras will be able to recognize residents and then unlock their smart front door locks automatically if desired, eliminating the need for keys.

And if, for example, an unauthorized person tries to trick the system by presenting a photograph of a registered family member’s face, the camera could use “3D liveness detection” to spot the fake and deny access. With these and other advances, professional monitoring service providers will have the opportunity to bring a new generation of access control panels to market.

Leveraging computer vision and deep neural networks

Ultimately, what camera makers strive for is customer engagement and customer loyalty. These new use cases—thanks to AI at the edge—will make home monitoring devices more useful and more engaging to consumers. Leveraging computer vision and deep neural networks, new cameras will be able to filter out and block false alerts, predict incidents, and send real-time notifications only when there is something that the consumer is truly interested in seeing.

AI and computer vision at the edge will enable a new generation of cameras that provide not only a higher level of security but that will fundamentally change the way consumers rely on and interact with their home monitoring devices.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version

Author profile

William Xu Senior Product Marketing Manager, Ambarella, Inc.

In case you missed it

Facial Recognition: Contactless Solutions For A Safe, Post-pandemic World
Facial Recognition: Contactless Solutions For A Safe, Post-pandemic World

Facial recognition technology has come a long way since it first came to market several years ago. Initially plagued with technical challenges and widely viewed as a futuristic solution, facial recognition is now firmly implanted in numerous consumer and business products and applications. New advancement in software, specifically in the areas of algorithms, neural networks and deep learning and/or artificial intelligence (AI), have all dramatically improved both the performance and accuracy of facial recognition, further expanding its use for an increasing number of applications. From a purely business perspective, facial recognition’s powerful identification and authentication capabilities make it ideal for two primary applications: first as a security tool, and second as a workforce management solution. The touchless, accurate credential solution Facial recognition readers meet the new emerging need to limit physical exposure to germs and viruses Even before the COVID-19 pandemic, the touchless nature of facial recognition as an access credential was gaining traction with physical and cyber security professionals. By using an individual’s face as an access control credential, facial recognition eliminates the need and expense of physical cards and proximity devices, or the need to physically enter PIN codes. In addition, facial recognition readers meet the new emerging need to limit physical exposure to germs and viruses by offering a highly accurate touchless access control credentialing solution. As a workforce management tool, facial recognition helps preserve the health of employees checking into work, while providing management with an infallible means of documenting employee time and attendance while providing a detailed history of overall workforce activity and individual personnel tracking. Both of which have been longstanding challenges due to easily compromised time tracking systems and practices. Now, nothing is left to question based on hard data. With the growing popularity of facial recognition technology, there are many choices already available with more undoubtedly on the way. Selecting the right solution for your specific access control and/or workforce management application is dependent on a very wide range of variables. But there are a few core characteristics that you should look for when evaluating facial recognition readers. Wide and near-angle LEDs Most facial recognition terminals employ some form of IR (Infrared) technology to help ensure high visibility by the unit’s image sensor. This often limits where the unit can be installed such as outdoors or near windows due to strong ambient light. More advanced facial recognition readers employ as many as 80 wide-angle near infrared LEDs and 60 narrow-angle near infrared LEDs, allowing the unit to recognize faces even in full daylight and brightly lit environments (not direct sun). This enables installation at indoor locations near windows, lobbies and building entries. 3D pixel intensity distribution analysis Another facial recognition reader advancement to look for involves three-dimensional pixel intensity analysis. Ambient lighting contains ultraviolet rays which can negate near infrared LED lighting, and can also cast shadows making it difficult for a facial recognition reader to pinpoint the facial recognition points required for identification and authentication. Three-dimensional pixel intensity distribution analysis minimizes the effects of ambient light when acquiring facial images by minimizing lighting contrasts. As a result, it is easier for the algorithm to recognize the shape of the face, enabling it to extract more facial features and create higher quality face templates, which are critical for accurate facial recognition. Functional ergonomics This results in a faster, more comfortable, and convenient user experience The angle and position of a facial recognition reader directly impact the performance of the unit. Facial recognition readers with different viewing angles for built-in visual and infrared cameras allows users to stand at positions that are most suitable for facial recognition with little or no effort of contortions. This results in a faster, more comfortable, and convenient user experience. High performance processing Like any intelligent edge device, the performance of a facial recognition solution is directly reliant on its processing power. New advanced facial recognition readers deliver exceptional performance by employing enhanced face template extraction technology combined with powerful processor. For example, a facial recognition reader with a 1.4 GHz quad-core processor can perform up to 3,000 facial database matches (1:N) within one second. More advanced solutions also feature Group Matching functionality capable of executing up to 30,000 matches within one second. Live face detection It is most important that the facial recognition readers you evaluate are capable of analyzing faces in real time to maintain fluid entry/egress even during high volumes of employee traffic. Hardware-dependent live face detection systems employing technologies such as facial thermogram recognition and facial vein recognition require expensive hardware components, provide less accurate matches and slower authentication performance, which is counterintuitive for mainstream access control and workforce management applications. Dual authentication for added security Although the use of an advanced facial recognition reader provides the convenience, health benefits and cost-savings of touchless identification and authentication, there are many applications where more than one credential may be necessary to ensure the highest levels of security. Advanced facial recognition readers with multimodal, multifactor credentialing capabilities provide this added security benefit. For example, facial recognition readers that support multiple RFID proximity devices supporting 125 kHz and 13.56 MHz provide varying degrees of protection and greater implementation versatility. Videophone or intercom capabilities Facial recognition readers with multifunctionality can solve several challenges with one solution Facial recognition readers with multifunctionality can solve several challenges with one solution. A perfect example includes devices with SIP (session initiation protocol) videophone capabilities which effectively eliminate the need and associated expense of  installing separate intercom devices while adding another layer of security to one’s facility. The COVID-19 pandemic, and hopefully soon to follow post-pandemic world, have surely accelerated the need for highly accurate, cost-efficient, and reliable facial recognition technologies to help get people back to work safely. Selecting the right facial recognition solution for your specific access control and/or workforce management is now more important than ever before, making a little extra due diligence during the evaluation process a smart decision.

Thermal Cameras and Smart Cities: Preventing COVID-19 in Public Places
Thermal Cameras and Smart Cities: Preventing COVID-19 in Public Places

With the pandemic still in full swing and no certainty as to when exactly it will come to an end, the world has been battling anxiety for months now. And with each day, circumstances change quickly and almost make it impossible to predict what will happen next, how events will unfold, and what actions to take in light of a new situation. But one thing is certain: the world has been shut down and paralyzed for way too long, and the eventual reopening is unavoidable – in fact, it’s well under way. In this situation, what is possible to control is how the world will continue reopening – and specifically, how to ensure the safest possible reopening that will ensure the return of some degree of normalcy to people’s lives and business operations, while also managing the risk of COVID’s spread in the most efficient way. Our highly digitised, technologically advanced world This is when the power of technology comes to rescue the day: what truly sets the global crisis we face today apart from other calamities that humanity has encountered over year is the fact that it has developed in a highly digitised, technologically advanced world where each day brings about innovations with a sole purpose to make daily life and operations easier and more streamlined. And among these, the star of the past decade has been artificial intelligence. The world has been shut down and paralyzed for way too long, and the eventual reopening is unavoidable – in fact, it’s well under way While AI has many avenues of introducing efficiency and fast problem-solving, there is one specific application that will further fuel the reopening of the world and successfully keep the spread of the virus abate. This “collaborative security” application includes a synthesis of smart video analytics, facial recognition, object identification/detection, and thermal cameras that can support the reopening of businesses globally when installed within those facilities frequented by customers. With such a level of sophistication that can ensure uninterrupted monitoring and analysis of large public spaces, these AI technologies can ideally operate best as cloud solutions to ensure a collaborative network with maximum scalability and widespread implementation. As these technologies increase in ubiquity and find their way into daily operations of businesses globally, the cost of the smart solutions will decrease proportionally to the growth of their reach. There are some highly specific ways to create this collaborative network of interconnected safety tools in the current climate. Here are some applications that have been successful to date and will increase in usability in the foreseeable future, creating “smart cities” working together towards a safer, more secure world. Maintaining social distancing practices The most important step everyone around the world has taken to contribute to the effort of slowing the spread of the virus has been social distancing. A six-foot-distance has become a new social norm that has quickly been adopted globally and become a habit to people who are naturally used to being close to others and socialising without giving distance a second thought. The star of the past decade has been artificial intelligence So, it is natural that such distancing measures take time to get accustomed to – and it is also natural that individuals may forget about them from time to time. To help maintain the six-foot distance between people at all times and give them slight nudges to keep the rule top of their minds, AI video technology can be trained to estimate the distance between individuals in public and commercial areas and identify the cases in which people get too close to each other. By notifying local merchants or authorities about such cases, the system can help ensure the safety of everyone in the area at all times while positively reinforcing the public to gradually get more accustomed to maintaining the distance and thus helping stop the spread of the virus. Detecting the virus through facial recognition Perhaps the straightforward application of such high-level technology is using video surveillance to identify persons of interest who have tested positive for the virus. Modern AI has the ability to identify facial features and characteristics with a unique level of granularity, making it possible to identify individuals whose records show they have antibodies from those who can be potential carriers of the virus. After the initial differentiation and identification, the system can then notify the employers and employees of the facility about the results of the conducted analysis and the pursuant results, allowing them to be more vigilant and take action where necessary to ensure a safe experience for everyone. PPE reinforcement Wearing a mask or some sort of face coverage in public spaces and especially within facilities (such as stores, for instance) has been - and will continue to be - a requirement for maintaining a safe and healthy environment for people to continue with their day-to-day lives and businesses to resume regular operations. To this extent, the object detection and identification abilities of smart cameras can further reinforce this requirement and ensure that the absence of protective equipment doesn’t go unnoticed.  Essentially, these cameras can easily identify if an individual has coverage at any given point of time or not, notifying the local authorities about any risks immediately and helping them maintain necessary safety measures without having to interrupt their workflow or worry about missing a visitor without a mask. Detecting high temperature One of the key (and the most widespread) symptoms of COVID-19 is a high fever - a certain indicator of whether an individual may have been infected with the virus or not. While identifying fever with a regular human eye is nearly impossible, AI can do so at a fraction of time by quickly scanning body temperatures of any incoming individuals and determine whether it’s above CDC’s recommended temperature of 100.4F in order to determine the risk factor and notify the local authorities to take action. Modern AI has the ability to identify facial features and characteristics with a unique level of granularity This technology is a good tactic to objectively assess potential risks that come with elevated temperatures - and sometimes, the people themselves might not realize they might (unconsciously) be carriers of the virus and thus endanger the safety of others in their vicinity. The technology is yet another step towards ensuring a safer reopening of the global economy and a more streamlined way of getting back on track while minimizing the risk of spreading the virus further. It’s not all about the theory  We have tested the described approaches in our own R&D campus in Europe. The latest release of the IREX cloud enables remote fever detection and monitoring of social isolation and mask policies with AI. We have integrated thermal cameras to detect people with elevated temperature and CCTV cameras for identification and notifying those who potentially ill. In case of any health threat, the venue manager gets an instant message with a picture and exact location. These preventive steps helped our employees return to the office months earlier than it's happening in other countries. Moreover, personnel coming back to the office by their own wish as now they feel a virus-free environment in the campus - even safer than in their own homes. Now we are launching a pilot project for a well-known pharmacy chain in Florida, USA. With the help of a Computer Vision platform, staff will be able to divide customer traffic into those with normal body temperature and those who come in with elevated temperatures, as well as effectively monitor social distance norms. The goal of our potential client is to maximize the safety of customers in the post-pandemic period. Also, IREX is already deployed across hundreds of locations in the UK and will add health monitoring capability soon.

Can CCTV Become A More Effective Tool?
Can CCTV Become A More Effective Tool?

We all know that having CCTV around your home can help to protect you and your family. Without CCTV, you could end up in danger and an intruder could get away with breaking into your house, hurting your loved ones and stealing your possessions. Similarly, without CCTV in the office, you’ll be leaving yourself open to all kinds of damage and could lose a lot of equipment in the process. In short, making sure you have CCTV is important for both home and business security. However, it can be improved to become a more effective system so that you’re better protected, and can even deter a potential intruder without having to panic. In our world of ever-changing technology, we’re able to upgrade and enhance our CCTV systems so they can become a monitored system. Remote CCTV monitoring is an ideal way to protect everyone and everything whether you’re at home or at a workplace. What is remote CCTV monitoring? For a long time, CCTV was one of the best ways to keep your home, the office and people safe. But people started to notice that it would only deter people so much of the time and often the cameras were ignored by intruders. They would just cover their faces and hope for the best as they steal from a home, office or any other premises. Remote CCTV monitoring is a system that can loop into your existing CCTV, or come preinstalled with a new system. This technology sends a feed to a control room full of trained operators that are on call 24/7. Within this control room, operators are able to respond to any sort of distress call or unauthorised movement on the property line.  How does it work? Remote CCTV monitoring works by attaching to a live feed of your CCTV system, existing or new, so that the signal and images can be passed to a team of operators. These operators are on hand 24/7 so that if there is a problem, you know that you're safe in the hands of a specialist team. The specialist team has been trained to mitigate the chances of somebody breaking into your home when the system is triggered; similarly they call the local authorities instantly so that the potential intruder has less time to flee the scene. This is especially important if an intruder is already inside your property because they have less time to steal your items and leave. Without CCTV in the office, you’ll be leaving yourself open to all kinds of damage and could lose a lot of equipment in the process One of the biggest questions that revolve around remote CCTV monitoring is the idea of operators watching the CCTV at all times. Luckily most remote CCTV monitoring systems will incorporate a motion detection system to accompany your CCTV. Motion detection offers the ability to alert a control room if there is an unauthorised entry to the property line. These motion detection systems are state-of-the-art and so, depending on the system that you choose, they can watch over your property from a number of angles. Once the motion detection system has been triggered and the alert has been sent to the control room, then and only then will the operators get involved. This means that until the motion detection system has been triggered, nobody will watch your live CCTV feed. After one of the systems has been triggered, one of the specialist operators will instantly jump into action. This means that they can take different measures to deter any potential intruder and make sure that they do everything in their power to stop any damage or theft from the premises. In addition to the motion detection system and CCTV, you are able to opt for a public address (PA) system too. This means that an operator is able to shout commands through the PA system and potentially scare away any intruder. The intruder will also be warned about the fact that local authorities have already been called to the location.  Remote monitoring versus traditional CCTV Motion detection offers the ability to alert a control room if there is an unauthorised entry to the property line Having a monitored CCTV system means that you're able to better protect yourself, your business, employees and even your loved ones. Whether you're at home or in the office, having someone looking over your shoulder protecting your every move is something that can be appreciated by everybody. The biggest problem with just having traditional CCTV, is that it is a reactive system. This means that rather than stopping crime, a CCTV system just records it. While the thought is that having a CCTV camera visible can deter some intruders, there's no real evidence to suggest that it stops anybody; anyone can simply cover their face and carry on breaking into your home, office or even your car. As mentioned, remote CCTV monitoring is going to tackle that problem and make sure that someone is on hand to protect you at all times.