Imagine a home surveillance camera monitoring an elderly parent and anticipating potential concerns while respecting their privacy. Imagine another camera predicting a home burglary based on suspicious behaviors, allowing time to notify the homeowner who can in turn notify the police before the event occurs—or an entire network of cameras working together to keep an eye on neighborhood safety.

Artificial Intelligence vision chips

A new gen of AI vision chips are pushing advanced capabilities such as behavior analysis and higher-level security

There's a new generation of artificial intelligence (AI) vision chips that are pushing advanced capabilities such as behavior analysis and higher-level security to the edge (directly on devices) for a customizable user experience—one that rivals the abilities of the consumer electronics devices we use every day.

Once considered nothing more than “the eyes” of a security system, home monitoring cameras of 2020 will leverage AI-vision processors for high-performance computer vision at low power consumption and affordable cost—at the edge—for greater privacy and ease of use as well as to enable behavior analysis for predictive and preemptive monitoring.

Advanced home monitoring cameras

With this shift, camera makers and home monitoring service providers alike will be able to develop new edge-based use cases for home monitoring and enable consumers to customize devices to meet their individual needs. The result will be increased user engagement with home monitoring devices—mirroring that of cellphones and smart watches and creating an overlap between the home monitoring and consumer electronics markets.

A quick step back reminds us that accomplishing these goals would have been cost prohibitive just a couple of years ago. Face recognition, behavior analysis, intelligent analytics, and decision-making at this level were extremely expensive to perform in the cloud. Additionally, the lag time associated with sending data to faraway servers for decoding and then processing made it impossible to achieve real-time results.

Cloud-based home security devices

The constraints of cloud processing certainly have not held the industry back, however. Home monitoring, a market just seven years young, has become a ubiquitous category of home security and home monitoring devices. Consumers can choose to install a single camera or doorbell that sends alerts to their phone, a family of devices and a monthly manufacturer’s plan, or a high-end professional monitoring solution.

While the majority of these devices do indeed rely on the cloud for processing, camera makers have been pushing for edge-based processing since around 2016. For them, the benefit has always been clear: the opportunity to perform intelligent analytics processing in real-time on the device. But until now, the balance between computer vision performance and power consumption was lacking and camera companies weren’t able to make the leap. So instead, they have focused on improving designs and the cloud-centric model has prevailed.

Hybrid security systems

Even with improvements, false alerts result in unnecessary notifications and video recording

Even with improvements, false alerts (like tree branches swaying in the wind or cats walking past a front door) result in unnecessary notifications and video recording— cameras remain active which, in the case of battery powered cameras, means using up valuable battery life.

Hybrid models do exist. Typically, they provide rudimentary motion detection on the camera itself and then send video to the cloud for decoding and analysis to suppress false alerts. Hybrids provide higher-level results for things like people and cars, but their approach comes at a cost for both the consumer and the manufacturer.

Advanced cloud analytics

Advanced cloud analytics are more expensive than newly possible edge-based alternatives, and consumers have to pay for subscriptions. In addition, because of processing delays and other issues, things like rain or lighting changes (or even bugs on the camera) can still trigger unnecessary alerts.

And the more alerts a user receives, the more they tend to ignore them—there are simply too many. In fact, it is estimated that users only pay attention to 5% of their notifications. This means that when a package is stolen or a car is burglarized, users often miss the real-time notification—only to find out about the incident after the fact. All of this will soon change with AI-based behavior analysis, predictive security, and real-time meaningful alerts.

Predictive monitoring while safeguarding user privacy

These days, consumers are putting more emphasis on privacy and have legitimate concerns about being recorded while in their homes. Soon, with AI advancements at the chip level, families will be able to select user apps that provide monitoring without the need to stream video to a company server, or they’ll have access to apps that record activity but obscure faces.

Devices will have the ability to only send alerts according to specific criteria. If, for example, an elderly parent being monitored seems particularly unsteady one day or seems especially inactive, an application could alert the responsible family member and suggest that they check in. By analyzing the elderly parent’s behavior, the application could also predict a potential fall and trigger an audio alert for the person and also the family.

AI-based behavior analysis

Ability to analyze massive amounts of data locally and identify trends is a key advantage of AI at the edge

The ability to analyze massive amounts of data locally and identify trends or perform searches is a key advantage of AI at the edge, for both individuals and neighborhoods. For example, an individual might be curious as to what animal is wreaking havoc in their backyard every night.

In this case, they could download a “small animal detector” app to their camera which would trigger an alert when a critter enters their yard. The animal could be scared off via an alarm and—armed with video proof—animal control would have useful data for setting a trap.

Edge cameras

A newly emerging category of “neighborhood watch” applications is already connecting neighbors for significantly improved monitoring and safety. As edge cameras become more commonplace, this category will become increasingly effective.

The idea is that if, for example, one neighbor captures a package thief, and then the entire network of neighbors will receive a notification and a synopsis video showing the theft. Or if, say, there is a rash of car break-ins and one neighbor captures video of a red sedan casing their home around the time of a recent incident, an AI vision-based camera could be queried for helpful information:

Residential monitoring and security

The camera could be asked for a summary of the dates and times that it has recorded that particular red car. A case could be made if incident times match those of the vehicle’s recent appearances in the neighborhood. Even better, if that particular red car was to reappear and seems (by AI behavior analysis) to be suspicious, alerts could be sent proactively to networked residents and police could be notified immediately.

Home monitoring in 2020 will bring positive change for users when it comes to monitoring and security, but it will also bring some fun. Consumers will, for example, be able to download apps that do things like monitor pet activity. They might query their device for a summary of their pet’s “unusual activity” and then use those clips to create cute, shareable videos. Who doesn’t love a video of a dog dragging a toilet paper roll around the house?

AI at the Edge for home access control

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring

Home access control via biometrics is one of many new edge-based use cases that will bring convenience to home monitoring, and it’s an application that is expected to take off soon. With smart biometrics, cameras will be able to recognize residents and then unlock their smart front door locks automatically if desired, eliminating the need for keys.

And if, for example, an unauthorized person tries to trick the system by presenting a photograph of a registered family member’s face, the camera could use “3D liveness detection” to spot the fake and deny access. With these and other advances, professional monitoring service providers will have the opportunity to bring a new generation of access control panels to market.

Leveraging computer vision and deep neural networks

Ultimately, what camera makers strive for is customer engagement and customer loyalty. These new use cases—thanks to AI at the edge—will make home monitoring devices more useful and more engaging to consumers. Leveraging computer vision and deep neural networks, new cameras will be able to filter out and block false alerts, predict incidents, and send real-time notifications only when there is something that the consumer is truly interested in seeing.

AI and computer vision at the edge will enable a new generation of cameras that provide not only a higher level of security but that will fundamentally change the way consumers rely on and interact with their home monitoring devices.

Share with LinkedIn Share with Twitter Share with Facebook Share with What's App Share with Facebook
Download PDF version Download PDF version

Author profile

William Xu Senior Product Marketing Manager, Ambarella, Inc.

In case you missed it

What You Need To Know About Open Source Intelligence (OSINT) For Emergency Preparedness?
What You Need To Know About Open Source Intelligence (OSINT) For Emergency Preparedness?

Have you ever stopped to consider the volume of new data created daily on social media? It’s staggering. Take Twitter, for instance. Approximately 500 million tweets are published every day, adding up to more than 200 billion posts per year. On Facebook, users upload an additional 350 million photos per day, and on YouTube, nearly 720,000 hours of new video content is added every 24 hours. While this overwhelming volume of information may be of no concern to your average social media user posting updates to keep up with family and friends, it’s of particular interest to corporate security and safety professionals who are increasingly using it to monitor current events and detect potential risks around their people and locations—all in real-time. Meet the fast-paced and oft-confusing world of open-source intelligence (OSINT). What is Open Source Intelligence (OSINT)? The U.S. Department of State defines OSINT as, “intelligence that is produced from publicly available information and is collected, exploited, and disseminated promptly to an appropriate audience to address a specific intelligence requirement.” The concept of monitoring and leveraging publicly available information sources for intelligence purposes dates back to the 1930s. The British Broadcast Corporation (BBC) was approached by the British government and asked to develop a new service that would capture and analyze print journalism from around the world. Monitoring and identifying potential threats Originally named the “Digest of Foreign Broadcast, the service (later renamed BBC Monitoring which still exists today) captured and analyzed nearly 1.25 million broadcast words every day to help British intelligence officials keep tabs on conversations taking place abroad and what foreign governments were saying to their constituents. OSINT encompasses any publicly accessible information that can be used to monitor and identify potential threats Today, OSINT broadly encompasses any publicly accessible information that can be used to monitor and identify potential threats and/or relevant events with the potential to impact safety or business operations. The potential of OSINT data is extraordinary. Not only can it enable security and safety teams to quickly identify pertinent information that may pose a material risk to their business or people, but it can also be captured by anyone with the right set of tools and training. OSINT for cybersecurity and physical threat detection Whether it be a significant weather event, supply chain disruptions, or a world health crisis few saw coming, the threats facing organizations continue to increase in size and scale. Luckily, OSINT has been able to accelerate how organizations detect, validate, and respond to these threats, and it has proved invaluable in reducing risk and informing decision-making – especially during emergencies. OSINT is typically shared in real-time, so once a situation is reported, security teams can then work on verifying critical details such as the location or time an incident occurred or provide the most up-to-date information about rapidly developing events on the ground. They can then continue to monitor online chatter about the crisis, increasing their situational awareness and speeding up their incident response times. OSINT applications OSINT can help detect when sensitive company information may have been accessed by hackers  Severe weather offers a good example of OSINT in action. Say an organization is located in the Great Plains. They could use OSINT from sources like the National Weather Service or National Oceanic and Atmospheric Administration (NOAA) to initiate emergency communications to employees about tornado warnings, high winds, or other dangerous conditions as they are reported. Another common use case for OSINT involves data breaches and cyber-attacks. OSINT can help detect when sensitive company information may have been accessed by hackers by monitoring dark web messaging boards and forums. In 2019, T-Cellphone suffered a data breach that affected more than a million customers, but it was able to quickly alert affected users after finding their personal data online. OSINT is a well-established field with countless applications. Unfortunately, in an ever-changing digital world, it’s not always enough to help organizations weather a crisis. Why OSINT alone isn’t enough? One of the core challenges with leveraging OSINT data, especially social media intelligence (SOCMINT), is that much of it is unstructured and spread across many disparate sources, making it difficult to sort through, manage, and organize. Consider the social media statistics above. Assuming a business wanted to monitor all conversations on Twitter to ensure all relevant information was captured, it would need to both capture and analyze 500 million individual posts every day. Assuming a trained analyst spent just three seconds analyzing each post, that would amount to 1.5 billion seconds of labor—equivalent to 416,666 hours—just to keep pace. While technology and filters can greatly reduce the burden and help organizations narrow the scope of their analysis, it’s easy to see how quickly human capital constraints can limit the utility of OSINT data—even for the largest companies. Challenges with OSINT OSINT data collection includes both passive and active techniques, each requiring a different level of effort and skill Additionally, collecting OSINT data is time-consuming and resource-intensive. Making sense of it remains a highly specialized skill set requiring years of training. In an emergency where every second count, the time required to sift through copious amounts of information takes far longer than the time in which an organization must take meaningful action to alter the outcome. Compounding the issue, OSINT data is noisy and difficult to filter. Even trained analysts find the need to constantly monitor, search, and filter voluminous troves of unstructured data tedious. Artificial intelligence and machine learning have helped weed through some of this data faster, but for organizations with multiple locations tasked with monitoring hundreds or thousands of employees, it’s still a challenging task. Adding to the complexity, collecting OSINT data isn’t easy. OSINT data collection includes both passive and active techniques, each requiring a different level of effort and skill. Passive vs Active OSINT Passive OSINT is typically anonymous and meant to avoid drawing attention to the person requesting the information. Scrolling user posts on public social media profiles is a good example of passive OSINT. Active OSINT refers to information proactively sought out, but it often requires a more purposeful effort to retrieve it. That may mean specific login details are needed to access a website where information is stored. Lastly, unverified OSINT data can’t always be trusted. Analysts often encounter false positives or fake reports, which not only take time to confirm accuracy, but if they act on misinformation, the result could be damage to their organization’s reputation or worse. So, how can companies take advantage of it without staffing an army of analysts or creating operational headaches? A new path for OSINT Organisations can leverage the benefits of OSINT to improve situational awareness and aid decision-making Fortunately, organizations can leverage the benefits of OSINT to improve situational awareness and aid decision-making without hiring a dedicated team of analysts to comb through the data. By combining OSINT data with third-party threat intelligence solutions, organizations can get a cleaner, more actionable view of what’s happening in the world. Threat intelligence solutions not only offer speed by monitoring for only the most relevant events 24/7/365, but they also offer more comprehensive coverage of a wide range of threat types. What’s more, the data is often verified and married with location intelligence to help organizations better understand if, how, and to what extent each threat poses a risk to their people, facilities, and assets. In a world with a never-ending stream of information available, learning how to parse and interpret it becomes all the more important. OSINT is a necessary piece to any organization’s threat intelligence and monitoring system, but it can’t be the only solution. Paired with external threat intelligence tools, OSINT can help reduce risk and keep employees safe during emergencies and critical events.

Baltimore Is The Latest U.S. City To Target Facial Recognition Technology
Baltimore Is The Latest U.S. City To Target Facial Recognition Technology

The city of Baltimore has banned the use of facial recognition systems by residents, businesses and the city government (except for police). The criminalization in a major U.S. city of an important emerging technology in the physical security industry is an extreme example of the continuing backlash against facial recognition throughout the United States. Facial recognition technology ban Several localities – from Portland, Oregon, to San Francisco, from Oakland, California, to Boston – have moved to limit use of the technology, and privacy groups have even proposed a national moratorium on use of facial recognition. The physical security industry, led by the Security Industry Association (SIA), vigorously opposed the ban in Baltimore, urging a measured approach and ‘more rational policymaking’ that preserve the technology’s value while managing any privacy or other concerns. Physical security industry opposes ban In such cases, it is local businesses and residents who stand to lose the most" “Unfortunately, an outright ban on facial recognition continues a distressing pattern in which the clear value of this technology is ignored,” said SIA’s Chief Executive Officer (CEO) Don Erickson, adding “In such cases, it is local businesses and residents who stand to lose the most.” At the national level, a letter to US President Biden from the U.S. Chamber of Commerce Coalition asserts the need for a national dialog over the appropriate use of facial recognition technology and expresses concern about ‘a blanket moratorium on federal government use and procurement of the technology’. (The coalition includes Security Industry Association (SIA) and other industry groups.) The negativity comes at a peak moment for facial recognition and other biometric technologies, which saw an increase of interest for a variety of public and business applications, during the COVID-19 pandemic’s prioritization to improve public health hygiene and to promote ‘contactless’ technologies. Prohibition on banks, retailers and online sellers The ordinance in Baltimore prohibits banks from using facial recognition to enhance consumer security in financial transactions. It prevents retailers from accelerating checkout lines with contactless payment and prohibits remote online identity document verification, which is needed by online sellers or gig economy workers, according to the Security Industry Association (SIA). At a human level, SIA points out that the prohibition of facial recognition undermines the use of customized accessibility tools for disabled persons, including those suffering with blindness, memory loss or prosopagnosia (face blindness). Ban out of line with current state of facial recognition Addressing the Baltimore prohibition, the Information Technology and Innovation Foundation reacted to the measure as ‘shockingly out of line with the current state of facial recognition technology and its growing adoption in many sectors of the economy’. Before Baltimore’s decision to target facial recognition, Portland, Oregon, had perhaps the strictest ban, prohibiting city government agencies and private businesses from using the technology on the city’s grounds. San Francisco was the first U.S. city to ban the technology, with Boston, Oakland; Cambridge, Massachusetts; and Berkeley, California, among others, following suit. Police and federal units can use biometrics Unlike other bans, the Baltimore moratorium does not apply to police uses Unlike other bans, the Baltimore moratorium does not apply to police uses, but targets private uses of the technology. It also includes a one-year ‘sunset’ clause that requires city council approval for an extension. The measure carves out an exemption for use of biometrics in access control systems. However, violations of the measure are punishable by 12 months in jail. The law also establishes a task force to evaluate the cost and effectiveness of surveillance tools. Transparency in public sector use of facial recognition Currently, the state of Maryland controls the Baltimore Police Department, so the city council does not have authority to ban police use of facial recognition, which has been a human rights concern driving the bans in other jurisdictions. A measure to return local control of police to the city could pass before the year lapses. SIA advocates transparency in public-sector applications of facial recognition in identity verification, security and law enforcement investigative applications. SIA’s CEO, Don Erickson stated, “As public sector uses are more likely to be part of processes with consequential outcomes, it is especially important for transparency and sound policies to accompany government applications.”

What Are The Security Challenges Of Protecting Critical Infrastructure?
What Are The Security Challenges Of Protecting Critical Infrastructure?

Many of us take critical infrastructure for granted in our everyday lives. We turn on a tap, flip a switch, push a button, and water, light, and heat are all readily available. But it is important to remember that computerized systems manage critical infrastructure facilities, making them vulnerable to cyber-attacks. The recent ransomware attack on the Colonial Pipeline is an example of the new types of threats. In addition, any number of physical attacks is also possibilities. We asked this week’s Expert Panel Roundtable: What are the security challenges of protecting critical infrastructure?