A new generation of video cameras is poised to boost capabilities dramatically at the edge of the IP network, including more powerful artificial intelligence (AI) and higher resolutions, and paving the way for new applications that would have previously been too expensive or complex.

Technologies at the heart of the coming new generation of video cameras are Ambarella’s newest systems on chips (SoCs). Ambarella’s CV5S and CV52S product families are bringing a new level of on-camera AI performance and integration to multi-imager and single-imager IP cameras. Both of these SoCs are manufactured in the ‘5 nm’ manufacturing process, bringing performance improvements and power savings, compared to the previous generation of SoCs manufactured at ‘10nm’.

CV5S and CV52S AI-powered SoCs

The CV5S, designed for multi-imager cameras, is able to process, encode and perform advanced AI on up to four imagers at 4Kp30 resolution, simultaneously and at less than 5 watts. This enables multi-headed camera designs with up to four 4K imagers looking at different portions of a scene, as well as very high-resolution, single-imager cameras of up to 32 MP resolution and beyond.

The CV52S, designed for single-imager cameras with very powerful onboard AI, is the next-generation of the company’s successful CV22S mainstream 4K camera AI chip. This new SoC family quadruples the AI processing performance, while keeping the same low power consumption of less than 3 watts for 4Kp60 encoding with advanced AI processing.

Faster and ubiquitous AI capabilities

Ambarella’s newest AI vision SoCs for security, the CV5S and CV52S, are competitive solutions"

Security system designers desire higher resolutions, increasing channel counts, and ever faster and more ubiquitous AI capabilities,” explains John Lorenz, Senior Technology and Market Analyst, Computing, at Yole Développement (Yole), a French market research firm.

John Lorenz adds, “Ambarella’s newest AI vision SoCs for security, the CV5S and CV52S, are competitive solutions for meeting the growing demands of the security IC (integrated circuit) sector, which our latest report forecasts to exceed US$ 4 billion by 2025, with two-thirds of that being chips with AI capabilities.

Edge AI vision processors

Ambarella’s new CV5S and CV52S edge AI vision processors enable new classes of cameras that would not have been possible in the past, with a single SoC architecture. For example, implementing a 4x 4K multi-imager with AI would have traditionally required at least two SoCs (at least one for encoding and one for AI), and the overall power consumption would have made those designs bulky and prohibitively expensive.

By reducing the number of required SoCs, the CV5S enables advanced camera designs such as AI-enabled 4x 4K imagers at price points much lower than would have previously been possible. “What we are usually trying to do with our SoCs is to keep the price points similar to the previous generations, given that camera retail prices tend to be fairly fixed,” said Jerome Gigot, Ambarella's Senior Director of Marketing.

4K multi-imager cameras

However, higher-end 4K multi-imager cameras tend to retail for thousands of dollars, and so even though there will be a small premium on the SoC for the 2X improvement in performance, this will not make a significant impact to the final MSRP of the camera,” adds Jerome Gigot.

In addition, the overall system cost might go down, Gigot notes, compared to what could be built today because there is no longer a need for external chips to perform AI, or extra components for power dissipation.

The new chips will be available in the second half of 2021, and it typically takes about 12 to 18 months for Ambarella’s customers (camera manufacturers) to produce final cameras. Therefore, the first cameras, based on these new SoCs, should hit the market sometime in the second half of 2022.

Reference boards for camera manufacturers

The software on these new SoCs is an evolution of our unified Linux SDK"

As with Ambarella’s previous generations of edge AI vision SoCs for security, the company will make available reference boards to camera manufacturers soon, allowing them to develop their cameras based on the new CV5S and CV52S SoC families.

The software on these new SoCs is an evolution of our unified Linux SDK that is already available on our previous generations SoCs, which makes the transition easy for our customers,” said Jerome Gigot.

Better crime detection

Detecting criminals in a crowd, using face recognition and/or license plate recognition, has been a daunting challenge for security, and one the new chips will help to address.

Actually, these applications are one of the main reasons why Ambarella is introducing these two new SoC families,” said Jerome Gigot.

Typically, resolutions of 4K and higher have been a smaller portion of the security market, given that they came at a premium price tag for the high-end optics, image sensor and SoC. Also, the cost and extra bandwidth of storing and streaming 4K video were not always worth it for the benefit of just viewing video at higher resolution.

4K AI processing on-camera

The advent of on-camera AI at 4K changes the paradigm. By enabling 4K AI processing on-camera, smaller objects at longer distances can now be detected and analyzed without having to go to a server, and with much higher detail and accuracy compared to what can be done on a 2 MP or 5 MP cameras.

This means that fewer false alarms will be generated, and each camera will now be able to cover a longer distance and wider area, offering more meaningful insights without necessarily having to stream and store that 4K video to a back-end server. “This is valuable, for example, for traffic cameras mounted on top of high poles, which need to be able to see very far out and identify cars and license plates that are hundreds of meters away,” said Jerome Gigot.

Ambarella’s new CV5S and CV52S SoCs truly allow the industry to take advantage of higher resolution on-camera for better analytics and wider coverage.
The advent of on-camera AI at 4K changes the paradigm

Enhanced video analytics and wider coverage

Ambarella’s new CV5S and CV52S SoCs truly allow the industry to take advantage of higher resolution on-camera for better analytics and wider coverage, but without all the costs typically incurred by having to stream high-quality 4K video out 24/7 to a remote server for offline analytics,” said Jerome Gigot.

He adds, “So, next-generation cameras will now be able to identify more criminals, faces and license plates, at longer distances, for an overall lower cost and with faster response times by doing it all locally on-camera.

Deployment in retail applications

Retail environments can be some of the toughest, as the cameras may be looking at hundreds of people at once

Retail applications are another big selling point. Retail environments can be some of the toughest, as the cameras may be looking at hundreds of people at once (e.g., in a mall), to provide not only security features, but also other business analytics, such as foot traffic and occupancy maps that can be used later to improve product placement.

The higher resolution and higher AI performance, enabled by the new Ambarella SoCs, provide a leap forward in addressing those scenarios. In a store setup, a ceiling-mounted camera with four 4K imagers can simultaneously look at the cashier line on one side of the store, sending alerts when a line is getting too long and a new cashier needs to be deployed, while at the same time looking at the entrance on the other side of the store, to count the people coming in and out.

This leaves two additional 4K imagers for monitoring specific product aisles and generating real-time business analytics.

Use in cashier-less stores

Another retail application is a cashier-less store. Here, a CV5S or CV52S-based camera mounted on the ceiling will have enough resolution and AI performance to track goods, while the customer grabs them and puts them in their cart, as well as to automatically track which customer is purchasing which item.

In a warehouse scenario, items and boxes moving across the floor could also be followed locally, on a single ceiling-mounted camera that covers a wide area of the warehouse. Additionally, these items and boxes could be tracked across the different imagers in a multi-headed camera setup, without the video having to be sent to a server to perform the tracking.

Updating on-camera AI networks

Another feature of Ambarella’s SoCs is that their on-camera AI networks can be updated on-the-fly, without having to stop the video recording and without losing any video frames.

So, for example in the case of a search for a missing vehicle, the characteristics of that missing vehicle (make, model, color, license plate) can be sent to a cluster of cameras in the general area, where the vehicle is thought to be missing, and all those cameras can be automatically updated to run a live search on that specific vehicle.

If any of the cameras gets a match, a remote operator can be notified and receive a picture, or even a live video feed of the scene.

Efficient traffic management

With the CV52S edge AI vision SoC, those decisions can be made locally at each intersection by the camera itself

Relating to traffic congestion, most big cities have thousands of intersections that they need to monitor and manage. Trying to do this from one central location is costly and difficult, as there is so much video data to process and analyze, in order to make those traffic decisions (to control the traffic lights, reverse lanes, etc.).

With the CV52S edge AI vision SoC, those decisions can be made locally at each intersection by the camera itself. The camera would then take actions autonomously (for example, adjust traffic-light timing) and only report a status update to the main traffic control center. So now, instead of having one central location trying to manage 1,000 intersections, a city can have 1,000 smart AI cameras, each managing its own location and providing updates and metadata to a central server.

Superior privacy

Privacy is always a concern with video. In this case, doing AI on-camera is inherently more private than streaming the video to a server for analysis. Less data transmission means fewer points of entry for a hacker trying to access the video.

On Ambarella’s CV5S and CV52S SoCs, the video can be analyzed locally and then discarded, with just a signature or metadata of the face being used to find a match. No actual video needs to be stored or transmitted, which ensures total privacy.

In addition, the chips contain a very secure hardware cyber security block, including OTP memory, Arm TrustZones, DRAM scrambling and I/O virtualization. This makes it very difficult for a hacker to replace the firmware on the camera, providing another level of security and privacy at the system level.

Privacy Masking

Another privacy feature is the concept of privacy masking. This feature enables portions of the video (say a door or a window) to be blocked out, before being encoded in the video stream. The blocked portions of the scene are not present in the recorded video, thus providing a privacy option for cameras that are facing private areas.

With on-camera AI, each device becomes its own smart endpoint, and can be reconfigured at will to serve the specific physical security needs of its installation,” said Jerome Gigot, adding “The possibilities are endless, and our mission as an SoC maker is really to provide a powerful and easy-to-use platform, complete with computer-vision tools, that enable our customers and their partners to easily deploy their own AI software on-camera.

Physical security in parking lots

With a CV5S or CV52S AI-enabled camera, the camera will be able to cover a much wider portion of the parking lot

One example is physical security in a parking lot. A camera today might be used to just record part of the parking lot, so that an operator can go back and look at the video if a car were broken into or some other incident occurred.

With a CV5S or CV52S AI-enabled camera, first of all, the camera will be able to cover a much wider portion of the parking lot. Additionally, it will be able to detect the license plates of all the cars going in and out, to automatically bill the owners.

If there is a special event, the camera can be reprogrammed to identify VIP vehicles and automatically redirect them to the VIP portion of the lot, while reporting to the entrance station or sign how many parking spots are available. It can even tell the cars approaching the lot where to go.

Advantages of using edge AI vision SoCs

Jerome Gigot said, “The possibilities are endless and they span across many verticals. The market is primed to embrace these new capabilities. Recent advances in edge AI vision SoCs have brought about a period of change in the physical security space. Companies that would have, historically, only provided security cameras, are now getting into adjacent verticals such as smart retail, smart cities and smart buildings.”

He adds, “These changes are providing a great opportunity for all the camera makers and software providers to really differentiate themselves by providing full systems that offer a new level of insights and efficiencies to, not only the physical security manager, but now also the store owner and the building manager.

He adds, “All of these new applications are extremely healthy for the industry, as they are growing the available market for cameras, while also increasing their value and the economies of scale they can provide. Ambarella is looking forward to seeing all the innovative products that our customers will build with this new generation of SoCs.

Share with LinkedIn Share with Twitter Share with Facebook Share with Facebook
Download PDF version Download PDF version

Author profile

Larry Anderson Editor, SecurityInformed.com & SourceSecurity.com

An experienced journalist and long-time presence in the US security industry, Larry is SecurityInformed.com's eyes and ears in the fast-changing security marketplace, attending industry and corporate events, interviewing security leaders and contributing original editorial content to the site. He leads SecurityInformed's team of dedicated editorial and content professionals, guiding the "editorial roadmap" to ensure the site provides the most relevant content for security professionals.

In case you missed it

The Post-Pandemic Mandate For Entertainment Venues: Digitally Transform Security Guards
The Post-Pandemic Mandate For Entertainment Venues: Digitally Transform Security Guards

As the COVID-19 pandemic wanes and sporting venues open-up to full capacity, a new disturbing trend has hit the headlines - poor fan behavior. Five NBA teams have issued indefinite bans on fans, who crossed the line of unacceptable behavior, during the NBA playoffs. Major League Baseball stadiums have a recurring problem with divisive political banners being strewn over walls, as part of an organized campaign, requiring fan ejections. There was a brawl between Clippers and Suns fans after Game 1 of their playoff series. And, the U.S. vs. Mexico Nations League soccer game over the Fourth of July weekend had to be halted, due to fans throwing objects at players and screaming offensive chants. Cracking down on poor fan behavior Security directors are consistently reporting a disturbing uptick in poor fan attitude and behavior With players across all major sports leagues commanding more power than ever before, they are demanding that sports venues crack down on poor fan behavior, particularly when they are the targets of that behavior. Whether it’s an extension of the social-media divisiveness that’s gripped society, or people unleashing pent up negative energy, following 15 months of social isolation, during the COVID-19 global pandemic, security directors are consistently reporting a disturbing uptick in poor fan attitude and behavior. They’re also reporting a chronic security guard shortage, like many businesses that rely on relatively low-cost labor, finding candidates to fill open positions has been incredibly difficult. Low police morale To add the third component to this perfect storm, many police departments are struggling with morale issues and officers are less likely to put themselves into positions, where they could wind up in a viral video. According to the Police Executive Research Forum, police officer retirements in the U.S. were up 45% in the April 2020 - April 2021 period, when compared to the previous year. Resignations were up 18%. In this environment, officers may be less likely to undertake fan intervention unless it’s absolutely necessary. This can seem like the worst of times for venue security directors, as they need more staff to handle increasingly unruly patrons, but that staff simply isn’t available. And, because the security guard staffing industry is a commoditized business, companies compete almost solely on price, which requires that they keep salaries as low as possible, which perpetuates the lack of interest in people participating in the profession. Digital Transformation There is only one way out of this conundrum and that is to make security personnel more efficient and effective. Other industries have solved similar staffing and cost challenges through digital transformation. For example, only a small percentage of the total population of restaurants in the U.S. used to offer home delivery, due to cost and staffing challenges of hiring dedicated delivery personnel. Advent of digital efficiency tools But with the advent of digital efficiency tools, now virtually all restaurants can offer delivery But with the advent of digital efficiency tools, such as UberEATS and DoorDash, now virtually all restaurants can offer delivery. Likewise, field-service personnel are digitally connected, so when new jobs arise, they can be notified and routed to the location. Compare this to the old paper-based days, when they wouldn’t know about any new jobs until they picked up their work schedule at the office, the next day and you can see how digital transformation makes each worker significantly more efficient. Security guards and manned guarding The security guard business has never undergone this kind of digital transformation. The state-of-the-art ‘technology’ has never changed - human eyes and ears. Yes, there are video cameras all over stadiums and other venues, but behind the scenes is a guard staring at a bunch of monitors, hoping to identify incidents that need attention. Meanwhile, there are other guards stationed around the stadium, spending most of their time watching people who are doing nothing wrong. Think about all the wasted time involved with these activities – not to mention the relentless boredom and ‘alert fatigue’ from false-positive incident reporting and you understand the fundamental inefficiencies of this labor-based approach to security. Now think about a world where there’s ubiquitous video surveillance and guards are automatically and pre-emptively notified and briefed, when situations arise. The fundamental nature of the security guards profession changes. Instead of being low paid ‘watchers’, they instead become digitally-empowered preventers. AI-based screening and monitoring technology This world is happening today, through Artificial Intelligence-based screening and monitoring technology. AI-powered weapons-detection gateways inform guards, when a patron entering the venue is carrying a gun, knife or other forbidden item. Instead of patting down every patron with metal in their pockets, which has been the standard practise since walk-through metal detectors were mandated by sports leagues following 9/11, guards can now target only those who are carrying these specific items. Video surveillance and AI-based analytics integration Combining surveillance video with AI-based advanced analytics can automatically identify fan disturbances Combining surveillance video with AI-based advanced analytics can automatically identify fan disturbances or other operational issues, and notify guards in real time, eliminating the need to have large numbers of guards monitoring video feeds and patrons. The business benefits of digitally transformed guards are compelling. A National Hockey League security director says he used to have 300 guards manning 100 walk-through metal detectors. By moving to AI solutions, he can significantly reduce the number of scanning portals and guards, and most importantly redeploy and gain further operational efficiencies with his overall operational strategy. Changing staffing strategy This changes the staffing strategy significantly and elevates the roles of guards. Suddenly, a US$ 20-per-hour ‘job’ becomes a US$ 40-per-hour profession, with guards transformed into digital knowledge workers delivering better outcomes with digitally enabled staffs. Beyond that, these digitally transformed guards can spend a much higher percentage of their time focused on tasks that impact the fan experience – whether it’s keeping weapons out of the building, pro-actively dealing with unruly fans before a broader disruption occurs, or managing business operations that positively impact fan patron experience. Digitally transforming security guards Perhaps most important, digitally transforming security guards elevates the profession to a more strategic level, which means better pay for the guards, better service for clients of guard services, and an overall better experience for fans. That’s a perfect storm of goodness for everyone.

Why Visualization Platforms Are Vital For An Effective Security Operation Center (SOC)
Why Visualization Platforms Are Vital For An Effective Security Operation Center (SOC)

Display solutions play a key role in SOCs in providing the screens needed for individuals and teams to visualize and share the multiple data sources needed in an SOC today. Security Operation Center (SOC) Every SOC has multiple sources and inputs, both physical and virtual, all of which provide numerous data points to operators, in order to provide the highest levels of physical and cyber security, including surveillance camera feeds, access control and alarm systems for physical security, as well as dashboards and web apps for cyber security applications. Today’s advancements in technology and computing power not only have increasingly made security systems much more scalable, by adding hundreds, if not thousands, of more data points to an SOC, but the rate at which the data comes in has significantly increased as well. Accurate monitoring and surveillance This has made monitoring and surveillance much more accurate and effective, but also more challenging for operators, as they can’t realistically monitor the hundreds, even thousands of cameras, dashboards, calls, etc. in a reactive manner. Lacking situational awareness is often one of the primary factors in poor decision making In order for operators in SOC’s to be able to mitigate incidents in a less reactive way and take meaningful action, streamlined actionable data is needed. This is what will ensure operators in SOC truly have situational awareness. Situational awareness is a key foundation of effective decision making. In its simplest form, ‘It is knowing what is going on’. Lacking situational awareness is often one of the primary factors in poor decision making and in accidents attributed to human error. Achieving ‘true’ situational awareness Situational awareness isn’t just what has already happened, but what is likely to happen next and to achieve ‘true’ situational awareness, a combination of actionable data and the ability to deliver that information or data to the right people, at the right time. This is where visualization platforms (known as visual networking platforms) that provide both the situational real estate, as well as support for computer vision and AI, can help SOCs achieve true situational awareness Role of computer vision and AI technologies Proactive situational awareness is when the data coming into the SOC is analyzed in real time and then, brought forward to operators who are decision makers and key stakeholders in near real time for actionable visualization. Computer vision is a field of Artificial Intelligence that trains computers to interpret and understand digital images and videos. It is a way to automate tasks that the human visual system can also carry out, the automatic extraction, analysis and understanding of useful information from a single image or a sequence of images. There are numerous potential value adds that computer vision can provide to operation centers of different kinds. Here are some examples: Face Recognition: Face detection algorithms can be applied to filter and identify an individual. Biometric Systems: AI can be applied to biometric descriptions such as fingerprint, iris, and face matching. Surveillance: Computer vision supports IoT cameras used to monitor activities and movements of just about any kind that might be related to security and safety, whether that's on the job safety or physical security. Smart Cities: AI and computer vision can be used to improve mobility through quantitative, objective and automated management of resource use (car parks, roads, public squares, etc.) based on the analysis of CCTV data. Event Recognition: Improve the visualization and the decision-making process of human operators or existing video surveillance solutions, by integrating real-time video data analysis algorithms to understand the content of the filmed scene and to extract the relevant information from it. Monitoring: Responding to specific tasks in terms of continuous monitoring and surveillance in many different application frameworks: improved management of logistics in storage warehouses, counting of people during event gatherings, monitoring of subway stations, coastal areas, etc. Computer Vision applications When considering a Computer Vision application, it’s important to ensure that the rest of the infrastructure in the Operation Center, for example the solution that drives the displays and video walls, will connect and work well with the computer vision application. The best way to do this of course is to use a software-driven approach to displaying information and data, rather than a traditional AV hardware approach, which may present incompatibilities. Software-defined and open technology solutions Software-defined and open technology solutions provide a wider support for any type of application the SOC may need Software-defined and open technology solutions provide a wider support for any type of application the SOC may need, including computer vision. In the modern world, with everything going digital, all security services and applications have become networked, and as such, they belong to IT. AV applications and services have increasingly become an integral part of an organization’s IT infrastructure. Software-defined approach to AV IT teams responsible for data protection are more in favor of a software-defined approach to AV that allow virtualised, open technologies as opposed to traditional hardware-based solutions. Software’s flexibility allows for more efficient refreshment cycles, expansions and upgrades. The rise of AV-over-IP technologies have enabled IT teams in SOC’s to effectively integrate AV solutions into their existing stack, greatly reducing overhead costs, when it comes to technology investments, staff training, maintenance, and even physical infrastructure. AV-over-IP software platforms Moreover, with AV-over-IP, software-defined AV platforms, IT teams can more easily integrate AI and Computer Vision applications within the SOC, and have better control of the data coming in, while achieving true situational awareness. Situational awareness is all about actionable data delivered to the right people, at the right time, in order to address security incidents and challenges. Situational awareness is all about actionable data delivered to the right people Often, the people who need to know about security risks or breaches are not physically present in the operation centers, so having the data and information locked up within the four walls of the SOC does not provide true situational awareness. hyper-scalable visual platforms Instead there is a need to be able to deliver the video stream, the dashboard of the data and information to any screen anywhere, at any time — including desktops, tablets phones — for the right people to see, whether that is an executive in a different office or working from home, or security guards walking the halls or streets. New technologies are continuing to extend the reach and the benefits of security operation centers. However, interoperability plays a key role in bringing together AI, machine learning and computer vision technologies, in order to ensure data is turned into actionable data, which is delivered to the right people to provide ‘true’ situational awareness. Software-defined, AV-over-IP platforms are the perfect medium to facilitate this for any organizations with physical and cyber security needs.

Gunshot Detectors Shorten Response Times And Make Cities Safer
Gunshot Detectors Shorten Response Times And Make Cities Safer

Gunshot detectors use digital microphones installed on (or in) buildings or along streets that listen for evidence of gunshots, provide near instantaneous notification, triangulate the location of shooters and direction of a shot, detect the type of gun and ultimately aid in catching fleeing suspects and solving crimes. Gunshot detection is just one technology playing a role in the larger trend by city agencies to improve core city services. Cities are turning to what are referred to as ‘smart city’ solutions – new, innovative technologies that improve and maintain a high quality of life and ‘liveability’ for citizens. Several cities in the United States have implemented gunshot detection systems. Identifying And Deterring Gun Violence ShotSpotter, a provider of gunshot detection solutions that help law enforcement officials and security personnel identify, locate and deter gun violence, announced that seven new cities have deployed ShotSpotter technology in their communities. Gunshot detection systems can shorten the response time in an active shooter situation The new cities include Cincinnati, OH; Jacksonville, FL; Louisville, KY; Newburgh, NY; Pittsfield, MA; Syracuse, NY and St. Louis County, MO – joining the more than 90 jurisdictions that rely on ShotSpotter to ensure a fast, accurate response to gunfire incidents. Three existing ShotSpotter cities, New York City, Chicago and Birmingham have also recently expanded their coverage areas. Gunshot detection systems can shorten the response time in an active shooter situation. Early detection should be a primary aim, second only to prevention. Security professionals must be part of both of these areas, working in partnership with relevant administrators, local government, law enforcement, first responders and the community to help prevent and better respond to gun violence. Gunshot Localization Solution In addition, active shooter events – large or small – are almost always sudden and unexpected, which places a burden on security personnel to manage these risks without creating a prison-like environment. A gunshot localization solution can turn a video camera system into a real-time safety system in the event of an active shooter A gunshot localization solution can turn a video camera system into a real-time safety system in the event of an active shooter. Called ShotPoint, the system is completely automated. Working with a video management system (VMS), it can enable a video image of an active shooter to be provided in seconds based on the location of a gunshot. ShotPoint is a network of sensors which can be mounted on walls, ceilings, streetlight poles or other indoor or outdoor locations. Using a ‘sensor mesh approach’, ShotPoint reliably detects and localizes the source of gunfire; ranging from small handguns to high caliber rifles. The system can cover large indoor or outdoor areas such as schools, office buildings, retail centers, campuses, and parks. Accurately Provides Gunshot Location Each sensor has an array of four acoustic channels (microphones) that can locate the source of a gunshot sound, the time of arrival and the time distance of arrival. ‘Hearing’ shots from several vantage points (using multiple sensors) enables the system to take into account the angle and time of the sound, which vary in different environments, thus accurately providing the location of the gunshot. A ‘fusion processor’ box (at the edge) listens to the various sensor nodes and computes the location of the gunshot, relative to a floorplan and/or based on global positioning system (GPS) location. In an outdoor location, additional information may also be inferred, such as the trajectory of the gunshot and/or the caliber of the firearm.