Audio is often overlooked in the security and video surveillance industry. There are some intercom installations where audio plays a key role, but it’s not typically thought about when it comes to security and event management. Audio takes a back seat in many security systems because audio captured from a surveillance camera can have a different impact on the privacy of those being monitored.

Audio surveillance is therefore subject to strict laws that vary from state to state. Many states require a clearly posted sign indicating audio recording is taking place in an area before a person enters. Analytic information derived from audio can be a useful tool and when implemented correctly, removes any concerns over privacy or legal compliance.

Audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera

Focused Responses To Events

Audio analytics processed in the camera, has been a niche and specialized area for many installers and end users. This could be due to state laws governing audio recording, however, audio analytics on the edge overcomes legal challenges as it never passes audio outside of the camera

Processing audio analytics in-camera provides excellent privacy since audio data is analysed internally with a set of algorithms that only compare and assess the audio content. Processing audio analytics on the edge also reduces latency compared with any system that needs to send the raw audio to an on-premises or cloud server for analysis.

Audio analytics can quickly pinpoint zones that security staff should focus on, which can dramatically shorten response times to incidents. Audio-derived data also provides a secondary layer of verification that an event is taking place which can help prioritise responses from police and emergency personnel.

Microphones And Algorithms

Having a SoC allows a manufacturer to reserve space for specialized features, and for audio analytics, a database of reference sounds is needed for comparisonMany IP-based cameras have small microphones embedded in the housing while some have a jack for connecting external microphones to the camera. Microphones on indoor cameras work well since the housing allows for a small hole to permit sound waves to reach the microphone.  Outdoor cameras that are IP66 certified against water and dust ingress will typically have less sensitivity since the microphone is not exposed. In cases like these, an outdoor microphone, strategically placed, can significantly improve outdoor analytic accuracy.

There are several companies that make excellent directional microphones for outdoor use, some of which can also combat wind noise. Any high-quality external microphone should easily outperform a camera’s internal microphone in terms of analytic accuracy, so it is worth considering in areas where audio information gathering is deemed most important.

Surveillance cameras with a dedicated SoC (System on Chip) have become available in recent years with in-built video and audio analytics that can detect and classify audio events and send alerts to staff and emergency for sounds such as gunshots, screams, glass breaks and explosions. Having a SoC allows a manufacturer to reserve space for specialized features. For audio analytics, a database of reference sounds is needed for comparison.

The camera extracts the characteristics of the audio source collected using the camera's internal or externally connected microphone and calculates its likelihood based on the pre-defined database. If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS.

Each camera must be custom configured for its particular environment to detect audio levels which exceed a user-defined level
If a match is found for a known sound, e.g., gunshot, explosion, glass break, or scream, an event is triggered, and the message is passed to the VMS

Configuring A Camera For Audio Analytics

  • Audio Detection
    The first job of a well-configured camera or camera/mic pair is to detect sounds of interest while rejecting ancillary sounds and noise below a preset threshold. Each camera must be custom configured for its particular environment to detect audio levels which exceed a user-defined level. Since audio levels are typically greater in abnormal situations, any audio levels exceeding the baseline set levels are detected as being a potential security event. Operators can be notified of any abnormal situations via event signals allowing the operator to take suitable measures. Finding a baseline of background noise and setting an appropriate threshold level is the first step.
  • Noise Reduction
    A simple threshold level may not be adequate enough to reduce false alarms depending on the environment where a camera or microphone is installed. Noise reduction is a feature on cameras that can reduce background noise greater than 55dB-65dB for increased detection accuracy. Installers should be able to enable or disable the noise reduction function and view the results to validate the optimum configuration during setup. With noise reduction enabled, the system analyzes the attenuated audio source. As such, the audio source classification performance may be hindered or generate errors, so it is important to use noise reduction technology sparingly.
  • Audio Source Classification
    It’s important to supply the analytic algorithm with a good audio level and a high signal-to-noise ratio to reduce the chance of generating false alarms under normal circumstances. Installers should experiment with ideal placement for both video as well as audio. While a ceiling corner might seem an ideal location for a camera, it might also cause background audio noise to be artificially amplified. Many cameras provide a graph which visualises audio source levels to allow for the intuitive checking of noise cancellation and detection levels.
  • Messages And Events
    It’s important to choose a VMS that has correctly integrated the camera’s API (application programming interface) in order to receive comprehensive audio analytic events that include the classification ID (explosion, glass break, gunshot, scream). A standard VMS that only supports generic alarms, may not be able to resolve all of the information. More advanced VMS solutions can identify different messages from the camera.

Well configured audio analytics can deliver critical information about a security event, accelerating response times and providing timely details beyond video-only surveillance.  Analytics take privacy concerns out of the equation and allow installers and end users to use camera audio responsibly. Hanwha Techwin's audio source classification technology, available in its X Series cameras, features three customizable settings for category, noise cancellation and detection level for optimum performance in a variety of installation environments.

Download PDF version

Author profile

In case you missed it

U.S. Partial Government Shutdown: What's The Impact On The Physical Security Industry?
U.S. Partial Government Shutdown: What's The Impact On The Physical Security Industry?

Security is arguably at the heart of the United States partial government shutdown: President Trump’s demand for $5.6 billion to start building a wall along the southern border with Mexico is the major bone of contention as gridlock in Washington prevents passage of a spending bill to keep the government in full operation. The partial government shutdown has affected security in other ways, too, and some of the impact could continue long after the impasse is settled. Some 800,000 federal employees are impacted, some on full or partial leave as a result of the shutdown and others working without knowing when they will get paid. Cybersecurity Initiatives Delayed Furloughed federal employees tasked with cybersecurity are not on hand to address the constant threat to government IT systems from possible hackers and other bad actors. In fact, hackers may decide the government shutdown is a vulnerable time to launch an attack. Data capture form to appear here! Hackers may decide the government shutdown is a vulnerable time to launch an attack Specifically, the new Cybersecurity Infrastructure Security Agency, launched last fall, is operating with a skeleton staff. If the shutdown encourages cybersecurity experts to seek other employment, the resulting drain of “knowledge capital” could be a lasting detriment. New cybersecurity initiatives are also being delayed, such as the 21st Century Integrated Digital Experience Act, aimed at creating a baseline of security defense across government web sites. Some government websites have had their Transport Layer Security (TLS) encryption certificates expire during the shutdown. Impact On TSA Agents Although Transportation Security Administration (TSA) agents remain on the job at major airports, they will not be paid again until after the shutdown is over. The resulting negative impact on morale has arguably slowed down airport security operations, although airports have not reported any major problems. In addition, some agents have called in “sick” and/or sought other employment to provide income during the shutdown. In the Dallas-Fort Worth airport, the number of agents calling out sick reportedly increased by 200% to 300%. The absences can aggravate existing TSA staffing shortages. Impact On Border Protection Agents Customs and border protection agents are also on the job with no assurance of when they will be paid. Other immigration agents in the Department of Homeland Security are also currently without pay. Hearings on immigration cases are being canceled, which can result in a large backlog to be addressed after the shutdown ends. Customs and border protection agents are also on the job with no assurance of when they will be paid What About Worker Documentation? A consequence of the shutdown is unavailability of the government’s E-Verify system, which is used to verify a worker’s immigration status prior to being hired by an employer. During the lapse in government appropriations, employers will not be able to access their E-Verify accounts, create an E-Verify case, edit company information, terminate accounts, run reports, etc. Long-Term Impact On Government Employment The longer the shutdown continues, the more employees will be encouraged to seek work elsewhere The shutdown may lead federal employees to seek work in the private sector, where their paycheck is not likely to be delayed because of a political impasse. The longer the shutdown continues, the more employees will be encouraged to seek work elsewhere, whether on a temporary basis or as a permanent alternative. Manpower shortages can translate into security risks. Trump has argued for funding of the border wall on security and humanitarian grounds and has sought to put pressure on newly empowered Democrats. However, adding physical barriers at the border is only part of the solution to border security, says the conservative Heritage Foundation. Also needed are improved technology to monitor the border and appropriately equipped border patrol agents. Holistic Approach To Border Security This holistic approach of combining barriers, technology, and people is the cost-effective way to secure the border, says Heritage Foundation. It’s also important to enforce immigration laws, and border security does nothing to stop visa overstays, another source of illegal immigration. The Democratic opposition has said Trump’s urgent call to build the wall is a “manufactured crisis,” and the wall would be “immoral” and “ineffective.” Weeks into the impasse, there is no end in sight.

A Glance At The Winners And Losers Of The Security Industry In 2018
A Glance At The Winners And Losers Of The Security Industry In 2018

In my coverage of China Tariffs impacting the security industry over four recent articles, products on the tariff schedules routinely integrated into security solutions included burglar and fire alarm control and transmission panels, video surveillance lenses, HDTV cameras used for broadcast use cases and fiber optic media converters. The general ‘callout’ of ADP (Automatic Data Processing) devices and peripherals technically includes servers, workstations and microcomputers, all of which are commonly used to support security solutions. The underperformance, from June 15 to August 24, of U.S. stocks with high revenue-exposure to China, and that of Chinese stocks with high revenue-exposure to the United States was significant and almost identical at 3.2%, significant losses to some investors already involved in security industry M&A activity. Significant Public Safety Facial Recognition (FR) vendors leveraging AI expanded their market focus to retail and public safety While it was not apparent that practitioners’ security program budgets kept pace with the growth of the more popular solution providers like video surveillance and cyber security, the ICT industries supporting the security economy continued to expand, especially in wireless and wired infrastructure, including preparations for 5G wireless rollouts. These omnipresent technologies drove significant public safety, smart city and public venue projects in 2018. Facial Recognition (FR) vendors leveraging AI expanded their market focus to retail and public safety. In 2018, virtually every public presentation, webinar and published Q&A on social media monitoring and facial recognition technologies I worked on, involved significant pushback from privacy advocates, almost to the point of alarmism. Massive Risk Reduction Several solution providers in these areas have made significant strides on data protection, accuracy, powered by AI and documented crime reduction cases; however, this real news is quickly shadowed by privacy advocates, seemingly ignoring massive risk reduction, especially in the case of active assailants and gang-related crime. Will FR become mainstream? The cautious security industry may take a cue from the maverick retail industry, sports venue and VIP verification solution providers that grew in 2018. 2019 trends: presupposition or repudiation; winners and losers. Chinese tariffs have had a huge impact on the security industry, which can be seen from changes to U.S and Chinese stocks  Although technology adoption forecasting is inexact, there are definitive opportunities in the security industry born on necessity. With the widespread problem of false alarm transmission and inability for first responders to ‘be everywhere,’ developers of solutions that provide automated verification and alternative security incident detection are expected to become mainstream. Promising Detection Systems The use of AI, NLP, LiDAR, UAS (Unmanned Aerial Vehicles aka drones) with surveillance and thermal imaging will grow, mostly due to higher acceptance in other industries like autonomous vehicles, rail safety, terrain and post devastation mapping/rescue. However, legacy ‘listing’ or certification organizations will be forced to make an important decision for their own survival: work toward integrating these promising detection systems into acceptance by insurance, licensing and standards development organizations. 2019’s ‘true’ Industrial Philanthropists will be needed to fund early warning tech for firefighters and the presence of active assailants 2019’s ‘true’ industrial philanthropists will be needed to fund early warning tech for firefighters and the presence of active assailants. For these use cases, 5G infrastructure rollouts, FR acceptance, lower cost perimeter detection and long range object and fire recognition by LiDAR and Thermal imaging will all be watched closely by investors. Should public agencies and philanthropical solution providers in the security industry cross paths, we may just yet see a successful, lifesaving impact. Cyber Risk Profile The ‘Digital twin’ refers to a digital replica of physical assets (physical twin), processes, people, places, systems and devices that can be used for various purposes. Your ‘Security Digital Twin’ has a similar physical and cyber risk profile, either through common threats, similar assets or both. Good news: managing your risk, protecting assets and securing your facilities in 2019 will get easier as security digital twin profiles will grow in maturity, while keeping their data sources private. This will be accelerated by the maturity of AI-based, auto-generated visualizations and image recognition, that happens to also drive the FR solutions. The 5G wireless infrastructure market is emerging as far more of a quantum leap in connectivity, like ‘wireless fiber optics’ performance, than an upgrade to 4G LTE. The 5G infrastructure market will be worth $2.86 billion by 2020 and $33.72 billion by 2026, growing at a compound annual growth rate (CAGR) of 50.9%. Intelligent Applications The explosion of ingested voice, video, and meta-data, the interconnectivity of devices, people and places, and the integration of intelligent applications into expanding ecosystems all require faster communications. To be more accurate, 5G rollouts will accelerate in 2019; however, current project funding will include and be impacted by future enterprise security connectivity: 5G and FWA (Fixed Wireless Access). 5G rollouts will accelerate in 2019; however, current project funding will include and be impacted by future enterprise security connectivity Quite simply put, larger solution providers are gently coaxing practitioners into seemingly ‘open systems;’ the negative discovery during an M&A process, audit or integration with a smart city’s public/private partnerships will continue to be revealed, and related industries will force reform. Autonomous things will be enabled by AI and image recognition. With few affordable rollouts of security robots and outdoor unmanned ground vehicles (UGV) that leveraged platforms popular with research and even NASA, the autonomous security robot was mostly MIA from a security practitioner’s program in 2018. Perimeter Intrusion Detection One platform was even accused of intimidating homeless people in a public place, at a major city. Industries mutually beneficial are often unaware of each other; this will change gradually: one major domestic airport is currently evaluating a UGV platform performing perimeter intrusion detection, runway weather conditions and potential aircraft taxiing dangers. The platform is being used largely in transportation research, yet offers significant opportunities to the security industry. Research firm Gartner estimates that 70% of today’s technology products and services can be enhanced with ‘multi-experience’-based VR/AR/MR The ‘immersive experience’ of virtually any security or threat detection is a twist on virtual/augmented/mixed reality (VR/AR/MR) with additional sensory features. Although VR/AR/MR is well underway in other industries, there are several companies with solutions like VR-based active assailant training that could provide a fighting chance for practitioners, employees, visitors, faculty and children. Research firm Gartner estimates that 70% of today’s technology products and services can be enhanced with ‘multi-experience’-based VR/AR/MR. Security Ecosystem Members Not necessarily MIA, but of special mention is the need of security and safety practitioners to prioritize communications systems over ‘nice to have’ expansive video surveillance systems for mass casualty threats. This will eventually improve with 5G for Enterprise solution rollouts. At the past GSX and upcoming CES Technology trade shows, a new roundup of technologies is discovered: a wider diversity of protection promise to save ASIS members on their technical security program is realized. With each of the ‘winners,’ (5G, AI, NLP, LiDAR, UAS [Unmanned Aerial Vehicles aka drones], thermal imaging, digital security twins and smart-city-friendly technologies) it is both exciting and challenging work for both security practitioners and solution providers. All things equal and with the necessary technology acceptance testing processes, this is a truly great time for security ecosystem members.

Security Industry Trends To Be Led By Focus On Cyber Security In 2019
Security Industry Trends To Be Led By Focus On Cyber Security In 2019

The Security Industry Association (SIA) looks forward to 2019, and it is apparent that physical security is moving into its most formative years. Changes presented by emerging technology, open systems and growing connectivity among devices and sensors will make a big difference for manufacturers, systems integrators/dealers and end users. With a more open, connected environment come cyber risk and data privacy concerns – which is why, in SIA’s 2019 Security Megatrends, cybersecurity’s impact on the physical security industry ranks number one on the list. Cybersecurity is affecting all areas of the industry landscape, from security implementation to attracting top talent to the workforce. Digital Transformation The digital transformation we are experiencing impacts many other parts of the security industry as well, bringing opportunities like evolving identity management and collecting and delivering big data to customers. At this critical point in the industry’s development, it is important to embrace change, leverage disruptive technology in ways that give companies a competitive advantage. To determine this year’s Megatrends, SIA surveyed hundreds of executives from member companies To determine this year’s Megatrends, SIA surveyed hundreds of executives from member companies, along with current and recent Securing New Ground speakers and attendees, to identify which previous trends were still relevant, which trends were no longer as impactful and which broad trends should be added to our report. This Year’s Security Megatrends 1. Cybersecurity’s Impact on Physical Security: It is important to prioritise cybersecurity for your business, your customers’ business and the vendors with which you work. This trend calls for continual process improvement and investment. 2. Internet of Things (IoT) and the Big Data Effect: The security industry makes use of IoT, analytics, artificial intelligence (AI), robotics and more, and data is coming from everywhere. The industry now faces the challenge of effectively managing and segmenting this information to be pertinent to the user. 3. Cloud Computing: Cloud platforms and applications are becoming prevalent across security solutions. This technology helps security integrators provide managed services and the advantages of off-site systems and services to customers. 4. Workforce Development: With historically low unemployment, finding skilled employees is a challenge to the whole security industry. Security stakeholders need talent with IT, cybersecurity, AI and even privacy expertise, presenting a need to grow students’ interest in the industry. 5. AI: Research firm Gartner predicts a new “democratisation of AI” that will impact more organisations than ever before. Companies are now testing this technology before offering it to customers and exploring how AI data can be used to improve security threat assessment and response. 6. Emphasis on Data Privacy: Growing connectivity brings new concerns over data privacy. Finding the balance between security and convenience is a dilemma the industry must now address. 7. Move to Service Models: The newest home security technologies are strongly impacting installing companies. Systems integrators must find ways to focus on services customers want and need and move to managed service models to make up revenues. 8. Security Integrated in Smart Environments: As everything becomes connected, smart environments will begin to proliferate. Buildings and cities are becoming more conscious, with connected systems now able to automatically respond to and even anticipate the needs of facility users and citizens. We must continue to find ways to make these environments smarter and safer. 9. Identity of the Future: With facial and voice recognition and biometrics growing in popularity and appeal, how will we enter buildings and access networks tomorrow? The industry will anticipate and adapt to constant technological change in identity and visitor management. 10. Impact of Consumer Electronics Companies: The influx of consumer electronics companies and DIY systems means changing rules and players in the security industry. This disruption presents both challenges and opportunities for security companies.