Automated Intelligent Surveillance
When the surveillance industry first started, the main activity was to record surveillance on video. As recording became more distributed, the technology was referred to as a Video Management System (VMS).
Over time, information from other sources was brought into the control room. If this information related to a building, e.g., information on temperatures, air conditioning and lighting), it was called a Building Management System (BMS).
Systems were then developed to list courses of action for particular events. The instructions were a documentation of processes that the operators were required to take. For instance, if a fire was detected, the operators needed to sound the alarm, ring the fire brigade and ask people to gather in a safe area outside the building. Such systems were called Physical Security Information Management (PSIM) systems.
Up to this point, there was little or no intelligence in the surveillance system. It was a recording and display system with the ability to provide instructions for the operator. Using the video to analyze and detect certain types of events became known as Video Analytics. In the last few years, the most sophisticated Video Analytics companies have also begun to provide recognition systems for vehicles and faces.
iOmniscient is already recognized as the technology leader in Video Analytics. However, it has taken the technology two further steps forward, first with the concept of Automated Surveillance and next with the concept of Automated Response Systems which will be described in the next section.
Conventional Techniques To Recognize A Person
In the past, Pan Tilt and Zoom (PTZ) cameras were used when one attempted to perform detection and recognition at the same time. As such, the camera would monitor the total scene. If an event occurs, e.g., a person jumps over a fence and enters the area that is monitored, the PTZ camera can zoom in onto the intruder who then can be recognized in the close-up view.
Initially, this maneuver was done manually. Once detection occurred, the operator would manually zoom in on the individual who had jumped over the fence. It was soon discovered that manual zooming was prone to error. It was very difficult for an individual to accurately zoom onto the target. Even for a skilled operator, it was not easy to make the camera close in directly on the targeted person. The task is equivalent to taking aim with a rifle sight. A slight movement of the camera meant that the person was not in the view.
This method was so difficult to use that when a capability was announced for cameras to automatically zoom in on their target, this was readily embraced. With this capability, the camera detects an intrusion and zooms in the PTZ camera's view to the intruder's co-ordinates.
This method had one very significant limitation. It was easy to defeat by anyone who was familiar with PTZ cameras. A decoy could be sent in. The camera would zoom in on the decoy and the real intruder could enter the scene from the other side. The next step in this evolution was to use two cameras. A fixed camera was used for detection. If an intrusion occurs, a separate PTZ camera zooms onto the intruder to perform recognition. However, the first camera continues detecting and if a second person intrudes, he is detected as well. Unfortunately, the second intruder is not identified as the PTZ camera is busy with the first intruder already.
The PTZ camera was usually programmed to follow the largest object in the scene. If a bird flew across the camera view, the PTZ camera would follow the bird (because it is closer and appeared larger) and it would ignore the intruder. Serious users of video surveillance systems have rejected the use of PTZ cameras for this type of detection and zooming because of its inherent limitations.
Automated Surveillance Using Higher Resolution Cameras
A really effective solution evolved through the introduction of a totally different technology - megapixel cameras. As megapixel cameras became more widely used, it was evident that by recording an image at a resolution of between 1 and 5 megapixels, the same image could be used for both detection and recognition. Multiple detections and recognitions can be done on the same image view. However, megapixel cameras have their own disadvantage. They are incredibly expensive to use because of the computing power required to process the images and the high bandwidth required for transmission. Furthermore, the large and dense images are expensive to store.
The standard camera image with around 384x288 pixels is equivalent to a 0.1 megapixel image. Using a 1 megapixel camera requires 10 times the computing power over a standard camera. A five megapixel camera requires 50 times the computing power. One would also require 50 times the storage capacity and 50 times the band width for transmitting images.
iQ-Hawk (Internationally Patented)
In 2008, iOmniscient developed a patented technology that finally solves the issue of the convergence of detection and recognition technologies. A working product called iQ-Hawk based on this patent was introduced to the market around the middle of 2008.
This revolutionary technology involved using a megapixel camera. It performed detection at very low resolution. Because of iOmniscient's powerful detection technologies, this could be achieved even at a very low frame rate (between 2 and 6 frames per second). The system then automatically digitally zoomed in and extracted the face or license plate of the person or vehicle in the incident to recognize in high resolution. It optimized the way how detection and recognition are performed at different resolutions to achieve several significant goals.
iQ-Hawk is the first and only technology which allows multiple detections and recognitions on a single camera without incurring the computing, storage and network bandwidth costs associated with the use of megapixel cameras.
1. Unlike PTZ cameras, it cannot be fooled by decoys.
2. Unlike standard megapixel camera based systems, it does not require massive computing, storage and networking resources. (When using a 5 megapixel camera the system requires 200 times less computing resources than an equivalent system processing images in the traditional way).
3. It can perform multiple detections (all types and at all iQ-levels) and subsequent recognition (both License Plate and Face Recognition) on the same scene using the same camera.
All this is performed automatically. If 10 events occur concurrently in the scene, the system will recognize the people and vehicles associated with all of them. At last, detection and recognition can finally be achieved on the same camera in a commercially viable way. And it can be done such that it cannot be defeated by someone with a cursory knowledge of the way these systems work. iQ-Hawk reduces the overall costs of the system by reducing storage and network bandwidth requirements by a huge factor.
This capability provides the foundation for Automated Surveillance systems. Incidents are automatically detected; people and vehicles involved in the incident can be automatically recognized. This patented capability was the first major technology breakthrough moving the industry beyond simple Video Analytics technology.
iOmniscient Automated Response
In surveillance systems, the response has usually been left to a human operator. However, a human operator may take considerable time to provide a response. His ability to provide an appropriate response is limited by the information that he is able to extract and absorb in a very short period of time under stressful conditions.
Yet, iOmniscient has pioneered another technology to address this issue - the ability to provide an Automated Response. The way the technology works will be described with a real example.
For a major Expo in Asia, the city council had the problem that large construction trucks were travelling on narrow streets where they were not allowed. They often got stuck in these streets causing traffic problems. The council set up cameras to detect these trucks and by using a GIS system, they could locate these trucks on iOmniscient Automated Response (Internationally Patented) on Google map. In a traditional system, an operator would then have called the police to check if there was a police vehicle in the vicinity to apprehend the truck. However, by the time the operator could locate the police and arrange a vehicle to be sent, the truck had usually disappeared.
iOmniscient has implemented its Automated Surveillance Action Platform (ASAP) for fast automated response. With this unique patented capability, once the truck is detected, the system searches the Police GPS system to determine if there is a police car in the vicinity. The system automatically sends the information on the truck to the police vehicle with instructions for apprehending it. The whole process is automatic and ensures a very fast response.
The communication is two-way. If the police vehicle is engaged somewhere else and is not in a position to respond to the alert, it can advise the system accordingly. The system can then send the information about the event to the next nearest vehicle.
The system is not limited to the police. It can be extended to all emergency services. If there is an accident, the information can be sent to the nearest ambulance. If there is a fire, the nearest fire brigade can be alerted and mobilized. Operators within Command and Control Centers may have the concern that direct communications between the cameras, the surveillance system and the response teams (e.g. in police vehicles) would diminish their control over their operations. This is easily addressed by providing all the relevant information to the operator for approval or veto.
Hence, if the camera sees an accident, it can locate the nearest police and ambulance vehicles. But rather than sending the information directly to these, it can send the information about the accident and about the potential responders to the Control Center. Having all these information, the operator is in a position to make an instantaneous decision to proceed according to the recommendations of the system, saving precious time which can save lives in an emergency. The operator is in a position to approve or disapprove the action and remain in full control of the situation. Therefore, the level of automation is configurable by the user. The higher the level of automation, the higher the speed of response.
The Automated Response System is the latest frontier in tools available to improve the productivity and efficiency of teams involved in improving the security, safety and services available for modern cities.
The system is not limited to use by responders in vehicles. Guards and policemen carrying smartphones can be located if they are closest to an incident. The concept is to speed up the response to an incident by automating the process of locating the nearest appropriate responder.