An electro-optical sensor system for maritime surveillance
Transport on safe sea routes provides the backbone of the global economy. The increasing incidence of piracy and asymmetric attacks on ships in waters remote from coastal areas has prompted the need for enhanced onboard protection measures for both civil ships and warships. A key element to effectively countering an attack is its early detection: enabling the ship's crew opportunity to prevent invaders from intruding the close-in range or even from boarding the ship. The possibility of permanent and continuous surveillance of a ship's surroundings would also reduce the need for crew personnel to be assigned to this task.
Over the past several years we have been developing a system for the automatic surveillance of a ship's surroundings. Our system, SIMONE (Ship Infrared Monitoring, Observation and Navigation Equipment), enables the early detection of approaching objects.1, 2 Using video imagery of the surroundings, a single crew member can essentially monitor the whole ship and raise the alarm to initiate counteractive measures. Even approaching objects possessing a low radar cross-section can be detected. The real-time video imagery allows the human operator to classify the type of threat.
SIMONE consists of a sensor suite either fully integrated into the ship (in case of new constructions) or retrofitted into existing platforms. Figure 1 shows an overview of the system architecture. Modules containing uncooled IR microbolometer sensors are arranged on the ship to continuously cover its surrounding. Using this configuration, the number of ‘dead zones’ caused by shadowing are minimized, and observation is possible from the ship's hull. Image data is transferred to a central processing cabinet (a computer) for real-time processing over Ethernet. Live video imagery and alarm data can be fed to several clients (computers that possess a network connection to the processing cabinet) simultaneously. All of the system components are connected by fiber-optic links.
During our extensive trials, we assessed different imaging sensors—including low-light-level cameras, daylight cameras, cameras sensitive in midwave-IR range, and cooled cameras—and sensor combinations for their potential to provide image data of low-contrast objects in maritime environments. Automatic processing for the purpose of object detection imposes specific demands on the image quality, such as stable performance with high detection rates and low false-alarm probability. We selected sensors operating in the long-wavelength IR spectral band because of their night-vision capabilities and their independence of external artificial (or natural) illumination. In addition, they allowed us to avoid sensor saturation. Long-wave IR sensors experience low impact from sun glint and external illumination. Thus, objects with low thermal contrast respective to their natural background (i.e., little temperature difference) can be resolved in the long-wave IR spectral range. Sensors with different spectral responses partially enhance the detection rate under specific diurnal and environmental conditions. However, considering cost effectiveness, the use of a sensor with universal application was preferable.
At the beginning of our research, we anticipated the need for developing microbolometer detectors that were capable of operating in large detector formats and with high thermal resolution. To achieve this, we used video graphics array-format microbolometer detectors possessing ∼50mK noise equivalent temperature difference (i.e., low camera noise), and exhibiting few pixel defects. The temporal response of microbolometer detectors is fully compatible with the scene dynamics for the maritime application. The detectors are equipped with fast IR optics that meet the demands of illuminating the full detector area and high modulation transfer function values (i.e., high imaging quality) over a wide range of temperatures. Importantly, cryogenic cooling is not required for operating these sensors, which extends their lifetime and reduces their need for maintenance. Careful calibration of the sensors ensures that spatial non-uniformity does not exceed the temporal noise.
The real-time processing of the image data obtained from electro-optical sensors involves the application of a variety of digital filters for object extraction and image enhancement. Once a potential threat is detected, a track is created to follow the object over time, which includes information on the object (such as its kinematics) and the corresponding alarm data. These are displayed on the human machine interface.
The homogeneous and simultaneous delivery of the imagery visualization is executed by the human machine interface. This characteristic is particularly important for scenes covering a wide range of temperatures and containing objects with little radiometric contrast compared to their background. Automatic image processing is performed on the full pixel depth. However, the interpretation of the scene content by human operators requires the contrast of the imagery to be improved because the human eye is limited in terms of the range of contrast distinguishable. To improve the contrast, we use a range of algorithms to process the imagery. Figure 2 shows the result of employing these algorithms to enhance the human operator's ability to capture the full visual content of the image. The top-left image is the raw data. Simple adjustments to brightness and contrast highlight details of the area marked by the red rectangle, whereas information outside this area is lost. Image enhancement algorithms automatically perform a local adjustment of brightness and contrast based on the image content. The result is the detailed image on the bottom right.
By distributing the sensors on the ship, we aim to create a panoramic view of the surroundings, which can be displayed on the human machine interface together with alarm data (see Figure 3). Indeed, our system has been qualified according to military standards for use in harsh naval environments, including electromagnetic compatibility.
In summary, we developed a robust electro-optical sensor system and an automatic image processing platform for detecting threats to naval vessels. In our future work, we will adapt the system to fulfill particular requirements of other naval platforms as well as upgrade the software to cover particular operational scenarios.
We appreciate the support of all contributing colleagues.
