Autonomous Remote Sensing – A Tale of Evolving, Emerging and Converging Technologies (Part 1)

A recent Centre for EO Instrumentation (CEOI) workshop discussed future applications of autonomous remote sensing in a wide range of sectors and found opportunities that will take the technology into major new markets.

Traditionally, remote sensing has been defined as the acquisition of information about an object or phenomenon without making physical contact. However, the emergence of new sensing techniques, miniaturization of electronics, more powerful software, and an ever-increasing range of applications has led to this definition being expanded to include terrestrial-based remote sensing and remote embedded sensing. At the same time, the emergence of technologies and applications for autonomy have led to a dramatic expansion in the use of remote sensing technologies in these autonomous systems and to the development of remote sensing systems that are themselves autonomous. These autonomous remote sensing (ARS) systems are the culmination of long-term development of existing technologies, emergence of disruptive new technical capabilities, and convergence of sensors, optics, electronics, and communications technologies.

Infrared image of Hurricane Harvey prior to making landfall along the Texas coast on August 25, 2017. Credit: NOAA/NASA.

The term remote sensing when applied to space-based or aerial sensor technologies refers to the detection and classification of processes and objects on Earth (atmosphere, oceans, surface) by means of propagated signals (e.g. electromagnetic radiation). It is split into active remote sensing -- when a signal is first emitted from aircraft or satellites (e.g. radar or lidar), or passive remote sensing -- when naturally occurring signals are recorded (e.g. from reflected or scattered sunlight). However, in recent years the definition has been extended significantly to new sensing modalities as the range of applications has developed. These include:

    • Terrestrial-Based Remote Sensing - instruments traditionally deployed on spacecraft or aircraft are being deployed in ground-based applications, e.g. for continuous monitoring of urban air quality or security threats.

    • New Applications of Remote Sensing - there are growing requirements to analyse and monitor different structures. Examples include the monitoring of tunnels or viaducts using remote sensing techniques to efficiently manage their construction and operation.

    • Remote-embedded Sensing - while remote sensing has traditionally been completely stand-off, the definition is now evolving to include the use of embedded sensors to remotely monitor structures and environments which are hazardous or difficult to access. Sensors are embedded at the point of interest and the data is transmitted to a central point for processing, analysis, and action, e.g. aero engine combustion chambers, rail track, and nuclear reactors. The challenge being addressed is sensing / monitoring from a distance environments that are difficult to access with conventional sensors / instruments due to distance, scale (1000 km), environment (temperature / radiation / pressure), etc.

CityScan air quality monitors were used to measure air pollution before and after the 2012 Olympic Games in London. Credit: University of Leicester

While remote sensing technologies have been evolving over several decades, capabilities in autonomy have emerged rapidly over the last few years. Autonomous systems technologies are truly transformational with benefits in cost / risk reduction and can enable entirely new capabilities in applications where direct human control is not possible due to inaccessibility, speed of decision making, or other human limiting factors. They have traditionally been used in situations where great precision and accurate repetition is important such as manufacturing and assembly. Autonomous systems are finding new applications in hazardous or challenging environments and areas where human lives may be exposed to greater risk. Examples include autonomous submarine vehicles for deep sea exploration and maintenance in oil and gas facilities, or remote systems conducting maintenance and repair in nuclear contaminated areas, and lunar and space exploration. But the disruptive future applications of autonomous systems will be in "systems of systems" such as intelligent transport systems across an entire region, or cost-effective management of integrated healthcare systems, from tele-diagnostics through stratified medicine and hospital admission to aftercare.

The Sensor Web Project uses a network of sensors linked by software and the internet to an autonomous satellite observation response capability. This system has been used to implement a global surveillance program to study volcanos. Credit: NASA/JPL

At the same time, there is a convergence of ARS systems and autonomous vehicles, both in the use of ARS to provide situational awareness to the autonomous vehicles (lidar, radar, IR, optical, sensing) and for autonomous vehicles (cubesats, UAVs, vehicles, ships) to provide mobility platforms for ARS systems. Applications include precision farming, forestry, monitoring of critical infrastructure, fire detection and flood monitoring, bio-security, and security / law enforcement.

In aerospace, services are becoming increasingly important to engine manufacturers wishing to maximise revenues while maintaining functionality and quality. To do this, real-time remote sensing of all parts of the airframe and engine are required (condition monitoring) to optimize operation and maintenance.

These three false-color images demonstrate some of the applications of remote sensing in precision farming. The images were acquired by the Daedalus sensor aboard a NASA aircraft flying over the Maricopa Agricultural Center in Arizona. Credit: NASA Earth Observatory

There is a movement away from man-heavy to man-light operations in the defense and security sector with an increasing need for remote sensing to monitor situations / environments on a continuous basis to provide situational awareness. However, this often requires a rapid refresh or continuous temporal monitoring due to the nature of the threats, which makes it difficult for satellites to be used. Remote sensing from satellites and aircraft can be used to spot things like clandestine laboratories as they are relatively slow to appear, but other approaches are needed for rapidly evolving situations.

A major challenge for the defense and security forces using ARS is finding rare events in very large data sets. Improved data processing, fusion, analysis, and interpretation techniques are needed before remote sensing will become really useful.

In the maritime world, there is a growing interest in autonomous operation and swarms of vessels. This change in operation will give rise to a number of challenges, especially navigation of autonomous ships in contended (?) and chaotic areas such as the Solent - the strait that separates the Isle of Wight from mainland England.

There are three major areas in the oil and gas sector where improvements in ARS are needed. First is sensors and intelligence in the drill head; intelligent drill bits can enable better control of the drilling process, and on-board data processing can reduce the amount of data to be transmitted to the surface. Second, communications are an issue as data transfer has low bit rates due to sonic modulation, and electrical connections are not reliable. The ability to transfer much larger amounts of data to the surface from intelligent drill bits will significantly improve drilling operations. The big prize is to avoid withdrawing the drill bit. Third is autonomous processing of seismic survey data which currently requires human intensive pre-processing.

A view of a city street from the Luminar lidar system. Credit: SPIE Newsroom

In the rail sector, embedded ARS systems are increasingly needed for asset management, where fixed assets typically have a 120-year lifetime. Trains could check their own infrastructure with data capture while on the move. The technology exists but it is hard to retrofit, leading to adoption only in new projects such as HS2 and HS3. Communications is a big issue for these applications, as data download is a challenge when travelling at 100m / sec. Another application is autonomous remote monitoring of tunnels under construction -- a dangerous environment. Maintenance of tunnels over long periods while in operation are also a challenge. Ideas under consideration include computer vision to monitor change, and 3D mapping with defect analysis to monitor tunnels at a larger scale.

"Data as a service" is also beginning to emerge as an alternative to traditional approaches where organizations run their own ARS systems or buy datasets from such systems. ARS could be as service in its own right (e.g. topographic data / pollution monitoring) or to enable another service offering, e.g. condition monitoring as part of an equipment maintenance service. The challenge will be to acquire appropriate data with sufficient "value add" to enable a viable business model.

While these future applications provide great potential for ARS systems, there are many technical, ethical, societal, and legal challenges to be overcome if they are to be practical. These are discussed in the Part 2 of this article.

Further information about the work of the Centre for Earth Observation Instrumentation can be found at www.ceoi.ac.uk. The website also includes information on the wide range of projects and programs funded by the CEOI. CEOI Director, Professor Mick Johnson can be reached at mick.johnson@airbus.com.

Robbin Higgons
Qi3 Limited

Robin Higgons is Managing Director of Qi3 Limited. A chemist by training, Higgons is a specialist in international technology marketing, focusing on strategy, marketing, sales, and technology translation. He is heavily involved in leading the private sector side of the business, helping companies to identify new markets via TME's and advising on product development.