Autonomous Remote Sensing – A Tale of Evolving, Emerging and Converging Technologies (Part 2)

Infrared image of Hurricane Harvey prior to making landfall along the Texas coast on August 25, 2017. Credit: NOAA/NASA.

Part 1 of this article discussed the wide range of future applications for autonomous remote sensing (ARS) systems uncovered in a recent Centre for Earth Observation Instrumentation (CEOI) workshop, which have potential to provide great commercial and social value. They include precision farming, forestry, monitoring critical infrastructure, fire detection, flood monitoring, bio-security, and security /law enforcement. While these are very exciting, Part 2 addresses the many technical, ethical, societal, and legal challenges to be overcome if they are to be practical.

The technical challenges are many and varied. Airborne and space-borne remote sensing platforms often incorporate in-scene ground truth measurements as a means of calibration and increasing the robustness of the model used to extract information from the imaged scene. In many of these future applications, it will often be impractical to utilize human observers for ground truth measurements due to temporal and physical restrictions. An in-situ sensor network may be needed to replace human observers, which could also provide a rich stream of data autonomously. This can enhance the product produced from the ARS platform.

This grouping of two test rovers and a flight spare provides a graphic comparison of three generations of Mars rovers developed at NASA Jet Propulsion Laboratory. Credit: NASA/JPL-Caltech

Fusion of data from satellite, UAV and ground sensors to give high-quality data and information is a major challenge. Other major data challenges include management and analysis in real-time monitoring, and security to ensure sensor systems are resilient in the face of cybersecurity threats.

Power harvesting / scavenging will become increasingly important as autonomous remote sensing systems are deployed into a wider range of applications. Remote locations without power have obvious issues, but the bigger challenge is for wearable systems developed to aid police, soldiers, or workers while operational. Such systems could provide critical information, but the power issues will need resolving before effective widespread adoption can take place.

Demonstration of a remote-controlled vehicle used to clear explosives in the Netherlands. Credit: Wikipedia

Artificial Intelligence is seen as vital for ARS, but there are a number of challenges that need to be addressed, including "what needs to be sensed;" "how to handle large data sets;" "monitoring of the sensing system itself;" "identifying complex solutions from large data sets;" and "situational awareness." There is the potential to generate huge volumes of data, but with fewer people available to process the data, machine learning paradigms to extract the relevant information will be needed. AI should be able to reduce processing time and level of false positives, undertake auto labelling / auto feature analysis, and flag anomalous events, thus enabling effective automatic learning and decision-making tools.

There are several challenges relating to autonomy in ARS systems. Autonomous determination of when to sense, selection of relevant spectral bands, and selection of relevant spatial regions are all of interest. Autonomous management of data processing, communications, self-forming networks, and navigation in difficult environments are enabling capabilities that need further development. Autonomous scene selection is of major interest. When combined with machine vision & photogrammetry for autonomous feature extraction and SLAM (simultaneous localization and mapping) techniques suitable for the built environment, it builds a powerful capability for practical applications of the data.

Reliability and robustness of ARS systems over long periods will be an issue, especially in hostile or mobile environments. Equally, the quality of the data acquired over long periods will also need to be validated. Software standards are also required for effective validation and verification of ARS systems. Validation methods and processes for autonomy are not good enough at present. While the automotive and aircraft sectors use utilisation time (just drive/fly for many hours), this is not possible for many applications. Development of an evidence base that could be shared by all sectors would be of great value.

The wider implications of the adoption of ARS systems have yet to be addressed. Small, closed loop systems used in, for example, aero engines have few wider implications, but large, open loop systems have many societal, legal, trust, security and ethical challenges that have yet to be resolved.

Ryan Bishop's paper in Cultural Politics, "Smart Dust and Remote Sensing - The Political Subject in Autonomous Systems"¹ starts to examine these issues and discusses the failures of ARS systems used in the Vietnam war caused by a lack of understanding of the outcomes of autonomy and the vulnerability to manipulation and deception by the enemy. Cyber-attack and warfare are the contemporary manifestations of these challenges.

And ethical considerations of using ARS systems are yet to be addressed. Where do we still need human judgement in the loop and where can humans bow out safely? There is an important distinction between "decision support" and "decision making" in such systems with significant legal and regulatory implications. These issues will need clarifying early in the development of any "decision making" systems.



Driverless compact tractors perform fully autonomous spraying tasks in a Texas vineyard. Credit: Wikipedia 

Societal acceptance of these systems cannot be taken for granted either. We have already seen major resistance to the deployment of drones and the public use of Google Glasses. Human interaction with and trust in autonomous systems will become a major issue. How easily humans can interact with ARS systems and the accuracy and reliability of the data produced will determine the level of trust we have in them. Performance evidence for users and regulators will become critical in addition to the normal HMI issues that are likely to arise.

Finally, legal issues resulting from actions or decisions based on the use of autonomous remote sensing analysis have yet to be identified and clarified. Certainly, traceability of the data from sensor to answer and validation of each step from acquisition decision making will be essential.

Autonomous remote sensing systems have the potential to improve the performance of many industries through new products or improved productivity and the effectiveness of significant parts of society. However, more than just technical progress will be needed. Challenges in trust, law, security and ethics must be addressed before the world can fully benefit from this emerging technology.

Further information about the work of the Centre for Earth Observation Instrumentation can be found at www.ceoi.ac.uk. The website also includes information on the wide range of projects and programs funded by the CEOI. CEOI Director, Professor Mick Johnson can be reached at mick.johnson@airbus.com.

Robbin Higgons
Qi3 Limited

Robin Higgons is Managing Director of Qi3 Limited. A chemist by training, Higgons is a specialist in international technology marketing, focusing on strategy, marketing, sales, and technology translation. He is heavily involved in leading the private sector side of the business, helping companies to identify new markets via TME's and advising on product development.

References: