Main focus on AI4EO

The AI and Deep Learning Revolution

AI is a general-purpose technology already transforming the global economy and many business models across industry sectors, including Space. In this context we believe AI has a huge, but still largely untapped potential for EO technology.

Described as the “new electricity” fuelling the fourth Industrial revolution in the “AI for the Earth” report presented at the World Economic Forum in 2018, AI research is a real driver of innovation, future growth and competitiveness for societies and industries worldwide that can bring enormous opportunities.

Currently, AI is in the midst of a true “renaissance”, driven by Moore’s Law and now super-fed by Big Data. It is deeply entrenched in our society and routinely used in everyday life in applications ranging from recommendation engines, language services (e.g. translation, speech recognition), face recognition, virtual assistants (e.g. Siri, Alexa) and autonomous vehicles (e.g. drones, self-driving cars) – transforming the way we work and live.  

What is more, over the last decade, Machine Learning (ML) has undergone a major revolution driven by the unique convergence of large-scale computing capability (e.g. Cloud computing, Graphics Processing Unit (GPU) architectures, High Performance Computing (HPC), easy access to large volumes of data through the Internet, and the availability of new algorithms enabling robust training of large-scale “deep” neural networks.  

Even more recently, DL and emerging Reinforcement Learning (RL) capabilities are becoming the “workhorses” of AI. DL algorithms are indeed bringing dramatic improvements in the automatic recognition of objects, as first demonstrated in 2012 by results for some tasks within the ImageNet Large Scale Visualisation Challenge [image-net.org].  

Specifically concerning EO, so far, AI applications have mainly involved Computer Vision (CV) to interpret and understand very high-resolution satellite imagery, but many other areas such as Earth Science, prediction and Big Data analytics could also benefit.

Therefore strengthening AI capabilities within ESA by leveraging European EO assets (including data and technical expertise) will help the science community and industry to realise the full potential of EO data in delivering socio-economic benefits and, at the same time, allow Europe to position itself in a rapidly changing AI landscape.

ESA White Paper on AI AI4EO portal

When AI meets big EO data

AI and EO is a marriage made in heaven, however with its own strengths as well as weaknesses. In a sense, we are now at an inflection point, at a kind of crossroads of opportunities, whereby on the one hand AI is becoming one of the most transformative technologies of the century, while on the other hand a growing European EO capability is delivering a totally unique, comprehensive and dynamic picture of the planet, thereby generating big open data sets to be explored by AI.  

Due to the rapid increase in the volume and variety of EO data sources, AI techniques become increasingly necessary to analyse the data in an automatic, flexible and scalable way. Today, EO data remotely sensed from space are particularly suited – but at the same time challenging – for AI processing as they are:

• Big in size and volume with terrabytes of data routinely streamed daily from space, which need to be transformed into “small” actionable information. For example, it would take an operator several hundred years to look at the trillions of pixels routinely acquired by Sentinel-2 on a weekly basis;
• Diverse including data from a variety of sensors, from optical (multispectral and hyperspectral) to radar data. Up until now, AI has been mainly applied to optical imagery, in particular at very high resolution by use of traditional Computer Vision techniques (using mainly RGB bands). More work is needed to make full use of all available spatial, temporal and spectral information of EO data at the global scale. E.g. exploiting the full information of the complex nature of radar data within AI schemes, including information on the amplitude, frequency, phase or polarization of the collected radar echoes;
• Complex and physically-based capturing dynamic features of a highly non-linear coupled Earth System. More meaningful data extraction requires integration of physical principles into the statistical approach, and goes well beyond mere automated feature recognition where a wide variety of training datasets are available.

Machines learning algorithms powered by AI are therefore critical to needed to accelerate “insight” into the data but should always be used in combination with domain experts vital to properly interpret the statistical correlations and data. The intersection of AI and EO remains an evolving, but a rapidly growing field. The application of ML to EO has developed rapidly over the last decade, but the emergence of DL has accelerated growth further, as illustrated by the increase in the number of publications. However, although very powerful, DL techniques suffer from their own inherent limitations; they are data hungry while lacking transparency and unable to distinguish causation.

This calls for more research which connects AI expertise to address EO problems. An objective of the Φ-lab is to foster the development of a seamless “EO Data Engine” comprising of AI4EO use cases, data sets and pre-trained algorithms (referred as Apps) that can then be used in a modular way. These Apps can also be chained and regularly updated with new information, much the same as an app on your mobile phone. Developing such a data engine will open new opportunities for innovative business models to provide users with updated information. The Φ-lab, in collaboration across ESA,  provides some elements of this huge and highly complex process.

Let the data talk. Building a Machine Learning software 2.0 stack to query our planet.

The revolution of Machine Learning (ML) is also a revolution in developing software in a radically new way. While traditional software 1.0 relies on “encoding” rules, the new paradigm of software 2.0 relies on learning from data to “reverse engineer” the software. And given the rapid growth in data, this will lead to a transformative way in developing sophisticated algorithms to solve complex problems.

Our overall goal is to accelerate insight into big EO data by fostering the development of a searchable database of our planet using state-of-the-art AI techniques to query and detect “relevant” changes, in a way analogous to what you would do with a search engine for linked data. 

Today, our current EO assets are data rich but label poor, and we are therefore only scratching the surface of what AI can do for EO. New Space companies, such as Planet, are actively working towards achieving such a vision of a searchable planet by combining high-resolution EO data with AI. A growing amount of data is available to fuel the ML revolution further, presenting exciting opportunities to exploring their potential to push the boundaries of AI in the field of EO. 

---

Some AI4EO Challenges

Connecting AI4EO innovators 

There is a pressing need to increase Europe’s strength and positioning in the area of AI research applied to space. In particular, EO big data presents a niche area of the data economy. Much more could be gained from these valuable data by using AI, thus increasing opportunities for a growing variety of applications and services. The investment, research and innovation efforts required to achieve such an ambitious goal are huge, and cannot be achieved alone, but only through teaming and partnership. ESA can play an important role in such endeavors along with other partners at European level.

In this context, and as part of a first element of a wider ESA response, the Φ-lab aims to operate as a “catalyst” enabling a network of AI and EO labs across Europe to advance our knowledge related to AI4EO and develop new innovative applications and solutions with EO. In particular, our vision is to become a kind of “hub” connecting a growing ecosystem of AI talents and capabilities across Europe to tackle a suite of challenges (see below). 

To do so, the Φ-lab is implementing a suite of collaborative schemes supporting the exchange of ideas and talents and developing new prototype solutions, including:

• Internal Research Fellowship opportunities;
• External Visiting Researchers, whereby the Φ-lab welcomes short to medium-term visits from scientists at MSc, PhD and Post-doctoral level working on topics of relevance and being seconded from their host organisation;
• Visiting Professors. Over the next few years, starting in late 2020, a team of visiting professors will provide advice, support ESA efforts and refine the research agenda to be pursued by ESA in general and the lab in particular;
• Industrial activities supporting the development of new AI schemes and pilot demonstration projects. 

---

If you are interested in these opportunities or have ideas to contribute please contact us.

Contact page