Context
	Objectives
	Architecture
	How it works
	Output

Context

Earth Observation (EO) images can contribute in a number of human activities, from the understanding of global phenomena to decision-making processes. EO images are representation of sensor signals in forms acceptable by human perception. Experts in EO sensing and in destination domains (e.g.: agriculture, meteorology, forestry, urbanism, …) join their knowledge and, interacting at semantic level, extract information from the relevant images (those acquired in the area and period of interest), by visual inspection or applying specific algorithms. This process permits to extract the few kilobytes of user interest from gigabytes of data. However it is complex, lengthy, human intensive and expensive. Therefore it cannot be systematically applied, thus limiting the availability of useful information in support to researchers, service providers and institutions active in non-EO domains. This lack of information flow might make processes or decisions more expensive or even impossible and leave relevant phenomena undetected or discovered too late.

Nowadays the situation is getting worse, since larger and larger quantities of higher and higher resolution EO images are acquired from an increasing variety of sensors and stored in archives reaching or surpassing the petabyte size, while emerging big applications (e.g.: change detection, global monitoring, disaster management support, etc.) demand more and more information. In future more automated, direct and human-centred methods should be provided for the information extraction process, which should rely on intelligent (knowledge-based and learning) and easy to use (semantic driven interactions) programming environments.

Automated is a consequence of the continuously increasing data size.
Direct responds to the need to reduce the steps between the user and the information (currently the user expresses his needs to one or more experts, who possibly get in touch with experts in other domains to collaborate in the extraction of the information).
Human-centred brings the focus on systems that could be managed also by non-EO experts via simple (semantic) interactions (like those between human beings).

During last years ESA funded a series of projects in the field:

Knowledge-based Information Mining (KIM)

Knowledge-centred Earth Observation (KEO)

Image Information Mining on Time Series (IIM-TS)

KEO Extensions and Installations (KEI)

Classification Application-services and References Datasets (CARD)

Support by Pre-classification to specific Applications (SPA)

Assessment of European Ortho-rectification and Co-registration Services (OrthoServ)

Open Access Ontology / Terminology for the GMES Space Component (OTEG)

These efforts led during years to design and refine the Knowledge-centred Earth Observation (KEO) prototype system, which permits users to interactively extract relevant features and information from EO data, either through a generic probabilistic technique (KIM) or by means of a modular and scalable Component-based Processing Environment (CPE), and to provide outputs, i.e. valuable information extracted from data, in easily accessible formats.

Objectives

In order to support users in the information extraction process, the KEO prototype aims at providing the following high-level functionalities:

		Ease the access to EO data and relevant information extracted from them
		Provide a large set of tools for EO data processing (bridging the gap between Data and Information)

		Expand the use of EO data by supporting and automating the identification and extraction of information relevant for users
		Encourage the use of a common scientific cooperative environment

Foster the use of standards

Within the KEO environment, the user can:

Create & semantically identify internal / external Processing Components

Create Processing Components from KIM training (also interactive use)

Graphically chain Processing Components into more complex Processing Chains

Store output into Web Servers (WFS, WMS, WCS)

Create and publish Web Services (from Processing Chains or output)

Architecture

The KEO architecture (shown in Figure 1) includes:

The Knowledge-based Information Mining (KIM) subsystem with enhanced functions

The Component-based Processing Environment (CPE) for distributed processing and graphic programming for automated extraction of information from EO images

The user interface client, named KEO Application Operating on Services (KAOS), to access KIM and the CPE

Figure 1

Processing Components can be either Software Modules or Feature Extraction Processors (FEPs). The Software Modules, deployed either on centralised KEO machines (i.e. at ESRIN premises) or on remote machines (e.g. partners’ premises), can be:

Written in JAVA

Written in any other programming language but wrapped by a Command Line Interface (CLI)

Provided as Web Services (WS)

The Software Modules are executed within the CPE only if embedded into FEPs. A FEP can include one or more Software Modules or combinations of other FEPs. The CPE core is a FEP Engine (see Figure 2), which activates FEPs via centralised or remote FEP Actuators according to the way in which they were chained using KAOS.

Figure 2

For development support, testing and validating of KEO, a Reference Data Sets infrastructure (RDS), managed within an GeoNetwork catalogue and accessible via standard browsers, is maintained by ESA. RDS contains collections of heterogeneous material (images, text, photos, digital elevation models, etc.) for specific topics or geographic areas.

How it works

Through the KAOS Client Application, the user can acces to all KEO functionalities. Currently the KEO prototype provides:

KIM - Automatic extraction from images of Primitive Features like:   o Spectral signature o Texture information o Geometric parameters o Discrete Cosine Transform o Semantic pre-classification

CPE - A large number of Processing Components for:   o Calibration and Classification of single images o Objects / Features Detection from single images o Signal Processing o Inter-equalisation and co-registration of time series of images o Change Detection and Hot Spot Monitoring o Basic processing (format conversion, segmentation, etc.)

RDS - Support application development or enhancement in the field of:   o Land Cover (AATSR-like and LANDSAT-like) o Land Use (SPOT-like) o Cloud Cover (AVNIR-2) o Ice Monitoring

Partners and entities expressed interest in KEO, which is installed at different premises (e.g. the Romanian Space Agency). The ESRIN installation is used also to support tests by other partners, like DLR, CNES, JRC.

More details on system use can be found in the KAOS User Manual.

A KEO Tutorial is also available for users in order to ease the access to the system and its functions.

Output

The KEO system prototype deployed at ESRIN is based on the following main components:
 

1. The KAOS Client Application, which permit to access to all system functionalities, including management and administration.

2. The KIM subsystem for Interactive Probabilistic Information Mining, which permits interactive detection of features (with a size compatible with image and ingestion resolution: higher resolution for smaller features at the expense of larger storage). It is possible to train the system to explore image collections for specific features, to obtain relevant image identifiers or feature maps / objects, to store the training for reuse also by others. The trained "feature label" can be associated to a semantic term for its storage and retrieval.

3. The CPE subsystem (Component-based Processing Environment), which permits to create, chain and execute, under the control of a workflow engine, new or available modules for the extraction of information form EO products. These modules can also be created by the system as result of trained "feature labels" (dynamic acquisition of new knowledge). Processing chains can be also published into the SSE as Web Services.
    

Outside the KEO prototype system, but connected to it:

1. Reference Data Sets (RDSs), a set of reference data used to test and validate KEO processing chains on specific applications.

2. OGC Web Servers, where the extracted information can be stored for reuse.

All KEO prototype resources are described in the following Technical Documents.

Further information can be found at http://keo-karisma.esrin.esa.int/keo-home/Welcome.html

KEO Training Course

A Training Course on the KEO system is available at the following link.

 VTT evaluation