ECCopernicus

Using pre-built user environments (VM images)

WEKEO offers several user environments with preconfigured software for data visualisation, data processing and software development in general. This page explains how to use these environments, and what is included in each one.

Pre-built user environments are available when you create a new VM in WEkEO. They have been designed to meet the needs of three user profiles:

  • WEKEO data users, mostly related to the scientific and application domains that typically don’t develop new software but rather analyse data in specific application suites.
  • Researchers or software developers that carry out advanced processing of data or write software code that leverages data accessible from WEKEO.
  • WEKEO developers or administrators that needs special tools for platform building, integration, testing, administration etc.

Available environments

Currently there are 11 pre-built virtual environments, distributed in the following way:

  • A Data Visualization and Analysis (VIS) designed for DIAS data users and available in five Linux flavours: RedHat, CentOS, Ubuntu, Debian, and Scientific Linux.
  • A Developer VM (DEV) also available in five Linux flavours, corresponding to five OSes. They inherit all the software from VIS and include additional software development suites. This is because the user who develops data-centric applications will likely need the same tools to visualise and analyse data as the rest of the users. The developer VM also includes the Tensorflow machine-learning engine, which is considered in par with other development tools. The virtual machine in particular contains the master node.
  • A Tensorflow Processing Node VM (TF) (coming up soon) which only contains Tensorflow and its libraries. If the user wants to scale the processing in Tensorflow it creates several instances of such VM and, properly configured, will plug to the Developer VM containing the Tensorflow master instance as cluster nodes.

For a detailed description on what is included in each environment, please refer to section "Pre-installed tools" below.

Available operating systems

The Operating Systems offered on the pre-built user environments are the following:

Operating systemVISDEVTF
Ubuntu 18.04.02 LTSXXX
Debian 9.8XX
Opensuse 15XX
Scientific Linux 7.6XX
Centos 7.6XX

Using pre-built user environments

In order to create a new VM based on one of the pre-built user environments, follow these steps:

  1. As a registered user, go to Dashboard and click on Virtual Machines.

  2. Navigate to Provisioning > Instances and click on +Add to create a new virtual machine instance.

  3. Select the instance type VM Catalog and click on Next.

    prefix: Selecting the instance type

  4. Configure the instance as per the image below. Version refers to the type of pre-built user environment (the BASE ones are empty, whereas DEV and VIS differ on the tools — see description below). For Plan, dedicated flavors start with wekeo.vmcatalog. The minimum storage size for VM Catalog images is 4 GB.

    prefix: Configuring the new instance

Pre-installed tools in each environment

ItemScopeVISDEVTFDescriptionURLLicense
Science Toolbox Exploitation PlatformApplication suitesXXThis is the community platform that will accompany SNAP and the Sentinel Toolboxes starting now.linkGPLv3
Orfeo Toolbox (OTB)Application suitesXXOrfeo ToolBox (OTB) is an open-source project for state-of-the-art remote sensing. Built on the shoulders of the open-source geospatial community, it can process high resolution optical, multispectral and radar images at the terabyte scale. A wide variety of applications are available: from ortho-rectification or pansharpening, all the way to classification, SAR processing, and much more! All of OTB’s algorithms are accessible from Monteverdi, QGIS, Python, the command line or C++. Monteverdi is an easy to use visualization tool with an emphasis on hardware accelerated rendering for high resolution imagery (optical and SAR).linkApache 2.0
Broadview Altimetry Toolbox (BRAT)Application suitesXXThe Broadview Radar Altimetry Toolbox is a tool designed to use radar altimetry data. It is available in 32-bit and 64-bit versions for Windows, Mac OS X and Linux. The Broadview Radar Altimetry Tutorial and Toolbox is a joint project between ESA and CNES to develop an open source tool (GPL-3) freely available to all the altimetry community.linkGPLv3
PanoplyApplication suitesXXPanoply netCDF, HDF and GRIB Data Viewer Slice and plot geo-referenced latitude-longitude, latitude-vertical, longitude-vertical, time-latitude or time-vertical arrays from larger multidimensional variables. Slice and plot "generic" 2D arrays from larger multidimensional variables. Slice 1D arrays from larger multidimensional variables and create line plots. Combine two geo-referenced arrays in one plot by differencing, summing or averaging. Plot lon-lat data on a global or regional map using any of over 100 map projections or make a zonal average line plot. Overlay continent outlines or masks on lon-lat map plots. Use any of numerous color tables for the scale colorbar, or apply your own custom ACT, CPT, or RGB color table. Save plots to disk GIF, JPEG, PNG or TIFF bitmap images or as PDF or PostScript graphics files. Export lon-lat map plots in KMZ format. Export animations as MP4 video or as a collection of invididual frame images. Explore remote THREDDS and OpenDAP catalogs and open datasets served from them. Panoply is a cross-platform application that runs on Macintosh, Windows, Linux and other desktop computers.linklink
ToolsUIApplication suitesXUser Interface to the netCDF-Java library The netCDF-Java library can read data from a wide range of scientific data formats (more details here). To check that your dataset can be read by netCDF-Java, use the "Viewer" tab in ToolsUI. Either cut-and-paste a URL or file path into the "dataset" field or click on the folder icon and select a file. (If you cut-and-paste, you will need to hit "Enter" to open the dataset.)linkLGPL / (MIT-style) NetCDF C library license
QGIS Geographic Information SystemApplication suitesXXQGIS (previously known as Quantum GIS) is a free and open-source cross-platform desktop geographic information system (GIS) application that supports viewing, editing, and analysis of geospatial data.linkGPL
Python 2.7, Python 3 Programming LanguageLanguages - R + PythonXlinkPSF
R Programming LanguageLanguages - R + PythonXlinkvarious
Interactive Python (Jupyter)Languages - R + PythonXlinkBSD 3-Clause License
Interactive R (Rstudio)Languages - R + PythonXlinkAGPLv3.0
Apache ZeppelinNotebookXlinkApache 2.0
EclipseIDEs & EditorsXlinkEclipse Public License (EPL)
IntelliJ IDEA (Community Edition)IDEs & EditorsXlinkApache 2.0
Visual Studio CodeIDEs & EditorsXlinkproprietary MIT for source code
Atom EditorIDEs & EditorsXlinkMIT
Brackets EditorIDEs & EditorsXlinkMIT
SciPyEssential Python libraries and toolsXlinkpropietary
NumpyEssential Python libraries and toolsXlinkpropietary
PandasEssential Python libraries and toolsXlinkBSD 3-Clause License
IpythonEssential Python libraries and toolsXlinkpropietary
MatPlotLibEssential Python libraries and toolsXlinkpropietary
SeabronEssential Python libraries and toolsXlinkBSD 2-Clause License
BokehEssential Python libraries and toolsXlinkBSD 2-Clause License
SciKit-learnEssential Python libraries and toolsXlinkBSD New
CaretEssential R libraries and toolsXlinkvarious (see R packages license archive)
KernlabEssential R libraries and toolsXlinkvarious (see R packages license archive)
randomForestEssential R libraries and toolsXlinkvarious (see R packages license archive)
nnetEssential R libraries and toolsXlinkvarious (see R packages license archive)
e1071Essential R libraries and toolsXlinkvarious (see R packages license archive)
klaREssential R libraries and toolsXlinkvarious (see R packages license archive)
treeEssential R libraries and toolsXlinkvarious (see R packages license archive)
rpartEssential R libraries and toolsXlinkvarious (see R packages license archive)
MICEEssential R libraries and toolsXlinkvarious (see R packages license archive)
Lasso2Essential R libraries and toolsXlinkvarious (see R packages license archive)
LarsEssential R libraries and toolsXlinkvarious (see R packages license archive)
GbmEssential R libraries and toolsXlinkvarious (see R packages license archive)
BayesTreeEssential R libraries and toolsXlinkvarious (see R packages license archive)
GDALPython remote sensing librariesXlinkvarious
GeorastersPython remote sensing librariesXlinkpropietary
GeoPandasPython remote sensing librariesXlinkBSD 3-Clause "New" or "Revised" License
CLIMAFPython remote sensing librariesXlinkCECILL-2.0
FionaPython remote sensing librariesXlinkpropietary
FmaskPython remote sensing librariesXlinkMIT
ShapelyPython remote sensing librariesXlinkpropietary
CartopyPython remote sensing librariesXlinkLGPL
RtreePython remote sensing librariesXlinkMIT
PySalPython remote sensing librariesXlinkpropietary
NanSatPython remote sensing librariesXlinkGPLv3
SatPyPython remote sensing librariesXlinkGPLv3
SpR remote sensing librariesXvarious (see R packages license archive)
RgdalR remote sensing librariesXlinkvarious (see R packages license archive)
FieldsR remote sensing librariesXlinkvarious (see R packages license archive)
MaptoolsR remote sensing librariesXlinkvarious (see R packages license archive)
RstoolboxR remote sensing librariesXlinkvarious (see R packages license archive)
SfR remote sensing librariesXvarious (see R packages license archive)
RastersR remote sensing librariesXlinkvarious (see R packages license archive)
GdistanceR remote sensing librariesXlinkvarious (see R packages license archive)
GeosphereR remote sensing librariesXlinkvarious (see R packages license archive)
SentinelsatCopernicus / Sentinel libraries and utilitiesXXSentinelsat makes searching, downloading and retrieving the metadata of Sentinel satellite images from the Copernicus Open Access Hub easy. Sentinelsat is both a Python library and a standalone CLI toollinkGPLv3
Sentinel2ProductingestorCopernicus / Sentinel libraries and utilitiesXXSentinel2ProductIngestor is used to download earth observation data provided by the http://www.esa.int/Our_Activities/Observing_the_Earth/Copernicus/Sentinel-2 satellites and store it to Amazon S3linkMIT
Sat-downloaderCopernicus / Sentinel libraries and utilitiesXXA python library for download Satellite Imagery. Currently support Landsat-8 and Sentinel-2A.linkMIT
Sat-apiCopernicus / Sentinel libraries and utilitiesXSat-api is a STAC compliant web API for searching and serving metadata for geospatial data (including but not limited to satellite imagery).linkMIT
sentinelhub-pyCopernicus / Sentinel libraries and utilitiesXXThe package supports Python integration of working with Sentinel Hub services and retreiving data from AWS in SAFE formatlinkMIT
eo-learnCopernicus / Sentinel libraries and utilitiesXXMachine learning workflows, integration with Jupyter Notebook.linkMIT
Peps-downloadCopernicus / Sentinel libraries and utilitiesXXa simple piece of code to automatically download the products provided by the French Sentinel collaborative ground segment named PEPS : https://peps.cnes.fr. PEPS is mirroring all the Sentinel data provided by ESA, and is providing a simplified access.linkN/A
TensorflowDeep / machine learningXXTensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.linkApache 2.0