INSPIRE Thematic Clusters

How to deal with huge volume of coverage data - Case 2. Delivery through WCS

633 Views

Source

Workshop "Implementation and potential of INSPIRE coverage data and WCS" - 30th September 2016 - INSPIRE Conference 2016, Barcelona.

Summary

WCS is a service explicitly designed to deliver and exploit coverage data.

For user convenience, data providers may offer all data files based on a common grid (common space allocation defined in a specific CRS, with predefined resolution levels) and range type, as one single coverage. Users have the opportunity to request only a subset of it. Nevertheless, the coverage may be tiled internally by some server for increasing efficiency of data access.

WCS allows hiding the complexity of tiles from the user so that any subset can be requested and is internally – in the server – reassembled to the desired result. Such a result can be internally tiled again when using some suitable data format such as TIFF which supports internal tiling.

EXAMPLE:  the Figure below shows an example where some 3-D x/y/t image timeseries object has some cubical tiling in the server; when requesting a 2-D timeslice, the result might be delivered – depending on the implementation and user requests – as one single raster or as a 2-D tile set.

 WCS - Server-side tiling (right) from which a request (left) extracts some rectangular subset which may be delivered in one tile or in some independent, client-oriented tiling

Figure - Server-side tiling (right) from which a request (left) extracts some rectangular subset which may be delivered in one tile or in some independent, client-oriented tiling.

Thus, the complexity of tiling schemas is hidden from the user who always just sees “a coverage”.

With the new CIS v1.1 standard, user may exploit the data according their needs – it supports flexible management of space (tiling), time, range format, etc. This enhances handling and exploiting raster data.

Regarding efficiency issues with WCS:

  • WCS services have proven to scale up to 250 TB serving 1-D to 5-D spatio-temporal data sets. At the workshop on-demand access and processing of coverages has been demonstrated, with a good response time (often, of sub-second order).
  • If any efficiency issues emerge from a specific implementation, an appropriate tuning shall be performed to the service or data architecture in order to achieve efficiency, as it was also mentioned in the case of ATOM or WFS.

EXAMPLE:  An easy practice would be to limit the maximum size of data volume that can be requested by a single ‘GetCoverage’ request. Note that, with WCS multiple hundred GBs can easily be requested by the user without noticing the volume they are actually requesting. This makes totally sense, since - in general - nobody wants to work with data files with sizes bigger than the RAM of the computer - In the times of Big Data, the uniformly accepted approach is to minimize data transfer and rather perform processing on server-side, close to the data.

In case of using WCS as a delivery option:

  • There is neither need to split the data nor set up guidelines on how to do so. This is dealt by the WCS server.
  • Rather, each data provider should adjust its WCS server parameters taking into account technical specifications: e.g. its server architecture, the band width available for distribution to the user, the maximum size of data to be delivered as a result of a single ‘GetCoverage’ request.

Conclusions

  • WCS is the natural way to deliver and exploit coverage data.
  • There is no need to prescribe any tiling on the client side, contrary to the case of delivering coverages data through predefined datasets.
  • It is highly advisable to limit the maximum volume of data that may be requested in a single query. As this is not part of the OGC standard, this has to be done at implementation level taking into account the capacities of the server (bandwidth) and of the client (RAM, storage).
Elevation, Orthoimagery, Reference Systems and Geographical Grids Cluster

Elevation, Orthoimagery, Reference Systems and Geographical Grids Cluster

INSPIRE Thematic Cluster Elevation, Orthoimagery, Reference systems, Geographical grids - Join this group to share your knowkledge, learn and collaborate in solving issues related to the Elevation, Orthoimagery, Reference systems and Geographical grids themes

Navigation