INSPIRE Community Forum

It seems that there is only one fully compliant solution to serve multiple harmonised datasets trough WFS 2.0

3318 Views

It seems that there is only one fully compliant solution to serve multiple harmonised datasets trough WFS 2.0

Started by Iurie MAXIM Replies (52)

We investigated several solutions to serve multiple harmonised datasets trough WFS 2.0 according to Technical Guidelines for Download Services. We are not discussing about providing access to only one dataset per server, but to provide access to more datasets.

 

Update 8 September 2016: The single compliant solution seems to be Snowflake GoPublisher WFS.

 

Update 9 May 2018: Because Stefania pointed in this tread that that Deegree is able to serve two different datasets for the same XML schema at different endpoints (asking for typeNames=ps:ProtectedSites in both cases) we started some tests.

So, it is still to see if Deegree is able to serve two different datasets based on multiple XML schemas at different endpoints in order to say that Deegree is compliant with Requirement 52, but is a big step forward in fullfiling the Requirement 52. For those data providers that need to serve feature types from one application schema at one endpoint Deegree is fulfiling the Req 52. We did not tested it yet to see if there are not other issues in order to confirm that it is able to serve multiple harmonised datasets based on multiple XML schemas trough WFS 2.0 according to Technical Guidelines for Download Services. Versions 3.4-RC (unstable) are passing the EPSG in URI format test, while 3.3 (stable) are not passing the test as EPSG is provided in URN format. As Deegree versions 3.4-RC are advertised as unstable and as there is no 3.4 stable version, it is not sure if Deegree is passing all the requirments of the Technical Guidelines for Download Services. However it should be noted that Deegree 3.4 version is OCG WFS 2.0 certified. If anyone had tested Deegree against each requirement from the TG, please share this information.

To conclude about Geoserver 2.13.0, for those data providers that need to serve feature types from one XML schema at one and only one endpoint, Geoserver 2.13.0 is fulfiling the Req 52. Geoserver seems not to be able yet to serve feature types from the same XML schema  at different endpoints, even if using the newly created isolated workspaces.

 

Till 8 September 2016 we thought that Go Publisher WFS (version 4.0.2) is not fullfiling the TG Requirement 46 ”Implementations shall conform to ISO 19142 Conformance Class Simple WFS”, more precise the WFS DescribeFeatureType request is not providing the expected result. Daniel Cocanu discovered that this is not related to GoPublisher WFS but it is related to XSD version 3.0 schemas.

 

The second most compliant solution seems to be Geoserver.  Unfortunately it does not fulfill at least one requirement, namely Requirement 52 ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.” Tested on version 2.8.3. 

On third place seems to be Deegree (very good for testing purposes as it alows to serve datasets from the GML stored in memory) and on forth place ESRI ArcGIS for INSPIRE. We did not tested any other solution and we do not know any other solution (probably FME from SAFE can be a solution as well). 

Does anyone knows how to provide separate WFS endpoints for each INSPIRE dataset with Geoserver? 

Does anyone know other solutions to be tested (i.e.: FME from Safe) ?

Deoes anyone know other INSPIRE TG Requirements or Recommandations that cant be fullfiled by existing solutions ?

Isnt'it quite strange that Technical Guidelines are setting requirements that cant be fullfiled by any existing solution ?

Best regards,

Iurie Maxim

Replies

    • Public

    By Iurie MAXIM

    Update 8 September 2016:

    Daniel Cocanu discovered that the issue with DescribeFeatureType is not related to GoPublisher or Geoserver. The DescribeFeatureType does not work on XSD 3.0 versions schemas, but works on XSD 4.0 versions schemas. This is due to the fact that the links provided in the targetNamespace are not resolvable in 3.0 schemas:

    targetNamespace="urn:x-inspire:specification:gmlas:ProtectedSites:3.0" version="3.0"

    targetNamespace="http://inspire.ec.europa.eu/schemas/ps/4.0" version="4.0">

    Therefore we closed the CASE 00001948. 

    We changed as well the title of the post into "It seems that there is only one fully compliant solution to serve multiple harmonised datasets trough WFS 2.0"

    Original post looked like:

    <<Regarding GoPublisher WFS, the DescribeFeatureType request:

    http://inspire.biodiversity.ro/WFS/RO_ENV_PS/wfs?service=wfs&version=2.0.0&request=DescribeFeatureType&typename=ps:ProtectedSite 

     

    is not providing the expected result, as Geoserver does:

    http://inspire.biodiversity.ro/geoserver/ows?service=wfs&version=2.0.0&request=DescribeFeatureType&typename=ps:ProtectedSite

     

    Therefore CASE 00001948 was opened at https://snowflakesupport.force.com/portal/>>

     

    • Public

    By Iurie MAXIM

    Geoserver unfortunately does not fulfill at least one requirement, namely

    Requirement 52 ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.” is not fullfiled.

    First of all it seems that Geoserver cant provide multiple WFS endpoints. A work around is to make multiple Geoserver installations on the same machine, but of course the server must have a lot of resources for this, and of course this is not a desired solution.

    Another workaround is to use virtual services. However only one dataset can be provided for each data scheme, so if multiple data providers will use the same server, they will not be able to provide two different datasets for the same data scheme (example protected sites for nature conservation from Ministry of Environment and protected sites for archeology from Ministry of Culture) because both of them need to share the same ”ps” endpoint.

    Example of virtual services as endpoints:

    http://inspire.biodiversity.ro/geoserver/ps/wfs?

    http://inspire.biodiversity.ro/geoserver/au/wfs?

    http://inspire.biodiversity.ro/geoserver/br/wfs?

    The worst issue related to virtual services is that there is a known bug in Geoserver and secondary prefixes are replaced by nulls in the output GML: https://osgeo-org.atlassian.net/browse/GEOS-4773

    When querying complex features by using virtual service (e.g. using http://inspire.biodiversity.ro/geoserver/ps/wfs?service=wfs&version=2.0.0&request=GetFeature&typename=ps:ProtectedSite&count=1   instead of http://inspire.biodiversity.ro/geoserver/ows?service=wfs&version=2.0.0&request=GetFeature&typename=ps:ProtectedSite&count=1 ) , any secondary namespaces (e.g. base) are replaced by nulls in the output gml.

    Consequently, instead of :

    <gml:identifier ...

    <base:Identifier ...

    Geoserver returns:

    <null:identifier ...

    <null:Identifier ...

    On the other hand, with Geoserver, currently it is not possible to fullfil the TG Recommandation 13 ”The following identifier should be used to identify the Stored Query for serving pre-defined Spatial Data Sets: http://inspire.ec.europa.eu/operation/download/GetSpatialDataSet”, as the following BUG exist https://osgeo-org.atlassian.net/browse/GEOS-5512

    • Public

    By Thorsten REITZ

    Hi Iurie,

    Thanks for the interesting testing information.

    Can you describe what the compliancy issues with deegree were?

    All the best,

    Thorsten

    • Public

    By Iurie MAXIM

    Hi Thorsten,

    We have no ideea how Deegree can pass the Requirement 52 ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.

    Maybe it can be configured to serve each dataset at a different endpoint (without multiple deploys). If you know how, please share this information.

    Even the manual says: "You may have multiple deegree workspaces on your machine, but only a single workspace can be active."

    In any case I do not understand exactly the reason of this TG Requirement 52. Most common is to have one endpoint for all services.

    If testing the request=DescribeFeatureType (that does not work properly for Snowflake GoPublisher WFS as well) on the Deegree Demo services at http://demo.deegree.org/inspire-workspace/services/wfs?service=wfs&version=2.0.0&request=DescribeFeatureType&typename=ad:Address the request will download a file instead of displaying a silimmilar result in the browser as the Geoserver does http://inspire.biodiversity.ro/geoserver/ows?service=wfs&version=2.0.0&request=DescribeFeatureType&typename=ps:ProtectedSite

    Therefore Deegree seems not to pass the TG Requirement 46 ”Implementations shall conform to ISO 19142 Conformance Class Simple WFS” as well.

    Best regards,

    Iurie Maxim

    • Public

    By Katharina SCHLEIDT

    Hiya,

    2 GeoServer bits:

    getting rid of the namespace null problem is fairly simple once one knows how, at least as of GeoServer 2.8 all you need to do is create a workspace for each subordinate namespace you need in your schema. Within this namespace you only have a namespace and workspace xml file providing the necessary namespace info, the rest works

    Also - for providing multiple datasets, you just need to have multiple workspaces. As long as a dataset is within a namespace, everything is good.

    What's still a bit unclear to me is how INSPIRE sees a dataset, i.e. is it just one feature type, or all features belonging to a sensible package. Trivial in most cases as only one main feature, but for things like EF you have quite a few. Wouldn't make sense to split, least not to my view.

    Example are the AQD services (WFS stuff all done with Geoserver):

    http://luft.umweltbundesamt.at/inspire/wfs?service=WFS&version=2.0.0&request=GetCapabilities

    :)

    Kathi

    • Public

    By Iurie MAXIM

    Dear Katharina,

    Of course there there are no nulls in the namespace if virtual services are not used. In the example you gave you are not using virtual services. As I explained, the problem is that Geoserver does not fulfill the Requirement 52 ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.” when serving more than one dataset trough a WFS 2.0 service. In your example it is only one dataset (aqd).

    Even more, if testing your service with the inspire validator at http://inspire-geoportal.ec.europa.eu/validator2/ by simply copying and pasting in the text box the link that you provided (http://luft.umweltbundesamt.at/inspire/wfs?service=WFS&version=2.0.0&request=GetCapabilities) the validator shows a lot of validation issues (it indicates an average conformity of 55%). The aqd dataset does not seems as well to be a harmonised dataset as it does not validates against any provided XSDs for INSPIRE Data Themes listed in annexes I - III of the Directive.

    To implement the Requirement 52 at least two datasets need to be provided via WFS on the same Geoserver instance, each dataset on a separate endpoint, as for example for dataset1 and dataset 2 these two need to be the GetCapabilities requests for the two datasets:

    http://luft.umweltbundesamt.at/dataset1/wfs?service=WFS&version=2.0.0&request=GetCapabilities

    http://luft.umweltbundesamt.at/dataset2/wfs?service=WFS&version=2.0.0&request=GetCapabilities

    Only in this way it is possible to comply to the Requirement 52 that states ”A separate WFS endpoint shall be provided for each INSPIRE dataset thus providing one dataset per GetCapabilities response.

    Of course ”dataset1” and ”dataset2” can be any texts, as for example ”ps” and ”au” (protected sites ,  administrative units).

    I indicated that a workaround to comply with Requirement 52 by using Geoserver is to use virtual services, but a bug exist while doing so.

    Maybe the INSPIRE community can help to solve these bugs at least by voting them at these links:

    https://osgeo-org.atlassian.net/browse/GEOS-4773
    https://osgeo-org.atlassian.net/browse/GEOS-5512

    If more votes will exist, maybe a programmer will try to solve them.

    Best regards,

    Iurie Maxim

    • Public

    By Thorsten REITZ

    Hi Iurie,

    I did a quick check on the two issues in Geoserver. I don't think they will be fixed by upvoting; they touch some fundamental workings of Geoserver and would require a funded effort to get resolved. With one of our customers, we've recently funded fixing of a different issue (support for multivalued xlink:href ClientProperty in GeoServer app-schema). Any ideas? Do you work on this for a customer?

    Regarding deegree, we've used it to provide one endpoint per dataset (WMS only publicly accessible at this time):

    https://deegree-gdi-suedhessen.wetransform.to/services/org.8.48f0797e-c3b2-4004-8d6f-f8c8a9757f84-transformed_wms?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.1.1

    https://deegree-gdi-suedhessen.wetransform.to/services/org.8.a1e99a63-ab39-4fb6-8a90-0b26320805ae-transformed_wms?SERVICE=WMS&REQUEST=GetCapabilities&VERSION=1.1.1

    If this looks like what you need, I'll ask one of the developers to explain the "How".

    All the best,

    Thorsten

    • Public

    By Katharina SCHLEIDT
    • Public

    By Iurie MAXIM

    Hi Thorsten,

    Thank you for the provided links. However as you also mentioned the links that you provided are not WFS 2.0. but for WMS 1.1.1.

    I said that it seems that there is no fully compliant solution to serve multiple harmonised datasets trough WFS 2.0 (as required by INSPIRE for download services).

    It seems that Geoserver has no problems with virtual services even with WFS 1.0 or WFS 1.1 (for simple features) and most probably for WMS as well because there are no secondary prefixes to be replaced by nulls when using virtual services.

    Therefore usefull would be to see WFS 2.0 services from Deegree (for complex features). As I mentioned before, at least the manual says: "You may have multiple deegree workspaces on your machine, but only a single workspace can be active." We were not able to configure Deegree to have multiple active workspaces and we have no ideea how it is possible to serve multiple WFSs, each one at a different endpoint with Deegree.

    However, it would be probably usefull to ask programmers if what they made for WMS on Degree can be replicated as well for WFS 2.0 for providing access to complex feature types.

    I started this topic because I am surprised to see that only one WFS has no issues according to Inspire Geoportal http://inspire-geoportal.ec.europa.eu/proxybrowser/ 

    As regards WMS there are 113 services that have no issues. Still few as well.

    As well there are 2 ATOM download services that have no issues, and 45 CSW services with no issues.

    Having in mind that so litle Network Services have no issues means that some serious problems actualy exist. And those problems that MS are facing can help other MS to overpass those problems or at least to know that they cant be overpased if using a certain solution. You see for example from Kathi's examples how difficult is even to understand that issues exist.

    And just to mention that currently the tests implemented on Geoportal are not validating the data exposed by the Network Services and are not testing all the requirements and recommandations from the TGs.

    We found that the creation of WFS 2.0 services for harmonised datasets that are passing current tests implemented in the Geoportal is not easy at all. Geoserver can be used if only one dataset is provided trough WFS 2.0. Snowflake GoPublisher WFS seems to be the only solution to provide more datasets trough WFS 2.0, but this solution is not too widespread.

    Therefore Member States have real problems to deliver what is required by the Technical Guidelines having in mind the 2017 deadline as no solution currently exist to deliver fully compliant WFS services.

    Another major issue is related to the registries that are missing the values, making imposible to encode codelists values as xlinks for the vast majority of data themes.

    Regarding your question, no, we are not working on this for a customer. We had a project trough which we delivered last year that WFS 2.0 service that is passing all the tests of the EC Geoportal, namely http://inspire.biodiversity.ro/WFS/RO_ENV_PS/wfs?service=wfs&version=2.0.0&request=GetCapabilities

    As you mentioned that a funded effort is required, maybe EC must guide the process because is quite strange to require Member States to implement all the requirements and most of the recomandations set in the TGs withouth ensuring and beeing sure that technical solutions are actualy available.

    Best regards,

    Iurie Maxim

     

     

     

    • Public

    By Iurie MAXIM

    Hi Kathi,

    Just to mention that the requests you made are for WFS versions 1.0.0 and 1.1 while the INSPIRE is requiring WFS 2.0.0 version.

    If you will copy and paste the second link that you provided (http://bolegweb.geof.unizg.hr/geoserver/ugsex/ows?service=WFS&version=1.1&request=GetFeature&typeName=ugsex:GovernmentalServiceExtension&maxFeatures=50)  in the browser and then hit enter you will see the ”null” writen instead of ”base”.

    <ugs:inspireId>
    <null:Identifier xmlns:null="http://inspire.ec.europa.eu/schemas/base/3.3">
    <null:localId>SCHULE_P_ETRS89.1910</null:localId>
    <null:namespace>StadtWien</null:namespace>
    <null:versionId>Version</null:versionId>
     

    Be sure that this result will not validate against the ugs and base XSD schemas.

    Now try the link without accessing virtual services http://bolegweb.geof.unizg.hr/geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&typeName=ugsex:GovernmentalServiceExtension&maxFeatures=50 (use version=2.0.0 and not version=1.1 or version=1.0.0 as the service is providing complex feature types. You may try as well with version=1.1 and 1.0.0 in order to better understand why version=2.0.0 is needed)

    You will see now:

    <ugs:inspireId>
    <base:Identifier>
    <base:localId>SCHULE_P_ETRS89.1910</base:localId>
    <base:namespace>StadtWien</base:namespace>
    <base:versionId>Version</base:versionId>
     

    Now the nulls does not exist anymore, but the Requirment 52 is not anymore fullfiled becasue you do not have anymore a separate endpoint for each dataset, if you need to serve multiple datasets.

    Without using virtual services there is only one endpoint for all datasets, namely http://bolegweb.geof.unizg.hr/geoserver/ows therefore the Requirement 52 is not fullfiled.

    Using virtual services the endpoint is http://bolegweb.geof.unizg.hr/geoserver/ugsex/ows but the request is replacing secondary prefixes with nulls and the dataset does not validate against the XSD schemas.

    Hope that now is clear based on the examples that you provided why Geoserver is not fully compliant to serve multiple harmonised datasets trough WFS 2.0

    You may see that implementations with Snowflake GoPublisher WFS does not have this issue, but it has another less critical as explained at the begining of this discution. Below you may see two datasets served with GoPublisherWFS at different endpoints RO_ENV_PS and ROENVPS (second is used for testing changes before entering in production, but it can be any other dataset):

    http://inspire.biodiversity.ro/WFS/RO_ENV_PS/wfs?service=wfs&version=2.0.0&request=GetCapabilities

    http://inspire.biodiversity.ro/WFS/ROENVPS/wfs?service=wfs&version=2.0.0&request=GetCapabilities

    Best regards,

    Iurie Maxim

     
     

This discussion is closed.

This discussion is closed and is not accepting new comments.

Biodiversity & Area Management

Biodiversity & Area Management

Thematic Biodiversity and Management Areas Cluster. If themes like Protected Sites, Area Management/Restriction/Regulation Zones and Reporting Units, Habitats and Biotopes, Species Distribution, Bio-geographical Regions matters to you, join these groups!