INSPIRE Thematic Clusters

missing a Statistical Unit type in the datamodel

539 Views

missing a Statistical Unit type in the datamodel

Started by Bresters PIETER Replies (16)

Recently Statistics Netherlands has published a INSPIRE harmonised Statistical Units WMS and WFS:

They can be reached via:

http://geodata.nationaalgeoregister.nl/su-vector/wms?
and
http://geodata.nationaalgeoregister.nl/su-vector/wfs?

The biggest problem we encountered for creating the harmonized service, was that we could only add one layer with one feature type in the service. In the original service we had one layer for each different SU, but now we had to put them all together. This is not very useful for the users, because they have to use a filter to get them separated again. We added the fields theme and year to be able to filter the separate SU's out again. Luckily, the validator accepted this. You now have to use an SQL query in QGIS to separate the different types again like I show in the picture below:

 

We also used the tessellation attribute to store the SU type and year. It is shown with the example below;

a rel="nofollow" href="http://geodata.nationaalgeoregister.nl/su-vector/wfs?request=GetFeature&version=2.0.0&typeNames=su-vector:AreaStatisticalUnit&filter=%3CFilter+xmlns=%22http://geodata.nationaalgeoregister.nl/su-vector/wfs?request=GetFeature&version=2.0.0&typeNames=su-vector:AreaStatisticalUnit&filter=%3CFilter+xmlns=%22http://www.opengis.net/ogc%22%3E%3CPropertyIsEqualTo%3E%3CPropertyName%3Esu-vector%3Atesselation%2Fsu-vector%3AStatisticalTessellation%2Fsu-vector%3AinspireId%2Fbase33%3AIdentifier%2Fbase33%3AlocalId%3C/PropertyName%3E%3CLiteral%3Eprovincie.2014%3C/Literal%3E%3C/PropertyIsEqualTo%3E%3C/Filter%3E

I'm curious to know how others solved this problem.

Best regards,

Pieter Bresters

Replies

    • Public

    By Francisco CALDEIRA

    Hello Pieter,

    Could not connect to your services using Qgis (2.18.4)  or Gaia (3.4.2).

    http://geodata.nationaalgeoregister.nl/su-vector/wms
    and
    http://geodata.nationaalgeoregister.nl/su-vector/wfs?

    Anyway I implemented more than one harmonized layer (all polygons) in one service with GeoServer (WMS).

    For instance data of the theme Human Health and Safety

    http://inspire.ine.pt/geoserver/COD2014_PT_CONTINENTAL/wms?REQUEST=GetCapabilities&SERVICE=WMS

    Which Software are you using to create the services?

    How did you implemented the complex feature for the WFS? I'm banging my head for days on the Appschema plugin for GeoServer.

     

    Regards

    Francisco Caldeira

     

      

     

     

    • Public

    By Bresters PIETER

    Hello Francisco,

    It is Strange that you can't connect to our services, since I use the same version of QGIS and have no problems with our services. I did manage to show your WMS but I can't add your WFS either. That might be a problem with complex features.

    Our services are published with Geoserver to, although I did not do it myself. They are hosted by www.PDOK.nl.

    I shall ask them wether they encountered the complex feature problem and how they solved it.

    http://geodata.nationaalgeoregister.nl/su-vector/wfs?request=DescribeFeatureType

    might help you understanding how we implemented the model to the WFS

    Regards

    Pieter Bresters

    • Public

    By Bresters PIETER

    My PDOK counterpart gave the following answer to the question about complex features:

    Hi Pieter,

    For the Statistical units we also used the Geoserver appschema plugin for mapping our features from our database schema to a complex type which is exposed in the WFS. We created the appschema manually, and I have to admit that this is not always easy and error prone. Recently we had a look at Hale which is a tool that helps you creating a mapping between schema’s (eg. between  database and inspire XSD’s ). An advantage of Hale is that all Inspire schema’s are preloaded and an appschema for Geoserver can be exported.

    Hale: http://www.dhpanel.eu/humboldt-framework/hale.html

     

    Kr Mark

    • Public

    By Francisco CALDEIRA

    Hello Pieter,

    Thanks for the answers! 

    Using ArcGIS 10.5 I did manage to load your WMS service. It's always nice to see implementations made elsewhere and compare with our information :-), to see if we made good or bad interpretations about what's expected.

    We are using Hale to do the harmonization, and at this moment using hale to create the appschema file like Kr Mark said. I'm quite impressed that Kr Mark is doing it manually, because there is quite a few documentation and examples. 

    I have just realize, with the help of Simon Temple, that my problem with the Appschema and Hale is that a didn't combine 2 schemas (Statistical Units and Human Health), and was working only with Human Health schema.

    I have another questions for you If you don't mind, did you use GeoServer to create the Table Join Service? Is that an Extension developed by PDOK? 

    Regards

    Francisco   

     

     

     

    • Public

    By Bresters PIETER

    Hello Fransisco,

    The proof of concept was based on a plugin on Geoserver hosted by PDOK on a test server. This was created by Thijs Brentjens from Geonovum and Joost Venema from the Dutch Kadaster. More info can be found via:

    https://themes.jrc.ec.europa.eu/file/view/113290/impact-analysis-of-a-table-joining-service

    We are now thinking of creating a TJS in the production environment of PDOK which is not as a plugin on Geoserver, but which does give geoservices created by Geoserver as output.

    Regards Pieter

    • Public

    By Katharina SCHLEIDT

    Hi all,

    a few comments:

    First off - are you using a WFS 2 plugin to try and access via QGIS? I had no problem accessing using this.

    Second: here's a (mostly) INSPIRE compliant VectorStatisticalUnit WFS I put up a while ago with data from eurostat:

    http://31.147.204.152:8080/geoserver/ows?service=WFS&version=2.0&request=GetFeature&typeNames=suv:VectorStatisticalUnit&outputFormat=gml32&count=50

    Also - while I haven't had time to play with inline filters, I have made headway on stored queries (though on PopulationDistribution), following works:

    http://31.147.204.152:8080/geoserver/ows?service=WFS&version=2.0.0&request=GetFeature&STOREDQUERY_ID=GetMeasureStatDist&measure=tps00170

    This work was done in the process of setting up a simple viewer for statistical data from eurostat, more info (i.e. the trick to getting stored queries running on complex features) in the following doc:

    http://datacove.eu/data/documents/DataCoveStatisticalViewerV1.1.pdf

    If you wanna play with our viewer (be gentle! still a pretty bloody prototype):

    http://bolegweb.geof.unizg.hr:2017/danubehack2/pd-viewer/

    As to GeoServer App Schema configuration - while it does cause a few grey hairs, one does get the hang of it eventually (least i have!)

    :)

    Kathi

    • Public

    By Bresters PIETER

    Dear Kathi,

    Yes I'm using the wfs 2.0 plugin in Qgis version 2.18.4.

    I'm not able yo read your WFS services either.

    The error message is gone before I can read it.

    sorry

    • Public

    By Francisco CALDEIRA

    Hello Kathi and Pieter,

    Pieter today did manage to load you WFS in QGIS, I don't think I did anything different but It worked today :-)

    This is an image of what It came out. https://ibb.co/gBizo5

    I noticed that I can't see the complex feaures, like InspireID.

    To Kathi I can't read your wfs either, using Gaia and Qgis (same version as Pieter). Althoug I can see the error message:

    Camada não é válida: A camada outputformat='gml32' restrictToRequestBBOX='1' srsname='EPSG:4326' typename='suv:VectorStatisticalUnit' url='http://31.147.204.152:8080/geoserver/ows?version=2.0&typeNames=suv:VectorStatisticalUnit&count=50' url='http://31.147.204.152:8080/geoserver/ows?version=2.0&typeNames=suv:VectorStatisticalUnit&outputFormat=gml32&count=50' version='auto' table="" sql= não é uma camada válida e não pode ser adicionada ao mapa

    I Know it's in Portuguese, but it says:

    Invalid layer: The layer outputformat ....sql= It's not a valid layer and It can't be added to the map.

    It's the same message for all the layers, and there are quite a few in that WFS.

    I can't access to your viewer either at:

    http://bolegweb.geof.unizg.hr:2017/danubehack2/pd-viewer/

    I'm almost there for the Appschema, I will post when I have something.

    Both to Kathi and Pieter thanks for sharing your thoughts and materials, it's gold for me! :-)

    Regards
    Francisco

    • Public

    By Katharina SCHLEIDT

    Hi Pieter,
    here I'm using qgis 2.14.7, Client for OGC Web Feature Service 2.0 Version 0.9.7, all work (both yours, mine and the ones from PT)
    for the simple ones (su-vector:SU.VectorStatisticalUnit) not even that is necessary.

    One quick side note - in su-vector:AreaStatisticalUnit you have problems with missing gml ids in the time periods, there's an easier encoding that needs less IDs (in the case i copied out without clear end, but also possible):

    <gml:TimePeriod gml:id="TPRAT">
        <gml:beginPosition>2012-12-31Z</gml:beginPosition>
        <gml:endPosition indeterminatePosition="unknown"/>
    </gml:TimePeriod>

    not sure why your filters aren't working, try and avoid restful encoding as must be a mess (the filter you have further up in the post can't work, syntax is scewed, initial filter tag not closed...). I tried to put up a stored query, but you're machine isn't allowing on WFS2 (pasted in below).

    Do believe having more data per layer and proper filters would make the data easier to handle; if you split it out by layers you're really just pushing the filtering problem elsewhere, don't think that the information available in the metadata will suffice. Reason I'd like some standardized stored queries added to the INSPIRE data specs

    :)

    Kathi

    <?xml version="1.0" encoding="UTF-8"?>
    <wfs:CreateStoredQuery service="WFS" version="2.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    xmlns:xlink="http://www.w3.org/1999/xlink"
     xmlns:wfs="http://www.opengis.net/wfs/2.0"
     xmlns:fes="http://www.opengis.org/fes/2.0"
     xmlns:gml="http://www.opengis.net/gml/3.2"
     xmlns:su-vector="http://inspire.ec.europa.eu/schemas/su-vector/3.0"
     xmlns:base33="http://inspire.ec.europa.eu/schemas/base/3.3"
     xsi:schemaLocation="http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd http://inspire.ec.europa.eu/schemas/pd/4.0 http://inspire.ec.europa.eu/schemas/pd/4.0/PopulationDistributionDemography.xsd http://www.opengis.net/gml/3.2 http://schemas.opengis.net/gml/3.2.1/gml.xsd"
     >
      <wfs:StoredQueryDefinition id="TestQueryKathi">
        <wfs:Parameter name="ID" type="xsd:string"/>
        <wfs:QueryExpressionText
         returnFeatureTypes="su-vector:AreaStatisticalUnit"
         language="urn:ogc:def:queryLanguage:OGC-WFS::WFS_QueryExpression"
         isPrivate="false">
          <wfs:Query typeNames="su-vector:AreaStatisticalUnit">
            <fes:Filter>
    <fes:PropertyIsEqualTo>
                <fes:ValueReference>su-vector:AreaStatisticalUnit/su-vector:inspireId/base33:Identifier/base33:localId</fes:ValueReference>
     <fes:Literal>${ID}</fes:Literal>
                 
                </fes:PropertyIsEqualTo>
            </fes:Filter>
          </wfs:Query>
        </wfs:QueryExpressionText>
      </wfs:StoredQueryDefinition>
    </wfs:CreateStoredQuery>

    • Public

    By Katharina SCHLEIDT

    Hi Francisco,

    > I noticed that I can't see the complex feaures, like InspireID.

    you need the right plugin!

    As for the WFS not working - devil is in the details (as always)! looks like you somehow managed to drop the request type in the request (least its missing in the error message)

    http://31.147.204.152:8080/geoserver/ows?version=2.0&typeNames=suv:VectorStatisticalUnit&count=50 (from the error message) can't work

    http://31.147.204.152:8080/geoserver/ows?service=WFS&version=2.0&request=GetFeature&typeNames=suv:VectorStatisticalUnit&outputFormat=gml32&count=50 is returning data (at least for me)

    side question to my viewer - is it not opening at all? I know something is currently going strange, not all features are being displayed, but thought I'd gotten the basics stored on all browsers - what are you using?

    :)

    Kathi

     

     

     

Statistical Units [SU]

Join this group if you would like to share knowledge or ask questions regarding the INSPIRE implementation of Statistical Units [SU] data theme.
Sub-Group of Statistical Cluster