QGIS on Windows: Oops … Could not load qgis_app.dll message

Sometimes after a Windows update, or after a QGIS update Windows users see the dreaded “Oops, looks like an error loading QGIS’… Could not load qgis_app.dll…” message

In short it means that one of the main libraries of QGIS cannot be fully loaded, because it is actually depending on other libraries, which (apparently) are not OK or available.

The 2 main reasons (I am aware of, please correct me if I am wrong), actually depend on your type of QGIS installation:

1) if you have installed QGIS with the “QGIS Standalone Installer” (the .msi version), the reason can be that the ‘opencl.dll’ version or install is messed up by Windows. For example see: https://answers.microsoft.com/en-us/windows/forum/all/opencldll-is-missing/de5a9687-c53d-4f33-8b28-47dc8115e745 As you can see it is not only QGIS having issues with it.

The solution is to make sure you have the good copy of opencl.dll either in the QGIS install dir OR in the c:/windows/system32 directory. See https://github.com/qgis/QGIS/issues/44806#issuecomment-908976764 for more info about it.

OpenCL is a framework to make it possible to share/move calculations to your graphics-card for parallel/faster processing: https://en.wikipedia.org/wiki/OpenCL.

(also check the second point below)

2) if you have installed QGIS using the OSGeo4W network Installer, the opencl.dll can also be a problem, BUT in that case there is sometimes an issue in which the scripts gis-bin.env or qgis-dev-bin.env files are vanished from your system. Those env files (in the QGIS/bin directory) are there to define the right PATH’s for QGIS and it’s libraries to find other libraries or elements needed.

As said sometimes (virus scanners?) do remove that script(s, one of each version of QGIS), so be sure those are there, or copy them from an other QGIS installation.

3) another opencl and QGIS related issue, see https://github.com/qgis/QGIS/issues/45507, is the one which tells you: “Can not find procedure entry point clCreateCommandQueueWithProperties in the DLL file C:\OSGeo4W\apps\qgis\bin\qgis_core.dll” or similar.

The solution to that is also to copy the right opencl.dll to make it available to QGIS again.

Hope this helps some people.

New PDOKServicesplugin (dutch public OWS services plugin)

Mostly interesting for dutchies: there is a new version of the PDOKServiceplugin, a plugin which makes it easier to set a WMS/WFS/WCS layer into QGIS from our national OWS services: PDOK.

Best addition for now: free High Resolution images of almost the whole of The Netherlands. To show of the old and the new version:

Wanne try yourself? Either install the plugin and search for “Luchtfoto 2021 Ortho HR” OR use the service url: https://service.pdok.nl/hwh/luchtfotorgb/wmts/v1_0?request=GetCapabilities&service=WMTS and search for that name (available in several CRS’s but our national one is EPSG:28992)

Institutional, centralized QGIS installation/configuration (QGIS.de)

QGIS more and more get’s to be installed ‘organisation wide’ by Windows Administrators (eg using SCCM, now Microsoft Endpoint Configuration Manager ), instead of personal installations by the GIS-people on their personal machines.

I get more and more questions about this (eg here)

So a short post about this. The good thing is that peeps from the QGIS user group Germany wrote a (also in english translated) page with A LOT of information and tips about how to do this. So I’m not going to copy that.

So for everybody wondering how to install QGIS with standard configuration or plugins:
READ https://qgis.de/doku.php?id=site:deployment:zentral_en . Thanks QGIS.de!

If you wonder about the loading order etc of all those settings, have a look into the code around here

Python development with PyCharm

Another pointing to… if you develop Python Plugins, and use PyCharm as IDE, read: about how to setup your IDE properly so PyCharm finds all QGIS and PyQt functions. Thanks Tudor!

Testing QGIS fixes on Windows

And as I am now crosslinking away, have a look at: https://github.com/qgis/QGIS/issues/39081
It shows the example about somebody having an issue, being fixed, and then being able to immediately test AND use the fix. By downloading that precise fixed QGIS. Very nice if you really really need that fix for your work TODAY. Disclaimer: in those QGIS applications NOT everybody is working (most notably Python stuff), it is not a full QGIS build like the ‘nightly builds’. Which are full installations of current development branch.

Happy QGIS-ing

Versie 3.5.0 van de PDOKServicesPlugin

OK, this is a short post to let the dutchies know there is a new version 3.5.0 of the PDOKServicesPlugin to load some of our national data as WMS, WFS or WCS.
For example: the GeoMorphological map of The Netherlands, or the data of the accidents that happened from 2008 till 2018. Here combined in one scary map of my surroundings:screenshot-20200619162849-1086x634 Read More

(English) Proj: Select Datum Transformations for EPSG:28992

(FOR REFERENCE, TODO: TO BE UPDATED AND TRANSLATED)
If you startup QGIS 3.8 / Zanzibar the first time to load a data in our national CRS (EPSG:28992) you are being presented with the following dialog:
projqgisveranderingen
I thought it had something todo with the fact that this OSGeo4W install maybe used the newer PROJ (6.0.1), but the About box of QGIS shows:
Compiled against PROJ5.2.0
Running against PROJ Rel. 5.2.0, September 15th, 2018

Checking my older version of QGIS I found the proj/Transformation string used there was:
Proj4: +proj=sterea +lat_0=52.15616055555555 +lon_0=5.38763888888889 +k=0.9999079 +x_0=155000 +y_0=463000 +ellps=bessel +towgs84=565.2369,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812 +units=m +no_defs
The first item in the dialog looks most like it…
Strange thing is that the first item is colored GREEN, so it looks like that is the preferred or current one?
But reading the information (from PROJ), you see:
+towgs84=565.237,50.0087,465.658,-0.406857,0.350733,-1.87035,4.0812
EPSG Transformation Code: 15934
Source CRS: Amersfoort
Destination CRS: WGS 84
Remarks: Parameter values from Amersfoort to ETRS89(3) (tfm code 15739) assuming that ETRS89 is equivalent to QGS84 withing the accuracy of the transformation.
Replaces Amersfoort to WGS 84 (2) (code 1672)
Replaced by Amersfoort to WGS 84 (4) (tfm code 4833)

+towgs84=593.16,26.15,478.54,-1.3044,-0.1033,-1.1445,4.0775
EPSG Transformation Code: 1112
Source CRS: Amersfoort
Destination CRS: WGS 84
Remarks: Replaced by Amersfoort to WGS84 (2) (code 1672).

+towgs84=565.04,49.91,465.84,-0.409394,0.359705,-1.86849,4.0772
EPSG Transformation Code: 1672
Source CRS: Amersfoort
Destination CRS: WGS 84
Remarks: Parameter values from Amersfoort to ETRS89(1) (code 1751) assuming that ETRS89 is equivalent to WGS84 within the accuracy of the transformation. Replaces Amersfoort to WGS84 (1) (code 1112). Replaced by Amersfoort to WGS84 (3) (code 15934).

+towgs84=565.417,50.3319,465.552,-0.398957,0.343988,-1.8774,4.0725
EPSG Transformation Code: 4833
Source CRS: Amersfoort
Destination CRS: WGS 84
Remarks: Parameter values from Amersfoort to ETRS89(5) (tfm code 4830) assuming that ETRS89 is equivalent to WGS84 within the accuracy of the transformation.
Replaces Amersfoort to WGS84 (3) (code 15934).

So my preliminary conclusion now is: do not use the GREEN (top) one, but the last one, as that states: Replaces Amersfoort to WGS84 (3) (code 15934)..
Ok, Nyall answered my question on the mailing list.
It appears that the transformations QGIS used earlier was a little messy, and he advices lo look into QGIS with proj6. Funny thing is then you are shown just 2 options (again the first one green). I’ll have to compile projinfo to see what that gives…
screenshot-20190627104509-716x467
Anybody better information or reasoning?

(English) QGISnetworklogger plugin or what are QGIS and my service talking about…

Just released a ‘new’ plugin:
QGIS Network Logger, install via the plugin manager of QGIS version 3.6 or higher (https://plugins.qgis.org/plugins/qgisnetworklogger/).
One of the things QGIS is pretty good in is talking to OGC services (WebMapService/WMS, WebFeatureService/WFS etc etc), QGIS even talks to Esri web services.
Something what was hard in this, is that if you are an average Windows user of QGIS, with a build without debug information, you were never able to see WHAT exactly QGIS was talking about with the server…
So when QGIS fails to show something on your map, it was not always clear if it was a data problem, a QGIS problem or ( very often 🙂 ) a service issue.
With this plugin, made possible by work of several core-devs in QGIS 3.6 or more, you can ‘justs’ see all the GET/POST/PUT/DELETE requests that QGIS is firing off to the servers. If you are a WebDeveloper: it even listens to F12 (for others: that is the key to show the web-console in most browsers, which also show you internal information of traffic and web pages).
There is a lot to see:
Information and context menu
You see all url’s being fired, the HTTP Operation, status, query, headers from Request and Reply and data/content from the Request etc etc.
You can even ‘replay’ or try out a request in your browser or terminal, by using the context menu on a url:
open
In this example you can see that you can copy a request as a CURL (https://curl.haxx.se/) command to fetch the data or image again.
OR if it is a GET url, you can just open it in your default browser.
The work was able because in QGIS 3.6 Nyall et al. changed some code, so it was easier to ‘listen’ to all the requests that QgsNetworkManager is doing. So this plugin just listens to the requests and show them in the window.
So as long as the provider (or plugin) is using the QgsNetworkManager interface to connect to online services, you will see the requests flying by in the logger window. I hope this is an argument for plugin builders to not use Requests anymore ;-), but stick to our own network machinery.
As said: this would not have been possible without help of Nyall and Alessandro. Nyall also helped with the treeview in my first rudimentary version.
That treeview also costs me some hair though. Because of the asynchrounous nature of the web, AND the fact that you can filter the requests, I bumped in all kind of threading issues…. The tree(view) is being filled, sorted, changed, updated etc etc so fast, that I suffered a lot of crashes… BUT I think I managed to make it stable now.
If not: let me know: https://github.com/rduivenvoorde/qgisnetworklogger/issues
Also if you want to pick up one of the Feature Requests we already have 🙂
Happy QGISsing.

About Layer tree embedded widgets and have your WM(T)S always crispy sharp

Around 2014/2015 Martin updated the whole Legend / Layermanager code in QGIS. He wrote some nice blogs about this new “Layer Tree API”: Part 1, Part 2 and Part 3 so people would better understand how “to talk PyQGIS to the Legend”.
In 2016 Martin merged some code on top of this, which would make it possible to create so called ‘Layer Tree embedded widgets’.
In the image below you see an example of this: a little opacity slider which can be used there to change the opacity of the Layer to which it is connected visible in the Layer Manager.
Such embedded widgets are inserted when you move then in the Layer Properties/Legend tab from ‘Available widgets’ to ‘Used widgets’, ONLY for the layer you are viewing the properties.
embeddedwidget_gui
Read More

WCS QGIS en PDOK

Rant?
This WCS investigation started off because my PDOKServicesPlugin was not working for the PDOK WCS services anymore. I wanted to see all fired network requests, so am creating a future plugin for that: https://github.com/rduivenvoorde/qgisnetworklogger. But I hit all kind of issues when trying out the service(s). This is some write up of my findings and hopefully is interesting enough for some people interested in WCS’s….
WCS?
WCS is the WebCoverageService, an OGC api to serve Coverage data. In OGC terms a Coverage is “raster-data”: like geo tiff’s, ecw’s being rasterized, DEM’s (Digital Elevation Models) etc.
Our national opendata service: ‘PDOK’ has an WCS endpoint to the AHN (our national DEM available at max 50cm, though governmental agencies can go up to 10cm resolution).
One of the drawbacks of WCS is that there are already so many different versions: 1.0.0, 1.0.1, 1.1.2, 2.0.0, 2.1.0 etc etc
QGIS WCS provider supports the versions: 1.0.0, 1.1.0, 1.1.2.
How?
WCS is part of the OWS-family of api’s: the ‘OGC Web Services’.
To determine the capabilities of a service you use (just like every other OWS service) a ‘GetCapabilities’-request, like:
https://geodata.nationaalgeoregister.nl/ahn3/wcs?request=GetCapabilities&service=wcs&version=1.1.2
This shows you the general information of the service, AND which datasets are available.
To fetch more information about one dataset, we use the ‘DescribeCoverage’-request, like:
https://geodata.nationaalgeoregister.nl/ahn3/wcs?request=DescribeCoverage&service=wcs&version=1.1.2
(in newer versions of the WCS you also have to add the CoverageId to the request)
After you received this information (well, actually nothing more then some crs, layernames and dimension information, I’ve never understood why you would not add data statistic in this also…) you are ready to actually GET the data with a so called ‘GetCoverage’-request. In this request you set the desired data type.
And example to retrieve a part of our 5m National DEM around Haarlem as a tiff raster image:
https://geodata.nationaalgeoregister.nl/ahn3/wcs?SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&FORMAT=image/tiff&COVERAGE=ahn3_5m_dsm&BBOX=99198.4917734710033983,485588.33618855383247137,106915.96004800940863788,494366.81641667388612404&CRS=EPSG:28992&RESPONSE_CRS=EPSG:28992&WIDTH=450&HEIGHT=512
Save this to disk and you can automagically load it in QGIS, because this is besides a true tiff-image also a geotiff, meaning it has some metadata in it keeping information about the crs (Coordinate Reference System/Projection) and it’s scale and place on earth:
ahn3tiff
As you can see, for example by using the identify tool, in this image there is just one ‘band’: Band 1. Where I clicked the data shows 43, note that this is NOT the height there, but the RGB value of the pixel in the position, going from 0 to 255. So this is NOT the actual data?
You can also load the following url in your browser to ask the service to provide a png image:
https://geodata.nationaalgeoregister.nl/ahn3/wcs?SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&FORMAT=image/png&COVERAGE=ahn3_5m_dsm&BBOX=99198.4917734710033983,485588.33618855383247137,106915.96004800940863788,494366.81641667388612404&CRS=EPSG:28992&RESPONSE_CRS=EPSG:28992&WIDTH=450&HEIGHT=512
Saving THAT one to disk and loading it in QGIS shows you that it will NOT be placed nicely in the right position of the map. That is because a png image does NOT contain the mentioned metadata (you can provide it as a separate worldfile next to it though).
When you use QGIS to look into the png image, you will find 3 bands: 1,2 and 3 (actually RGB) all holding the same values.
So the service encoded the (actual) raster data into some kind of image where the values are spread over the range 0 till 255…. It is still possible to have some general view of the height variations in this area (actually not that much, besides the dunes in the west 🙂 ) by providing some styling to the png or tiff:
ahn3png
But what are the real data values of the cells of the raster?
To find that you can (either use the capabilities url and load the service via QGIS, OR use the following request, which should get exact the same data but encoded as 32 bit floats in a tiff (GEOTIFF_FLOAT32):
https://geodata.nationaalgeoregister.nl/ahn3/wcs?SERVICE=WCS&VERSION=1.0.0&REQUEST=GetCoverage&FORMAT=GEOTIFF_FLOAT32&COVERAGE=ahn3_05m_dsm&BBOX=99198.4917734710033983,485588.33618855383247137,106915.96004800940863788,494366.81641667388612404&CRS=EPSG:28992&RESPONSE_CRS=EPSG:28992&WIDTH=450&HEIGHT=512
NOW you receive actual data, data in every cell has it’s (modelled) height in meters, ranging from -4.485 meter below seelevel (in the lower right corner) upto 55.5212 meter: I think some ‘high’ dunes in the west or maybe some tower 🙂
ahn3floattiff
Mmm, not sure what is going on here, because measuring the cellen, they are not 0.5 meter x 0.5 meter ???
Let’s try the actual service then: use the GetCapabilities service url from above, create a WCS connection and request the ahn3_05m_dsm (raw data) in the format FLOAT32. This takes some time but NOW you will have 0.5×0.5m cells (though it takes some time to retrieve the data….
Open the Layer props, and from the ‘Render type’ dropdown just select ‘Hillshade’. Zoomed in at Haarlem, you will see:
hillshade
Nice and easy he? The ‘mountain’ in the middle is an old ‘landfill’, now ‘recreation area’, 17 meters high.
So?
What are the issues I hit with WCS?
WCS as Standard
– There are just too many variables and options in the ‘standard’. Because of that both WCS client-implementations AND WCS server-implementations often do not play well together. In my case the server change at PDOK from Geoserver -> Mapserver made the Formats totally different. NOT so much a problem if you have a highly configurable client like QGIS. But if you have a more simple client (as my plugin, or some OpenLayers or Leaflet plugin) it would be good if those formats stay more or like the same.
– All the capability documents are (very) different in the different versions
– So: please OGC: Keep It Simple and backward compatible please!
WCS and Server and Services
– the idea is that a WCS is able to server actual data. Why then are most server implementations also serving those image/rgb format (why a PNG with values of 0 – 255 if you want to know the height???). IF you want something like that, use a WMS to serve the data. So PDOK, maybe just only server the FLOAT32 version? As the height/data is in meters so having integers makes the height-resolution rather low.
– why do you not get back the real data if you request an area in float32 in a browser?
– Not sure how realistic it is to server a DEM from a small country like NL via WCS. As you can imagine the actual data is HUGE, so I was thinking: why did we not come up with WCTC (a Tiled version of the WCS) ???
WCS as client
– looking at all the requests QGIS is firing to the server, it seems like it tries to find out some stats first (for the whole dataset), killing the WCS or at least timing out pretty often. Not sure how valid the stats are from the whole extent (but created from the pyramid by gdal?). Anyway: in my view QGIS fires way to much requests to the server. In case of a tiny tiff serving as wcs that is not a problem, but the AHN/DEM of NL is just to big for that. WHY is that raster-statistics info not available in the GetCapabilities or DescribeCoverage response?
PDOK serving WCS’s
– why is PDOK (or it’s data-providers) using different values for the NODATA value for every different service. Can we try to use some standard for that (at least between those services of the same kin)? Like -9999 (as we know nowhere in NL we go that low…).
– should the AHN not being served as float32 only (and a WMS next to it if you want images)?
Conclusion
Serving the AHN as WCS is doable, but you really have to be patient, raise your network timeout value, hope that PDOK internally does not time out, find out about the data first by a lot of trial and error and THEN you can have fun.