OSGeo Planet

gvSIG Team: Another big lie: interoperability without open source software

OSGeo Planet - 5 hours 20 min ago

001

The interoperability concept is currently on everyone’s tongue. Everything has to be interoperable. It’s necessary, it’s ideal, it’s legal. Implementation of an IT system or solution wouldn’t come to mind to anybody. Or, at least, consciously.

But, what is the interoperability?

From a technological perspective, the Institute of Electrical and Electronics Engineers defined the interoperability as the ability of a system or a product to work with other systems or products without special effort on the part of the customer.

That initial definition have been increased and currently the interoperability is understood as a greater concept. The e-Government Ibero-American Interoperability Framework, together with the European Commission, has defined it with one of the most complete definitions: the interoperability is the ability of disparate and diverse organizations to interact towards mutually beneficial and agreed common goals. It involves the sharing of information and knowledge between the organizations, through the business processes they support, by means of the exchange of data between their respective ICT systems.

Wait…

One moment…

Sharing knowledge? Not only information?

Getting mutual benefits? Common goals?

Doesn’t it sound familiar? Open source software?

Perhaps for that the Interoperability National Schema in Spain (and in other countries there will be similar laws) has sentences like these ones:

  • “The licensing conditions of the applications and associated documentation, and any other information objects which public authorities are holders of intellectual property rights and that they can make available to other public administrations and citizens, without nothing in return or any agreement, they will take into account that the aim pursued is the use and reuse. “
  • “For open source applications, administrations will use licenses that assure that programs, data or information that are shared: a) Can be executed for any purpose. b) Allow to know their source code. c) Can be modified and improved . d) Can be redistribute to other users with or without changes, but keeping these four guarantees.”
  • “Administrations will link application directories for their free reuse…”.
  • “Indicates that the Administrations should take into account the available solutions for free reuse, that can satisfy (all or part) of the needs of the new systems and services or improving and updating those already implemented.”
  • “Public administrations will try to publish the application source code, in development or finished, for their free reuse in order to favor sharing, reusing and collaborating actions, in benefit of a better efficiency.”

That’s interesting, isn’t that?

I’m going to read this: “the aim pursued is the use and reuse”. And “should take into account the available solutions for free reuse”, and “improving and updating those already implemented”.

All of this… how was it?… wait… yes, “in benefit of a better efficiency”.

That’s a wonder! I don’t know why we complain!

Now we only hope that it is achieved and our public administrations leave the Middle Age and bet on open source software, share technologies, invest in improving the existing solutions, and… finally, they are interoperable.

 


Filed under: english, opinion Tagged: interoperability, open source
Categories: OSGeo Planet

QGIS Blog: Report back: 15th QGIS hackfest in Girona, Spain

OSGeo Planet - 11 hours 45 min ago

Time flies when you are having fun! It seems like only yesterday that I was writing about the 14th Hackfest in Gran Canaria. At the end of May 2016, we held the 15th QGIS hackfest! QGIS has been on an incredible journey since the project was started by Gary Sherman 14 years ago, and the fact that the hackfest was held in tandem with the 2nd QGIS International User conference is a testament to the growth and strength of the project.

2nd International QGIS Conference

Isn’t it amazing – we just held the second international QGIS User’s Conference! We really need to give credit to the amazing team who ran a totally seamless operation to organise the event: Gemma Pons, Toni Hernández, Josep Sitjar, Alexandre Busquets, Ferran Orduña, Rosa Olivella, Laura Olivas, & Lluís Vicens

Girona Organising Team

We would like to give a special thank you to the University of Girona’s Director of GIS Service (SIGTE) – Gemma Boix who helped to organised the event as well as ensuring the institutional support for the event. In case I have missed mentioning someone by name, our thanks to all the other volunteers and the sponsors of the conference. Not only did the conference team host the conference event, they also covered a large part of the costs of the hackfest that followed the conference – for which we can’t thank them enough! OSGeo also supported the hack fest financially for which we are extremely grateful.

The conference team also extends their thanks to the attendees and presenters, instructors and developers who also actively participated on the event!  For those interested in viewing the various talks at the conference, here are some handy links:

 

State of QGIS

The QGIS.org project is in a very healthy state right now. For my talk at the user conference (video here) I got some fresh download stats from our servers and the numbers are quite astounding: QGIS 2.8 LTR (release Feb 2015) has been downloaded over 679, 000 times for the Windows Standalone installer (which includes all bug fix releases). Even after controlling for overestimates (from bot downloads, multiple downloads per user, “try and don’t use”) and underestimates (single downloads being distributed to many users) it is clear that we have an extremely large and constantly growing user base. By another metric, our most popular plugin, the OpenLayers plugin has been downloaded over 700 000 times!

Something else that is clear from the make up of the talks, workshop topics and attendees at the user conference and hackfest: QGIS is increasingly moving from single user environments to large multi-user deployments. This ‘edge of network effect’ is a common phenomenon in FOSS and is largely how Linux came to be such a lynchpin in the dev-ops world. Sys admins and (in the case of QGIS) GIS power users, test out the software on their own devices, see the potential for it in their workplace and start integrating it into their workflows in the office until eventually it has become a mission critical piece of software for an organisation.

One thing my slides probably do not make clear is that there is a huge amount of investment being made into QGIS and plugins for QGIS to fulfil a wide variety of needs. These investments are largely external to the project (i.e. not factored into the financial figures I mentioned in the talk) and happen in direct client-to-developer relationships completely bypassing (from a financial sense) the upstream  QGIS.ORG project. This is a really good model since we do not need to deal with contract delivery, competing interests etc.

If we do a simple calculation based on direct QGIS.org revenue for 2015/2016 (around EUR 69, 000) to downloads, the average revenue per download of QGIS 2.8.x was around EUR 0.10. All of the money we receive into the project goes into improving the QGIS, maintaining infrastructure and funding travel and accommodation for hackfests. I mention these numbers both because they are interesting and because it is good to emphasise how incredibly grateful we are to each and every one of our sponsors and donors that support the project. We really do run on a shoe-string budget and we have audacious goals and a vision to put spatial decision making tools into the hands of everyone on the planet who wants to use them. Your sponsorship and donations are a key enabler to making this vision a reality!

By the way, my apologies for not mentioning Gary Sherman (our project founder) by name at the start of my talk – that was totally unintentional! Before I talk about the main activities, let me make a quick aside to mention the small excursion we took:

Underwater autonomous vehicles & QGIS

Natalia

One of the really cool things we did at the hackfest was take a little side trip to visit the Computer Vision and Robotics Institute. Natália Hurtós (who works for the institute) kindly gave a bunch of us QGIS geeks a tour. The work they are doing building [relatively] cheap underwater autonomous vehicles is really awesome and inspiring – all the more so because Natalia is planning to build the mission planning tool using QGIS libraries!

Ok so what actually happened at the hackfest? Lets dig in and find out!

Cool stuff from the QGIS Hackfest

I am only going to focus on the hackfest here because the videos from the talks at the QGIS User Conference have been posted online (see above), so you can take in all the QGIS goodness you like from those. There was a lot going on at the hackfest, so these are only the nuggets I managed to cherry-pick from the talks.

Give processing some love Screen Shot 2016-06-12 at 22.13.15

During the hackfest and user conference, Victor Olaya (lead developer for the QGIS processing framework) really did a great job of promoting the idea of writing processing plugins rather than ‘normal’ plugins. His argument is that most plugins that are intended to provide single purpose analytical capabilities (we are looking at you geeks about to write the 300th buffer plugin!) would  be better off implemented as processing plugins:

  • the plugin author would not need to spend valuable time writing user interfaces, input validators and so on.
  • users of the plugins could chain the tool into complex workflows easily, rather than only being able to use it on a once off basis.
  • we would grow the amount of options available in the processing toolbox while at the same time reducing the sometimes overwhelming amount of choice in the QGIS Plugin Manager.
ILWIS processing tools coming soon

Also on the topic of plugins, Bas Restsios from the ILWIS Project gave a demo of the ILWIS software in order for QGIS developers to be more aware of its capabilities. Although currently Windows only, the ILWIS developers are in the process of porting the software to be based on the Qt5 framework, which means us Linux and OSX users will get to enjoy using it too soon. ILWIS is Open Source and does its rendering using OpenGL. Because of their smart rendering system, everything draws lightning fast. ILWIS packs in many remote sensing tools and raster analysis tools and should be on anyone’s radar if they are interested in FOSSGIS. The best part (from my point of view) is that Bas and his team members are also busy creating a set of processing plugins for QGIS that call out to ILWIS’s command line tools. This means you can expect a huge leap forward in the number of raster based analysis functions you can do with QGIS in the near future.

GeoPackage, JPEG2000, WFS improvements

It was really great to have Even Rouault (maintainer of GDAL/OGR) present at the hackfest. Even recently received core committer rights to the QGIS code repository and has been making great contributions by adding support for the OGC Geopackage format (death to shapefiles!). The shapefile format is long in the tooth and doesn’t serve the GIS community well as a de facto standard for GIS data interchange. It also doesn’t make a good format for intermediate  representation of data processing outputs (e.g. when using multi algorithm processing models). Even has also been working on improvements to WFS support in QGIS which many will appreciate.

Not directly related to QGIS, but Even also mentioned he has been giving the GDAL driver for JPEG2000 some love – gaining good performance increases. This gives me some hope that there will be a viable open format alternative to ECW and MrSid in the future – something which QGIS will benefit from greatly. One of the interesting things that GDAL supports with the JPEG2000 specification is embedded vectors – you can write the raster with gdal_translate and then during creation pass a shapefile or GML stream to the gdal_translate command. I look forward to the day (not currently on the roadmap) where we can ship JP2 images with embedded vector masks and use those embedded vectors  seamlessly in QGIS.

Planning for 3.0 release

We did some planning for the 3.0 release of QGIS and, in particular, fine tuned the plans for how we will manage the transition from QGIS 2.x to 3.x. In February 2016 I posted an outline of the general approach we planned to follow. Some developers felt that we would be better off having the core of the 3.0 transition work happening on the master branch of QGIS so that it has more attention and testing focussed on it. I guess there are two main interest groups to consider here, so I will break down the outcome between take-home points for developers and general users:

For developers:

  • 2.16 gets released off master.
  • After 2.16 we create a 2x branch and the 2x branch will be put into caretaker mode.
  • Support for Qt4 and Py2 will be discontinued in master and we do packaging only against Qt5 and Py3.
  • Once Qt5 and Py3 only are supported in master, nightly build packages may not be immediately available on master as we need to get the packaging systems update.
  • After 2.16 is released API breaking changes will be allowed in master (but code must build and packaging not broken, plugins will be broken).
  • All API breaking changes should be annotated by means of a Doxygen page patch to indicate what changes were made.
  • We target the API change window to end in Feb 2017 so that we can have a 3.0 release in March 2017.
  • We may do a 2.18 release off the 2x branch  if there have been substantial changes in the 2x branch. If there are no substantial changes in the 2x branch following 2.16, we will not do a 2.18 release.
  • In January 2017 we will have a review to establish if all the API breaking changes are complete. If someone has specific implementation plans that are in progress and they need more time, we will (under agreement from the dev community) extend the API breaking window and may push out the release date.
  • Following the release of 3.0 we will implement API freeze again and polish up the codebase for an LTR release based on 3.2
  • 2.14 LTR support will be extended until 3.2 is ready
  • Note: for QGIS 3.x, a minimum of Qt 5.5 is recommended
  • Note: for QGIS 3.x a minimum of Python 3.4 is recommended
  • Note: We are looking for a volunteer to set up a  windows build system using vagrant on win 7 so that we can test and build automaticallyl

For General Users

We propose that for general users, you rather share the following more easy to digest bullet list as it does not contain all the technical details above that mainly developers will care about.

  • 2.18 release is not guaranteed
  • We are planning a 3.0 release in March 2017
  • This release date may be postponed – we will do a review in January to establish whether we are on track for the release date
  • The 3.0 release will break your plugins – we will publish a migration guide and tools to help you migrate your plugin to the new platform
  • If you have any queries about the process please feel free to contact us (via the developer or user mailing lists preferably)
Style repository

Akbar Gumbira, our Google Summer of Code (GSOC) student joined us at the hackfest. For his GSOC project, Akbar is working on a unified way to share styles, symbols, ramps and markers in QGIS. The idea will be very similar to the QGIS plugin repository, where users can host their favourite cartographic elements for everyone else to enjoy. There was quite a lot of technical discussion about the exact mechanisms to use for hosting shared symbology with the key elements of the debate being about whether to use git as a hosting system, simple zip files or some other mechanism. If you wish to chat to Akbar about his work, visit this chat room or comment on the QEP.

OSX Packaging and Building

QGIS is an attractive proposition for OSX users since the big commercial vendors typically don’t support OSX. Since QGIS is built on cross platform technologies, this does not pose a huge limitation for us, but there are few OSX developers in the QGIS developer team and the platform requirements and constraints are pretty complex. Larry Shaffer has been doing awesome work to cut through the various issues and simplify the installation process – both for end users and for developers. It’s finally pretty easy to get a development environment up and running on a Mac now. Using brew (a package manager for OSX) you can install all the needed dependencies and get  QGIS compiling in Qt Creator with full debugging support. The next challenge is going to be supporting this under Qt5 for QGIS 3.0 and Larry has been doing a bunch of great work to make that happen.

Lizmap!

There are several web client frontends for QGIS out there. Lizmap is probably the most feature rich of them. Michaël Douchin (@kimaidou on twitter) showed off the latest version of Lizmap. If you are doing any web mapping, you really should check it out!

QField

Marco Bernasocchi showed off the work they have been doing on QField – an Android app for your mobile device. The workflow for using QField is simple: Create a QGIS project on your desktop, copy it over to your device’s SD Card or internal storage, then open the same project on your device. Since it uses the same QGIS 2.14 backend as you are probably running on your desktop, all the cartographic elements from your desktop are supported – including the new 2.5D rendering. The main use case for QField is field data collection and new in QField is the ability to capture point data and edit / update feature attributes. This opens many possibilities for asset management and field survey work.

Marco shared some roadmap plans for QField including broader support for form elements. Value maps are already implemented, value relations, date and picture support are coming soon!. There is also beta support for digitising lines which should land in your installed version soon. I am eagerly looking forward to seeing how QField develops!

During the hackfest I did some testing of QFied using a PostGIS layer in a project with the data coming from a remote QGIS server – and it works (assuming you have internet connectivity). I also did some testing using BTSync to create a synchronised file system between my mobile device and my desktop. Edits to layers on the desktop and the mobile device can easily be pushed back and forth, making it very very easy to push out maps and new data to workers in the field.

Testing

There were lots of interesting things going on for those interested in testing. Alessandro Pasotti showed off some really cool stuff he has been working on for running python tests directly in QGIS instead of using a mock QGIS iface object. There are huge benefits to doing this since your tests run in  a ‘real’ QGIS environment. He also showed off how they are testing QGIS plugins in Docker using the above mentioned technique.

If you do need / want to use an mock iface object, Matthias Kuhn has been promoting the use of the new qgis.testing python module improvements which includes an iface object which is comprehensive in terms of API stubs. For pythonistas:

from qgis.testing import start_app, unittest 

Matthias has also created a unittest subclass that you can use which includes nice goodies like letting you do asserts that geometries match.

Matthias Kuhn, Nyall Dawson and others have really been leading the charge to build a more comprehensive test suite in QGIS and there were lots of other interesting tips and tricks been shown like how to make your travis tests run against multiple versions of QGIS – which can be very handy for plugin authors.

Victor Olaya also showed off the tester plugin – for automated GUI testing based on recorded interaction sequences.

Nice things for developers

Martin Dobias showed off some of the tools they have been developing including:

  • The report plugin (see http://plugins.qgis.org/plugins/report/) that lets you trap python exceptions and send a bug report directly to a github issue tracker.
  • Martin also showed of the First Aid plugin which lets you capture tracebacks in a more elegant way and also debug your plugin, stepping through the code and execute python instructions in the current run context. The First AID plugin makes a great alternative to using remote debugging from an IDE which is time consuming to set up and technical for less experienced developers to do.

Training plugin

Victor Olaya showed of a plugin (the Lessons plugin) they are working on to facilitate interactive training in QGIS. It uses a simple dock interface to guide the user through a series of activities. Interestingly you can ask it to play recorded macros (saved as python code) of the active being explained so that you can see how it is done if you can’t figure it out yourself. Their authoring system also allows you to record these  macros.

 

Docker

Docker is increasingly becoming a useful building block for those wishing to deploy QGIS in server side contexts. Patrick Valsecchi and Stéphane Brunner did some really awesome work refactoring the QGIS Server docker image – you can get a preview here  https://github.com/pvalsecc/QGIS/tree/docker (read the README for usage notes). Patrick’s work strips down the size of the docker image to make it much less of an overhead to pull the image. We would like to eventually merge this into the upstream QGIS repo so that we have a versioned docker build set up for each QGIS release.

 

Crayfish

Martin Dobias showed off the Crayfish plugin. Crayfish includes some really awesome multi temporal visualisation tools – really useful if you have time slice data in NetCDF or similar formats and you want to view layers sequentially as animations. It also has cool symbology additions to show flow / directionality arrows over your raster.

 

GeoNode QGIS Server Demo

Etienne Trimaille demonstrated the work he, Ismail Sunni and Rizky Maulana have been doing to implement a QGIS Server backend for GeoNode. GeoNode is a spatial content management system that allows you to easily upload and share geospatial data on the web. With their implementation you can upload a shape file or TIFF along with a QGIS .qml style file and the layer is published directly as a tile service in the original QGIS styling. Bye-bye hard do create SLD’s, hello easy to create QGIS styles. (Disclaimer: I work with Etienne, Ismail and Rizky so I am tooting our own horn a bit here).

Documentation

The work of the documentation team is often overlooked in the frenzy to enjoy the new cool stuff that developers churn out. Take a look over at the documentation GitHub pulse page to see how just as much effort is going on in the documentation work. During the hackfest Yves Jacolin and the rest of the documentation team were hard at work getting things ready for the next QGIS Manual release. If you are able to lend a hand with the documentation effort (no coding skills required), please do contact Otto Dassau, the QGIS Documentation Lead!

Lightning talks

Here is also a quick-fire list of features that were shown off -that are coming down the QGIS conveyor belt – mostly for QGIS 2.16 but also new plugins and other efforts:

  • Style dock in QGIS 2.16 with undo / redo – Nathan Woodrow
  • Dynamic hillside rendering – Nathan Woodrow
  • Awesome new gradient editor – Nyall Dawson
  • D3 & Plotly charting with chart interaction showing on the layer – Matteo Ghetta and Michael Douchin
  • Forms improvements: the ‘drag and drop’ editor will support multiple columns – Matthias Kuhn
  • Table view will be able to filter which columns to show (to let you hide unwanted columns) – Matthias Kuhn
  • Action widget columns in the attribute table: you can place one or more action widgets (and icons) into the attribute table making it very easy to fire off an action (e.g. python script) for a particular record – Matthias Kuhn
  • Support for changing column order display in attribute table – Matthias Kuhn
  • In composer you can get have JSON of all layer relations that you can use in your html widget – Nyall Dawson
  • Aggregation in expressions. We will explain this more in the visual changeling for the next release, but you will be able to compute attribute aggregates (e.g. sum of areas) in your reports – Nyall Dawson
  • Default values for fields – Matthias Kuhn showed upcoming work to add support for field defaults in when adding a new record to a vector layer.
  • Transaction groups / cascaded editing mode when you are editing a feature so that related tables get put into edit mode (project property needs to be enabled) – again we will explain this more in the upcoming 2.16 changelog – Matthias Kuhn
  • SectorPlot plugin: a plugin to let you make ‘pizza plots’ of your data – Raymond Nijssen and Richard Duivenvoorde
  • Denis Rouzaud showed off work he has been doing to create a standard settings dialog that you can use in your plugins. It has support for different input types and layer chooser etc. They provide a settings dialog base class that you can derive from in your plugin which will take care of reading and writing your settings to Settings.
  • User profiles plugin – lets you create customised user interface for different user categories – Alex Bruy
  • Georeferenced PDFs and outputs from composer (sorry not yet GeoPDF) but you can use composer to create a  PDF – and then add that into QGIS just like any other raster layer. Nyall Dawson
  • Nyall Dawson chatted about his work on QGIS Task manager – a tool to let you easily create concurrent background tasks and show the progress of each task in a simple UI. I believe he is looking for funding to help him get this finished, so if you are interested please contact him.
  • Bivariate legends plugin (not published yet).  Thomas Gratier showed off work he is doing to support the creation of bivariate legends.

 

Albireo QGIS Fork

albireo

Sandro Mani showed off an awesome new QGIS front end they have been working on at Sourcepole. Their implementation creates a brand dew ribbon based UI paradigm for QGIS (don’t scoff, it’s actually pretty cool!). They have pared down the number of features and grouped functionality into discrete areas, both the menus and ribbon icons changing based on the context of what you are doing. You can see the source code here: https://github.com/sourcepole/kadas-albireo.  Here are some bullet notes I made while watching his presentation:

  • 3D Globe updates to get it ready for production use. They have also integrated the 3D extrusion support and lots of improvements to the globe that Matthias Kuhn did as part of his masters thesis. Its really exciting to think that these improvements may make their way into the official QGIS builds sometime soon and we can all enjoy them.
  • Redlining – you can sketch onto the map without saving those geometries in a specific layer.
  • Measuring & profile tools
  • Shift-drag for zoom – just like you can do in OpenLayers and Leaflet (Nyall then went and added this to QGIS for 2.16 – yay!)
  • Switch coordinate reference systems used for cursor position display easily from the status bar

There were many other cool features – some of them may make their way into the mainstream QGIS desktop. If you are looking to support users who need a simplified QGIS user interface, Albireo QGIS spin off is something to watch.

Conclusion

Girona was an absolutely awesome venue for the QGIS Conference and Hackfest. I hope the participants of the conference gained good benefit from the experience. The hackfest had around 50 participants and beyond the above notes, there was such a lot going on including work on the documentation system, discussions on the proposed community voter system (more on that in a follow up blog post) and many other things. I really encourage you to attend these events if you want to keep track of the leading edge of QGIS developers.

 

 

 


Categories: OSGeo Planet

Cameron Shorter: OSGeo-Live 10.0 beta1 released. Test sprint / Doc update

OSGeo Planet - 20 hours 5 min ago

OSGeo-Live 10.0 beta1 is ready for download at [1]. We're looking for testing applications and updating docs and translations.
In particular, we need help to:
  1. Check if any new features need to be added to Project Overviews
  2. Run Quickstarts and verify they work as described. Please join us on irc://freenode.net#osgeolive this weekend, 2 & 3 July 2016, for our testing sprint.
  3. Update review status in our spreadsheet [2]. (It really helps us know if an application has been reviewed, and by who).
  4. Translate docs that have been updated.
  5. OSGeo-Live is scheduled to be the default installation at FOSS4G 2016 workshops. We strongly urge all workshop leaders to test OSGeo-Live now and provide feedback, while there is still time to tweak anything required for the workshop.
[1] Download: http://aiolos.survey.ntua.gr/gisvm/10.0/ Mirror: http://osprey.ucdavis.edu/downloads/osgeo/gisvm/gisvm/10.0beta1/
[2] https://docs.google.com/spreadsheets/d/1Q5BaEgQtgw4O1bXyeWMlM8XtAOhUgcjZ7Y2O0FZc2H0/edit?hl=en_GB#gid=2014800150What's Changed:New applications:
  • PyWPS 3.2.5
The following applications have been updated:
  • 52nWPS 3.3.1 -> 3.4.0
  • 52nSOS 4.3.0 -> 4.3.6
  • GpsPrune 17.2 ->18.3
  • GeoMoose 2.8.0 -> 2.9.0
  • GRASS 7.0.3 -> 7.0.4
  • GeoServer 2.8.2 -> 2.8.3
  • GMT 5.1.2 -> 5.2.1
  • Iris 1.9.0 -> 1.9.2
  • istSOS 2.2.0 -> 2.3.0
  • Marble 1.9.2 -> 15.12.3
  • Mapnik 2.3.0 -> 3.0.11
  • MapProxy 1.8.0 -> 1.8.2
  • mb-system 5.5.2252 -> 5.5.2274
  • OpenCPN 4.0.0 -> 4.2.0
  • OSSIM 1.8.20 -> 1.8.20-3
  • JOSM 8159 -> 9329
  • Merkaartor 1.18.1 -> 1.18.2
  • OTB 5.2.0 -> 5.4.0
  • pgRouting 2.0.0 -> 2.2.3
  • PostGIS 2.2.1 -> 2.2.2
  • Proj4 4.8.0 -> 4.9.2
  • pycsw 1.10.3 -> 1.10.4
  • QGIS 2.14.0 -> 2.14.3
  • R 3.2.1 -> 3.3.1
  • Saga 2.2.4 -> 2.2.7
  • Spatialite 4.3.0 -> 4.3.0a
  • Viking 1.4.2 -> 1.6.0
  • Zygrib 6.2.1 -> 7.0.0
  • ZOO-Project 1.3.0 -> 1.5.0
Retired applications:
  • Tilemill (not compatible with the latest version of Mapnik included in the disk)
Full changelog:
https://git.osgeo.org/gogs/osgeo/OSGeoLive/src/master/CHANGES.txt
Schedule:
  • 03 Jul 2016 Testing Sprint
  • 11 Jul 2016 English Project Overviews & Quickstarts complete
  • 16 Jul 2016 Translations complete
  • 18 Jul 2016 Release Candidate 1
  • 01 Aug 2016 Final ISO sent to printers
Full Schedule: https://docs.google.com/spreadsheets/d/1kO6zzmLFfprZGgp5x7Sjwi-EVN6NTGDR4KXvFVtNpR0/edit?hl=en_GB#gid=0About OSGeo-Live:OSGeo-live is a Lubuntu based distribution of Geospatial Open Source Software, available via a Live DVD, Virtual Machine and USB. You can use OSGeo-Live to try a wide variety of open source geospatial software without installing anything.
Categories: OSGeo Planet

From GIS to Remote Sensing: Sentinel-2 Download Issues Using the Semi-Automatic Classification Plugin: Solved

OSGeo Planet - Wed, 2016-06-29 12:35

Recently several users of the Semi-Automatic Classification Plugin (SCP) for QGIS reported errors that prevented the search and download of Sentinel-2 images.

The issues were caused mainly by the cryptographic protocols used for accessing data and the way the SCP tries to connect to the Copernics Data Hub (https://scihub.copernicus.eu).In particular, the issues started after some recent changes of the Copernics Data Hub, when the protocols used for accessing data were upgraded (i.e. TLS v1.1+).
The SCP relies on Python for downloading Sentinel-2 images.As described here https://docs.python.org/2/library/ssl.html the protocols TLS v1.1 and TLS v1.2 are available only in Python 2.7.9+ with openssl version 1.0.1+.Unfortunately QGIS comes with a previous version of Python where TLS v1.1 and TLS v1.2 are not available. Therefore the Sentinel-2 download process fails.
Categories: OSGeo Planet

Nathan Woodrow: QGIS style dock – Part 2 – Plugin panels

OSGeo Planet - Wed, 2016-06-29 11:53

In part 1 I talked about the new style dock and the in built functions it has in this post I would like to cover how you can extend that dock using plugin panels.

While building the dock it became evident that being able to add your own panels with be super neat, why lock the user to what styling controls we have built into core.

After one, or two, API iterations it is now possible to add your own panels to the style dock.

First we need to make a new widget using QgsLayerStylingPanel

class CustomPanel(QgsLayerStylingPanel): def __init__(self, layer, canvas, parent): super(CustomPanel, self).__init__(layer, canvas, parent) self.layer = layer def apply(self): pass

then we need to create a factory object (QgsLayerStylingPanelFactory) which will hold the metadata for the panel widget, things like icon, title, and the widget it self as required. The factory object looks like this:

class PanelFactory(QgsLayerStylingPanelFactory): def icon(self): return QIcon(r"F:\icons\SVG\book.svg") def title(self): return "" def supportsLayer( self, layer): return layer.type() == QgsMapLayer.VectorLayer def createPanel(self, layer, canvas, parent): return CustomPanel(layer, canvas, parent) # Also make a instance and tell the style dock about it # make sure you keep a instance of the the factory around factory = PanelFactory() iface.registerMapStylePanelFactory(factory)

You can also see we have a supportsLayer() method. This must return True if the layer can be supported for the widget, if the layer isn’t support it’s not shown in the layer style dock. We also have the createPanel method which just returns the widget, you could also use this method to return different widgets based on layer type if you needed. This createPanel method will be called any time the item is selected in the left panel of the style dock. Using the above code we will see a new icon on the side panel

style

Remember, the create method is called any time this new item is selected.    If your widget is expensive to create you can create a cache here.

Now lets add some logic.  As an example I’m going to load the style xml and show it in the panel. Adding some logic to the __init__ method of the CustomPanel class

class CustomPanel(QgsLayerStylingPanel): def __init__(self, layer, canvas, parent): super(CustomPanel, self).__init__(layer, canvas, parent) self.layer = layer self.setLayout(QVBoxLayout()) # Create the editor and set the xml from the layer. self.editor = QgsCodeEditorHTML() self.editor.setLexer(QsciLexerXML()) doc = QDomDocument( "style" ) rootNode = doc.createElement( "qgis" ) doc.appendChild( rootNode ) iface.activeLayer().writeStyle( rootNode, doc, "") xml = doc.toString() self.editor.setText(xml) self.layout().addWidget(self.editor)

xml

Nifty.  Now can we 1 UP it and make it live edit?  Sure can!  The style dock will call apply() whenever we raise the widgetChanged signal.  We can just connect this to the text changed event and add some logic to the apply method.

class CustomPanel(QgsLayerStylingPanel): def __init__(self, layer, canvas, parent): super(CustomPanel, self).__init__(layer, canvas, parent) ..... self.editor.textChanged.connect(self.widgetChanged.emit) def apply(self): # Read the xml and set it back to the style. doc = QDomDocument( "style" ) doc.setContent(self.editor.text()) node = doc.documentElement() self.layer.readStyle(node, "") self.layer.triggerRepaint()

live

WINNING!

Pro Tip: Raise widgetChanged whenever you need to tell the style dock to trigger the apply method. The style state of the layer will also be saved to the undo stack so you don’t have to worry about that.

This is just an example, hand editing xml is well gross, but I’m sure you can see great uses for being able to expand the style dock.


Filed under: Open Source, qgis
Categories: OSGeo Planet

Fernando Quadro: OpenStreetMap realiza melhorias nos mapas para as Olimpíadas

OSGeo Planet - Wed, 2016-06-29 10:30

A comunidade brasileira do OpenStreetMap (OSM) realizou no último fim de semana uma maratona de mapeamento com o objetivo de melhorar o mapa do Rio para as Olimpíadas. Durante todo o sábado e o domingo, colaboradores do OSM não apenas do Rio de Janeiro, mas também de outras cidades do Brasil e de outros países da América Latina dedicaram-se a atualizar o mapa da Cidade Olímpica.

O foco principal da atividade foi atualizar o mapa das áreas onde serão realizadas as competições. “Muitas edificações foram construídas para os jogos e também tivemos novas vias como a TransOlímpica e serviços de transporte como o VLT”, afirma Arlindo Pereira, mapeador da cidade do Rio de Janeiro. A imagem abaixo apresenta as áreas que ocorreram edições:

RioOlympicsMappingEdits

Uma das áreas que mais recebeu contribuições foi o bairro de Deodoro, na zona oeste do Rio, onde acontecerão as competições de esportes como Mountain Bike, Tiro, Canoagem Slalom, Hockey, entre outros.

“Os Jogos Olímpicos devem atrair muitos turistas ao Rio, por isso é importante termos um mapa bem atualizado e preciso” afirma Wille Marcel, um dos membros do OpenStreetMap no Brasil. Foram realizadas 290 edições no mapa do Rio durante o período. A atividade teve a colaboração da startup Mapbox, a qual disponibilizou imagens de satélite atualizadas do Rio de Janeiro.

Navegue pelo mapa do Rio no OpenStreetMap: http://osm.org/go/OVcch20–

Fonte: OpenStreetMap

Posts RelacionadosZemanta
Categories: OSGeo Planet

Fernando Quadro: Diretiva para o OpenLayers 3 no AngularJS

OSGeo Planet - Tue, 2016-06-28 10:30

AngularJS é um framework JavaScript open-source, mantido pelo Google, que auxilia na execução de single-page applications. Seu objetivo é aumentar aplicativos que podem ser acessados por um navegador web, foi construído sob o padrão model-view-view-model (MVVM), em um esforço para facilitar tanto o desenvolvimento quanto o teste dos aplicativos.

No AngularJS encontramos as diretivas que permitem ao desenvolvedor especificar tags HTML personalizadas e reusáveis, que personalizam o comportamento de certos elementos.

Partindo desse princípio David Rubert criou a diretiva angular-openlayers-directive com a intenção de incorporar mapas geridos pela biblioteca OpenLayers 3.

Veja como é fácil usar a diretiva:

1. Primeiro de tudo, carregue o AngularJS e OpenLayers (V3) em seu arquivo HTML.

2. Depois disso, você deve incluir a dependência da openlayers-directiva em seu módulo angular:

var app = angular.module ( "demoapp", [ "openlayers-directiva"]);

3. O comportamento padrão irá mostrar uma camada OpenStreetMap e os controles de zoom e atribuições, mas você pode personalizar os valores padrão para atender às suas necessidades. Além disso, você pode definir uma atualização bi-direcional de valores entre o mapa e as variáveis ​​do seu controler. Desta forma, você pode interagir facilmente com o mapa. Exemplo:

angular.extend($scope, {
    center: {
        lat: 51.505,
        lon: -0.09,
        zoom: 8
    }
});

4. Finalmente, você deve incluir a diretiva em sua página HTML, desta forma:

<openlayers ol-center="center" height="480px" width="640px"></openlayers>

Para conferir informações detalhadas e baixar a diretiva você pode acessar o link do GitHub:

https://github.com/tombatossals/angular-openlayers-directive

Bons estudos!

Fonte: Spatially Oriented

Posts RelacionadosZemanta
Categories: OSGeo Planet

GeoSpatial Camptocamp: Rencontres développeurs et utilisateurs de QGIS

OSGeo Planet - Tue, 2016-06-28 09:22

Camptocamp était présent à la 2ème rencontre utilisateurs et au 15ème Hackfest de QGIS.

Cet article Rencontres développeurs et utilisateurs de QGIS est apparu en premier sur Camptocamp.

Categories: OSGeo Planet

gvSIG Team: How to solve the problem about label and symbol size when exporting to PDF file in gvSIG

OSGeo Planet - Tue, 2016-06-28 07:52

When we create a layout in gvSIG, and we export it to PDF format or we print it, we sometimes can see that labels that we had added to the different elements of the map appear in a different size at the PDF file or in the paper. We can see the labeling in a correct way at the gvSIG layout, without being overlapped, but they are overlapped or they are not visualized because they are very small at the PDF file or at the paper.

It’s the same than the symbology. We can have several point or line symbol (for example, an airport symbol or a motorway line), that we see them in a correct size in a map, but when we print it or we export it to a PDF file they are very big or they can’t be seen.

The cause is because we are working in pixels instead of length unities (meters, kilometers…). The pixel is an immaterial concept that doesn’t have a concrete measurement, it is a cell measurement unity, they are “screen” unities.

For this reason, when we have the labels or symbols in a layout in pixel unities, we can see them correctly at the screen, but when we see them at the “paper space” we don’t have pixel unities, so it can change depending on the zoom/scale that we have at the screen.

PDF file created from a layout, with pixel unities

We can solve it selecting length units, like meters. At this way, we will have at the paper or PDF file the same visualization than in the screen. If symbols or labels are not overlapped at the screen, they won’t be overlapped at the PDF file or at the paper, we will see them at the same way.

PDF file created from a layout, with meter unities

Depending on the scale that we are using we will have to select a different value. For example, if we are working with a complete country in a view, if we want to label the different regions we would be able to write 10.000, 20.000… meters for example, and if we are working in a local area, with municipality names, it can be 500, 1000… meter (of course it will depend on the size of our country).

It will be the same than the point (for example a hospital or a tree) or line (motorway, national road…) symbols, where we can work with a few meters. For example the size of a tree symbol size would be able to be about 4-5 meters.

To get the best results we will have to check it with different values until we visualize what we want at the layout.

In addition, at the labeling, apart from playing with the units as we have told, we also can use the advanced labeling (selecting “User defined labels” option at the first pull-down menu of Labeling tab at layer properties), where we can customize our labels (labeling each label in a different way, adding fixed text and a text from the attribute table…) as well as delete the repeated labels, avoid their overlap (indicating the priority of their location)…

We also will be able to select the interval of scale where the symbols or labels will be shown. For that we will install the extension about Create labels or Create legends by scale.

We hope that this information is useful for you!


Filed under: english, gvSIG Desktop, training Tagged: labelling, pdf, print, symbology
Categories: OSGeo Planet

Fernando Quadro: Integração do Google Maps com o OpenLayers 3

OSGeo Planet - Mon, 2016-06-27 10:30

Para quem estava acostumado a usar o OpenLayers 2 e migrou para a versão 3.0 deve ter percebido que a nova versão não dá suporte as camadas do Google Maps e segundo Andreas Hocevar não dará a menos que o Google mude sua política e permita o acesso direto aos tiles do seu mapa.

Devido a esse problema a mapGears resolveu criar uma biblioteca que realiza essa integração das camadas do Google Maps com o OpenLayers 3.

OL3-Google-Maps é uma biblioteca de código aberto que permite aos usuários adicionar camadas do Google Maps a um mapa que utiliza a biblioteca OpenLayers 3. Ela funciona através da replicação e sincronização dos objetos do OL3 com o Google Maps, utilizando o seu próprio mapeamento da API. Todas as solicitações feitas do lado do usuário são escritas como chamadas do OL3, portanto, não há nova sintaxe para se aprender.

dicas-google-maps

Para usar a biblioteca, você precisa incluir o script OL3-Google-Maps, bem como a API do Google Maps:

<script src="https://maps.googleapis.com/maps/api/js?v=3&key=mykey"></script>
<script src="ol3gm.js"></script>

No exemplo abaixo, mykey é a sua chave da API do Google Maps, e ol3gm.js é o script OL3-Google-Maps compilado. Você não precisa carregar a biblioteca do OL3 uma vez que ela já está incluída com a OL3-Google-Maps.

// Create a Google Maps layer
var googleLayer = new olgm.layer.Google();
// Create a regular OL3 map, containing our Google layer
var map = new ol.Map({
  // use OL3-Google-Maps recommended default interactions
  interactions: olgm.interaction.defaults(),
  layers: [
    googleLayer
  ],
  target: 'map',
  view: new ol.View({
    center: center,
    zoom: 12
  })
});
// Activate the library
var olGM = new olgm.OLGoogleMaps({map: map});
olGM.activate();

No site do projeto você pode conferir mais exemplos e também a lista de limitações do projeto.

Posts RelacionadosZemanta
Categories: OSGeo Planet

gvSIG Team: Cómo solucionar en gvSIG el problema de que las etiquetas o símbolos estén a diferente tamaño al exportar a PDF

OSGeo Planet - Mon, 2016-06-27 10:19

Cuando creamos un mapa en gvSIG, y luego lo exportamos a formato PDF o lo imprimimos a papel, en ocasiones vemos que las etiquetas que habíamos insertado en los elementos del mapa nos aparecen en el papel o en el PDF en un tamaño distinto. Podemos ver el etiquetado de forma correcta en el mapa de gvSIG, sin superponerse, pero después en el papel o el PDF se superponen entre ellos o directamente no se visualizan porque quedan demasiado pequeños.

Lo mismo ocurre con la simbología. Podemos tener ciertos símbolos de puntos o líneas (por ejemplo el símbolo de un aeropuerto, o la línea de una autopista), que en el mapa lo vemos a un tamaño correcto, pero cuando lo imprimimos o lo exportamos a formato PDF nos aparece demasiado grande o apenas se ve.

El problema viene porque estamos trabajando en píxeles en lugar de unidades de longitud (metros, kilómetros…). El píxel es un concepto inmaterial que no tiene una medida concreta, es una unidad de medida de celdilla, son unidades de la “pantalla”.

Por este motivo, cuando tenemos el etiquetado o simbología insertados en unidades de píxeles, podemos verlo bien en la pantalla, pero al trasladarlo al espacio papel ya no tenemos unidades de píxel, por lo que nos puede cambiar según el zoom/escala que tengamos en ese momento en la pantalla.

Mapa-etiquetado-pixeles_

Fichero PDF creado a partir de un Mapa con unidades en píxeles

La solución pasaría por seleccionar unidades de longitud tales como metros. De esta forma, lo mismo que vemos en la pantalla será lo que veamos en el papel (en el PDF o al imprimir). Si tenemos varios símbolos con un tamaño concreto, o varias etiquetas, sin superponerse entre ellas, en el PDF y en el papel nos aparecerán igual.

Mapa-etiquetado-metros_

Fichero PDF creado a partir de un Mapa con unidades en metros

Según la escala a la que estemos trabajando deberemos poner una cantidad u otra. Por ejemplo, si trabajamos en una vista con un país completo, las unidades para el etiquetado de las distintas regiones pueden ser por ejemplo 10.000, 20.000… metros, y si trabajamos en una zona local, con nombres de municipios, puede ser de 500, 1000… metros (por supuesto dependerá del tamaño de nuestro país).

Lo mismo ocurrirá para los símbolos puntuales (por ejemplo un hospital o un árbol) o lineales (autopista, carretera nacional…), donde podemos trabajar con unos pocos metros. Por ejemplo, el tamaño del símbolo de un árbol en un mapa de un barrio podría ser de unos 4-5 metros.

Para poder obtener los resultados óptimos habría que probar distintas cantidades hasta que lo que visualicemos en el mapa sea lo que deseemos.

Por otro lado, en el etiquetado, aparte de jugar con las unidades como hemos comentado, podemos utilizar también el etiquetado avanzado (seleccionando la opción “Etiquetas definidas por el usuario” en el primer desplegable de la pestaña de “Etiquetados” en las propiedades de la capa), donde aparte de poder personalizar nuestro etiquetado (etiquetando cada elemento de una forma, añadiendo texto fijo y texto obtenido de la tabla de atributos…), podemos seleccionar que elimine las etiquetas repetidas, que no se superpongan entre ellas (pudiendo indicar la prioridad de su localización)…

También podremos seleccionar las escalas entre las cuales queremos que se visualicen los símbolos o etiquetas. Para ello deberemos instalar la extensión para Generar etiquetas y Generar simbología por escala.

¡Esperamos que os sea útil esta información!


Filed under: gvSIG Desktop, spanish, training Tagged: etiquetado, impresión, pdf, simbología
Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Exploring CKAN data portals with QGIS

OSGeo Planet - Sun, 2016-06-26 20:28

CKAN is for data portals what QGIS is for GIS. The project describes itself as

CKAN is a powerful data management system that makes data accessible – by providing tools to streamline publishing, sharing, finding and using data. CKAN is aimed at data publishers wanting to make their data open and available.

Many open (government) data platforms rely on CKAN and while the web interface is pretty good, there’s still the hassle of finding and downloading the data using a web browser.

This is where the QGIS CKAN-Browser plugin comes in useful. The plugin has been developed by BergWerkGIS for the state of Carinthia, Austria and added to the public plugin repo earlier this year. CKAN-Browser comes preconfigured with some Austrian and European CKAN URLs for testing, so you can get going really quickly. It is easy to search for specific datasets or explore the portal’s data categories and it is just one click to download and load the data into your QGIS map:

Screenshot 2016-06-26 22.25.00

Here’s a quick demo of loading a vector dataset as well as raster tiles:

For the full usage guide, visit the plugin’s Github page.

It’s great to see how well CKAN and QGIS can play together to enable seamless access to open data!


Categories: OSGeo Planet

Gis-Lab: Опыт классификации космоснимка Landsat с помощью Semi-Automatic Classification Plugin в QGIS

OSGeo Planet - Sat, 2016-06-25 16:15

Новая статья описывает опыт работы с Semi-Automatic Classification Plugin для QGIS для классификации снимка Landsat с целью выявления лесонарушений на примере НП “Орловское полесье”, а также содержит пошаговую инструкцию для лесного дешифрирования снимка с помощью данного плагина.

Автор: Карпачев Андрей Петрович, научный сотрудник НП “Орловское полесье”.

Прочитать | Обсудить

null

null

Categories: OSGeo Planet

Paulo van Breugel: Climate data sets, which one to select?

OSGeo Planet - Sat, 2016-06-25 15:07
For species or vegetation modelling, one of the first choices to make is the selection of explanatory variables, which in most cases will include climatic or bioclimatic data sets. One of the most widely used global climate data sets in biogeographic and ecological research is from Worldclim (Hijmans et al., 2005). Alternative global rainfall data […]
Categories: OSGeo Planet

Nathan Woodrow: QGIS style dock – Part 1 – Live Styles!

OSGeo Planet - Sat, 2016-06-25 07:54

This post is part of a series exploring the new style dock.

  • Part 1 – Styles
  • Part 2 – Plugin pages
  • Part 3 – New panel API

A while ago did a post about the new style dock in QGIS. The first stage was getting labels working and then moving on to styles. After many iterations of the UIs and the APIs I’m pretty happy with the release state of the dock. We now have styles (vector and raster), labels, saved styles, undo redo, and plugin support (yep!)

Of all the features I have added to QGIS this is my favorite. Nothing has made me feel more productive then working on this feature and seeing the gains it allows in day to day work.

If you haven’t used the style dock yet I would strongly encourage you to grab the nightly QGIS builds and give it a crack, F7 to open anytime, or use the paint brush icon on the layers panel.   I’m not bragging when I say that it will change your life in QGIS for the better, it has become a part of my daily workflow. No going back!

Dock break down

breakdown

As you can see in the image there is a few main things in the dock, the layer selection, style options selector (come back to this later), undo/redo, live update and apply.

Live Updates

One of the key things for me when doing this dock was instant feedback. I really hated the open close, change, apply, repeat workflow of the old dialogs.  Every widget in the dock can request a redraw on change so you can see feedback instantly in the canvas.  This might not always be what you want so you can disable it for those cases and use manual apply.

Undo Stack

Live updates come with a risk. What happens if you apply something you didn’t mean. There is no cancel button. Well not to fear we now have a undo/redo stack.  Any style change will now add an entry to the style undo stack so you can rollback and forward any changes. Undo/Redo now allows for fearless style changes, so style away to your hearts content and rollback anything you don’t like.

I will also add that the undo stack is stored on the layer object itself, so closing the dock doesn’t clear the stack nor does changing the active layer. You can also access the stack via the Python bindings if needed.

Style Options

On the side of the dock we have the different style options.  These buttons change the style options in the main panel.

optionsVector Style Options
  • Style
  • Label
  • Saved styles
  • Undo/Redo
optionsrasterRaster Style Options
  • Style
  • Transparency
  • Histogram
  • Saved styles
  • Undo/Redo

The Saved Styles and Undo/Redo will always be the last in the stack regardless of the active layer type.

Style UIs (Note: GIF heavy section)

Almost all the widgets have been tweaked to allow for better dock layout. Widgets are now in a more uniform layout which flows better when moving around in the dock and just generally look better.

layout

When I talked about live updates before this applies to everything, even the data defined buttons.  Any changes in those will be reflected into the canvas without any extra clicks.

dd

Changing renderers is now simple and quick. You go from nothing to seeing instant results in seconds.

change

A excellent side effect of the live updates is workflows that were never exposed before and now possible. Using the graduated renderer histogram for example. As the histogram allows for live edits we can now update the map on the fly from adjusting the histogram.

hiso.gif

Style UIs (Raster)

As soon as I finished vectors it became pretty evident that raster layers were going to be the next main thing to move.   So the style dock fully supports raster styling.

raster

Undo/Redo

For the final trick. Here we have the undo/redo stack in action. Unlimited undo/redo  on all layers when changing styles. Fearless change he we come

undo

Trying to find a picture to sum up my feelings about the style dock and I think this fits

but of course I’m super bias.

Stay tuned for Part 2 where I talk about adding your own style panels to the style dock.


Filed under: Open Source, qgis
Categories: OSGeo Planet

portailSIG: QGIS: des Ellipses de Déviation Standard (SDE), un plugin, « Standard Deviational Ellipse », des scripts R (processing) et Python et une approche critique...

OSGeo Planet - Fri, 2016-06-24 13:40
Niveau Intermédiaire Logiciels utilisés QGIS
R
Python
QGIS Plugin: Standard Deviational Ellipse
Plateforme Windows | Mac | Linux | FreeBSD

Parmi les commandes d’ArcGIS qui manquaient dans QGIS, figurait l’ Ellipse de Déviation Standard (Standard Deviational Ellipse ou SDE). Ce n’est plus le cas actuellement puisque c’était déjà possible de le faire avec des scripts R dans la Boîte à outils de Traitements (Processing) avec des packages dédiés et le nouveau plugin Standard Deviational Ellipse. Ses premiers résultats ont suscité des réactions de ma part (problem with the rsd values compared with the aspace or phonTools R packages ) qui ont amené Håvard Tveite à améliorer et compléter son plugin. 

Je vais essayer ici d’expliquer le pourquoi des choses, de manière visuelle et sans trop de statistiques, car ce concept a sérieusement été remis en cause récemment par Gong, J (2002), Clarifying the standard deviational ellipse. Geographical Analysis 34, 155–167.(pdf) et Wang B, Shi W, Miao Z (2015) Confidence Analysis of Standard Deviational Ellipse and Its Extension into Higher Dimensional Euclidean Space. PLoS ONE 10(3): e0118537. doi:10.1371/journal.pone.0118537,

C’est quoi ?

Si la dispersion d’une variable en une dimension est facile à calculer et à visualiser (histogrammes, etc.), c’est moins le cas pour des distributions bivariées (ou multivariées). La solution classique est de calculer séparément les écarts types de chaque variable et de représenter leur distribution par des ellipses (ou des ellipsoïdes) dont les axes sont ces écarts types. La méthode permet aussi de mesurer une tendance directionnelle (angle de l’ellipse).

Dans la réalité ce sont en fait des isocontours d’une surface, similaires à des courbes de niveau.

Et le graphique réalisé avec le package R psych  qui illustre le fait que cette démarche n'est pas limitée au domaine géospatial, loin de là.

Cette ellipse est donc théoriquement déterminée par trois mesures :

  1. calcul du point « moyen » du nuage des points (cela peut être le centroïde, le barycentre ...)
  2. calcul de l’angle de rotation de l’ellipse (orientation)
  3. calcul de la dispersion par les écarts-types des coordonnées x et y à partir du  point « moyen ». Ceux-ci vont définir les axes de l’ellipse (c’est pourquoi ESRI nomme aussi l’ellipse  « ellipse de l’écart type ».

Toutes les combinaisons sont possibles: affecter une masse (valeur) à chaque point, utiliser des moyennes pondérées, des médianes ou la covariance des données (pondérée ou non). Le calcul peut aussi varier en fonction de la répartition des points et il est possible d’utiliser soit la distance euclidienne soit d'autres distances ( comme la Distance de Mahalanobis ou la Distance de Manhattan).        

Je n’illustrerai ici que le cas de valeurs non pondérées (le reste n'est qu'un facteur d'échelle et de rotation).

Statistisquement parlant, l'objectif recherché est d'obtenir les valeurs de  la classique Règle 68-95-99.7 (3 sigma) empirique ( graphique réalisé avec Python et le module matplotlib) dans le cas d'une distribution univariée, et pas bivariée (Wang et al. (2015) contestent ces valeurs et en proposent des nouvelles pour les distributions multivariées, voir plus bas)

Un peu d’histoire ou comment une ellipse n’est pas une ellipse, mais...

Proposée dès 1926 par Lefever (Measuring Geographic Concentration by Means of the Standard Deviational Ellipse. American Journal of Sociology. 1926; 32(1):88–94), il a vite été démontré que le résultat n’était pas une ellipse (Furfey PH. A Note on Lefever’s "Standard Deviational Ellipse". American Journal of Sociology. 1927; 33(1):94–8), mais à ce qui est nommé actuellement une « Courbe de Déviation Standard ».  Depuis lors, bien évidemment, de nombreuses corrections ont été apportées pour tenter de résoudre le problème avec des résultats variables, mais la controverse continue... 

Il faut noter que le concept est né sous l’hypothèse que les variables observées suivaient la loi normale, mais que cette condition n’est plus indispensable avec les algorithmes utilisés actuellement. Elle est théoriquement dérivée de la distribution bivariée (ou bidimensionnelle)

A l’heure actuelle, dans le monde geospatial, ne subsistent que deux grands algorithmes et des solutions intermédiaires:

  • celui proposé par Yuill, R. S.(1971) dans The Standard Deviational Ellipse: An Updated Tool for Spatial Description, Geografiska Annaler 53B(1),28-39) qui est une énième adaptation réussie de la solution de Lefever, avec de nombreuses variantes postérieures;
  • celui implanté  dans le  logiciel CrimeStat III (Ned Levine, 2010, A Spatial Statistics Program for the Analysis of Crime Incident Locations (version 3.3).  Ned Levine & Associates, Houston, TX.; National Institute of Justice, Washington, DC.), spécialement dans la note 4 de Part II: Spatial Description (pdf).

Wang et al. (2015) ont récemment proposé un nouvel algorithme basé sur la décomposition spectrale de la covariance, plus exact selon eux.

Pourtant les géomaticiens continuent à utiliser cette méthode dans de nombreux domaines, sans se préoccuper ni de l’algorithme utilisé, ni de la signification statistique réelle du résultat.

Préambules statistiques: l’éllipse de covariance

Au départ, pour toute distribution bivariée, il est possible de calculer l’ellipse de covariance (ou ellipse de dispersion, ou ellipse des erreurs, ou...). Une matrice de covariance peut en effet être décomposée en (voir A geometric interpretation of the covariance matrix, par example):

  • vecteurs propres qui permettent de calculer les 2 angles de l’ellipse. Le premier vecteur (violet) est la direction dans laquelle les données varient le plus, le second (bleu) est orthogonal au premier.
  • valeurs propres qui sont les 2 diamètres de l’ellipse. Les rayons sont les racines carrées des valeurs propres et correspondent aux écarts types modifiés (translation + rotation) de x et y, σx' et σy’

La courbe limite de l’ellipse (Δχ2) correspond à un contour de probabilité de 1 écart type avec une probabilité de ± 39,35 % (c'est le genre de phrase qui nécessiterait de très nombreuses explications qu’il n’est pas possible de développer ici et je vous renvoie à vos manuels de statistiques).

 

Un script Python qui permet de le faire est disponible à par_covar.py. Le même résultat peut être obtenu avec les packages R ellipse:ellipse, siar: standard.ellipse , PhonTools: sdellipse, mvdalab: ellipse.mvdalab et autres.

Résultats:

Centre.x Centre.y σx σx Angle (en degrés par rapport au N) Aire Excentricité 320,66 215.66 45,67 112.86 77.12 16194.14 0,9144

En multipliant les valeurs obtenues par des facteurs d’échelle appropriés (loi normale, voir How to draw a covariance error ellipse?), il est donc possible de calculer des ellipses d’équiprobabilités. Ceci sera fait avec le package R car: dataEllipse() qui permet de superposer les contours d'une distribution bivariée normale de deux variables sur un semis de points. Un script R pour tracer ces ellipses dans la Boîte à Outils est disponible à car_ellipses.rsx.

Et il est possible de constater que l’ellipse de covariance correspond bien à une probabilité de ± 39,35 %.


Ces ellipses vont nous servir de repère pour superposer les différentes ellipses calculées dans la suite. Elles ne sont en effet que des adaptions et c'est pourquoi les coordonnées du point « moyen »,  les angles et l’excentricité restent les mêmes pour toutes les solutions.

L'interface du plugin

L’ellipse de Yuill

Les calculs d’estimation des rayons/écarts types (standard deviation) sont:

Elle est calculée avec l'option Yull du plugin.

Résultats (pour rappel, les valeurs des angles et de l’excentricité sont les mêmes que celles obtenues pour l’ellipse de covariance): 

σx σy Aire 44,12 109,03 15114,53

 C’est aussi l’ellipse calculée par: 

L’ellipse de CrimeStats III

Ned Levine, dans le cas de CrimeStats III, démontre dans la note 4 (voir référence plus haut) qu’avec les traitements précédents, l’ellipse résultante est trop petite (d’un facteur racine carrée de 2) pour deux raisons statistiques, le calcul des axes et le fait que les autres solutions ne sont pas correctes au niveau des degrés de liberté (n-2 au lieu de n). Les résultats sont beaucoup plus grands qu’avec l’algorithme de Yull.

Elle peut être calculée avec le package R aspace: calc_sde()  (un script R pour la Boîte à Outils de Traitement (Processing) est disponible à aspace_SDE.rsx ) ou avec l'option CrimeStat du plugin.

Résultats: 

σx σy Aire 67,03 165,63 34879.69

 

La solution d’ArcGIS  10.x

Entre la version 9.x et la version 10.x, ArcGIS a modifié son algorithme en utilisant une des corrections de CrimeStats (racine carrée de 2) sans autre explication ni justification (ArcMap 10.x: Fonctionnement de Directional Distribution (Standard Deviational Ellipse))

Elle est calculée avec les options Yuill + sqrt(2) correction  du plugin et avec divers packages R (ellipse pour 1 écart type ici)

 

Résultats:

σx σy Aire 62,40 154,196 30229.06

Le plugin offre aussi la possibilité d’utiliser la deuxième correction de CrimeStats seule avec l’option Yull + DF correction.
 

L'ellispse de Wang et al. (2015)

Le  nouvel algorithme proposé par Wang et al (2015) se base aussi sur la matrice de covariance, mais en modifiant le cacul  (1/n au lieu de 1/n-1 pour les connaisseurs). Ensuite le traitement est le même, voir le script ellipse_wang.py.

Résultats (c'est le même que celui de Yuill): 

σx σy Aire 44,12 109,03 15114,53

L'important est qu'ils calculent les valeurs de  la classique règle empirique des 3 sigmas pour des distributions multivariées. Ce sont les valeurs approximatives de 39,35%, 84,47% et 98,89% et non 68,27%, 95,45% et 99.73% comme le stipule ESRI.

Conclusions

Que d’ellipses, me direz-vous, mais elles ne se différencient que par les résultats du calcul des axes de l’ellipse et donc de l’aire. Du  fait de la signification statistique des résultats, c’est là où naissent les problèmes et les controverses.

  • dans le cas d’ESRI, il est stipulé que « un polygone d’ellipse d’un écart type recouvre approximativement 68 pour cent des entités» (que ce soit avec les résultats de la version 9.x ou ceux la version 10.x) , en supposant que variables concernées suivent une distribution spatiale normale;
  • Cette conclusion est remise en question par Wang et all (2015) qui opinent que les conclusions d’ESRI correspondent à la classique Règle 68-95-99.7 (3 sigma) des distributions univariées normales, mais pas pour des  distributions bivariées (ou multivariées). Ils proposent donc des nouvelles valeurs: "Obviously, such confusing interpretation may mislead the GIS users to believe the univariate 3-sigma rule remains valid in two-dimensional Euclidean space, or even higher dimensions". Ils calculent des valeurs correctes selon eux;
  • la solution de CrimeStat s'éloigne des précédentes;
  • les ellipses basées sur la seule matrice de covariance sont théoriquement les plus correctes.

Alors quelle est la bonne ellipse ? Le nouveau plugin permet de toutes les figurer (hormis les eclipses de covariance et les axes) et à vous choisir la bonne...

Pour ceux qui ne s’intéressent qu’aux orientations (angles et excentricité), comme moi, elles sont toutes bonnes, mais pour ceux qui traitent les surfaces et leurs interprétations statistiques, c’est autre chose... Il serait nécessaire en tout cas de toujours spécifier la solution retenue comme dans la majorités des articles scientifiques.

Et oui, il n'y a donc pas une seule Elipse de Deviation Standard comme pourraient le penser les utisateurs d'ArcGIS avec leur commande standardisée. Je remercie aussi Håvard Tveite d'avoir tenu compte de mes remarques (j'utilise depuis longtemps ce genre d'ellipse dans un contexte non spatial).

En route pour les ellipses de covariance 3D et les ellipses de Deviation Standard bayesiennes.

Annexe: Sommaire des résultats obtenus avec les points utilisés

Pour rappel, les coordonnées du point « moyen », les angles et l'excentricité sont les mêmes pour toutes les solutions (77,12° par rapport au N et 0,9144).

solution σx σy Aire 3σ: σ1 spécifié Plugin: Yuill 44,13 109,03 15114,53 variable Arcview 3.3 (ArcMap 9.x) 44,13 109,03 15114,53 68,23% Wang et al (2015) 44,13 109,03 15114,53 39,35% R avec le package siar 45,67 112,86 16194.14   Covariance 45,67 112,86 16194.14 39,35% Plugin:Yuill avec l'option DF 47,40 117,12 17439.85   ArcMap 10.x 62,40 154,20 30229.06 68,23% Plugin:Yuill avec l'option sqrt(2) 62,40 154,20 30229.06   R avec le package aspace 67,03 165,63 34879.69   Plugin avec l'option "CrimeStat" 67,03 165,63 34879.69  

 

Tous les traitements présentés ont été effectués sur Mac OS X ou Linux avec QGIS 2.14.x, R version 3.x et Python 2.7.x. ArcGIS 10.1 sur Windows a été utilisé pour obtenir des ellipses pour comparer les résultats. LaTeX a été utilisé pour les équations.

Site officiel : Calculating and visualizing standard deviations in two dimensions


Creative Commons License
licence Creative Commons Paternité-Pas d'Utilisation Commerciale-Pas de Modification 2.0 France

Categories: OSGeo Planet

Fernando Quadro: Livro: QGIS Design Map

OSGeo Planet - Fri, 2016-06-24 10:30

Criar mapas pode não ser uma tarefa tão árdua pra você, mas deixar o mapa com uma aparência agradável e harmoniosa provavelmente sim, isso porque a maioria de nós (analistas de sistemas, geógrafos, cartógrafos, entre outros) não temos muitas noções de design.

Afim de nos ajudar nesse quesito Anita Graser e Gretchen N. Peterson escreveram um livro intitulado “QGIS Design Map” com o objetivo de nos ensinar a utilizando o QGIS elevar nossos produtos cartográficos ao mais alto nível.

qgis-map-design

Com instruções passo-a-passo para criar os mais modernos designs de mapa este livro cobre tudo, desde um estilo básico a técnicas de rotulagem avançadas como contornos iluminados e mascaramento dinâmico.

Se você está querendo aprofundar seus conhecimentos e melhorar a aparência dos seus mapas, vale a pena a leitura.

Posts RelacionadosZemanta
Categories: OSGeo Planet

Jackie Ng: Let's introduce the other major player

OSGeo Planet - Thu, 2016-06-23 15:44
My plans for the VSCode map extension now extend beyond just OpenLayers



At this point there are some small oddities:

  • The timeline button SVG has broken hrefs. You can see it from the missing play/pause/rewind graphics.
  • The Cesium InfoBox doesn't work.
This is probably due to a combination of:
  • Running cesium from the local file system
  • The VSCode/Electron webview or Cesium itself imposing sandboxing restrictions that breaks the InfoBox
Still, for the purposes of being able to quick and easily preview CZML content, this is a very nice starting point.
Categories: OSGeo Planet

Fernando Quadro: Habilitando cache no QGIS para camadas WMS

OSGeo Planet - Thu, 2016-06-23 10:30

Quando se está trabalhando no QGIS com camadas através de conexões WMS em alguns momentos pode-se perceber que as camadas estão tendo algum delay, por motivos da rede ou até mesmo do servidor em que estão hospedadas as camadas.

Curiosamente o cache para camadas WMS não está habilitado por padrão no QGIS. Para aproveitar melhor as conexões WMS, WMS-C ou WMTS – e se você tem algum espaço em disco para dedicar – é uma boa idéia habilitar o cache nestes poucos passos a seguir:

Vá até a opção de menu Configurações (Settings) >> Opções (Options):

qgis_wms_cache1

Ao abrir a tela de Opções vá até a aba ” Rede (Network)”. Nessa tela, em “Configurações de cache (Cache settings)”, selecione uma pasta e escolha uma pasta para armazenar os arquivos de cache. Certifique-se de que há espaço livre na pasta para armazenadar os arquivos ao longo do tempo.

Aqui eu escolhi a pasta “/tmp” e criei dentro dela uma nova pasta chamada “qgis_wms_cache”:

qgis_wms_cache2-768x498

Eventualmente podemos mudar o período de expiração padrão das tiles WMS-C/WMTS (horas) para mais de 24 horas.

Aqui eu vou mudar para 24 * 30 = 720 horas, e o tamanho do cache vou aumentar para 250MB (escrito como 250000, uma vez que a unidade utilizada aqui é o kilobytes):

qgis_wms_cache3-768x498

Agora basta pressionar o botão “OK” e fechar a janela e o seu QGIS já estará com a geração de cache habilitada para conexões WMS.

Fonte: GFOSS Blog

Posts RelacionadosZemanta
Categories: OSGeo Planet

Fernando Quadro: Apresentações do 3º QGIS Portuguese User Meeting

OSGeo Planet - Wed, 2016-06-22 10:30

O 3º Encontro de Utilizadores QGIS Portugal, foi um encontro informal que teve como tema central o uso do software Open Source QGIS e aconteceu entre os dias 17 e 18 de Junho de 2016, na cidade do Porto (Portugal).

O evento contou com 11 apresentações com duração aproximada de 15 minutos, dois workshops demonstrativos com a duração aproximada de 4 horas cada um e um Hackfest com o objetivo de apresentar o modelo de desenvolvimento colaborativo do QGIS e contribuir efetivamente para o projeto através da realização de traduções, elaboração de documentação, e do reporte e correção de bugs (bug reporting e bug fixing).

As palestras e workshops já estão disponíveis e podem ser baixadas (no formato PDF):

Utilização de ferramentas open source para a caracterização de uma paisagem. O módulo LecoS do QGIS.
O uso combinado de LiDAR, QGIS e LAStools aplicado ao estudo de paisagens arqueológicas
Desenvolvimento de aplicações em QGIS para modelos de gestão de risco
Aplicação SIG open source – MicMac
Urban Traffic Emission Modelling in QGIS – Qtraffic
Expansion of Alien Invasive Plants Along the Roads: a Remote Sensing Approach
Mapeamentos com QGIS de uma Colecção de Pinturas de Adriano de Sousa Lopes do Acervo da Faculdade de Belas Artes da Universidade de Lisboa
“QGIS na Administração Publica Intermunicipal” e “QGIS/QGIS Server – Plano Operacional Municipal – Acesso a dados geográficos em modo OFFLINE”
O uso de software open source na investigação e desenvolvimento florestal
Processo de implementação de QGIS como ferramenta de trabalho na Câmara Municipal de Oeiras
O que há de novo no QGIS 2.12-2.16
Iniciação ao QGIS (Workshop)
Análise Espacial com QGIS + dataset (Workshop)

Posts RelacionadosZemanta
Categories: OSGeo Planet
Syndicate content