OSGeo Planet

Equipo Geotux: Publicar un servicio de teselas de mapas con QMetatiles y GitHub

OSGeo Planet - Fri, 2017-10-13 15:07

Para realizar una publicación de un servicio WMTS usando simplemente el almacenamiento estático en un servidor Web, es necesario utilizar una serie de herramientas que permiten, en primera medida generar las teselas o baldosas de imágenes y segundo una herramienta que nos permita generar el archivo XML.. Para el primer caso, se va hacer uso de las siguientes extensiones o complementos de QGIS.



Nota: Este documento hace parte del material de guías de talleres del curso de Servicios Web Geográficos de la Maestría en Geomática de la Universidad Nacional de Colombia Sede Bogotá, mayor información en http://www.aulageo.cloud/course/unal-ogc-2017/ 1. IDENTIFICACIÓN DE LAS ESCALAS DE TESELAS CON EL COMPLEMENTO DE QGIS DE TILELAYER.

Una vez instalados los complementos de QGIS, se procede a cargar el proyecto de Laguna de Tota, recuerde que el sistema por defecto de este proyecto debe ser EPSG:3857 o Web Mercator, ya que las herramientas a utilizar sólo son compatibles con este sistema de referencia de coordenadas. Una vez el proyecto es desplegado, se procede a cargar el esquema de matrices de teselas desde el menú Web → TileLayer Plugin → Add TileLayer …, en este caso para el servicio WMTS es el esquema XYZFrame.

TileLayer Plugin

El esquema se representa con una nueva capa de nombre XYZFrame, y permite identificar cual es rango mínimo y máximo de zoom, así cómo el número e índice de las teselas. Para el siguiente caso, el zoom mínimo que permite el almacenamiento de nuestra zona de estudio en una tesela en 13, y el índice de origen en la matriz de teselas es 2434,3967.



Una vez identificado los niveles de zoom o escalas de las matrices, se procede a generarlas las teselas con el complemento QMetaTiles disponible en la ruta del menú Complementos → QMetaTiles → QMetaTiles.

Los parámetros solicitados por esta herramienta son los que se muestran en la siguiente imagen.


  • Output: La ruta del directorio en el cual va a crear las teselas. Recomendable usar, si esta usando el servidor de GeoTux Server, la ruta que corresponda al punto de montaje /gisdata/tiles
  • Tileset name: hace referencia al nombre del proyecto o conjunto de matriz de teselas, en este caso “z11to17”.
  • Extent: Hace referencia a la extensión geográfica de la generación de teselas, para este caso use una capa para restringir la extensión geográfica para la generación de teselas.
  • Zoom: son los niveles de zoom para generar el conjunto de matrices de teselas, para este caso se ha identificado anteriormente un conjunto de matrices de teselas de 11 a 17.

Categories: OSGeo Planet

Even Rouault: Optimizing JPEG2000 decoding

OSGeo Planet - Thu, 2017-10-12 16:41
Over this summer I have spent 40 days (*) in the guts of the OpenJPEG open-source library (BSD 2-clause licensed) optimizing the decoding speed and memory consumption. The result of this work is now available in the OpenJPEG 2.3.0 release.
For those who are not familiar with JPEG-2000, and they have a lot of excuse given its complexity, this is a standard for image compression, that supports lossless and lossy methods. It uses discrete wavelet transform for multi-resolution analysis, and a context-driven binary arithmetic coder for encoding of bit plane coefficients. When you go into the depths of the format, what is striking is the number of independent variables that can be tuned:
- use of tiling or not, and tile dimensions
- number of resolutions
- number of quality layers
- code-block dimensions
- 6 independent options regarding how code-blocks are encoded (code-block styles): use of Selective arithmetic coding bypass, use of Reset context probabilities on coding pass boundaries, use of Termination on each coding pass, use of Vertically stripe causal context, use of Predictable termination, use of Segmentation Symbols. Some can bring decoding speed advantages (notably selective arithmetic coding bypass), at the price of less compression efficiency. Others might help hardware based implementations. Others can help detecting corruption in the codestream (predictable termination)- spatial partition of code-blocks into so-called precincts, whose dimension may vary per resolution- progression order, ie the criterion to decide how packets are ordered, which is a permutation of the 4 variables: Precincts, Component, Resolution, Layer. The standard allows for 5 different permutations. To add extra fun, the progression order might be configured to change several times among the 5 possible (something I haven't yet had the opportunity to really understand)- division of packets into tile-parts
- use of multi-component transform or not
- choice of lossless or lossy wavelet transforms
- use of start of packet / end of packet markers
- use of  Region Of Interest, to have higher quality in some areas
- choice of image origin and tiling origins with respect to a reference grid (the image and tile origin are not necessarily pixel (0,0))
And if that was not enough, some/most of those parameters may vary per-tile! If you already found that TIFF/GeoTIFF had too many parameters to tune (tiling or not, pixel or band interleaving, compression method), JPEG-2000 is probably one or two orders of magnitude more complex. JPEG-2000 is truly a technological and mathematical jewel. But needless to say that having a compliant JPEG-2000 encoder/decoder, which OpenJPEG is (it is an official reference implementation of the standard) is already something complex. Having it perform optimally is yet another target.
Previously to that latest optimization round, I had already worked at enabling multi-threaded decoding at the code-block level, since they can be decoded independently (once you've re-assembled from the code-stream the bytes that encode a code-block), and in the inverse wavelet transform as well (during the horizontal pass, resp vertical pass, rows, resp columns, can be transformed independently). But the single-thread use had yet to be improved. Roughly, around 80 to 90% of the time during JPEG-2000 decoding is spent in the context-driven binary arithmetic decoder, around 10% in the inverse wavelet transform and the rest in other operations such as multi-component transform. I managed to get around 10% improvement in the global decompression time by porting to the decoder an optimization that had been proposed by Carl Hetherington for the encoding side, in the code that determines which bit of wavelet transformed coefficient must be encoded during which coding pass. The trick here was to reduce the memory needed for the context flags, so as to decrease the pressure on the CPU cache. Other optimizations in that area have consisted in making sure that some critical variables are kept preferably in CPU registers rather than in memory. I've spent a good deal of time looking at the disassembly of the compiled code.
I've also optimized the reversible (lossless) inverse transform to use the Intel SSE2 (or AVX2) instruction sets to be able to process several rows, which can result up to 3x speed-up for that stage (so a global 3% improvement)
I've also worked on reducing the memory consumption needed to decode images, by removing the use of intermediate buffers when possible. The result is that the amount of memory needed to do full-image decoding was reduced by 2.4.
Another major work direction was to optimize speed and memory consumption for sub-window decoding. Up to now, the minimal unit of decompression was a tile. Which is OK for tiles of reasonable dimensions (let's say 1024x1024 pixels), but definitely not on images that don't use tiling, and that hardly fit into memory. In particular, OpenJPEG couldn't open images of more than 4 billion pixels. The work has consisted in 3 steps :- identifying which precincts and code-blocks are needed for the reconstruction of a spatial region- optimize the inverse wavelet transform to operate only on rows and columns needed- reducing the allocation of buffers to the amount strictly needed for the subwindow of interestThe overall result is that the decoding time and memory consumption are now roughly proportional to the size of the subwindow to decode, whereas they were previously constant. For example decoding 256x256 pixels in a 13498x9944x3 bands image takes now only 190 ms, versus about 40 seconds before.
As a side activity, I've also fixed 2 different annoying bugs that could cause lossless encoding to not be lossless for some combinations of tile sizes and number of resolutions, or when some code-block style options were used.
I've just updated the GDAL OpenJPEG driver (in GDAL trunk) to be more efficient when dealing with untiled JPEG-2000 images.
There are many more things that could be done in the OpenJPEG library :- port a number of optimizations on the encoding side: multi-threadig, discrete wavelet transform optimizations, etc...- on the decoding side, reduce again the memory consumption, particularly in the untiled case. Currently we need to ingest into memory the whole codestream for a tile (so the whole compressed file, on a untiled image)- linked to the above, use of TLM and PLT marker segments (kind of indexes to tiles and packets)- on the decoding side, investigate further improvements for the code specific of irreversible / lossy compression- make the opj_decompress utility do a better use of the API and consume less memory. Currently it decodes a full image into memory instead of proceeding by chunks (you won't have this issue if using gdal_translate)- investigate how using GPGPU capabilities (CUDA or OpenCL) could help reduce the time spent in context-driven binary arithmetic decoder.
Contact me if you are interested in some of those items (or others !)

(*) funding provided by academic institutions and archival organizations, namely
… And logistic support from the International Image Interoperability Framework (IIIF), the Council on Library and Information Resources (CLIR), intoPIX, and of course the Image and Signal Processing Group (ISPGroup) from University of Louvain (UCL, Belgium) hosting the OpenJPEG project.
Categories: OSGeo Planet

Paul Ramsey: Adding PgSQL to PHP on OSX

OSGeo Planet - Thu, 2017-10-12 14:00

I’m yak shaving this morning, and one of the yaks I need to ensmooth is running a PHP script that connects to a PgSQL database.

No problem, OSX ships with PHP! Oh wait, that PHP does not include PgSQL database support.

Adding PgSQL to PHP on OSX

At this point, you can either run to completely replace your in-build PHP with another PHP (probably good if you’re doing modern PHP development and want something newer than 5.5) or you can add PgSQL to your existing PHP installation. I chose the latter.

The key is to build the extension you want without building the whole thing. This is a nice trick available in PHP, similar to the Apache module system for independent module development.

First, figure out what version of PHP you will be extending:

> php --info | grep "PHP Version" PHP Version => 5.5.38

For my version of OSX, Apple shipped 5.5.38, so I’ll pull down the code package for that version.

Then, unbundle it and go to the php extension directory:

tar xvfz php-5.5.38.tar.bz2 cd php-5.5.38/ext/pgsql

Now the magic part. In order to build the extension, without building the whole of PHP, we need to tell the extension how the PHP that Apple ships was built and configured. How do we do that? We run phpize in the extension directory.

> /usr/bin/phpize Configuring for: PHP Api Version: 20121113 Zend Module Api No: 20121212 Zend Extension Api No: 220121212

The phpize process reads the configuration of the installed PHP and sets up a local build environment just for the extension. All of a sudden we have a ./configure script, and we’re ready to build (assuming you have installed the MacOSX command-line developers tools with XCode).

> ./configure \ --with-php-config=/usr/bin/php-config \ --with-pgsql=/opt/pgsql/10 > make

Note that I have my own build of PostgreSQL in /opt/pgsql. You’ll need to supply the path to your own install of PgSQL so that the PHP extension can find the PgSQL libraries and headers to build against.

When the build is complete, you’ll have a new modules/ directory in the extension directory. Now figure out where your system wants extensions copied, and copy the module there.

> php --info | grep extension_dir extension_dir => /usr/lib/php/extensions/no-debug-non-zts-20121212 => /usr/lib/php/extensions/no-debug-non-zts-20121212 > sudo cp modules/pgsql.so /usr/lib/php/extensions/no-debug-non-zts-20121212

Finally, you need to edit the /etc/php.ini file to enable the new module. If the file doesn’t already exist, you’ll have to copy in the template version and then edit that.

sudo cp /etc/php.ini.default /etc/php.ini sudo vi /etc/php.ini

Find the line for the PgSQL module and uncomment and edit it appropriately.

;extension=php_pdo_sqlite.dll extension=pgsql.so ;extension=php_pspell.dll

Now you can check and see if it has picked up the PgSQL module.

> php --info | grep PostgreSQL PostgreSQL Support => enabled PostgreSQL(libpq) Version => 10.0 PostgreSQL(libpq) => PostgreSQL 10.0 on x86_64-apple-darwin15.6.0, compiled by Apple LLVM version 8.0.0 (clang-800.0.42.1)

That’s it!

Categories: OSGeo Planet

Gis-Lab: Обработка данных аэрофотосъемки средствами открытого пакета OpenDroneMap

OSGeo Planet - Thu, 2017-10-12 07:46

В связи с бурным развитием как фотограмметрических технологий, так и индустрии простых в освоении БПЛА оснащенных фото/видео-аппаратурой, у специалистов самых разных профилей стал расти интерес к возможностям организации аэрофотосъемки и обработки получаемых данных для дальнейшей работы с географическими продуктами, такими как ортофотопланы, цифровые модели местности, трёхмерные модели. На рынке представлено большое количество решений как аппаратных (преимущественно БПЛА), так и программных. Программные продукты для фотограмметрической обработки данных стали разрабатывать практически все крупные вендоры (Autodesk, Trimble, …), также появилось множество новых компаний, продвигающих собственные пакеты (Agisoft, Pix4D, DroneDeploy, …). Параллельно начали развиваться и проекты с открытым исходным кодом. Подробное описание установки на виртуальную машину и основы использования одного из наиболее удачных открытых пакетов – OpenDroneMap – рассмотрены в представленной статье.

Прочитать | Обсудить


Categories: OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 5.1 ‘Servicios web (Introducción a las IDE)’

OSGeo Planet - Wed, 2017-10-11 22:29

Con el primer vídeo del quinto módulo, que trata sobre el acceso a servicios web desde gvSIG, nos introducimos en un concepto fundamental cuando hablamos de la gestión eficiente de la información geográfica: las Infraestructuras de Datos Espaciales (IDE). Es tal su importancia que son cada vez los países o regiones del mundo que legislan para hacer efectiva su implantación en toda administración que genere información geográfica digital.

Las IDE se consideran el sistema idóneo para gestionar, en su totalidad, la información geográfica de cualquier organización y, por supuesto, de un ayuntamiento. En futuros módulos de este curso veremos gvSIG Online, la solución libre para ponerlas en marcha. En el módulo de hoy veremos cómo trabajar con los servicios web de mapas que pueden generar las IDE desde el SIG de escritorio.

Actualmente, una gran cantidad de administraciones ofrecen su cartografía de forma pública para poder cargar a través de servicios web. Gracias al uso de determinados estándares es posible poder acceder a estos servicios desde gvSIG Desktop, lo que nos permite cargar cartografía en nuestro proyecto sin necesidad de descargar nada en disco.

Para poder entender mejor esta parte en gvSIG comenzaremos con un primer vídeo teórico sobre introducción a las Infraestructuras de Datos Espaciales, donde explicaremos qué son los servicios web, qué tipos hay, y algunos enlaces donde se recopilan algunos estos servicios disponibles.

En este módulo no es necesario descargar ninguna cartografía, ya que es un vídeo totalmente teórico.

El primer vídeo-tutorial de este quinto módulo es el siguiente:

Post relacionados:

Filed under: gvSIG Desktop, IDE, spanish, training Tagged: IDE, OGC, Servicios web
Categories: OSGeo Planet

Even Rouault: GDAL and cloud storage

OSGeo Planet - Wed, 2017-10-11 17:48
In the past weeks, a number of improvements related to access to cloud storage have been committed to GDAL trunk (future GDAL 2.3.0)

Cloud based virtual file systems
There was already support to access private data in Amazon S3 buckets through the /vsis3/ virtual file system (VFS). Besides a few robustness fixes, a few new capabilities have been added, like creation and deletion of directories inside a bucket with VSIMkdir() / VSIRmdir(). The authentication methods have also been extended to support, beyond the AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID environment variables, the other ways accepted by the "aws" command line utilities, that is to say storing credentials in the ~/.aws/credentials or ~/.aws/config files. If GDAL is executed since a Amazon EC2 instance that has been assigned rights to buckets, GDAL will automatically fetch the instance profile credentials.
The existing read-only /vsigs/ VFS for Google Cloud Storage as being extended with write capabilities (creation of new files), to be on feature parity with /vsis3/. The authentication methods have also been extended to support OAuth2 authentication with a refresh token, or with service account authentication. The credentials can be stored in a ~/.boto configuration file. And when run from a Google Compute Engine virtual machine, GDAL will automatically fetch the instance profile credentials.
Two new VFS have also been added, /vsiaz/ for Microsoft Azure Blobs and /vsioss/ for Alibaba Cloud Object Storage Service. They support read and write operations similarly to the two previously mentioned VFS.

To make file and directory management easy, a number of Python sample scripts have been created or improved:
gdal_cp.py my.tif /vsis3/mybucket/raster/
gdal_cp.py -r /vsis3/mybucket/raster /vsigs/somebucket
gdal_ls.py -lr /vsis3/mybucket
gdal_rm.py /vsis3/mybucket/raster/my.tif
gdal_mkdir.py /vsis3/mybucket/newdir
gdal_rmdir.py -r /vsis3/mybucket/newdir

Cloud Optimized GeoTIFF
Over the last past few months, there has been adoption by various actors of the cloud optimized formulation of GeoTIFF files, which enables clients to efficiently open and access portions of a GeoTIFF file available through HTTP GET range requests.

Source code for a online service that offers validation of cloud optimized GeoTIFF (using GDAL and the validate_cloud_optimized_geotiff.py script underneath) and can run as a AWS Lambda function is available. Note: as the current definition of what is or is not a cloud optimized formulation has been uniteraly decided up to now, it cannot be excluded that it might be changed on some points (for example relaxing constraints on the ordering of the data of each overview level, or enforcing that tiles are ordered in a top-to-bottom left-to-right way)

GDAL trunk has received improvements to speed up access to sub windows of a GeoTIFF file by making sure that the tiles that participate to a sub-window of interest are requested in parallel (this is true for public files accessed through /vsicurl/ or with the four above mentioned specialized cloud VFS), by reducing the amount of data fetched to the strict minimum and merging requests for consecutive ranges. In some environments, particularly when accessing to the storage service of a virtual machine of the same provider, HTTP/2 can be used by setting the GDAL_HTTP_VERSION=2 configuration option (provided you have a libcurl recent enough and built against nghttp2). In that case, HTTP/2 multiplexing will be used to request and retrieve data on the same HTTP connection (saving time to establish TLS for example). Otherwise, GDAL will default to several parallel HTTP/1.1 connections. For long lived processes, efforts have been made to re-use as much as possible existing HTTP connections.

Categories: OSGeo Planet

gvSIG Team: Impresiones tras las 4as Jornadas gvSIG México

OSGeo Planet - Tue, 2017-10-10 01:24

La pasada semana tuvieron lugar las 4as Jornadas gvSIG México. Por cuarto año consecutivo la Comunidad Mexicana de gvSIG ha celebrado un encuentro lleno de actividades, con interesantes ponencias y talleres llenos a rebosar.

Creo que la mejor forma de mostrar mis impresiones de cómo fueron la jornadas es publicar unas pocas imágenes sobre las mismas. Tan sólo repetiré alguna de las palabras que utilicé en la clausura del evento: las jornadas de gvSIG en México han sido semillas que han sumado a un proceso que va más allá de la migración a software libre de tal o cual organismo, han puesto en marcha algo mucho más importante desde el área de la geomática, el camino hacia la independencia tecnológica de un país.

Unas jornadas que se llevaron a cabo al mismo tiempo que se estaban celebrando las 9as Jornadas de Latinoamérica y Caribe; en dos semanas tenemos las 13as Jornadas Internacionales…indicadores, junto a muchos otros, que muestran que gvSIG es un proyecto consolidado y en constante crecimiento, cuyo uso se extiende a más de 160 países. Y esperaros al 2018, que va a deparar muchas sorpresas…

Filed under: events, spanish Tagged: Conferencia, Culiacán, jornadas, México, Sinaloa
Categories: OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 4.2 ‘Tablas de atributos (unión de tablas)’

OSGeo Planet - Mon, 2017-10-09 17:46

Ya está disponible el segundo vídeo del cuarto módulo, en el que continuamos trabajando con las tablas de atributos. En este caso lo que haremos será ver cómo realizar unión de tablas, muy útil para cuando tenemos una capa con cartografía de nuestro ayuntamiento (como por ejemplo una capa con las parcelas o barrios), y queremos unirle información de una tabla externa como puede ser la población de cada parcela o barrio. Para ello necesitaremos que ambas tablas tengan un campo con valores comunes, que será por los que las unirá.

En el tercer módulo tenéis toda la información sobre cómo instalar gvSIG, y en el segundo, en el apartado de preguntas frecuentes tenéis toda la información sobre cómo preguntar las dudas que tengáis durante el curso.

En este módulo se trabaja con un asistente para abrir un fichero CSV, que por defecto no suele ir instalado en gvSIG 2.3.1. Para poder instalarlo podéis ver la información del siguiente post.

La cartografía a utilizar en este módulo ya estaba disponible en el post anterior, pero si no la habéis descargado podéis hacerlo desde el siguiente enlace.

El segundo vídeo-tutorial de este cuarto módulo es el siguiente:

Post relacionados:

Módulo 1: Diferencias entre SIG y CAD

Módulo 2: Introducción a los Sistemas de Referencia

Módulo 3: Vistas, capas, simbología, etiquetado

Módulo 4.1: Tablas de atributos (información alfanumérica)

Filed under: gvSIG Desktop, spanish, training Tagged: ayuntamientos, gestión municipal, información alfanumérica, Tablas, tablas de atributos, unión de tablas
Categories: OSGeo Planet

From GIS to Remote Sensing: Developing the SCP 6: Main interface and Multiple band sets

OSGeo Planet - Mon, 2017-10-09 08:00
I am updating the Semi-Automatic Classification Plugin (SCP) to version 6 (codename Greenbelt) which will be compatible with the upcoming QGIS 3.

In this previous post, I described the main changes to the SCP dock, and in particular the new tabs designed to optimize the space on the display.
In this post I present the redesigned Main interface that contains the SCP tools.
In addition, I have implemented a profound change to the SCP core, which is the ability to define multiple band setsthis opens many possibilities for new tools that can exploit the combination of bands (I am thinking about raster mosaic and more tools) which I'll try to include in SCP 6.

Main interface
Categories: OSGeo Planet

OSGeo.nl: GRASS GIS crash course

OSGeo Planet - Sun, 2017-10-08 18:44

On 3 November 2017 there will be a GRASS GIS crash course, organized by OSGeo.nl and the Faculty of Geo-Information Science and Earth Observation of ITC.  The course will provide participants with an overview of the software capabilities and a hands-on experience in raster, vector and time series processing with the Open Source software GRASS GIS 7. For more information, go to https://osgeo.nl/grassgis-course/. You can register on our Meetup page [update: maximum number of participants has been reached, registration closed].



Course structure/contents

The course consists of mainly practical sessions with a short intro to basic concepts at the beginning. All the code and material will be publicly available.

  • We will cover the following introductory topics, among others:
  • GRASS database, locations and mapsets
  • Working with different data types (vector, raster, 3D raster formats, time series)
  • Different interfaces (GUI, CLI, Python)
  • Region and mask
  • Scripting examples
  • Visualization of spatial data; scale bar, symbols, grids, color tables, histograms
  • Where to find help

After the introduction with simple examples, we will go together through a guided exercise to demonstrate a full workflow in GRASS GIS, involving raster, vector and temporal data. In the end, participants will have the chance to follow three different tutorials: Remote sensing analysis using satellite data, Time series processing and spatial point interpolation. Teachers will be available for questions and explanations.

When and where?
  • Friday, November 3rd, 2017 from 10.30 to 16.30 (with 1-hour for lunch break)
  • ITC – Faculty of Geo-Information Science and Earth Observation. University of Twente. Hengelosestraat 99, 7514 AE, Enschede. The Netherlands.
Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Movement data in GIS #8: edge bundling for flow maps

OSGeo Planet - Sun, 2017-10-08 17:50

If you follow this blog, you’ll probably remember that I published a QGIS style for flow maps a while ago. The example showed domestic migration between the nine Austrian states, a rather small dataset. Even so, it required some manual tweaking to make the flow map readable. Even with only 72 edges, the map quickly gets messy:

Raw migration flows between Austrian states, line width scaled by flow strength

One popular approach in the data viz community to deal with this problem is edge bundling. The idea is to reduce visual clutter by generate bundles of similar edges. 

Surprisingly, edge bundling is not available in desktop GIS. Existing implementations in the visual analytics field often run on GPUs because edge bundling is computationally expensive. Nonetheless, we have set out to implement force-directed edge bundling for the QGIS Processing toolbox [0]. The resulting scripts are available at https://github.com/dts-ait/qgis-edge-bundling.

The main procedure consists of two tools: bundle edges and summarize. Bundle edges takes the raw straight lines, and incrementally adds intermediate nodes (called control points) and shifts them according to computed spring and electrostatic forces. If the input are 72 lines, the output again are 72 lines but each line geometry has been bent so that similar lines overlap and form a bundle.

After this edge bundling step, most common implementations compute a line heatmap, that is, for each map pixel, determine the number of lines passing through the pixel. But QGIS does not support line heatmaps and this approach also has issues distinguishing lines that run in opposite directions. We have therefore implemented a summarize tool that computes the local strength of the generated bundles.

Continuing our previous example, if the input are 72 lines, summarize breaks each line into its individual segments and determines the number of segments from other lines that are part of the same bundle. If a weight field is specified, each line is not just counted once but according to its weight value. The resulting bundle strength can be used to create a line layer style with data-defined line width:

Bundled migration flows

To avoid overlaps of flows in opposing directions, we define a line offset. Finally, summarize also adds a sequence number to the line segments. This sequence number is used to assign a line color on the gradient that indicates flow direction.

I already mentioned that edge bundling is computationally expensive. One reason is that we need to perform pairwise comparison of edges to determine if they are similar and should be bundled. This comparison results in a compatibility matrix and depending on the defined compatibility threshold, different bundles can be generated.

The following U.S. dataset contains around 4000 lines and bundling it takes a considerable amount of time.

One approach to speed up computations is to first use a quick clustering algorithm and then perform edge bundling on each cluster individually. If done correctly, clustering significantly reduces the size of each compatibility matrix.

In this example, we divided the edges into six clusters before bundling them. If you compare this result to the visualization at the top of this post (which did not use clustering), you’ll see some differences here and there but, overall, the results are quite similar:

Looking at these examples, you’ll probably spot a couple of issues. There are many additional ideas for potential improvements from existing literature which we have not implemented yet. If you are interested in improving these tools, please go ahead! The code and more examples are available on Github.

For more details, leave your email in a comment below and I’ll gladly send you the pre-print of our paper.

[0] Graser, A., Schmidt, J., Roth, F., & Brändle, N. (accepted) Untangling Origin-Destination Flows in Geographic Information Systems. Information Visualization – Special Issue on Visual Movement Analytics.

Read more:

Categories: OSGeo Planet

OSGeo-fr: 5eme rencontre des utilisateurs de QGIS

OSGeo Planet - Fri, 2017-10-06 11:16

L'OSGeo-fr et Montpellier SupAgro organisent la cinquième rencontre de la communauté francophone QGIS.

Cette rencontre se déroulera les 14 et 15 décembre à Montpellier SupAgro.

La première journée est organisée sous forme de barcamp : des ateliers informels où vous (utilisateurs) pouvez proposer des sujets, présenter des travaux, échanger, discuter. Ce moment a vocation de permettre l'échange entre utilisateurs, contributeurs et financeurs. N'hésitez pas à venir découvrir une forme originale de contributions autour de QGIS !

Voici quelques sujets évoqués en 2015-2016 :

  • comment traduire QGIS ?
  • comment utiliser QGIS sur une tablette (QField) ?
  • comment créer une carte avec QGIS et la diffuser sur le net ?

Durant la seconde journée, plusieurs conférences seront proposées. Cette année, le thème de cette journée est "QGIS 3.0 : que va changer cette nouvelle version pour les utilisateurs ?".

L'appel à mécénat a été diffusé cette semaine et celui pour les présentations le sera tout bientôt. Restez attentif !

Pour toute information ou demande, utilisez le formulaire de contact prévu à cet effet.

À propos de Montpellier SupAgro et la formation AgroTIC

Montpellier SupAgro est un établissement d'enseignement et de recherche qui forme des ingénieurs agronomes. La formation AgroTIC est une des formations portées par Montpellier SupAgro. Chaque année nous formons une quinzaine d'ingénieurs agronomes avec une double compétence en géomatique. Qgis constitue un outil central de notre formation d'une part pour les fonctionnalités de gestion de l'information spatiale qu'il offre et d'autre part parce qu'il est libre et gratuit et pourra être réutilisé par nos futurs diplomés quelque soit leur contexte professionnel.
Depuis 3 ans, nous organisons un évènement d'échange autour de Qgis au cours duquel nos étudiants peuvent rencontrer des professionnels qui utilisent ce logiciel. Cet évènement leur permet aussi de valoriser les enseignements qu'ils ont suivi puisqu'ils présentent également les nouvelles fonctionnalités de QGIS devant les professionnels.
Montpellier SupAgro : www.supagro.fr
AgroTIC : www.agrotic.org

À propos de l'OSGeo-fr

L’association OSGeo.fr est la représentation Francophone de la fondation Open Source Geospatial dont la mission est d’aider et de promouvoir le développement collaboratif des données et des technologies géospatiales ouvertes. L’association sert d’entité légale envers qui les membres de la communauté peuvent contribuer au code, aux finances et aux autres ressources, s’assurer que leurs contributions seront maintenues au bénéfice du public.

L’OSGeo.fr sert également d’organisation de référence et d’assistance pour la communauté géospatiale libre, et fournit un forum commun et une infrastructure partagée pour améliorer la collaboration entre les projets.

La participation est ouverte à l’ensemble de la communauté Open Source. Tous les travaux de l’association sont publiés sur des forums publics où peut s’investir une communauté libre de participants. Les projets de la Fondation OSGeo sont tous librement disponibles et utilisables sous une licence Open Source certifiée par l’OSI.


Categories: OSGeo Planet

Andrea Antonello: JGrasstools back in black: The Horton Machine - Part 1

OSGeo Planet - Thu, 2017-10-05 15:21
I have to admit that when in far 2002, working at the University of Trento, Faculty of Environmental Engineering, we published the first release of The Horton Machine, I never thought we would ever change that name. The Horton Machine was a collection of around 40 GRASS modules written in C and dedicated to advanced Hydrology and Geomorphology. They represented the effort of the passed 10 years (now around 20) of professor Riccardo Rigon and his team.
At that time Riccardo and I were dreaming about a nice to use GUI for GRASS that would allow GRASS to be used more outside of the academic domain. In 2003 we started the JGrass project with that objective: create a userfriendly GUI for GRASS. The reaction of the GRASS community was bad, mostly (so they said) because Java was not open source. Also QGis was coming and becoming the natural choice for being an interface to GRASS.
Not at all in the mood of religious wars in 2006 we decided to join the java tribe and moved our resources to support the uDig project, where we happily lived and developed for many many years. We kind of stayed between two worlds, still using GRASS and its mapsets, but living in the userfriendly java world. :-)
At that time the processing libraries for Hydrology and Geomorphology (as well as LiDAR and forestry later on) were extracted into a library that could be used in standalone mode or inside uDig. That library, as a logical follow-up, got the name JGrasstools.
Had I only known better! In some (rare, but still!) occasions I and other JGrasstools developers have been asked why we still use the name JGrasstools, since we are not directly "bound" to GRASS. Well, I have been fighting over this a few times and had no hard feeling about this, apart of the huge work that it would have been to change everything. 
The last time it happened, was at the Foss4g conference in Paris. At the end of a great presentation given by Silvia about the tools we developed for forestry management using LiDAR data, (again) a member of the GRASS community asked the same old question in the questions interval dedicated to the presentation: Why do you still call it JGrasstools....?
This was the final straw for me. I still have to understand why people do certain things, but one thing was sure for me: we had to change that name, to make some of the GRASS community members sleep sweet dreams, and to be finally free!
So this is it. After 15 years of continuous development on the JGrasstools core, we go back to our origins: The Horton Machine.
The Horton Machine is now something more than just Hydrology and Geomorphology, there are other projects that support interaction with the mobile digital field mapping app Geopaparazzi, a module that supports interaction with spatial databases (also on android), the LESTO modules developed with the team of professor Giustino Tonon at the Free University of Bolzano... and beyond other things... also the plugins for the Desktop GIS gvSIG.
It has been quite an exercise to make this namespace migration and has taken me days between code refactoring, domain registration, maven publishing updates, documentation updating (still much work to be done there) and and and... but it is done now. Many will sleep sweet dreams, and I will be the first. And maybe at the next conference someone will ask a question related to the content of the presentation. Don't know what? A hint: "How did you get such a high single tree extraction rate from LiDAR data with your tools?" ;-)  What will happen now?Before the 13 international gvSIG conference we will do the first HortonMachine branded release and the together with it the connected release if gvSIG plugins.
Also maven releases of the modules will be done. At the time JGrasstools was at version 0.8.1. The HortonMachine will most probably start at 0.9.0. It sure should have been a major number, but well, we still need to reach the first major. :-)
In the next post we will show you what you will find in the release. Stay tuned!

Categories: OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 4.1 ‘Tablas de atributos (información alfanumérica)’

OSGeo Planet - Thu, 2017-10-05 07:08

Ya está disponible el primer vídeo del cuarto módulo, en el que veremos cómo trabajar con las tablas de atributos. En este primer vídeo de este módulo veremos cómo realizar selección de elementos, tanto gráficamente como desde la tabla, y también utilizando diferentes funcionalidades, como por ejemplo los filtros.

En el módulo anterior tenéis toda la información sobre cómo instalar gvSIG, y en el segundo módulo, en el apartado de preguntas frecuentes tenéis toda la información sobre cómo preguntar las dudas que tengáis durante el curso.

Podéis descargar la cartografía a utilizar en este módulo desde el siguiente enlace.

El primer vídeo-tutorial de este cuarto módulo es el siguiente:

Post relacionados:

Módulo 1: Diferencias entre SIG y CAD

Módulo 2: Introducción a los Sistemas de Referencia

Módulo 3: Vistas, capas, simbología, etiquetado

Filed under: gvSIG Desktop, spanish, training Tagged: ayuntamientos, filtros, gestión municipal, información alfanumérica, tablas de atributos
Categories: OSGeo Planet

OSGeo.nl: OSGeo.nl Dag 2017 – Geo.Samen.Doen.

OSGeo Planet - Wed, 2017-10-04 21:53

Het is inmiddels traditie dat de jaarlijkse OSGeo.nl Dag op GeoBuzz plaatsvindt. De afgelopen jaren was ons programma altijd het drukst bezochte en best gewaardeerde onderdeel. Ook dit jaar wordt het natuurlijk weer geweldig! Zet dus 22 November 2017 in je agenda! Onder het motto “Samenwerking Versnelt” haken wij in bij het GeoBuzz-thema: “Geo Versnelt”. 

Wat gaan we doen? Het programma is nog in voorbereiding, maar de rode draad zal “Samenwerking” zijn. In grote lijnen kun je het volgende verwachten:

De Ochtend – QGIS Morning!

Thema van het ochtendprogramma is Samenwerking & QGIS. De laatste jaren is QGIS de meest zichtbare en populaire  component die de Open Source geo-wereld heeft voortgebracht. Maar wat is de oorzaak van haar groei en populariteit? Wat maakt QGIS “anders”? Hoe QGIS effectief in te zetten? We laten zien hoe het QGIS Project zelf een samenwerking tussen gebruikers en ontwikkelaars bevordert. Hoe “de markt” hierop inhaakt met een aanbod van QGIS cursussen en ondersteuning. Hoe QGIS gebruikers zelfstandig complexe oplossingen voor hun organisatie ontwikkelen. Slogan: “Doe het niet alleen, zoek de samenwerking”. Onder de sprekers zijn de voornaamste QGIS-experts in Nederland. Zij zullen voorbeelden en “cases” geven ter inspiratie en je uitnodigen om onderdeel te worden van de Nederlandse QGIS community in aanloop naar de QGIS Gebruikersdag op 31 januari 2018.

De Middag – Gebruikers en Aanbieders!

Open Source Geo lijkt op afstand een wat ondoorgrondelijk geheel van projecten en producten. Toch is er een groeiend aantal bedrijven en organisaties dat hier effectief gebruik van maakt. Hoe doen zij dit? Wat is hun geheim? We kunnen natuurlijk verklappen dat het antwoord in “Samenwerking” zit, maar dat is te vrijblijvend. Daarom staan in het middagprogramma vooral gebruikers centraal die effectief Open Source hebben ingezet. Zij zullen uiteenzetten welke stappen zij hebben ondernomen, welke hobbels zij daarbij zijn tegengekomen. Ook zullen aanbieders van Open Source Geo oplossingen uiteenzetten hoe zij hun klanten hierbij hebben geholpen.

Laat je daarom ook dit jaar ‘updaten’ op de OSGeo.nl Dag. Kom luisteren naar en praten met gebruikers die ervaringen delen en ontwikkelaars die hun projecten tonen. Kijk voor meer info en/of als je iets wilt presenteren op de website https://osgeo.nl of stuur een email naar events@osgeo.nl.

Categories: OSGeo Planet

Fernando Quadro: Curso de QGIS Básico

OSGeo Planet - Wed, 2017-10-04 17:44

Neste curso de QGIS Básico ministrado pela Geocursos você terá uma introdução ao software QGIS Desktop. São apresentados desde como iniciar um projeto, passando pelos procedimentos básicos de edição de dados geográficos e criação de mapas temáticos até a geração de mapas para impressão.

O curso é uma excelente oportunidade para aqueles que desejam conhecer o QGIS, suas ferramentas e aplicabilidade em projetos de SIG. Embora não existam requisitos prévios rígidos, a familiaridade com os conceitos básicos de Geotecnologias é recomendável.

01. Apresentação do QGIS
02. Interface do Software
03. Inicialização de Projetos no QGIS
04. Ferramentas de Seleção
05. Consultas por Atributo
06. Simbologia e Rotulação
07. Elaboração de Mapas Temáticos
08. Manipulação da Tabela de Atributos
09. Edição de Atributos
10. Criação e configuração de Hiperlink
11. Medição de Áreas e Distâncias
12. União de Tabelas
13. Gerar camada a partir de Coordenadas
14. Extração de Coordenadas
15. Operações Espaciais
16. Integração com Base de Dados Espacial
17. Geração de Mapas para Impressão (Layout)
18. Outros Tópicos Relevantes

Para saber mais informações, e realizar a sua inscrição basta acessar o site:


A Geocursos informa que estão abertas as inscrições, e as vagas são limitadas para o Curso Online de QGIS Básico que acontecerá entre os dias 04 e 13 de Dezembro nas segunda, quartas e sextas das 20h00 as 23h00 (Horário de Brasília).

Categories: OSGeo Planet

Oslandia: Oslandia is baking some awesome QGIS 3 new features

OSGeo Planet - Wed, 2017-10-04 15:35

QGIS 3.0 is now getting closer and closer, it’s the right moment to write about some major refactor and new features we have been baking at Oslandia.

A quick word about the release calendar, you probably felt like QGIS 3 freeze was expected for the end of August, didn’t you?

In fact, we have so many new major changes in the queue that the steering committee (PSC), advised by the core developers, decided to push twice the release date up up to the 27 of October. Release date has not be been pushed (yet).

At Oslandia we got involved in a dark list of hidden features of QGIS3.

They mostly aren’t easy to advertised visually, but you’ll appreciate them for sure!

  • Add  capabilities to store data in the project
    • add a new .qgz zipped file format container
    • have editable joins, with upsert capabilities (Insert Or Update)
    • Transparently store  and maintain in sync data in a sqlite database. Now custom labeling is pretty easy!
  • Coordinating work and tests on new node tool for data editing
  • Improving Z / m handling in edit tools and layer creation dialogs
  • Ticket reviewing and cleaning

Next articles will describe some of those tasks soon.

This work was a great opportunity to ramp up a new talented developer with commit rights on the repository! Welcome and congratulations to Paul our new core committer !

All this was possible with the support of many actors, but also thanks to the fundings of QGIS.org via Grant Applications or direct funding of QGIS server!

A last word, please help us in testing QGIS3, it’s the perfect moment to stress it, bugfix period is about to start !




Categories: OSGeo Planet

gvSIG Team: Software libre, Economía Circular y gestión eficiente de los recursos

OSGeo Planet - Wed, 2017-10-04 14:06

Una Europa que utilice eficazmente los recursos” es una de las siete iniciativas emblemáticas que forman parte de la estrategia Europa 2020 que pretende generar un crecimiento inteligente, sostenible e integrador. Actualmente es la principal estrategia de Europa para generar crecimiento y empleo, con el respaldo del Parlamento Europeo y el Consejo Europeo.

Esta iniciativa pretende crear un marco político destinado a apoyar el cambio a una economía eficiente en el uso de los recursos, contemplando objetivos como:

  • Mejorar los resultados económicos al tiempo que se reduce el uso de los recursos;
  • Identificar y crear nuevas oportunidades de crecimiento económico e impulsar la innovación y la competitividad de la UE;
  • Garantizar la seguridad del suministro de recursos esenciales;

Para todos aquellos que trabajamos en proyectos libres como gvSIG la relación directa de estos objetivos y la utilización (y re-utilización) de software libre en las administraciones públicas es indiscutible.

Software libre es reutilización de recursos, software libre es impulso de la innovación y la competitividad, software libre es seguridad de acceso a suministros (tecnología, sistemas de información), software es -en definitiva- una apuesta por el crecimiento inteligente, sostenible e integrador.

Además, esta iniciativa introduce en las políticas de la UE el concepto de “economía circular”, un concepto económico ligado con la sostenibilidad, y cuyo objetivo es que el valor de los productos, los materiales y los recursos se mantenga en la economía durante el mayor tiempo posible, y que se reduzca al mínimo la generación de residuos. Se trata de implementar una nueva economía, circular -no lineal-, basada en el principio de «cerrar el ciclo de vida» de los productos, los servicios, los residuos, los materiales, el agua y la energía.

El concepto de economía circular rara vez lo he visto asociado directamente al software libre, estando más centrado en “recursos materiales” que en “conocimiento”, sin embargo creo que acopla perfectamente y no debería desligarse su impulso de la adopción de las únicas licencias de software que garantizan los objetivos que persigue la economía circular. Valga este post para aportar nuestro granito de arena a ello.

Filed under: opinion, software libre, spanish Tagged: economía circular
Categories: OSGeo Planet

gvSIG Team: Towards gvSIG 2.4: Projects preview

OSGeo Planet - Wed, 2017-10-04 13:14

Every new gvSIG Desktop version includes small improvements, some of which deserves a special mention because of its usefulness. It’s the case of the project preview that is included in gvSIG 2.4.

It allows a thing as easy as seeing an image of the gvSIG Desktop projects, that can be very helpful sometimes to identify the project that we want to open. This image is updated each time that we save changes at the project.

Here you can watch a video about how it works:

Filed under: english, gvSIG Desktop, testing Tagged: gvSIG 2.4
Categories: OSGeo Planet

Oslandia: Refresh your maps FROM postgreSQL !

OSGeo Planet - Wed, 2017-10-04 13:09

Continuing our love story with PostgreSQL and QGIS, we asked QGIS.org a grant application during early 2017 spring.

The idea was to take benefit of very advanced PostgreSQL features, that probably never were used in a Desktop GIS client before.

Today, let’s see what we can do with the PostgreSQL NOTIFY feature!

Ever dreamt of being able to trigger things from outside QGIS? Ever wanted a magic stick to trigger actions in some clients from a database action?

X All The Y Meme | REFRESH QGIS FROM THE DATABASE !!! | image tagged in memes,x all the y | made w/ Imgflip meme maker


NOTIFY is a PostgreSQL specific feature allowing to generate notifications on a channel and optionally send a message — a payload in PG’s dialect .

In short, from within a transaction, we can raise a signal in a PostgreSQL queue and listen to it from a client.

In action

We hardcoded a channel named “qgis” and made QGIS able to LISTEN to NOTIFY events and transform them into Qt’s signals. The signals are connected to layer refresh when you switch on this rendering option.

Optionnally, adding a message filter will only redraw the layer for some specific events.

This mechanism is really versatile and we now can imagine many possibilities, maybe like trigger a notification message to your users from the database, interact with plugins, or even code a chat between users of the same database  (ok, this is stupid) !


More than just refresh layers?

The first implementation we chose was to trigger a layer refresh because we believe this is a good way for users to discover this new feature.

But QGIS rocks hey, doing crazy things for limited uses is not the way.

Thanks to feedback on the Pull Request, we added the possibility to trigger layer actions on notification.

That should be pretty versatile since you can do almost anything with those actions now.


QGIS will open a permanent connection to PostgreSQL to watch the notify signals. Please keep that in mind if you have several clients and a limited number of connections.

Notify signals are only transmitted with the transaction, so when the COMMIT is raised. So be aware that this might not help you if users are inside an edit session.

QGIS has a lot of different caches, for attribute table for instance. We currently have no specific way to invalidate a specific cache, and then order QGIS to refresh it’s attribute table.

There is no way in PG to list all channels of a database session, that’s why we couldn’t propose a combobox list of available signals in the renderer option dialog. Anyway, to avoid too many issues, we decided to hardcode the channel name in QGIS with the name “qgis”. If this is somehow not enough for your needs, please contact us!


The github pull request is here : https://github.com/qgis/QGIS/pull/5179

We are convinced this would be really useful for real time application, let us know if that makes some bells ring on your side!

More to come soon, stay tuned!



Categories: OSGeo Planet
Syndicate content