OSGeo Planet

Free and Open Source GIS Ramblings: OSGeo Code Sprint in Bonn

OSGeo Planet - 9 hours 25 min ago

It’s been a great week in Bonn! I joined the other members of the QGIS project at the pre-FOSS4G code sprint at the Basecamp, the weirdest location we’ve had for a developer meeting so far. We used this opportunity to have a face-to-face meeting of the QGIS PSC  with special guests Matthias Kuhn (on QGIS 3.0 and bug tracker discussions) and Lene Fischer (on community team issues)  – notes here.

picture by Tim Sutton

QGIS PSC meeting in action (from left to right: Otto Dassau, Paolo Cavallini, Anita Graser, Andreas Neumann, Jürgen E. Fischer), picture by Tim Sutton

I also finally took the time to compile a blog post on the results of the QGIS user survey 2015.

The code sprint was also a great opportunity to present the results of Akbar Gumbira’s Google Summer of Code project: the QGIS Resource Sharing plugin. This plugin makes it possible to easily share resources (such as SVG icons, symbol definitions, QGIS styles, and Processing scripts) with other QGIS users through an interface that closely resembles the well-known QGIS Plugin Manager. Akbar has also prepared great presentation with background info and screencasts showcasing his project.

QGIS Resource Sharing presentation, picture by @foss4g

QGIS Resource Sharing presentation, picture by @foss4g

The plugin is now available in the Plugin Repository and we have created the official QGIS Resources repository on Github. If you have symbols or styles that you want to share with the community, please create a resource collection and send us a pull request to add it to the official list.

Thanks to all the organizers who worked hard to make this one of the most well-organized and enjoyable code sprints I’ve ever been to. You are awesome!


Categories: OSGeo Planet

gvSIG Team: NASA WORLD WIND Europa Challenge 2016. List of nominees and gvSIG projects

OSGeo Planet - Fri, 2016-08-26 15:10

nasa

“The Europa Challenge has always had Europe’s INSPIRE Directive to guide project development. This year we continue to have INSPIRE guide us and more specifically, we are looking for urban management solutions. This year the Europa Challenge is asking the world’s *best and brightest* to deliver solutions serving city needs.

Almost every city needs the same data management tools as every other city. How can we help cities work together to be more sustainable, more livable and more resilient? If cities were able to share their solutions with each other, this would multiply their investment by the number of cities participating. Each city could develop different functionalities and then ‘share’ these with each other, massively increasing our planet’s collective productivity. “

This year gvSIG Association has two projects in the final list of nominees:

  • Improving the integration of NWW in gvSIG Desktop: Extrusion, vector data and EPSG support. More info: here.
  • gvSIG Online and Web World Wind, the solution for Spatial Data Infrastructures on Open Source software with 3D View. More info: here.

The 2016 projects are:

http://eurochallenge.como.polimi.it/projects2016


Filed under: events, gvSIG Desktop, gvSIG Online, press office, spanish Tagged: awards, Europa Challenge, nasa world wind
Categories: OSGeo Planet

QGIS Blog: What are trusted plugins?

OSGeo Planet - Fri, 2016-08-26 10:16

The core team of QGIS strives hard to provide the most advanced and user friendly GIS for free use by everyone. In the core QGIS project, every line of code that gets committed is subject to peer review when contributed by a non core developer. This gives us an opportunity to identify and correct inadvertent (or intentional) security issues that a developer may introduce into the code base. By contrast, all of the plugins that are published via the QGIS plugin repository are reviewed by the plugin developers themselves and we don’t have good insight into how much due diligence is applied to plugin code management.

The vast majority of our plugins (listed in http://plugins.qgis.org/ and inside your copy of QGIS) are developed by third parties, either individuals, companies, and institutions. As such, they are outside our direct control and the developers often relatively unknown to the QGIS community. We view this as a potential security risk. We are convinced the risk is small, because of many factors including the “many eyes” principle (the code is visible to everybody, and in use by thousands of people), but cannot exclude the possibility that someone tries to inject malicious code into a plugin.

In order to address this situation, we looked into the opportunity of implementing automatic tools to scan plugins, before their publication, and spot potential problems. Our research indicated that this approach would be difficult and costly, and easy to circumvent.

We (the PSC) therefore decided to implement a simple yet robust approach to security, based on the ‘web of trust’ principle: we trust people we know well in the community. You will see on the http://plugins.qgis.org web site that there is a ‘Trusted Author’ tag has been applied to plugins created by those members of the community that we know and trust.

The criteria for ‘Trusted Authors’ includes those community members that regularly meet at our QGIS developer meetings, and and those that are in almost daily contact with the core team via our developer mailing lists or background project discussions. The remaining plugins (and there are wonderful, reliable, robust, and useful plugins in the list) have not been given the ‘trusted’ label.

We would be delighted if a side effect of this choice would be to stimulate more active and direct involvement of plugin developers in the QGIS community. All plugin developers are therefore invited to join us at one of the next developer meetings (AKA HackFest), or otherwise become a recognized, active member of the community, so they can be integrated as ‘trusted’ plugin developers.


Categories: OSGeo Planet

gvSIG Team: FOSS4G 2016: gvSIG Video recordings (1)

OSGeo Planet - Fri, 2016-08-26 10:00

3D tools in gvSIG using Nasa World Wind

Digital field mapping with Geopaparazzi and gvSIG


Filed under: english, events, gvSIG Desktop, gvSIG Mobile, gvSIG Online Tagged: foss4g, Geopaparazzi, osgeo
Categories: OSGeo Planet

From GIS to Remote Sensing: Tutorial: Land Cover Classification and Mosaic of Several Landsat images

OSGeo Planet - Thu, 2016-08-25 15:07
This tutorial is about the land cover classification of several Landsat images in order to create a classification of a large study area using the Semi-Automatic Classification Plugin (SCP). For very basic tutorials see Tutorial 1: Your First Land Cover Classification and Tutorial 2: Land Cover Classification of Landsat Images.The study area of this tutorial is Costa Rica , a Country in Central America that has an extension of about 51,000 square kilometres. In particular, we are going to classify Landsat 8 and Landsat 7 images, masking clouds and creating a mosaic of classifications. We are going to identify the following land cover classes:
  1. Built-up;
  2. Vegetation;
  3. Soil;
  4. Water.
Following the video of this tutorial.



Categories: OSGeo Planet

From GIS to Remote Sensing: Major Update: Semi-Automatic Classification Plugin v. 4.9.0 - Sentinel-2 Download and Conversion to Reflectance

OSGeo Planet - Thu, 2016-08-25 14:55
This post is about a major update for the Semi-Automatic Classification Plugin for QGIS, version 4.9.0.


Following the changelog:
-updated the Sentinel-2 download for downloading single granules and selected bands
-updated the Sentinel-2 Pre processing tab for converting bands to TOA reflectance and surface reflectance (using DOS1)
Categories: OSGeo Planet

Paul Ramsey: PgSQL Indexes and "LIKE"

OSGeo Planet - Thu, 2016-08-25 09:05

Do you write queries like this:

SELECT * FROM users WHERE name LIKE 'G%'

Are your queries unexpectedly slow in PostgreSQL? Is the index not doing what you expect? Surprise! You’ve just discovered a PostgreSQL quirk.

TL;DR: If you are running a locale other than “C” (show LC_COLLATE to check) you need to create a special index to support pattern searching with the LIKE operator: CREATE INDEX myindex ON mytable (mytextcolumn text_pattern_ops). Note the specification of the text_pattern_ops operator class after the column name.

As a beginner SQL student, you might have asked “will the index make my ‘like’ query fast” and been answered “as long as the wildcard character is at the end of the string, it will.”

PgSQL Indexes and "LIKE"

That statement is only true in general if your database is initialized using the “C” locale (the North America/English-friendly UNIX default). Running with “C” used to be extremely common, but is less and less so, as modern operating systems automagically choose appropriate regional locales to provide approriate time and formatting for end users.

For example, I run Mac OSX and I live in British Columbia, an English-speaking chunk of North America. I could use “C” just fine, but when I check my database locale (via my collation), I see this:

pramsey=# show LC_COLLATE; lc_collate ------------- en_CA.UTF-8 (1 row)

It’s a good choice, it’s where I live, it supports lots of characters via UTF-8. However, it’s not “C”, so there are some quirks.

I have a big table of data linked to postal codes, this is what the table looks like:

Table "gis.postal_segments" Column | Type | Modifiers -------------------+--------------+----------- postal_code | text | not null segment | character(4) | Indexes: "postal_segments_pkey" PRIMARY KEY, btree (postal_code)

Note the index on the postal code, a standard btree.

I want to search rows based on a postal code prefix string, so I run:

EXPLAIN ANALYZE SELECT * FROM postal_segments WHERE postal_code LIKE 'V8V1X%'; QUERY PLAN ------------------------------------------------------------------------------------------------------ Seq Scan on postal_segments (cost=0.00..2496.85 rows=10 width=68) (actual time=30.320..34.219 rows=4 loops=1) Filter: (postal_code ~~ 'V8V1X%'::text) Rows Removed by Filter: 100144 Planning time: 0.250 ms Execution time: 34.263 ms (5 rows)

Ruh roh!

I have an index on the postal code, so why am I getting a sequence scan?!?! Because my index is no good for doing pattern matching in any collation other than “C”. I need a special index for that, which I create like this.

CREATE INDEX postal_segments_text_x ON postal_segments (postal_code text_pattern_ops);

The magic part is at the end, invoking text_pattern_ops as the opclass for this index. Now my query works as expected:

EXPLAIN ANALYZE SELECT * FROM postal_segments WHERE postal_code LIKE 'V8V1X%'; QUERY PLAN -------------------------------------------------------------------------------------------------------------------------------- Index Scan using postal_segments_text_x on postal_segments (cost=0.29..8.31 rows=10 width=68) (actual time=0.067..0.073 rows=4 loops=1) Index Cond: ((postal_code ~>=~ 'V8V1X'::text) AND (postal_code ~<~ 'V8V1Y'::text)) Filter: (postal_code ~~ 'V8V1X%'::text) Planning time: 0.532 ms Execution time: 0.117 ms (5 rows)

I have gotten so used to PostgreSQL doing exactly the right thing automatically that it took quite a long time to track down this quirk when I ran into it. I hope this page helps others save some time!

Categories: OSGeo Planet

gvSIG Team: 12as Jornadas Internacionales gvSIG: “Conoce el territorio. Gestiona la realidad”

OSGeo Planet - Thu, 2016-08-25 06:50

xx_12as J gvSIG esp

Del 30 de noviembre al 2 de diciembre de 2016 tendrán lugar en Valencia, España, las 12as Jornadas Internacionales gvSIG, organizadas por la Asociación gvSIG bajo el lema “Conoce el territorio. Gestiona la realidad“.

Este año las jornadas se celebrarán en una nueva sede, concretamente en la Escuela Técnica Superior de Ingeniería Geodésica, Cartográfica y Topográfica (Universitat Politècnica de València). En la página web del evento se facilitará más información sobre las salas de conferencias y talleres.

Ya está abierto el periodo para el envío de propuestas para comunicaciones para las Jornadas. Desde hoy pueden enviarse las propuestas a la dirección de correo electrónico conference-contact@gvsig.com, y serán valoradas por el comité científico de cara a su inclusión en el programa de las Jornadas. Existen dos modalidades de comunicación: ponencia y póster. Toda la información sobre las normas para la presentación de comunicaciones puede consultarse en el apartado ‘Comunicaciones’ de la web. El periodo de recepción de resúmenes finalizará el próximo 22 de septiembre.

Las organizaciones interesadas en colaborar en el evento pueden encontrar información en el apartado ‘¿Cómo colaborar?’ de la web de las jornadas.

¡Esperamos vuestra participación!


Filed under: events, press office, spanish Tagged: 12as Jornadas Internacionales gvSIG
Categories: OSGeo Planet

gvSIG Team: 12th International gvSIG Conference: “Know the territory. Manage the reality”. Call for papers is open.

OSGeo Planet - Wed, 2016-08-24 21:35

xxx_gvSIG eng

The 12th International gvSIG Conference, organized by the gvSIG Association, will be held from November 30th to December 2nd 2016 in Valencia, Spain, under the slogan “Know the territory. Manage the reality“.

This year they will be held in a new location, the School of Engineering in Geodesy, Cartography and Surveying (Polytechnic University of Valencia). More information about the conference rooms will be available at the event website soon.

Call for papers is now open. As of today communication proposals can be sent to the email address: conference-contact@gvsig.com; they will be evaluated by the scientific committee as to their inclusion in the conference program.

There are two types of communication: paper or poster. Information regarding to regulations on communication presentations can be found in the Communications section of the website. Abstracts will be accepted until September 22nd.

Organizations interested in collaborating in the event can find information in the section: How to collaborate.

We expect your participation!


Filed under: english, events, press office Tagged: 12th International gvSIG Conference
Categories: OSGeo Planet

gvSIG Team: GeoTIC: Solución para gestión del inventario de recursos TIC

OSGeo Planet - Wed, 2016-08-24 12:51

geoportal_geotic

GeoTIC es uno de los proyectos realizados en la Asociación gvSIG que ejemplifica el uso de nuestras tecnologías en software libre para la gestión de inventarios. Si con gvSIG Roads se dispone de un producto para gestión del inventario de carreteras, en este proyecto se pone en marcha una solución para optimizar la gestión del inventario de recursos TIC.

El Servicio de Atención al Usuario y al Puesto de Trabajo (SAUPT) de la Dirección General de Tecnologías de la Información y Comunicación (DGTIC) de la Generalitat Valenciana (GVA) tenía una serie de necesidades en su proceso de gestión.

La problemática principal a resolver era que gran parte de la gestión se realizaba en hojas de cálculo y que la información que necesitaban para realizar sus funciones estaba descentralizada, necesitando acceder a diferentes sistemas para obtenerla, y por lo tanto no se disponía de una visión de conjunto. Como solución a esta problemática nació el proyecto GeoTIC.

GeoTIC es tanto una herramienta para técnicos, la cual facilita su trabajo diario, como un cuadro de mandos, que ayuda en la toma de decisiones para optimizar los recursos de los que dispone dicho servicio.

geotic_alfa

La aplicación se ha desarrollado utilizando gvNIX -un producto impulsado por la Asociación gvSIG- y que destaca por el aumento de la productividad, en lo que se refiere a la disminución de los tiempos de desarrollo. Además, para este tipo de soluciones de inventario, que combinan gestión alfanumérica (realmente la más importante) con parte cartográfica, gvNIX facilita la incorporación de la componente geográfica, integrándola en el sistema como una más. Como base de datos se ha utilizado PostgreSQL + PostGIS.

Respecto a la información geográfica, el geoportal de GeoTIC está dedicado al manejo de recursos TIC de la DGTIC. Este geoportal muestra las sedes a la que presta servicio el Servicio de Atención al Usuario y al Puesto de Trabajo (SAUPT), pudiendo acceder y gestionar la información a través de ellas.

Otra de las características de GeoTIC es que es una aplicación web adaptativa, pudiéndose utilizar en dispositivos con diferentes resoluciones: sobremesas, portátiles, tablets o móviles .

En cuanto a la información, GeoTIC se nutre de diferentes bases de datos (GUC, CESTA, GeoTIC), que utilizan diferentes tecnologías (PostgreSQL y Oracle) y de diferentes servicios web (JIRA, SAFE, OSM, Cartografía oficial):

  • CESTA es una base de datos Oracle, que contiene el inventario de los activos TIC de la GVA.
  • GUC también es una base de datos Oracle, con el inventario de sedes de la GVA.
  • La base de datos de GeoTIC está implementada con postgreSQL + PostGIS donde se almacena la información que estaba en hojas de cálculo. Además, está base de datos es la que le da el soporte espacial a la aplicación.
  • JIRA es una aplicación para la gestión y planificación de proyectos mediante la gestión de incidencias.
  • SAFE es el servicio de autenticación y autorización que ofrece la GVA.

GeoTIC realiza sincronizaciones automáticas por la noche, cuando los usuarios no están trabajando, para tener siempre actualizados los datos que obtiene de GUC y JIRA. También ofrece la posibilidad de realizar sincronizaciones manuales.

La aplicación también necesita acceder a servidores de mapas para obtener la cartografía. Obtiene los mapas de OpenStreetMap y de diversas fuentes de cartografía oficial.

Para más información sobre el proyecto (y ver una demo de su funcionamiento) podéis consultar la presentación que se hizo del mismo en las pasadas Jornadas Internacionales de gvSIG:

Si en vuestra organización tenéis necesidades equivalentes para gestionar vuestro inventario (sea de TIC o de cualquier otro tipo de información), os animamos a que os pongáis en contacto con nosotros: info@gvsig.com . Además de contar con los mejores expertos en geomática libre estaréis contribuyendo al mantenimiento y desarrollo del proyecto gvSIG.


Filed under: geoportal, gvNIX, Inventory, spanish Tagged: gestión de recursos, inventario, TIC
Categories: OSGeo Planet

Ivan Minčík: Automatic PostgreSQL tuning in Ansible

OSGeo Planet - Wed, 2016-08-24 10:27
Once upon a time, there was a tool called pgtune, which I heavily used for setting reasonable defaults for automatically deployed PostgreSQL instances.

Unfortunately, this tool is not maintained for a long time, doesn't support new PostgreSQL versions and finally it was removed from Debian and all it's derivates.

I understand, that proper database configuration is result of deep knowledge of your data and queries, but in current world of automated deployment and containers, it is much more better to configure your PostgreSQL instance with some computed values based on current environment, then leaving default values designed for some very low memory system.

Therefore I have started to search Internet for some pgtune alternative and found great, new web based pgtune replacement developed by Alexey Vasiliev (le0pard).

Since I am using Ansible for my systems orchestration, I have decided to rewrite it's computing logic as a new Ansible postgresql_tune module.

Module requires following information to compute results:
  • PostgreSQL version
  • deployment type
  • total memory
  • maximal number or connections
  • path to resulting PostgreSQL file
  • path to resulting sysctl file

Example usage:

- name: Install PostgreSQL configuration
  postgresql_tune:
    db_version: "{{ POSTGRESQL-VERSION }}"
    db_type: dw
    total_memory: "{{ ansible_memtotal_mb }}MB"
    max_connections: "{{ POSTGRESQL-CONNECTIONS }}"
    postgresql_file: "{{ pgsql_config_dir }}/99-postgresql-tune.conf"
    sysctl_file: "/etc/sysctl.d/99-postgresql-tune.conf"
  notify:
    - service postgresql restart
    - sysctl postgresql reload
  become: yes

Source code is here and any feedback is very much appreciated.
Categories: OSGeo Planet

gvSIG Team: Problemas en la Alianza Rebelde: Star Wars y gvSIG

OSGeo Planet - Tue, 2016-08-23 17:31

Skywalker_gvSIG«Estaban en el lugar equivocado en el momento equivocado. Naturalmente, se convirtieron en héroes.» .-Leia

 

 

No es muy habitual dedicar un post a un bug, pero en este caso es lo suficientemente curioso como para que me anime a ello. Quién sabe si no es una venganza del Imperio por jugar en gvSIG con la imagen ráster de la Estrella de la Muerte… aunque también es un buen ejemplo de lo escondidos que pueden estar algunos bugs.

El caso es que uno de los desarrolladores de gvSIG estaba intentando instalar la RC2 de gvSIG 2.3 en su casa…sin éxito. Algo raro pasaba, pues en la oficina no había tenido ningún problema, con el mismo sistema operativo y mismas características. Ubuntu 16.04 de 64 bits para más señas.

Primero probó con el instalador .run pero cuando se iniciaba la instalación, se cerraba sin motivo aparente. Tras esto probó con el instalador .jar con el mismo resultado.

Por alguna razón el instalador decidía que allí no se podía instalar gvSIG. Siguió dando vueltas al asunto intentando averiguar los motivos…y nada.

La siguiente prueba sería con la versión portable. gvSIG fallaba al intentar arrancar, sin llegar a mostrar el splash y sin generar el fichero “gvSIG.log” que es el que permite averiguar qué podía estar pasando.

Entonces ve en el fichero “gvSIG-installer.log” que se estaba intentando ejecutar un archivo denominado “cygcheck.exe” y que no lo encontraba.

¿cygchec.exe?¿Un ejecutable de Windows en la instalación de Linux? Estaba claro que ahí estaba el problema…¿pero qué estaba pasando? La conclusión no tenía lógica: gvSIG pensaba que se estaba ejecutando en un sistema Windows.

Y, claro, esto sólo le pasaba a él y sólo en el equipo de su casa.

Y ahora viene la explicación: nuestro desarrollador, en el equipo de casa, había puesto el nombre del sistema como Xwing, un tipo de nave de la saga Star Wars utilizado por la Alianza Rebelde. Y lo que nadie podía imaginar estaba pasando…si tú sistema Linux se llama Xwing no puedes instalar ni ejecutar gvSIG.

Cuando gvSIG le pide al sistema sus características para averiguar la arquitectura y sistema operativo ejecuta el comando:

# uname -a

Linux XWing 3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

…como resultado ve que contiene las letras “win” y decide que es un sistema operativo Windows…y claro, a partir de ahí, es normal que nada funcionara como tocaba.

Aunque haremos las pruebas necesarias, la corrección a priori es sencilla, haciendo que gvSIG utilice “uname -p -o” en lugar de “uname -a”, lo que informa sólo de la arquitectura del procesador y del sistema operativo (y no del nombre que haya decidido ponerle el usuario a su sistema). Con esto parece que la Alianza Rebelde podrá seguir utilizando gvSIG en su lucha contra el Imperio.

¿Merece o no ser nombrado el bug del año?

¡Y que la fuerza os acompañe!

Categories: OSGeo Planet

QGIS Blog: Results of the QGIS user survey 2015

OSGeo Planet - Tue, 2016-08-23 15:36

In autumn last year, we ran a rather large-scale user survey, which was translated into many languages and advertised here on this blog. The final reports can be found here:

(Let me know if you have links to other language versions which were not sent to the mailing list.)

Looking at the English report, most responses were filed by regular (49.7%) and advanced users (35.9%) who use QGIS at least several times per week. One interesting result is that responders feel that the project should still prioritise new features when spending funds:

Top 3 “highest priority for spending QGIS funds”

  1. Important features that are missing (50%)
  2. More bugfixing (24.1%)
  3. Improved user documentation (12.4%)

This is also confirmed by the free comments section were roughly 23% of responders were asking for new features, 19% called for more stability (fewer releases and new features), and 9% for better documentation.

Documentation improvements were followed closely by calls for a more structured approach to plugins (making it easier to find the right tool for the job), stricter plugin documentation requirements, consolidation of plugins with similar functionality, and integration of key plugins into core.

When interpreting these results, it’s important to keep in mind that responses are skewed towards experienced users, who are more likely to require specialist functionality. Beginners on the other hand might rank stability, ease of use of core functionality, and good documentation higher.


Categories: OSGeo Planet

Petr Pridal: PDFium GeoPDF driver in GDAL 2.1

OSGeo Planet - Tue, 2016-08-23 08:14


GDAL is a very popular open-source library for decoding/encoding of geospatial data formats and raster processing. It is used by QGIS, Grass, MapServer, but also by Google Earth or ESRI ArcGIS and many other GIS software tools. It powers our MapTiler software too.

A new implementation of efficient reading of PDF and GeoPDF file formats is available in GDAL library since version 2.1. Klokan Technologies GmbH team is behind the implementation of this format driver, which has been accepted and merged by Even Rouault and released publicly in May 2016.

The driver is powered by the open-source PDFium library, which has been released by Google Inc. for easier previewing of the .pdf files used in the open-source Chrome/Chromium browser. The free PDFium library is BSD licensed light version of the PDF Foxit framework. We have prepared compiling scripts for Linux, Mac OS X and Windows, which were adopted, included and documented as part of the GDAL.

The advantages of the new PDF driver:
  • Significantly higher performance (compared to the previous PoDoFo and Poppler engines)
  • Support for larger PDF files with smaller memory footprint - even large AutoCAD plans or huge GeoPDFs can be processed efficiently now.
  • A non-restrictive BSD license! The copy-left GPL prevented the existence of applications supporting both PDF and MrSID/ECW formats for example.


The driver is in production use in our MapTiler software for over a year now.

We are pleased to share the source code with the community and offer it for integration in all open-source projects and third-party products. Next time, when you launch your favorite open-source GIS tool after the GDAL library has been updated, you may benefit from the faster PDF reading thanks to our work!
Categories: OSGeo Planet

Akshat Tandon: The How part (6) - Concatenating OSM way chunks

OSGeo Planet - Mon, 2016-08-22 23:30

( This post is related to my GSoC project with Marble. For more information you can read the introductory post )

Problem

As of now, If you load an openstreetmap file containing linestrings such as highways, lanes or streets, in Marble and zoom to level 11, you will find that the highways or streets are not contiguous and appear as broken.

You will find the roads/streets as broken

roads_broken

Instead of getting contiguous roads

road_joined

Cause

One of the primary reasons for this discontiguity is that often a particular highway/lane/street is described using multiple contiguous OSM way elements instead of a single OSM way element. Marble treats each of these specific way element as a linestring object. However Marble omits rendering any objects which are smaller than 2px, in order to improve rendering performance. Due to this, many of the OSM way elements, which are smaller than 2px don’t get rendered. Hence the street appears broken since only those of its OSM way elements are rendered which are larger than 2px.

One of the reasons which I can think of and which justifies this highway description using multiple elements is that a street might be called by a certain name for a specific stretch and might be called by some other different name for the remaining stretch. However, at level 11 we are mostly concerned with the type of highways (primary, secondary, motorway , residential) rather than the specifics such as names.

Usually, the multiple OSM way elements of a single highway share the first and the last node ID references. For example consider <1n…2n> as an OSM way element where 1n and 2n corresponds to the first and last node IDs of the way. A highway described by an ordered list of node IDs 1 to 56 can then usually be represented by the following OSM way elements <1…5>, <5…13>, <13…28>, <28…36>, <36…56>

I exploited this way of representation to create a way concatenating module in the existing osm-module tool. For the above example, the module would concat all the 5 OSM way elements into a single OSM way element <1…56>

osm-simplify -t highway=* -w input.osm

The above command concatenates all the highways present in the input file and produces a new osm file as output.

Apart from solving the problem of discontinuity, way concatenation also results in data reduction since it is eliminating redundant node references and way elements. This data reduction in turn results in faster file parsing as well as improved rendering performance since now to render a single stretch of highway one only needs to create and render a single linestring object as opposed to multiple linestring objects.

Algorithm

The tricky part of coding this OSM way concatenator is actually coming up with an algorithm which concatenates all the linestrings present in an input file. Finally I and my mentors Dennis Nienhüser and Torsten Rahn were able to come up with a working algorithm for concatenating osm ways of a file in reasonable time (approximately O(n) time).

The algorithm involves a made up data structure called WayChunk which basically is a list of contiguous ways. It also stores a GeoDataVisualCategory which represents the type of linestring. For example in case of highways GeoDataVisualCategory will contain the kind of highway, whether it is a motorway, primary, secondary or a residential type of highway.

The algorithm utilizes a multi-hash-map to bundle together OSM way elements which share a common starting or terminating node. This multi hash map has nodeID’s as keys and WayChunk’s as the values. The idea is that at any given instant, this map will contain the starting and ending point of the ways which have been evaluated till now, as keys and the corresponding way chunk as the value pointed to by these keys. Now whenever we encounter a new way, and if it’s starting or ending node matches with any of the existing way chunks as well as the type i.e. GeoDataVisualCategory of the way matches with the type of the way chunk, then this way is added to the way chunk and the values of the map are adjusted so that the map’s keys are the starting or ending nodes of some way chunk and not the intermediary ones. This way, eventually, we are able to concat the multiple small OSM chunks of highways, railroads into singular way elements.

The reason we are using multi-hash-maps instead of regular hash maps is that at a particular node, two or more highways(linestrings) of different types may emanate or terminate. Hence a given node may be associated with two or more way chunks having different type(GeoDataVisualCategory).

The algorithm in more detail is described below:

Concat(input_osm_file): Iterate over all of the way elements having key=value tags specified during input Check if the first or the last node ID of the way is present in the multi-hash-map. If neither of the IDs are present CreateWayChunk(way) If only the first ID is present in the map Check if any chunk exsits in the map which has the key as that of first ID and GeoDataVisualCategory as that of the way. If such a chunk exists Append the way to this chunk accordingly(reverse the way if required) Delete the first ID from the map Insert a new entry into the multi map having the key as last node ID of the way and value as that of the found chunk If not CreateWayChunk(way) If only the last ID is present in the map Check if any chunk exsits in the map which has the key as that of last ID and GeoDataVisualCategory as that of the way. If such a chunk exists Append the way to this chunk accordingly(reverse the way if required) Delete the last ID from the map Insert a new entry into the multi map having the key as first node ID of the way and value as that of the found chunk If not CreateWayChunk(way) If both the IDs are present in the map Check if any chunk exsits in the map which has the key as that of first ID and GeoDataVisualCategory as that of the way. Check if any chunk exsits in the map which has the key as that of last ID and GeoDataVisualCategory as that of the way. If both the chunks exist Append the way and the second chunk to the first chunk and modify the map accordingly If only first chunk exists Append the way to the first chunk and modify the map accordingly If only last chunk exists Append the way to this last chunk and modify the map accordingly If none of the chunks exist CreateWayChunk(way) Finally iterate over all the WayChunks and merge the ways in each of their lists to form one single osm way for each chunk CreateWayChunk(way): Create a new WayChunk Append the way to this new way chunk Set the GeoDataVisualCategory of the chunk as that of the way Insert two new entries in the multi map having the starting and ending node IDs of the way as keys and the created WayChunk as the value. Results

The first image having discontiguous highways represents the raw input OSM file. This file has 2704 ways in total and has a size of 4.7 MB

The way-concatenator reduced the number of ways to 812 and the size to 2.9 MB. The second image having contiguous roads represent the output of the way concatenator.

If we remove the redundant nodes from the above output using the node reducing module described in the previous post, we get a resulting file having a size of 2.5MB. This node reducer removes 15146 redundant nodes (keeping the resolution at level 11).

If you suspect that due to node reduction there will loss in quality of rendering, then look at the below rendered image and compare it with the above ones.

node_road_joined

The node reducer and the way concatenator have resulted in a size reduction of approx 46% for the above sample without any significant loss in rendering quality.

Categories: OSGeo Planet

GeoSpatial Camptocamp: QGIS, contribuez sans coder !

OSGeo Planet - Mon, 2016-08-22 11:36

Participer au projet QGIS ne nécessite pas forcément un investissement financier : un investissement humain peut également apporter autant à QGIS qu'à vous-même !

Cet article QGIS, contribuez sans coder ! est apparu en premier sur Camptocamp.

Categories: OSGeo Planet

gvSIG Team: Libro gratuito: Epidemiología panorámica. Introducción al uso de herramientas geoespaciales aplicadas a la salud pública

OSGeo Planet - Mon, 2016-08-22 08:29

epidemiologia_SIG_gvSIG_geomaticaEl portal NOSOLOSIG, que merece estar entre los favoritos de cualquier interesado en la geomática, se ha hecho eco de una publicación que desde aquí recomendamos.

Se trata del libro “Epidemiología panorámica. Introducción al uso de herramientas geoespaciales aplicadas a la salud pública”, que tal y como se indica, es una obra que surge en el marco y como resultado de más de 10 años de cooperación entre el Ministerio de Salud de la Nación y la Comisión Nacional de Actividades Espaciales CONAE (Argentina), para generar herramientas operativas de utilidad en el campo de la salud basadas en tecnología geoespacial.

El libro incluye ejercicios prácticos con gvSIG, pudiendo descargar gratuitamente tanto el propio libro como los datos utilizados en los ejercicios.

Tenéis toda la información en:

http://www.nosolosig.com/bibliografia/item/epidemilogia-panoramica-introduccion-al-uso-de-herramientas-geoespaciales-aplicadas-a-la-salud-publica


Filed under: gvSIG Desktop, spanish, training Tagged: epidemeología, salud
Categories: OSGeo Planet

Bjorn Sandvik: The history of the Telemark Canal - projected on a physical landscape model

OSGeo Planet - Mon, 2016-08-22 07:17
Together with Jon Olav Eikenes and Christan Løverås, I'm part of a new startup called Norviz. The main focus so far has been on projecting animated graphics onto physical landscape models. Our first job was to tell the history of the Telemark Canal, a beautiful waterway connecting the sea and the interior through eight locks at a distance of 105 km from Skien to Dalen in Norway.


The installation was made for West Telemark museum, and is now on show in Vrangfoss, the largest lock complex on the canal with five locks and a lift of 23 metres. The 3D model displays a 10 minutes map animation showing the history of the canal together with historical images, voice and sound effects.

Here are some of the technical details which might interest my readers :-)

The digital elevation model was prepared in Blender and cutted with a a CNC router. It took the machine about 30 hours to finish the whole model.

Cutting a large 240x110 cm model of the Telemark Canal in Valchromat. Time-lapse of the cutting process:


After two coats of white paint, our model became a nice canvas for our map animation. Fun to see the geological structures from above.

Setup and calibration in an old barn at Vrangfoss. The video projector was mounted 4 meters above the model.   The various maps telling the story of the canal was rendered with Mapnik before adding transitions and special effects in Adobe After Effects. With the help of three.js and some homemade scripts, we were able to align our map animation with the uneven landscape surface. Lastly we used JavaScript for Automation (JSX) to link and synchronise the the different parts of the installation.

The final installation showing graphics projected on the physical landscape model. From Varden newspaper 28 June 2016.
See it live at Vrangfoss during summer season while the canal boats are operating!

Are you interested in collaborating with us or help us fill the world with engaging visualizations? Please don’t hesitate contacting us!


Map data from Kartverket.
Concept and story by Indici and West Telemark museum.
Photos above taken by Jon Olav Eikenes.
Categories: OSGeo Planet

gvSIG Team: 3as Jornadas gvSIG México: Guía de talleres

OSGeo Planet - Mon, 2016-08-22 06:57

gvSIG_México_talleres

Ya podéis consultar la guía de talleres y geotemas de las 3as Jornadas gvSIG México, y que son parte importante de las actividades que se celebrarán del 7 al 9 de septiembre durante estas jornadas. Las Jornadas tendrán lugar en Instituto de Geografía de la Universidad Nacional Autónoma de México.

Todos los talleres son gratuitos. Para poder asistir a ellos es necesario inscribirse en las jornadas. A tener en cuenta que tienen cupo limitado, por lo que el primer día de jornadas, en la mesa de registro, los asistentes se podrán inscribir a los talleres que les interesen. Para inscribirse en las jornadas podéis hacerlo a través del siguiente enlace:

http://www.gvsig.com/es/eventos/jornadas-mexico/2016/inscripcion

Estos talleres pretenden ser mini-cursos que permitan a los usuarios formarse en distintos aspectos de la geomática libre. Veamos un breve resumen de lo que ofrecen estas jornadas en cuanto a talleres:

  • Talleres generales de gvSIG Desktop para usuarios:
    • Introducción al uso de SIG libre con gvSIG Desktop. El objetivo es que los asistentes aprendan el manejo de gvSIG desde cero. De forma sencilla y mediante ejercicios prácticos se comprenderán las principales funcionalidades de gvSIG. Al terminar el taller los asistentes podrán continuar sin dificultad con la utilización y aprendizaje del programa.
    • Novedades de gvSIG. Permitirá conocer y aprender a manejar las nuevas funcionalidades y características que presenta la nueva versión de gvSIG Desktop 2.3 (LiDAR, Segmentación dinámica, 3D,…) mediante ejercicios prácticos.
  • Talleres temáticos de gvSIG Desktop para usuarios:
    • Manejo de estrategías electorales con gvSIG. Tiene como objetivo proporcionar los elementos básicos de manejo de gvSIG para el análisis geoelectoral, y la importancia de su aplicación en la toma de decisiones en el ámbito electoral, con participación activa y práctica.
    • Aplicación de los SIG al urbanismo con gvSIG. Busca despertar el interés por el uso de los SIG a los urbanistas, mediante una introducción a gvSIG mediante una serie de ejercicios prácticos relacionados con el urbanismo.
    • Arqueología en gvSIG. Los asistentes aprenderán a integrar el uso de gvSIG al trabajo arqueológico diario, por lo que tendrá un planteamiento práctico y a base de ejemplos reales.
  • Talleres de gvSIG Desktop para desarrolladores e interesados en iniciarse en el desarrollo:
    • Introducción al scripting en Python con gvSIG. El objetivo es conocer las posibilidades del módulo de scripting en gvSIG y el proceso de manipulación de datos espaciales para la creación de capas a partir de otras existentes. Para este curso no son necesarios conocimientos en programación.
    • Desarrollo avanzado en gvSIG con scripting. Permitirá conocer de forma más profunda el entorno de programación en gvSIG con Python. Para este curso es recomendable tener conocimientos en Python.
  • Taller de gvSIG Online:
    • gvSIG Online, Infraestructuras de Datos Espaciales en software libre. Permitirá aprender a manejar gvSIG Online, una plataforma integral para la implantación y gestión de IDE. Los alumnos aprenderán a crear geoportales y administrar la información geográfica de una Infraestructuras de Datos Espaciales.
  • Talleres de geomática libre:
    • Alternativas de geocodificación en software libre. Por medio de este taller el asistente conocerá las diferentes formas de geocodificación que existen en software libre, tanto on-line como desktop.
    • Software para análisis fractal: FROG. El objetivo es que el asistente conozca el software FROG para el cálculo de la dimensión fractal de los objetos utilizando diversos tratamientos, a partir de imágenes con formatos raw o bmp.
    • Space Synthax con Depth Map. Tiene como objetivo examinar desde una perspectiva teórica y luego práctica el significado de las variables de gramática espacial y como utilizarlas en la planeación de espacios urbanos. Conocer el concepto de Space Syntax y su utilidad para conocer el grado de integración y conectividad de cada segmento de la ciudad, conocimiento útil para estudios urbanos y de transporte en general.
    • Mapeo libre: mapeando territorios en situación de riesgo. El territorio mexicano cuenta con distintas características que lo hace vulnerable a diversos tipos de desastres naturales como temblores, huracanes, lluvias torrenciales, deslaves, entre otros. El objetivo del taller es organizar a través de OpenStreetMaps el mapeo de zonas que pueden sufrir las consecuencias de alguno de estos fenómenos naturales.

La guía completa de talleres y geotemas está disponible en:

http://www.gits.igg.unam.mx/descargas/guiaTalleres.pdf

Podéis revisar el programa completo de las 3as Jornadas gvSIG México en:

http://www.gvsig.com/es/eventos/jornadas-mexico/2016/programa


Filed under: events, gvSIG Desktop, gvSIG Online, spanish, training Tagged: México
Categories: OSGeo Planet
Syndicate content