OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 8.1 ‘Creación de capas de puntos a partir de tablas (Geocodificación: Puntos a partir de tabla con direcciones)’

OSGeo Planet - Thu, 2017-11-02 14:52

Ya está disponible el primer vídeo del octavo módulo, que trata sobre cómo crear capas de puntos a partir de tablas. En este primer vídeo veremos la parte de geocodificación, creando una capa de puntos a partir de una tabla con direcciones.

La tabla, aparte de direcciones también podría contener elementos característicos como museos, monumentos, instalaciones deportivas…, es decir, todo lugar que podríamos encontrar en buscadores como Google Maps, OpenStreetMap…, ya que emplea dichos motores de búsqueda para crear dicha capa.

Esta funcionalidad es muy útil en un ayuntamiento, ya que si disponemos por ejemplo de varias tablas con dichos elementos o sus direcciones, no tendríamos que ir digitalizando uno a uno. Gracias a este geoproceso lo podremos realizar de forma automática, y solo tendríamos que abortar una fase de control de calidad al final para ver que los resultados son correctos (ya que como sabemos dichos motores de búsqueda también tienen ciertos errores). Aparte, podemos indicarle en el geoproceso que nos muestre qué elementos o direcciones no han sido encontrados, para así poder digitalizarlos de otra forma.

También mostraremos cómo acceder a Google Street View desde gvSIG, muy interesante para el trabajo de oficina, para ciertas consultas que nos evitarían el tener que acercarnos al lugar en persona.

La cartografía a utilizar en este vídeo la podéis descargar desde el siguiente enlace.

El primer vídeo-tutorial de este octavo módulo es el siguiente:

Post relacionados:


Filed under: gvSIG Desktop, spanish, training Tagged: ayuntamientos, direcciones, geocodificación, gestión municipal, Google Maps, OpenStreetMap, OSM
Categories: OSGeo Planet

gvSIG Team: State of art, What is the difference between Geopaparazzi and gvSIG Mobile, tools for gvSIG Desktop and much more

OSGeo Planet - Thu, 2017-11-02 08:40

During the 13th International gvSIG Conference Andrea Antonello of HydroloGIS spoke us about Geopaparazzi and gvSIG Mobile.

If you need an engineering survey tool, an open source Mobile GIS don’t miss this video:


Filed under: english, Geopaparazzi, gvSIG Mobile
Categories: OSGeo Planet

Equipo Geotux: Generación de metadatos con QSphere en QGIS

OSGeo Planet - Wed, 2017-11-01 15:13

QSphere es un complemento de QGIS que de acuerdo a la documentación oficial "es una extensión que se llevó a cabo en el marco de jornadas de información en la atención a ADL (Administradores de Datos) de la Secretaría de Desarrollo Sustentable del Estado francés para la implementación de catálogos, que permite la documentación de metadatos". Desarrollo por Christophe MASSE, del Centro de servicios informáticos e ingeniería del Ministerio de Ambiente de Francia.

Básicamente QSphere permite generar la documentación de metadatos conforme a la norma ISO 19139 de recursos geográficos en QGIS Desktop, así como la exploración de metadatos a través de los servicios de catálogos.

QSphere presenta una interfaz gráfica amigable para generar la documentación de metadatos de acuerdo a la iniciativa INSPIRE, usando ayudantes, formularios, editor y salidas en XML y presentación en HTML para la salida en un Servicio Web de Catálogos.

Para mayor información consulte el sitio Web del proyecto: https://qgis.projets.developpement-durable.gouv.fr/projects/qsphere

 

 

Nota: Este documento hace parte del material de guías de talleres del curso de Servicios Web Geográficos de la Maestría en Geomática de la Universidad Nacional de Colombia Sede Bogotá, mayor información en http://www.aulageo.cloud/course/unal-ogc-2017/ INSTALACIÓN DE QSPHERE

Para la instalación del complemento se requiere agregar el repositorio de QGIS del Ministerio de Ambiente de Francia:

http://piece-jointe-carto.developpement-durable.gouv.fr/NAT002/QGIS/plugins/plugins.xml

Repositorio QSpehre

Luego de sincronizar el repositorio de complementos, se puede realizar la instalación del módulo de QSphere.

La versión instalada a la fecha, corresponde a la versión 4.3.8 en QGIS 2.18.x. Las versiones más actuales del complemento está orientada a las versión de desarrollo de QGIS 3.

Nota 1:…


Read more...
Categories: OSGeo Planet

Paul Ramsey: Parallel PostGIS II

OSGeo Planet - Tue, 2017-10-31 20:00

A year and a half ago, with the release of PostgreSQL 9.6 on the horizon, I evaluated the parallel query infrastructure and how well PostGIS worked with it.

The results at the time were mixed: parallel query worked, when poked just the right way, with the correct parameters set on the PostGIS functions, and on the PostgreSQL back-end. However, under default settings, parallel queries did not materialize. Not for scans, not for joins, not for aggregates.

With the recent release of PostgreSQL 10, another generation of improvement has been added to parallel query processing, so it’s fair to ask, “how well does PostGIS parallelize now?”

Parallel PostGIS II

TL;DR:

The answer is, better than before:

  • Parallel aggregations now work out-of-the-box and parallelize in reasonable real-world conditions.
  • Parallel scans still require higher function costs to come into action, even in reasonable cases.
  • Parallel joins on spatial conditions still seem to have poor planning, requiring a good deal of manual poking to get parallel plans.
Setup

In order to run these tests yourself, you will need:

  • PostgreSQL 10
  • PostGIS 2.4

You’ll also need a multi-core computer to see actual performance changes. I used a 4-core desktop for my tests, so I could expect 4x improvements at best.

For testing, I used the same ~70K Canadian polling division polygons as last time.

createdb parallel psql -c 'create extension postgis' parallel shp2pgsql -s 3347 -I -D -W latin1 PD_A.shp pd | psql parallel

PDs

To support join queries, and on larger tables, I built a set of point tables based on the polling divisions. One point per polygon:

CREATE TABLE pts AS SELECT ST_PointOnSurface(geom)::Geometry(point, 3347) AS geom, gid, fed_num FROM pd; CREATE INDEX pts_gix ON pts USING GIST (geom);

Ten points per polygon (for about 700K points):

CREATE TABLE pts_10 AS SELECT (ST_Dump(ST_GeneratePoints(geom, 10))).geom::Geometry(point, 3347) AS geom, gid, fed_num FROM pd; CREATE INDEX pts_10_gix ON pts_10 USING GIST (geom);

One hundred points per polygon (for about 7M points):

CREATE TABLE pts_100 AS SELECT (ST_Dump(ST_GeneratePoints(geom, 100))).geom::Geometry(point, 3347) AS geom, gid, fed_num FROM pd; CREATE INDEX pts_100_gix ON pts_100 USING GIST (geom);

The configuration parameters for parallel query have changed since the last test, and are (in my opinion) a lot easier to understand.

These parameters are used to fine-tune the planner and execution. Usually you don’t need to change them.

  • parallel_setup_cost sets the planner’s estimate of the cost of launching parallel worker processes. Default 1000.
  • parallel_tuple_cost sets the planner’s estimate of the cost of transferring one tuple from a parallel worker process to another process. Default 0.1.
  • min_parallel_table_scan_size sets the minimum amount of table data that must be scanned in order for a parallel scan to be considered. Default 8MB.
  • min_parallel_index_scan_size sets the minimum amount of index data that must be scanned in order for a parallel scan to be considered. Default 512kB.
  • force_parallel_mode forces the planner to parallelize is wanted. Values: off | on | regress
  • effective_io_concurrency for some platforms and hardware setups allows true concurrent read. Values from 1 (for one spinning disk) to ~100 (for an SSD drive). Default 1.

These parameters control how many parallel processes are launched for a query.

  • max_worker_processes sets the maximum number of background processes that the system can support. Default 8.
  • max_parallel_workers sets the maximum number of workers that the system can support for parallel queries. Default 8.
  • max_parallel_workers_per_gather sets the maximum number of workers that can be started by a single Gather or Gather Merge node. Setting this value to 0 disables parallel query execution. Default 2.

Once you get to the point where #processes == #cores there’s not a lot of advantage in adding more processes. However, each process does exact a cost in terms of memory: a worker process consumes work_mem the same as any other backend, so when planning memory usage take both max_connections and max_worker_processes into consideration.

Before running tests, make sure you have a handle on what your parameters are set to: I frequently found I accidentally tested with max_parallel_workers set to 1.

show max_worker_processes; show max_parallel_workers; show max_parallel_workers_per_gather; Aggregates

First, set max_parallel_workers and max_parallel_workers_per_gather to 8, so that the planner has as much room as it wants to parallelize the workload.

PostGIS only has one true spatial aggregate, the ST_MemUnion function, which is comically inefficient due to lack of input ordering. However, it’s possible to see some aggregate parallelism in action by wrapping a spatial function in a parallelizable aggregate, like Sum():

SET max_parallel_workers = 8; SET max_parallel_workers_per_gather = 8; EXPLAIN ANALYZE SELECT Sum(ST_Area(geom)) FROM pd

Boom! We get a 3-worker parallel plan and execution about 3x faster than the sequential plan.

Finalize Aggregate (cost=15417.45..15417.46 rows=1 width=8) (actual time=236.925..236.925 rows=1 loops=1) -> Gather (cost=15417.13..15417.44 rows=3 width=8) (actual time=236.915..236.921 rows=4 loops=1) Workers Planned: 3 Workers Launched: 3 -> Partial Aggregate (cost=14417.13..14417.14 rows=1 width=8) (actual time=231.724..231.724 rows=1 loops=4) -> Parallel Seq Scan on pd (cost=0.00..13800.30 rows=22430 width=2308) (actual time=0.049..30.407 rows=17384 loops=4) Planning time: 0.111 ms Execution time: 238.785 ms

Just to confirm, re-run it with parallelism turned off:

SET max_parallel_workers_per_gather = 0; EXPLAIN ANALYZE SELECT Sum(ST_Area(geom)) FROM pd

Back to one thread and taking about 3 times as long, as expected.

Scans

The simplest spatial parallel scan adds a spatial function to the filter clause.

SET max_parallel_workers = 8; SET max_parallel_workers_per_gather = 8; EXPLAIN ANALYZE SELECT * FROM pd WHERE ST_Area(geom) > 10000;

Unfortunately, that does not give us a parallel plan.

The ST_Area() function is defined with a COST of 10. If we move it up, to 100, we can get a parallel plan.

SET max_parallel_workers_per_gather = 8; ALTER FUNCTION ST_Area(geometry) COST 100; EXPLAIN ANALYZE SELECT * FROM pd WHERE ST_Area(geom) > 10000;

Boom! Parallel scan with three workers:

Gather (cost=1000.00..20544.33 rows=23178 width=2554) (actual time=0.253..293.016 rows=62158 loops=1) Workers Planned: 5 Workers Launched: 5 -> Parallel Seq Scan on pd (cost=0.00..17226.53 rows=4636 width=2554) (actual time=0.091..210.581 rows=10360 loops=6) Filter: (st_area(geom) > '10000'::double precision) Rows Removed by Filter: 1229 Planning time: 0.128 ms Execution time: 302.600 ms

It appears our spatial function costs may still be too low in general to get good planning. And as we will see with joins, it’s possible the planner is still discounting function costs too much in deciding whether to go parallel or not.

Joins

Starting with a simple join of all the polygons to the 100 points-per-polygon table, we get:

SET max_parallel_workers_per_gather = 4; EXPLAIN SELECT * FROM pd JOIN pts_100 pts ON ST_Intersects(pd.geom, pts.geom);

PDs & Points

In order to give the PostgreSQL planner a fair chance, I started with the largest table, thinking that the planner would recognize that a “70K rows against 7M rows” join could use some parallel love, but no dice:

Nested Loop (cost=0.41..13555950.61 rows=1718613817 width=2594) -> Seq Scan on pd (cost=0.00..14271.34 rows=69534 width=2554) -> Index Scan using pts_gix on pts (cost=0.41..192.43 rows=232 width=40) Index Cond: (pd.geom && geom) Filter: _st_intersects(pd.geom, geom)

There are a number of knobs we can press on. There are two global parameters:

  • parallel_setup_cost defaults to 1000, but no amount of lowering the value, even to zero, causes a parallel plan.
  • parallel_tuple_cost defaults to 0.1. Reducing it by a factor of 100, to 0.001 causes the plan to flip over into a parallel plan.
SET parallel_tuple_cost = 0.001;

As with all parallel plans, it is a nested loop, but that’s fine since all PostGIS joins are nested loops.

Gather (cost=0.28..4315272.73 rows=1718613817 width=2594) Workers Planned: 4 -> Nested Loop (cost=0.28..2596658.92 rows=286435636 width=2594) -> Parallel Seq Scan on pts_100 pts (cost=0.00..69534.00 rows=1158900 width=40) -> Index Scan using pd_geom_idx on pd (cost=0.28..2.16 rows=2 width=2554) Index Cond: (geom && pts.geom) Filter: _st_intersects(geom, pts.geom)

Running the parallel plan to completion on the 700K point table takes 18s with four workers and 53s with a sequential plan. We are not getting an optimal speed up from parallel processing anymore: four workers are completing in 1/3 of the time instead of 1/4.

If we set parallel_setup_cost and parallel_tuple_cost back to their defaults, we can also change the plan by fiddling with the function costs.

First, note that our query can be re-written like this, to expose the components of the spatial join:

SET parallel_tuple_cost=0.1; SET parallel_setup_cost=1000; SET max_parallel_workers_per_gather = 4; EXPLAIN SELECT * FROM pd JOIN pts_100 pts ON pd.geom && pts.geom AND _ST_Intersects(pd.geom, pts.geom);

The default cost of _ST_Intersects() is 100. If we adjust it up by a factor of 100, we can get a parallel plan.

ALTER FUNCTION _ST_Intersects(geometry, geometry) COST 10000;

However, what if our query only used a single spatial operator in the join filter? Can we still force a parallel plan on this query?

SET parallel_tuple_cost=0.1; SET parallel_setup_cost=1000; SET max_parallel_workers_per_gather = 4; EXPLAIN SELECT * FROM pd JOIN pts_100 pts ON pd.geom && pts.geom;

The && operator could activate one of two functions:

  • geometry_overlaps(geom, geom) is bound to the && operator
  • geometry_gist_consistent_2d(internal, geometry, int4) is bound to the 2d spatial index

However, no amount of increasing their COST causes the operator-only query plan to flip into a parallel mode:

ALTER FUNCTION geometry_overlaps(geometry, geometry) COST 1000000000000; ALTER FUNCTION geometry_gist_consistent_2d(internal, geometry, int4) COST 10000000000000;

So for operator-only queries, it seems the only way to force a spatial join is to muck with the parallel_tuple_cost parameter.

More Joins

Can we parallelize a common GIS use case: the spatial overlay?

Shifted PDs

Here is a table that simply shifts the polling divisions up and over, so that they can be overlaid to create a new set of smaller polygons.

CREATE TABLE pd_translate AS SELECT ST_Translate(geom, 100, 100) AS geom, fed_num, pd_num FROM pd; CREATE INDEX pd_translate_gix ON pd_translate USING GIST (geom); CREATE INDEX pd_fed_num_x ON pd (fed_num); CREATE INDEX pdt_fed_num_x ON pd_translate (fed_num);

The overlay operation finds, for each geometry on one side, all the overlapping geometries, and then calculates the shape of those overlaps (the “intersection” of the pair). Calculating intersections is expensive, so it’s something want to happen in parallel, even more than we want the join to happen in parallel.

This query calculates the overlay of all polling divisions (and their translations) in British Columbia (fed_num > 59000):

EXPLAIN SELECT ST_Intersection(pd.geom, pdt.geom) AS geom FROM pd JOIN pd_translate pdt ON ST_Intersects(pd.geom, pdt.geom) WHERE pd.fed_num > 59000 AND pdt.fed_num > 59000;

Unfortunately, the default remains a non-parallel plan. The parallel_tuple_cost has to be adjusted down to 0.01 or the cost of _ST_Intersects() adjusted upwards to get a parallel plan.

Conclusions
  • The costs assigned to PostGIS functions still do not provide the planner a good enough guide to determine when to invoke parallelism. Costs assigned currently vary widely without any coherent reasons.
  • The planner behaviour on spatial joins remains hard to predict: is the deciding factor the join operator cost, the number of rows of resultants, or something else altogether? Counter-intuitively, it was easier to get join behaviour from a relatively small 6K x 6K polygon/polygon overlay join than it was for the 70K x 7M point/polygon overlay.
Categories: OSGeo Planet

GeoSolutions: New release of MapStore with Annotations and Quick Search

OSGeo Planet - Mon, 2017-10-30 15:00

Release 2017.05.00

Dear Reader,

we are pleased to announce release 2017.05.00 of MapStore, our flagship Open Source webgis product. The full list of changes for this release can be found here, but the most interesting additions are the following:

  • General Review of Map Layout, reorgainizing tools in the bottom-right corner and adding the footer.
  • TOC UI Review, so that now the map's table of contents is organized in cards.
  • Catalog UI Review, to make the catalog widget more intuitive. You can add as many sources as you want and save them in the map.
  • Initial Support for annotations, now you can quickly annotate the map with markers having also the ability to customize their style and description.
  • Quick Filter on Feature Grid, now you can search directly from the featuregrid without opening the query panel. You can also synchronize the content of the map with the filtered data.
General Review of Map Layout We have started an in-depth review of the map look&feel; now the scale and coordinate indicators are placed in a footer, together with the attribution. The table of contents has been reorganized, allowing filtering and grouping all the layer tools in the top toolbar; the layers are now organized in cards that can be expanded or collapsed to see the legend. [gallery type="slideshow" size="full" ids="3752,3750"] The catalog widget has also been reviewed. Now you can add OGC CSW, WMS, or WMTS services in a more intuitive way and store them in the map configuration for later reuse.  [caption id="attachment_3748" align="aligncenter" width="1024"]New Catalog UI New Catalog UI[/caption]   [caption id="attachment_3749" align="aligncenter" width="1024"]Add Catalog Services Add Catalog Services[/caption] Initial version of Map Annotations We have implemented a first version of map annotations that can be used to implement markers. You can easily add to the map your own markers, customizing content, icon and color. You can  use this functionalities to mark your favorite places, the places to visit for work or whatever you want. Annotations are saved in the map context, hence you can share them with your friends, coworkers or the users of your geoportal. More feature and look&feel improvements will be performed in the next releases. [caption id="attachment_3765" align="aligncenter" width="1024"]Annotations Annotations[/caption] Quick Search

We have been struggling over the years with complex interactions with visual query builders to filter data exposed via the OGC WFS services. We have therefore decided to try and build a simplified filtering capability to easily filter vector layers directly from the data grid column headers. You can now enter text or insert a formula (like > 10 or <= -23) right in the header and see the data grid filter rows automatically. You can also synchronize the content of the map to match the content of the data grid. 

[caption id="attachment_3769" align="aligncenter" width="1024"]Quick Search and Map Sync Quick Search and Map Sync[/caption] Future Work

We have great plans for the future which translates into many new features in the working. Just to give you a little preview of an interesting one, for the next release we are implementing charting capabilities for vector layers directly inside the map. Here below a preview.

[caption id="attachment_3770" align="aligncenter" width="1024"]Preview of Charts functionality Preview of Charts functionalities[/caption]

In addition, there are other things in the pipeline, in sparse order:

  • Integration with GeoNode
  • Integrate styler for GeoServer
  • Revamp of the Catalog Widget to make it more usable
  • Support for layers with TIME
  • Support general map annotations, beyond simple markers

If you are interested in learning about how we can help you achieving your goals with open source products like GeoServerMapstore, GeoNode and GeoNetwork through our Enterprise Support Services and GeoServer Deployment Warranty offerings, feel free to contact us!

The GeoSolutions team,
Categories: OSGeo Planet

OTB Team: Orfeo ToolBox 6.2 is out!

OSGeo Planet - Mon, 2017-10-30 13:38
We are happy to announce that Orfeo ToolBox 6.2 is out! As usual, ready-to-use binary packages are available for Windows, Linux and Mac OS X: OTB 6.2 You can also checkout the source directly with git: git clone https://git@git.orfeo-toolbox.org/git/otb.git OTB -b release-6.2 We welcome your feedback and requests, and encourage you to join the OTB […]
Categories: OSGeo Planet

From GIS to Remote Sensing: Developing the SCP 6: Download products and Sentinel-3

OSGeo Planet - Mon, 2017-10-30 09:00
I am updating the Semi-Automatic Classification Plugin (SCP) to version 6 (codename Greenbelt) which will be compatible with the upcoming QGIS 3.

In this previous posts, I described the main changes to the SCP dock, the Main interface and the ability to create multiple band sets.
In this post I present the new interface for downloading free products such as Landsat, Sentinel-2, ASTER, MODIS and the new addition Sentinel-3.

Download products
Categories: OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 7.2 ‘Edición (SHP de geometrías derivadas)’

OSGeo Planet - Mon, 2017-10-30 08:07

Ya está disponible el segundo vídeo del séptimo módulo, donde mostraremos una funcionalidad más de la parte de edición, muy importante para el trabajo en un ayuntamiento.

La funcionalidad que veremos en este vídeo es la de crear ficheros .SHP de geometrías derivadas. Esto nos permitirá crear una capa de polígonos a partir de una de puntos o de líneas, y también una capa de líneas a partir de una de puntos. Nos será muy útil por ejemplo cuando tenemos los puntos que forman el eje de una calle de nuestro municipio, donde tenemos un campo con el orden que llevarían (si no lo tenemos debemos comprobar en la Vista qué puntos son cuando los seleccionamos para ver en qué orden están), así no tendríamos que ir digitalizando punto a punto. Lo mismo ocurriría cuando tenemos los puntos que forman una parcela. Si tenemos por ejemplo una parcela formada por 500 puntos, utilizando esta herramienta no tendríamos que ir digitalizando uno a uno esos puntos para formar el polígono, ya que se crearía de forma automática.

La cartografía a utilizar en este vídeo la podéis descargar desde el siguiente enlace.

El segundo vídeo-tutorial de este séptimo módulo es el siguiente:

Post relacionados:


Filed under: gvSIG Desktop, spanish, training Tagged: ayuntamientos, edición, geometrías derivadas, gestión municipal
Categories: OSGeo Planet

QGIS Blog: QGIS Server refactoring is done!

OSGeo Planet - Sun, 2017-10-29 22:31

As you may know, QGIS is jumping to a new major version. (Yes!) Doing so was made necessary because of the need to switch to Python 3, Qt5, but also because we needed to break the QGIS API in several places.

A year ago there was an appeal on the QGIS developer mailing list about the strong need for love that the QGIS server code base required. Indeed, the API was locked by some old methods of QGIS server. In short, QGIS server was reparsing the .qgs project file in its own way, and created dependencies to parts of QGIS we needed to drop.

As outsourcing the server code base was not an option, so we had to refactor it. The involved parties decided to get engaged in a code sprint in the city of Lyon , France dedicated to sharing their vision, planning the work and finally making all the following happen:

Higher level refactoring

All services (WMS GetMap, WFS GetFeature, GetLegendGraphics, WCS, GetPrint etc..) have been rewritten. Some like WMS were entirely rewritten. Kudos to the devs!

New features

Deep, complex and unrewarding tasks

  • Remove all singleton calls
  • Cut all the dependencies to the old QGIS project file parser
  • Minimize dependencies to GUI library. Since fonts are necessary to render maps, totally removing them was not feasible.

Infrastructure tasks

Additionally, some of these new developments have already been presented at FOSS4G-EU in July.

Congratulations to the developers who worked hard on this!

Now this deserves to be well tested, please report back any issues!


Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Movement data in GIS #10: open tools for AIS tracks from MarineCadastre.gov

OSGeo Planet - Sat, 2017-10-28 13:22

MarineCadastre.gov is a great source for AIS data along the US coast. Their data formats and tools though are less open. Luckily, GDAL – and therefore QGIS – can read ESRI File Geodatabases (.gdb).

They also offer a Track Builder script that creates lines out of the broadcast points. (It can also join additional information from the vessel and voyage layers.) We could reproduce the line creation step using tools such as Processing’s Point to path. But this post will show how to create PostGIS trajectories instead.

First, we have to import the points into PostGIS using either DB Manager or Processing’s Import into PostGIS tool:

Then we can create the trajectories. I’ve opted to create a materialized view:

The first part of the query creates a temporary table called ptm (short for PointM). This step adds time stamp information to each point. The second part of the query then aggregates these PointMs into trajectories of type LineStringM.

CREATE MATERIALIZED VIEW ais.trajectory AS WITH ptm AS ( SELECT b.mmsi, st_makepointm( st_x(b.geom), st_y(b.geom), date_part('epoch', b.basedatetime) ) AS pt, b.basedatetime t FROM ais.broadcast b ORDER BY mmsi, basedatetime ) SELECT row_number() OVER () AS id, st_makeline(ptm.pt) AS st_makeline, ptm.mmsi, min(ptm.t) AS min_t, max(ptm.t) AS max_t FROM ptm GROUP BY ptm.mmsi WITH DATA;

The trajectory start and end times (min_t and max_t) are optional but they can help speed up future queries.

One of the advantages of creating trajectory lines is that they render many times faster than the original points.

Of course, we end up with some artifacts at the border of the dataset extent. (Files are split by UTM zone.) Trajectories connect the last known position before the vessel left the observed area with the position of reentry. This results, for example, in vertical lines which you can see in the bottom left corner of the above screenshot.

With the trajectories ready, we can go ahead and start exploring the dataset. For example, we can visualize trajectory speed and/or create animations:

Purple trajectory segments are slow while green segments are faster

We can also perform trajectory analysis, such as trajectory generalization:

This is a first proof of concept. It would be great to have a script that automatically fetches the datasets for a specified time frame and list of UTM zones and loads them into PostGIS for further processing. In addition, it would be great to also make use of the information in the vessel and voyage tables, thus splitting up trajectories into individual voyages.

Read more:


Categories: OSGeo Planet

From GIS to Remote Sensing: SCP Questions of This Month: October

OSGeo Planet - Fri, 2017-10-27 08:00
This post is a collection of questions and answers about the Semi-Automatic Classification Plugin (SCP) and remote sensing which were discussed in the Facebook group and the Google+ Community this month.
These questions vary from supervised classification technique to software issues, and can be useful to the readers of this blog for solving issues about the use of SCP.

Categories: OSGeo Planet

GIS for Thought: #Ireland 2023

OSGeo Planet - Thu, 2017-10-26 09:53

Ireland is bidding for the 2023 Rugby World Cup.

They have submitted 12 stadiums in their bid. They cover all four provinces and the breadth of the island.

Ranging from Europe’s third biggest stadium Croke Park in Dublin. Welcoming 1.5 million fans every year. Packing in 82,300 dedicated fans every year for the GAA (Gaelic Athletic Association) Hurling and Gaelic Football finals. An integral part of Ireland’s history through sweat, blood, and identity.

The Aviva Stadium in Dublin, the worlds oldest international rugby stadium. Venue for the 2011 Europa League Final between Portuguese sides Porto and Braga.

Ravenhill Stadium in Belfast. Home of Ulster Rugby and in 1991 venue for Japan’s first match victory in a Rugby World Cup.

Thomond Park in Limerick. Heart of the community and host to a 12 year unbeaten run for Munster rugby. Winner of the ‘Best Rugby Stadium in the World’ vote in 2013.

Ireland 2023 Stadiums

Categories: OSGeo Planet

gvSIG Team: Material de los Talleres de Introducción a Scripting con gvSIG realizados en Valencia y Culiacan

OSGeo Planet - Thu, 2017-10-26 09:45

Durante las 13as Jornadas Internacionales gvSIG en Valencia y las 4as Jornadas gvSIG de México, se dieron varios talleres de Introducción a Scripting con gvSIG.

Puedes descargar el taller completo desde el siguiente enlace: Taller Introducción de Scripting.

El taller está realizado sobre la versión 2.3.1 de gvSIG y está todo explicado en diapositivas paso a paso, incluidas dentro del paquete y de manera online. Para instalar el paquete solo hay que ir al Administrador de Complementos -> Instalación desde fichero.

Esperamos que os sirvan de utilidad. Cualquier duda sobre el taller o el módulo de Scripting podéis hacerlas en las Listas de Correo.


Filed under: development, gvSIG Desktop, gvSIG development, scripting, spanish
Categories: OSGeo Planet

gvSIG Team: SIG aplicado a Gestión Municipal: Módulo 7.1 ‘Edición (Creación de nuevas capas, edición gráfica, edición alfanumérica)’

OSGeo Planet - Thu, 2017-10-26 07:22

Ya está disponible el primer vídeo del séptimo módulo, en el que comenzaremos con la parte de edición.

La edición es una parte muy importante dentro de los Sistemas de Información Geográfica, ya que nos permite por ejemplo crear nuevas capas vectoriales, digitalizar elementos, añadir información alfanumérica a dichas geometrías… Esto es lo que veremos en esta primera parte del módulo.

Son muchas las herramientas disponibles en el módulo de edición de gvSIG, entre las que destacan la de crear nuevos elementos (puntos, líneas, polígonos…), rotarlos, escalarlos, desplazarlos, moverlos, crear paralelas, alargar o recortar líneas, unir o partir geometrías, crear autopolígonos o polígonos isla, etc.

Podemos digitalizar tanto con cartografía de referencia, por ejemplo una ortofoto, como utilizar la consola de edición para escribir las coordenadas de inserción.

La cartografía a utilizar en este vídeo la podéis descargar desde el siguiente enlace.

El primer vídeo-tutorial de este séptimo módulo es el siguiente:

Post relacionados:


Filed under: gvSIG Desktop, spanish, training Tagged: ayuntamientos, edición, Edición alfanumérica, edición gráfica, gestión municipal
Categories: OSGeo Planet

gvSIG Team: Workshops about Geostatistics with R and gvSIG given in Valencia and Cualiacan

OSGeo Planet - Tue, 2017-10-24 12:26

During the 13th International gvSIG Conference in Valencia and 4th gvSIG Conference in Mexico, several workshops were given about R integration in gvSIG and some development examples with R.

From the gvSIG Association we want to share all this material to be avalaible for test and consulting. You can download it from this link. Besides from a layer about Brasil, all the data in the workshop has been downloaded from the UK Police Open Data portal.

We are going to explain some of the examples:

  • Test1 – Kriging example. based on a example found in here. The target was learn how to adapt a script to gvSIG.

  • Test 11: Descomposition. Reading csv files about UK crimes we’ll try to analyze if there is a seasonal component based on time series. We also create a prognostication about the crimes in the future for that region.

  • Test12: Mean Center. An analysis of the mean center for crimes point cloud in the region of North Wales. Also, there is the data to calculate the city of London. In the North of Wales zone it can be appreciated that the Y-axis, corresponding to latitude value, it has a seasonal component. The mean center moves to the East in Winter, and to the West in Summer. We also calculate the number of total crimes (ncrimes column).

  • Test 2:R example for transformation of multiple csv files (UK Crimes) to an unique shapefile. The input parameter is the folder of the csv files.

  • Test 3: Mapa. Simple example to show how to pass parameters from gvSIG to R.

  • Test 4: Simple example of R plot.

The workshops were made using gvSIG 2.3.1. If you are using a gvSIG portable version, the R plugin is already inside. If you are using a gvSIG installable version, you should install the “R” plugin from the Plugin Administrator.

An error that can occur is the lack of some R libraries. To do this you should install them from the R Console manually. This steps could be automatize. For example, if an error similar to “Error library tcltk2 not found”. If this happen, you need to install the libraries writing in the R console: install.packages(“tcltk2”).

The libraries that you could need are: tcltk2, forecast, xts, lattice.To open the R console, you need to go to the tab “System” in the scripting composer, search the module r.extension, and execute the script RShell as we explain in this documentation.

To install the data of the workshoop, you can do it from  the Plugin Administrator -> Install package from file. select the previous downloaded file. Once is installed, go to the Scripting Composer inside Tools -> Scripting. Inside the addons folder you will find the workshop.

If you want to execute one example of r, you will have to execute it from the python file that handle the execution. Each ejemploN.py execute testN.r

There are only to script that need special attention:

  • test2: you need to open a View with the projection 4326
  • test3: you need to have a View open with the layer UFEBRASIL. In that view, you need to select in the Table of Contents the layer, this one will appear in bold.

With a little bit of development, these conditions could be managed but as the purpose of the workshop is keep it simple we didn’t do it.

Any question can be asked in the mailing list.

Hope you like it!

Some of the references we have used for the workshop:


Filed under: development, english, gvSIG Desktop, gvSIG development, scripting Tagged: r
Categories: OSGeo Planet

gvSIG Team: Final notes: 13th International gvSIG Conference

OSGeo Planet - Tue, 2017-10-24 10:19

Fantastic, great, excellent presentations, full workshops…, with these words and with many thanks, most of the more than 300 attendees to the 13th International gvSIG Conference have transmitted us a feeling of satisfaction that we have had during the event.

That’s a reflection of the situation of the gvSIG project. It started developing a desktop GIS and it has continued with a whole catalog of open source software solutions, the gvSIG Suite with ‘horizontal’ solutions: gvSIG Desktop, gvSIG Mobile, gvSIG Online and an a wide range of sector products: gvSIG Roads, gvSIG Educa, gvSIG Crime …

The gvSIG Association, responsible for the sustainability and evolution of this suite, has become a model of professional services in open source geomatics. During the conference some of these projects related to critical sectors were presented. For example about the Provincial Consortium of Firemen of Valencia, the Civil Protection National Department of Spain or the Observatory for Security and Citizen Coexistence of the Government of Cordoba in Argentina.

The vitality of the project is also defined by the innumerable novelties that were presented during the first day of the event. gvSIG Desktop 2.4 includes a lot of improvements, gvSIG Online too, the new gvSIG Mobile based on Geopaparazzi is being released now, gvSIG Crime…

And complementing the professional sector with a set of outstanding papers from the field of research and education. At the beginning of the project we spoke about synergies between administration, university and company, and now they are being materialized around gvSIG.

The gvSIG conference are always a good place to expand training. Workshops with full capacity until the last minute of the event, it has been the same if they were for users or for developers. Another excellent sign.

Finally, on Friday, there was a working day of the different teams that are working on the development of the different gvSIG products. The feeling: next year will be even better.

Understand that we are happy. And that we transmit it.


Filed under: english, events Tagged: 13th gvSIG Conference
Categories: OSGeo Planet

gvSIG Team: Material de los Talleres de Geoestadística realizados con R y gvSIG en Valencia y Culiacán

OSGeo Planet - Tue, 2017-10-24 08:51

Durante las 13as Jornadas Internacionales gvSIG en Valencia y las 4as Jornadas gvSIG de México, se dieron varios talleres muy similares explicando la Integración de R en gvSIG y ejemplos de desarrollo con R.

Desde  la Asociación queremos poner todo este material disponible para la consulta y prueba de cualquiera. Os lo podéis descargar desde este enlace. Además de una capa de Brasil, el contenido principal del taller ha sido descargado del portal de Open Data de crímenes en UK.

Vamos a explicar los diferentes ejemplos que se han realizado:

  • Test1 – Ejemplo de Kriging. Basado en un ejemplo encontrado en Internet en el siguiente enlace. El objetivo era aprender a adaptar ejemplos ya hechos a gvSIG.

  • Test 11: Descomposition. Se realiza una lectura de los csv de crímenes en UK y se busca analizar si tienen una componente estacional basada en series de tiempo. Con el resultado se busca poder pronosticar el comportamiento a futuro de los crímenes en la región.

  • Test12: Mean Center. Se realiza un análisis del eje mediano de la nube de puntos de crímenes en la región del Norte de Gales. También se incluye la opción de realizar el cálculo sobre la ciudad de Londres. En la zona del Norte de Gales se puede apreciar que en el eje Y que corresponde a la longitud de se centro mediano, sufre un desplazamiento estacional, teniendo picos en invierno (desplazamiento hacia el Este del centro mediano) y bajadas en verano (desplazamiento hacia el Oeste). También se hace teniendo en cuenta el Numero de Crímenes totales (columna ncrimes).

  • Test 2: Ejemplo en R de transformación de toda una serie de ficheros csv en un único fichero shapefile con todos los crimenes de UK para una cantidad de tiempo. El único parámetro de entrada es la carpeta donde se encuentran los csv.

  • Test 3: Mapa. Ejemplo sencillo de pase de parámetros y de mostrar un mapa a través de R.

  • Test 4: Ejemplo sencillo de gráfica en R.

Los talleres fueron realizados con la versión 2.3.1 de gvSIG. Si se usa la versión portable, el plugin de R debería de venir instalado por defecto. Si usáis la versión instalable, deberéis de ir al administrador de complementos y descargar el plugin correspondiente de “R”.

Un error que puede aparecer es que te avise de que no tiene ciertas librerías instaladas. En ese caso tienes que ir a la consola de R y instalarla manualmente (también se podría automatizar esto dentro del script). Por ejemplo, si aparece un Error library ggplot2 not found o similar deberías de hacer un install.packages(“tcltk2”) en la consola. En principio las únicas librerías de R que harían falta para instalar serían: tcltk2, forecast, xts, lattice

Esta explicado en la documentación del Taller realizado el año anterior:

http://downloads.gvsig.org/download/web/es/build/html/workshops/workshop_12gvsig_r/index.html#librerias-de-r

Para utilizar los ejemplos del taller, desde el Administrador de Complementos -> Instalación desde Archivo, seleccionaremos el fichero descargado anteriormente. Una vez realizado esto, si nos vamos a Herramientas -> Scripting -> Editor de Scripts, dentro de la carpeta de /addons/, nos debería de aparecer una carpeta con el nombre del Taller.

Si queremos ejecutar estos scripts, debemos de abrir el script correspondiente de Python que lo ejecuta. En este caso, cada ejemploN.py ejecuta el script de R testN.r

Los dos scripts que necesitas alguna condición previa para su ejecución son:

  • test2: necesita tener una Vista con la proyección 4326 asignada
  • test3: necesita tener una Vista abierta con la capa de UFEBRASIL que contiene el propio script, y con esta capa seleccionada en la Tabla de Contenidos (TOC) de la Vista.

Con algo más de desarrollo estas condiciones se podrían manejar mejor, pero por mantener un nivel de iniciados en el taller no se han implementado en los scripts.

Cualquier duda, pregunta o recomendación sobre el taller podéis escribirnos a las Listas de Correo.

¡Espero que los talleres fueran de utilidad y de su interés!

Alguna de la bibliografía base usada para los talleres:


Filed under: development, events, gvSIG Desktop, gvSIG development, scripting, spanish
Categories: OSGeo Planet

GeoTools Team: GeoTools 17.3 released

OSGeo Planet - Tue, 2017-10-24 06:04
The GeoTools team is pleased to announce the release of GeoTools 17.3:This release is also available from our maven repository.

This release is made in conjunction with GeoServer 2.11.3.

GeoTools 17.3  marks the switch of the 17.x series to maintenance mode (as 18.x takes the role of stable release) is a recommended upgrade for projects already using the 17.x series. This release comes with 15 assorted fixes and a couple of minor improvements:
  • Several improvements in image mosaic and raster rendering (in particular related to mosaics with mixed CRS, filtering and sorting, and direct modification of the mosaic index)
  • Avoid rendering empty outputs on raster data when the oversampling factor reaches very high values
  • Better mapping of dates in Oracle datastore, in particular, DATE is now mapped to java.sql.TimeStamp
  • Better toString for temporal filters (handy if you're debugging some code involving them)
  • Fixed an issue preventing to parse GeoJSON having a "crs" key among its properties
And more! For more information please see the release notes (17.3 | 17.2 | 17.1 | 17.0 | 17-RC1 | 17-beta).About GeoTools 17
  • The wfs-ng module is now a drop in replacement and will be replacing gt-wfs
  • The NetCDF module now uses NetCDF-Java 4.6.6
Upgrading
  • The AbstractDataStore has finally been removed, please transition any custom DataStore implementations to ContentDataStore (tutorial available).
Categories: OSGeo Planet
Syndicate content