OSGeo Planet

gvSIG Team: La Suite gvSIG premiada en el “Helsinki NASA Europa Challenge”

OSGeo Planet - Tue, 2017-09-05 15:52

Lo hemos vuelto hacer. Un nuevo reconocimiento. La Suite gvSIG ha sido galardonada en Helsinki con el “NASA Europa Challenge” que este año llevaba por lema “Make Earth More Livable”.

Durante este año en la Asociación gvSIG hemos trabajado duro para ir construyendo un catálogo de productos con gvSIG Desktop, gvSIG Online y gvSIG Mobile que permitan abordar cualquier proyecto con necesidades “geo” con esta suite de soluciones en software libre. Esto, además, está permitiendo ir ampliando nuestras soluciones sectoriales: gvSIG Roads, gvSIG Educa, gvSIG Crime…

Una parte principal de este trabajo se fundamenta en disponer de unas potentes herramientas 3D. Y aquí hemos apostado claramente por World Wind, el software 3D de la NASA y sus distintas variantes (Java World Wind para gvSIG Desktop, Web World Wind para gvSIG Online). Una apuesta que se ha visto refrendada en los últimos meses por la ESA (Agencia Espacial Europea), que se ha sumado también al proyecto.

Este premio es, a su vez, un reconocimiento a todas las personas y organizaciones que ayudan a hacer más grande a gvSIG día a día (y esperamos que se sumen muchas más!). Y, especialmente, desde la Asociación gvSIG queremos reconocer la apuesta de la Generalitat Valenciana por la integración de World Wind y gvSIG.

Nos despedimos con una promesa…en el próximo año os esperan novedades muy interesantes en 3D en la Suite gvSIG.

Filed under: gvSIG Suite, premios, spanish Tagged: Europa Challenge, NASA, World Wind
Categories: OSGeo Planet

gvSIG Team: 9as Jornadas de Latinoamérica y Caribe de gvSIG: Ampliación del plazo de envío de comunicaciones

OSGeo Planet - Tue, 2017-09-05 13:15

Se ha ampliado el plazo de envío de resúmenes para las 9as Jornadas de Latinoamérica y Caribe de gvSIG, siendo la nueva fecha límite el día 14 de septiembre.

Las jornadas se celebrarán del 4 al 6 de octubre de 2017 en Santa María (Rio Grande do Sul – Brasil).

Os animamos a participar no solo como asistentes, sino como ponentes, presentando vuestros trabajos realizados sobre gvSIG. Toda la información sobre el envío de comunicaciones podéis encontrarla en la web del evento.

Recordamos también que sigue abierto el periodo de inscripción, siendo las jornadas totalmente gratuitas, incluidos los talleres. Podéis inscribiros desde el formulario de la web.

¡Os esperamos!

Filed under: community, events, portuguese, spanish, training Tagged: 9as Jornadas LAC
Categories: OSGeo Planet

QGIS Blog: Do you want to host a QGIS developer meeting?

OSGeo Planet - Sat, 2017-09-02 21:49

Each year the QGIS.ORG community holds two developer meetings. These events are an important part of  our project – they provide an invaluable opportunity for us all to meet face to face and share ideas, discuss issues and plan the future of QGIS.

The host of the developer meeting gets a special bonus for hosting the meeting: One of our releases will be named after the town / village / city etc. where the event was held – like this:

Screen Shot 2017-09-02 at 11.11.31 PM.png

We want to have a better idea of which venues we will be using for future events to help with out planning. So I am putting out a call for venue proposals:

If you would like to host a QGIS developer meeting (estimated 50 people per event) or a QGIS Conference (estimated 100-150 people per event) please contact us!

Please don’t submit proposals unless you have the authority to make such a proposal and are willing to act as the local organiser for the event. To make a proposal, fill out this form and tell us about your great venue!


Categories: OSGeo Planet

PostGIS Development: PostGIS 2.4.0beta1 Released

OSGeo Planet - Sat, 2017-09-02 00:00

The PostGIS development team is pleased to announce the release of PostGIS 2.4.0beta1. PostGIS 2.4.0 will be the first version to support PostgreSQL 10. Best served with PostgreSQL 10beta4 and pgRouting 2.5.0beta See the full list of changes in the news file.

We encourage everyone to test and in particular package maintainers to insure no surprises at final release time.


From this point forward until release of PostGIS 2.4.0, no new functions will be added.
Only bug fixes will be addressed in future 2.4.0 betas and rcs.

In order to have Map Box Vector Tiles support enabled, you’ll need to compile with protobuf support and pkg-config to verify the correct minimum version of protobuf-c see protobuf for details. ST_FrechetDistance function will not be enabled if PostGIS is compiled with lower than GEOS 3.7.0. GEOS 3.7.0 will be released around the same time as PostGIS 2.4.0 and will have a beta release in a week.

Continue Reading by clicking title hyperlink ..
Categories: OSGeo Planet

Volker Mische: FOSS4G 2017

OSGeo Planet - Fri, 2017-09-01 20:56

The Global FOSS4G 2017 conference was a great experience as every year. Meeting all those people you know already for years, but also those you’ve so far met only virtually.

The talks

The program committee did a great job with the selection. Especially since there were so many to select from. Here are the most memorable talks:

  • “Optimizing Spatiotemporal Analysis Using Multidimensional Indexing with GeoWave” by Richard Fecher: The talked also touched the technical details on how they solve building a multidimensional index on top of distributed key-value stores. Currently they support Apache Accumulo, Apache HBase and Google’s Bigtable, but in theory they could also support any distributed key-value store, hence also [Apache CouchDB](http://couchdb.apache.org/ or Couchbase. I really enjoyed the technical depth and that it is based on solid research and evaluations.

  • “DIY mapping with drones and open source in a humanitarian context” by Dan Joseph: It was really nice to see that not everyone is using quadcopters for drone mapping, but that there’s also fixed-wing drones (they look like planes). It gave good details about failures and success. I wish good look with future models and the mapping itself.

  • “GPUs & Their Role in Geovisualization” by Todd Mostak: GPUs are now so powerful that you can do your multidimensional queries on points with doing table scans. That’s quite impressive. It’s also good to see that the core of MapD got open sourced under the Apache License 2.0.

Sadly I’ve missed two talks I wanted to see. One was [Steven Ottens](https://twitter.com/stvno_ speaking about “D3.js in postgres with plv8 V8/JavaScript”. It sounds daunting at first, but if you think about the numerous JavaScript libraries for geo processing that are out there, it makes sense for rapid prototyping.

The other one was Steven Feldman’s talk on “Fake Maps”. I always enjoy Steven’s talks as he digs into maps as much as I’d love to, but sadly don’t take the time to. Though he said that once the recording of the talk is out, I should grab a beer and enjoy watching it. I’m looking forward to do so.

My own talk went really well. Originally I thought being on the last slot on Friday — the last conference day — is bad, as people don’t have my time to approach you after the talk. But in the end it was actually good as I had several days to promote it to people who are interested. I loved that I was in the smallest room of the venue, hence it was packed. I’ll write more about the talk once I’ve cleaned up the code and pushed it to the master branch, so that you can all play with the spatial features yourself.

The keynotes

This year there were 5 keynotes, which I think is a good number. You always need to keep in mind that depending on the length, you might kick out 10–20 normal speaker slots. I enjoyed all of them, although in my opinion, for most of them 30 mins (instead of 45 mins) would’ve been sufficient. But I have to admit that I could probably see Paul Ramsey talking for hours and it would still be great.

Of course one keynote — the one from Richard Stallman — stood out. It surely lead to lively discussion within the community, which is really a great thing. I share the opinion of Jeff McKenna that I really respect what Stallman did and is doing and how much he is into it. Though it came clear to me that I am an Open Source developer who cares about openness and transparency.

The venue

The venue was a typical conference center, which had the benefit that the rooms were close together. This made switching rooms even within slots easily possible.

One thing I didn’t like was the air conditioning. Some rooms were cooled down way to much. Did anyone measure? I know, it’s a cultural thing and not the fault of the organizers. Though I wonder how much energy and money could’ve been saved when the temperature would’ve been lowered to an acceptable level only.

Sometimes there are discussions about the location of the OSGeo booth within the exhibition area. I think this year it was in a good spot. It wasn’t at the most prestigious place, that’s for Diamond sponsors. But at a spot where people actually gather/hang out, that’s a way better fit in my opinion.

The social events

The social events were nice and I was happy that I was able to bring in a well known and liked former community member into the icebreaker event. The icebreaker reminded me a bit of last year’s one. There it was possible to bring anyone who wanted to go there. I think the attendees had some vouchers, but I can’t recall really the details. Anyway, I think it’s a good idea to have one social event where you can bring in people that are in the area, but don’t attend the conference.

The code sprint

The code sprint was hosted at the District Hall which is a innovation/startup/co-working place. We had the whole space which was really nice. The different tribes, Java, MapServer, Postgres and Fancy Shit assembled at different spots and put up signs, so it was easy to find your way to the right group.


I also need to mention that the day before the FOSSS4G there was the JS.Geo at the same place as the code sprint. It was a really nice event and if I ever organize an English single track geo conference, I’ll get Brian Timoney as a moderator. He was so entertaining and really contributed to the great vibe this conference had.


This year there wasn’t a printed program brochure. It was all just available online at — the certainly cool — https://foss4g.guide/ or as app. I on my FirefoxOS phone was using the website. I think it could’ve been better to navigate, but it was OK and I didn’t really miss the brochure. The website based guide was OK when you were on your phone and on-site, to see which talks are up next. I don’t think it worked well if you tried to do some ahead of time planning.

The FOSS4G t-shirts look great, but I’m a bit sad that they were grey (a nice one though) and the Local Team had t-shirts in my favourite orange color.

Notes for future years

It might really make sense to not producing a printed program brochure anymore as probably all attendees have a smartphone anyway (though this needs to be checked by Local Team depending on the area). If you decide to go Web only, you should make sure it works offline and perhaps spending the time you would’ve spent on the printed one instead on the usability of the web one.

Categories: OSGeo Planet

Fernando Quadro: Infraestrutura de dados espaciais com GeoNode – Parte 2

OSGeo Planet - Fri, 2017-09-01 12:23

No último post, tivemos uma breve introdução do que é o GeoNode. Neste post iremos falar do que você precisa pra instalar a sua instância deste gerenciador de informações espaciais.

1. Requisitos para o servidor

O GeoNode é composto por várias camadas de software e serviços, veja abaixo:

– tomcat7/jetty: Para atender as requisições e gerenciar as conexões com os servlets do geoserver
– apache2: Para atender as requisições web em conjunto com o Django
– Django: Usado para fornecer os dados para o frontend e controlar a administração dos metadados.
– Geoserver: Usado para gerenciar os mapas.
– PostgreSQL: Servidor de banco de dados.
– PyCSW: Servidor de catálogo de metadados, pode ser substituido pelo Geonetwork.

Em testes realizados em máquina com menos de 2GB de RAM foram não obtidos bons resultados de desempenho. A camada Java (Spring FW) consome muitos recursos, apesar de ter um desempenho bom quando acessada por milhares de requisições, porém para um sistema com poucos usuários é bem pesado.

A recomendação mínima são 6 GB de RAM, 100 GB de disco e processador com 64bits 2Ghz, preferencialmente com mais de um núcleo de processamento para operações concorrentes.

2. Instalação

A instalação pode ser feita de várias formas, mas por se tratar de um sistema complexo, com mais de um backend funcionando ao mesmo tempo recomendo as formas mais simples, que são:

2.1 Ubuntu

Essa forma é a ideal para fornecer uma VPS ou hardware dedicado. O procedimento faz toda configuração inicial, mas não leva em conta configurações específicas para um melhor desempenho do SO, principalmente para máquinas com pouca memória.

$ sudo add-apt-repository ppa:geonode/stable $ sudo apt-get update $ sudo apt-get install -y build-essential libxml2-dev libxslt1-dev libpq-dev zlib1g-dev $ sudo apt-get install geonode

Importante: Não esqueça de configurar o sistema para responder no seu endereço de IP ou domínio. Caso isso não seja feito o sistema vai retornar Erro 400 ( Bad request ). Para realizar esse procedimento, o comando é :

sudo geonode-updateip

É importante também criar o usuário admin do sistema. Veja como:

$ geonode createsuperuser

2.2 Docker

É possível usar o docker principalmente para fazer desenvolvimento do geonode, o processo é simples:

# constroi o container docker docker build -t geonode . # executa o container docker run -d -p 8111:8000 -p 8181:8080 geonode

Ele vai disponibilizar toda a base que o geonode precisa como servidores web, servidor de banco de dados, infraestrutura java, etc.

O processo demora um pouco, e exige uma máquina com pelo menos 2GB de RAM ou pode apresentar falhas.

2.3 Outras possibilidades

Além das duas opções citadas acima (Ubuntu e Docker) foi disponibilizado a partir da versão 2.4 um arquivo de instalação para o Windows, MacOSX, além de você poder criar uma VM a partir do Vagrant.

Para maiores informações sobre estes tipos de instalação, você pode clicar aqui.

Categories: OSGeo Planet

QGIS Polska: blog:i_spotkanie_uzytkownikow_qgis

OSGeo Planet - Fri, 2017-09-01 10:09
Zapraszamy na I spotkanie użytkowników QGIS. Mamy nadzieję, że każdy znajdzie coś dla siebie - zarówno osoby zaczynające przygodę z systemami informacji przestrzennej, jak i programiści zajmujący się zawodowo geoinformacją. Postaramy się zaprezentować praktyczne zagadnienia związane z wykorzystaniem programu QGIS oraz przedstawić nowości z wersji 3, która pojawi się niebawem. Jeśli w swojej pracy wykorzystujesz QGIS - ta impreza jest dla Ciebie!
Categories: OSGeo Planet

Fernando Quadro: Infraestrutura de dados espaciais com GeoNode – Parte 1

OSGeo Planet - Thu, 2017-08-31 12:58

GeoNode é um sistema de gerenciamento de conteúdo geoespacial, uma plataforma para gerenciamento e publicação de dados geoespaciais. Ele reúne projetos de software open source maduros e estáveis sob uma interface consistente e fácil de usar, permitindo que usuários não especializados compartilhem dados e criem mapas interativos.

As ferramentas de gerenciamento de dados incorporadas no GeoNode permitem a criação integrada de dados, metadados e visualizações de mapas. Cada conjunto de dados no sistema pode ser compartilhado publicamente ou restringido para permitir o acesso a apenas usuários específicos. Recursos sociais como perfis de usuários e sistemas de comentários e avaliações permitem o desenvolvimento de comunidades em torno de cada plataforma para facilitar o uso, gerenciamento e controle de qualidade dos dados que contém a instância GeoNode.

Também é projetado para ser uma plataforma flexível que os desenvolvedores de software podem ampliar, modificar ou integrar para atender aos requisitos em suas próprias aplicações.

1. Armazenamento de dados geoespaciais

O GeoNode permite que o usuário faça upload de dados vetoriais (atualmente apenas shapefiles) e dados raster em suas projeções originais. Os dados vetoriais são carregados no ESRI Shapefile e imagens de satélite e outros tipos de dados raster são carregados como GeoTIFFs.

É dada uma importância especial aos formatos de metadados como o padrão ISO 19139. Após a conclusão do upload, o usuário recebe um formulário para preencher os metadados e disponibilizá-los usando uma interface CSW. Os usuários também podem carregar um documento XML de metadados (no formato ISO, FGDC ou Dublin Core) para preencher automaticamente os elementos de metadados no GeoNode.

Da mesma forma, o GeoNode fornece um estilizador baseado na web, que permite ao usuário alterar a aparência dos dados e visualizar as mudanças em tempo real.

2. Mistura de dados e criação de mapas

Uma vez que os dados foram carregados, o GeoNode permite ao usuário buscá-lo geograficamente ou através de palavras-chave e também criar mapas.

Todas as camadas são automaticamente re-projetadas para Web Mercator para exibição de mapas, possibilitando o uso de diferentes camadas base populares, como o OpenStreetMap, o Google Satellite ou as camadas Bing.

Uma vez que os mapas são salvos, é possível inseri-los em qualquer página da Web ou obter uma versão em PDF para impressão.

3. Live Demo

O GeoNode disponibiliza uma versão de demonstração da última compilação estável, que está disponível em demo.geonode.org.

Qualquer pessoa pode criar uma conta de usuário, criar dados, compartilhar mapas e alterar as permissões desses dados. Como é um site de demonstração, não é possível garantir que seus dados e mapas estarão sempre lá. Mas permiti que você visualize facilmente os recursos disponíveis no GeoNode.

Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Fixing invalid polygon geometries

OSGeo Planet - Tue, 2017-08-29 20:08

Invalid geometries can cause a lot of headache: from missing features to odd analysis results.

This post aims to illustrate one of the most common issues and presents an approach that can help with these errors.

The dataset used for this example is the Alaska Shapefile from the QGIS sample data:

This dataset has a couple of issues. One way to find out if a dataset contains errors is the Check Validity tool in the Processing toolbox:

If there are errors, a layer called Error output will be loaded. In our case, there are multiple issues:

If we try to use this dataset for spatial analysis, there will likely be errors. For example, using the Fixed distance buffer tool results in missing features:

Note the errors in the Processing log message panel:

Feature ### has invalid geometry. Skipping ...

So what can we do?

In my experience, GRASS can work wonders for fixing these kind of issues. The idea is to run v.buffer.distance with the distance set to zero:

This will import the dataset into GRASS and run the buffer algorithm without actually growing the polygons. Finally, it should export a fixed version of the geometries:

A quick validity check with the Check validity tool confirms that there are no issues left.


Categories: OSGeo Planet

Geomatic Blog: Geocamp ES 2017

OSGeo Planet - Tue, 2017-08-29 13:20

Quedan poco más de dos semanas para la GeocampES 2017, que este año se va a celebrar el sábado 16 de Septiembre en Almería, organizada por Geoinquietos AlmeríaHacklab Almería y UNIA. Como cada año la idea es organizar un evento sencillo, divertido y muy técnico y participativo. No hay agenda, siguiendo el modelo de desconferencia habitual, al llegar allí se espera que un buen porcentaje de los asistentes se postulen para salir a contar algo: puede ser una charla convencional, una demostracíon de algo en lo que estás trabajando, una dinámica de grupo que te parezca interesante, etc. Cualquier actividad con una componente geo bien definida (o no tanto) es bienvenida.

La idea de Geocamp ES se tomó siguiendo el liderazgo de los compañeros de Portugal. A diferencia sus geocamps en pequeños pueblos más o menos alejados de todo, las anteriores Geocamp españolas se han celebrado en ciudades. En cualquier caso este año nos vamos a Almería, que está un poco más difícil de acceder (sin exagerar) pero sigue siendo una ciudad que seguro va a ofrecernos un espacio agradable y totalmente adecuado para pasarlo bien aprendiendo.

Si lo que te he contado te parece mínimamente interesante, anímate y prepara un fin de semana por el sur de España con el resto de la comunidad geoespacial.

Apúntate aquí.

Filed under: GIS
Categories: OSGeo Planet

OSGeo News: GeoMoose 3.0.0 Released

OSGeo Planet - Mon, 2017-08-28 21:58
Categories: OSGeo Planet

Jackie Ng: React-ing to the need for a modern MapGuide viewer (Part 19): Highlighting selected features

OSGeo Planet - Sun, 2017-08-27 16:59
When I was implementing the Selection Panel for mapguide-react-layout, this super-ancient Fusion ticket was on the back of my mind as being able to highlight selected features is a very useful feature to have.

Whereas I didn't really have an idea how this would be done in Fusion, for mapguide-react-layout I had an idea how this would be done.

The trick is to use the v2.6 QUERYMAPFEATURES operation with the following parameters:

  • REQUESTDATA=2 (asking for inline selection image only)
  • LAYERATTRIBUTEFILTER=0 (so that selection is not constrained by layer selectability and our current view)
  • FEATUREFILTER=(the selection XML sub-fragment that represents the selected feature)
  • PERSIST=0 (the most important parameter. We want this operation to not alter the selection set)
Once the QUERYMAPFEATURES operation is sent and we get a response, the key is to be able to line up the inline selection image (data URI) with the current map. To handle this part, we use an OL static image source to store the inline selection image.
When all the pieces are brought together, we finally have the ability to highlight selected features.

This will be in the next release of mapguide-react-layout.
Categories: OSGeo Planet

Free and Open Source GIS Ramblings: Getting started with GeoMesa using Geodocker

OSGeo Planet - Sun, 2017-08-27 15:21

In a previous post, I showed how to use docker to run a single application (GeoServer) in a container and connect to it from your local QGIS install. Today’s post is about running a whole bunch of containers that interact with each other. More specifically, I’m using the images provided by Geodocker. The Geodocker repository provides a setup containing Accumulo, GeoMesa, and GeoServer. If you are not familiar with GeoMesa yet:

GeoMesa is an open-source, distributed, spatio-temporal database built on a number of distributed cloud data storage systems … GeoMesa aims to provide as much of the spatial querying and data manipulation to Accumulo as PostGIS does to Postgres.

The following sections show how to load data into GeoMesa, perform basic queries via command line, and finally publish data to GeoServer. The content is based largely on two GeoMesa tutorials: Geodocker: Bootstrapping GeoMesa Accumulo and Spark on AWS and Map-Reduce Ingest of GDELT, as well as Diethard Steiner’s post on Accumulo basics. The key difference is that this tutorial is written to be run locally (rather than on AWS or similar infrastructure) and that it spells out all user names and passwords preconfigured in Geodocker.

This guide was tested on Ubuntu and assumes that Docker is already installed. If you haven’t yet, you can install Docker as described in Install using the repository.

To get Geodocker set up, we need to get the code from Github and run the docker-compose command:

$ git clone https://github.com/geodocker/geodocker-geomesa.git $ cd geodocker-geomesa/geodocker-accumulo-geomesa/ $ docker-compose up

This will take a while.

When docker-compose is finished, use a second console to check the status of all containers:

$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 4a238494e15f quay.io/geomesa/accumulo-geomesa:latest "/sbin/entrypoint...." 19 hours ago Up 23 seconds geodockeraccumulogeomesa_accumulo-tserver_1 e2e0df3cae98 quay.io/geomesa/accumulo-geomesa:latest "/sbin/entrypoint...." 19 hours ago Up 22 seconds>50095/tcp geodockeraccumulogeomesa_accumulo-monitor_1 e7056f552ef0 quay.io/geomesa/accumulo-geomesa:latest "/sbin/entrypoint...." 19 hours ago Up 24 seconds geodockeraccumulogeomesa_accumulo-master_1 dbc0ffa6c39c quay.io/geomesa/hdfs:latest "/sbin/entrypoint...." 19 hours ago Up 23 seconds geodockeraccumulogeomesa_hdfs-data_1 20e90a847c5b quay.io/geomesa/zookeeper:latest "/sbin/entrypoint...." 19 hours ago Up 24 seconds 2888/tcp,>2181/tcp, 3888/tcp geodockeraccumulogeomesa_zookeeper_1 997b0e5d6699 quay.io/geomesa/geoserver:latest "/opt/tomcat/bin/c..." 19 hours ago Up 22 seconds>9090/tcp geodockeraccumulogeomesa_geoserver_1 c17e149cda50 quay.io/geomesa/hdfs:latest "/sbin/entrypoint...." 19 hours ago Up 23 seconds>50070/tcp geodockeraccumulogeomesa_hdfs-name_1

At the time of writing this post, the Geomesa version installed in this way is 1.3.2:

$ docker exec geodockeraccumulogeomesa_accumulo-master_1 geomesa version GeoMesa tools version: 1.3.2 Commit ID: 2b66489e3d1dbe9464a9860925cca745198c637c Branch: 2b66489e3d1dbe9464a9860925cca745198c637c Build date: 2017-07-21T19:56:41+0000

Loading data

First we need to get some data. The available tutorials often refer to data published by the GDELT project. Let’s download data for three days, unzip it and copy it to the geodockeraccumulogeomesa_accumulo-master_1 container for further processing:

$ wget http://data.gdeltproject.org/events/20170710.export.CSV.zip $ wget http://data.gdeltproject.org/events/20170711.export.CSV.zip $ wget http://data.gdeltproject.org/events/20170712.export.CSV.zip $ unzip 20170710.export.CSV.zip $ unzip 20170711.export.CSV.zip $ unzip 20170712.export.CSV.zip $ docker cp ~/Downloads/geomesa/gdelt/20170710.export.CSV geodockeraccumulogeomesa_accumulo-master_1:/tmp/20170710.export.CSV $ docker cp ~/Downloads/geomesa/gdelt/20170711.export.CSV geodockeraccumulogeomesa_accumulo-master_1:/tmp/20170711.export.CSV $ docker cp ~/Downloads/geomesa/gdelt/20170712.export.CSV geodockeraccumulogeomesa_accumulo-master_1:/tmp/20170712.export.CSV

Loading or importing data is called “ingesting” in Geomesa parlance. Since the format of GDELT data is already predefined (the CSV mapping is defined in geomesa-tools/conf/sfts/gdelt/reference.conf), we can ingest the data:

$ docker exec geodockeraccumulogeomesa_accumulo-master_1 geomesa ingest -c geomesa.gdelt -C gdelt -f gdelt -s gdelt -u root -p GisPwd /tmp/20170710.export.CSV $ docker exec geodockeraccumulogeomesa_accumulo-master_1 geomesa ingest -c geomesa.gdelt -C gdelt -f gdelt -s gdelt -u root -p GisPwd /tmp/20170711.export.CSV $ docker exec geodockeraccumulogeomesa_accumulo-master_1 geomesa ingest -c geomesa.gdelt -C gdelt -f gdelt -s gdelt -u root -p GisPwd /tmp/20170712.export.CSV

Once the data is ingested, we can have a look at the the created table by asking GeoMesa to describe the created schema:

$ docker exec geodockeraccumulogeomesa_accumulo-master_1 geomesa describe-schema -c geomesa.gdelt -f gdelt -u root -p GisPwd INFO Describing attributes of feature 'gdelt' globalEventId | String eventCode | String eventBaseCode | String eventRootCode | String isRootEvent | Integer actor1Name | String actor1Code | String actor1CountryCode | String actor1GroupCode | String actor1EthnicCode | String actor1Religion1Code | String actor1Religion2Code | String actor2Name | String actor2Code | String actor2CountryCode | String actor2GroupCode | String actor2EthnicCode | String actor2Religion1Code | String actor2Religion2Code | String quadClass | Integer goldsteinScale | Double numMentions | Integer numSources | Integer numArticles | Integer avgTone | Double dtg | Date (Spatio-temporally indexed) geom | Point (Spatially indexed) User data: geomesa.index.dtg | dtg geomesa.indices | z3:4:3,z2:3:3,records:2:3 geomesa.table.sharing | false

In the background, our data is stored in Accumulo tables. For a closer look, open an interactive terminal in the Accumulo master image:

$ docker exec -i -t geodockeraccumulogeomesa_accumulo-master_1 /bin/bash

and open the Accumulo shell:

# accumulo shell -u root -p GisPwd

When we store data in GeoMesa, there is not only one table but several. Each table has a specific purpose: storing metadata, records, or indexes. All tables get prefixed with the catalog table name:

root@accumulo> tables accumulo.metadata accumulo.replication accumulo.root geomesa.gdelt geomesa.gdelt_gdelt_records_v2 geomesa.gdelt_gdelt_z2_v3 geomesa.gdelt_gdelt_z3_v4 geomesa.gdelt_queries geomesa.gdelt_stats

By default, GeoMesa creates three indices:
Z2: for queries with a spatial component but no temporal component.
Z3: for queries with both a spatial and temporal component.
Record: for queries by feature ID.

But let’s get back to GeoMesa …

Querying data

Now we are ready to query the data. Let’s perform a simple attribute query first. Make sure that you are in the interactive terminal in the Accumulo master image:

$ docker exec -i -t geodockeraccumulogeomesa_accumulo-master_1 /bin/bash

This query filters for a certain event id:

# geomesa export -c geomesa.gdelt -f gdelt -u root -p GisPwd -q "globalEventId='671867776'" Using GEOMESA_ACCUMULO_HOME = /opt/geomesa id,globalEventId:String,eventCode:String,eventBaseCode:String,eventRootCode:String,isRootEvent:Integer,actor1Name:String,actor1Code:String,actor1CountryCode:String,actor1GroupCode:String,actor1EthnicCode:String,actor1Religion1Code:String,actor1Religion2Code:String,actor2Name:String,actor2Code:String,actor2CountryCode:String,actor2GroupCode:String,actor2EthnicCode:String,actor2Religion1Code:String,actor2Religion2Code:String,quadClass:Integer,goldsteinScale:Double,numMentions:Integer,numSources:Integer,numArticles:Integer,avgTone:Double,dtg:Date,*geom:Point:srid=4326 d9e6ab555785827f4e5f03d6810bbf05,671867776,120,120,12,1,UNITED STATES,USA,USA,,,,,,,,,,,,3,-4.0,20,2,20,8.77192982456137,2007-07-13T00:00:00.000Z,POINT (-97 38) INFO Feature export complete to standard out in 2290ms for 1 features

If the attribute query runs successfully, we can advance to some geo goodness … that’s why we are interested in GeoMesa after all … and perform a spatial query:

# geomesa export -c geomesa.gdelt -f gdelt -u root -p GisPwd -q "CONTAINS(POLYGON ((0 0, 0 90, 90 90, 90 0, 0 0)),geom)" -m 3 Using GEOMESA_ACCUMULO_HOME = /opt/geomesa id,globalEventId:String,eventCode:String,eventBaseCode:String,eventRootCode:String,isRootEvent:Integer,actor1Name:String,actor1Code:String,actor1CountryCode:String,actor1GroupCode:String,actor1EthnicCode:String,actor1Religion1Code:String,actor1Religion2Code:String,actor2Name:String,actor2Code:String,actor2CountryCode:String,actor2GroupCode:String,actor2EthnicCode:String,actor2Religion1Code:String,actor2Religion2Code:String,quadClass:Integer,goldsteinScale:Double,numMentions:Integer,numSources:Integer,numArticles:Integer,avgTone:Double,dtg:Date,*geom:Point:srid=4326 139346754923c07e4f6a3ee01a3f7d83,671713129,030,030,03,1,NIGERIA,NGA,NGA,,,,,LIBYA,LBY,LBY,,,,,1,4.0,16,2,16,-1.4060533085217,2017-07-10T00:00:00.000Z,POINT (5.43827 5.35886) 9e8e885e63116253956e40132c62c139,671928676,042,042,04,1,NIGERIA,NGA,NGA,,,,,OPEC,IGOBUSOPC,,OPC,,,,1,1.9,5,1,5,-0.90909090909091,2017-07-10T00:00:00.000Z,POINT (5.43827 5.35886) d6c6162d83c72bc369f68bcb4b992e2d,671817380,043,043,04,0,OPEC,IGOBUSOPC,,OPC,,,,RUSSIA,RUS,RUS,,,,,1,2.8,2,1,2,-1.59453302961275,2017-07-09T00:00:00.000Z,POINT (5.43827 5.35886) INFO Feature export complete to standard out in 2127ms for 3 features

Functions that can be used in export command queries/filters are (E)CQL functions from geotools for the most part. More sophisticated queries require SparkSQL.

Publishing GeoMesa tables with GeoServer

To view data in GeoServer, go to http://localhost:9090/geoserver/web. Login with admin:geoserver.

First, we create a new workspace called “geomesa”.

Then, we can create a new store of type Accumulo (GeoMesa) called “gdelt”. Use the following parameters:

instanceId = accumulo zookeepers = zookeeper user = root password = GisPwd tableName = geomesa.gdelt


Then we can configure a Layer that publishes the content of our new data store. It is good to check the coordinate reference system settings and insert the bounding box information:


To preview the WMS, go to GeoServer’s preview:


Which will look something like this:


GeoMesa data filtered using CQL in GeoServer preview

For more display options, check the official GeoMesa tutorial.

If you check the preview URL more closely, you will notice that it specifies a time window:


This is exactly where QGIS TimeManager could come in: Using TimeManager for WMS-T layers. Interoperatbility for the win!

Categories: OSGeo Planet

Antonio Santiago: Graceful shutdown NodeJS HTTP server when using PM2

OSGeo Planet - Sun, 2017-08-27 12:10

So you have created a NodeJS server that receives tons of requests and you are really happy but, as every piece of software, you found a bug or add a new feature to it. It is clear you will need to shutdown your NodeJS process/es and restart again so that the new code takes place. The question is: how can you do that in a graceful way that allows continue serving incoming requests?

Categories: OSGeo Planet

QGIS Blog: Plotting the future of QGIS

OSGeo Planet - Fri, 2017-08-25 11:42

During the developer hackfest at our recent QGIS Conference in Nødebo, the developers present had a discussion session about the future (post 3.0) road map for QGIS. Note that the ideas laid out here do no necessarily represent a consensus between all the QGIS developers and community members since those present at the hackfest were only a subset of the great QGIS community. However the discussion probably provides a good idea of the kind of things on our minds as we move forward to QGIS 3.0 and beyond. Just a note before you get too excited reading the article below: This was a future looking session of great ideas that will take QGIS forward, but there may not be anybody actively working on these ideas (if you are looking for something to fund it would be a great start!). Here are twelve ideas that were raised (in no particular order)…

1. We need to beef up the analytical capabilities in QGIS

There was a general feeling that we should have stronger analytical capabilities in QGIS. Somewhere along the line we lost ManageR (the R integration with QGIS) and we have missed the boat in having something like Pandas / Jupyter Notebooks, embedded into QGIS (with iface available to the console). Whilst many data scientists are using R, going the python route with Pandas and Jupyter Notebooks might be a better fit in terms of being harmonious with the other work that has been done to provide python bindings for QGIS. But hey, why not provide both a Jupyter Notebook that supports both Python and R out of the box? Technically curious may want to look here for some hints on how we might go about integrating Jupyter into the QGIS application…

2. We need to improve our ‘first open’ experience

Especially for new users and novice GIS users, starting a QGIS project with a blank white canvas and many buttons and menus can be quite intimidating. We want to provide some basic projects (e.g. based on OpenStreetmap tiles) that can appear as a default layer when you open the QGIS application so that you can immediately get a sense of place and space – much like you would get in Google maps or any web mapping application. Naturally we will provide the option to disable this for those who are not interested in this functionality, but we would make it a default behaviour for new users…

3. We need a better way of communicating with our users

We do not even know simple things like how many users we have (I estimate broadly between 500 000 and 1 000 000 users based on downloads). Most users are silent users – they never communicate with the upstream project via our mailing lists or other communication mechanisms. Not knowing stuff about our users makes it hard to build a better product for them, and not having a communication channel with our users makes it hard for us to let them know about important updates, bug fixes, events etc. and it is a bit silly to be in this situation because every time a user opens QGIS, we have an opportunity to share this kind of information with them. So in the future it would be nice to have a way to provide timed and targeted messages to our users (for example letting them know when we have made a new blog post on the official QGIS blog). It would be nice to have the notification system scriptable by plugin. Of course it should be easy to opt out of or filter the messages by category (e.g. don’t show me event announcements) we share with our users. Imagine on the projects list view you see when you first open QGIS that we have a panel to the right of the projects list which just lists the headlines of the latest announcements. Perhaps there are other ways we can communicate with our users, but we should really make it a priority to get to know our users and this seems like a good start. By seeing how many times a given article gets read after it as been posted in the QGIS announcement area, we might get a better indication of how many users we have. Another example – when a new LTR bug fix comes out, we can publicise it better to make sure users are aware of the important fixes.

4. We need to focus on Quality Assurance (QA)

Especially as relates to reducing the incidence of side effects, QA is going to be critical as the project grows and gains a user base that uses it for critical functions. Side effects happen when e.g. a developer implements one feature that (probably unbeknownst to him) breaks another feature. Side effects are bad because they are hard to test for and hard to trace back to the root cause. The development of QGIS happens in a largely ad hoc manner – developers get contracts to build features their clients need, there is no top-down approach to how we roll out new features. This makes it difficult for us to ensure that side effects do not happen. We are not only concerned with side effects, but QA in general and would like to have the time and resources to spend on really taking the work that has already been put in place (automated testing on Travis for example) to the next level.

5. We need more dedicated (paid) effort to take care of the project

QGIS has become too big of a project to rely entirely on volunteers to take care of all aspects of the project. Many of us still contribute many hours of unpaid volunteer time to the project and will continue to do so. It has long been my vision that we eventually recruited a corps of professional (i.e. paid) contributors to work on QGIS, especially to take care of things that contract work will never cover. For example triaging the pull request queue (which is extremely time consuming), managing the issues in the issue tracker, expanding our test suite coverage, writing documentation, fixing bugs and building ‘cross cutting functionality that typically would not be funded by client work but that everyone will derive benefit from. Our project revenue (from sponsorships and donations) has been steadily growing (thank you to all of those that have contributed!) and if we can increase the revenue a little more we will reach the point where we can start to recruit some of our community members to work for QGIS.org on a professional basis – maybe on a part time basis in the beginning, but eventually building a corps of full time paid staff. This has long been a vision of mine for QGIS and if it is the one thing we achieve while I am project chair, I will be a happy chappie!

6. We need to automate trace captures

This relates somewhat to 3. above – when a user experiences a crash in QGIS, we have no automated way to get that crash information (and no Apple / Microsoft do not pass along the tracebacks to us when they offer to let you post them to their domain

Categories: OSGeo Planet

From GIS to Remote Sensing: SCP Questions of This Month: August

OSGeo Planet - Fri, 2017-08-25 09:38
This post is a collection of questions and answers about the Semi-Automatic Classification Plugin (SCP) and remote sensing which were discussed in the Facebook group and the Google+ Community this month.
These questions vary from supervised classification technique to software issues, and can be useful to the readers of this blog for solving issues about the use of SCP.

Categories: OSGeo Planet

GIScussions: FOSS4G – it’s getting better all the time

OSGeo Planet - Thu, 2017-08-24 16:21

FOSS4G just gets better and better, perhaps it’s the delight of being on a geobusman’s holiday rather than being the conference chair?

I’d been in conversation with Michael Terner and Guido Stein, the co-chairs of FOSS4G 2017, for over a year in my role as a past chair and member of the OSGeo Conference Committee. I know how much passion and effort they and their team put into bringing FOSS4G to Boston, their vision and attention to detail paid off in spades with the largest FOSS4G ever.

#FOSS4G by the numbers

1141 attendees
1309 workshop people at 56 sessions #AWESOME

— (((Steven Feldman))) (@StevenFeldman) August 18, 2017

The Boston LOC for #FOSS4G. We owe you enormous thanks. Time to have a well deserved rest & enjoy what you've created pic.twitter.com/j5eDGbmvI8

— (((Steven Feldman))) (@StevenFeldman) August 18, 2017

So where do I start in describing this feast of open source  geo to folk who weren’t there and may never have attended a FOSS4G? Here is a snapshot of my personal experience punctuated with some tweets that give a flavour of the event.

Before the event I had helped to organise the FOSS4G Travel Grant Programme. I wrote about “Paying it Forward” before the event, with support from OSGeo sponsors, a generous donor and a big crowdfunding campaign we were able to bring 10 people from all over the world to Boston. What an amazing pleasure to meet these people, hear their stories and know that we had done something good that would ripple through their lives and impact the communities that they would return to. We also did a tiny bit to promote diversity which is definitely an ongoing topic for OSGeo and most of the tech world. I’ll post some more on the grantees experiences in a few weeks time once they have written back to me.

If you missed all the noise about the TGP for 2018, we are trying to raise $50,000 to really make a difference next year and support a lot of people coming to FOSS4G who wouldn’t otherwise be able to make it, you can (and should) chip in here.

Monday was spent working with Marc Vloemans on our Sales and Marketing 101 workshop and familiarising ourselves with Boston, a very cool city, and finding the conference venue.

And the 1st sign is up! pic.twitter.com/Cwb44excug

— (((Steven Feldman))) (@StevenFeldman) August 14, 2017

And meeting up with old friends that I hadn’t seen for a while.

Meeting of the British former #FOSS4G chairs club – good dinner with @StevenFeldman pic.twitter.com/ehMedVoEVC

— Peter Batty (@pmbatty) August 15, 2017

Tuesday was workshop day. An early start to get a stack of donuts to energise our delegates and we are off.

Workshop Sales&Marketing101 @foss4g is well underway with @StevenFeldman and I pic.twitter.com/AdlG0P7IWd

— Marc Vloemans (@MarcVloemans) August 15, 2017

We worked hard today in our sales & marketing workshop shout out to @MarcVloemans & @StevenFeldman #foss4g pic.twitter.com/ahLn0B9fLT

— Matt Krusemark (@kspatial) August 16, 2017

Learning about marketing from @StevenFeldman at #FOSS4G2017 with a British pun: don't sell open SOURCE; sell secret SAUCE

— Sarah Chow (@chowslc) August 15, 2017

The workshop seems to “work” well and the feedback has been very positive so I am planning to run the workshop again in London (and anywhere else that wants me), you can read my write up on S&M101 and view the slides

Tuesday night brought the first party of FOSS4G, loads of friends all gathered together including Vasile and Maria who both told me that they would be submitting proposals for Bucharest and Seville respectively to host FOSS4G 2019. We haven’t even kicked off the main conference and the competition is on for 2 years time!

Maybe #FOSS4G 2019 will be in Bucharest or Seville. Either way these 2 will be friends @vcraciunescu @delawen pic.twitter.com/TgtSeYjF2r

— (((Steven Feldman))) (@StevenFeldman) August 16, 2017

Wednesday morning and the main event opens, Michael Terner in his welcome, remarks that it is 25 months since they started writing their proposal to host FOSS4G. Why would anyone do this?

What if you throw a party & nobody comes?

What if that party costs at least $500,000?

The worries of a @foss4g chair

Thanks for coming

— (((Steven Feldman))) (@StevenFeldman) August 16, 2017

What 1000 open source Geo enthusiasts look like – congrats to the Boston #FOSS4G team pic.twitter.com/BcweqoUso2

— Ed Parsons (@edparsons) August 16, 2017

The quick summary of sessions I attended
  • Paul Ramsey is a superb speaker, his keynote “Why we code” is a must watch when the videos are uploaded. You can view the slides here
  • “Wikipedia maps” was a session about how data from wikipedia and OSM can be merged to provide better multi-lingual labelling, some smart stuff going on. Worth checking out.
  • “Breaking up is easy to do” was a pretty regular case study from a small not for profit but articulated the ease and savings of migrating from ArcGIS to QGIS – sadly the NFP had no policy to contribute back some of their savings to the QGIS project
  • “What’s up with diversity?” by Maria Arias de Reyna was one of the only talks on diversity (that I have heard) that did not turn into an anguished wail. I encourage you to look out for the video when posted, if not for the important content then for her amazing slide decks

What's up with diversity? @delawen nailing it pic.twitter.com/XKUo2s53Tw

— (((Steven Feldman))) (@StevenFeldman) August 16, 2017

"Please respect safe spaces even if you don't understand them"@delawen on diversity #Foss4g

— (((Steven Feldman))) (@StevenFeldman) August 16, 2017

  • “Making money & building a business with open source” was a panel discussion. Not very customer focussed but there was one great line that I am sure I will borrow to use in a presentation

“Users may come to open source for the wrong reasons, we hope they will stay for the right reasons”

  • “Google Earth Enterprise goes open source”  Ed Parsons gave a fascinating explanation of how Google acquired Keyhole, why they went enterprise and why they stopped. And he had a great slide about enterprise sales people!

"our sales people have very short timespans of attention" says @edparsons #FOSS4G pic.twitter.com/ZsEiyLP4Cz

— (((Steven Feldman))) (@StevenFeldman) August 17, 2017

  • “The Utility of beautiful geovisualisations” by Laura Tatesosian demonstrated some amazing techniques for visualising massive multivariate datasets (heat mappers eat your hearts out), one for the cartographers
  • “Critical Cartography” – Mathew Cloyd – maps and algorithms have bias (even when we think they are objective) some great examples – this was thought provoking and one of the best talks I attended – look out for the video
  • “Non cartographic visualisations of geographic data”  by Andrew Turner. Esri have developed an open source library called Cedar which incorporates Vega  – incredible potential for custom charts plus an enormous library of out the box stuff – if you have apps that need charts you should look at this. It’s good to see a company like Esri, who are often demonised within the open source community (unfairly IMO), making a commitment to open source and sharing something really useful.
  • “Free Software: Freedom, Privacy, Sovereignty” was the afternoon keynote by Richard Stallman, the founder of the Free Software Foundation and the/a developer of GNU Project. This wasn’t my cup of tea but the purists seemed to enjoy it, all 90 minutes of it!

An alarming talk by Richard Stallman about Orwellian surveillance and the evils of proprietary software aka malware #foss4g pic.twitter.com/TEJjlsQOLd

— Jamie Gaynor (@geomodler) August 17, 2017

Stallman had a great, great quote

"when you have a white supremacist bully in the White House freedom is never safe" Stallman #FOSS4g

— (((Steven Feldman))) (@StevenFeldman) August 17, 2017

#foss4g getting the full Stallman treatment, putting lotion on the feet & Saint InGNUTius of the eMacs church pic.twitter.com/J3hTFptFcD

— Joshua S Campbell (@disruptivegeo) August 17, 2017

  • “State of QGIS” was the annual round up of new features and plans for QGIS. There will be masses of goodies in QGIS 3, better, faster, more stable, watch the video as soon as it is up.
  • “Towards an improved metadata management in QGIS: Vision and Roadmap” by Joana Simoes, GeoCat – exciting stuff coming integrating metadata across QGIS and GeoNetwork. One for the metadata geeks to watch. On which note my favourite metadata slide

Meta-cat as a metaphor for metadata

Categories: OSGeo Planet

Fernando Quadro: Configurações de segurança no GeoServer – Parte 4

OSGeo Planet - Thu, 2017-08-24 10:30

Nos últimos posts (parte 1, parte 2 e parte 3) realizamos algumas configurações no módulo de segurança do GeoServer. Mas, como garantir que você realizou todos os passos corretamente?

Existem alguns testes diferentes para garantir que as regras foram implementadas corretamente, veja:

1. O acesso anônimo

Para testar o acesso anônimo, sair do GeoServer e executar os seguintes testes:

2. Acesso com usuário public_editor

Para testar o acesso via usuário public_editor, faça o login como esse usuário e executar os seguintes testes:

3. Acesso com usuário private_viewer

Para testar o acesso via usuário private_viewer, faça o login como esse usuário e executar os seguintes testes:

4. Acesso com usuário private_editor

Para testar o acesso via usuário private_editor, faça o login como esse usuário e executar os seguintes testes:

Além disso, a conta de administrador deve ser capaz de realizar todos os testes acima referidos.

Este post é uma tradução livre do tutorial “Working with multiple GeoServer users” escrito originalmente pela equipe da Boundless.

Categories: OSGeo Planet
Syndicate content