Case Study: Creating a Geospatial Web Service for National Geographic

National Geographic needed a geospatial web service to get data and display on a map. Celerity chose Ade Labs to help build this geospatial web service. The application would involve creating a web service with park location information so that the information could be plotted on a map. Ade Labs was in charge of building the Web Service/API.

Software Tools Used

The initial technologies that the client wanted to use were the following Hapijs, PostgreSQL. I also chose to use Postgis, Knexjs, Bookshelfjs to incorporate geospatial capabilities and an ORM into the backend/web service.
Here is a description of tools:

PostGIS
Postgis is a geospatial database extension of PostgreSQL. This technology is can be used to find the geometry, longitude, latitude, distance and other measures. Postgis has some features such as geometric types that are specific to geospatial databases. These measures can be used to location of an object, finding the near objects to a point, and other use cases. More detailed information about some of PostGIS’s geospatial features are explained here.

Mapbox

Mapbox is software used to create smart vector maps complete with instant rendering to your web or mobile app. Also, Mapbox has an API that allows developers to reverse geo code coordinates, get routing directions, get distance calcuations. Also, when creating a map, there are styles and tiles, and other features that make Mapbox an ideal Map API to use. More information about some of Mapbox’s API is explained here.

GeojSON

GeoJSON is a JSON based format for encoding geospatial data structures especially geometric objects that have additional properties. This format supports Point, LineString, Polygon, Multipoint, MultiLineString and MultiPolygon types.

GDAL

GDAL (Geospatial Data Abstraction Library ) is a Translator library for geospatial formats. GDAL is used to transform one format to another format. A well known GDAL tool to use that transforms files is the ogr2ogr functionality. It is known to transform shapefiles, geojson into a postgres tables and other formats.  Also, Postgis has a tool to load shapefiles to a postgres database called shp2pgsql that is similar but ogr2ogr. However it is believed here that ogr2ogr is the better tool.

Geospatial Web Service

Loading and Updating Data

The first step one was to load initial park data from National Geographic into the postgres database. When starting the National Geographic project, we were given geojson data that needed to be loaded to the Postgres database. We ended up using GDAL’s ogr2ogr functionality to load data into a database to create a table.

To import geojson to a database table:

ogr2ogr -f PostgreSQL PG:”dbname=dbname user=user password=password” parks.geojson -nln nationalgeographicdatabase

Once the geojson was loaded into the table, the dataset used had missing data. The dataset was missing street, city, state or zipcode. Also, the longitude and latitude was not a friendly geometric format. Postgis was then used to convert geometric measures of the parks to longitude and latitude values. Since the dataset was missing street, city, state or zipcode for parks, we used the Mapbox Geocoding API to reverse geocode the longitude and latitude of the parks data to get location information.

Reverse Geocoding

Reverse geocoding latitude and longitude coordinates as input. The response includes one result from each geocoding dataset.

Here is what the URL looks like:

https://api.mapbox.com/v4/geocode/mapbox.places/’+ xCoordinate+ ‘,’ + yCoordinate + ‘.json?access_token=<access token>

Here is an example use of the URL with the x-coordinate/longitude = -110.547 and y coordinate/latitude = 44.596:

https://api.mapbox.com/v4/geocode/mapbox.places/-110.547,44.596.json.json?access_token=<access token>

Here is sample of the response:

{“type”:”FeatureCollection”,”query”:[-110.547,44.596],”features”:[{“id”:”place.55914″,”type”:”Feature”,”text”:”Yellowstone National Park”,”place_name”:”Yellowstone National Park, Wyoming, United States”,”relevance”:1,”properties”:{},”bbox”:[-111.056889009999,44.1312909900081,-109.825486991003,45.0039050099999],”center”:[-110.83,44.46],”geometry”:{“type”:”Point”,”coordinates”:[-110.83,44.46]},”context”:[{“id”:”postcode.8237635196769770″,”text”:”82190″},{“id”:”region.10947505346724150″,”text”:”Wyoming”},{“id”:”country.5877825732302570″,”text”:”United States”,”short_code”:”us”}]}

To get the rest of the data, you will actually have to make the HTTP request in a browser or with a programming language. Limiting the input coordinates to at most 5 decimal places usually improves precision and performance of the reverse geocoding process. If you want more information about using the API to reverse geocode, you can go here.

After the location information was received the data was updated with that information in the postgres database.

Geospatial Web/Service API Methods

Once the data is in the database, the web service methods can be created so that the data can be displayed on the map. Inside the methods, sql statements are created to get the parks data to be displayed on the map.

Some of the web service functionality built was:

Items Search Method

The Items Search Method based sql statement to get all the item information by a search query parameter. The item information is returned in geojson format.

Geojson Format Support

The Postgis Extension has built-in GeoJSON format support using the ST_AsGeoJSON() method. This method can be used to addGeoJSON format can to be added in the sql query. The ST_AsGeoJSON() method formats a string into GeoJSON format in the sql query.

Here is an example PostgreSQL query with ST_AsGeoJSON method to format the result in GeoJSON:

SELECT ST_AsGeoJSON(geometry) as geometry   FROM points limit 1;

And here is the GeoJSON Query Result from a query with the ST_AsGeoJSON method:

{“type”:”Point”,”coordinates”:[-76.0301621537909,43.1440190508815]} 

Nearest Parks Method

The Nearest Parks Method to return a number of parks near a location given a point and other possible parameters. This functionality is based of a sql statement given the point and other possible parameters. The park information is returned in GeoJSON format.

PostGIS’s Nearest Neighbor Method

The basis for the PostGIS geographical type is a point. Given a geometry it aims to find the x number of nearest neighbors and n geometries of data. A  query with to find the nearest objects to a geometric value has the following clause, ORDER BY geometry_column <-> geometric value LIMIT k. <-> in the ORDER BY clause is a “distance” operator that when combined with ORDER BY clause and LIMIT k allows query to find the k nearest objects to a geometric value. We will focus on using a point, so geometric value = ST_MakePoint(double x , double y), where x is longitude and y is latitude.

Here is an example PostgreSQL query with ST_MakePoint method to get the k = 5 points nearest to the given point (-110.547348214819,44.5964236630063) in the database:

SELECT ST_AsGeoJSON(geometry) as geometry   FROM points ORDER BY ST_MakePoint(longitude,latitude) <-> ST_MakePoint(-110.547348214819,44.5964236630063) limit 5;

And here is the Nearest Neighbor Query Result from the query above :

{“type”:”Point”,”coordinates”:[-110.547348214819,44.5964236630063]} {“type”:”Point”,”coordinates”:[-110.709738503157,45.0298832732411]} {“type”:”Point”,”coordinates”:[-110.663536323832,44.0947376621328]} {“type”:”Point”,”coordinates”:[-110.728358647307,44.0885386251708]} {“type”:”Point”,”coordinates”:[-111.100823317053,44.6630655719391]}

If you are interested in discussing more on how to create a geospatial application, feel free to contact me here or email me at adetola@adelabs.com.