Here’s the full list. Please apply and/or spread the word!

The sprint wiki page lays out some of the topics on the agenda: routing, parallel processing, 3D, point clouds, performance, and of course the ever-present meta-topic, git.  It seems likely that PostGIS will move to git in the near future unless one of the development team is strongly opposed. Being merely ambivalent, I won’t throw myself into the gears of “progress”.

My personal development priorities for 2.1 are performance, performance, performance: I have speed improvements in mind for geometry, geography and raster. The geometry and geography improvements are based on internal indexing, as I described almost three years ago! They were too radical to go into 1.5, which was nearing release at the time, and other things (re-writing the core serialization/parsers/emitters) distracted me during 2.0. So 2.1 is it! And for raster, someone might beat me to it, but the ST_Union(raster) function should be relatively easy to speed up with the same memory management approach that the vector aggregate functions use.

This evening the developers will be meeting with a local group of power users for a discussion about roadmap priorities and where they think our development effort would be best spent. I’m looking forward to hearing some more about how folks are using PostGIS in the wild.

À bientôt!

 This is a bug fix release, addressing issues that have been
 filed since the 1.5.3 release.

 - Bug Fixes
   - #547, ST_Contains memory problems (Sandro Santilli)
   - #621, Problem finding intersections with geography (Paul Ramsey)
   - #627, PostGIS/PostgreSQL process die on invalid geometry (Paul Ramsey)
   - #810, Increase accuracy of area calculation (Paul Ramsey)
   - #852, improve spatial predicates robustness (Sandro Santilli, Nicklas Avén)
   - #877, ST_Estimated_Extent returns NULL on empty tables (Sandro Santilli)
   - #1028, ST_AsSVG kills whole postgres server when fails (Paul Ramsey)
   - #1056, Fix boxes of arcs and circle stroking code (Paul Ramsey)
   - #1135, improve testsuite predictability (Andreas 'ads' Scherbaum)
   - #1146, images generator crashes (bronaugh)
   - #1170, North Pole intersection fails (Paul Ramsey)
   - #1179, ST_AsText crash with bad value (kjurka)
   - #1184, honour DESTDIR in documentation Makefile (Bryce L Nordgren)
   - #1227, server crash on invalid GML
   - #1252, SRID appearing in WKT (Paul Ramsey)
   - #1264, st_dwithin(g, g, 0) doesn't work (Paul Ramsey)
   - #1344, allow exporting tables with invalid geometries (Sandro Santilli)
   - #1389, wrong proj4text for SRID 31300 and 31370 (Paul Ramsey)
   - #1406, shp2pgsql crashes when loading into geography (Sandro Santilli)
   - #1595, fixed SRID redundancy in ST_Line_SubString (Sandro Santilli)
   - #1596, check SRID in UpdateGeometrySRID (Mike Toews, Sandro Santilli)
   - #1602, fix ST_Polygonize to retain Z (Sandro Santilli)
   - #1697, fix crash with EMPTY entries in GiST index (Paul Ramsey)
   - #1772, fix ST_Line_Locate_Point with collapsed input (Sandro Santilli)
   - #1799, Protect ST_Segmentize from max_length=0  (Sandro Santilli)
   - Alter parameter order in 900913 (Paul Ramsey)
   - Support builds with "gmake" (Greg Troxel)

This recent ticket took a little while to visualize and confirm, and it’s worth showing so hopefully other folks can debug their issues on their own. The submitter basically said, “look, I have two squares, and I know the smaller one is contained in the larger, and I know that they both contain a marker, and yet when I run containment tests, they say the marker is only contained by the smaller square, but if the larger square contains the smaller it must logically also contain the marker!” And, helpfully, he included the SQL for the tests, which included the squares.

You can see where his concern comes from. The smaller square is inside the larger.

And the marker is inside both.

However, this is a visualization in mercator. The lines between vertices are being drawn as linear interpolations in planar space. The geography calculations involve great circles between vertices in spherical space. What does that look like?

Hm, the lower lines don’t actually line up.

And the marker is actually between the lower lines. It’s contained by the smaller square, but not by the larger one. Just like the geography tests said.

The moral of the story is, we aren’t good at visually reasoning about boundaries on a sphere. Our practice is all cartesian and it causes us to make mistakes.

This is something I learned during the development of geography, when I spent many hours debugging test cases that the code was getting “wrong” only to find in the end that the problem was entirely between my ears. (Example, does POINT(0 1) fall on LINESTRING(-1 1, 1 1)? If you said “yes”, you too have problems between your ears.)

So, plug your problem into KML LineStrings (not Polygons, they aren’t rendered as great circles!) and have a look before reporting correctness issues in geography, please!

SELECT status, content_type, content FROM http_get('http://localhost');

 status | content_type |                   content
--------+--------------+----------------------------------------------
    200 | text/html    | <html><body><h1>It works!</h1></body></html>
(1 row)

What good is it? Well, the first non-silly use case I came up with was setting up a geocoding hook so that you can call a geocoding web service as new addresses are added to a table, and this morning I thought I would write it up to see what it looked like.

The Google Geocoding API takes in URLs that look like this:

http://maps.googleapis.com/maps/api/geocode/xml?sensor=false&address=1234+main+st+chicago

And returns results that look like this:

<?xml version="1.0" encoding="UTF-8"?>
<GeocodeResponse>
 <status>OK</status>
 <result>
  <type>street_address</type>
  <formatted_address>1234 Main St, Evanston, IL 60202, USA</formatted_address>
  <address_component>
   <long_name>1234</long_name>
   <short_name>1234</short_name>
   <type>street_number</type>
  </address_component>
  <address_component>
   <long_name>Main St</long_name>
   <short_name>Main St</short_name>
   <type>route</type>
  </address_component>
  ......
  <geometry>
   <location>
    <lat>41.6372458</lat>
    <lng>-87.5860067</lng>
   </location>
  </geometry>
  <partial_match>true</partial_match>
 </result>
</GeocodeResponse>

The HTTP library can get us the content, and we just need to extract the coordinates. There are all sorts of interesting text processing functions in PostgreSQL, including a full regular expressions system, but for maximum nerdiness it makes sense to process the XML document with the native PostgreSQL XML support (you will need to have your PostgreSQL compiled with XML support for this to work).

Here’s a function that takes in a street address, calls the geocoder, strips out the coordinates and constructs a PostGIS geometry return value.

CREATE OR REPLACE FUNCTION address_to_geom(text)
RETURNS geometry
AS
$$
WITH http AS (
  SELECT status, content_type, content::xml
  FROM http_get('https://maps.googleapis.com/maps/api/geocode/xml?sensor=false&address=' || replace($1, ' ', '+'))
),
parts AS (
  SELECT
    status, content_type,
    (xpath('//location/lat/text()',content))[1] AS lat,
    (xpath('//location/lng/text()',content))[1] AS lng
  FROM http
)
SELECT
  ST_SetSRID(ST_MakePoint(lng::text::float8, lat::text::float8),4326) AS geom
FROM parts

$$
LANGUAGE 'SQL';

I’m using my new favorite PostgreSQL syntactic sugar, the “WITH” keyword, to order the subqueries from top to bottom so you can read them serially.

First we run the HTTP call, lightly URL-encoding the input address before appending it to the base URL. (Future enhancement, do a real URL encoding.)

Then we extract the coordinates from the XML, using an XPath query! Note that we’re only extracting the first result, but with XPath we could extract them all into arrays if we wished. (Future enhancement, watch the return status and check for web service error conditions at this step.)

Finally we convert the coordinates into a geometry, casting the coordinates from XML to text to float, and then setting the SRID to 4326, since Google only works in WGS84.

Now that we have the function, writing a trigger is trivial.

CREATE TABLE addresses (
  id SERIAL PRIMARY KEY,
  address VARCHAR,
  geom GEOMETRY(Point, 4326)
);

CREATE OR REPLACE FUNCTION update_address_geometry()
RETURNS TRIGGER AS
$$
  BEGIN
  NEW.geom = address_to_geom(NEW.address);
  RETURN NEW;
  END;
$$
LANGUAGE plpgsql;

CREATE TRIGGER address_trigger
  BEFORE UPDATE OR INSERT
  ON addresses
  FOR EACH ROW EXECUTE PROCEDURE update_address_geometry();

Now every time an address is inserted or updated, the associated geometry will be looked up in the geocoding service automagically.  What could possibly go wrong? Well, if the Google web service is slow, each insert and update will take as long as the web service, which would not be good in a high-volume environment, to say the least. And as noted above, this example includes no error handling at all. However, it’s a cute trick and I’m sure there will be more to come.

The reason I wrote this little library is to allow PostgreSQL and PostGIS to talk to GeoWebCache directly, so that edits at the database level can inform the caching layer that the cache needs to be updated where the edits have occurred. This is important for any system that uses cached tiles on top of live data. This update will be in the OpenGeo Suite 3.X series, which is coming later this year.

What’s a harmonized layer name? Simply put it means that the WMS needs to advertise its layer according to a pre-defined name which is listed  in the INSPIRE data specifications. If you were to look at table 19 of the data specification for transport networks you’d see that there is a layer which should be named “TN.CommonTransportElements.TransportNode”. Historically you were never able to achieve a harmonized layer name because GeoServer outputs the namespace prefix in front of the user defined name.

To remedy this OpenGeo utilized work done on GeoServer virtual services. The namespace prefix on a virtual service does not make layer names unique and was dropped. For technical details see  JIRA issue 5016 and please note that this work is only available in the GeoServer 2.2 release.

Our thanks to our client Ordnance Survey for supporting this enhancement, and to Juan Marin and Justin Deoliveira for the development work.

There are a lot of user-visible changes, but it’s also hard to overstate how much changed under the covers: the storage format, the indexes, the parsers and emitters were all re-written. That is, all the code that formed the initial release of PostGIS back in 2001 was swapped out. It’s all fresh and shiny under the hood. Here’s the official release announcement:

The PostGIS development team is super excited,
can hardly believe that they are actually doing this,
aren't maybe even sure that they are ready to make
this kind of commitment, not so young, and not when
we have so much more living to do, but:

PostGIS 2.0.0 is complete and available for download.

http://postgis.org/download/postgis-2.0.0.tar.gz

The development process for 2.0 has been very long,
but has resulted in a release with a number of exciting
new features.

 * Raster data and raster/vector analysis in the database
 * Topological models to handle objects with shared boundaries
 * PostgreSQL typmod integration, for an automagical
   geometry_columns table
 * 3D and 4D indexing
 * Index-based high performance nearest-neighbour searching
 * Many more vector functions including
   * ST_Split
   * ST_Node
   * ST_MakeValid
   * ST_OffsetCurve
   * ST_ConcaveHull
   * ST_AsX3D
   * ST_GeomFromGeoJSON
   * ST_3DDistance
 * Integration with the PostgreSQL 9.1 extension system
 * Improved commandline shapefile loader/dumper
 * Multi-file import support in the shapefile GUI
 * Multi-table export support in the shapefile GUI
 * A geo-coder optimized for free US Census
   TIGER (2010) data

We are greatly indebted to our large community of beta testers
who valiantly tested PostGIS 2.0.0 and reported bugs so we could
squash them before release time.

And also we want to thank our parents for making PostGIS possible.

Yours,

The PostGIS development team

In addition to all my colleagues on the PostGIS team, I’d like to also thank OpenGeo, who have given me the time to work on PostGIS over the past couple years.
 

This project has gained national attention and put the SFMTA in the spotlight as a forward looking innovator aiming to improve the status quo – values that OpenGeo respects and shares. The SFMTA has been a great client and we look forward to our continued partnership.

• Points are never invalid. How can you screw up a point?
• Linestrings are almost never invalid. Only a two-point line where the start and end are the same is invalid.
• Polygons have lots of ways to be invalid. Rings have to close. Rings can’t self-intersect. Rings can only touch other rings once, at a point. Inner rings have to be inside outer rings.
• Polyhedralsurfaces… I don’t have enough space.  Imagine a face defined as a 3D polygon and you get the picture. All the rules of polygons, plus rules like requiring all points on all rings to be co-planar.

My favorite invalid polygon (doesn’t everyone have a favorite invalid polygon?) is the figure-eight, because it’s so obviously wrong, and because it demonstrates the importance of validity rules so elegantly.

POLYGON((-1 -1, -1 0, 1 0, 1 1, 0 1, 0 -1, -1 -1))

Want to see why validity is important? Run ST_Area() on this polygon. The answer? Zero. The area algorithm expects that rings bound areas using a consistent orientation of edges to the bounded area. But you can see in the figure-eight diagram above, one lobe is bounded by clockwise edges and the other lobe by anti-clockwise edges. So the calculation looks like this:

The way to represent this area validly is with a MultiPolygon consisting of two one-unit squares, so, run ST_MakeValid() and see what comes out!

SELECT ST_AsText(ST_MakeValid('POLYGON((-1 -1, -1 0, 1 0, 1 1, 0 1, 0 -1, -1 -1))'))
MULTIPOLYGON (((-1 -1,-1 0,0 0,0 -1,-1 -1)),((0 0,0 1,1 1,1 0,0 0)))

ST_MakeValid() will try to create a valid output without dropping vertices, so for some invalidities, the result is not exactly what one might like. For example, many hand digitized polygons have very small versions of the figure-eight case embedded in their edges. Users probably just want those little “polygon warts” to go away, but ST_MakeValid() will retain them, as tiny elements of a multipolygon. ST_MakeClean(), to remove small errors using a tolerance, awaits funding from some enterprising organization!

Check out the podcast and the transcript below or download on our Media Page .

Thanks to the host Rich Bowen and the SourceForge blog!

GeoServer on The Anvil Podcast: