Blog

Network Walking in PostGIS

One of the new features in PostgreSQL 8.4 was the “WITH RECURSIVE” clause available for queries. It allows you to define a subquery based on a recursive term — fancy language for a function that calls itself. One of the favorite uses of recursion is walking a network. Geospatial applications use networks all the time: electrical grids, stream systems, and storm sewers are all directed networks (they have unidirectional flow).

Here’s an example of network walking using a simple collection of segments. As is common in many GIS applications, the segment are implicitly connected — their end points are coincident with the start points of other segments.

CREATE TABLE network ( segment geometry, id integer primary key );

INSERT INTO network VALUES ('LINESTRING(1 1, 0 0)', 1);
INSERT INTO network VALUES ('LINESTRING(2 1, 1 1)', 2);
INSERT INTO network VALUES ('LINESTRING(1 2, 1 1)', 3);
INSERT INTO network VALUES ('LINESTRING(3 1, 2 1)', 4);
INSERT INTO network VALUES ('LINESTRING(3 2, 2 1)', 5);
INSERT INTO network VALUES ('LINESTRING(2 3, 1 2)', 6);
INSERT INTO network VALUES ('LINESTRING(1 3, 1 2)', 7);
INSERT INTO network VALUES ('LINESTRING(4 2, 3 2)', 8);
INSERT INTO network VALUES ('LINESTRING(3 4, 2 3)', 9);
INSERT INTO network VALUES ('LINESTRING(2 4, 2 3)', 10);
INSERT INTO network VALUES ('LINESTRING(1 4, 1 3)', 11);
INSERT INTO network VALUES ('LINESTRING(4 3, 4 2)', 12);
INSERT INTO network VALUES ('LINESTRING(4 4, 3 4)', 13);

CREATE INDEX network_gix ON network USING GIST (segment);

Visually, the network looks like this:

Network

To walk our network, use a WITH clause that starts with one segment, then repeatedly adds the next downstream segment to the collection. In our case, the “next downstream segment” is defined as a segment whose start point is close to the end point of the current segment. We’ll walk down from segment 6.

WITH RECURSIVE walk_network(id, segment) AS (
    SELECT id, segment FROM network WHERE id = 6
  UNION ALL
    SELECT n.id, n.segment
    FROM network n, walk_network w
    WHERE ST_DWithin(ST_EndPoint(w.segment),ST_StartPoint(n.segment),0.01)
  )
SELECT id
FROM walk_network

Which returns:

 id
----
  6
  3
  1
(3 rows)

From 6 to 3 to 1, correct! Once we have our walker producing the results we want, we can wrap more PostGIS and PostgreSQL functions around the walker to produce a finished product. Here’s a function that takes in an edge identifier and outputs a linestring based on the downstream path.

CREATE OR REPLACE FUNCTION downstream(integer)
RETURNS geometry
LANGUAGE sql
AS '
WITH RECURSIVE walk_network(id, segment) AS (
    SELECT id, segment FROM network WHERE id = $1
  UNION ALL
    SELECT n.id, n.segment
    FROM network n, walk_network w
    WHERE ST_DWithin(ST_EndPoint(w.segment),ST_StartPoint(n.segment),0.01)
  )
SELECT ST_MakeLine(ST_EndPoint(segment))
FROM walk_network
' IMMUTABLE;

And here’s the function in action, generating the downstream path from segment 12.

=# SELECT ST_AsText(Downstream(12));

            st_astext
---------------------------------
 LINESTRING(4 2,3 2,2 1,1 1,0 0)
(1 row)

Check the generated path against our network picture — looks good!

Path 12

OpenGeo Suite Community Edition 2.1.0

Last month, we released the OpenGeo Suite Enterprise Edition 2.0. Today we bring Community Edition users up to date with the latest 2.0 features, as well as a number of experimental new ones, with the OpenGeo Suite Community Edition 2.1.0.

GeoExplorer now includes a printing feature. With a simple export to PDF you can now impress your offline friends and your mom’s refrigerator. Another GeoExplorer improvement includes version 3 of the Google Maps API.  You no longer need an API key in order to host maps with Google layers on your site. And for design aficionados, Styler now preserves GeoServer SLD vendor options. We also fixed bugs around GeoWebCache layer groups and GeoServer layer names.

With 2.1.0 there is a lot more than meets the eye. We invite you to try it out and let us know what you think!

You can find answers to your questions at our growing OpenGeo Suite Community Forum.  Remember, the Community Edition is free (open source and no charge), but unsupported.  If you’re going into production, you’ll definitely want the OpenGeo Suite Enterprise Edition.

Unlimited bug fixes: The shocking truth

In conversations with our OpenGeo Suite Enterprise Edition clients, we have more than once heard the following lament:

It’s too bad that we don’t have any hours in our contract, because we’d really like this particular bug fixed.

Luckily for them, this is a simple misunderstanding between core service (”implementation”) hours, and bug fixing hours.

Core service hours are billed time towards strategically improving the capabilities of the OpenGeo Suite in the direction you desire. This includes, but isn’t limited to, items on our roadmap (such as WMS 1.3 required for INSPIRE compliance, extra dimensionality in PostGIS, and offline editing in OpenLayers). When you sign up for an Enterprise Strategic package, this comes with 350 implementation hours that you can use as you please. Currently, this is the only package that comes with implementation hours.

Bug fixes are a different matter, however. We realize that bugs happen, so bug fixes are in everyone’s best interest. That is why we offer unlimited bug fixes on every one of our Enterprise Edition packages. Unlimited. We mean it. If you’re an Enterprise Edition client and you find a bug in our software, let us know and we’ll fix it, no questions asked (well, except for maybe a log file and the steps required to reproduce the bug).

So if you’re wondering why you should become an OpenGeo Suite Enterprise Edition client, you can add the peace of mind that comes with constantly improving software. And now you are free to lament about other things.

Tulips, Pancakes and Open Source Mapping

“What are the Netherlands known for, Alex?”

OK, maybe football too (I guess we’ll find out on Sunday). In the meanwhile, take a look at these recommendations for the Netherlands’ national SDI (PDOK, which is “Publieke Dienstverlening op de Kaart”, or “Mapping Services for the Public” in English).

On the policy side, in Section 2, the document notes “Dutch public administrations need to provide explanations if they do not use open standards and if they decide against using open source software when making procurement decisions”, but then goes on to provide two direct operational reasons for going with open source:

1. PDOK is foreseen to become a relatively large system. It has to support hundreds of data services some of which may experience very high demands. In order to address these high performance demands a great number of servers may be needed (with a load balancing mechanism). Proprietary software is often licensed by number of cores/cpu’s which are used by the software simultaneously. A highly scalable system may therefore be confronted with high license costs.

2. Open source software components will not provide all functionality which is required by PDOK, but this will also be the case with proprietary software. In case of open source there are more means to exert influence on the directions of future developments. PDOK may also contribute to the development of open source components. These developments will then directly be beneficial to PDOK and its partners but also to the entire (international) community which makes a good case for spending public funds.

Lots of foresight there. The unit cost of any enhancements that open source components require to meet the functional needs of the system will be spread out over every server deployed in every country in Europe that follows a similar architecture.

In Section 4, we get to software recommendations.

Geoservices system components

First, a nice familiar decomposition of the web mapping architecture, which provides lots of flexibility in choosing components. And what components do they recommend for PDOK? I’m glad you asked!

From the point of view of an enterprise GIS system, GeoServer seems to be the most promising solution.

GeoServer includes a version of GeoWebCache. This is a proxy server between a map client and a map server.

A PostgreSQL database requires limited resources and can easily be implemented in a virtual environment. The advantage of this is that the virtualisation platform can be used to scale the database … For that reason, PostgreSQL is the database for PDOK; it is open source and it integrates very well with GeoServer

OpenLayers (http://openlayers.org/) seems to be the most suitable library for client-side webmapping for the requirements. OpenLayers is an open source web mapping client library and has many required features out of the box, supporting the required formats and much more.

The GeoExt project combines ExtJS with OpenLayers already, which may be useful depending on the extra client-functionality needed.

We have been working hard over the past couple years to make sure all these components of the geospatial stack play will together, and more recently selling a bundled package of components, but it’s great to get the external validation of a big organization looking at all the parts and saying “these work great together, we should use them”.

I can’t wait to open up the Amsterdam office!

Social goals are (more) motivating

I work for a purpose-driven organization, the social purpose of the organization aligns with my own social purposes, and the work is challenging and sophisticated.

Open source fits the profile of an intrinsically motivating pursuit (for folks of the right technical bent) exceedingly well. A great talk and visual presentation.

OpenGeo Suite 2.0 released

og-intro-eval

Since our first release of the OpenGeo Suite in January, our team has been working consistently to improve and refine the software. Much of this was based on feedback from our users, leading us to create a Community Forum where users can share their ideas and solutions. We have been periodically releasing a free Community Edition of the OpenGeo Suite, both as a testing environment for new features, and as a way for users to get the very latest in our technology.

Today, we release version 2.0 of the OpenGeo Suite Enterprise Edition. Here are just some of the many new features that comprise this software:

  • Fully integrated PostGIS, including the new pgShapeLoader, a graphical shapefile loading utility.
  • A new application, GeoEditor, that allows for web-based editing of geospatial features.
  • New code samples and recipes to help you design your own mapping applications.
  • Individualized packages of each application to integrate with an existing IT infrastructure.

And of course, all components are updated to the most recent stable version: PostGIS 1.5.1, GeoServer 2.0.2, GeoWebCache 1.2.2, and GeoExt 0.7.

With the OpenGeo Suite, you have all you need at your disposal to publish your data and maps on the web, as well as create virtually any web-based geospatial application for your organization. We hope that the new version 2.0 of the OpenGeo Suite will make your work even easier.

Register now to get your copy of the OpenGeo Suite 2.0. And please let us know what you think.

Our First Reseller

Not as exciting as our first kiss, but still pretty exciting: our first OpenGeo re-seller is Spatialytics in Quebec. That means that users of the OpenGeo Suite in Quebec will be able to receive front-line support from local staff who (importantly) speak French (with the right accent)!

We expect resellers to form the backbone of the ecosystem around the OpenGeo Suite. In a world with 40 time zones and hundreds of languages there is no way one organization can be everywhere. We want to focus as much of our internal effort as possible on improving the Suite, the core Suite projects, and the learning materials that go with them. Partners will be the bridge between our core team and customers around the world.

Welcome Spatialytics, to the OpenGeo family!

(Want to explore becoming a re-seller? Drop us a line, inquiry at opengeo dot org.)

Prj2EPSG

Once upon a time, a group of smart people got together to define a common standards base for geographic map services, a “Web Map Service” specification, if you will.

They wanted their map services to be interoperable, but different maps can be rendered using different projections, and in order to overlay one map onto another, they needed to know (and advertise) the projections of both.

There was an existing standard for representing map projections, called “well-known text” (which is also, confusingly, the name that describes a standard for representing geometries) but it was quite verbose. Who, after all, could remember this:

GEOGCS["WGS 84",
    DATUM["WGS_1984",
        SPHEROID["WGS 84",6378137,298.257223563,
            AUTHORITY["EPSG","7030"]],
        AUTHORITY["EPSG","6326"]],
    PRIMEM["Greenwich",0,
        AUTHORITY["EPSG","8901"]],
    UNIT["degree",0.01745329251994328,
        AUTHORITY["EPSG","9122"]],
    AUTHORITY["EPSG","4326"]]

More importantly, how would this fit cleanly into a URL?

Fortunately, there already existed a large database of commonly used map projections: the EPSG database. This provided a single numeric ID for each common map projection. So it was decided that all map services must advertise their projection using a unique number defined by an authority, and set EPSG as the first authority. And so ESPG:4326 came into the world (WGS84 geographic coordinates) along with ESPG:26910 (NAD83 UTM Zone 10 North), and many others.

But, unlike me, most GIS practitioners haven’t memorized the EPSG database. So they frequently ask questions like “what is the EPSG number for Oregon State-Plane South?” and “how do I find the EPSG number for this shapefile?” One could search spatialreference.org, a site for understanding spatial reference systems. My own answer used to be a fairly unhelpful set of directions for doing a text search of the PostGIS SPATIAL_REF_SYS table:

SELECT srid, srtext FROM spatial_ref_sys WHERE srtext ILIKE '%oregon%';

But today, I can provide a much simpler answer: Use prj2epsg.org. With prj2epsg.org, you can paste in full well-known text descriptions, you can type in shorter keyword searches, and you can even read a .prj file directly.

This free public service is provided by OpenGeo and our cloud services provider SkyGone. The code is naturally all open source and the service is built on top of the same GeoTools library that is at the heart of our OpenGeo Suite.

And now we’re all hopefully one step closer to living happily ever after.

OpenGeo Suite 1.9.1 Community Edition

We are excited to announce the OpenGeo Suite 1.9.1 Community Edition, the latest maintenance release of our leading geospatial platform.

The 1.9.1 Edition adds stability to our previous community release, featuring:

  • PostGIS is now integrated into the OpenGeo Suite install experience! You now get a complete spatial mapping stack: database, map server, tile cache, web interface.
  • pgShapeLoader, a graphical interface for loading shapefiles into your PostGIS database.
  • pgAdmin, a graphical interface for database administration.
  • GeoEditor, a web application. GeoEditor allows for web based editing of data, regardless of the underlying data format (shapefiles, PostGIS, ArcSDE, Oracle Spatial, DB2, etc.).

The integrated installer is now available for Linux (32 bit only), as well as Mac OS X (v10.5 or later) and Windows XP/Vista/7. The Linux installer does not yet include PostGIS.

The OpenGeo Suite Community Editions are previews of future enterprise packages. Please download, contribute issues, and participate in driving the Suite’s future on our OpenGeo Suite Community Forum. And stay tuned for the Enterprise 2.0 release of the OpenGeo Suite this month.

Download OpenGeo Suite 1.9.1 Community Edition

What’s an OBIS?

It’s an Ocean Biogeographic Information System, that’s what it is! It’s also our newest customer for the OpenGeo Suite.

We visited the OBIS team at Rutgers University last year and found they were already planning a system using the components of the Suite: PostGIS, GeoServer, OpenLayers and others. As they move through implementation into production, their Suite contract will provide support for all the components. OBIS has decided to get their support publicly via our web community page (customers have the option of private lists too) so you can see the questions they are asking if you go have a look.