OpenGeo

In preparing some data for our next round of training courses, I spent a fair amount of time today processing and cleaning the US Federal Electoral Commission (FEC) database for 2008. The FEC is extremely good about releasing their data, even though it looks like they have to dump it out of a very old database system.

I processed the three main files, and then converted the associated code tables into side tables, so the whole thing is pretty self-contained and hopefully self-explanatory. I had originally hoped to fully replicate something like the FundRace site from 2008, but since the FEC data only has zip-code as a location entity, that is not going to happen this time around. I assume the FundRace folks also had access to a nationwide telephone directory or some other way of taking name and zip-code and using that to leverage out an actual street address.

If you are interested in playing with the FEC data and don’t feel like spending a couple hours mucking about in Perl to get it into tables, I’ve placed a PostgreSQL dump file online. Candidates are linked to individuals via committees. The FEC model has a lot of complexity hiding in it, with some committees not associated with candidates, and so on, so using the data correctly will probably require a little care.

Update: The FundRace folks did in fact only use FEC data, the trick is, they used the original filings, rather than the FEC database dump. The original filings include a street address for each donation.

On Thursday, the latest major release of PostGIS came out: version 1.5.  This release adds a long-wished-for feature to the open source spatial database—direct support for “geodetic” coordinates.

Geodetics are more commonly known as “lat/lon” coordinates. While you could load and work with lat/lon coordinates in earlier versions of PostGIS, the indexing and calculation code did not make any allowance for the fact that the coordinates were angular units, not cartesian units. As a result, objects that crossed the poles or datelines would not index properly, and calculations of areas, lengths and distances returned strange looking answers in “degrees” rather than meters.

With PostGIS 1.5, the new “geography” type is a 100% sphere-aware type, which can be indexed globally and returns answers in meters, using calculations on the spheroid for maximum correctness. It is built on top of a new disk storage and index format, which the existing “geometry” type will also transition to in version 2.0.

The development of the “geography” type was funded as a PostGIS core development task, by a company that chooses to remain anonymous.  Since the main geography development is complete the old task has been updated to a new version, outlining extra functions and performance additions that could be added to geography support.

We expect that the geography type will make it easier for new users to store their data in PostGIS (without having to learn about projections and coordinate systems before starting) and also allow global data managers to store and query international data sets for effectively.

The US National Weather Service has a lot of weather to watch! They have sensors all over the continent, and in space, ranging from simple thermometers to orbiting satellites. When weather happens (and let’s be honest, weather is always happening, it’s happening on me right now) it happens fast — how does the NWS take in the whole situation at a glance and make decisions? On a map, of course.

The NWS Central Region Headquarters has built a test-bed for putting their situational information onto web maps, and the toolset they used is the OpenGeo suite of applications: PostGIS, Geoserver, OpenLayers. They also used raster-data standby GDAL for handling gridded data conversions.

A paper on their work (”Development of Web-based GIS Applications for Decision Support and Situational Awareness“) was presented by Brian Walawender at the  American Meteorological Association Annual Meeting this week.

This week, OpenGeo released version 1.0 of our OpenGeo Suite.

When we initially announced the OpenGeo Suite, it was a notion — a collection of individual software pieces we would professionally support as a whole. Now, it’s a product in the conventional sense — one download that provides all the pieces in a simple installer for Windows, Mac, or Linux:

• GeoServer — a geospatial data and map server;
• GeoWebCache — a map accelerator;
• OpenLayers/GeoExt — user interface libraries for building map applications;
• (New!) GeoExplorer — a browser-based map composer and publisher;
• (New!) Styler — a WYSIWIG editor for map styles (SLD);
• (New!) Recipe Book — code samples and documentation for building your own map applications;
• Full documentation for all components; and
• (New!) Dashboard — a unified administration panel for starting and managing the components of the OpenGeo Suite.

Up to this point, we have concentrated on clients already adept at downloading, integrating, and using the pieces of the Suite. With version 1.0, anybody can start publishing their data and building applications right out of the box.

This ease of entry aligns with the OpenGeo mission.  As an organization, we want to democratize mapping. That means offering tools available under non-discriminatory legal terms, like open source. It also means lowering barriers so that more people can use, build, and grow these tools.

Both novice and expert benefit from the seamless integration of the OpenGeo Suite.  By placing everything together in one place — software, documentation, examples, administration — the Suite offers a central resource to navigate, configure, and support the various pieces of your mapping application.

At a decision making level, we provide a corporate entity tightly bound to the software, providing support, expertise and training services to the community of users and administrators.

Version 1.0 is the first step in a long journey, but we know where we are going. Every day we ask ourselves: can we make our product easier to use? can we make it easier to learn? can we make it easier to try? We would love your feedback, so download the free 30 day trial and let us know what you think!

We’re looking forward to an exciting 2010, meeting those goals and growing our community.

We are pleased to announce the launch of GeoNode.org, the official website of the GeoNode project.

The GeoNode project is a partnership between the World Bank’s CAPRA initiative, OpenGeo, and other organizations from around the world.  Our aim is to take the principles and practices of openness which have empowered the modern web, and use them to build a spatial data infrastructure solution appropriate for large NGO’s and government agencies.

The GeoNode is both an organizational partnership and an open source software project, built on the familiar projects of our OpenGeo Stack (GeoServer, OpenLayers, GeoExt, etc.) as well as Django and GeoNetwork.  After several months of incubation and prototyping, GeoNode.org is the kick off to our new focus on the GeoNode community.  Look there for the latest news on GeoNode technology, partnerships, and community resources.

GEOS, the geometry engine underneath the PostGIS spatial database (part of the OpenGeo Suite), has achieved a version 3.2.0 release! The latest release includes performance improvements in buffering, general C++ performance improvements, and an implementation of single-sided buffering. PostGIS users who upgrade their GEOS will get all the performance improvements automatically. The upcoming release of PostGIS 1.5 (about which we are very excited) will also tie in support for the new single-sided buffers.

It’s not just stand-up comedians wondering that anymore! In the brave new world of “serverizing” (nee Geoweb (nee SDI)) being promoted by ESRI’s Jack Dangermond (and us, and many others) the health of your server could directly impact the health of real live people.

When you stand up a public server with useful services, people will start using that server, and eventually will expect it to be there when they need it. And the better you are at keeping your server up, the stronger that assumption of reliability will become! At the highest levels of expected reliability, outages become newsworthy events in and of themselves.

So, to run a good service, choose good, reliable software — and then don’t trust it! The FGDC has recognized that monitoring is a key to providing reliable SDI services, and has stood up a public system for checking the health of spatial services, the Service Status Checker. As an added bonus, it’s not just for feds, anyone can use it, so give it a try!

One of the items we launched with our new web site this spring was what we have been internally calling “the menu”, and ended up calling “core development“. The premise is that a generation of proprietary software experiences have broken customers of the idea that they can directly pay a vendor for a new feature — as customers we’ve been trained to just wait until the next version and hope.  But in an open source world, developers (us) are happy to work on new features directly for customers. So in our core development “menu” we try to provide customers with some guidance about what is possible, writing up some descriptions of larger development pieces and enumerating the functionality they would provide.

One of the items I put in my PostGIS menu last spring was “geodetic types“, native support for latitude/longitude coordinates that allows for indexing of features that cross the poles or dateline, provides direct calculation of distances and areas on the spheroid, and integrates with the other functions in PostGIS.  And a few months ago, that menu item was funded by a client!  We are currently approaching the final delivery date, the code is committed to the PostGIS SVN repository, and I’m spending the rest of the week testing and polishing.

Amazingly, the open source development started paying off almost immediately — I was getting testing and bug reports from third parties very early in the process, which means the final delivery will be that much stronger for the client. I’ve also added a large number of functions above and beyond those itemized in the contract terms, since this code is going to be in wide use as soon as it is released.

To get a feel for the functions that have been added, check out the documentation for the upcoming PostGIS release. For more technical details on using the new type, see this post.

Once again, OpenGeo is participating in the popular annual WMS benchmarking tests that are presented at FOSS4G. For the last couple years (2005, 2007, 2008) MapServer and Geoserver have been tested, against PostGIS, Shape file, and image file data sources. This year, in addition to the regular components, there are some new and exciting additions: ESRI is participating with ArcGIS Server, and the list of data sources has been expanded to include Oracle Spatial and ArcSDE-on-Oracle.

Our own Andrea Aime is coordinating the Geoserver set-up and configuration, as well as providing expertise in benchmarking that he gained working on the tests in previous years.

OpenGeo’s had a busy month as we have been ramping up to drive our technology and business in exciting new directions. Here is a quick preview of what is in store for us:

The OpenGeo Suite Installer. In an effort to bring open source geospatial software to new markets, we are unifying the OpenGeo Suite into one tight package. Soon we will be releasing a distribution of PostGIS, GeoServer, and GeoWebCache with a easy installer, an improved data importing interface, and an integrated Styler application based on Openlayers and GeoExt.

The GeoNode. In collaboration with the World Bank’s Central American Probabilistic Risk Assessment (CAPRA), we have been building the next generation of SDI technology with modern web principles in mind. This summer we have built a simple data clearing house with GeoServer and GeoExt. Soon we will extend it with reporting tools useful for disaster risk management experts in the field, as well as uploading and styling workflows. Keep an eye out here for updates on the GeoNode’s technical and community development.

Training and FOSS4G. We are improving our training curriculum and giving several workshops on our stack. We will have a strong presence at FOSS4G this year, and are eager to spread the news and knowledge of our latest open source developments, like GeoExt, to the rest of the open source geospatial community.