![[BACK]](/icons/back.gif){kind=icon}
Up to [cvs.NetBSD.org] / pkgsrc / geography / osm2pgsql
Request diff between arbitrary revisions
Keyword substitution: kv
Default branch: MAIN
geography/osm2pgsl: Fix distinfo from previous
geography/osm2pgsal: Update to 1.11.0 Upstream NEWS content: Release 1.11.0 This release makes the new middle database format the default. If you have not switched already, you need to reimport your database to take advantage of that. We have changed the way we are parsing the command line options. The new code uses the CLI11 library (a copy of which is included in the repository) and is much cleaner and also much stricter. You now get warnings (and sometimes errors) for many combinations of options that don't make sense. Please check the output from osm2pgsql and osm2pgsql-replication for such messages and fix your command lines accordingly. Note especially that duplicated options are not allowed any more. This can happen, for instance, when using osm2pgsql-replication which adds the database connection parameters (such as -d) when it calls osm2pgsql. Release 1.10.0 This is a relatively small but still important release. The new middle table format has changed slightly: the tags field can now be NULL. This makes storage more efficient and indexing faster. The new middle format is now declared stable and production ready. To use it, use the command line option --middle-database-format=new, in a future version of osm2pgsql this will become the new default. If you have used this option already with one of the 1.9.x versions of osm2pgsql you have to reload your database or use this SQL command to update the table: ALTER TABLE <name> ALTER COLUMN tags DROP NOT NULL;, for <name> use planet_osm_nodes, planet_osm_ways, and planet_osm_rels or the equivalents if you are using a different table name prefix.
geography/osm2pgsql: Update to 1.9.2
The old version was quite behind and stopped building. This version
packages and the main program runs; therefore it is better, even if
not tested.
Packaging changes:
Take MAINTAINERship (ok wiedi@).
Adjust dependencies to match upstream docs.
Hack in finding boost headers (should not be necessary).
Upstream NEWS content: (Much has changed and this summary omits much,
including bug-fix releases entirely. See
https://github.com/osm2pgsql-dev/osm2pgsql/releases for more details
-- there is no NEWS.)
Release 1.9.0
This release brings three new major features:
a new osm2pgsql_properties table that saves command line options and reuses them on updates
a new database middle saves raw OSM data in JSONB format and is explicitly designed to be queried by the user
the new (and still experimental) osm2pgsql-gen adds geometry generalization to osm2pgsql (thanks @joto)
Release 1.8.0
The largest change is the addition of much more flexible index support in the flex output. The table definitions have a new (optional) field called indexes now which takes a list of index definitions. If the field is not there, we fall back to what we did before and create a GIST index on the only/first geometry column of a table. But you can also define any kind of index you want: define which index method (BTREE, GIST, ...) to use on which columns, define WHERE clauses and expression indexes and much more. See the flex-config/indexes.lua Lua config for some usage examples and the manual for all the details. You can also force osm2pgsql to always build the id indexes which are normally only built in slim mode.
The gazetteer output and the command line option --with-forward-dependencies are deprecated in this release and will be removed soon. They were only needed for Nominatim which switched to using the flex output recently.
Here are the other changes:
Fix a problem when using osm2pgsql with a projection other than WGS84 (EPSG:4326) or Web Mercator (EPSG:3857) which made the program really slow.
New pole_of_inaccessibility() Lua function to generate reasonably good label points from polygons. (This function is currently marked as experimental, which means it can change without notice at any time.)
Release 1.7.0
For this version we rebuilt a lot of the code around geometry processing and around expire. The different parts -- creation of geometries from OSM data, transforming geometries (like merging and splitting linestrings) and finally writing them out in WKB format for import into the database -- are now well separated and tested on their own. And we added some functions for geometry processing, too. osm2pgsql can now calculate the centroid of a polygon and simplify linestrings using the Douglas-Peucker algorithm.
But the best part is that all of that new geometry goodness is now available from the Lua config files when using the flex output. There are many new ways of processing geometries from Lua:
The get_bbox() is now available for relations, too.
There are new functions as_point(), as_linestring(), as_polygon(), as_multilinestring(), as_multipolygon(), and as_geometrycollection() to create geometries from OSM objects.
Geometries can be manipulated in Lua with several functions modeled after the same functions in PostGIS: area(), centroid(), geometry_type(), line_merge(), num_geometries(), segmentize(), simplify(), srid(), and transform(). We expect more to come in the future. This way you can do more geometry processing on import removing the need for some post-processing in SQL.
We used to have the somewhat magic handling of geometries with the add_row() function which only allowed a limited set of operations. This function is still available for backwards compatibility, but there is a new function insert() now which doesn't have this magic. Instead geometries are treated like any other data type giving you a lot more flexiblity. Check out the example config files addresses.lua, generic.lua, simple.lua and geometries-using-insert.lua in the flex-config directory for some ideas on what can be done.
In this version we enabled the bucket index for way nodes by default. This had been around for a while but you needed a command line option to enable it. After some positive feedback from the community we decided to make this the new default. It will be used on new imports (existing databases will keep using the old index). The new bucket index is much smaller and can save you hundreds of gigabytes of disk space. See
https://osm2pgsql.org/doc/manual.html#bucket-index-for-slim-mode for the details.
Release 1.6.0
The osm2pgsql-replication script which has been included in the osm2pgsql repository for a while will now be installed together with its man page on "make install". To use it you need Python3, psycopg2 (or psycopg3), and PyOsmium installed. See the manual for details.
Release 1.5.0
This release brings quite a lot of improvements. We removed the "experimental" label from the flex output which we introduced in version 1.3.0. There are some small changes you might have to make to your flex configurations, see the Upgrading chapter of the manual for details.
This release also contains a rewrite of the code used to temporarily store OSM data in memory while processing the data in non-slim mode, i.e. when you import data without --slim. It now uses much less memory.
Release 1.4.0
The project has a new website at https://osm2pgsql.org now with extensive documentation and examples, and with sections on support, contributing, news, etc. Most of the documentation from the repository and the OSM wiki was moved there. We still have a man page, it is now maintained in markdown format. All the documentation, man page, help texts etc. have been cleaned up, made more consistent and brought up to date.
The program has a much improved log output now. Each line is prefixed with a date/timestamp and by default osm2pgsql isn't as verbose any more. You can change the verbosity using several options. You can even have super-verbose logging of all SQL commands issued and all data written to the database. Warnings and errors now appear in red color if your console supports it. Progress output can be disabled, for instance, when the output is redirected to a file. When printing how low something took, osm2pgsql will now not only print the seconds but also a more human readable format with hours, minutes, and seconds.
In the last release (version 1.3.0) we have already added a warning when you used input files with negative OSM object ids or input files which are not ordered correctly. These are now not allowed any more and osm2pgsql will stop with an error if it detects these. See the manual for how to work around this. This allowed us to improve the handling of multiple input files. Osm2pgsql now reads multiple input files at the same time merging the contents. This means that you can now import several extracts in one go. Note that the extracts still have to come from the same point in time!
Changes in the flex output (which is still marked experimental). Not that some of these are breaking changes compared to the behaviour in version 1.3.0:
Fix: Flex output sometimes created two id indexes on the same table.
Set projection for geometry columns in the table configuration. The command line options --latlong, -m, --merc, -E, and --proj are not used by the flex output any more.
Flex mode setting type_column fixed. Now also supports id columns compatible with Imposm.
Optionally wrap polygon geometries in multipolygons if the geometry column of the target table is of type MultiPolygon.
Switch multipolygon generation from default off to default on. The multi option on the area geometry transformation has been removed and there is a new option split_at.
Add several Lua helper functions for flex config files.
geography: Replace RMD160 checksums with BLAKE2s checksums All checksums have been double-checked against existing RMD160 and SHA512 hashes
geography: Remove SHA1 hashes for distfiles
geography/osm2pgsql: Update to 1.3.0
Upstream changes:
1.3.0:
This release introduces the new "flex" output. It allows a more flexible
definition of output tables and columns. It also adds a second stage of
processing which makes it possible to get information from relations to
their members, allowing, for instance, to render tags from bicycle route
relations on their member ways. The "flex" output is configured through
Lua scripts.
The flex output is currently still marked as experimental, because it is new
and we want to collect feedback from the community before finalizing the API.
But it already works well and users are encouraged to try it out. Some new
features are only or will only be available in the flex output and we expect
that it will replace the other outputs in the long term.
Some features have been marked as deprecated:
The "multi" output will be removed in a future version of osm2pgsql. If you
are using the multi output, switch to the flex output now and tell us if
you have any problems.
When the input file uses negative OSM object IDs a warning is now generated.
Negative IDs never worked correctly for all use cases. Future versions of
osm2pgsql will not allow negative IDs at all. Use "osmium renumber"
to get rid of the negative IDs.
Input files that are not ordered generate a warning. Future versions of
osm2pgsql will not work any more with unordered files. If you have unordered
files use "osmium sort" to order them.
Further changes:
The multi output now looks for lua script relative to the style.json file.
This is a breaking change. Users might have to change the file names of
their lua scripts in the style files.
Use the fmt library for formatting strings now instead of a mixture of
boost::format and hand-written mechanisms. A version of fmt is included
in the contrib directory.
Make PROJ library optional. If the proj library cannot be found by cmake,
do not offer the option to use arbitrary projections. Only WGS84 and
WebMercator are supported then.
Don't use ST_GeoHash for ordering tables by geometry on Postgis >= 2.4.
Instead use the default ordering which works better now.
Fix: Always print correct relations count and more correct count per seconds
when showing processing stats.
Fix: If a function run in the thread pool throws an exception, this exception
was never "collected", it was silently ignored. This meant that some errors,
especially in communication with the database, were not detected correctly.
The dependency management, the part of the code which tracks which changes
in the OSM data trigger which changes in the outputs, was reorganized
making in much cleaner and removing the last remnants of code written to
support "old style" multipolygons.
Tests have been moved to the Catch framework, extended and the regression
tests have been reorganised, so they can run independently of each other.
A lot of code was cleaned up, modernized, made more robust, and sometimes
removed.
1.2.2:
This release only updates the bundled version of libosmium. The new
version 2.15.6 fixes an issue where complicated multipolygons make
osm2pgsql hang.
geography/osm2pgsql: Update to 1.2.1 Upstream psudo-NEWS: 1.2.1 and 1.2.0 are bugfix releases. 1.0.0: This release finally drops support for old-style multipolygons. Doing so allowed a major overhaul of the processing pipeline. Imports are now entirely done in the first processing stage. The second, much slower processing stage is only needed when updating existing databases. Data is entirely streamed into the database using COPY, which reduces the number of database connections needed. Other major changes include - better error handling in Lua backend - process all OSM objects again when extra attributes are requested - enable running tests in pg_virtualenv - add support for configurable Gazetteer style - allow to disable RAM node cache with -C 0
osm2pgsql: Update to 0.96.0 Upstream does not provide NEWS, but reading the git commit messages from 0.94, this is mostly bugfixes and a number of efficiency improvements.
osm2pgsql: update to 0.94.0 Release 0.94.0, a new stable branch This is the last release which will support old-style multipolygons. Upgrading to this version requires a reimport for the slim table changes. Major changes since 0.92.0 are - Coordinates are now stored unprojected in slim tables, and an osmium dense file array is used for flat nodes. This dense file array can be read by other libosmium-based programs. - Libosmium is used for geometry building instead of GEOS This offers speed increases, improves code, and avoids relying on a large library for a small portion of its functionality. Other changes are - Tile expiry has been rewritten, fixing bugs, including one that dropped large portions of expiry lists. - Node tags are no longer stored in slim tables, consistent with flat-nodes. THis reduces space required for --slim imports without --flat-nodes. - A default database name of gis is no longer assumed. An explicit -d option is recommended if compatibility with 0.92 and earlier.
Update osm2pgsql to 0.92.1 Release 0.92.1 This fixes two important bugs which impacted relations with excessive members and consuming updates with the multi backend Release 0.92.0, a new stable branch Major changes since 0.90.0 are - PostgreSQL 9.1 + PostGIS 2.0 or later are now required, which has allowed performance improvements and cleanups - EPSG 3857 is now default. You can get the old behavior by manually specifying 900913 - Invlid geometries are re-checked for validity after buffering, instead of assuming GEOS returns a valid geom Other changes are - A new option to change the max bbox size at which polygons will expire all the tiles in them, not just the boundary - Behavior fixes for C transforms and tables with no columns - More numeric datatypes are allowed for table columns in C tagtransforms - Lua is now required by default - Code fixes, particularly replacement of C memory management
Add SHA512 digests for distfiles for geography category One mismatched digest found in geography/libmemphis02: # package libmemphis02 recorded SHA1 (memphis-0.2.3.tar.gz) = dbc2f61e49b996dc9ca91df0de9a08eb7adbfa9b calculated SHA1 (memphis-0.2.3.tar.gz) = 85993bce12c3616fcf6e7682a70b9605883edec2 No changes were made to the libmemphis02 distinfo file Otherwise, existing SHA1 digests verified and found to be the same on the machine holding the existing distfiles (morden). All existing SHA1 digests retained for now as an audit trail.
Update geography/osm2pgsql to 0.88.1 Release 0.88.1 This maintainance release fixes duplicates in the job queues for pending ways and relations, minor compiler errors, and updates memory usage documentation. The duplicates in job queues (#419) only impacted multithreaded updates and was not observed in most cases. Release 0.88.0, a new stable branch If upgrading and using a database created with 0.86.0 or earlier, the schema migrations in docs/migrations.md are required. Major changes since 0.86.0 - Osm2pgsql is now C++ and requires the Boost libraries - A new backend has been added, the “multi” backend. This allows multiple tables which can each contain different types of features. More documentation is available at docs/multi.md - In-database pending way tracking has been replaced with in-memory tracking, offering significant performance gains. - Rendering tables are ordered by GeoHash when created, resulting in significant performance improvements. - z_logic has been improved, taking into account more recent work across multiple styles. - The node storage has been improved, and out of order nodes and nodes at 0,0 should now always be handled correctly - A new test suite with unit tests - Many bug-fixes Changes since 0.87.3 include - C++ cleanups - default.style cleanups to improve ease of modification - Remove the lockfree queue pending implementation and default to what was --without-lockfree, which uses less RAM - RAM node storage cleanups - Better node storage test coverage
Import osm2pgsql-0.87.3 as geography/osm2pgsql Converts OSM planet.osm data to a PostgreSQL / PostGIS database suitable for specific applications like rendering into map tiles by Mapnik or geocoding with Nominatim.