--- Begin Message ---
- To: Debian Bug Tracking System <submit@bugs.debian.org>
- Subject: unblock/pre-approval: geoip/1.6.2-3
- From: Patrick Matthäi <pmatthaei@debian.org>
- Date: Tue, 30 Dec 2014 19:44:42 +0100
- Message-id: <20141230184442.22569.53316.reportbug@srv1.linux-dev.org>
Package: release.debian.org
Severity: normal
User: release.debian.org@packages.debian.org
Usertags: unblock
Hello,
I am asking for a pre-approval for geoip. It just fixes (or better it adds support) a problem
on generating the database, if upstream csv is not correct built, as it is the case since
two months now..
The fix is also in experimental since the 13.11.2014 (introduced with version 1.6.3-2).
Would you allow this change? If yes I would upload it to unstable and mail you again after it has been altered.
diff -Naur '--exclude=.svn' tags/1.6.2-2/debian/changelog branches/jessie/debian/changelog
--- tags/1.6.2-2/debian/changelog 2014-10-27 19:31:48.626784609 +0100
+++ branches/jessie/debian/changelog 2014-12-30 19:35:21.634201307 +0100
@@ -1,3 +1,10 @@
+geoip (1.6.2-3) unstable; urgency=low
+
+ * geoip-generator: Add support for skipping locations if the location ID is
+ not correctly ordered in the upstream CSV.
+
+ -- Patrick Matthäi <pmatthaei@debian.org> Tue, 30 Dec 2014 19:35:02 +0100
+
geoip (1.6.2-2) unstable; urgency=low
* Add patch for geoip-csv-to-dat to add support for building GeoIP city DB.
diff -Naur '--exclude=.svn' tags/1.6.2-2/debian/src/geoip-csv-to-dat.cpp branches/jessie/debian/src/geoip-csv-to-dat.cpp
--- tags/1.6.2-2/debian/src/geoip-csv-to-dat.cpp 2014-10-27 19:31:48.626784609 +0100
+++ branches/jessie/debian/src/geoip-csv-to-dat.cpp 2014-12-30 19:32:51.588379341 +0100
@@ -694,12 +694,15 @@
class city_dat_writer : public dat_writer
{
public:
- // All serialized location information
+ // All serialized location information, in one big
+ // undifferentiated block
std::stringstream location_stream;
// Seek offset of each location within
// location_stream (relative to the beginning of
- // location_stream)
+ // location_stream). An offset of -1 means that that
+ // location is not in the table (can happen if the
+ // location info's out of order).
std::vector<int> location_pos;
city_dat_writer(const char *dat_file_name, GeoIPDBTypes database_type);
@@ -744,14 +747,22 @@
{
if (it->edges[0] == 0x1000000) // No data
it->edges[0] = trie_size;
- else if (it->edges[0] > 0x1000000) // Ptr to location block
- it->edges[0] = location_pos[it->edges[0] - 0x1000000 - 1] + trie_size;
+ else if (it->edges[0] > 0x1000000) { // Ptr to location block
+ int loc_id = it->edges[0] - 0x1000000;
+ if (loc_id >= location_pos.size() || location_pos[loc_id] == -1)
+ error(EX_DATAERR, 1, "Location %d exists in blocks but not in locations", loc_id);
+ it->edges[0] = location_pos[loc_id] + trie_size;
+ }
// Any other value would indicate a non-leaf node
if (it->edges[1] == 0x1000000) // No data
it->edges[1] = trie_size;
- else if (it->edges[1] > 0x1000000) // Ptr to location block
- it->edges[1] = location_pos[it->edges[1] - 0x1000000 - 1] + trie_size;
+ else if (it->edges[1] > 0x1000000) { // Ptr to location block
+ int loc_id = it->edges[1] - 0x1000000;
+ if (loc_id >= location_pos.size() || location_pos[loc_id] == -1)
+ error(EX_DATAERR, 1, "Location %d exists in blocks but not in locations", loc_id);
+ it->edges[1] = location_pos[loc_id] + trie_size;
+ }
// Any other value would indicate a non-leaf node
}
}
@@ -801,19 +812,33 @@
const char *input_file_name,
int input_line_number)
{
- // There's nothing wrong with location info being out of
- // order, but currently we don't support it (i.e. the code
- // won't work if the info's out of order). Sanity check that
- // things are in order.
+ // First, we save the offset of this location block.
int loc_id = ::atoi(info[0].c_str());
- if (loc_id != location_pos.size() + 1) {
- error_at_line(EX_DATAERR, 0, input_file_name, input_line_number,
- "Location info not in order (currently not supported)");
- return;
- }
+ if (loc_id >= location_pos.size()) {
+ // We need to add to the location table (this is the
+ // usual case).
+
+ while(loc_id > location_pos.size()) {
+ // If some numbers were skipped in the data,
+ // then we need to add some empty locations to
+ // the table before we find our spot.
+ location_pos.push_back(-1);
+ }
- // First, we save the offset of this location block.
- location_pos.push_back(location_stream.tellp());
+ // Now we have our spot, insert this location.
+ location_pos.push_back(location_stream.tellp());
+ } else {
+ // We already have a space in the table for this location --
+ // if it's not empty, then we have two locations with the same
+ // ID, and we print an error.
+ if (location_pos[loc_id] != -1) {
+ error_at_line(EX_DATAERR, 0, input_file_name,
+ input_line_number,
+ "Duplicate location info for ID %d",
+ loc_id);
+ }
+ location_pos[loc_id] = location_stream.tellp();
+ }
// Country ID
int country_id;
@@ -1319,8 +1344,8 @@
return;
}
- const int locid = atoi(csv_fields[CSV_BLOCK_FIELD_LOC].c_str());
- const binary_trie::edge_type leaf = 0x1000000 + locid;
+ const int loc_id = atoi(csv_fields[CSV_BLOCK_FIELD_LOC].c_str());
+ const binary_trie::edge_type leaf = 0x1000000 + loc_id;
if (cmdline.address_family != AF_INET) {
error(EX_SOFTWARE, 1, "IPv6 with city database is unimplemented.");
-- System Information:
Debian Release: 7.7
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 3.2.0-4-amd64 (SMP w/2 CPU cores)
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
--- End Message ---