Add option for skipping invalid ranges

[patryk4815]

Add option for skipping invalid "ranges".

For example:

Error: merging databases: writing range 10.200.200.0-10.200.200.255: inserting 10.200.200.0/24: attempt to insert 10.200.200.0/24 into 10.0.0.0/8, which is a reserved network

Error: merging databases: writing range 2001::-2001:0:ffff:ffff:ffff:ffff:ffff:ffff: inserting 2001::/32: attempt to insert 2001::/32 into 2001::/32, which is an aliased network

It is not possible to easily merge mmdb's

[horgh]

I believe you can set include_reserved_networks = true to allow the first kind: https://github.com/maxmind/mmdbconvert/blob/main/docs/config.md#mmdb-options

I don't think we expose an option for allowing the second kind (aliased networks). I suppose we could add that in theory.

However I wonder why you would want such networks in your database. It would likely be better to filter them out of whatever is originally adding them. Could you tell me more about the databases you're using? And the use case for allowing these networks?

[patryk4815]

To clarify my use case: I explicitly do not want bogon or otherwise non-routable IP networks in the resulting database. My goal is to filter them out entirely. From that perspective, adding such networks to an allow list (for example via include_reserved_networks = true) seems counterintuitive.

[horgh]

I see. Thanks for clarifying that!

I think we could have options to skip them. We could have one option for reserved networks and one option for aliased networks. We could probably detect the error type from the mmdbwriter insert and skip if these options are enabled. mmdbwriter has error types for each of these.

Until such an option exists, you could rewrite your databases without these networks before using them with mmdbconvert. You could do this using mmdbwriter (https://github.com/maxmind/mmdbwriter).

It's possible skipping the aliased networks isn't what you'd want to do in all cases. You may want to normalize them such that they could be inserted.