debian: Rework historical changelog

  Rewriting history is generally considered a "not-so-good" thing,
however here the historical data does not align with best practises
and therefore it is beneficial to provide a better example going
forward.

There is only one initial release. Everything following that should
list some kind of release notes or changelog, or at the very least
just say something along the lines of "New version" rather than
"Initial release".

In this commit, the Git history is used for this task,
filtering out "Makefile" changes as to retain only changes
that are visible to users, excluding building tooling.

For Debian packages, upon release, the target distribution should be
updated to "unstable" (or "experimental" if preferred for any reason)
when a release is finalized. During development, an invalid
distribution name is expected to be there for tracking unreleased
changes. That is why "UNRELEASED" is the standard way of specifying
ongoing development, being an invalid distribution name itself.

The "(Closes: #XXXXXX)" tag is intended for linking to Debian bug
tracker, such as linking to the initial Intent to Package ticket,
or later update/bugfix tickets. There does not appear to be a bug
tracker in use for this task here, and the XXXXXX bug ticket number
does not take you anywhere. It's therefore better to just remove it.
---
 debian/changelog | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
  

Hello Valters,

thanks for your patch.

Indeed, the historical changelog of libloc currently contains - um - information in a
non-optimal fashion. I guess this was due to the lack of time back then, and nobody of
us had good experience with packaging stuff for Debian. Thanks for improving this.

Eventually, we hoped libloc would be used by other distributions as well, since a decent
part of the open source community is facing license trouble after MaxMind changed their
terms and conditions. I remember Michael having a discussion with some members of the
Debian development team, but my memories fail me when it comes to it's results.

Therefore, I am not sure if libloc is ready in a way we would move from "UNRELEASED" to
"unstable". On the one hand, it is used in production for IPFire since a while, on the
other hand, nobody else is using the libloc _code_ as such - at least no one I am aware of.

@Michael: What do you think?

Thanks, and best regards,
Peter Müller


> Rewriting history is generally considered a "not-so-good" thing,
> however here the historical data does not align with best practises
> and therefore it is beneficial to provide a better example going
> forward.
> 
> There is only one initial release. Everything following that should
> list some kind of release notes or changelog, or at the very least
> just say something along the lines of "New version" rather than
> "Initial release".
> 
> In this commit, the Git history is used for this task,
> filtering out "Makefile" changes as to retain only changes
> that are visible to users, excluding building tooling.
> 
> For Debian packages, upon release, the target distribution should be
> updated to "unstable" (or "experimental" if preferred for any reason)
> when a release is finalized. During development, an invalid
> distribution name is expected to be there for tracking unreleased
> changes. That is why "UNRELEASED" is the standard way of specifying
> ongoing development, being an invalid distribution name itself.
> 
> The "(Closes: #XXXXXX)" tag is intended for linking to Debian bug
> tracker, such as linking to the initial Intent to Package ticket,
> or later update/bugfix tickets. There does not appear to be a bug
> tracker in use for this task here, and the XXXXXX bug ticket number
> does not take you anywhere. It's therefore better to just remove it.
> ---
>  debian/changelog | 15 +++++++++++----
>  1 file changed, 11 insertions(+), 4 deletions(-)
> 
> diff --git a/debian/changelog b/debian/changelog
> index e0be397..e58c0ca 100644
> --- a/debian/changelog
> +++ b/debian/changelog
> @@ -1,11 +1,18 @@
> -libloc (0.9.6-1) UNRELEASED; urgency=medium
> +libloc (0.9.6-1) unstable; urgency=medium
> 
> -  * Initial release. (Closes: #XXXXXX)
> +  * location-importer.in: skip networks with unknown country codes
> +  * location-importer.in: process unaligned IP ranges in RIR data files
> +    correctly
> +  * database: Free mmapped countries section
> +  * location-importer.in: reduce log noise for unusable networks
> +  * location-importer.in: delete 6to4 IPv6 space as well
> +  * location-importer.in: fix typo
> +  * location: Fix list-networks-by-as
> 
>   -- Michael Tremer <michael.tremer@ipfire.org>  Wed, 31 Mar 2021 14:06:00 +0100
> 
> -libloc (0.9.5-1) UNRELEASED; urgency=medium
> +libloc (0.9.5-1) unstable; urgency=medium
> 
> -  * Initial release. (Closes: #XXXXXX)
> +  * Initial release.
> 
>   -- Stefan Schantl <stefan.schantl@ipfire.org>  Sun, 27 Oct 2019 18:55:44 +0100
>
  

On Tue, Apr 13, 2021 at 6:38 PM Peter Müller <peter.mueller@ipfire.org> wrote:
> Eventually, we hoped libloc would be used by other distributions as well, since a decent
> part of the open source community is facing license trouble after MaxMind changed their
> terms and conditions. I remember Michael having a discussion with some members of the
> Debian development team, but my memories fail me when it comes to it's results.
>
> Therefore, I am not sure if libloc is ready in a way we would move from "UNRELEASED" to
> "unstable". On the one hand, it is used in production for IPFire since a while, on the
> other hand, nobody else is using the libloc _code_ as such - at least no one I am aware of.

I am in that boat actually, as I ended up looking at the repository
with the goal of migrating away from MaxMind, however I am on Ubuntu.
The build currently fails due to a bad test invocation which I hope to
take a closer look at. Additionally I would like to update the
debhelper compatibility level while I am at it, but that also needs to
be looked into - whether the resulting build is the same, however for
that I would like to have the automated build tooling in place (which
needs those test changes).

Regarding the topic of "UNRELEASED" vs "unstable": Having "unstable"
for a _released_ version is the standard way for Debian-native
packages.

You can take the `debmirror` tool as a simple example. The official
upstream changelog there can be seen in the source containing
"unstable": https://salsa.debian.org/debian/debmirror/-/blob/debian/1%252.33/debian/changelog

As people are working on future changes, "UNRELEASED" is used for
tracking changes until the release is tagged (by replacing
"UNRELEASED" with "unstable", and updating the maintainer name/email
and date). A sample of work in progress in source can be seen:
https://salsa.debian.org/debian/debmirror/-/blob/0f9992cdb9b535bd42958a9ff6cb07723f064006/debian/changelog

The tool is available in Ubuntu repositories as well, where additional
patches are applied -- replacing Debian defaults with Ubuntu defaults
as required for the package. As a result, in Ubuntu a separate version
with a 'ubuntu' suffix gets created, while the history still lists
"unstable" throughout from its upstream:
https://changelogs.ubuntu.com/changelogs/pool/universe/d/debmirror/debmirror_2.33ubuntu1/changelog

--Valters
  

Hello,

Thank you Valters for the patch :)

> On 13 Apr 2021, at 16:38, Peter Müller <peter.mueller@ipfire.org> wrote:
> 
> Hello Valters,
> 
> thanks for your patch.
> 
> Indeed, the historical changelog of libloc currently contains - um - information in a
> non-optimal fashion. I guess this was due to the lack of time back then, and nobody of
> us had good experience with packaging stuff for Debian. Thanks for improving this.
> 
> Eventually, we hoped libloc would be used by other distributions as well, since a decent
> part of the open source community is facing license trouble after MaxMind changed their
> terms and conditions. I remember Michael having a discussion with some members of the
> Debian development team, but my memories fail me when it comes to it's results.

You wanted to reach out to them to find out what it takes to get our package into Debian :)

> Therefore, I am not sure if libloc is ready in a way we would move from "UNRELEASED" to
> "unstable". On the one hand, it is used in production for IPFire since a while, on the
> other hand, nobody else is using the libloc _code_ as such - at least no one I am aware of.

libloc is stable. We should technically give it the 1.0 version tag soon.

However, the Debian package might have some issues, but I do not see that as a reason that we mark it as “please stay away and use something else”. That would send the wrong signal about libloc.

I cannot disclose any other users of libloc apart from those that I have already shared publicly, but we have plenty of downloads of the library so I assume that there are some silent users out there :)

We should work more on making people aware that there now is an alternative to other products available which is truly free software.

-Michael

> @Michael: What do you think?
> 
> Thanks, and best regards,
> Peter Müller
> 
> 
>> Rewriting history is generally considered a "not-so-good" thing,
>> however here the historical data does not align with best practises
>> and therefore it is beneficial to provide a better example going
>> forward.
>> 
>> There is only one initial release. Everything following that should
>> list some kind of release notes or changelog, or at the very least
>> just say something along the lines of "New version" rather than
>> "Initial release".
>> 
>> In this commit, the Git history is used for this task,
>> filtering out "Makefile" changes as to retain only changes
>> that are visible to users, excluding building tooling.
>> 
>> For Debian packages, upon release, the target distribution should be
>> updated to "unstable" (or "experimental" if preferred for any reason)
>> when a release is finalized. During development, an invalid
>> distribution name is expected to be there for tracking unreleased
>> changes. That is why "UNRELEASED" is the standard way of specifying
>> ongoing development, being an invalid distribution name itself.
>> 
>> The "(Closes: #XXXXXX)" tag is intended for linking to Debian bug
>> tracker, such as linking to the initial Intent to Package ticket,
>> or later update/bugfix tickets. There does not appear to be a bug
>> tracker in use for this task here, and the XXXXXX bug ticket number
>> does not take you anywhere. It's therefore better to just remove it.
>> ---
>> debian/changelog | 15 +++++++++++----
>> 1 file changed, 11 insertions(+), 4 deletions(-)
>> 
>> diff --git a/debian/changelog b/debian/changelog
>> index e0be397..e58c0ca 100644
>> --- a/debian/changelog
>> +++ b/debian/changelog
>> @@ -1,11 +1,18 @@
>> -libloc (0.9.6-1) UNRELEASED; urgency=medium
>> +libloc (0.9.6-1) unstable; urgency=medium
>> 
>> -  * Initial release. (Closes: #XXXXXX)
>> +  * location-importer.in: skip networks with unknown country codes
>> +  * location-importer.in: process unaligned IP ranges in RIR data files
>> +    correctly
>> +  * database: Free mmapped countries section
>> +  * location-importer.in: reduce log noise for unusable networks
>> +  * location-importer.in: delete 6to4 IPv6 space as well
>> +  * location-importer.in: fix typo
>> +  * location: Fix list-networks-by-as
>> 
>>  -- Michael Tremer <michael.tremer@ipfire.org>  Wed, 31 Mar 2021 14:06:00 +0100
>> 
>> -libloc (0.9.5-1) UNRELEASED; urgency=medium
>> +libloc (0.9.5-1) unstable; urgency=medium
>> 
>> -  * Initial release. (Closes: #XXXXXX)
>> +  * Initial release.
>> 
>>  -- Stefan Schantl <stefan.schantl@ipfire.org>  Sun, 27 Oct 2019 18:55:44 +0100
>>
  

Hello,

> On 13 Apr 2021, at 17:41, Valters Jansons <valter.jansons@gmail.com> wrote:
> 
> On Tue, Apr 13, 2021 at 6:38 PM Peter Müller <peter.mueller@ipfire.org> wrote:
>> Eventually, we hoped libloc would be used by other distributions as well, since a decent
>> part of the open source community is facing license trouble after MaxMind changed their
>> terms and conditions. I remember Michael having a discussion with some members of the
>> Debian development team, but my memories fail me when it comes to it's results.
>> 
>> Therefore, I am not sure if libloc is ready in a way we would move from "UNRELEASED" to
>> "unstable". On the one hand, it is used in production for IPFire since a while, on the
>> other hand, nobody else is using the libloc _code_ as such - at least no one I am aware of.
> 
> I am in that boat actually, as I ended up looking at the repository
> with the goal of migrating away from MaxMind, however I am on Ubuntu.
> The build currently fails due to a bad test invocation which I hope to
> take a closer look at. Additionally I would like to update the
> debhelper compatibility level while I am at it, but that also needs to
> be looked into - whether the resulting build is the same, however for
> that I would like to have the automated build tooling in place (which
> needs those test changes).

It should work just fine on Ubuntu. We are only dependent on a POSIX-compatible system so Windows might be a bit tricky. I used to build it on Mac OS X, too.

If there is interest, I wouldn’t mind publishing Ubuntu packages. Better would of course be to make it an upstream package.

> Regarding the topic of "UNRELEASED" vs "unstable": Having "unstable"
> for a _released_ version is the standard way for Debian-native
> packages.

unstable is right for us then.

> You can take the `debmirror` tool as a simple example. The official
> upstream changelog there can be seen in the source containing
> "unstable": https://salsa.debian.org/debian/debmirror/-/blob/debian/1%252.33/debian/changelog
> 
> As people are working on future changes, "UNRELEASED" is used for
> tracking changes until the release is tagged (by replacing
> "UNRELEASED" with "unstable", and updating the maintainer name/email
> and date). A sample of work in progress in source can be seen:
> https://salsa.debian.org/debian/debmirror/-/blob/0f9992cdb9b535bd42958a9ff6cb07723f064006/debian/changelog

We normally do not build packages with a development version.

> The tool is available in Ubuntu repositories as well, where additional
> patches are applied -- replacing Debian defaults with Ubuntu defaults
> as required for the package. As a result, in Ubuntu a separate version
> with a 'ubuntu' suffix gets created, while the history still lists
> "unstable" throughout from its upstream:
> https://changelogs.ubuntu.com/changelogs/pool/universe/d/debmirror/debmirror_2.33ubuntu1/changelog
> 
> --Valters

-Michael
  

On Wed, Apr 14, 2021 at 12:31 PM Michael Tremer
<michael.tremer@ipfire.org> wrote:
> It should work just fine on Ubuntu. We are only dependent on a POSIX-compatible system so Windows might be a bit tricky. I used to build it on Mac OS X, too.
>
> If there is interest, I wouldn’t mind publishing Ubuntu packages. Better would of course be to make it an upstream package.

Understandable - and I completely agree in the benefit of having the
package available in Debian to be pulled into all Debian derivative
distributions that way.

The building problem is not directly linked with Ubuntu. Instead, it
is about auto_test failing, due to `make check` failing on the root
Makefile. Testsuite for libloc in the root directory passes, however
the check-recursive target then tries to `make check` inside of po
subdirectory which fails with: "No rule to make target
'../src/python/__init__.py', needed by 'libloc.pot'.  Stop."

The broken scenario that needs patching can simplified to:
$ autoreconf --install --symlink
$ intltoolize --force --automake
$ ./configure --prefix=/usr --sysconfdir=/etc --libdir=/lib
$ make -j$(nproc)
$ make -j$(nproc) check

I will shortly provide a patch with an updated po/POTFILES.in as
generated by `rm po/POTFILES.in && make po/POTFILES.in`.

> > You can take the `debmirror` tool as a simple example. The official
> > upstream changelog there can be seen in the source containing
> > "unstable": https://salsa.debian.org/debian/debmirror/-/blob/debian/1%252.33/debian/changelog
> >
> > As people are working on future changes, "UNRELEASED" is used for
> > tracking changes until the release is tagged (by replacing
> > "UNRELEASED" with "unstable", and updating the maintainer name/email
> > and date). A sample of work in progress in source can be seen:
> > https://salsa.debian.org/debian/debmirror/-/blob/0f9992cdb9b535bd42958a9ff6cb07723f064006/debian/changelog
>
> We normally do not build packages with a development version.

The UNRELEASED distribution is tagged for that reason, as to signal
that a package should not be built from that source. I was simply
highlighting a development workflow in place for one Debian package
which tracks changes during development, where individual
commits/patches also update the changelog file. This approach ensures
that at release time only the s/UNRELEASED/unstable/ replacement needs
to happen along with `update-maintainer` -- without having to worry
about collecting the list of changes.

--Valters
  

Hello,

> On 14 Apr 2021, at 11:03, Valters Jansons <valter.jansons@gmail.com> wrote:
> 
> On Wed, Apr 14, 2021 at 12:31 PM Michael Tremer
> <michael.tremer@ipfire.org> wrote:
>> It should work just fine on Ubuntu. We are only dependent on a POSIX-compatible system so Windows might be a bit tricky. I used to build it on Mac OS X, too.
>> 
>> If there is interest, I wouldn’t mind publishing Ubuntu packages. Better would of course be to make it an upstream package.
> 
> Understandable - and I completely agree in the benefit of having the
> package available in Debian to be pulled into all Debian derivative
> distributions that way.
> 
> The building problem is not directly linked with Ubuntu. Instead, it
> is about auto_test failing, due to `make check` failing on the root
> Makefile. Testsuite for libloc in the root directory passes, however
> the check-recursive target then tries to `make check` inside of po
> subdirectory which fails with: "No rule to make target
> '../src/python/__init__.py', needed by 'libloc.pot'.  Stop."
> 
> The broken scenario that needs patching can simplified to:
> $ autoreconf --install --symlink
> $ intltoolize --force --automake
> $ ./configure --prefix=/usr --sysconfdir=/etc --libdir=/lib
> $ make -j$(nproc)
> $ make -j$(nproc) check
> 
> I will shortly provide a patch with an updated po/POTFILES.in as
> generated by `rm po/POTFILES.in && make po/POTFILES.in`.

This does not show any changes on my system.

>>> You can take the `debmirror` tool as a simple example. The official
>>> upstream changelog there can be seen in the source containing
>>> "unstable": https://salsa.debian.org/debian/debmirror/-/blob/debian/1%252.33/debian/changelog
>>> 
>>> As people are working on future changes, "UNRELEASED" is used for
>>> tracking changes until the release is tagged (by replacing
>>> "UNRELEASED" with "unstable", and updating the maintainer name/email
>>> and date). A sample of work in progress in source can be seen:
>>> https://salsa.debian.org/debian/debmirror/-/blob/0f9992cdb9b535bd42958a9ff6cb07723f064006/debian/changelog
>> 
>> We normally do not build packages with a development version.
> 
> The UNRELEASED distribution is tagged for that reason, as to signal
> that a package should not be built from that source. I was simply
> highlighting a development workflow in place for one Debian package
> which tracks changes during development, where individual
> commits/patches also update the changelog file. This approach ensures
> that at release time only the s/UNRELEASED/unstable/ replacement needs
> to happen along with `update-maintainer` -- without having to worry
> about collecting the list of changes.
> 
> --Valters
  

On Wed, Apr 14, 2021 at 1:05 PM Michael Tremer
<michael.tremer@ipfire.org> wrote:
> > The broken scenario that needs patching can simplified to:
> > $ autoreconf --install --symlink
> > $ intltoolize --force --automake
> > $ ./configure --prefix=/usr --sysconfdir=/etc --libdir=/lib
> > $ make -j$(nproc)
> > $ make -j$(nproc) check
> >
> > I will shortly provide a patch with an updated po/POTFILES.in as
> > generated by `rm po/POTFILES.in && make po/POTFILES.in`.
>
> This does not show any changes on my system.

Based on a clean clone of the repository? Maybe the
src/python/__init__.py gets generated with some other targets?

I personally get a diff of:

--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -1,0 +2 @@
+src/python/__init__.py.in
@@ -7,2 +7,0 @@
-src/python/__init__.py
-src/python/__init__.py.in

--Valters