Add tool to format rootfiles

  There are some rules which files should be definitely not included in a
rootfile. Applying these rules by hand is error prone and annoying. So I
wrote these simple command, because computer can apply these rules
faster and better than humans :-;

It removes all lines starting with '-' and comment out all lines which
point to files which should not be included in the distribution. This
list needs to be extended, but I think this is a good starting point to
make the creating of rootfiles easier.

Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
---
 make.sh | 37 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 36 insertions(+), 1 deletion(-)
  

Hallo Jonatan,

I see that you have a rule to remove all .so shared libraries from /usr/lib

With the bacula addon, this will give a problem as bacula have a different approach to the naming of their shared libraries

/usr/lib/libbac-9.6.5.so
/usr/lib/libbaccfg-9.6.5.so
/usr/lib/libbaccfg.so -> libbaccfg-9.6.5.so
/usr/lib/libbacfind-9.6.5.so
/usr/lib/libbacfind.so -> libbacfind-9.6.5.so
/usr/lib/libbac.so -> libbac-9.6.5.so

As you can see the .so is at the end of all the files so that rule would remove all shared libraries causing bacula to fail to load.

If this rule is implemented in make.sh then the only option I can think of for bacula would be to change the library location from /usr/lib to /lib unless someone else has any suggestions for how I could deal with it.


Regards,

Adolf

On 05/09/2020 10:40, Jonatan Schlag wrote:
> There are some rules which files should be definitely not included in a
> rootfile. Applying these rules by hand is error prone and annoying. So I
> wrote these simple command, because computer can apply these rules
> faster and better than humans :-;
>
> It removes all lines starting with '-' and comment out all lines which
> point to files which should not be included in the distribution. This
> list needs to be extended, but I think this is a good starting point to
> make the creating of rootfiles easier.
>
> Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
> ---
>   make.sh | 37 ++++++++++++++++++++++++++++++++++++-
>   1 file changed, 36 insertions(+), 1 deletion(-)
>
> diff --git a/make.sh b/make.sh
> index 4a9dd3cb6..909d4a4de 100755
> --- a/make.sh
> +++ b/make.sh
> @@ -1895,6 +1895,41 @@ downloadsrc)
>   	fi
>   	cd - >/dev/null 2>&1
>   	;;
> +format-rootfile)
> +	if [ ! -f $2 ]; then
> +		echo -n "'$2' is not a regular file"
> +		beautify message FAIL
> +		exit 1
> +	fi
> +
> +	ROOTFILE_TO_FORMAT="$2"
> +
> +	# Remove all .so files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .a files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +
> +	# Remove all .la files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .hpp files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .h files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .ipp files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .m4 files in /usr/share/aclocal
> +	sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all lines starting with -
> +	sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
> +
> +;;
>   toolchain)
>   	# Clear screen
>   	${INTERACTIVE} && clear
> @@ -1990,7 +2025,7 @@ find-dependencies)
>   	exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build" "$@"
>   	;;
>   *)
> -	echo "Usage: $0 {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
> +	echo "Usage: $0 {build|changelog|clean|format-rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
>   	cat doc/make.sh-usage
>   	;;
>   esac
  

Hallo Jonatan,

Having looked at /usr/lib further I have noted that programmes such as conntrack-tools and collectd just use .so files and the solution with them is to have their own directory under /usr/lib.

So I could do the same with bacula and change the libdir in configure to be /usr/lib/bacula. I think that should solve the problem. Just have to remember to do the change when the next bacula update occurs.

Regards,

Adolf

On 05/09/2020 21:14, Adolf Belka wrote:
> Hallo Jonatan,
>
> I see that you have a rule to remove all .so shared libraries from /usr/lib
>
> With the bacula addon, this will give a problem as bacula have a different approach to the naming of their shared libraries
>
> /usr/lib/libbac-9.6.5.so
> /usr/lib/libbaccfg-9.6.5.so
> /usr/lib/libbaccfg.so -> libbaccfg-9.6.5.so
> /usr/lib/libbacfind-9.6.5.so
> /usr/lib/libbacfind.so -> libbacfind-9.6.5.so
> /usr/lib/libbac.so -> libbac-9.6.5.so
>
> As you can see the .so is at the end of all the files so that rule would remove all shared libraries causing bacula to fail to load.
>
> If this rule is implemented in make.sh then the only option I can think of for bacula would be to change the library location from /usr/lib to /lib unless someone else has any suggestions for how I could deal with it.
>
>
> Regards,
>
> Adolf
>
> On 05/09/2020 10:40, Jonatan Schlag wrote:
>> There are some rules which files should be definitely not included in a
>> rootfile. Applying these rules by hand is error prone and annoying. So I
>> wrote these simple command, because computer can apply these rules
>> faster and better than humans :-;
>>
>> It removes all lines starting with '-' and comment out all lines which
>> point to files which should not be included in the distribution. This
>> list needs to be extended, but I think this is a good starting point to
>> make the creating of rootfiles easier.
>>
>> Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
>> ---
>>   make.sh | 37 ++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 36 insertions(+), 1 deletion(-)
>>
>> diff --git a/make.sh b/make.sh
>> index 4a9dd3cb6..909d4a4de 100755
>> --- a/make.sh
>> +++ b/make.sh
>> @@ -1895,6 +1895,41 @@ downloadsrc)
>>       fi
>>       cd - >/dev/null 2>&1
>>       ;;
>> +format-rootfile)
>> +    if [ ! -f $2 ]; then
>> +        echo -n "'$2' is not a regular file"
>> +        beautify message FAIL
>> +        exit 1
>> +    fi
>> +
>> +    ROOTFILE_TO_FORMAT="$2"
>> +
>> +    # Remove all .so files in /usr/lib
>> +    sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all .a files in /usr/lib
>> +    sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +
>> +    # Remove all .la files in /usr/lib
>> +    sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all .hpp files in /usr/include
>> +    sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all .h files in /usr/include
>> +    sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all .ipp files in /usr/include
>> +    sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all .m4 files in /usr/share/aclocal
>> +    sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>> +
>> +    # Remove all lines starting with -
>> +    sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
>> +
>> +;;
>>   toolchain)
>>       # Clear screen
>>       ${INTERACTIVE} && clear
>> @@ -1990,7 +2025,7 @@ find-dependencies)
>>       exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build" "$@"
>>       ;;
>>   *)
>> -    echo "Usage: $0 {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
>> +    echo "Usage: $0 {build|changelog|clean|format-rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
>>       cat doc/make.sh-usage
>>       ;;
>>   esac
  

Hi,

Thanks for having a look at this.

> Am 05.09.2020 um 21:30 schrieb Adolf Belka <ahb.ipfire@gmail.com>:
> 
> Hallo Jonatan,
> 
> Having looked at /usr/lib further I have noted that programmes such as conntrack-tools and collectd just use .so files and the solution with them is to have their own directory under /usr/lib.
> 
> So I could do the same with bacula and change the libdir in configure to be /usr/lib/bacula. I think that should solve the problem. Just have to remember to do the change when the next bacula update occurs.
> 
> Regards,
> 
> Adolf
> 
>> On 05/09/2020 21:14, Adolf Belka wrote:
>> Hallo Jonatan,
>> 
>> I see that you have a rule to remove all .so shared libraries from /usr/lib
>> 
>> With the bacula addon, this will give a problem as bacula have a different approach to the naming of their shared libraries
>> 
>> /usr/lib/libbac-9.6.5.so
>> /usr/lib/libbaccfg-9.6.5.so
>> /usr/lib/libbaccfg.so -> libbaccfg-9.6.5.so
>> /usr/lib/libbacfind-9.6.5.so
>> /usr/lib/libbacfind.so -> libbacfind-9.6.5.so
>> /usr/lib/libbac.so -> libbac-9.6.5.so

Thanks for pointing this out. So the patch shows that our wiki is inaccurate. You are right that the patch in the current version, is not suitable for production.
>> 
>> As you can see the .so is at the end of all the files so that rule would remove all shared libraries causing bacula to fail to load.
>> 
>> If this rule is implemented in make.sh then the only option I can think of for bacula would be to change the library location from /usr/lib to /lib unless someone else has any suggestions for how I could deal with it.
>> 
I suggest dropping the patch in the current version 😉. Sometimes a good workaround is worth a lot, but here I should improve the script to detect these situations and handle them gracefully.

So no need to do anything 🙂. The only one which need to do things is myself 😒.
I will raise this topic in the telephone conference tomorrow.



Greetings Jonatan
>> 
>> Regards,
>> 
>> Adolf
>> 
>>> On 05/09/2020 10:40, Jonatan Schlag wrote:
>>> There are some rules which files should be definitely not included in a
>>> rootfile. Applying these rules by hand is error prone and annoying. So I
>>> wrote these simple command, because computer can apply these rules
>>> faster and better than humans :-;
>>> 
>>> It removes all lines starting with '-' and comment out all lines which
>>> point to files which should not be included in the distribution. This
>>> list needs to be extended, but I think this is a good starting point to
>>> make the creating of rootfiles easier.
>>> 
>>> Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
>>> ---
>>>   make.sh | 37 ++++++++++++++++++++++++++++++++++++-
>>>   1 file changed, 36 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/make.sh b/make.sh
>>> index 4a9dd3cb6..909d4a4de 100755
>>> --- a/make.sh
>>> +++ b/make.sh
>>> @@ -1895,6 +1895,41 @@ downloadsrc)
>>>       fi
>>>       cd - >/dev/null 2>&1
>>>       ;;
>>> +format-rootfile)
>>> +    if [ ! -f $2 ]; then
>>> +        echo -n "'$2' is not a regular file"
>>> +        beautify message FAIL
>>> +        exit 1
>>> +    fi
>>> +
>>> +    ROOTFILE_TO_FORMAT="$2"
>>> +
>>> +    # Remove all .so files in /usr/lib
>>> +    sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all .a files in /usr/lib
>>> +    sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +
>>> +    # Remove all .la files in /usr/lib
>>> +    sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all .hpp files in /usr/include
>>> +    sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all .h files in /usr/include
>>> +    sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all .ipp files in /usr/include
>>> +    sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all .m4 files in /usr/share/aclocal
>>> +    sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +    # Remove all lines starting with -
>>> +    sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +;;
>>>   toolchain)
>>>       # Clear screen
>>>       ${INTERACTIVE} && clear
>>> @@ -1990,7 +2025,7 @@ find-dependencies)
>>>       exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build" "$@"
>>>       ;;
>>>   *)
>>> -    echo "Usage: $0 {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
>>> +    echo "Usage: $0 {build|changelog|clean|format-rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
>>>       cat doc/make.sh-usage
>>>       ;;
>>>   esac
<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr">Hi,</div><div dir="ltr"><br></div><div dir="ltr">Thanks for having a look at this.</div><div dir="ltr"><br><blockquote type="cite">Am 05.09.2020 um 21:30 schrieb Adolf Belka &lt;ahb.ipfire@gmail.com&gt;:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><span>Hallo Jonatan,</span><br><span></span><br><span>Having looked at /usr/lib further I have noted that programmes such as conntrack-tools and collectd just use .so files and the solution with them is to have their own directory under /usr/lib.</span><br><span></span><br><span>So I could do the same with bacula and change the libdir in configure to be /usr/lib/bacula. I think that should solve the problem. Just have to remember to do the change when the next bacula update occurs.</span><br><span></span><br><span>Regards,</span><br><span></span><br><span>Adolf</span><br><span></span><br><span>On 05/09/2020 21:14, Adolf Belka wrote:</span><br><blockquote type="cite"><span>Hallo Jonatan,</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>I see that you have a rule to remove all .so shared libraries from /usr/lib</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>With the bacula addon, this will give a problem as bacula have a different approach to the naming of their shared libraries</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbac-9.6.5.so</span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbaccfg-9.6.5.so</span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbaccfg.so -&gt; libbaccfg-9.6.5.so</span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbacfind-9.6.5.so</span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbacfind.so -&gt; libbacfind-9.6.5.so</span><br></blockquote><blockquote type="cite"><span>/usr/lib/libbac.so -&gt; libbac-9.6.5.so</span><br></blockquote></div></blockquote><div><br></div>Thanks for pointing this out. So the patch shows that our wiki is inaccurate. You are right that the patch in the current version, is not suitable for production.<br><blockquote type="cite"><div dir="ltr"><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>As you can see the .so is at the end of all the files so that rule would remove all shared libraries causing bacula to fail to load.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>If this rule is implemented in make.sh then the only option I can think of for bacula would be to change the library location from /usr/lib to /lib unless someone else has any suggestions for how I could deal with it.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote></div></blockquote><div><div><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">I&nbsp;</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">suggest dropping</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">&nbsp;the patch in the current version 😉. Sometimes a good&nbsp;</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">workaround</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">&nbsp;is worth a lot, but here I should improve the script to detect&nbsp;</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">these</span><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">&nbsp;situations and handle them gracefully.</span><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">So no need to do anything 🙂. The only one which need to do things is myself 😒.</span><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"><span style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px; background-color: rgb(255, 255, 255);">I will raise this topic in the telephone conference tomorrow.</span><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"><br style="font-family: &quot;Source Sans Pro&quot;, sans-serif; font-size: 18px;"></div><div><br></div><div>Greetings Jonatan<br><blockquote type="cite"><div dir="ltr"><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Regards,</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>Adolf</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>On 05/09/2020 10:40, Jonatan Schlag wrote:</span><br></blockquote><blockquote type="cite"><blockquote type="cite"><span>There are some rules which files should be definitely not included in a</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>rootfile. Applying these rules by hand is error prone and annoying. So I</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>wrote these simple command, because computer can apply these rules</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>faster and better than humans :-;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span></span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>It removes all lines starting with '-' and comment out all lines which</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>point to files which should not be included in the distribution. This</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>list needs to be extended, but I think this is a good starting point to</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>make the creating of rootfiles easier.</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span></span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>Signed-off-by: Jonatan Schlag &lt;jonatan.schlag@ipfire.org&gt;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>---</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp; make.sh | 37 ++++++++++++++++++++++++++++++++++++-</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp; 1 file changed, 36 insertions(+), 1 deletion(-)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span></span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>diff --git a/make.sh b/make.sh</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>index 4a9dd3cb6..909d4a4de 100755</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>--- a/make.sh</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+++ b/make.sh</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>@@ -1895,6 +1895,41 @@ downloadsrc)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; fi</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; cd - &gt;/dev/null 2&gt;&amp;1</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ;;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+format-rootfile)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; if [ ! -f $2 ]; then</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; echo -n "'$2' is not a regular file"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; beautify message FAIL</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exit 1</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; fi</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; ROOTFILE_TO_FORMAT="$2"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .so files in /usr/lib</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .a files in /usr/lib</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .la files in /usr/lib</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .hpp files in /usr/include</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .h files in /usr/include</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .ipp files in /usr/include</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all .m4 files in /usr/share/aclocal</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; # Remove all lines starting with -</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+;;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp; toolchain)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; # Clear screen</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ${INTERACTIVE} &amp;&amp; clear</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>@@ -1990,7 +2025,7 @@ find-dependencies)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build" "$@"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ;;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp; *)</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>-&nbsp;&nbsp;&nbsp; echo "Usage: $0 {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>+&nbsp;&nbsp;&nbsp; echo "Usage: $0 {build|changelog|clean|format-rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; cat doc/make.sh-usage</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; ;;</span><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><span>&nbsp; esac</span><br></blockquote></blockquote></div></blockquote></div></div></body></html>
  

Hello Jonatan,

We have already talked about this patch on the last telephone conference, but I would still like to put forward this:

Can we not change this patch, so that it only tells us what could/should potentially be changed? So the command will simply say things like “/usr/lib/libabc.so is a development file and should not be included”.

Wouldn’t that avoid the problem of any automatic errors and we still have a helper that makes the whole process easier for developers?

For that, the sed statements simply have to be changed to grep and the output of that has to be dealt with.

-Michael

> On 5 Sep 2020, at 09:40, Jonatan Schlag <jonatan.schlag@ipfire.org> wrote:
> 
> There are some rules which files should be definitely not included in a
> rootfile. Applying these rules by hand is error prone and annoying. So I
> wrote these simple command, because computer can apply these rules
> faster and better than humans :-;
> 
> It removes all lines starting with '-' and comment out all lines which
> point to files which should not be included in the distribution. This
> list needs to be extended, but I think this is a good starting point to
> make the creating of rootfiles easier.
> 
> Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
> ---
> make.sh | 37 ++++++++++++++++++++++++++++++++++++-
> 1 file changed, 36 insertions(+), 1 deletion(-)
> 
> diff --git a/make.sh b/make.sh
> index 4a9dd3cb6..909d4a4de 100755
> --- a/make.sh
> +++ b/make.sh
> @@ -1895,6 +1895,41 @@ downloadsrc)
> 	fi
> 	cd - >/dev/null 2>&1
> 	;;
> +format-rootfile)
> +	if [ ! -f $2 ]; then
> +		echo -n "'$2' is not a regular file"
> +		beautify message FAIL
> +		exit 1
> +	fi
> +
> +	ROOTFILE_TO_FORMAT="$2"
> +
> +	# Remove all .so files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .a files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +
> +	# Remove all .la files in /usr/lib
> +	sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .hpp files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .h files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .ipp files in /usr/include
> +	sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all .m4 files in /usr/share/aclocal
> +	sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> +
> +	# Remove all lines starting with -
> +	sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
> +
> +;;
> toolchain)
> 	# Clear screen
> 	${INTERACTIVE} && clear
> @@ -1990,7 +2025,7 @@ find-dependencies)
> 	exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build" "$@"
> 	;;
> *)
> -	echo "Usage: $0 {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
> +	echo "Usage: $0 {build|changelog|clean|format-rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-contributors|find-dependencies}"
> 	cat doc/make.sh-usage
> 	;;
> esac
> -- 
> 2.20.1
>
  

Hi Michael,

this approach would be similar to the root file checker already
included in the "tools directory.  Maybe this approach is now the
better one because we can recheck this tool against existing correct
root files, which should make it easy to find errors in the
expressions.

As such a tool should be well tested and I have to try out BDD
(Behaviour Driven Development) I will try, to combine these and create
a little software for this task. 

I will report back when  I have results.

Jonatan

Am Donnerstag, den 10.09.2020, 12:45 +0100 schrieb Michael Tremer:
> Hello Jonatan,
> 
> We have already talked about this patch on the last telephone
> conference, but I would still like to put forward this:
> 
> Can we not change this patch, so that it only tells us what
> could/should potentially be changed? So the command will simply say
> things like “/usr/lib/libabc.so is a development file and should not
> be included”.
> 
> Wouldn’t that avoid the problem of any automatic errors and we still
> have a helper that makes the whole process easier for developers?
> 
> For that, the sed statements simply have to be changed to grep and
> the output of that has to be dealt with.
> 
> -Michael
> 
> > On 5 Sep 2020, at 09:40, Jonatan Schlag <jonatan.schlag@ipfire.org>
> > wrote:
> > 
> > There are some rules which files should be definitely not included
> > in a
> > rootfile. Applying these rules by hand is error prone and annoying.
> > So I
> > wrote these simple command, because computer can apply these rules
> > faster and better than humans :-;
> > 
> > It removes all lines starting with '-' and comment out all lines
> > which
> > point to files which should not be included in the distribution.
> > This
> > list needs to be extended, but I think this is a good starting
> > point to
> > make the creating of rootfiles easier.
> > 
> > Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
> > ---
> > make.sh | 37 ++++++++++++++++++++++++++++++++++++-
> > 1 file changed, 36 insertions(+), 1 deletion(-)
> > 
> > diff --git a/make.sh b/make.sh
> > index 4a9dd3cb6..909d4a4de 100755
> > --- a/make.sh
> > +++ b/make.sh
> > @@ -1895,6 +1895,41 @@ downloadsrc)
> > 	fi
> > 	cd - >/dev/null 2>&1
> > 	;;
> > +format-rootfile)
> > +	if [ ! -f $2 ]; then
> > +		echo -n "'$2' is not a regular file"
> > +		beautify message FAIL
> > +		exit 1
> > +	fi
> > +
> > +	ROOTFILE_TO_FORMAT="$2"
> > +
> > +	# Remove all .so files in /usr/lib
> > +	sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all .a files in /usr/lib
> > +	sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> > +
> > +
> > +	# Remove all .la files in /usr/lib
> > +	sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all .hpp files in /usr/include
> > +	sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g'
> > "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all .h files in /usr/include
> > +	sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all .ipp files in /usr/include
> > +	sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g'
> > "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all .m4 files in /usr/share/aclocal
> > +	sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g'
> > "$ROOTFILE_TO_FORMAT"
> > +
> > +	# Remove all lines starting with -
> > +	sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
> > +
> > +;;
> > toolchain)
> > 	# Clear screen
> > 	${INTERACTIVE} && clear
> > @@ -1990,7 +2025,7 @@ find-dependencies)
> > 	exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build"
> > "$@"
> > 	;;
> > *)
> > -	echo "Usage: $0
> > {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchai
> > n|update-contributors|find-dependencies}"
> > +	echo "Usage: $0 {build|changelog|clean|format-
> > rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-
> > contributors|find-dependencies}"
> > 	cat doc/make.sh-usage
> > 	;;
> > esac
> > -- 
> > 2.20.1
> >
  

Hello,

> On 14 Sep 2020, at 12:02, Jonatan Schlag <jonatan.schlag@ipfire.org> wrote:
> 
> Hi Michael,
> 
> this approach would be similar to the root file checker already
> included in the "tools directory.  Maybe this approach is now the
> better one because we can recheck this tool against existing correct
> root files, which should make it easy to find errors in the
> expressions.

Hmm… Two tools doing exactly the same does not seem to be a good idea.

> As such a tool should be well tested and I have to try out BDD
> (Behaviour Driven Development) I will try, to combine these and create
> a little software for this task. 

BDD? Someone has been paying attention in uni :)

Let’s discuss any ideas here first. You can prototype things of course, but do not fully build it out before you have collected a couple of opinions from other people, please.

Best,
-Michael

> I will report back when  I have results.
> 
> Jonatan
> 
> Am Donnerstag, den 10.09.2020, 12:45 +0100 schrieb Michael Tremer:
>> Hello Jonatan,
>> 
>> We have already talked about this patch on the last telephone
>> conference, but I would still like to put forward this:
>> 
>> Can we not change this patch, so that it only tells us what
>> could/should potentially be changed? So the command will simply say
>> things like “/usr/lib/libabc.so is a development file and should not
>> be included”.
>> 
>> Wouldn’t that avoid the problem of any automatic errors and we still
>> have a helper that makes the whole process easier for developers?
>> 
>> For that, the sed statements simply have to be changed to grep and
>> the output of that has to be dealt with.
>> 
>> -Michael
>> 
>>> On 5 Sep 2020, at 09:40, Jonatan Schlag <jonatan.schlag@ipfire.org>
>>> wrote:
>>> 
>>> There are some rules which files should be definitely not included
>>> in a
>>> rootfile. Applying these rules by hand is error prone and annoying.
>>> So I
>>> wrote these simple command, because computer can apply these rules
>>> faster and better than humans :-;
>>> 
>>> It removes all lines starting with '-' and comment out all lines
>>> which
>>> point to files which should not be included in the distribution.
>>> This
>>> list needs to be extended, but I think this is a good starting
>>> point to
>>> make the creating of rootfiles easier.
>>> 
>>> Signed-off-by: Jonatan Schlag <jonatan.schlag@ipfire.org>
>>> ---
>>> make.sh | 37 ++++++++++++++++++++++++++++++++++++-
>>> 1 file changed, 36 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/make.sh b/make.sh
>>> index 4a9dd3cb6..909d4a4de 100755
>>> --- a/make.sh
>>> +++ b/make.sh
>>> @@ -1895,6 +1895,41 @@ downloadsrc)
>>> 	fi
>>> 	cd - >/dev/null 2>&1
>>> 	;;
>>> +format-rootfile)
>>> +	if [ ! -f $2 ]; then
>>> +		echo -n "'$2' is not a regular file"
>>> +		beautify message FAIL
>>> +		exit 1
>>> +	fi
>>> +
>>> +	ROOTFILE_TO_FORMAT="$2"
>>> +
>>> +	# Remove all .so files in /usr/lib
>>> +	sed -i '/^\+usr\/lib\/.*\.so$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all .a files in /usr/lib
>>> +	sed -i '/^\+usr\/lib\/.*\.a$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +
>>> +	# Remove all .la files in /usr/lib
>>> +	sed -i '/^\+usr\/lib\/.*\.la$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all .hpp files in /usr/include
>>> +	sed -i '/^\+usr\/include\/.*\.hpp$/s/+/#/g'
>>> "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all .h files in /usr/include
>>> +	sed -i '/^\+usr\/include\/.*\.h$/s/+/#/g' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all .ipp files in /usr/include
>>> +	sed -i '/^\+usr\/include\/.*\.ipp$/s/+/#/g'
>>> "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all .m4 files in /usr/share/aclocal
>>> +	sed -i '/^\+usr\/share\/aclocal\/.*\.h$/s/+/#/g'
>>> "$ROOTFILE_TO_FORMAT"
>>> +
>>> +	# Remove all lines starting with -
>>> +	sed -i '/^-.*$/d' "$ROOTFILE_TO_FORMAT"
>>> +
>>> +;;
>>> toolchain)
>>> 	# Clear screen
>>> 	${INTERACTIVE} && clear
>>> @@ -1990,7 +2025,7 @@ find-dependencies)
>>> 	exec "${BASEDIR}/tools/find-dependencies" "${BASEDIR}/build"
>>> "$@"
>>> 	;;
>>> *)
>>> -	echo "Usage: $0
>>> {build|changelog|clean|gettoolchain|downloadsrc|shell|sync|toolchai
>>> n|update-contributors|find-dependencies}"
>>> +	echo "Usage: $0 {build|changelog|clean|format-
>>> rootfile|gettoolchain|downloadsrc|shell|sync|toolchain|update-
>>> contributors|find-dependencies}"
>>> 	cat doc/make.sh-usage
>>> 	;;
>>> esac
>>> -- 
>>> 2.20.1
>>> 
>