squid init v_final

  Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)

Changelog:

- Raised 'while'-loop-time for stopping squid to 360 seconds until
  '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
  I know, this timeout could not be enough for really BIG caches.
  The only other choice I know would be to leave this loop alone without
  the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
  But if it fails, I fear that we could end up in an endless loop.
  Suggestions are welcome!

- Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
  and redirect_wrappers - 'while'-loop counter wouldn't work because of
  this line.
  Suggestions for better handling?

- Process detection looks for leftover '(squid-1)'-process using 'pgrep'.

- Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
  (still) too much? I'd prefer to get a warning (an announcement) if
  '/var/log/cache/swap.state' was deleted.

- Changed the 'flush'-command to really delete the entire
  '/var/log/cache'-structure, it will automatically be rebuild
  during the next start.
  Reason: a CLEAR cache command should really CLEAR the cache!

Best,
Matthias

Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
---
 src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++----------------
 1 file changed, 35 insertions(+), 20 deletions(-)
  

Hi,

thanks. I finally merged this with a few changes:

  http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d81201591b2e8a76520579a4e7b7c5d46

and added some more changes:

  http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54b3f7fc8d6f3b98377b3c9a470
  http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab1886244a647dd7cf98f00e5c555

I used boot_mesg instead of echo in some situations. Refrased the warning
message. Removed -r from rm. And finally if squid shut down gracefully the shut
down time is logged to syslog instead of being written to the console.

I also added some more comments to the script so that we know what is going on
in a few months down from here.

This is working fine for me. Even with a small and almost empty cache squid
takes about 14 seconds to shut down on my IPFire Prime Box.

Please test everything again. I am going to deploy this on a few more test
systems this week.

Best,
-Michael

On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)
> 
> Changelog:
> 
> - Raised 'while'-loop-time for stopping squid to 360 seconds until
>   '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
>   I know, this timeout could not be enough for really BIG caches.
>   The only other choice I know would be to leave this loop alone without
>   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
>   But if it fails, I fear that we could end up in an endless loop.
>   Suggestions are welcome!
> 
> - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
>   and redirect_wrappers - 'while'-loop counter wouldn't work because of
>   this line.
>   Suggestions for better handling?
> 
> - Process detection looks for leftover '(squid-1)'-process using 'pgrep'.
> 
> - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
>   (still) too much? I'd prefer to get a warning (an announcement) if
>   '/var/log/cache/swap.state' was deleted.
> 
> - Changed the 'flush'-command to really delete the entire
>   '/var/log/cache'-structure, it will automatically be rebuild
>   during the next start.
>   Reason: a CLEAR cache command should really CLEAR the cache!
> 
> Best,
> Matthias
> 
> Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
> ---
>  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++---------------
> -
>  1 file changed, 35 insertions(+), 20 deletions(-)
> 
> diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid
> index abed90a..1b369ea 100644
> --- a/src/initscripts/init.d/squid
> +++ b/src/initscripts/init.d/squid
> @@ -94,9 +94,9 @@ case "$1" in
>  	stop)
>  			iptables -t nat -F SQUID
>  		if [ -e /var/run/squid.pid ]; then
> -			boot_mesg "Stopping Squid Proxy Server..."
> -			squid -k shutdown >/dev/null 2>&1
> -			evaluate_retval
> +			boot_mesg -n "Stopping Squid Proxy Server...\n"
> +			boot_mesg "(this may take up to a few minutes)"
> +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
>  
>  			# Stop squidGuard, updxlrator, squidclamav
>  			# and redirect_wrappers.
> @@ -105,25 +105,41 @@ case "$1" in
>  			killproc /usr/bin/squidclamav >/dev/null &
>  			killproc /usr/sbin/redirect_wrapper >/dev/null &
>  
> -			# Wait until all redirectors have been stopped.
> -			wait
> -
> -			# If squid is still running, wait up to 30 seconds
> -			# before we go on to kill it.
> -			counter=30
> -
> -			while [ ${counter} -gt 0 ]; do
> -				statusproc /usr/sbin/squid >/dev/null &&
> break;
> -				sleep 1
> -				counter=$(( ${counter} - 1))
> +			# If some squid processes are still running, wait up
> to 360 seconds
> +			# before we go on to kill the remaining process(es)
> and delete damaged
> +			# '/var/log/cache/swap.state'.
> +			n=0
> +			while squid -k check > /dev/null 2>&1 && [ $n -lt 360
> ]; do
> +				sleep 2
> +				n=$(( ${n} + 2 ))
> +				echo -n .
>  			done
> -
> -			# Kill squid service, if still running.
> -			killproc /usr/sbin/squid >/dev/null
> +			echo ""
> +			echo "Done."
> +			echo "Shutdown time:" "$n" "seconds"
> +
> +			# If (squid-1) is still running after 360 seconds,
> +			# kill all squid processes and delete
> '/var/log/cache/swap.state'.
> +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 ); then
> +				killproc /usr/sbin/squid >/dev/null
> +				rm -rf /var/log/cache/swap.state
> +				echo ""
> +				boot_mesg -n "You should not be reading this
> warning.\n"
> +				boot_mesg -n "Some squid-processes had to be
> killed after 360 seconds,\n"
> +				boot_mesg -n "so the cache index file was
> damaged and had to be deleted.\n"
> +				boot_mesg -n "This file will be rebuild
> during the next start."
> +				echo_warning
> +				echo ""
> +			else
> +				boot_mesg "All squid processes exited
> normally."
> +				echo_ok
> +				echo ""
> +			fi
> +		fi
>  
>  			# Trash remain pid file from squid.
>  			rm -rf /var/run/squid.pid
> -		fi
> +
>  		;;
>  
>  	restart)
> @@ -143,8 +159,7 @@ case "$1" in
>  
>  	flush)
>  		$0 stop
> -		echo > /var/log/cache/swap.state
> -		chown squid.squid /var/log/cache/swap.state
> +		rm -rf /var/log/cache/*
>  		sleep 1
>  		$0 start
>  		;;
  

Just coming back from work...

On 18.05.2016 20:15, Michael Tremer wrote:
> Hi,

Hi,
 
> thanks. I finally merged this with a few changes:
> 
>   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d81201591b2e8a76520579a4e7b7c5d46
> 
> and added some more changes:
> 
>   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54b3f7fc8d6f3b98377b3c9a470
>   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab1886244a647dd7cf98f00e5c555
> 
> I used boot_mesg instead of echo in some situations. Refrased the warning
> message. Removed -r from rm. And finally if squid shut down gracefully the shut
> down time is logged to syslog instead of being written to the console.

I took a quick look. For me, the warning message goes across the screen.
Nevertheless, message will clearly be seen.

That 'while'-counters look as if you got some cats running across your keyboard!? ;-)
Such things are "a bit" beyond my programming skills. Nice.

And I never thought of moving that d*** 'killproc'/'wait'-block to the bottom.
I'm really curious what 'squidclamav' will make of this.

> I also added some more comments to the script so that we know what is going on
> in a few months down from here.
> 
> This is working fine for me. Even with a small and almost empty cache squid
> takes about 14 seconds to shut down on my IPFire Prime Box.

14 seconds! Not bad.
I thought that it would take *some* time, but not that much.
One problem remains - what happens to really BIG caches.
I don't have a solution for that by now.

> Please test everything again. I am going to deploy this on a few more test
> systems this week.

Testmachine didn't complain - it did everything right while testing
with "360" and "3" seconds.

Now running on production machine.
Tomorrow I'll see what the logs really think of this. ;-)

Best, Matthias

> 
> Best,
> -Michael
> 
> On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
>> Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)
>> 
>> Changelog:
>> 
>> - Raised 'while'-loop-time for stopping squid to 360 seconds until
>>   '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
>>   I know, this timeout could not be enough for really BIG caches.
>>   The only other choice I know would be to leave this loop alone without
>>   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
>>   But if it fails, I fear that we could end up in an endless loop.
>>   Suggestions are welcome!
>> 
>> - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
>>   and redirect_wrappers - 'while'-loop counter wouldn't work because of
>>   this line.
>>   Suggestions for better handling?
>> 
>> - Process detection looks for leftover '(squid-1)'-process using 'pgrep'.
>> 
>> - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
>>   (still) too much? I'd prefer to get a warning (an announcement) if
>>   '/var/log/cache/swap.state' was deleted.
>> 
>> - Changed the 'flush'-command to really delete the entire
>>   '/var/log/cache'-structure, it will automatically be rebuild
>>   during the next start.
>>   Reason: a CLEAR cache command should really CLEAR the cache!
>> 
>> Best,
>> Matthias
>> 
>> Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
>> ---
>>  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++---------------
>> -
>>  1 file changed, 35 insertions(+), 20 deletions(-)
>> 
>> diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid
>> index abed90a..1b369ea 100644
>> --- a/src/initscripts/init.d/squid
>> +++ b/src/initscripts/init.d/squid
>> @@ -94,9 +94,9 @@ case "$1" in
>>  	stop)
>>  			iptables -t nat -F SQUID
>>  		if [ -e /var/run/squid.pid ]; then
>> -			boot_mesg "Stopping Squid Proxy Server..."
>> -			squid -k shutdown >/dev/null 2>&1
>> -			evaluate_retval
>> +			boot_mesg -n "Stopping Squid Proxy Server...\n"
>> +			boot_mesg "(this may take up to a few minutes)"
>> +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
>>  
>>  			# Stop squidGuard, updxlrator, squidclamav
>>  			# and redirect_wrappers.
>> @@ -105,25 +105,41 @@ case "$1" in
>>  			killproc /usr/bin/squidclamav >/dev/null &
>>  			killproc /usr/sbin/redirect_wrapper >/dev/null &
>>  
>> -			# Wait until all redirectors have been stopped.
>> -			wait
>> -
>> -			# If squid is still running, wait up to 30 seconds
>> -			# before we go on to kill it.
>> -			counter=30
>> -
>> -			while [ ${counter} -gt 0 ]; do
>> -				statusproc /usr/sbin/squid >/dev/null &&
>> break;
>> -				sleep 1
>> -				counter=$(( ${counter} - 1))
>> +			# If some squid processes are still running, wait up
>> to 360 seconds
>> +			# before we go on to kill the remaining process(es)
>> and delete damaged
>> +			# '/var/log/cache/swap.state'.
>> +			n=0
>> +			while squid -k check > /dev/null 2>&1 && [ $n -lt 360
>> ]; do
>> +				sleep 2
>> +				n=$(( ${n} + 2 ))
>> +				echo -n .
>>  			done
>> -
>> -			# Kill squid service, if still running.
>> -			killproc /usr/sbin/squid >/dev/null
>> +			echo ""
>> +			echo "Done."
>> +			echo "Shutdown time:" "$n" "seconds"
>> +
>> +			# If (squid-1) is still running after 360 seconds,
>> +			# kill all squid processes and delete
>> '/var/log/cache/swap.state'.
>> +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 ); then
>> +				killproc /usr/sbin/squid >/dev/null
>> +				rm -rf /var/log/cache/swap.state
>> +				echo ""
>> +				boot_mesg -n "You should not be reading this
>> warning.\n"
>> +				boot_mesg -n "Some squid-processes had to be
>> killed after 360 seconds,\n"
>> +				boot_mesg -n "so the cache index file was
>> damaged and had to be deleted.\n"
>> +				boot_mesg -n "This file will be rebuild
>> during the next start."
>> +				echo_warning
>> +				echo ""
>> +			else
>> +				boot_mesg "All squid processes exited
>> normally."
>> +				echo_ok
>> +				echo ""
>> +			fi
>> +		fi
>>  
>>  			# Trash remain pid file from squid.
>>  			rm -rf /var/run/squid.pid
>> -		fi
>> +
>>  		;;
>>  
>>  	restart)
>> @@ -143,8 +159,7 @@ case "$1" in
>>  
>>  	flush)
>>  		$0 stop
>> -		echo > /var/log/cache/swap.state
>> -		chown squid.squid /var/log/cache/swap.state
>> +		rm -rf /var/log/cache/*
>>  		sleep 1
>>  		$0 start
>>  		;;
>
  

Hi,

On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
> Just coming back from work...
> 
> On 18.05.2016 20:15, Michael Tremer wrote:
> > 
> > Hi,
> Hi,
>  
> > 
> > thanks. I finally merged this with a few changes:
> > 
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d81201591b2e
> > 8a76520579a4e7b7c5d46
> > 
> > and added some more changes:
> > 
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54b3f7
> > fc8d6f3b98377b3c9a470
> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab188624
> > 4a647dd7cf98f00e5c555
> > 
> > I used boot_mesg instead of echo in some situations. Refrased the warning
> > message. Removed -r from rm. And finally if squid shut down gracefully the
> > shut
> > down time is logged to syslog instead of being written to the console.
> I took a quick look. For me, the warning message goes across the screen.
> Nevertheless, message will clearly be seen.

Yes, it is kind of long, but will line-break then.

> That 'while'-counters look as if you got some cats running across your
> keyboard!? ;-)
> Such things are "a bit" beyond my programming skills. Nice.

That is just modulus. It calculates the remainder of a division of n by 6. If
that is 0, then a dot will be printed. That way every 6 cycles only one dot is
printed.

> And I never thought of moving that d*** 'killproc'/'wait'-block to the bottom.
> I'm really curious what 'squidclamav' will make of this.

Actually this should never do anything because squid should take care of killing
all processes. However we had issues with that in the past so that we are
killing those manually.

> > 
> > I also added some more comments to the script so that we know what is going
> > on
> > in a few months down from here.
> > 
> > This is working fine for me. Even with a small and almost empty cache squid
> > takes about 14 seconds to shut down on my IPFire Prime Box.
> 14 seconds! Not bad.
> I thought that it would take *some* time, but not that much.
> One problem remains - what happens to really BIG caches.
> I don't have a solution for that by now.

Well, they should be closed within 6 minutes. If not they will rebuild every
time.

But I think if it doesn't close within in 6 minutes your cache is way too big
for your hardware and should be downsized.

We might have to wait for some feedback.

> 
> > 
> > Please test everything again. I am going to deploy this on a few more test
> > systems this week.
> Testmachine didn't complain - it did everything right while testing
> with "360" and "3" seconds.
> 
> Now running on production machine.
> Tomorrow I'll see what the logs really think of this. ;-)

I installed it on one of the big test machines. Waiting for feedback...

Best,
-Michael

> 
> Best, Matthias
> 
> > 
> > 
> > Best,
> > -Michael
> > 
> > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> > > 
> > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)
> > > 
> > > Changelog:
> > > 
> > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
> > >   '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
> > >   I know, this timeout could not be enough for really BIG caches.
> > >   The only other choice I know would be to leave this loop alone without
> > >   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
> > >   But if it fails, I fear that we could end up in an endless loop.
> > >   Suggestions are welcome!
> > > 
> > > - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
> > >   and redirect_wrappers - 'while'-loop counter wouldn't work because of
> > >   this line.
> > >   Suggestions for better handling?
> > > 
> > > - Process detection looks for leftover '(squid-1)'-process using 'pgrep'.
> > > 
> > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
> > >   (still) too much? I'd prefer to get a warning (an announcement) if
> > >   '/var/log/cache/swap.state' was deleted.
> > > 
> > > - Changed the 'flush'-command to really delete the entire
> > >   '/var/log/cache'-structure, it will automatically be rebuild
> > >   during the next start.
> > >   Reason: a CLEAR cache command should really CLEAR the cache!
> > > 
> > > Best,
> > > Matthias
> > > 
> > > Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
> > > ---
> > >  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-----------
> > > ----
> > > -
> > >  1 file changed, 35 insertions(+), 20 deletions(-)
> > > 
> > > diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid
> > > index abed90a..1b369ea 100644
> > > --- a/src/initscripts/init.d/squid
> > > +++ b/src/initscripts/init.d/squid
> > > @@ -94,9 +94,9 @@ case "$1" in
> > >  	stop)
> > >  			iptables -t nat -F SQUID
> > >  		if [ -e /var/run/squid.pid ]; then
> > > -			boot_mesg "Stopping Squid Proxy Server..."
> > > -			squid -k shutdown >/dev/null 2>&1
> > > -			evaluate_retval
> > > +			boot_mesg -n "Stopping Squid Proxy Server...\n"
> > > +			boot_mesg "(this may take up to a few minutes)"
> > > +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
> > >  
> > >  			# Stop squidGuard, updxlrator, squidclamav
> > >  			# and redirect_wrappers.
> > > @@ -105,25 +105,41 @@ case "$1" in
> > >  			killproc /usr/bin/squidclamav >/dev/null &
> > >  			killproc /usr/sbin/redirect_wrapper >/dev/null &
> > >  
> > > -			# Wait until all redirectors have been stopped.
> > > -			wait
> > > -
> > > -			# If squid is still running, wait up to 30
> > > seconds
> > > -			# before we go on to kill it.
> > > -			counter=30
> > > -
> > > -			while [ ${counter} -gt 0 ]; do
> > > -				statusproc /usr/sbin/squid >/dev/null &&
> > > break;
> > > -				sleep 1
> > > -				counter=$(( ${counter} - 1))
> > > +			# If some squid processes are still running, wait
> > > up
> > > to 360 seconds
> > > +			# before we go on to kill the remaining
> > > process(es)
> > > and delete damaged
> > > +			# '/var/log/cache/swap.state'.
> > > +			n=0
> > > +			while squid -k check > /dev/null 2>&1 && [ $n -lt
> > > 360
> > > ]; do
> > > +				sleep 2
> > > +				n=$(( ${n} + 2 ))
> > > +				echo -n .
> > >  			done
> > > -
> > > -			# Kill squid service, if still running.
> > > -			killproc /usr/sbin/squid >/dev/null
> > > +			echo ""
> > > +			echo "Done."
> > > +			echo "Shutdown time:" "$n" "seconds"
> > > +
> > > +			# If (squid-1) is still running after 360
> > > seconds,
> > > +			# kill all squid processes and delete
> > > '/var/log/cache/swap.state'.
> > > +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 );
> > > then
> > > +				killproc /usr/sbin/squid >/dev/null
> > > +				rm -rf /var/log/cache/swap.state
> > > +				echo ""
> > > +				boot_mesg -n "You should not be reading
> > > this
> > > warning.\n"
> > > +				boot_mesg -n "Some squid-processes had to
> > > be
> > > killed after 360 seconds,\n"
> > > +				boot_mesg -n "so the cache index file was
> > > damaged and had to be deleted.\n"
> > > +				boot_mesg -n "This file will be rebuild
> > > during the next start."
> > > +				echo_warning
> > > +				echo ""
> > > +			else
> > > +				boot_mesg "All squid processes exited
> > > normally."
> > > +				echo_ok
> > > +				echo ""
> > > +			fi
> > > +		fi
> > >  
> > >  			# Trash remain pid file from squid.
> > >  			rm -rf /var/run/squid.pid
> > > -		fi
> > > +
> > >  		;;
> > >  
> > >  	restart)
> > > @@ -143,8 +159,7 @@ case "$1" in
> > >  
> > >  	flush)
> > >  		$0 stop
> > > -		echo > /var/log/cache/swap.state
> > > -		chown squid.squid /var/log/cache/swap.state
> > > +		rm -rf /var/log/cache/*
> > >  		sleep 1
> > >  		$0 start
> > >  		;;
  

Hi,

first feedback from here:

Works. ;-)

No squidclamav-crashes, clean log.

Max Mem size: 32768 KB
Max Swap size: 262144 KB
~3000 Objects (not much in cache)

Took about 8 seconds for stopping '(squid-1)'.

I'll watch this during the next days.

Best,
Matthias

On 18.05.2016 23:22, Michael Tremer wrote:
> Hi,
> 
> On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
>> Just coming back from work...
>> 
>> On 18.05.2016 20:15, Michael Tremer wrote:
>> > 
>> > Hi,
>> Hi,
>>  
>> > 
>> > thanks. I finally merged this with a few changes:
>> > 
>> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d81201591b2e
>> > 8a76520579a4e7b7c5d46
>> > 
>> > and added some more changes:
>> > 
>> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54b3f7
>> > fc8d6f3b98377b3c9a470
>> >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab188624
>> > 4a647dd7cf98f00e5c555
>> > 
>> > I used boot_mesg instead of echo in some situations. Refrased the warning
>> > message. Removed -r from rm. And finally if squid shut down gracefully the
>> > shut
>> > down time is logged to syslog instead of being written to the console.
>> I took a quick look. For me, the warning message goes across the screen.
>> Nevertheless, message will clearly be seen.
> 
> Yes, it is kind of long, but will line-break then.
> 
>> That 'while'-counters look as if you got some cats running across your
>> keyboard!? ;-)
>> Such things are "a bit" beyond my programming skills. Nice.
> 
> That is just modulus. It calculates the remainder of a division of n by 6. If
> that is 0, then a dot will be printed. That way every 6 cycles only one dot is
> printed.
> 
>> And I never thought of moving that d*** 'killproc'/'wait'-block to the bottom.
>> I'm really curious what 'squidclamav' will make of this.
> 
> Actually this should never do anything because squid should take care of killing
> all processes. However we had issues with that in the past so that we are
> killing those manually.
> 
>> > 
>> > I also added some more comments to the script so that we know what is going
>> > on
>> > in a few months down from here.
>> > 
>> > This is working fine for me. Even with a small and almost empty cache squid
>> > takes about 14 seconds to shut down on my IPFire Prime Box.
>> 14 seconds! Not bad.
>> I thought that it would take *some* time, but not that much.
>> One problem remains - what happens to really BIG caches.
>> I don't have a solution for that by now.
> 
> Well, they should be closed within 6 minutes. If not they will rebuild every
> time.
> 
> But I think if it doesn't close within in 6 minutes your cache is way too big
> for your hardware and should be downsized.
> 
> We might have to wait for some feedback.
> 
>> 
>> > 
>> > Please test everything again. I am going to deploy this on a few more test
>> > systems this week.
>> Testmachine didn't complain - it did everything right while testing
>> with "360" and "3" seconds.
>> 
>> Now running on production machine.
>> Tomorrow I'll see what the logs really think of this. ;-)
> 
> I installed it on one of the big test machines. Waiting for feedback...
> 
> Best,
> -Michael
> 
>> 
>> Best, Matthias
>> 
>> > 
>> > 
>> > Best,
>> > -Michael
>> > 
>> > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
>> > > 
>> > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-)
>> > > 
>> > > Changelog:
>> > > 
>> > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
>> > >   '/var/log/cache/swap.state' is deleted. NOT the whole cache structure!
>> > >   I know, this timeout could not be enough for really BIG caches.
>> > >   The only other choice I know would be to leave this loop alone without
>> > >   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles this.
>> > >   But if it fails, I fear that we could end up in an endless loop.
>> > >   Suggestions are welcome!
>> > > 
>> > > - Had to delete 'wait' after killing squidguard, updxlrator, squidclamav
>> > >   and redirect_wrappers - 'while'-loop counter wouldn't work because of
>> > >   this line.
>> > >   Suggestions for better handling?
>> > > 
>> > > - Process detection looks for leftover '(squid-1)'-process using 'pgrep'.
>> > > 
>> > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
>> > >   (still) too much? I'd prefer to get a warning (an announcement) if
>> > >   '/var/log/cache/swap.state' was deleted.
>> > > 
>> > > - Changed the 'flush'-command to really delete the entire
>> > >   '/var/log/cache'-structure, it will automatically be rebuild
>> > >   during the next start.
>> > >   Reason: a CLEAR cache command should really CLEAR the cache!
>> > > 
>> > > Best,
>> > > Matthias
>> > > 
>> > > Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
>> > > ---
>> > >  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-----------
>> > > ----
>> > > -
>> > >  1 file changed, 35 insertions(+), 20 deletions(-)
>> > > 
>> > > diff --git a/src/initscripts/init.d/squid b/src/initscripts/init.d/squid
>> > > index abed90a..1b369ea 100644
>> > > --- a/src/initscripts/init.d/squid
>> > > +++ b/src/initscripts/init.d/squid
>> > > @@ -94,9 +94,9 @@ case "$1" in
>> > >  	stop)
>> > >  			iptables -t nat -F SQUID
>> > >  		if [ -e /var/run/squid.pid ]; then
>> > > -			boot_mesg "Stopping Squid Proxy Server..."
>> > > -			squid -k shutdown >/dev/null 2>&1
>> > > -			evaluate_retval
>> > > +			boot_mesg -n "Stopping Squid Proxy Server...\n"
>> > > +			boot_mesg "(this may take up to a few minutes)"
>> > > +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
>> > >  
>> > >  			# Stop squidGuard, updxlrator, squidclamav
>> > >  			# and redirect_wrappers.
>> > > @@ -105,25 +105,41 @@ case "$1" in
>> > >  			killproc /usr/bin/squidclamav >/dev/null &
>> > >  			killproc /usr/sbin/redirect_wrapper >/dev/null &
>> > >  
>> > > -			# Wait until all redirectors have been stopped.
>> > > -			wait
>> > > -
>> > > -			# If squid is still running, wait up to 30
>> > > seconds
>> > > -			# before we go on to kill it.
>> > > -			counter=30
>> > > -
>> > > -			while [ ${counter} -gt 0 ]; do
>> > > -				statusproc /usr/sbin/squid >/dev/null &&
>> > > break;
>> > > -				sleep 1
>> > > -				counter=$(( ${counter} - 1))
>> > > +			# If some squid processes are still running, wait
>> > > up
>> > > to 360 seconds
>> > > +			# before we go on to kill the remaining
>> > > process(es)
>> > > and delete damaged
>> > > +			# '/var/log/cache/swap.state'.
>> > > +			n=0
>> > > +			while squid -k check > /dev/null 2>&1 && [ $n -lt
>> > > 360
>> > > ]; do
>> > > +				sleep 2
>> > > +				n=$(( ${n} + 2 ))
>> > > +				echo -n .
>> > >  			done
>> > > -
>> > > -			# Kill squid service, if still running.
>> > > -			killproc /usr/sbin/squid >/dev/null
>> > > +			echo ""
>> > > +			echo "Done."
>> > > +			echo "Shutdown time:" "$n" "seconds"
>> > > +
>> > > +			# If (squid-1) is still running after 360
>> > > seconds,
>> > > +			# kill all squid processes and delete
>> > > '/var/log/cache/swap.state'.
>> > > +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1 );
>> > > then
>> > > +				killproc /usr/sbin/squid >/dev/null
>> > > +				rm -rf /var/log/cache/swap.state
>> > > +				echo ""
>> > > +				boot_mesg -n "You should not be reading
>> > > this
>> > > warning.\n"
>> > > +				boot_mesg -n "Some squid-processes had to
>> > > be
>> > > killed after 360 seconds,\n"
>> > > +				boot_mesg -n "so the cache index file was
>> > > damaged and had to be deleted.\n"
>> > > +				boot_mesg -n "This file will be rebuild
>> > > during the next start."
>> > > +				echo_warning
>> > > +				echo ""
>> > > +			else
>> > > +				boot_mesg "All squid processes exited
>> > > normally."
>> > > +				echo_ok
>> > > +				echo ""
>> > > +			fi
>> > > +		fi
>> > >  
>> > >  			# Trash remain pid file from squid.
>> > >  			rm -rf /var/run/squid.pid
>> > > -		fi
>> > > +
>> > >  		;;
>> > >  
>> > >  	restart)
>> > > @@ -143,8 +159,7 @@ case "$1" in
>> > >  
>> > >  	flush)
>> > >  		$0 stop
>> > > -		echo > /var/log/cache/swap.state
>> > > -		chown squid.squid /var/log/cache/swap.state
>> > > +		rm -rf /var/log/cache/*
>> > >  		sleep 1
>> > >  		$0 start
>> > >  		;;
>
  

Looking good on my end, too.

We will have to wait for Monday when all the students are coming back.

-Michael

On Thu, 2016-05-19 at 21:03 +0200, Matthias Fischer wrote:
> Hi,
> 
> first feedback from here:
> 
> Works. ;-)
> 
> No squidclamav-crashes, clean log.
> 
> Max Mem size: 32768 KB
> Max Swap size: 262144 KB
> ~3000 Objects (not much in cache)
> 
> Took about 8 seconds for stopping '(squid-1)'.
> 
> I'll watch this during the next days.
> 
> Best,
> Matthias
> 
> On 18.05.2016 23:22, Michael Tremer wrote:
> > 
> > Hi,
> > 
> > On Wed, 2016-05-18 at 21:03 +0200, Matthias Fischer wrote:
> > > 
> > > Just coming back from work...
> > > 
> > > On 18.05.2016 20:15, Michael Tremer wrote:
> > > > 
> > > > 
> > > > Hi,
> > > Hi,
> > >  
> > > > 
> > > > 
> > > > thanks. I finally merged this with a few changes:
> > > > 
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=6113575d8120159
> > > > 1b2e
> > > > 8a76520579a4e7b7c5d46
> > > > 
> > > > and added some more changes:
> > > > 
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=691b2836c0ffa54
> > > > b3f7
> > > > fc8d6f3b98377b3c9a470
> > > >   http://git.ipfire.org/?p=ipfire-2.x.git;a=commitdiff;h=3fcc9b67fc7ab18
> > > > 8624
> > > > 4a647dd7cf98f00e5c555
> > > > 
> > > > I used boot_mesg instead of echo in some situations. Refrased the
> > > > warning
> > > > message. Removed -r from rm. And finally if squid shut down gracefully
> > > > the
> > > > shut
> > > > down time is logged to syslog instead of being written to the console.
> > > I took a quick look. For me, the warning message goes across the screen.
> > > Nevertheless, message will clearly be seen.
> > Yes, it is kind of long, but will line-break then.
> > 
> > > 
> > > That 'while'-counters look as if you got some cats running across your
> > > keyboard!? ;-)
> > > Such things are "a bit" beyond my programming skills. Nice.
> > That is just modulus. It calculates the remainder of a division of n by 6.
> > If
> > that is 0, then a dot will be printed. That way every 6 cycles only one dot
> > is
> > printed.
> > 
> > > 
> > > And I never thought of moving that d*** 'killproc'/'wait'-block to the
> > > bottom.
> > > I'm really curious what 'squidclamav' will make of this.
> > Actually this should never do anything because squid should take care of
> > killing
> > all processes. However we had issues with that in the past so that we are
> > killing those manually.
> > 
> > > 
> > > > 
> > > > 
> > > > I also added some more comments to the script so that we know what is
> > > > going
> > > > on
> > > > in a few months down from here.
> > > > 
> > > > This is working fine for me. Even with a small and almost empty cache
> > > > squid
> > > > takes about 14 seconds to shut down on my IPFire Prime Box.
> > > 14 seconds! Not bad.
> > > I thought that it would take *some* time, but not that much.
> > > One problem remains - what happens to really BIG caches.
> > > I don't have a solution for that by now.
> > Well, they should be closed within 6 minutes. If not they will rebuild every
> > time.
> > 
> > But I think if it doesn't close within in 6 minutes your cache is way too
> > big
> > for your hardware and should be downsized.
> > 
> > We might have to wait for some feedback.
> > 
> > > 
> > > 
> > > > 
> > > > 
> > > > Please test everything again. I am going to deploy this on a few more
> > > > test
> > > > systems this week.
> > > Testmachine didn't complain - it did everything right while testing
> > > with "360" and "3" seconds.
> > > 
> > > Now running on production machine.
> > > Tomorrow I'll see what the logs really think of this. ;-)
> > I installed it on one of the big test machines. Waiting for feedback...
> > 
> > Best,
> > -Michael
> > 
> > > 
> > > 
> > > Best, Matthias
> > > 
> > > > 
> > > > 
> > > > 
> > > > Best,
> > > > -Michael
> > > > 
> > > > On Tue, 2016-05-17 at 21:33 +0200, Matthias Fischer wrote:
> > > > > 
> > > > > 
> > > > > Last update suggestions for (3.5.xx) initscript, awaiting feedback. ;-
> > > > > )
> > > > > 
> > > > > Changelog:
> > > > > 
> > > > > - Raised 'while'-loop-time for stopping squid to 360 seconds until
> > > > >   '/var/log/cache/swap.state' is deleted. NOT the whole cache
> > > > > structure!
> > > > >   I know, this timeout could not be enough for really BIG caches.
> > > > >   The only other choice I know would be to leave this loop alone
> > > > > without
> > > > >   the "xxx seconds"-counter and wait how 'squid -k shutdown' handles
> > > > > this.
> > > > >   But if it fails, I fear that we could end up in an endless loop.
> > > > >   Suggestions are welcome!
> > > > > 
> > > > > - Had to delete 'wait' after killing squidguard, updxlrator,
> > > > > squidclamav
> > > > >   and redirect_wrappers - 'while'-loop counter wouldn't work because
> > > > > of
> > > > >   this line.
> > > > >   Suggestions for better handling?
> > > > > 
> > > > > - Process detection looks for leftover '(squid-1)'-process using
> > > > > 'pgrep'.
> > > > > 
> > > > > - Cosmetic changes to some 'boot_mesg' lines. Added a few. Is it
> > > > >   (still) too much? I'd prefer to get a warning (an announcement) if
> > > > >   '/var/log/cache/swap.state' was deleted.
> > > > > 
> > > > > - Changed the 'flush'-command to really delete the entire
> > > > >   '/var/log/cache'-structure, it will automatically be rebuild
> > > > >   during the next start.
> > > > >   Reason: a CLEAR cache command should really CLEAR the cache!
> > > > > 
> > > > > Best,
> > > > > Matthias
> > > > > 
> > > > > Signed-off-by: Matthias Fischer <matthias.fischer@ipfire.org>
> > > > > ---
> > > > >  src/initscripts/init.d/squid | 55 ++++++++++++++++++++++++++++-------
> > > > > ----
> > > > > ----
> > > > > -
> > > > >  1 file changed, 35 insertions(+), 20 deletions(-)
> > > > > 
> > > > > diff --git a/src/initscripts/init.d/squid
> > > > > b/src/initscripts/init.d/squid
> > > > > index abed90a..1b369ea 100644
> > > > > --- a/src/initscripts/init.d/squid
> > > > > +++ b/src/initscripts/init.d/squid
> > > > > @@ -94,9 +94,9 @@ case "$1" in
> > > > >  	stop)
> > > > >  			iptables -t nat -F SQUID
> > > > >  		if [ -e /var/run/squid.pid ]; then
> > > > > -			boot_mesg "Stopping Squid Proxy Server..."
> > > > > -			squid -k shutdown >/dev/null 2>&1
> > > > > -			evaluate_retval
> > > > > +			boot_mesg -n "Stopping Squid Proxy
> > > > > Server...\n"
> > > > > +			boot_mesg "(this may take up to a few
> > > > > minutes)"
> > > > > +			/usr/sbin/squid -k shutdown >/dev/null 2>&1
> > > > >  
> > > > >  			# Stop squidGuard, updxlrator, squidclamav
> > > > >  			# and redirect_wrappers.
> > > > > @@ -105,25 +105,41 @@ case "$1" in
> > > > >  			killproc /usr/bin/squidclamav >/dev/null &
> > > > >  			killproc /usr/sbin/redirect_wrapper
> > > > > >/dev/null &
> > > > >  
> > > > > -			# Wait until all redirectors have been
> > > > > stopped.
> > > > > -			wait
> > > > > -
> > > > > -			# If squid is still running, wait up to 30
> > > > > seconds
> > > > > -			# before we go on to kill it.
> > > > > -			counter=30
> > > > > -
> > > > > -			while [ ${counter} -gt 0 ]; do
> > > > > -				statusproc /usr/sbin/squid >/dev/null
> > > > > &&
> > > > > break;
> > > > > -				sleep 1
> > > > > -				counter=$(( ${counter} - 1))
> > > > > +			# If some squid processes are still running,
> > > > > wait
> > > > > up
> > > > > to 360 seconds
> > > > > +			# before we go on to kill the remaining
> > > > > process(es)
> > > > > and delete damaged
> > > > > +			# '/var/log/cache/swap.state'.
> > > > > +			n=0
> > > > > +			while squid -k check > /dev/null 2>&1 && [ $n
> > > > > -lt
> > > > > 360
> > > > > ]; do
> > > > > +				sleep 2
> > > > > +				n=$(( ${n} + 2 ))
> > > > > +				echo -n .
> > > > >  			done
> > > > > -
> > > > > -			# Kill squid service, if still running.
> > > > > -			killproc /usr/sbin/squid >/dev/null
> > > > > +			echo ""
> > > > > +			echo "Done."
> > > > > +			echo "Shutdown time:" "$n" "seconds"
> > > > > +
> > > > > +			# If (squid-1) is still running after 360
> > > > > seconds,
> > > > > +			# kill all squid processes and delete
> > > > > '/var/log/cache/swap.state'.
> > > > > +			if ( pgrep -fl "(squid-1)" > /dev/null 2>&1
> > > > > );
> > > > > then
> > > > > +				killproc /usr/sbin/squid >/dev/null
> > > > > +				rm -rf /var/log/cache/swap.state
> > > > > +				echo ""
> > > > > +				boot_mesg -n "You should not be
> > > > > reading
> > > > > this
> > > > > warning.\n"
> > > > > +				boot_mesg -n "Some squid-processes
> > > > > had to
> > > > > be
> > > > > killed after 360 seconds,\n"
> > > > > +				boot_mesg -n "so the cache index file
> > > > > was
> > > > > damaged and had to be deleted.\n"
> > > > > +				boot_mesg -n "This file will be
> > > > > rebuild
> > > > > during the next start."
> > > > > +				echo_warning
> > > > > +				echo ""
> > > > > +			else
> > > > > +				boot_mesg "All squid processes exited
> > > > > normally."
> > > > > +				echo_ok
> > > > > +				echo ""
> > > > > +			fi
> > > > > +		fi
> > > > >  
> > > > >  			# Trash remain pid file from squid.
> > > > >  			rm -rf /var/run/squid.pid
> > > > > -		fi
> > > > > +
> > > > >  		;;
> > > > >  
> > > > >  	restart)
> > > > > @@ -143,8 +159,7 @@ case "$1" in
> > > > >  
> > > > >  	flush)
> > > > >  		$0 stop
> > > > > -		echo > /var/log/cache/swap.state
> > > > > -		chown squid.squid /var/log/cache/swap.state
> > > > > +		rm -rf /var/log/cache/*
> > > > >  		sleep 1
> > > > >  		$0 start
> > > > >  		;;