| Message ID | 1555236523-3509-1-git-send-email-ipfire@starkstromkonsument.de |
|---|---|
| State | Accepted |
| Commit | ddc5602ac6674b5ede85068bcad16528199d2bfe |
| Headers |
Return-Path: <development-bounces@lists.ipfire.org> Received: from mail01.ipfire.org (mail01.i.ipfire.org [172.28.1.200]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail01.ipfire.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by web07.i.ipfire.org (Postfix) with ESMTPS id D7A3488B51F for <patchwork@web07.i.ipfire.org>; Sun, 14 Apr 2019 11:08:59 +0100 (BST) Received: from mail01.i.ipfire.org (localhost [IPv6:::1]) by mail01.ipfire.org (Postfix) with ESMTP id 44hnQy5zwsz55JnN; Sun, 14 Apr 2019 11:08:58 +0100 (BST) Received: from nx112.node02.secure-mailgate.com (nx112.node02.secure-mailgate.com [192.162.87.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mail01.ipfire.org (Postfix) with ESMTPS id 44hnQv6YVdz55Jn5 for <development@lists.ipfire.org>; Sun, 14 Apr 2019 11:08:55 +0100 (BST) Received: from dehamd003.servertools24.de ([31.47.254.18]) by node02.secure-mailgate.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.89) (envelope-from <ipfire@starkstromkonsument.de>) id 1hFc47-0003iI-M6 for development@lists.ipfire.org; Sun, 14 Apr 2019 12:08:48 +0200 Received: from balthasar.starkstromlahn.spdns.org (dslb-084-058-233-099.084.058.pools.vodafone-ip.de [84.58.233.99]) by dehamd003.servertools24.de (Postfix) with ESMTPSA id C59878272F for <development@lists.ipfire.org>; Sun, 14 Apr 2019 12:08:46 +0200 (CEST) From: Alexander Koch <ipfire@starkstromkonsument.de> To: development@lists.ipfire.org Subject: [PATCH] squid / WPAD: Add exception-files for generation of proxy.pac Date: Sun, 14 Apr 2019 12:08:43 +0200 Message-Id: <1555236523-3509-1-git-send-email-ipfire@starkstromkonsument.de> X-Mailer: git-send-email 2.7.4 X-Originating-IP: 31.47.254.18 X-SecureMailgate-Domain: dehamd003.servertools24.de X-SecureMailgate-Username: 31.47.254.18 X-SecureMailgate-Outgoing-Class: ham X-SecureMailgate-Outgoing-Evidence: Combined (0.05) X-Recommended-Action: accept X-Filter-ID: EX5BVjFpneJeBchSMxfU5iSjwZDk6zQhatR23hJtHQh602E9L7XzfQH6nu9C/Fh9KJzpNe6xgvOx q3u0UDjvO234slfrnEdiMqZNFIl3KV30wqEzzsz0qey7Lab92ZzcVTGbEWWnE2yTxqfzAQugcZ+y dutNMSJpU9Vfrqx3tQ1IeZjZv1oz6oWKgngYgisMFP/fMVP0svmpWqdKA4I3N3DrsgLoQIlol112 md4PmZ3JlwazEuJ5FLeAveAcO7l0w+pJdu/U5y6mAHsHsNznwP0sNOmRi6cQXeybw4h5I237Nx2K MWdgPAi++APAaUY2H/cuh5evaj1lUzaVjWgVraO5TPqzGal/PlyCuaPMa+bGc8PaqZZ/FzEG53Yo 1Az82WYSex+Yt07uCbJsusiJ1enFl6U7B/q9vr26RhU4BBU/xuIZVofF0iEmlfZIX9+tP9OlDLyR N9GjtiXFVLi3Vl19D65Nxn6Sz8wAkuPxgFgLMK9HC+pUQBAyR03+Yzd8UhwxKCq1wptT0tr4AeII UH2+5gx4OhewwkMP0JkVbYMdd65ri6y7K51mXtklaaDlrSOIPpeqwlm2NDGXIJ2x7OHYlJMxQPrm gMtcg8AzxfsiDo+IpU12FumWzopltXdKYBWhV+tPoIzjsCN3CmE6sf1PeOWddjCtOmdRMrZN8jgM yeNRq1jbE+1JbC4bXhPaBnH7tqS9e5bS4qA/jFxLRoV9VXyPpwhrSnOhfpMqWhrxRIfd0bAuQbjq t+XCpWwA0Q8kyfrtwrpahI8V1plAkerePyqbXbof/9laAGxPqvJ+8+AaVO4XC3/UMaxMESTW8ZZD w6MG3E9U62qnI+GE+vFq5rhWpcEtzkb8VVIzoQ3rjuU3A3io6xJmD59iFVkbj3Ez2xu8SRHhFCYw xJ6gkcv/S4qaprz9Djv6d2Y4kBdsIH8EF3joBheiindVvWjMwqLzoKXgT5Up7fvpQRUuERvipbQ5 EcjyCGH1YtofXcIDWDyg/Q/09Ieu7aO1pJGxeyyFY8GlmzRmIAOnS+GEX99IXHPnR/WBAxQU9Pct nno= X-Report-Abuse-To: spam@node01.secure-mailgate.com Authentication-Results: mail01.ipfire.org; dkim=none; dmarc=none; spf=pass (mail01.ipfire.org: domain of ipfire@starkstromkonsument.de designates 192.162.87.112 as permitted sender) smtp.mailfrom=ipfire@starkstromkonsument.de X-Rspamd-Queue-Id: 44hnQv6YVdz55Jn5 X-Spamd-Result: default: False [-3.32 / 11.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:192.162.87.0/24]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[development@lists.ipfire.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DMARC_NA(0.00)[starkstromkonsument.de]; MX_GOOD(-0.01)[mail.starkstromkonsument.de]; RCVD_IN_DNSWL_NONE(0.00)[112.87.162.192.list.dnswl.org : 127.0.5.0]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM(-2.47)[-0.822,0]; IP_SCORE(-0.55)[asn: 45031(-2.64), country: DE(-0.09)]; RECEIVED_SPAMHAUS_PBL(0.00)[99.233.58.84.zen.spamhaus.org : 127.0.0.10]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:45031, ipnet:192.162.84.0/22, country:DE]; RCVD_TLS_ALL(0.00)[]; BAYES_HAM(-0.99)[87.11%]; FROM_EQ_ENVFROM(0.00)[] X-Rspamd-Server: mail01.i.ipfire.org X-BeenThere: development@lists.ipfire.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: IPFire development talk <development.lists.ipfire.org> List-Unsubscribe: <https://lists.ipfire.org/mailman/options/development>, <mailto:development-request@lists.ipfire.org?subject=unsubscribe> List-Archive: <https://lists.ipfire.org/pipermail/development/> List-Post: <mailto:development@lists.ipfire.org> List-Help: <mailto:development-request@lists.ipfire.org?subject=help> List-Subscribe: <https://lists.ipfire.org/mailman/listinfo/development>, <mailto:development-request@lists.ipfire.org?subject=subscribe> Errors-To: development-bounces@lists.ipfire.org Sender: "Development" <development-bounces@lists.ipfire.org> |
| Series |
squid / WPAD: Add exception-files for generation of proxy.pac
|
|
Commit Message
Alexander Koch
April 14, 2019, 8:08 p.m. UTC
This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files:
- /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl
- /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl
as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri
These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released.
Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de>
---
html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++
1 file changed, 39 insertions(+)
Comments
Hello Alex, Thanks for submitting the patch. I guess the code looks fine, but where is the UI? Why should this not be configurable on the web interface? -Michael > On 14 Apr 2019, at 11:08, Alexander Koch <ipfire@starkstromkonsument.de> wrote: > > This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files: > > - /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl > - /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl > > as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri > > These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released. > > Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de> > --- > html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++ > 1 file changed, 39 insertions(+) > > diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi > index 6daa7fb..369a5cb 100644 > --- a/html/cgi-bin/proxy.cgi > +++ b/html/cgi-bin/proxy.cgi > @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; > my $acl_ports_ssl = "$acldir/ports_ssl.acl"; > my $acl_include = "$acldir/include.acl"; > > +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; > +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; > + > my $updaccelversion = 'n/a'; > my $urlfilterversion = 'n/a'; > > @@ -2763,6 +2766,42 @@ END > print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; > } > > + # Additional exceptions for URLs > + # The file has to be created by the user and should contain one entry per line > + # Line-Format: <URL incl. wildcards> > + # e.g. *ipfire.org* > + if (-s "$acl_dst_noproxy_url") { > + undef @templist; > + > + open(NOPROXY,"$acl_dst_noproxy_url"); > + @templist = <NOPROXY>; > + close(NOPROXY); > + chomp (@templist); > + > + foreach (@templist) > + { > + print FILE " (shExpMatch(url, \"$_\")) ||\n"; > + } > + } > + > + # Additional exceptions for Subnets > + # The file has to be created by the user and should contain one entry per line > + # Line-Format: "<IP>", "<SUBNET MASK>" > + # e.g. "192.168.0.0", "255.255.255.0" > + if (-s "$acl_dst_noproxy_ip") { > + undef @templist; > + > + open(NOPROXY,"$acl_dst_noproxy_ip"); > + @templist = <NOPROXY>; > + close(NOPROXY); > + chomp (@templist); > + > + foreach (@templist) > + { > + print FILE " (isInNet(host, $_)) ||\n"; > + } > + } > + > print FILE <<END > (isInNet(host, "169.254.0.0", "255.255.0.0")) > ) > -- > 2.7.4 >
Hello Michael, my motivation for the patch is to provide a possibility to make exceptions survive an update of squid, as I'm repatching proxy.cgi by myself after each upgrade. I suppose there are more people out there with the same issue. I agree that it would by very nice to have it on the GUI as well, but unfortunately I don't have any experience with CGI yet and I don't have the time to learn it right now. I think patching the integration of the exception files into proxy.cgi is a good first step. It can be used as the base for extending the GUI. Maybe somebody else with CGI experience can help out? It's "just" two textareas and some file i/o basically... As far as I know, the WPAD-Feature does not have any GUI support in general (e.g. checkboxes for enabled, enabled on a per subnet basis, etc.) until now. Additionally the WPAD-Feature requires the user to set up the extra apache-vhost or haproxy-frontend for port 80 (for http://wpad.<IPFire-Network-Domain>/wpad.dat) via CLI by himself anyway (another ToDo for a future patch ;-). Having this said, I think it is reasonable for the users to maintain their exceptions via CLI in the first instance until a GUI is available. Usually these things are not changed very often. It is still better than having to fix them after each upgrade of proxy.cgi If nobody else grabs this, I might possibly come back to it by myself at a later date. Should I write a bug report for the WPAD-GUI feature request? Best regards, Alex Am 15.04.2019 um 11:43 schrieb Michael Tremer: > Hello Alex, > > Thanks for submitting the patch. > > I guess the code looks fine, but where is the UI? > > Why should this not be configurable on the web interface? > > -Michael > >> On 14 Apr 2019, at 11:08, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >> >> This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files: >> >> - /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl >> - /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl >> >> as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri >> >> These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released. >> >> Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de> >> --- >> html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++ >> 1 file changed, 39 insertions(+) >> >> diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi >> index 6daa7fb..369a5cb 100644 >> --- a/html/cgi-bin/proxy.cgi >> +++ b/html/cgi-bin/proxy.cgi >> @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; >> my $acl_ports_ssl = "$acldir/ports_ssl.acl"; >> my $acl_include = "$acldir/include.acl"; >> >> +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; >> +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; >> + >> my $updaccelversion = 'n/a'; >> my $urlfilterversion = 'n/a'; >> >> @@ -2763,6 +2766,42 @@ END >> print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; >> } >> >> + # Additional exceptions for URLs >> + # The file has to be created by the user and should contain one entry per line >> + # Line-Format: <URL incl. wildcards> >> + # e.g. *ipfire.org* >> + if (-s "$acl_dst_noproxy_url") { >> + undef @templist; >> + >> + open(NOPROXY,"$acl_dst_noproxy_url"); >> + @templist = <NOPROXY>; >> + close(NOPROXY); >> + chomp (@templist); >> + >> + foreach (@templist) >> + { >> + print FILE " (shExpMatch(url, \"$_\")) ||\n"; >> + } >> + } >> + >> + # Additional exceptions for Subnets >> + # The file has to be created by the user and should contain one entry per line >> + # Line-Format: "<IP>", "<SUBNET MASK>" >> + # e.g. "192.168.0.0", "255.255.255.0" >> + if (-s "$acl_dst_noproxy_ip") { >> + undef @templist; >> + >> + open(NOPROXY,"$acl_dst_noproxy_ip"); >> + @templist = <NOPROXY>; >> + close(NOPROXY); >> + chomp (@templist); >> + >> + foreach (@templist) >> + { >> + print FILE " (isInNet(host, $_)) ||\n"; >> + } >> + } >> + >> print FILE <<END >> (isInNet(host, "169.254.0.0", "255.255.0.0")) >> ) >> -- >> 2.7.4 >> >
Hi, > On 15 Apr 2019, at 21:12, Alexander Koch <ipfire@starkstromkonsument.de> wrote: > > Hello Michael, > > my motivation for the patch is to provide a possibility to make exceptions survive an update of squid, as I'm repatching proxy.cgi by myself after each upgrade. I suppose there are more people out there with the same issue. I agree that it would by very nice to have it on the GUI as well, but unfortunately I don't have any experience with CGI yet and I don't have the time to learn it right now. I think patching the integration of the exception files into proxy.cgi is a good first step. It can be used as the base for extending the GUI. Maybe somebody else with CGI experience can help out? It's "just" two textareas and some file i/o basically… You can literally just copy and paste that. Give it a try! > As far as I know, the WPAD-Feature does not have any GUI support in general (e.g. checkboxes for enabled, enabled on a per subnet basis, etc.) until now. Additionally the WPAD-Feature requires the user to set up the extra apache-vhost or haproxy-frontend for port 80 (for http://wpad.<IPFire-Network-Domain>/wpad.dat) via CLI by himself anyway (another ToDo for a future patch ;-). It is available on http://<ipfire>:81/wpad.dat. No need for an extra host. > Having this said, I think it is reasonable for the users to maintain their exceptions via CLI in the first instance until a GUI is available. Usually these things are not changed very often. It is still better than having to fix them after each upgrade of proxy.cgi If nobody else grabs this, I might possibly come back to it by myself at a later date. > > Should I write a bug report for the WPAD-GUI feature request? If you want to track it, why not. -Michael > > Best regards, > Alex > > > Am 15.04.2019 um 11:43 schrieb Michael Tremer: >> Hello Alex, >> >> Thanks for submitting the patch. >> >> I guess the code looks fine, but where is the UI? >> >> Why should this not be configurable on the web interface? >> >> -Michael >> >>> On 14 Apr 2019, at 11:08, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >>> >>> This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files: >>> >>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl >>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl >>> >>> as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri >>> >>> These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released. >>> >>> Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de> >>> --- >>> html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++ >>> 1 file changed, 39 insertions(+) >>> >>> diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi >>> index 6daa7fb..369a5cb 100644 >>> --- a/html/cgi-bin/proxy.cgi >>> +++ b/html/cgi-bin/proxy.cgi >>> @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; >>> my $acl_ports_ssl = "$acldir/ports_ssl.acl"; >>> my $acl_include = "$acldir/include.acl"; >>> >>> +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; >>> +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; >>> + >>> my $updaccelversion = 'n/a'; >>> my $urlfilterversion = 'n/a'; >>> >>> @@ -2763,6 +2766,42 @@ END >>> print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; >>> } >>> >>> + # Additional exceptions for URLs >>> + # The file has to be created by the user and should contain one entry per line >>> + # Line-Format: <URL incl. wildcards> >>> + # e.g. *ipfire.org* >>> + if (-s "$acl_dst_noproxy_url") { >>> + undef @templist; >>> + >>> + open(NOPROXY,"$acl_dst_noproxy_url"); >>> + @templist = <NOPROXY>; >>> + close(NOPROXY); >>> + chomp (@templist); >>> + >>> + foreach (@templist) >>> + { >>> + print FILE " (shExpMatch(url, \"$_\")) ||\n"; >>> + } >>> + } >>> + >>> + # Additional exceptions for Subnets >>> + # The file has to be created by the user and should contain one entry per line >>> + # Line-Format: "<IP>", "<SUBNET MASK>" >>> + # e.g. "192.168.0.0", "255.255.255.0" >>> + if (-s "$acl_dst_noproxy_ip") { >>> + undef @templist; >>> + >>> + open(NOPROXY,"$acl_dst_noproxy_ip"); >>> + @templist = <NOPROXY>; >>> + close(NOPROXY); >>> + chomp (@templist); >>> + >>> + foreach (@templist) >>> + { >>> + print FILE " (isInNet(host, $_)) ||\n"; >>> + } >>> + } >>> + >>> print FILE <<END >>> (isInNet(host, "169.254.0.0", "255.255.0.0")) >>> ) >>> -- >>> 2.7.4 >>> >>
Hi, Am 17.04.2019 um 16:08 schrieb Michael Tremer: > Hi, > >> On 15 Apr 2019, at 21:12, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >> >> Hello Michael, >> >> my motivation for the patch is to provide a possibility to make exceptions survive an update of squid, as I'm repatching proxy.cgi by myself after each upgrade. I suppose there are more people out there with the same issue. I agree that it would by very nice to have it on the GUI as well, but unfortunately I don't have any experience with CGI yet and I don't have the time to learn it right now. I think patching the integration of the exception files into proxy.cgi is a good first step. It can be used as the base for extending the GUI. Maybe somebody else with CGI experience can help out? It's "just" two textareas and some file i/o basically… > > You can literally just copy and paste that. Give it a try! Have a look at it please, I just sent in an additional patch ... the translations for all languages except en and de need to be revised, how is this usually done? I copied the english versions into the language files I'm not able to translate by myself to avoid empty texts in the frontend. > >> As far as I know, the WPAD-Feature does not have any GUI support in general (e.g. checkboxes for enabled, enabled on a per subnet basis, etc.) until now. Additionally the WPAD-Feature requires the user to set up the extra apache-vhost or haproxy-frontend for port 80 (for http://wpad.<IPFire-Network-Domain>/wpad.dat) via CLI by himself anyway (another ToDo for a future patch ;-). > > It is available on http://<ipfire>:81/wpad.dat. No need for an extra host. This only provides WPAD via DHCP (if option 252 is configured by the user). Firefox for example does not support this (see http://findproxyforurl.com/browser-support/) and it alternatively uses WPAD via DNS. This requires one of the following URL's to work: http://wpad.<IPFire-Network-Domain>/wpad.dat or http://wpad/wpad.dat Port 80 does not seem to be in use on a new IPFire-Host by default. I could provide a patch for an additional apache-vhost. I'm not sure whether this is a good idea though. If users are running a haproxy on port 80/443 for example, this could break their running setup ... shipping some working example lines for haproxy.cfg to provide a frontend/backend-pair for wpad on port 80 is also a possibility. Or a Checkbox in the GUI to enable the vhost. Or just leave it as it is and provide the infos on the Wiki. What do you think? Best regards, Alex > >> Having this said, I think it is reasonable for the users to maintain their exceptions via CLI in the first instance until a GUI is available. Usually these things are not changed very often. It is still better than having to fix them after each upgrade of proxy.cgi If nobody else grabs this, I might possibly come back to it by myself at a later date. >> >> Should I write a bug report for the WPAD-GUI feature request? > > If you want to track it, why not. > > -Michael > >> >> Best regards, >> Alex >> >> >> Am 15.04.2019 um 11:43 schrieb Michael Tremer: >>> Hello Alex, >>> >>> Thanks for submitting the patch. >>> >>> I guess the code looks fine, but where is the UI? >>> >>> Why should this not be configurable on the web interface? >>> >>> -Michael >>> >>>> On 14 Apr 2019, at 11:08, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >>>> >>>> This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files: >>>> >>>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl >>>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl >>>> >>>> as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri >>>> >>>> These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released. >>>> >>>> Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de> >>>> --- >>>> html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++ >>>> 1 file changed, 39 insertions(+) >>>> >>>> diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi >>>> index 6daa7fb..369a5cb 100644 >>>> --- a/html/cgi-bin/proxy.cgi >>>> +++ b/html/cgi-bin/proxy.cgi >>>> @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; >>>> my $acl_ports_ssl = "$acldir/ports_ssl.acl"; >>>> my $acl_include = "$acldir/include.acl"; >>>> >>>> +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; >>>> +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; >>>> + >>>> my $updaccelversion = 'n/a'; >>>> my $urlfilterversion = 'n/a'; >>>> >>>> @@ -2763,6 +2766,42 @@ END >>>> print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; >>>> } >>>> >>>> + # Additional exceptions for URLs >>>> + # The file has to be created by the user and should contain one entry per line >>>> + # Line-Format: <URL incl. wildcards> >>>> + # e.g. *ipfire.org* >>>> + if (-s "$acl_dst_noproxy_url") { >>>> + undef @templist; >>>> + >>>> + open(NOPROXY,"$acl_dst_noproxy_url"); >>>> + @templist = <NOPROXY>; >>>> + close(NOPROXY); >>>> + chomp (@templist); >>>> + >>>> + foreach (@templist) >>>> + { >>>> + print FILE " (shExpMatch(url, \"$_\")) ||\n"; >>>> + } >>>> + } >>>> + >>>> + # Additional exceptions for Subnets >>>> + # The file has to be created by the user and should contain one entry per line >>>> + # Line-Format: "<IP>", "<SUBNET MASK>" >>>> + # e.g. "192.168.0.0", "255.255.255.0" >>>> + if (-s "$acl_dst_noproxy_ip") { >>>> + undef @templist; >>>> + >>>> + open(NOPROXY,"$acl_dst_noproxy_ip"); >>>> + @templist = <NOPROXY>; >>>> + close(NOPROXY); >>>> + chomp (@templist); >>>> + >>>> + foreach (@templist) >>>> + { >>>> + print FILE " (isInNet(host, $_)) ||\n"; >>>> + } >>>> + } >>>> + >>>> print FILE <<END >>>> (isInNet(host, "169.254.0.0", "255.255.0.0")) >>>> ) >>>> -- >>>> 2.7.4 >>>> >>> >
Hi, > On 18 Apr 2019, at 02:41, Alexander Koch <ipfire@starkstromkonsument.de> wrote: > > Hi, > > Am 17.04.2019 um 16:08 schrieb Michael Tremer: >> Hi, >> >>> On 15 Apr 2019, at 21:12, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >>> >>> Hello Michael, >>> >>> my motivation for the patch is to provide a possibility to make exceptions survive an update of squid, as I'm repatching proxy.cgi by myself after each upgrade. I suppose there are more people out there with the same issue. I agree that it would by very nice to have it on the GUI as well, but unfortunately I don't have any experience with CGI yet and I don't have the time to learn it right now. I think patching the integration of the exception files into proxy.cgi is a good first step. It can be used as the base for extending the GUI. Maybe somebody else with CGI experience can help out? It's "just" two textareas and some file i/o basically… >> >> You can literally just copy and paste that. Give it a try! > > Have a look at it please, I just sent in an additional patch ... the translations for all languages except en and de need to be revised, how is this usually done? I copied the english versions into the language files I'm not able to translate by myself to avoid empty texts in the frontend. I already replied to this on the patch. Just leave them empty if you don’t have a translation. English is a must. Do not use Google Translate or something. That never goes well. >>> As far as I know, the WPAD-Feature does not have any GUI support in general (e.g. checkboxes for enabled, enabled on a per subnet basis, etc.) until now. Additionally the WPAD-Feature requires the user to set up the extra apache-vhost or haproxy-frontend for port 80 (for http://wpad.<IPFire-Network-Domain>/wpad.dat) via CLI by himself anyway (another ToDo for a future patch ;-). >> >> It is available on http://<ipfire>:81/wpad.dat. No need for an extra host. > > This only provides WPAD via DHCP (if option 252 is configured by the user). Firefox for example does not support this (see http://findproxyforurl.com/browser-support/) and it alternatively uses WPAD via DNS. This requires one of the following URL's to work: http://wpad.<IPFire-Network-Domain>/wpad.dat or http://wpad/wpad.dat Yeah that is indeed a problem. > Port 80 does not seem to be in use on a new IPFire-Host by default. I could provide a patch for an additional apache-vhost. I'm not sure whether this is a good idea though. If users are running a haproxy on port 80/443 for example, this could break their running setup ... shipping some working example lines for haproxy.cfg to provide a frontend/backend-pair for wpad on port 80 is also a possibility. Or a Checkbox in the GUI to enable the vhost. Or just leave it as it is and provide the infos on the Wiki. How do we solve conflicts then when people either run a web server on IPFire or use a port-forwarding? A checkbox is quite complicated. We could use an iptables redirect rule or something but that all creates new problems. I really would like to support WPAD across platforms, but WPAD over DNS is a nightmare. There is no clean way to “make it just work”. -Michael > > What do you think? > > Best regards, Alex > >> >>> Having this said, I think it is reasonable for the users to maintain their exceptions via CLI in the first instance until a GUI is available. Usually these things are not changed very often. It is still better than having to fix them after each upgrade of proxy.cgi If nobody else grabs this, I might possibly come back to it by myself at a later date. >>> >>> Should I write a bug report for the WPAD-GUI feature request? >> >> If you want to track it, why not. >> >> -Michael >> >>> >>> Best regards, >>> Alex >>> >>> >>> Am 15.04.2019 um 11:43 schrieb Michael Tremer: >>>> Hello Alex, >>>> >>>> Thanks for submitting the patch. >>>> >>>> I guess the code looks fine, but where is the UI? >>>> >>>> Why should this not be configurable on the web interface? >>>> >>>> -Michael >>>> >>>>> On 14 Apr 2019, at 11:08, Alexander Koch <ipfire@starkstromkonsument.de> wrote: >>>>> >>>>> This patch extends the script /srv/web/ipfire/cgi-bin/proxy.cgi by additional code for reading exceptions for URL's and IP's/Subnets from two new files: >>>>> >>>>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_url.acl >>>>> - /var/ipfire/proxy/advanced/acls/dst_noproxy_ip.acl >>>>> >>>>> as described in: https://wiki.ipfire.org/configuration/network/proxy/extend/add_distri >>>>> >>>>> These can be used to define additional URL's, IP's and Subnets that should be retrieved "DIRECT" and not via the proxy. The files have to be created by the user, as the WPAD-Feature is not enabled by default anyway. If the files are not present or their size is 0, nothing is done. I'll revise the wiki-page, after the patch is merged and the core update is released. >>>>> >>>>> Signed-off-by: Alexander Koch <ipfire@starkstromkonsument.de> >>>>> --- >>>>> html/cgi-bin/proxy.cgi | 39 +++++++++++++++++++++++++++++++++++++++ >>>>> 1 file changed, 39 insertions(+) >>>>> >>>>> diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi >>>>> index 6daa7fb..369a5cb 100644 >>>>> --- a/html/cgi-bin/proxy.cgi >>>>> +++ b/html/cgi-bin/proxy.cgi >>>>> @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; >>>>> my $acl_ports_ssl = "$acldir/ports_ssl.acl"; >>>>> my $acl_include = "$acldir/include.acl"; >>>>> >>>>> +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; >>>>> +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; >>>>> + >>>>> my $updaccelversion = 'n/a'; >>>>> my $urlfilterversion = 'n/a'; >>>>> >>>>> @@ -2763,6 +2766,42 @@ END >>>>> print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; >>>>> } >>>>> >>>>> + # Additional exceptions for URLs >>>>> + # The file has to be created by the user and should contain one entry per line >>>>> + # Line-Format: <URL incl. wildcards> >>>>> + # e.g. *ipfire.org* >>>>> + if (-s "$acl_dst_noproxy_url") { >>>>> + undef @templist; >>>>> + >>>>> + open(NOPROXY,"$acl_dst_noproxy_url"); >>>>> + @templist = <NOPROXY>; >>>>> + close(NOPROXY); >>>>> + chomp (@templist); >>>>> + >>>>> + foreach (@templist) >>>>> + { >>>>> + print FILE " (shExpMatch(url, \"$_\")) ||\n"; >>>>> + } >>>>> + } >>>>> + >>>>> + # Additional exceptions for Subnets >>>>> + # The file has to be created by the user and should contain one entry per line >>>>> + # Line-Format: "<IP>", "<SUBNET MASK>" >>>>> + # e.g. "192.168.0.0", "255.255.255.0" >>>>> + if (-s "$acl_dst_noproxy_ip") { >>>>> + undef @templist; >>>>> + >>>>> + open(NOPROXY,"$acl_dst_noproxy_ip"); >>>>> + @templist = <NOPROXY>; >>>>> + close(NOPROXY); >>>>> + chomp (@templist); >>>>> + >>>>> + foreach (@templist) >>>>> + { >>>>> + print FILE " (isInNet(host, $_)) ||\n"; >>>>> + } >>>>> + } >>>>> + >>>>> print FILE <<END >>>>> (isInNet(host, "169.254.0.0", "255.255.0.0")) >>>>> ) >>>>> -- >>>>> 2.7.4 >>>>> >>>> >>
diff --git a/html/cgi-bin/proxy.cgi b/html/cgi-bin/proxy.cgi index 6daa7fb..369a5cb 100644 --- a/html/cgi-bin/proxy.cgi +++ b/html/cgi-bin/proxy.cgi @@ -124,6 +124,9 @@ my $acl_ports_safe = "$acldir/ports_safe.acl"; my $acl_ports_ssl = "$acldir/ports_ssl.acl"; my $acl_include = "$acldir/include.acl"; +my $acl_dst_noproxy_url = "$acldir/dst_noproxy_url.acl"; +my $acl_dst_noproxy_ip = "$acldir/dst_noproxy_ip.acl"; + my $updaccelversion = 'n/a'; my $urlfilterversion = 'n/a'; @@ -2763,6 +2766,42 @@ END print FILE " (isInNet(host, \"$netsettings{'ORANGE_NETADDRESS'}\", \"$netsettings{'ORANGE_NETMASK'}\")) ||\n"; } + # Additional exceptions for URLs + # The file has to be created by the user and should contain one entry per line + # Line-Format: <URL incl. wildcards> + # e.g. *ipfire.org* + if (-s "$acl_dst_noproxy_url") { + undef @templist; + + open(NOPROXY,"$acl_dst_noproxy_url"); + @templist = <NOPROXY>; + close(NOPROXY); + chomp (@templist); + + foreach (@templist) + { + print FILE " (shExpMatch(url, \"$_\")) ||\n"; + } + } + + # Additional exceptions for Subnets + # The file has to be created by the user and should contain one entry per line + # Line-Format: "<IP>", "<SUBNET MASK>" + # e.g. "192.168.0.0", "255.255.255.0" + if (-s "$acl_dst_noproxy_ip") { + undef @templist; + + open(NOPROXY,"$acl_dst_noproxy_ip"); + @templist = <NOPROXY>; + close(NOPROXY); + chomp (@templist); + + foreach (@templist) + { + print FILE " (isInNet(host, $_)) ||\n"; + } + } + print FILE <<END (isInNet(host, "169.254.0.0", "255.255.0.0")) )