The Un-Official Proxomitron Forum

Full Version: ProxHTTPSProxyMII: Reloaded
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
(Sep. 16, 2018 06:37 PM)vlad_s Wrote: [ -> ]is it possible to specify the time when the certificate should start validating more early, rather than at the time the certificate was generated?

Yep. In CertTool.py change lines
Code:
cert.gmtime_adj_notBefore(0)
to
Code:
cert.gmtime_adj_notBefore(-60 * 60 * 24 * 2)
and all your new certificates will be generated two days ago from current time
There are also one thing that bothers me. Don't you think, that default encryption between browser and ProxHTTPSProxyMII is too powerful? Really, AES256-GCM for localhost is a little... excessively.
Ciphersuite for connection to front-proxy can be set in ProxHTTPSProxy.py line
Code:
ssl_sock = ssl.wrap_socket(self.connection, keyfile=dummycert, certfile=dummycert, server_side=True)
by change it like
Code:
ssl_sock = ssl.wrap_socket(self.connection, ciphers='ECDHE-ECDSA-AES128-GCM-SHA256', keyfile=dummycert, certfile=dummycert, server_side=True)
It's better to use ECDHE-ECDSA-AES128-GCM-SHA256 if CPU have AES-NI, ECDH+CHACHA20 if AES acceleration unavailable or even !aNULL for avoiding double encrypt-decrypt if connection security managed by upstream proxy like compy
So, maybe ciphersuite selection option should be placed in config.ini as advanced option?
(Sep. 17, 2018 01:15 AM)Sudenr Wrote: [ -> ]
(Sep. 16, 2018 06:37 PM)vlad_s Wrote: [ -> ]is it possible to specify the time when the certificate should start validating more early, rather than at the time the certificate was generated?

Yep. In CertTool.py change lines
Code:
cert.gmtime_adj_notBefore(0)
to
Code:
cert.gmtime_adj_notBefore(-60 * 60 * 24 * 2)
and all your new certificates will be generated two days ago from current time
Ok, it works.
It is impossible to open the site just.ru and rbt.ru, while in the section [SSL Pass-Thru] is not added *.variti.de. The script from *.variti.de is loaded normally in any case. So I did not understand who was to blame for the privoxy or proxhttpsproxy.
(Sep. 19, 2018 04:38 PM)vlad_s Wrote: [ -> ]who was to blame for the privoxy or proxhttpsproxy.
Privoxy. I do not use it's default filters and actions, and both sites load normally.
In the config privoxy I commented out all filter and action and it did not help Sad. Only the [SSL Pass-Thru] section helps, then *.variti.de can be removed from there and some time works.
Did you try regenerate *.variti.de certificate? What's in your ProxHTTPSProxy log, when you go to variti.de?
(Sep. 17, 2018 01:20 AM)Sudenr Wrote: [ -> ]There are also one thing that bothers me. Don't you think, that default encryption between browser and ProxHTTPSProxyMII is too powerful? Really, AES256-GCM for localhost is a little... excessively.
Ciphersuite for connection to front-proxy can be set in ProxHTTPSProxy.py line
Code:
ssl_sock = ssl.wrap_socket(self.connection, keyfile=dummycert, certfile=dummycert, server_side=True)
by change it like
Code:
ssl_sock = ssl.wrap_socket(self.connection, ciphers='ECDHE-ECDSA-AES128-GCM-SHA256', keyfile=dummycert, certfile=dummycert, server_side=True)
It's better to use ECDHE-ECDSA-AES128-GCM-SHA256 if CPU have AES-NI, ECDH+CHACHA20 if AES acceleration unavailable or even !aNULL for avoiding double encrypt-decrypt if connection security managed by upstream proxy like compy
So, maybe ciphersuite selection option should be placed in config.ini as advanced option?
Unfortunately I don't think many, if any, browsers will like to use null SSL. Perhaps the weakest cipher they accept would be ideal, but that seems to change frequently.
(Sep. 21, 2018 12:03 PM)amy Wrote: [ -> ]Unfortunately I don't think many, if any, browsers will like to use null SSL. Perhaps the weakest cipher they accept would be ideal, but that seems to change frequently.
It's not for browsers, it's for another proxy, if they encrypt connection by themselves.
(Sep. 20, 2018 06:03 PM)Sudenr Wrote: [ -> ]Did you try regenerate *.variti.de certificate? What's in your ProxHTTPSProxy log, when you go to variti.de?
With proxhttps, the script loads, if I just try to download it (by link). But when I open those sites, the problem also arises. The log is empty (no errors).
(Sep. 23, 2018 10:42 PM)vlad_s Wrote: [ -> ]The log is empty (no errors).
Open browser network console (F12) and reload page. Is there CORS related errors?
(Sep. 25, 2018 08:39 PM)Sudenr Wrote: [ -> ]
(Sep. 23, 2018 10:42 PM)vlad_s Wrote: [ -> ]The log is empty (no errors).
Open browser network console (F12) and reload page. Is there CORS related errors?
There are no errors, just the script with variti.de reloads all the time.
There is a similar problem with the site cdek.ru, with opening it redirects to ohio8.vchecks.me and so on to infinity. If in [SSL Passs-Thru] add ohio8.vchecks.me, then the site opens. Ohio8.vchecks.me added to the privoxy and use filters
Code:
{fragile \
}
ohio8.vchecks.me/
It seems to be turning off filtering. But it does not help.
There are errors in the log:
Code:
[11:34] 260 ProxHTTPSProxyMII FrontProxy/v1.5 [Errno 32] Broken pipe
[11:34] 266 ProxHTTPSProxyMII FrontProxy/v1.5 [Errno 32] Broken pipe
[11:35] 275 ProxHTTPSProxyMII FrontProxy/v1.5 [Errno 32] Broken pipe
[11:35] 285 ProxHTTPSProxyMII FrontProxy/v1.5 [Errno 32] Broken pipe
[11:36] 288 ProxHTTPSProxyMII FrontProxy/v1.5 [Errno 32] Broken pipe
Etc. Three-digit number grows.
I recently switched browsers, and can not seem to get ProxHTTPSProxyMII to work. Or rather, it seems like ProxHTTPSProxy IS working, but Privoxy is still not filtering HTTPS pages. It works in Opera Presto, but not in Firefox.

For testing, I set a simple filter in Privoxy that changes a word on google.com home page. I can confirm this works in Opera, filtering an HTTPS page, and using the ProxHTTPS certificate:
[Image: prxtst_1.jpg]
Note the text "[TEST!!]" at the bottom, replacing the word "celebrate"

The same page in Firefox:
[Image: prxtst_2.jpg]
In this shot, the word was not replaced.


And for comparison, the proxy settings for Opera:
[Image: prxtst_3.jpg]

And Firefox:
[Image: prxtst_4.jpg]

I have also tried "Use system proxy settings" in Firefox with the same result:
[Image: prxtst_5.jpg]

Now here is the proxy log, showing the same port numbers as I have specified in the proxy configs. Then the log lines at 02:14 are when loading the page in Firefox, and the lines at 02:15 loading the same url in Opera. Both of them are obviously flowing through ProxHTTPSProxy, and both also show up in the Privoxy log, but the html is NOT being filtered in Firefox.

Code:
C:\Programs\Privoxy\ProxHTTPS>C:\Programs\Python\python.exe ProxHTTPSProxy.py
============================================================================
ProxHTTPSProxyMII v1.4 (urllib3/1.21.1)

  FrontServer  : localhost:8079
  RearServer   : localhost:8081
  ParentServer : None
  Proxomitron  : http://localhost:8118
============================================================================
[02:14] 001 [D] "POST https://www.google.com/gen_204?atyp=csi&ei=6B0KXIffBLCJgge195iwCQ&s=jsa&jsi=s,et.focus,n.iDPoPb,cn.2&zx=1544166897562 0" 204 0
[02:14] 002 [D] "POST https://www.google.com/gen_204?atyp=csi&ei=6B0KXIffBLCJgge195iwCQ&s=jsa&jsi=s,et.focus,n.iDPoPb,cn.3&zx=1544166898240 0" 204 0
[02:14] 003 [D] "GET https://www.google.com/" 200 -
[02:14] 004 [D] "POST https://www.google.com/gen_204?s=webhp&t=aft&atyp=csi&ei=9x0KXIKhHsHl_QbdsIPYDQ&rt=wsrt.485,aft.38,prt.38 0" 204 0
[02:14] 005 [D] "POST https://www.google.com/gen_204?atyp=csi&ei=9x0KXIKhHsHl_QbdsIPYDQ&s=jsa&jsi=s,t.0,et.focus,n.iDPoPb,cn.1&zx=1544166899011 0" 204 0
[02:14] 006 [D] "POST https://www.google.com/gen_204?atyp=csi&ei=9x0KXIKhHsHl_QbdsIPYDQ&s=webhp&t=all&imn=1&adh=&ima=1&ime=1&imeb=0&imeo=0&sys=hc.16&rt=aft.38,dcl.137,iml.38,ol.301,prt.38,xjs.333,xjsee.332,xjses.242,xjsls.116,wsrt.485,cst.0,dnst.0,rqst.311,rspt.1,sslt.1,rqstt.40,unt.1,cstt.1,dit.536&zx=1544166899071 0" 204 0
[02:15] 008 [D] "GET https://www.google.com/" 200 -
[02:15] 009 [D] "GET https://www.google.com/gen_204?s=webhp&t=aft&atyp=csi&ei=AB4KXIGDLoTK_Qbgqpto&rt=wsrt.undefined,aft.18,prt.18" 204 0
[02:15] 010 [D] "GET https://clients1.google.com/generate_204" 204 0

And I just now tried Chrome (which *only* gives you option to use system proxy ::facepalm:: ), and it too is not filtering HTTPS. Opera has never had this problem for as long as I've been using ProxHTTPSProxy. I have tried everything I could possibly think of, and can't even begin to guess what the problem is. Anybody have any ideas?

EDIT: I just had a random thought... Opera Presto is very old, Firefox and Chrome are both new versions. Is it possible that Firefox and Chrome are requesting pages in a compression/encoding/whatever format that Privoxy can't understand? On the other hand, HTTP pages *are* still being filtered properly.
Same result here.
Opera and vivaldi filtering succeed.
Palemoon and firefox faild.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Reference URL's