The Un-Official Proxomitron Forum
ProxHTTPSProxyMII: Reloaded - Printable Version

+- The Un-Official Proxomitron Forum (https://www.prxbx.com/forums)
+-- Forum: Forum Related (/forumdisplay.php?fid=37)
+--- Forum: ProxHTTPSProxy (/forumdisplay.php?fid=48)
+--- Thread: ProxHTTPSProxyMII: Reloaded (/showthread.php?tid=2172)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25


RE: ProxHTTPSProxyMII: Reloaded - vlad_s - Apr. 09, 2018 06:38 PM

I used the file proposed by you, 1.1.1.1 still does not work.
On account of the browser, what kind of certificate do I need to export? There are three of them on that site (1.1.1.1).
[Image: 2018_04_09_213228.png]

The error that I see:
[Image: 2018_04_09_221507.png]
The file .1.1.1.crt is created in the directory Certs. I understand that there should be 1.1.1.1, and not *.1.1.1?


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 10, 2018 06:02 AM

(Apr. 09, 2018 06:38 PM)vlad_s Wrote:  The error that I see:
[attachment=1026]
The file .1.1.1.crt is created in the directory Certs. I understand that there should be 1.1.1.1, and not *.1.1.1?

This appears to be a problem with the cert that ProxHTTPSProxyMII creates.

I didn't see it because I have disabled browser warnings for ssl.
I apologize for my poor memory. Sorry. D'oh!


RE: ProxHTTPSProxyMII: Reloaded - vlad_s - Apr. 10, 2018 04:18 PM

(Apr. 10, 2018 06:02 AM)JJoe Wrote:  This appears to be a problem with the cert that ProxHTTPSProxyMII creates.
Ok, I already understood.


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 14, 2018 08:37 PM

(Apr. 09, 2018 06:38 PM)vlad_s Wrote:  I understand that there should be 1.1.1.1, and not *.1.1.1?

In the past, the common name could be an ip address.
I don't remember if a wildcard was allowed in a CN ip address.
I think most of MII's cert problems are cause by a missing SubjectAltNames field.
Regardless...




I have uploaded
ProxHTTPSProxyMII 1.5wip 34cx_freeze5.0.1urllib3v1.2Win32OpenSSL_Light-1_0_2k-1_1_0e.zip, to
https://1fichier.com/?1qa8qglsv6
, 7.03 MB.

Changes:
Added SubjectAltNames support for DNS and IP... No guarantees, warranties, etc., but it appears to work.
Common name will no longer use a leading '*'. This means less code but a larger cert folder.

Notes:
Built with outdated WinXP compatable software. So, may work with WinXP.


About 1fichier:
In the past, free use was supported by pop over and under advertising.
Do not install any of the advertised programs or browser extensions without additional study...

Now, free downloads are throttled and limited to one every 2 hours.
Still good enough for this. Smile!

HTH

Edited to reflect change at 1fichier
Edited to strike download link


RE: ProxHTTPSProxyMII: Reloaded - Thomas S. - Apr. 18, 2018 06:29 PM

Thanks for this work.
Is it possible for you to offer the py code for download?
I have done little adjustments for my usage with old v1.4 - but i am not able to do such a change.
I can compile my own exe with actual packages (for example cryptography 2.2.2)


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 19, 2018 02:16 AM

(Apr. 18, 2018 06:29 PM)Thomas S. Wrote:  Is it possible for you to offer the py code for download?

https://www.prxbx.com/forums/showthread.php?tid=2191&pid=19245#pid19245

Try this one, minor mods and edits. It should work.

Have Fun


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 19, 2018 02:34 AM

ProxHTTPSProxyMII 1.5wip 34cx_freeze5.0.1urllib3v1.2Win32OpenSSL_Light-1_0_2k-1_1_0e.zip

Download link:
https://1fichier.com/?n96fnmk401
7 MB

Changes:
__version__ updated
minor mods and edits. It should still work.

Have Fun

Edited to strike download link


RE: ProxHTTPSProxyMII: Reloaded - Thomas S. - Apr. 19, 2018 08:13 PM

Thanks very much for the code.

For your information:
I have done a first short test, all seams to be good - but https://1.1.1.1 will not work.
I have got a certificate error under IE8 WinXP with the new version and have to load the site "on my own risk":

"The security certificate of this website has been issued for a different address of the website"

If I look (with WinXP certificate manager) in the certificate "1.1.1.1.crt" it list NO CN, so IE8 rejects this.
The field is empty.

It is a little bit useless because the site will not rendered OK in IE8, but for test it is good.

With the old version 1.4 is load without my extra confirmation.
And the certificate ".1.1.1.crt" has the CN *.1.1.1

In the next days I made more tests, may be all other site works.


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 19, 2018 11:00 PM

(Apr. 19, 2018 08:13 PM)Thomas S. Wrote:  If I look (with WinXP certificate manager) in the certificate "1.1.1.1.crt" it list NO CN, so IE8 rejects this.
The field is empty.

Thanks, I didn't notice this was missing.
My browsers on Win7 and Win10 don't care.

I'll try to add the field.

Files updated.


ProxHTTPSProxyMII 1.5wipa 34cx_freeze5.0.1urllib3v1.2Win32OpenSSL_Light-1_0_2k-1_1_0e.zip

Download link:
https://1fichier.com/?0hzpeavdn0
7 MB

https://www.prxbx.com/forums/showthread.php?tid=2191&pid=19245#pid19245

Changes:
Common Name returns


RE: ProxHTTPSProxyMII: Reloaded - JJoe - Apr. 21, 2018 09:11 PM

ProxHTTPSProxyMII 1.5wipb 34cx_freeze5.0.1urllib3v1.22Win32OpenSSL_Light-1_0_2o-1_1_0h.zip

Download link https://1fichier.com/?6azh99hfzl
7.01 MB

http://www.prxbx.com/forums/showthread.php?tid=2191&pid=19252#pid19252

Changes:
'*' returns to cert's Subject field due to some hosts using more than the 64 characters that are allowed. Example: 18cfdfd73150f69310ab-4d842a0601d0ae955a714605e7fb6d6f.ssl.cf2.rackcdn.com.
urllib3 updated to v1.22
OpenSSL updated to Win32OpenSSL_Light-1_0_2k-1_1_0e


RE: ProxHTTPSProxyMII: Reloaded - ryszardzonk - May. 08, 2018 09:45 AM

Hi
I am redirecting all HTTP/S traffic to squid for caching
Code:
iptables -t nat -A PREROUTING -i ${INT_IF} -p tcp -s 192.168.101.0/24 ! -d 192.168.101.0/24 --dport 80 -j REDIRECT --to-port 8080
iptables -t nat -A PREROUTING -i ${INT_IF} -p tcp -s 192.168.101.0/24 ! -d 192.168.101.0/24 --dport 443 -j REDIRECT --to-port 8090

which than I am receiving in squid for transparent caching separately for http & https traffic

Code:
http_port 192.168.101.101:8080 intercept
https_port 192.168.101.101:8090 intercept ssl-bump \
  cert=/etc/squid/ssl_cert/squid.pem \
  generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
sslcrtd_program /usr/libexec/squid/ssl_crtd -s /var/lib/ssl_db -M 4MB

acl broken_sites ssl::server_name .wikipedia.org .nsatc.net .microsoft.com
ssl_bump splice broken_sites
ssl_bump peek all
ssl_bump bump all

All that is forwarded to privoxy for filtering where as privoxy does not handle ssl traffic is filtered only for http sites
Code:
cache_peer 127.0.0.1 parent 3128 0 no-query no-digest

What I am planning to do is to separate traffic for http & https
Code:
acl ACL_HTTP proto HTTP
acl ACL_HTTPS proto HTTPS
acl ACL_HTTPS2 port 443        
cache_peer 127.0.0.1 parent 3128 0 name=http_peer no-query no-digest
cache_peer 127.0.0.1 parent 3129 0 name=https_peer ssl no-query no-digest
cache_peer_access https_peer allow ACL_HTTPS
cache_peer_access https_peer allow ACL_HTTPS2
cache_peer_access http_peer allow ACL_HTTP
never_direct allow all
http traffic would be than forwarded to privoxy. Where to send https to?
Way I see it that from squid I send https traffic to ProxHTTPSProxyMII which sends it to privoxy for filtering and gets it back from privoxy to send to actual server. Is this correct approach and if it is how do I configure privoxy for it. So far I have rather simple configuration which does not differentiate between front and rear server

Code:
...
listen-address  127.0.0.1:3128
permit-access  localhost
permit-access  192.168.101.0/24
        forward         192.168.*.* .
        forward         127.*.*.*/  .
        forward         :443 .

My question is do I need to edit privoxy config to listen on more than port 3128 or do I need to simply edit config.ini from ProxHTTPSProxyMII into this?

Code:
ProxAddr = http://localhost:3128
FrontPort = 3129
RearPort = 3130

EDIT: It turned out that to use squid for ssl parent proxy I had to add option "ssl" to that proxy otherwise squid would fail with
Code:
2018/05/09 07:50:44 kid1| assertion failed: PeerConnector.cc:116: "peer->use_ssl"
so proper line for https traffic is
cache_peer 127.0.0.1 parent 3129 0 name=https_peer ssl no-query no-digest
instead of
cache_peer 127.0.0.1 parent 3129 0 name=https_peer no-query no-digest


RE: ProxHTTPSProxyMII: Reloaded - JJoe - May. 09, 2018 03:58 AM

I think I understand but I haven't actually done it.
So...

(May. 08, 2018 09:45 AM)ryszardzonk Wrote:  http traffic would be than forwarded to privoxy. Where to send https to?

You send https to ProxHTTPSProxyMII front server at 3129. The front server adds a 'tagged' header to https requests and forwards to privoxy at 3128. Privoxy forwards 'tagged' requests to ProxHTTPSProxyMII rear server, 3130.

(May. 08, 2018 09:45 AM)ryszardzonk Wrote:  My question is do I need to edit privoxy config to listen on more than port 3128 or do I need to simply edit config.ini from ProxHTTPSProxyMII into this?

Code:
ProxAddr = http://localhost:3128
FrontPort = 3129
RearPort = 3130

You need to configure privoxy to recognize the 'tagged' requests and forward them to the rear server.

I believe this is step 7 of https://prxbx.com/forums/showthread.php?tid=2224
*Code edited to use port 3130*


(Jul. 26, 2015 11:09 AM)Faxopita Wrote:  Step 7
Add these lines to user.filter file:
Code:
CLIENT-HEADER-TAGGER: tagger4https
s@^.*Tagged:.*ProxHTTPSProxyMII.*FrontProxy.*$@$0@i

Add these lines to user.action file:
Code:
{ +client-header-tagger{tagger4https} }
/
{ +forward-override{forward 127.0.0.1:3130} }
TAG:.*?ProxHTTPSProxyMII



RE: ProxHTTPSProxyMII: Reloaded - ryszardzonk - May. 09, 2018 07:03 AM

(May. 09, 2018 03:58 AM)JJoe Wrote:  I believe this is step 7 of https://prxbx.com/forums/showthread.php?tid=2224
*Code edited to use port 3130*

Yes sir. This is what I was missing is how privoxy would know to send traffic back to ProxHTTPSProxyMII. Simple code additions You pointed out in the howto (which btw I quite likely would never find by myself) made the traffic go like it should which is ProxHTTPSProxyMII -> privoxy -> ProxHTTPSProxyMII.

It is however quite problematic to enable it like that network wide for intercepting proxy as any https website tried to use required confirming certificate to work

This is what firefox 52.7 would show while using latest dev version from https://www.prxbx.com/forums/showthread.php?tid=2191&pid=19252#pid19252

Quote:The certificate is not trusted because the issuer certificate is unknown. The server might not be sending the appropriate intermediate certificates. An additional root certificate may need to be imported. Error code: SEC_ERROR_UNKNOWN_ISSUER

To make sure that message is not coming from my first proxy in chain I skipped squid and pointed browser to use ProxHTTPSProxyMII for https and privoxy for http.

To fix it I got the idea of copying ca-certificates from system to "Certs" directory, but than I saw all those certs to websites I tried to use written there so it seems firefox has problem with certificate ProxHTTPSProxyMII issues.


RE: ProxHTTPSProxyMII: Reloaded - JJoe - May. 09, 2018 11:59 AM

(May. 09, 2018 07:03 AM)ryszardzonk Wrote:  any https website tried to use required confirming certificate to work
...
it seems firefox has problem with certificate ProxHTTPSProxyMII issues.

You will need to add ProxHTTPSProxy's "CA.crt" to each Client's (Firefox) store of trusted certificate authorities.

There is a copy of "CA.crt" in ProxHTTPSProxyMII_py 1.5wipb.zip


RE: ProxHTTPSProxyMII: Reloaded - ryszardzonk - May. 09, 2018 12:41 PM

(May. 09, 2018 11:59 AM)JJoe Wrote:  You will need to add ProxHTTPSProxy's "CA.crt" to each Client's (Firefox) store of trusted certificate authorities.

There is a copy of "CA.crt" in ProxHTTPSProxyMII_py 1.5wipb.zip

Yes there was one and hence I do have apache running on that server so for easy installation I placed it in root of the website and then in client machine I went ahead I typed "192.168.101.101/CA.crt". Window poped-up asking would I like to install this certificate and asked for which options I would use it. Depending whether it was Firefox on Linux it asked to use it for
- web site identification (checked)
- email
- software
Chrome on Android device
- VPN and apps (checked)
- Wifi

However that did not help as on both devices same error message as previously appeared Sad