Post Reply 
ProxHTTPSProxyMII: Reloaded
Jun. 24, 2018, 08:06 PM
Post: #271
RE: ProxHTTPSProxyMII: Reloaded
(Jun. 23, 2018 09:29 AM)kichrot Wrote:  Hello!
...
Sorry for my English, I'm writing through an interpreter.

Welcome!

Thank you for your work and post. Thumbs Up

The English was good enough. Smile!
Add Thank You Quote this message in a reply
Jun. 25, 2018, 03:49 AM
Post: #272
RE: ProxHTTPSProxyMII: Reloaded
(Jun. 24, 2018 07:41 PM)JJoe Wrote:  What are you using to generate and apply the patches?
I have been unable to find a utility that will apply all the patches correctly on Windows 10.
It's exported from git (git format-patch) and you should be able to re-apply it using git apply.

Quote:Could you zip the source and upload?
Sure. I am uploading everything including git info - it stared as a clone of https://github.com/wheever/ProxHTTPSProxyMII, I attached branch jjoe with your changes to it, then attached branch pepak with my changes.

Quote:For our purposes, you could temporarily tag it as 1.5.1wip.

I prefer to leave the versioning to you.

Quote:ProxHTTPSProxyMII: Development may be the more appropriate thread.
It might, but then again, if I post here, I consider it a part of a discussion and hopefully everyone will consider it so, rather than an official release. I think my changes are working fine, but as they are my first attempt at developing in Python, I would rather have someone read the changes before they are committed.


Attached File(s)
.zip  ProxHTTPSProxy.zip (Size: 570.8 KB / Downloads: 100)
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to pepak for this post:
vlad_s
Jul. 10, 2018, 07:48 AM (This post was last modified: Jul. 10, 2018 07:49 AM by ryszardzonk.)
Post: #273
RE: ProxHTTPSProxyMII: Reloaded
I got problem with certificate validation on ProxHTTPSProxyMII 1.5 on that site https://www.ssllabs.com/ssltest/analyze.....240.18.19 even after adding proper certtificate to cacert.pem from http://cacerts.digicert.com/DigiCertSHA2...rverCA.crt Any ideas what may be wrong?

Code:
502: HTTPError

The following error occurred while trying to access https://static.xx.fbcdn.net/

HTTPSConnectionPool(host='static.xx.fbcdn.net', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))
Generated on 2018-07-10 09:49:06.136996 by ProxHTTPSProxyMII RearProxy/v1.5
Add Thank You Quote this message in a reply
Jul. 11, 2018, 03:43 AM
Post: #274
RE: ProxHTTPSProxyMII: Reloaded
(Jul. 10, 2018 07:48 AM)ryszardzonk Wrote:  Any ideas what may be wrong?

'static.xx.fbcdn.net' works for me.
I think, validation only requires 'DigiCert High Assurance EV Root CA'.

'502: Bad Gateway' message may be caused by site's server failing to respond. Which could be caused by incorrect url, server or network problems, router, dns, missing or incorrect data in the client's request, etc.

Try https://static.xx.fbcdn.net/rsrc.php/v3/yb/r/GsNJNwuI-UM.gif
which loads for me in a 'new private window', (no cookies, no referer, etc).
Using Opera portable.
Add Thank You Quote this message in a reply
Jul. 11, 2018, 06:18 AM
Post: #275
RE: ProxHTTPSProxyMII: Reloaded
Yes you are right. It works. Turned out for some reason it ended up in my hosts file so it was a network problem after all.
Add Thank You Quote this message in a reply
Sep. 02, 2018, 05:57 PM (This post was last modified: Sep. 02, 2018 06:01 PM by vlad_s.)
Post: #276
RE: ProxHTTPSProxyMII: Reloaded
Hello! I can not get the right column of two sites to be displayed https://vc.ru/ and https://tjournal.ru/
Specifically on trying to add to the section [SSL Pass-Thru] and [BYPASS URL] it vc.ru/chan/ and https://vc.ru/chan/* accordingly does not help. With the rule iptables
Code:
iptables -t nat -I PREROUTING -s 192.168.2.211/32 -p tcp -m tcp --dport 443 -j ACCEPT
everything works correctly, but it does not fit, because past the proxy.
[Image: image.jpg]
Add Thank You Quote this message in a reply
Sep. 04, 2018, 04:01 AM
Post: #277
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 02, 2018 05:57 PM)vlad_s Wrote:  Hello! I can not get the right column of two sites to be displayed https://vc.ru/ and https://tjournal.ru/

Code:
[SSL Pass-Thru]
vc.ru
tjournal.ru

Works for me but yuck.

When we do something that they do not like we get "bad_user_visit"

MII shows

Code:
584 [D] "GET https://tt.onthe.io/?k[]=12300:bad_user_visit...

The server does not send the stream to 'bad_users' Wink

You may need to clear cookies and data. Also, there may be a time penalty assigned to your browser and/or ip address.
Add Thank You Quote this message in a reply
Sep. 04, 2018, 09:21 PM (This post was last modified: Sep. 04, 2018 09:23 PM by Sudenr.)
Post: #278
RE: ProxHTTPSProxyMII: Reloaded
Hi! I'm using ProxHTTPSProxMII v1.5 on Python 3.6.6 on Windows and sometimes (especially when program generates many certificates at once) I have errors like

SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:841)" while trying to establish local SSL tunnel for [site.example.com:443]

If I delete certificate for site.example.com and renew page, re-created certificate usually works as it should be.

I'm using EC prime256 EC certificate and key. It works perfectly with v1.4 but not with 1.5

Thank you!
Add Thank You Quote this message in a reply
Sep. 04, 2018, 11:57 PM (This post was last modified: Sep. 05, 2018 01:59 AM by JJoe.)
Post: #279
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 04, 2018 09:21 PM)Sudenr Wrote:  If I delete certificate for site.example.com and renew page, re-created certificate usually works as it should be.

Have you tried renewing the page with the original certificate? If yes, what happened?
Have you compared the certificates?
Add Thank You Quote this message in a reply
Sep. 09, 2018, 02:01 PM (This post was last modified: Sep. 09, 2018 02:02 PM by Sudenr.)
Post: #280
RE: ProxHTTPSProxyMII: Reloaded
UPD:
Alas, even without "SSL Accelerator" addon in Firefox, ProxHTTPSProxyMII still continues to spawn errors, albeit somewhat less often.
So I did some research:

1. Verification showed that the problem only occurs with Firefox, Chrome-based is not affected.

2. The problem arises even in the clean, fresh-installed Firefox.

3. The problem arises if a site is opened that loads a lot of other encrypted sites simultaneously. Most often this is a variety of imgNN.example.com

4. The problem occurs regardless of certificate type - EC or RSA
But generated certificates are valid in both cases (if check it with Windows)

5. When I try to download a picture from subdomain with an incorrect certificate, Firefox gives an error:
"SEC_ERROR_REUSED_ISSUER_AND_SERIAL"

It seems, that it's caused by identical serial number in generated certificates (and paranoid Firefox security), so I check how certs generated, and found line
Code:
cert.set_serial_number(int(time.time()*10000))
in CertTool.py and thats explains everything.
I changed it to
Code:
cert.set_serial_number(int(time.time()*random.randint(1, 10000)))
(yep, dirty hack), delete Certs folder and restart ProxHTTPSProxyMII.
No SSLv3 errors for 3 days.
Add Thank You Quote this message in a reply
Sep. 11, 2018, 03:53 PM
Post: #281
RE: ProxHTTPSProxyMII: Reloaded
Personally, I opted for:
Code:
cert.set_serial_number(int.from_bytes(os.urandom(16), byteorder='big'))
This version is cryptographically secure, anything based on time or random is not.
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to pepak for this post:
Sudenr
Sep. 11, 2018, 08:13 PM
Post: #282
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 11, 2018 03:53 PM)pepak Wrote:  This version is cryptographically secure, anything based on time or random is not.

Thanks, that's better. I'm not a programmer myself, but I understand that time*random is dirty hack Smile!
Add Thank You Quote this message in a reply
Sep. 12, 2018, 06:58 AM (This post was last modified: Sep. 12, 2018 08:41 AM by ryszardzonk.)
Post: #283
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 04, 2018 04:01 AM)JJoe Wrote:  
(Sep. 02, 2018 05:57 PM)vlad_s Wrote:  Hello! I can not get the right column of two sites to be displayed https://vc.ru/ and https://tjournal.ru/

Code:
[SSL Pass-Thru]
vc.ru
tjournal.ru

Works for me but yuck.

(Sep. 09, 2018 02:01 PM)Sudenr Wrote:  5. When I try to download a picture from subdomain with an incorrect certificate, Firefox gives an error:
"SEC_ERROR_REUSED_ISSUER_AND_SERIAL"

It seems, that it's caused by identical serial number in generated certificates (and paranoid Firefox security), so I check how certs generated, and found line
Code:
cert.set_serial_number(int(time.time()*10000))
in CertTool.py and thats explains everything.
I changed it to
Code:
cert.set_serial_number(int(time.time()*random.randint(1, 10000)))
(yep, dirty hack), delete Certs folder and restart ProxHTTPSProxyMII.
No SSLv3 errors for 3 days.

I don't think I ever stumbled upon "SEC_ERROR_REUSED_ISSUER_AND_SERIAL" in the logs and I use Firefox almost exclusively, but maybe it was only my luck to visit some sites first before they were subdomains in others.
Anyways would this code change also fix problems with vc.ru and tjournal.ru so they would not need [SSL Pass-Thru]? Clearly they were subdomains in the shown example

EDIT:
OpenSSL now supports TLS1.3. Does that mean ProxHTTPSProxyMII would have to be updated to use this new updated library or OpenSSL-1.1.1 can be safely used?

more here:
https://www.openssl.org/blog/blog/2018/0...elease111/
Add Thank You Quote this message in a reply
Sep. 16, 2018, 04:16 AM
Post: #284
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 12, 2018 06:58 AM)ryszardzonk Wrote:  I don't think I ever stumbled upon "SEC_ERROR_REUSED_ISSUER_AND_SERIAL" in the logs and I use Firefox almost exclusively, but maybe it was only my luck to visit some sites first before they were subdomains in others.
I did. Had to restart Firefox or re-generate the certificates when that happened.

It also seems that with this change, the quite frequent ResponseNotReady errors generated by ProxHTTPSProxy for some sites are a thing of the past. So I would quite recommend using this patch.

Quote:EDIT:
OpenSSL now supports TLS1.3. Does that mean ProxHTTPSProxyMII would have to be updated to use this new updated library or OpenSSL-1.1.1 can be safely used?
ProxHTTPSProxy does not need any change, but a modification may be necessary for the underlying OpenSSL bindings. Although a quick check suggest that the necessary change may already be in.
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to pepak for this post:
vlad_s
Sep. 16, 2018, 06:37 PM (This post was last modified: Sep. 16, 2018 06:38 PM by vlad_s.)
Post: #285
RE: ProxHTTPSProxyMII: Reloaded
(Sep. 12, 2018 06:58 AM)ryszardzonk Wrote:  Anyways would this code change also fix problems with vc.ru and tjournal.ru so they would not need [SSL Pass-Thru]? Clearly they were subdomains in the shown example
Yes, just like subdomains. The proposed option in [SSL Pass-Thru] works, but this method is not desirable.

pepak, question to you or to someone who understands. I work on a router (ubuntu server 16.04). And sometimes there is an error that the certificate is not valid because of different time on the router and clients. Clients are synchronized from this router to NTP, but not always accurately, there is a difference of 0.5 seconds and this error occurs. I can not make exact synchronization of time. I wrote about this here https://prxbx.com/forums/showthread.php?...5#pid19135
The question is, is it possible to specify the time when the certificate should start validating more early, rather than at the time the certificate was generated?
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: