Post Reply 
ProxHTTPSProxyMII: Reloaded
Jun. 12, 2018, 06:18 AM (This post was last modified: Jun. 13, 2018 01:01 PM by ryszardzonk.)
Post: #256
RE: ProxHTTPSProxyMII: Reloaded
(Jun. 11, 2018 01:20 PM)whenever Wrote:  Feel free to do it.
I am not sure what do You mean by that. It is your repository after all Wink Besides even if You give me access to it I still would not know what to do with it as my git/github skills are very limited Sad

By running diff on your git and JJoe's package I got this patch which if applied along with updated certs would bring code up to date.

Code:
diff -Naur wheever-ProxHTTPSProxyMII-da06c09/CertTool.py "ProxHTTPSProxyMII_py 1.5wipb/CertTool.py"
--- wheever-ProxHTTPSProxyMII-da06c09/CertTool.py       2017-06-19 22:20:22.000000000 +0200
+++ "ProxHTTPSProxyMII_py 1.5wipb/CertTool.py"  2018-04-20 16:26:28.000000000 +0200
@@ -14,6 +14,7 @@
import os
import time
import OpenSSL
+import ipaddress

def create_CA(capath):
     key = OpenSSL.crypto.PKey()
@@ -77,15 +78,24 @@
         cert.gmtime_adj_notBefore(0)
         cert.gmtime_adj_notAfter(60 * 60 * 24 * 3652)
         cert.set_issuer(ca.get_subject())
-        if commonname.startswith('.'):
-          domain = '*' + commonname
-        else:
-          domain = commonname
-        cert.get_subject().CN = domain
+        try:
+            ip = ipaddress.ip_address(commonname)
+            cert.get_subject().CN = commonname
+            san = 'IP: ' + commonname
+            cert.add_extensions([OpenSSL.crypto.X509Extension(b"subjectAltName", False, san.encode())])
+#            print('IP')
+        except ValueError:
+            # protocol limits common name field to 64 characters.
+            # commonnameshort may use wildcard to 'shorten' commonname.
+            commonnameshort = '*.' + commonname.partition('.')[-1] if commonname.count('.') >= 2 else commonname
+            cert.get_subject().CN = commonnameshort
+            san = 'DNS: ' + commonname
+            cert.add_extensions([OpenSSL.crypto.X509Extension(b"subjectAltName", False, san.encode())])
+#            print('DNS')
+        except:
+            print('Address not found')
         cert.set_serial_number(int(time.time()*10000))
         cert.set_pubkey(ca.get_pubkey())
-        cert.add_extensions(
-           [OpenSSL.crypto.X509Extension(b"subjectAltName", False, str.encode("DNS:"+domain))])
         cert.sign(key, "sha256")
         with open(certfile, 'wb') as fp:
             fp.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
diff -Naur wheever-ProxHTTPSProxyMII-da06c09/ProxHTTPSProxy.py "ProxHTTPSProxyMII_py 1.5wipb/ProxHTTPSProxy.py"
--- wheever-ProxHTTPSProxyMII-da06c09/ProxHTTPSProxy.py 2017-06-19 22:20:22.000000000 +0200
+++ "ProxHTTPSProxyMII_py 1.5wipb/ProxHTTPSProxy.py"    2018-04-20 16:08:56.000000000 +0200
@@ -5,7 +5,7 @@

_name = 'ProxHTTPSProxyMII'
__author__ = 'phoenix'
-__version__ = 'v1.4'
+__version__ = 'v1.5wipb'

CONFIG = "config.ini"
CA_CERTS = "cacert.pem"
@@ -138,7 +138,7 @@
     server_version = "%s FrontProxy/%s" % (_name, __version__)

     def do_CONNECT(self):
-        "Descrypt https request and dispatch to http handler"
+        "Decrypt https request and dispatch to http handler"

         # request line: CONNECT www.example.com:443 HTTP/1.1
         self.host, self.port = self.path.split(":")
@@ -162,7 +162,8 @@
             self.wfile.write(("HTTP/1.1 200 Connection established\r\n" +
                               "Proxy-agent: %s\r\n" % self.version_string() +
                               "\r\n").encode('ascii'))
-            commonname = '.' + self.host.partition('.')[-1] if self.host.count('.') >= 2 else self.host
+#            commonname = '.' + self.host.partition('.')[-1] if self.host.count('.') >= 2 else self.host
+            commonname = self.host
             dummycert = get_cert(commonname)
             # set a flag for do_METHOD
             self.ssltunnel = True

I would welcome it a lot if you updated repo with it and certs mentioned earlier Pray

EDIT:
I am unable to visit losyziemi.pl due to cert error.
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))
I went to SSLLabls and it pointed me to other server https://www.ssllabs.com/ssltest/analyze....num.edu.pl From that I downloaded http://secure2.alphassl.com/cacert/gsalphasha2g2r1.crt and converted it to PEM and added to cacert.pem but it still does not work
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'losyziemi.pl' doesn't match either of '*.platinum.edu.pl', 'platinum.edu.pl'",),))
Is there anything that can be done about? If I read SSLLabs right it as it is likely missing interm cert.

EDIT 2:
There is more of those matching errors like
Code:
HTTPSConnectionPool(host='mf24.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'mf24.pl' doesn't match 's4.masternet.pl'",),))
Perhaps disabling this specific error only for specific site is possible?
Add Thank You Quote this message in a reply
Jun. 13, 2018, 11:46 PM
Post: #257
RE: ProxHTTPSProxyMII: Reloaded
The problem is the hostname (losyziemi.pl, mf24.pl) is not on the certificate. These sites are probably intended to be http only. I added the 'losyziemi.pl' to [SSL No-Verify] in config.ini and the server returned a redirect to http.

Solution is to use http.

(Jun. 12, 2018 06:18 AM)ryszardzonk Wrote:  EDIT:
I am unable to visit losyziemi.pl due to cert error.
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))
I went to SSLLabls and it pointed me to other server https://www.ssllabs.com/ssltest/analyze....num.edu.pl From that I downloaded http://secure2.alphassl.com/cacert/gsalphasha2g2r1.crt and converted it to PEM and added to cacert.pem but it still does not work
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'losyziemi.pl' doesn't match either of '*.platinum.edu.pl', 'platinum.edu.pl'",),))
Is there anything that can be done about? If I read SSLLabs right it as it is likely missing interm cert.

EDIT 2:
There is more of those matching errors like
Code:
HTTPSConnectionPool(host='mf24.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'mf24.pl' doesn't match 's4.masternet.pl'",),))
Perhaps disabling this specific error only for specific site is possible?
Add Thank You Quote this message in a reply
Jun. 13, 2018, 11:52 PM
Post: #258
RE: ProxHTTPSProxyMII: Reloaded
(Jun. 11, 2018 01:20 PM)whenever Wrote:  Sorry for late reply. Cheers

No worries. Smile!

I've got some work to do before I publish.
Add Thank You Quote this message in a reply
Jun. 17, 2018, 01:59 AM
Post: #259
RE: ProxHTTPSProxyMII: Reloaded
http://www.prxbx.com/forums/showthread.php?tid=2172

updated.

(Jun. 11, 2018 01:20 PM)whenever Wrote:  JJoe should could modify the post.

@JJoe, please check email for file hosting details.

Sorry for late reply. Cheers
Add Thank You Quote this message in a reply
Yesterday, 12:58 PM
Post: #260
RE: ProxHTTPSProxyMII: Reloaded
I keep getting this exception in the logs. Anything can be done about it?

Code:
Exception happened during processing of request from ('127.0.0.1', 38518)
Traceback (most recent call last):
  File "/usr/lib64/python3.5/socketserver.py", line 625, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python3.5/socketserver.py", line 354, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python3.5/socketserver.py", line 681, in __init__
    self.handle()
  File "/usr/lib64/python3.5/http/server.py", line 422, in handle
    self.handle_one_request()
  File "/opt/proxhttpsproxy/ProxyTool.py", line 115, in handle_one_request
    BaseHTTPRequestHandler.handle_one_request(self)
  File "/usr/lib64/python3.5/http/server.py", line 410, in handle_one_request
    method()
  File "/opt/proxhttpsproxy/ProxHTTPSProxy.py", line 196, in do_METHOD
    if any((fnmatch.fnmatch(self.host, pattern) for pattern in pools.blacklist)):
  File "/opt/proxhttpsproxy/ProxHTTPSProxy.py", line 196, in <genexpr>
    if any((fnmatch.fnmatch(self.host, pattern) for pattern in pools.blacklist)):
  File "/usr/lib64/python3.5/fnmatch.py", line 34, in fnmatch
    name = os.path.normcase(name)
  File "/usr/lib/python-exec/python3.5/../../../lib64/python3.5/posixpath.py", line 54, in normcase
    "not '{}'".format(s.__class__.__name__))
TypeError: normcase() argument must be str or bytes, not 'NoneType'
Add Thank You Quote this message in a reply
Yesterday, 08:04 PM
Post: #261
RE: ProxHTTPSProxyMII: Reloaded
(Yesterday 12:58 PM)ryszardzonk Wrote:  I keep getting this exception in the logs.

Do you see this in 1.5 or 1.5wipb or both?
Add Thank You Quote this message in a reply
Yesterday, 08:59 PM
Post: #262
RE: ProxHTTPSProxyMII: Reloaded
(Yesterday 08:04 PM)JJoe Wrote:  
(Yesterday 12:58 PM)ryszardzonk Wrote:  I keep getting this exception in the logs.

Do you see this in 1.5 or 1.5wipb or both?

So far I have only used 1.5wipb and didn't change to 1.5 as *.py files are identical in it. Now I updated cert file to new version, but I doubt it would influence anything.
It is also hard to say when it happens as program keeps running fine regardless.
For 33 hours application has run it appeared 35 times in the log.
Add Thank You Quote this message in a reply
Yesterday, 10:01 PM
Post: #263
RE: ProxHTTPSProxyMII: Reloaded
(Yesterday 08:59 PM)ryszardzonk Wrote:  *.py files are identical in it.

That's correct. This may have been easier to trobleshoot, had you said this came with 1.5. Wink

Is there something wrong or odd in the blacklist section of 'config.ini'?
The error appears to be that 'name' has no value. In the traceback, os.path.normcase(name) follows a pattern match check for the blacklist.

I haven't seen this error but I haven't used python3.5+.
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: