The Un-Official Proxomitron Forum

Full Version: ProxHTTPSProxyMII: Reloaded
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
(Jun. 11, 2018 01:20 PM)whenever Wrote: [ -> ]Feel free to do it.
I am not sure what do You mean by that. It is your repository after all Wink Besides even if You give me access to it I still would not know what to do with it as my git/github skills are very limited Sad

By running diff on your git and JJoe's package I got this patch which if applied along with updated certs would bring code up to date.

Code:
diff -Naur wheever-ProxHTTPSProxyMII-da06c09/CertTool.py "ProxHTTPSProxyMII_py 1.5wipb/CertTool.py"
--- wheever-ProxHTTPSProxyMII-da06c09/CertTool.py       2017-06-19 22:20:22.000000000 +0200
+++ "ProxHTTPSProxyMII_py 1.5wipb/CertTool.py"  2018-04-20 16:26:28.000000000 +0200
@@ -14,6 +14,7 @@
import os
import time
import OpenSSL
+import ipaddress

def create_CA(capath):
     key = OpenSSL.crypto.PKey()
@@ -77,15 +78,24 @@
         cert.gmtime_adj_notBefore(0)
         cert.gmtime_adj_notAfter(60 * 60 * 24 * 3652)
         cert.set_issuer(ca.get_subject())
-        if commonname.startswith('.'):
-          domain = '*' + commonname
-        else:
-          domain = commonname
-        cert.get_subject().CN = domain
+        try:
+            ip = ipaddress.ip_address(commonname)
+            cert.get_subject().CN = commonname
+            san = 'IP: ' + commonname
+            cert.add_extensions([OpenSSL.crypto.X509Extension(b"subjectAltName", False, san.encode())])
+#            print('IP')
+        except ValueError:
+            # protocol limits common name field to 64 characters.
+            # commonnameshort may use wildcard to 'shorten' commonname.
+            commonnameshort = '*.' + commonname.partition('.')[-1] if commonname.count('.') >= 2 else commonname
+            cert.get_subject().CN = commonnameshort
+            san = 'DNS: ' + commonname
+            cert.add_extensions([OpenSSL.crypto.X509Extension(b"subjectAltName", False, san.encode())])
+#            print('DNS')
+        except:
+            print('Address not found')
         cert.set_serial_number(int(time.time()*10000))
         cert.set_pubkey(ca.get_pubkey())
-        cert.add_extensions(
-           [OpenSSL.crypto.X509Extension(b"subjectAltName", False, str.encode("DNS:"+domain))])
         cert.sign(key, "sha256")
         with open(certfile, 'wb') as fp:
             fp.write(OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert))
diff -Naur wheever-ProxHTTPSProxyMII-da06c09/ProxHTTPSProxy.py "ProxHTTPSProxyMII_py 1.5wipb/ProxHTTPSProxy.py"
--- wheever-ProxHTTPSProxyMII-da06c09/ProxHTTPSProxy.py 2017-06-19 22:20:22.000000000 +0200
+++ "ProxHTTPSProxyMII_py 1.5wipb/ProxHTTPSProxy.py"    2018-04-20 16:08:56.000000000 +0200
@@ -5,7 +5,7 @@

_name = 'ProxHTTPSProxyMII'
__author__ = 'phoenix'
-__version__ = 'v1.4'
+__version__ = 'v1.5wipb'

CONFIG = "config.ini"
CA_CERTS = "cacert.pem"
@@ -138,7 +138,7 @@
     server_version = "%s FrontProxy/%s" % (_name, __version__)

     def do_CONNECT(self):
-        "Descrypt https request and dispatch to http handler"
+        "Decrypt https request and dispatch to http handler"

         # request line: CONNECT www.example.com:443 HTTP/1.1
         self.host, self.port = self.path.split(":")
@@ -162,7 +162,8 @@
             self.wfile.write(("HTTP/1.1 200 Connection established\r\n" +
                               "Proxy-agent: %s\r\n" % self.version_string() +
                               "\r\n").encode('ascii'))
-            commonname = '.' + self.host.partition('.')[-1] if self.host.count('.') >= 2 else self.host
+#            commonname = '.' + self.host.partition('.')[-1] if self.host.count('.') >= 2 else self.host
+            commonname = self.host
             dummycert = get_cert(commonname)
             # set a flag for do_METHOD
             self.ssltunnel = True

I would welcome it a lot if you updated repo with it and certs mentioned earlier Pray

EDIT:
I am unable to visit losyziemi.pl due to cert error.
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))
I went to SSLLabls and it pointed me to other server https://www.ssllabs.com/ssltest/analyze....num.edu.pl From that I downloaded http://secure2.alphassl.com/cacert/gsalphasha2g2r1.crt and converted it to PEM and added to cacert.pem but it still does not work
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'losyziemi.pl' doesn't match either of '*.platinum.edu.pl', 'platinum.edu.pl'",),))
Is there anything that can be done about? If I read SSLLabs right it as it is likely missing interm cert.

EDIT 2:
There is more of those matching errors like
Code:
HTTPSConnectionPool(host='mf24.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'mf24.pl' doesn't match 's4.masternet.pl'",),))
Perhaps disabling this specific error only for specific site is possible?
The problem is the hostname (losyziemi.pl, mf24.pl) is not on the certificate. These sites are probably intended to be http only. I added the 'losyziemi.pl' to [SSL No-Verify] in config.ini and the server returned a redirect to http.

Solution is to use http.

(Jun. 12, 2018 06:18 AM)ryszardzonk Wrote: [ -> ]EDIT:
I am unable to visit losyziemi.pl due to cert error.
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)'),))
I went to SSLLabls and it pointed me to other server https://www.ssllabs.com/ssltest/analyze....num.edu.pl From that I downloaded http://secure2.alphassl.com/cacert/gsalphasha2g2r1.crt and converted it to PEM and added to cacert.pem but it still does not work
Code:
HTTPSConnectionPool(host='losyziemi.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'losyziemi.pl' doesn't match either of '*.platinum.edu.pl', 'platinum.edu.pl'",),))
Is there anything that can be done about? If I read SSLLabs right it as it is likely missing interm cert.

EDIT 2:
There is more of those matching errors like
Code:
HTTPSConnectionPool(host='mf24.pl', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'mf24.pl' doesn't match 's4.masternet.pl'",),))
Perhaps disabling this specific error only for specific site is possible?
(Jun. 11, 2018 01:20 PM)whenever Wrote: [ -> ]Sorry for late reply. Cheers

No worries. Smile!

I've got some work to do before I publish.
http://www.prxbx.com/forums/showthread.php?tid=2172

updated.

(Jun. 11, 2018 01:20 PM)whenever Wrote: [ -> ]JJoe should could modify the post.

@JJoe, please check email for file hosting details.

Sorry for late reply. Cheers
I keep getting this exception in the logs. Anything can be done about it?

Code:
Exception happened during processing of request from ('127.0.0.1', 38518)
Traceback (most recent call last):
  File "/usr/lib64/python3.5/socketserver.py", line 625, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python3.5/socketserver.py", line 354, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib64/python3.5/socketserver.py", line 681, in __init__
    self.handle()
  File "/usr/lib64/python3.5/http/server.py", line 422, in handle
    self.handle_one_request()
  File "/opt/proxhttpsproxy/ProxyTool.py", line 115, in handle_one_request
    BaseHTTPRequestHandler.handle_one_request(self)
  File "/usr/lib64/python3.5/http/server.py", line 410, in handle_one_request
    method()
  File "/opt/proxhttpsproxy/ProxHTTPSProxy.py", line 196, in do_METHOD
    if any((fnmatch.fnmatch(self.host, pattern) for pattern in pools.blacklist)):
  File "/opt/proxhttpsproxy/ProxHTTPSProxy.py", line 196, in <genexpr>
    if any((fnmatch.fnmatch(self.host, pattern) for pattern in pools.blacklist)):
  File "/usr/lib64/python3.5/fnmatch.py", line 34, in fnmatch
    name = os.path.normcase(name)
  File "/usr/lib/python-exec/python3.5/../../../lib64/python3.5/posixpath.py", line 54, in normcase
    "not '{}'".format(s.__class__.__name__))
TypeError: normcase() argument must be str or bytes, not 'NoneType'
(Jun. 18, 2018 12:58 PM)ryszardzonk Wrote: [ -> ]I keep getting this exception in the logs.

Do you see this in 1.5 or 1.5wipb or both?
(Jun. 18, 2018 08:04 PM)JJoe Wrote: [ -> ]
(Jun. 18, 2018 12:58 PM)ryszardzonk Wrote: [ -> ]I keep getting this exception in the logs.

Do you see this in 1.5 or 1.5wipb or both?

So far I have only used 1.5wipb and didn't change to 1.5 as *.py files are identical in it. Now I updated cert file to new version, but I doubt it would influence anything.
It is also hard to say when it happens as program keeps running fine regardless.
For 33 hours application has run it appeared 35 times in the log.
(Jun. 18, 2018 08:59 PM)ryszardzonk Wrote: [ -> ]*.py files are identical in it.

That's correct. This may have been easier to trobleshoot, had you said this came with 1.5. Wink

Is there something wrong or odd in the blacklist section of 'config.ini'?
The error appears to be that 'name' has no value. In the traceback, os.path.normcase(name) follows a pattern match check for the blacklist.

I haven't seen this error but I haven't used python3.5+.
(Jan. 07, 2015 11:16 AM)whenever Wrote: [ -> ]Attached is a launcher modified from https://github.com/phuslu/taskbar. If extracted into the ProxHTTPSProxyMII directory, the Launcher.exe can start the program with the console window minimized to the system tray. Left click on the system tray icon can restore or minimize the window again.

The right click menu has more options but is hard coded with Chinese characters. You have to either modify the source code and recompile, or hex edit the exe file to change, which I'm not good at either.

Another choice is RBTray, which can minimize almost any window to the system tray.

Hello!
Many thanks for ProxHTTPSProxyMII. I use this program in conjunction with HandyCache. Everything is great.
The only thing is that the proposed Launcher is not quite convenient for me.
I wrote my version of the launcher for ProxHTTPSProxyMII, which in my opinion is more convenient.

Launcher_ProxHTTPSProxy: https://yadi.sk/d/1w0C8R1X3YHUqd

This utility allows you to:
- hide / show the ProxHTTPSProxyMII window by clicking on the icon in the system tray or through the context menu of this icon;
- automatically run ProxHTTPSProxyMII, if its window is accidentally closed;
- open the context menu of the icon in the system tray config.ini file for viewing and editing.

Together with Launcher_ProxHTTPSProxy I spread its source code written on HiAsm 4.4 build 186: http://hiasm.com/

I think that Launcher_ProxHTTPSProxy maybe someone like it and come in handy.
Sorry for my English, I'm writing through an interpreter.

Edit by JJoe: Fixed link to https://github.com/phuslu/taskbar
(Jun. 18, 2018 10:01 PM)JJoe Wrote: [ -> ]Is there something wrong or odd in the blacklist section of 'config.ini'?
The error appears to be that 'name' has no value. In the traceback, os.path.normcase(name) follows a pattern match check for the blacklist.

my [blacklist] entries had nothing out of ordinary, but for testing purposes I hashed all those entries out, but it did not help as errors still appear in the logs.

(Jun. 18, 2018 10:01 PM)JJoe Wrote: [ -> ]I haven't seen this error but I haven't used python3.5+.
Which python version do You use?
(Jun. 23, 2018 08:24 PM)ryszardzonk Wrote: [ -> ]
(Jun. 18, 2018 10:01 PM)JJoe Wrote: [ -> ]I haven't seen this error but I haven't used python3.5+.
Which python version do You use?
3.4.4.
On Gentoo python 3.4 series got deprecated on May 22 and python 3.6 became default. Theoretically I can install it back, but it would not last much longer as:
Quote:If you are still using Python 3.4, please consider switching to a newer
version as it is reaching its end-of-life. The end-of-life dates
for the currently used versions are:

Python 3.4 2019-03-16
Python 2.7 2020-01-01
Python 3.5 2020-09-13 [1]
https://devguide.python.org/#status-of-python-branches

So testing with newer python versions would be welcomed Wink
Attached you will find eight patches to Version 1.5 (20180616):

1) Notes about creating an .exe file of ProxHTTPSProxy on Windows. Given the directory contents, it is obvious you are using a different mechanism, but this one works fine for me, so I made a note of that.

2) All configuration data got moved to the Config subdirectory. The purpose of that is to adhere to the least-privilege principle - the configs may need to be writable, but the code should only be read-only.

3) If a config file is missing, display a message rather than an exception trace.

4) Introduced a more reliable check for the Certs directory. This one works with junctions and also verifies that the directory is writable.

5) Created an object for the configuration data. This object can be shared between different modules easily.

6) Added support for on-the-fly certificate whitelisting: If a certificate error is displayed, the page now contains a link which will add the certificate to a whitelist temporarily (until ProxHTTPSProxy's restart).

7) Make the locations of trusted certificates file, signing certificate file and generated certificates directory configurable.

8) Made the location of the config file itself configurable.

Please bear with me - this is my first attempt to write anything in Python.
(Jun. 24, 2018 06:43 AM)ryszardzonk Wrote: [ -> ]So testing with newer python versions would be welcomed Wink

Agreed but Python 3.5+ cannot be used on Windows XP or earlier.
(Jun. 24, 2018 11:25 AM)pepak Wrote: [ -> ]Attached you will find eight patches to Version 1.5 (20180616):

Looks good and most welcome. Thumbs Up

But...

What are you using to generate and apply the patches?
I have been unable to find a utility that will apply all the patches correctly on Windows 10.

Could you zip the source and upload?
For our purposes, you could temporarily tag it as 1.5.1wip.
ProxHTTPSProxyMII: Development may be the more appropriate thread.

Thanks
Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Reference URL's