ProxHTTPSProxy, a Proxomitron SSL Helper Program

[whenever]

ProxHTTPSProxy is not being developed any more. It's replaced by ProxHTTPSProxyMII.

One of the purposes I decided to learn Python Programming Language is to write a proxy which can do https interception to solve Proxomitron's SSL issue, so here comes the ProxHTTPSProxy, a Proxomitron SSL Helper Program.

How does it work?

- for every https request, it returns a "307 Moved Temporarily" response with a "Location" header pointing to the http version of the request. The purpose is to switch the browser from https mode to http mode.

- for every http request, it fetches the content through https protocol and feeds the decrypted content to the browser. The communication between ProxHTTPSProxy and the remote server is https while the communication between ProxHTTPSProxy and the browser is still http so Proxomitron gets a chance to filter the content.

How to install?

- install Python 2.x branch from http://www.python.org/download/
- install OpenSSL from http://www.slproweb.com/products/Win32OpenSSL.html
- copy Proxomitron's proxcert.pem into ProxHTTPSProxy.py directory or simply throw ProxHTTPSProxy.py into Proxomitron directory
- execute ProxHTTPSProxy.py

ProxHTTPSProxy is designed originally to work as a parent proxy for Proxomitron but now it can be used as a general purpose proxy server too. It supports http and socks5 parent proxy and is able to display outgoing and incoming headers. See config.py for available options.

How to work with Proxomitron?

For https pages you want to filter, simply tell Proxomitron to use ProxHTTPSProxy as the parent proxy. Remember to forward both http and https urls, so ProxHTTPSProxy can issue a 307 redirect for https and fetch https content for http.

This is how it look like in sidki's Exceptions-U.ptxt:

- If you want to filter all https, set your browser's https proxy to ProxHTTPSProxy

Code:

# ProxHTTPSProxy
[^/]++:443&$URL(http://*) $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

- If you want to filter specific https

Code:

# ProxHTTPSProxy
# Add specific URLs like below
ssl.scroogle.org:         $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

Since Proxomitron is not needed to do https interception any more, you can optionally uncheck the https setting in Proxomitron's config settings.

Exe build: http://proxfilter.net/ProxHTTPSProxy.zip

Any feedbacks is welcome.

Warning: Currently ProxHTTPSProxy is not doing any kind of certificate check, use it as your own risk!

[ProxRocks]

AWESOME!...

i'll have to do more extensive testing over the weekend, but from what i've seen so far, this is TOTALLY COOL!

[ecli]

Although I don't currently filter HTTPS, I might take a look in the future. Thank you for sharing!

[JJoe]

Well...

I downloaded and installed Python 2.6.5. 32bit and ProxHTTPSProxy on Win7 64bit, added the line to Exceptions-U.ptxt, started ProxHTTPSProxy, and tried https://ssl.scroogle.org/.
It doesn't work for me. Yet...
But lets wait for more feedback. It could be just me.

All I see is many connections being opened to ssl.scroogle.org.

http://local.ptron/.pinfo/urls/ shows

Code:

Closed    429    000        0    http://ssl.scroogle.org:443/
Closed    427    000        0    http://ssl.scroogle.org:443/
Closed    425    000        0    http://ssl.scroogle.org:443/
Closed    423    000        0    http://ssl.scroogle.org:443/
Closed    421    000        0    http://ssl.scroogle.org:443/
Closed    419    000        0    http://ssl.scroogle.org:443/
Closed    417    000        0    http://ssl.scroogle.org:443/
Closed    415    000        0    http://ssl.scroogle.org:443/

Log shows

Code:

BlockList 415: in Exceptions-U, line 976
GET 415 : User Keywords: .i_proxy:3.
BlockList 415: in User-Agents, line 45

+++GET 415+++
Using Proxy - 127.0.0.1:8081
GET http://ssl.scroogle.org:443/ HTTP/1.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Host: ssl.scroogle.org:443
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
Connection: keep-alive
BlockList 416: in Exceptions-U, line 976
GET 416 : User Keywords: .i_proxy:3.

+++GET 416+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.0

+++SSL 416:+++
SSL Pass-Thru: CONNECT https://ssl.scroogle.org:443/
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5
+++CLOSE 416+++
BlockList 417: in Exceptions-U, line 976
GET 417 : User Keywords: .i_proxy:3.
BlockList 417: in User-Agents, line 45

and repeats till I end ProxHTTPSProxy.

I'm also wondering about the HTTP/1.0?
Is the port number removed from the url that is passed? Proxo may treat 443 as https.
If the https urls were 'tagged', like Proxo's https.., you wouldn't need to keep a list.

Hmm... I better get some work done.

[ProxRocks]

admittedly, i haven't tried YET...
it's on my weekend-to-do-list...

my fingers are crossed that this will solve some "half-ssl" problems i've been having at ING Direct where 'some' content is still sneaking in "non-halved"...

[Graycode]

(May. 19, 2010 06:56 AM)whenever Wrote:  Any feedbacks is welcome.

That's very cool!

Code:

if method == 'POST':
    post_data_len = int(self.headers['Content-Length'])

Within do_METHOD it might be better to just check a Length header for non-zero, regardless of method=POST. I think for example AJAX may allow a POST having zero content, and I don't know what happens if Python tries to read or write zero length.

Code:

post_data_len = int(self.headers['Content-Length'])
if post_data_len > 0:

Code:

self.send_response(200)

I'm lost there. What if the server's response code was not 200? For example a 304 is very common. Also unclear whether that's a HTTP/1.0 or 1.1 response, but maybe connection persistence is not a factor in what you're using this for.

Code:

if keyword.lower() in ('transfer-encoding',):
    #print '%s: %s removed' % (keyword, value)
    continue

Dropping the Transfer-Encoding header seems odd, apparently Python already accounted for chunked data but not for gzip, deflate, etc? I'm not sure if any consideration for Content-Encoding is desired in that situation. Transfer-Encoding is generally hop-by-hop, Content-Encoding is more end-to-end.

Code:

def do_CONNECT(self):
        host_port = self.path

I don't see how the Python proxy would know the requested path. The SSL CONNECT method normally just has '/' even if the browser wanted '/something/other.htm'. It looks to my non-Python eyes that the 307 redirection would always send the browser to the root of the SSL host and not to the location that was wanted.

Code:

resp = 'HTTP/1.1 307 Moved Temporarily\r\nLocation: http://%s%s\r\n\r\n' % (host_port, path)
        ssl_sock.send(resp)
        ssl_sock.close()

The response is HTTP/1.1 so Proxo should assume persistence, but then its socket gets quickly closed.
Consider adding 'Connection: Close\r\n' to that, and maybe also 'Content-Length: 0\r\n'.

[whenever]

(May. 19, 2010 08:01 PM)JJoe Wrote:  It doesn't work for me. Yet...

From you log I didn't see following http request, it seems ProxHTTPSProxy failed to issue the 307 response.

What did you see in ProxHTTPSProxy's log window? It might be because the SSL tunnel failed to establish.

(May. 19, 2010 08:01 PM)JJoe Wrote:  I downloaded and installed Python 2.6.5. 32bit and ProxHTTPSProxy on Win7 64bit

I don't have a 64bit machine to test. Could you please test with the 64bit Python 2.6.5?

(May. 19, 2010 08:01 PM)JJoe Wrote:  I'm also wondering about the HTTP/1.0?

If you mean the "Connection established" message ProxHTTPSProxy issued back, the HTTP/1.0 doesn't matter. It just tells the client it is ready to establish the SSL tunnel and only from there, all traffic is encrypted with SSL.

(May. 19, 2010 08:01 PM)JJoe Wrote:  Is the port number removed from the url that is passed? Proxo may treat 443 as https.

No. You can see the port is there when the browser and Proxo pass through the message:

Code:

CONNECT https://ssl.scroogle.org:443/ HTTP/1.0

ProxHTTPSProxy will pass this port to the http request in case some HTTPS server is not using the regular 443 port.

(May. 19, 2010 08:01 PM)JJoe Wrote:  If the https urls were 'tagged', like Proxo's https.., you wouldn't need to keep a list.

I didn't get your point. ProxHTTPSProxy doesn't keep a list.

[whenever]

(May. 19, 2010 09:26 PM)Graycode Wrote:  I think for example AJAX may allow a POST having zero content

Thanks. Now it is:

Code:

post_data = self.rfile.read(post_data_len) if post_data_len > 0 else ''

(May. 19, 2010 09:26 PM)Graycode Wrote:  

Code:

self.send_response(200)

I'm lost there. What if the server's response code was not 200? For example a 304 is very common.

Python's urllib2 module handles redirections, cookies and many other things so I assume it will finally fetch the content.

If not for example 304 it will raise an Exception, which will be caught in my try statement and the response code will be passed to the browser.

(May. 19, 2010 09:26 PM)Graycode Wrote:  Also unclear whether that's a HTTP/1.0 or 1.1 response, but maybe connection persistence is not a factor in what you're using this for.

To keep things simple, I am adding "Connection: close" to each request.

Code:

req.add_header("Connection", "close")

(May. 19, 2010 09:26 PM)Graycode Wrote:  

Code:

if keyword.lower() in ('transfer-encoding',):
    #print '%s: %s removed' % (keyword, value)
    continue

Dropping the Transfer-Encoding header seems odd, apparently Python already accounted for chunked data but not for gzip, deflate, etc? I'm not sure if any consideration for Content-Encoding is desired in that situation.

Yes, urllib2 handles chunked data but won't touch gzip, deflate etc. That's why I dropped Transfer-Encoding header while keep Content-Encoding header.

(May. 19, 2010 09:26 PM)Graycode Wrote:  

Code:

def do_CONNECT(self):
        host_port = self.path

I don't see how the Python proxy would know the requested path. The SSL CONNECT method normally just has '/' even if the browser wanted '/something/other.htm'. It looks to my non-Python eyes that the 307 redirection would always send the browser to the root of the SSL host and not to the location that was wanted.

As the statement itself explained, I get only "host_port" from "self.path".

I am reading the requested path from the following http request in the SSL tunnel:

Code:

# Read 512 bytes to extract url path
# Will it cause issue not reading all data?
path = ssl_sock.recv(512).split(None, 2)[1]

From your point of view, will it cause issue not reading all data from the socket?

(May. 19, 2010 09:26 PM)Graycode Wrote:  

Code:

resp = 'HTTP/1.1 307 Moved Temporarily\r\nLocation: http://%s%s\r\n\r\n' % (host_port, path)
        ssl_sock.send(resp)
        ssl_sock.close()

The response is HTTP/1.1 so Proxo should assume persistence, but then its socket gets quickly closed.
Consider adding 'Connection: Close\r\n' to that, and maybe also 'Content-Length: 0\r\n'.

Thanks. Now I understand why sidki's config was adding 'Content-Length: 0' to my response. Where are you sidki? Come back please ...

Now it is:

Code:

resp = ('HTTP/1.1 307 Moved Temporarily\r\n',
                'Location: http://%s%s\r\n' % (host_port, path),
                'Connection: Close\r\n',
                'Content-Length: 0\r\n',
                '\r\n')
        ssl_sock.send(''.join(resp))

[Graycode]

(May. 20, 2010 03:19 AM)whenever Wrote:  As the statement itself explained, I get only "host_port" from "self.path".

I am reading the requested path from the following http request in the SSL tunnel:

Code:

# Read 512 bytes to extract url path
# Will it cause issue not reading all data?
path = ssl_sock.recv(512).split(None, 2)[1]

From your point of view, will it cause issue not reading all data from the socket?

It seems like an issue. If in a browser I enter (https)example.com:443/path/more/stuff/for/page.htm and send that through a proxy, the CONNECT command nor any header will contain the requested path portion. It will be just a '/' slash and nothing about path, page, htm.

It's only enough information for a proxy to build a blind-tunnel with the target server within which encrypted content would flow, and in that muck is where the actual path / page will be transmitted. Hence it seems like the 307 redirection could only always send the browser to the root of the site but not to the path / page that was desired. That level of detail is not provided with the CONNECT method, it provides no clue about what is being requested.

In order to determine what is being requested the Python would have to establish the SSL hand-shakes with the browser to a sufficient state that the browser would be willing to cough up what it was wanting. That might be possible since you already have a 'proxcert.pem' that the browser would trust. Hopefully it's obvious that I don't know much about Python.

[whenever]

Graycode, give ProxHTTPSProxy a try and you will see how it works.

[Graycode]

I will, but it may be a few days before I can install Python.

I see now that your Python is using SSL before referencing the path:

Code:

ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=True)

So apparently the read() after that is getting the browser's "real" HTTP headers within SSL, and Python is handling the decryption? If so then that seems really slick, and the requested path is indeed identified for the 307 redirection.

[JJoe]

Of course, 64bit Python 2.6.5.

This time with Proxo filtering ssl

Code:

BlockList 1665: in Exceptions-U, line 976
GET 1665 : User Keywords: .i_proxy:3.
BlockList 1665: in User-Agents, line 45

+++GET 1665+++
Using Proxy - 127.0.0.1:8081
GET http://ssl.scroogle.org:443/ HTTP/1.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Host: ssl.scroogle.org:443
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
Connection: keep-alive
BlockList 1666: in Exceptions-U, line 976
GET 1666 : User Keywords: .i_proxy:3.

+++GET 1666+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.0
BlockList 1666: in Exceptions-U, line 976
GET 1666 : User Keywords: .i_proxy:3.
BlockList 1666: in User-Agents, line 45

+++SSL:GET 1666+++
Using Proxy - 127.0.0.1:8081
SSL cipher TLSv1 AES256-SHA (256 bits)
GET https://ssl.scroogle.org:443/ HTTP/1.1
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Host: ssl.scroogle.org:443
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
Connection: keep-alive
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5

+++SSL:RESP 1666+++
SSL cipher TLSv1 AES256-SHA (256 bits)
HTTP/1.1 307 Moved Temporarily
Location: http://ssl.scroogle.org:443/
+++CLOSE 1666+++

and repeats till I end ProxHTTPSProxy.

http://local.ptron/.pinfo/urls/ shows

Code:

Closed    1673    000        0    http://ssl.scroogle.org:443/
Closed    1672    307        0    https://ssl.scroogle.org:443/
Closed    1671    000        0    http://ssl.scroogle.org:443/
Closed    1670    307        0    https://ssl.scroogle.org:443/
Closed    1669    000        0    http://ssl.scroogle.org:443/
Closed    1668    307        0    https://ssl.scroogle.org:443/
Closed    1667    000        0    http://ssl.scroogle.org:443/
Closed    1666    307        0    https://ssl.scroogle.org:443/
Closed    1665    000        0    http://ssl.scroogle.org:443/

ProxHTTPSProxy's log window shows

Code:

ProxHTTPSProxy/0.1 serving on localhost:8081, <Ctrl-C> to stop ...
E3-PC - - [19/May/2010 23:06:41] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:47] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:47] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:47] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:48] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:48] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:48] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:48] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:48] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:49] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:49] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:49] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:49] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:49] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:50] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:50] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:50] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:50] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:50] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:51] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:51] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:51] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:51] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:51] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:52] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:53] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:53] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:53] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
Traceback (most recent call last):
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 78, in <module>
    server.serve_forever()
  File "C:\Python26amd64\lib\SocketServer.py", line 224, in serve_forever
    r, w, e = select.select([self], [], [], poll_interval)
KeyboardInterrupt

Well... All 4 possibilities are now showing something like

Code:

E3-PC - - [19/May/2010 23:37:04] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52659)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------

Time for restart

And now...

32 bit no ssl filtering by Proxo

Code:

E3-PC - - [20/May/2010 00:06:47] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49165)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------


BlockList 1: in Exceptions-U, line 976
GET 1 : User Keywords: .i_proxy:3.
BlockList 1: in User-Agents, line 45

+++GET 1+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Proxy-Connection: keep-alive
Host: ssl.scroogle.org
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang

+++SSL 1:+++
SSL Pass-Thru: CONNECT https://ssl.scroogle.org:443/
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5
+++CLOSE 1+++

and repeats but slower now.

32 bit Proxo filtering ssl

Code:

E3-PC - - [20/May/2010 00:15:26] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49207)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------


BlockList 25: in Exceptions-U, line 976
GET 25 : User Keywords: .i_proxy:3.
BlockList 25: in User-Agents, line 45

+++GET 25+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Proxy-Connection: keep-alive
Host: ssl.scroogle.org
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
BlockList 25: in Exceptions-U, line 976
GET 25 : User Keywords: .i_proxy:3.
BlockList 25: in User-Agents, line 45

+++SSL:GET 25+++
Using Proxy - 127.0.0.1:8081
SSL cipher SSLv3 AES256-SHA (256 bits)
GET https://ssl.scroogle.org:443/ HTTP/1.1
Host: ssl.scroogle.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5

and repeats.

64 bit no ssl filtering by Proxo

Code:

Exception happened during processing of request from ('127.0.0.1', 49215)
Traceback (most recent call last):
  File "C:\Python26amd64\lib\SocketServer.py", line 558, in process_request_thre
ad
    self.finish_request(request, client_address)
  File "C:\Python26amd64\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26amd64\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 323, in handle_one_request

    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26amd64\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26amd64\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------


+++CLOSE 30+++
BlockList 31: in Exceptions-U, line 976
GET 31 : User Keywords: .i_proxy:3.
BlockList 31: in User-Agents, line 45

+++GET 31+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Proxy-Connection: keep-alive
Host: ssl.scroogle.org
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang

+++SSL 31:+++
SSL Pass-Thru: CONNECT https://ssl.scroogle.org:443/
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5

no repeating seen.

64 bit Proxo filtering ssl

Code:

E3-PC - - [20/May/2010 00:26:38] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49217)
Traceback (most recent call last):
  File "C:\Python26amd64\lib\SocketServer.py", line 558, in process_request_thre
ad
    self.finish_request(request, client_address)
  File "C:\Python26amd64\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26amd64\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 323, in handle_one_request

    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26amd64\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26amd64\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------


BlockList 32: in Exceptions-U, line 976
GET 32 : User Keywords: .i_proxy:3.
BlockList 32: in User-Agents, line 45

+++GET 32+++
Using Proxy - 127.0.0.1:8081
CONNECT https://ssl.scroogle.org:443/ HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Proxy-Connection: keep-alive
Host: ssl.scroogle.org
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
BlockList 32: in Exceptions-U, line 976
GET 32 : User Keywords: .i_proxy:3.
GET 32 : Cache-Control killed: max-age=0
BlockList 32: in User-Agents, line 45

+++SSL:GET 32+++
Using Proxy - 127.0.0.1:8081
SSL cipher SSLv3 AES256-SHA (256 bits)
GET https://ssl.scroogle.org:443/ HTTP/1.1
Host: ssl.scroogle.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.9.9
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Referer: http://slashdot.org/search/referrer-karma.php?q=Big+Bang
HTTP/1.0 200 Connection established
HTTP/1.0 Proxy-agent: ProxHTTPSProxy/0.1 Python/2.6.5

no repeating seen.

Quote:I didn't get your point. ProxHTTPSProxy doesn't keep a list.

Probably too late for me to think straight but wouldn't the user be expected to maintain a list?

Quote:This is how it look like in sidki's Exceptions-U.ptxt

Proxo could build a list for you, I think. Some sort of tag might remove the need.

What happens when https and http share an address or part of one?

Sorry for the lengthy post folks. I may shorten it after whenever reads it.

[whenever]

(May. 20, 2010 05:50 AM)Graycode Wrote:  and Python is handling the decryption?

Yes, python handles the decryption and encryption for you when reading and writing the SSL tunnel once the tunnel is established.

[whenever]

(May. 20, 2010 06:01 AM)JJoe Wrote:  

Code:

+++SSL:RESP 1666+++
SSL cipher TLSv1 AES256-SHA (256 bits)
HTTP/1.1 307 Moved Temporarily
Location: http://ssl.scroogle.org:443/
+++CLOSE 1666+++

So, 307 response was successfully issued.

(May. 20, 2010 06:01 AM)JJoe Wrote:  http://local.ptron/.pinfo/urls/ shows

Code:

Closed    1673    000        0    http://ssl.scroogle.org:443/
Closed    1672    307        0    https://ssl.scroogle.org:443/

http request follows https request, that's what we expect.

(May. 20, 2010 06:01 AM)JJoe Wrote:  ProxHTTPSProxy's log window shows

Code:

ProxHTTPSProxy/0.1 serving on localhost:8081, <Ctrl-C> to stop ...
E3-PC - - [19/May/2010 23:06:41] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
E3-PC - - [19/May/2010 23:06:47] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -

Only https requests here. Where are the http requests? Are you sure you have made Proxo to forward http requests to ProxHTTPSProxy too?

Let's bypass Proxo totally. Could you please test with the browser's http and https proxies both set to ProxHTTPSProxy?

(May. 20, 2010 06:01 AM)JJoe Wrote:  Probably too late for me to think straight but wouldn't the user be expected to maintain a list?

Yes, but it is Proxo that maintains the list while not ProxHTTPSProxy.

(May. 20, 2010 06:01 AM)JJoe Wrote:  Proxo could build a list for you, I think. Some sort of tag might remove the need.

What happens when https and http share an address or part of one?

That's what we have to think about later. After all it is just ver 0.1 now. The first thing is to get it running on your machine so you can take a hand in

[JJoe]

(May. 20, 2010 06:35 AM)whenever Wrote:  So, 307 response was successfully issued.

But I haven't seen one since restarting the machine.

(May. 20, 2010 06:35 AM)whenever Wrote:  Are you sure you have made Proxo to forward http requests to ProxHTTPSProxy too?

I have
ssl.scroogle.org[:/] $SET(0=i_proxy:3.) $SETPROXY(127.0.0.1:8081)

(May. 20, 2010 06:35 AM)whenever Wrote:  Let's bypass Proxo totally. Could you please test with the browser's http and https proxies both set to ProxHTTPSProxy?

Edit: Results are not to be trusted.
Although Opera's http and https were set to use ProxHTTPSProxy,
I now believe that Opera may have used the Proxomitron,
http://prxbx.com/forums/showthread.php?t...4#pid14244 .

Code:

E3-PC - - [20/May/2010 02:14:15] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49668)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------

and

Code:

E3-PC - - [20/May/2010 02:29:03] "CONNECT ssl.scroogle.org:443 HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49706)
Traceback (most recent call last):
  File "C:\Python26amd64\lib\SocketServer.py", line 558, in process_request_thre
ad
    self.finish_request(request, client_address)
  File "C:\Python26amd64\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26amd64\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26amd64\lib\BaseHTTPServer.py", line 323, in handle_one_request

    method()
  File "C:\Users\E3\Programs\PrxBlx\ProxHTTPSProxy.py", line 57, in do_CONNECT
    ssl_sock = ssl.wrap_socket(self.connection, certfile=cert_file, server_side=
True)
  File "C:\Python26amd64\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26amd64\lib\ssl.py", line 113, in __init__
    cert_reqs, ssl_version, ca_certs)
SSLError: [Errno 336265218] _ssl.c:337: error:140B0002:SSL routines:SSL_CTX_use_
PrivateKey_file:system lib
----------------------------------------

Both on Opera and Win7. No repeating seen.

(May. 20, 2010 06:35 AM)whenever Wrote:  That's what we have to think about later. After all it is just ver 0.1 now. The first thing is to get it running on your machine so you can take a hand in

lol...

Have you added anything to your python install?
I'm assuming that you have installed only python-2.6.5.

Later