Post Reply 
ProxHTTPSProxy, a Proxomitron SSL Helper Program
Jun. 04, 2010, 05:29 PM
Post: #91
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
apologies for being so late to the punch bowl...


seems to be working flawlessly...
though i'm not seeing the "difference" between 'half_ssl = 0' and 'half_ssl = 1' ???
Add Thank You Quote this message in a reply
Jun. 04, 2010, 06:11 PM
Post: #92
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 19, 2010 06:56 AM)whenever Wrote:  How to install?

- install Python 2.x branch from http://www.python.org/download/
- install OpenSSL from http://www.slproweb.com/products/Win32OpenSSL.html
- copy Proxomitron's proxcert.pem into ProxHTTPSProxy.py directory or simply throw ProxHTTPSProxy.py into Proxomitron directory
- execute ProxHTTPSProxy.py

is it possible to copy the OpenSSL's .dll's to the .py directory like we are the .pem, rather than having OpenSSL "installed"...
Add Thank You Quote this message in a reply
Jun. 04, 2010, 06:48 PM
Post: #93
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 28, 2010 06:56 AM)whenever Wrote:  I uploaded an exe version here in case somebody doesn't like to install python.

this python newbie can't figure out how to create the .exe (i installed the py2exe) - am i overlooking something obvious?
Add Thank You Quote this message in a reply
Jun. 05, 2010, 02:53 AM (This post was last modified: Jun. 05, 2010 02:55 AM by whenever.)
Post: #94
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 04, 2010 05:29 PM)ProxRocks Wrote:  though i'm not seeing the "difference" between 'half_ssl = 0' and 'half_ssl = 1' ???

If 'half_ssl = 1', you should see http:// while not https:// in the address bar for https pages.

(Jun. 04, 2010 06:11 PM)ProxRocks Wrote:  is it possible to copy the OpenSSL's .dll's to the .py directory like we are the .pem, rather than having OpenSSL "installed"...

I don't know. The 2 machines I tested have already OpenSSL installed though I can't remember when. Maybe got installed with other software.

If you could test if the simple copy of DLLs works or not, that would be appreciated. Smile!

(Jun. 04, 2010 06:48 PM)ProxRocks Wrote:  this python newbie can't figure out how to create the .exe (i installed the py2exe) - am i overlooking something obvious?

I haven't tried py2exe. I use cx_Freeze, which is much simpler. The 0.3 exe is at: http://proxfilter.net/ProxHTTPSProxy.zip

Modify config.py to change settings.
Add Thank You Quote this message in a reply
Jun. 05, 2010, 04:30 AM
Post: #95
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Sorry I'm late.

(Jun. 04, 2010 03:43 PM)whenever Wrote:  Please test and report issue. Have fun!

Scroogle works. developer.mozilla.org failed to load once.
The other two haven't loaded yet:

Code:
ProxHTTPSProxy/0.3 serving now, <Ctrl-C> to stop ...
Listen Addr  : localhost:8081
Parent Proxy :
Half_SSL     : No
Debug        : On
--------------
E3-PC - - [04/Jun/2010 23:18:34] GET https://login.yahoo.com:443/ HTTP/1.1

accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
accept-encoding: gzip,deflate
accept-language: en-us,en;q=0.5
accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai
n;q=0.8,image/png,*/*;q=0.5
connection: close
host: login.yahoo.com
user-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/2
0081217 Firefox/2.0.0.20



--------------
E3-PC - - [04/Jun/2010 23:18:39] GET https://bugzilla.mozilla.org:443/ HTTP/1.1

accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
accept-encoding: gzip,deflate
accept-language: en-us,en;q=0.5
accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai
n;q=0.8,image/png,*/*;q=0.5
connection: close
host: bugzilla.mozilla.org
user-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/2
0081217 Firefox/2.0.0.20



--------------

The Proxomitron shows the connection closed and no reply headers.

Will check the other features tomorrow. Smile!

Bedtime
Add Thank You Quote this message in a reply
Jun. 05, 2010, 07:27 AM
Post: #96
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 05, 2010 04:30 AM)JJoe Wrote:  Scroogle works. developer.mozilla.org failed to load once.
The other two haven't loaded yet

Well, it is the old socket shutdown issue. Version 0.3a commented out all server sockets shutdown statements while client socket are still explicit shutdown.

All the 4 https sites working now.

Like the exe build, settings are moved to config.py.


Attached File(s)
.zip  ProxHTTPSProxy 0.3a.zip (Size: 7.9 KB / Downloads: 689)
Add Thank You Quote this message in a reply
Jun. 05, 2010, 07:26 PM (This post was last modified: Jun. 05, 2010 07:27 PM by JJoe.)
Post: #97
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 05, 2010 07:27 AM)whenever Wrote:  Version 0.3a commented out all server sockets shutdown statements while client socket are still explicit shutdown.

hmmm

Assuming that you are still sliding down the slippery slope...

A problem passing plain http while in Half_SSL mode with socks.

Code:
ProxHTTPSProxy/0.3a serving now, <Ctrl-C> to stop ...
Listen Addr  : localhost:8081
Parent Proxy : 127.0.0.1:9050@socks5
Half_SSL     : Yes
Debug        : On
--------------
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51894)
Traceback (most recent call last):
  File "C:\Python26\lib\SocketServer.py", line 558, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Python26\lib\SocketServer.py", line 320, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Python26\lib\SocketServer.py", line 615, in __init__
    self.handle()
  File "C:\Python26\lib\BaseHTTPServer.py", line 329, in handle
    self.handle_one_request()
  File "C:\Python26\lib\BaseHTTPServer.py", line 323, in handle_one_request
    method()
  File "C:\Users\E3\Programs\ProxHTTPSProxy 0.3a\ProxHTTPSProxy.py", line 48, in
do_METHOD
    server_sock = ssl.wrap_socket(server_sock)
  File "C:\Python26\lib\ssl.py", line 350, in wrap_socket
    suppress_ragged_eofs=suppress_ragged_eofs)
  File "C:\Python26\lib\ssl.py", line 118, in __init__
    self.do_handshake()
  File "C:\Python26\lib\ssl.py", line 293, in do_handshake
    self._sslobj.do_handshake()
SSLError: [Errno 1] _ssl.c:480: error:140770FC:SSL routines:SSL23_GET_SERVER_HEL
LO:unknown protocol
----------------------------------------

Code:
+++GET 4784+++
Using Proxy - 127.0.0.1:8081
GET http://www.scroogle.org/ HTTP/1.1
Host: www.scroogle.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/20081217 Firefox/2.0.0.20
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
+++CLOSE 4784+++

Works without Half_SSL, so I'm assuming feature creep. Wink

The rest seems ok.

Back after dinner.
Add Thank You Quote this message in a reply
Jun. 05, 2010, 11:17 PM
Post: #98
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 05, 2010 07:26 PM)JJoe Wrote:  A problem passing plain http while in Half_SSL mode with socks.

It doesn't seem to be a problem.

It behaves like the old ProxHTTPSProxy when in Half_SSL mode. That is, all http requests are treated like results of 307 redirecting and will be fetched through https protocol.

Obviously http://www.scroogle.org is not speaking https.

You still need those Proxo entries to work under Half_SSL mode.
Add Thank You Quote this message in a reply
Jun. 06, 2010, 12:30 AM
Post: #99
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Sorry, I think I was too cute with my words.

ProxHTTPSProxy, when not in Half_SSL mode, does pass plain http.
Polipo is not needed to handle socks when using Tor, I think.

Code:
ProxHTTPSProxy/0.3a serving now, <Ctrl-C> to stop ...
Listen Addr  : localhost:8081
Parent Proxy : 127.0.0.1:9050@socks5
Half_SSL     : No
Debug        : On
--------------
E3-PC - - [05/Jun/2010 18:59:13] GET http://scroogle.org/ HTTP/1.1

accept-charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
accept-encoding: gzip,deflate
accept-language: en-us,en;q=0.5
accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plai
n;q=0.8,image/png,*/*;q=0.5
cache-control: max-age=0
connection: close
host: scroogle.org
user-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.20) Gecko/2
0081217 Firefox/2.0.0.20

HTTP/1.1 200 OK
Date: Sat, 05 Jun 2010 23:59:05 GMT
Server: Apache/2.2.3 (CentOS)
Last-Modified: Fri, 04 Jun 2010 21:57:02 GMT
ETag: "112c0d5-11c8-67288780"
Accept-Ranges: bytes
Content-Length: 4552
Connection: close
Content-Type: text/html; charset=UTF-8
--------------

ProxHTTPSProxy, when in Half_SSL mode, does not pass plain http.

(Jun. 05, 2010 11:17 PM)whenever Wrote:  It behaves like the old ProxHTTPSProxy when in Half_SSL mode. That is, all http requests are treated like results of 307 redirecting and will be fetched through https protocol.

I thought that you might want both options to behave the same.
I think I'd vote for allowing plain http to pass. Only http with ":443" would be 'converted'.

The Proxomitron was set to send all to ProxHTTPSProxy for both options.
Add Thank You Quote this message in a reply
Jun. 06, 2010, 12:50 AM (This post was last modified: Jun. 06, 2010 12:53 AM by whenever.)
Post: #100
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 06, 2010 12:30 AM)JJoe Wrote:  Polipo is not needed to handle socks when using Tor, I think.
Yes, unless you got problems.

(Jun. 06, 2010 12:30 AM)JJoe Wrote:  I think I'd vote for allowing plain http to pass. Only http with ":443" would be 'converted'.

So, what about https configured at 80, 8080? I know it is hardly to see but we have to take it into account.

Maybe we'd better to tag the https request like Graycode advised? a prefix to the port? or a URL query sting? Which method is supposed to break things less?
Add Thank You Quote this message in a reply
Jun. 06, 2010, 04:44 AM (This post was last modified: Jun. 06, 2010 04:50 AM by JJoe.)
Post: #101
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 06, 2010 12:50 AM)whenever Wrote:  So, what about https configured at 80, 8080? I know it is hardly to see but we have to take it into account.

Maybe we'd better to tag the https request like Graycode advised? a prefix to the port? or a URL query sting? Which method is supposed to break things less?

Since relative links don't inherit the query string, I'd guess prefix.

Tag has to be > http://somewhere.in:here/. SRL went with "https..".

There may also be cookie issues to be considered, whether you prefix the address or port number. I believe there is a optional port attribute for cookies.

Currently, the Proxomitron is expected to handle the exceptions with a small list. Could this work be done by ProxHTTPSProxy? ProxHTTPSProxy could keep a list of the odd 307s that it issued. The user would probably have to add the ones for plain http on 443. Should still be a small rarely used list, I think. I still can't estimate how much work this would be for the python programmer, sorry.
Add Thank You Quote this message in a reply
Jun. 06, 2010, 08:10 AM
Post: #102
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 06, 2010 04:44 AM)JJoe Wrote:  Since relative links don't inherit the query string, I'd guess prefix.

Doesn't a proxy server always get absolute urls, or am I misunderstanding?

(Jun. 06, 2010 04:44 AM)JJoe Wrote:  ProxHTTPSProxy could keep a list of the odd 307s that it issued.

Try ver 0.4. Smile!

ProxHTTPSProxy now maintains a host:port list in memory for 307s.

(Jun. 06, 2010 04:44 AM)JJoe Wrote:  The user would probably have to add the ones for plain http on 443.

No need. Plain http on 443 won't get into ProxHTTPSProxy's list.

(Jun. 06, 2010 04:44 AM)JJoe Wrote:  I still can't estimate how much work this would be for the python programmer, sorry.

I just added/modified 2 lines. Testing took more time. Wink


Attached File(s)
.zip  ProxHTTPSProxy 0.4.zip (Size: 7.94 KB / Downloads: 610)
Add Thank You Quote this message in a reply
Jun. 06, 2010, 02:25 PM
Post: #103
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 06, 2010 08:10 AM)whenever Wrote:  
(Jun. 06, 2010 04:44 AM)JJoe Wrote:  Since relative links don't inherit the query string, I'd guess prefix.

Doesn't a proxy server always get absolute urls, or am I misunderstanding?

Consider bugzilla's, https://bugzilla.mozilla.org/ , html:

Code:
<link href="skins/standard/global.css"
            rel="stylesheet"
            type="text/css">

The browser creates https://bugzilla.mozilla.org/skins/custom/global.css from the relative link "skins/standard/global.css" and the base address.

A ProxHTTPSProxy 307 with query string tag would create an base address like http://bugzilla.mozilla.org:443/?=gotcha. The browser would then create http://bugzilla.mozilla.org:443/skins/custom/global.css. Query string tagging ProxHTTPSProxy would mistakenly pass this 'global.css' address as plain http over 443. I think.

(Jun. 06, 2010 08:10 AM)whenever Wrote:  
(Jun. 06, 2010 04:44 AM)JJoe Wrote:  ProxHTTPSProxy could keep a list of the odd 307s that it issued.

Try ver 0.4. Smile!

ProxHTTPSProxy now maintains a host:port list in memory for 307s.

(Jun. 06, 2010 04:44 AM)JJoe Wrote:  The user would probably have to add the ones for plain http on 443.

No need. Plain http on 443 won't get into ProxHTTPSProxy's list.

This is probably the better choice. Documentation should be simpler. Easier for the user to understand and configure. List should still be fairly small.

Half_SSL working at all sites tested. Smile!

(Jun. 06, 2010 08:10 AM)whenever Wrote:  
(Jun. 06, 2010 04:44 AM)JJoe Wrote:  I still can't estimate how much work this would be for the python programmer, sorry.

I just added/modified 2 lines. Testing took more time. Wink

A little less than I guessed but you took a simpler path.
Python appears to be (at least) a fairly good prototyping language.

Have fun
Add Thank You Quote this message in a reply
Jun. 06, 2010, 03:54 PM
Post: #104
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Sorry, I am updating again.

The memory list get lost upon program restart, which is inconvenient if we have already half_ssl pages open in browser (they are not in the new memory list and will be fetched via http hence failed).

Version 0.4a adds port443_exceptions and half_ssl_lists sections in config.py and treats below URLs from top to below as half_ssl:

- host:port in memory list (automatically generated under half_ssl mode)
- port is 443 while host not in port443_exceptions section (any mode)
- host:port in half_ssl_lists section (any mode)

(Jun. 06, 2010 02:25 PM)JJoe Wrote:  A ProxHTTPSProxy 307 with query string tag would create an base address like http://bugzilla.mozilla.org:443/?=gotcha. The browser would then create http://bugzilla.mozilla.org:443/skins/custom/global.css. Query string tagging ProxHTTPSProxy would mistakenly pass this 'global.css' address as plain http over 443. I think.

If I were going to tag the url, I would add the query string to the end of the original url like http://bugzilla.mozilla.org:443/skins/custom/global.css&taggedhere. The http handler would detect the tag string and remove it before make final request.


Attached File(s)
.zip  ProxHTTPSProxy 0.4a.zip (Size: 8.19 KB / Downloads: 583)
Add Thank You Quote this message in a reply
Jun. 06, 2010, 08:55 PM
Post: #105
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 06, 2010 03:54 PM)whenever Wrote:  Sorry, I am updating again.

The memory list get lost upon program restart, which is inconvenient if we have already half_ssl pages open in browser (they are not in the new memory list and will be fetched via http hence failed).

Version 0.4a adds port443_exceptions and half_ssl_lists sections in config.py and treats below URLs from top to below as half_ssl:

- host:port in memory list (automatically generated under half_ssl mode)
- port is 443 while host not in port443_exceptions section (any mode)
- host:port in half_ssl_lists section (any mode)

I haven't tested the lists. The rest works as expected, so far.

No need to be sorry. Smile!

(Jun. 06, 2010 03:54 PM)whenever Wrote:  
(Jun. 06, 2010 02:25 PM)JJoe Wrote:  A ProxHTTPSProxy 307 with query string tag would create an base address like http://bugzilla.mozilla.org:443/?=gotcha. The browser would then create http://bugzilla.mozilla.org:443/skins/custom/global.css. Query string tagging ProxHTTPSProxy would mistakenly pass this 'global.css' address as plain http over 443. I think.

If I were going to tag the url, I would add the query string to the end of the original url like http://bugzilla.mozilla.org:443/skins/custom/global.css&taggedhere. The http handler would detect the tag string and remove it before make final request.

Maybe I'm missing the point or was misunderstood or am wrong but how would the tag get there?

The browser would request https://bugzilla.mozilla.org/.
ProxHTTPSProxy would 307 that to http://bugzilla.mozilla.org:443/&taggedhere.
When the browser created the absolute links from the relative links in the html and http://bugzilla.mozilla.org:443/&taggedhere, wouldn't the tag be lost?

Absolute https links in the html would be 307 and tagged by ProxHTTPSProxy.
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: