ProxHTTPSProxy, a Proxomitron SSL Helper Program
|
May. 31, 2010, 03:11 AM
Post: #76
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 02:21 AM)whenever Wrote: Graycode, is it normal a "/" come after "CONNECT" method? How could that happened? I think Proxo doesn't get the host:port where to establish the tunnel so it popped up the error message box. No. The CONNECT method is supposed to be only for proxies, and proxied requests should have the host-port as part of the request line. Non-proxied request (something a web server would get) Code: GET / HTTP/1.1 Proxied request (something a proxy would get) Code: GET http://www.example.com/ HTTP/1.1 For a CONNECT given to a proxy, it's normal for the request to contain host and port while the 'Host:' header does not contain the port by some browsers. Code: CONNECT www.example.com:443 HTTP/1.1 Did your Python proxy get the headers you showed? Is Proxomitron just showing that format in its logging, or did it really send those headers to your proxy? If Proxo knows the communication is going to be sent through your proxy the it should have sent proxy formatting. I'm confused. |
|||
May. 31, 2010, 03:19 AM
Post: #77
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 01:30 AM)JJoe Wrote: Perhaps "continuing problem with httplib multiple set-cookie headers" at http://bugs.python.org/issue1660009 ? A quick fix for that problem. |
|||
May. 31, 2010, 03:38 AM
Post: #78
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 03:11 AM)Graycode Wrote: Did your Python proxy get the headers you showed? Is Proxomitron just showing that format in its logging, or did it really send those headers to your proxy? If Proxo knows the communication is going to be sent through your proxy the it should have sent proxy formatting. I am not logging headers but I use host-port to build the https url to be 307 redirected in ProxHTTPSProxy or to be fetched in HTTPSProxy. I hadn't seen exceptions about that yet. Proxo is not showing that format in its logging. For normal communication, it does log "CONNECT http://www.example.com:443 HTTP/1.1". The host-port is so important that I don't think the browser will get it wrong. I am also confused why Proxo doesn't get it right. It's even not going into SSL layer yet. |
|||
May. 31, 2010, 04:21 AM
Post: #79
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 03:19 AM)whenever Wrote:(May. 31, 2010 01:30 AM)JJoe Wrote: Perhaps "continuing problem with httplib multiple set-cookie headers" at http://bugs.python.org/issue1660009 ? With that and a ssl_sock.settimeout of .5 sec, I've logged in to yahoo. Thanks |
|||
May. 31, 2010, 09:09 AM
Post: #80
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 03:38 AM)whenever Wrote: The host-port is so important that I don't think the browser will get it wrong. I am also confused why Proxo doesn't get it right. It's even not going into SSL layer yet. I just tried to telnet to Proxo's listening port and issued "CONNECT / HTTP/1.1" command manually. It didn't trigger the error message box but responded with below text, which indicated Proxo did have detection for malformed command: Code: HTTP/1.1 403 Connection refused I forgot to add that when the error was raised, the error message box was keeping poping up and the Proxo log window was keeping outputing "CONNECT / HTTP/1.1". It seems something went wrong within Proxo internal and it was keeping trying. (May. 31, 2010 04:21 AM)JJoe Wrote: With that and a ssl_sock.settimeout of .5 sec, I've logged in to yahoo. A longer timeout value will make the proxy block longer. I don't know what value should be reasonable. It is local communication between the browser and the proxy, which I had thought could be done very quickly. |
|||
May. 31, 2010, 05:12 PM
(This post was last modified: May. 31, 2010 05:27 PM by Graycode.)
Post: #81
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Are you using ProxHTTPSProxy or the other HTTPSProxy?
At the bottom of do_CONNECT in HTTPSProxy it seems to be closing the socket immediately after doing the shutdown(). Using shutdown for write should make the socket layer send a FIN after the last packet is sent (flushed), then the other side should ACK to close the socket, and final clean closure determined by a read of 0 bytes. It may need to be something like: Code: # socket.SHUT_WR == 1 I've specified a 6 second timeout, don't know how much data will need to hit the wire, be processed by the other side and have the other side ACK to the FIN. When things work as they should, doing that adds no unnecessary delay. Considered using settimeout(0) or setblocking(True) for more permanent wait, maybe that's better. Something should be specified otherwise the previous settimeout(0.1) or (0.5) will still be in effect, and that may be insufficient to get all the data flushed back to Proxo. There's a lot of things I don't know or don't understand about Python. What you've been able to accomplish is not easy or simple. |
|||
May. 31, 2010, 08:32 PM
Post: #82
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 09:09 AM)whenever Wrote:(May. 31, 2010 04:21 AM)JJoe Wrote: With that and a ssl_sock.settimeout of .5 sec, I've logged in to yahoo. It looks like it has to be long enough to allow interaction with the SSL warning dialogs, "Allow for Session"? If I wait to click more than ssl_sock.settimeout, I have to reload the page. (May. 31, 2010 05:12 PM)Graycode Wrote: Are you using ProxHTTPSProxy or the other HTTPSProxy? HTTPSProxy. Thanks |
|||
Jun. 01, 2010, 02:59 AM
Post: #83
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(May. 31, 2010 05:12 PM)Graycode Wrote: It may need to be something like: That's what I do in ProxHTTPSProxy to solve the reset issue. It seems not needed in HTTPSProxy because there is no pending readable data. Anyway, being strict is better. I added the code to HTTPSProxy too. (May. 31, 2010 05:12 PM)Graycode Wrote: Something should be specified otherwise the previous settimeout(0.1) or (0.5) will still be in effect, and that may be insufficient to get all the data flushed back to Proxo. I forgot to set the socket back to blocking mode. Fixed. (May. 31, 2010 08:32 PM)JJoe Wrote: It looks like it has to be long enough to allow interaction with the SSL warning dialogs, "Allow for Session"? If I wait to click more than ssl_sock.settimeout, I have to reload the page. Well, even 1 hour is not long enough. The socket should block forever until we finish interaction with the SSL warning dialogs. Should be fixed in version 0.1b. |
|||
Jun. 01, 2010, 04:03 AM
Post: #84
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 01, 2010 02:59 AM)whenever Wrote: I forgot to set the socket back to blocking mode. I think you are doing great. This isn't... It's suppose to be some kind of fun. Don't forget to play with the kid. (Jun. 01, 2010 02:59 AM)whenever Wrote:(May. 31, 2010 08:32 PM)JJoe Wrote: It looks like it has to be long enough to allow interaction with the SSL warning dialogs, "Allow for Session"? If I wait to click more than ssl_sock.settimeout, I have to reload the page. Appears to be fixed. I do still need a ssl_sock.settimeout of .2 to get all the yahoo cookies. I'm testing at https://ssl.scroogle.org/ , https://login.yahoo.com/ , https://bugzilla.mozilla.org/ , https://developer.mozilla.org/ . What do you see at https://developer.mozilla.org/ ? I see Code: New Message Log Window.... and Code: Exception happened during processing of request from ('127.0.0.1', 59345) Loads without HTTPSProxy. |
|||
Jun. 01, 2010, 07:22 AM
Post: #85
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
(Jun. 01, 2010 04:03 AM)JJoe Wrote: Don't forget to play with the kid. Don't worry. It is not taking my family time. I am thinking replacing urllib2 with httplib module for lower level operations. urllib2 follows redirects by default, which may cause issue. For example, the browser doesn't know a 301/302 redirect happened (they should know); if a login page set cookies with a redirect, the browser won't see it. Give me some time to work on it. Let's wait and see if it could solve the https://developer.mozilla.org/ issue. |
|||
Jun. 01, 2010, 04:12 PM
Post: #86
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Well, I decided to go more lower level, so here is a just working socket version. It seems faster and worked with https://developer.mozilla.org/.
Have fun and report issue please. I might won't be able to respond until Friday. Go to bed now. |
|||
Jun. 01, 2010, 07:01 PM
Post: #87
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
This HTTPSProxy works at all sites checked.
And now the log shows outgoing headers. What next? You might want to update this topic's first post. The link is to ProxHTTPSProxy 0.1b. I will be leaving tomorrow. Expect to be back Thursday or Friday. |
|||
Jun. 01, 2010, 11:46 PM
Post: #88
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program | |||
Jun. 02, 2010, 02:09 AM
Post: #89
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program | |||
Jun. 04, 2010, 03:43 PM
(This post was last modified: Jun. 04, 2010 03:43 PM by whenever.)
Post: #90
|
|||
|
|||
RE: ProxHTTPSProxy, a Proxomitron SSL Helper Program
Well, you all have helped a lot. Thanks!
Here is the version 0.3 for playing around. What's new: Code: - Rewrite ProxHTTPSProxy with socket module too, faster than urllib2 version Please test and report issue. Have fun! |
|||
« Next Oldest | Next Newest »
|