Post Reply 
Proxomitron Reborn
Dec. 15, 2018, 07:53 PM
Post: #121
RE: Proxomitron Reborn
(Dec. 13, 2018 05:50 AM)JJoe Wrote:  HTTPS port number in Settings has been set to 8443, certs generated and installed, config saved, Proxomitron restarted, and computers rebooted.
Have I overlooked something?

The Proxomitron must load an "HTTPS port number" enabled cfg at start. So, I have modified Scott's default.

Looks like it is working, Edge, IE, Firefox.

Opera is still complaining but this may be fixed in 4.6.0.1.
Add Thank You Quote this message in a reply
Dec. 16, 2018, 06:47 PM
Post: #122
RE: Proxomitron Reborn
(Dec. 15, 2018 04:39 AM)JJoe Wrote:  I wonder tho, would the dialog be more self-explanatory, if 'Include Scheme' always applied?
The dialog never added the scheme to the blocklist entries, so I didn't want to break the existing behaviour and instead added that option (default false). After I fix it, manually entered entries will be put into the blocklist as-is, while selecting the existing recent URLs will depend on Include Scheme whether they will be added with a prefix of http:// or https:// .

I am still investigating the local bypass bug. Note that if your browser supports excluding the use of a proxy for certain URLs, you can add local.ptron to that list.
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to amy for this post:
mizzmona
Dec. 17, 2018, 12:19 AM
Post: #123
RE: Proxomitron Reborn
I think I've figured it out: HTTPS local sites look like they have no host, and start with '/', because the browser creates a loopback connection through Proxomitron itself and then asks for the path directly, treating it like a normal webserver. Thus, to bypass all local paths,
Code:
/
in your blockfile is enough Smile!

If you want to bypass a specific local path, then specify it as usual, but omit the host:
Code:
/foo/bar/bypassed.html
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to amy for this post:
referrer
Dec. 17, 2018, 01:44 AM
Post: #124
RE: Proxomitron Reborn
(Dec. 16, 2018 06:47 PM)amy Wrote:  
(Dec. 15, 2018 04:39 AM)JJoe Wrote:  I wonder tho, would the dialog be more self-explanatory, if 'Include Scheme' always applied?
The dialog never added the scheme to the blocklist entries, so I didn't want to break the existing behaviour and instead added that option (default false). After I fix it, manually entered entries will be put into the blocklist as-is, while selecting the existing recent URLs will depend on Include Scheme whether they will be added with a prefix of http:// or https:// .

I understood.

My thought was that a user (having seen the 'Include Scheme' option) might expect the scheme to be stripped from a pasted a URL, more so if they were aware of the existing behaviour.
Applying the option to all URLs would keep someone from adding 'unless typed or pasted' to instructions that are too often not read.

Just a thought. Smile!
Add Thank You Quote this message in a reply
Dec. 17, 2018, 11:49 AM (This post was last modified: Dec. 17, 2018 11:50 AM by referrer.)
Post: #125
RE: Proxomitron Reborn
Another question I get when I trying let proxoR 4520 and 4600 share one blockfile.

RedirectList
Code:
#1st jump
line12: (http(|s)://)\1www.example.com(:443)+{0,1}/index.html $SET(RDIR=\1local.ptron/test/test.html)
#2nd jump
line14: (https://local.ptron)\1:443 $SET(RDIR=\1:8443)

BypassList
Code:
#for 2nd jump
line20: #local.ptron
#for 4520
line22: local.ptron/
#for 4600
line24: /test/

visit https://www.example.com/index.html
proxoR4600's log:
Code:
+++GET 267+++
CONNECT / HTTP/1.1
Host: www.example.com:443
BlockList 267: in RedirectList, line 12
JumpTo: https://local.ptron/test/test.html

+++SSL:GET 267+++
SSL cipher TLSv1.2 AES128-SHA (128 bits)
GET /test/test.html HTTP/1.1
Host: local.ptron
Upgrade-Insecure-Requests: 1

+++CLOSE 267+++
Client Connection Reused: 1
Client closed: total 0
Client opened: total 1
BlockList 268: in RedirectList, line 14
JumpTo: https://local.ptron:8443/TTP/1.

+++GET 268+++
CONNECT /TTP/1. HTTP/0.9
Host: local.ptron

+++SSL 268:+++
SSL Pass-Thru: CONNECT https://local.ptron:8443/TTP/1.
Client opened: total 2
Getting certificate for local.ptron
Certificate found in cache
BlockList 269: in BypassList, line 24

Where's the TTP/1. coming from?
Add Thank You Quote this message in a reply
[-] The following 1 user says Thank You to referrer for this post:
amy
Dec. 18, 2018, 03:50 AM
Post: #126
RE: Proxomitron Reborn
It is coming from a bug Wink Thanks, will be fixed in 4.6.0.1.
Add Thank You Quote this message in a reply
[-] The following 3 users say Thank You to amy for this post:
referrer, mizzmona, usr
Dec. 22, 2018, 04:36 AM
Post: #127
RE: Proxomitron Reborn
(Dec. 16, 2018 06:47 PM)amy Wrote:  The dialog never added the scheme to the blocklist entries, so I didn't want to break the existing behaviour and instead added that option (default false). After I fix it, manually entered entries will be put into the blocklist as-is, while selecting the existing recent URLs will depend on Include Scheme whether they will be added with a prefix of http:// or https:// .

What about when "filtering secure pages" is enabled in "Config -> HTTP" tab, making it default to filter both :443 and :80 (when disabled filtering :80 only)? I think it's more nature this way given https has been so popular nowadays.

The benefits:

- Users don't have to change their blocklists to explicitly write (:443)+{0,1} to filter https. Instead, they can keep their blocklists as is and only need to change items for https sites they don't want to filter.

- You don't need to add the scheme to the blocklist entries, and you don't break the existing behaviour.
Add Thank You Quote this message in a reply
Dec. 22, 2018, 07:27 AM (This post was last modified: Dec. 22, 2018 07:29 AM by whenever.)
Post: #128
RE: Proxomitron Reborn
Say https port is set with 8443, program starts to display "waiting for connections on 8443" instead of the proxy port. Just a display issue.


Attached File(s)
.png  8443.png (Size: 3.84 KB / Downloads: 488)
Add Thank You Quote this message in a reply
Dec. 23, 2018, 05:29 AM
Post: #129
RE: Proxomitron Reborn
(Dec. 17, 2018 12:19 AM)amy Wrote:  I think I've figured it out: HTTPS local sites look like they have no host, and start with '/', because the browser creates a loopback connection through Proxomitron itself and then asks for the path directly, treating it like a normal webserver. Thus, to bypass all local paths,
Code:
/
in your blockfile is enough Smile!

If you want to bypass a specific local path, then specify it as usual, but omit the host:
Code:
/foo/bar/bypassed.html

This is not inline with the original Proxomitron. The original version is able to bypass a whole https site based on its hostname (though doesn't work on path). local.ptron should be enough to make it bypassed if the local server is treated as an external server.

I think there should be 2 phases of URL filtering for https:

HTTPS (with Proxomitron and SSL-filtering)
- Browser connects to localhost port 8080
- Browser sends CONNECT example.com:443 HTTP/1.1 request
<===== The 1st URL filtering here, should could bypass/kill based on hostname. I think there is where the original version works.
- Proxomitron establishes encrypted tunnel with browser
- Proxomitron connects to example.com port 443 and establishes encrypted tunnel with it
- Browser sends GET / HTTP/1.1 request and headers through encrypted tunnel to Proxomitron
<===== The 2nd URL filtering here, should build the complete URL with the hostname available in the 1st phase and the path available here. This phase should could bypass/kill based on full url.
- Proxomitron decrypts, processes/filters request, then re-encrypts and sends it to the site server
- Site server sends reply to Proxomitron, which decrypts, processes/filters the reply, and re-encrypts it to send to the browser
Add Thank You Quote this message in a reply
Dec. 24, 2018, 04:40 AM
Post: #130
RE: Proxomitron Reborn
4.6.0.1 will be available soon.
(Dec. 22, 2018 04:36 AM)whenever Wrote:  What about when "filtering secure pages" is enabled in "Config -> HTTP" tab, making it default to filter both :443 and :80 (when disabled filtering :80 only)? I think it's more nature this way given https has been so popular nowadays.

The benefits:

- Users don't have to change their blocklists to explicitly write (:443)+{0,1} to filter https. Instead, they can keep their blocklists as is and only need to change items for https sites they don't want to filter.

- You don't need to add the scheme to the blocklist entries, and you don't break the existing behaviour.
Unfortunately this would require some very extensive changes in the way requests are processed. I'd love to rewrite the request processing code, because I'm aware of a bunch of limitations it has, but that risks breaking some edge-case existing behaviour as well as introducing new bugs. By "extensive" I mean it would, if there was sufficient demand for it, probably appear as a version 5.x.
(Dec. 22, 2018 07:27 AM)whenever Wrote:  Say https port is set with 8443, program starts to display "waiting for connections on 8443" instead of the proxy port. Just a display issue.
The status line shows what was last written to it, and whereas before there was only one thread (the HTTP listener) that wrote to it once it started listening, now there's two threads that write to it once they start listening, once for HTTP and once for HTTPS. The order in which they write the status is unpredictable so you may see either one. Do you have any suggestions?
(Dec. 23, 2018 05:29 AM)whenever Wrote:  This is not inline with the original Proxomitron. The original version is able to bypass a whole https site based on its hostname (though doesn't work on path). local.ptron should be enough to make it bypassed if the local server is treated as an external server.
Therein lies the crux of the problem: local.ptron is special, and has to be to avoid a recursive loop of the Proxomitron connecting back to itself because the same listener is used for both local and remote requests. In fact, if you visit it via https://local.ptron:8443/xxx , the first CONNECT is automatically bypassed as you can see from the "SSL Pass-Thru" appearing in the log. Once it is bypassed, the listener then gets the actual request, and for local requests, this is just a path with no hostname. That's how it knows NOT to initiate a remote connection and instead process it as if it was a local file path.
Add Thank You Quote this message in a reply
Dec. 24, 2018, 01:47 PM (This post was last modified: Dec. 24, 2018 01:49 PM by whenever.)
Post: #131
RE: Proxomitron Reborn
(Dec. 24, 2018 04:40 AM)amy Wrote:  4.6.0.1 will be available soon.

Well done! Thumbs Up

(Dec. 24, 2018 04:40 AM)amy Wrote:  Unfortunately this would require some very extensive changes in the way requests are processed
...
if there was sufficient demand for it, probably appear as a version 5.x.

It would be good to have but we can live without it. Don't feel pressure.

(Dec. 24, 2018 04:40 AM)amy Wrote:  now there's two threads that write to it once they start listening, once for HTTP and once for HTTPS. The order in which they write the status is unpredictable so you may see either one. Do you have any suggestions?

Maybe let the HTTPS thread not write the status?

(Dec. 24, 2018 04:40 AM)amy Wrote:  Once it is bypassed, the listener then gets the actual request, and for local requests, this is just a path with no hostname.

Maybe build a URL at this step? If we see just a path, is it safe to assume the hostname is just local.ptron?

Also, the URL matching is handled inside the HTTPS server while not by the Proxy server (because of "SSL Pass-Thru")? Is this the same way how the program handle pure http local.ptron requests? I had thought the matching/filtering was done by the proxy server.


(Dec. 24, 2018 04:40 AM)amy Wrote:  the same listener is used for both local and remote requests.
...
That's how it knows NOT to initiate a remote connection and instead process it as if it was a local file path.

A little curious when it will initiate a remote connection?

Also, http://local.ptron is not exposed to the outside via port 80 so you can't use http://127.0.0.1:80 to access it. The new https addition (Thank you!) is making its service available via https://127.0.0.1:8443 and I'm not sure if it's a good idea. It may expose security risks. What about making it available only via https://local.ptron, or making it an option? For example, if we set the https port number to 0, it just disable the external service while the internal service is always available via https://local.ptron (at default port 443).

BTW, I have been getting this several times.


Attached File(s)
.png  GUI missing.png (Size: 2.74 KB / Downloads: 424)
Add Thank You Quote this message in a reply
Dec. 24, 2018, 07:01 PM (This post was last modified: Dec. 24, 2018 07:10 PM by mizzmona.)
Post: #132
RE: Proxomitron Reborn
Just thought of something for the wish list, no rush or anything...

Strict byte-value matches for checking uppercase vs lowercase (e.g., [%41] vs [%61]).
Add Thank You Quote this message in a reply
Dec. 25, 2018, 05:47 AM
Post: #133
RE: Proxomitron Reborn
(Dec. 24, 2018 01:47 PM)whenever Wrote:  Also, http://local.ptron is not exposed...

It can be exposed, however. 'Promiscuous' mode is controlled by the Access dialog, https://proxomitron.info/45/help/CfgT3.html#foo , and default is disabled.

(Dec. 24, 2018 01:47 PM)whenever Wrote:  The new https addition (Thank you!) is making its service available via https://127.0.0.1:8443 and I'm not sure if it's a good idea.

Best solution would be to have the Access dialog control both.
Add Thank You Quote this message in a reply
Dec. 25, 2018, 10:15 PM
Post: #134
RE: Proxomitron Reborn
4.6.0.1 has been released! Merry Christmas!

Changes in this release:
- Generate certificate serial number from hostname
- Always add SAN to certificates
- Fix buffer overflow in request parsing
- Fix adding manually entered blockfile entries
- Fix parsing of CONNECT URLs
- Fix listener startup message

(Dec. 24, 2018 01:47 PM)whenever Wrote:  Maybe let the HTTPS thread not write the status?
That is what I've done for 4.6.0.1. You will still see two messages in the log when you abort (which restarts the listeners) if you have both ports enabled, however.
(Dec. 24, 2018 01:47 PM)whenever Wrote:  Maybe build a URL at this step? If we see just a path, is it safe to assume the hostname is just local.ptron?

Also, the URL matching is handled inside the HTTPS server while not by the Proxy server (because of "SSL Pass-Thru")? Is this the same way how the program handle pure http local.ptron requests? I had thought the matching/filtering was done by the proxy server.
Yes, that might be possible but like I said earlier, I still don't understand the request processing code sufficiently and it might break some other things if I change it. For non-local HTTPS requests, the listener goes into secure mode and then reads the request, and URL matching is done there. For local HTTPS requests, the listener goes into bypass mode, hence no URL matching is done by it. Your suggestions have been logged for version 5.x improvements.
(Dec. 24, 2018 01:47 PM)whenever Wrote:  A little curious when it will initiate a remote connection?
When it receives a CONNECT request that isn't local.ptron, or a request with a full hostname (e.g. GET http://example.com/ HTTP/1.1)... I think.
(Dec. 24, 2018 01:47 PM)whenever Wrote:  Also, http://local.ptron is not exposed to the outside via port 80 so you can't use http://127.0.0.1:80 to access it. The new https addition (Thank you!) is making its service available via https://127.0.0.1:8443 and I'm not sure if it's a good idea. It may expose security risks. What about making it available only via https://local.ptron, or making it an option? For example, if we set the https port number to 0, it just disable the external service while the internal service is always available via https://local.ptron (at default port 443).
The access controls for both listeners are managed by the settings under the Access tab.
Quote:
BTW, I have been getting this several times.
That is unusual. Does it occur with Scott's original? Is it just the UI not getting drawn (i.e. you can still click where the buttons would be and they work), or do the controls not even respond?
(Dec. 24, 2018 07:01 PM)mizzmona Wrote:  Just thought of something for the wish list, no rush or anything...

Strict byte-value matches for checking uppercase vs lowercase (e.g., [%41] vs [%61]).
If you can come up with a precise detailed specification of the syntax and behaviour, preferably one which won't break existing patterns, then I can consider adding it in the next feature release (4.6.1).
(Dec. 25, 2018 05:47 AM)JJoe Wrote:  Best solution would be to have the Access dialog control both.
That's how it currently works. Thumbs Up
Add Thank You Quote this message in a reply
[-] The following 5 users say Thank You to amy for this post:
referrer, mizzmona, usr, defconnect, Callahan
Dec. 26, 2018, 02:17 AM
Post: #135
RE: Proxomitron Reborn
(Dec. 25, 2018 05:47 AM)JJoe Wrote:  It can be exposed, however. 'Promiscuous' mode is controlled by the Access dialog, https://proxomitron.info/45/help/CfgT3.html#foo , and default is disabled.

That controls if to limit the access to 8080 (the proxy port). If you enable that option, it changes the proxy's listening address from 127.0.0.1 to 0.0.0.0, so that other PCs can access this PC's 8080 port (the proxy).

(Dec. 25, 2018 10:15 PM)amy Wrote:  
(Dec. 25, 2018 05:47 AM)JJoe Wrote:  Best solution would be to have the Access dialog control both.
That's how it currently works. Thumbs Up

I'm afraid you all didn't get my point. What I'm talking about is if to make the resources at https://local.ptron available via https://127.0.0.1:(443|8443). My opinion is NO because it has security risks and the original version doesn't do it either (the resources at http://local.ptron is not available via http://127.0.0.1).

(Dec. 25, 2018 10:15 PM)amy Wrote:  Your suggestions have been logged for version 5.x improvements.

Thanks. My personal suggestion is:

- the proxy handles all things like URL matching, filtering ...
- the proxy treats requests to local.ptron just like requests to non-local sites, the only difference is it knows where to send the local.ptron requests to.
- leave the part that handles local.ptron alone, no URL matching or filtering there.

(Dec. 25, 2018 10:15 PM)amy Wrote:  That is unusual. Does it occur with Scott's original? Is it just the UI not getting drawn (i.e. you can still click where the buttons would be and they work), or do the controls not even respond?

It occurs with Scott's original too, and I can reproduce it on my PC (win10x64).

- Start the program freshly
- Click its icon on the tray to make its main window appear
- Press and hold the Alt key

The controls would appear and respond when you click their positions.
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: