Connection Timer...

[ProxRocks]

"thinking out loud"...

in the spirit of the 'page load timer', i'm wondering if an actual connection can be "timed" - and more importantly, if Proxo can KILL/disconnect a connection after a preset duration...


any thoughts?

[bugger]

prefences > http > local connection persist

[ProxRocks]

except i'm not looking for a "persist"...

i'm looking to KILL/disconnect an ACTIVE connection, a connection that IS transferring data...

ie, go here - http://www.ehowa.com/ (must do TODAY, 'cause the blog will change tomorrow!)
my "what the Proxo is thinking" page is showing that it took 88, EIGHTY EIGHT!!! connections...

this is RETARDED!!!...

and the page takes TEN AND A HALF seconds to load...

again, RETARDED!!!...


so okay, maybe "timing" the connection is tackling this RETARDATION the wrong way...


let's rephrase the question...

is there a way that Proxo can PREVENT a given web site from opening close to NINETY connections? say, limit the connections per "page" to 20?

[lnminente]

If we could find a way to remember the value of a variable from a connection to another maybe.

[sidki3003]

An astonishing fact we found in the discussion here is that Proxomitron is obeying its own (i.e. user-added) incoming header.

So you could try playing with the keep-alive header, which Graycode was explaining here.
For instance (incoming -- no clue if it has any perceivable effect on Prox at all!):

Code:

Keep-Alive: timeout=30, max=20
Connection: Keep-Alive

Naturally, this would require the remote server to actually send a response. IOW, *if* the problem yesterday was that the image server thumbs.ehowa.com was temporarily dead, mentioned approach hadn't helped.

[ProxRocks]

hmmm...


i just thought of a different fix (and one i know how to do myself, lol)...

i'm going to just "toggle" (off by default, no "connection" made until toggled "on") any-and-all images coming from a site with "thumb" in the URL...

that'll get me where i need to be


still surprises me that ANYONE would "compile" a web site requiring the viewers to open near NINETY connections for the dang page to load !!!

asinine times nine to the power of ten plus one...

[Graycode]

Most sites that try to protect their images by checking the Referer: header work well when No such header is sent. But that site insists on the Referer for at least their 'thumbs.' and 'images.' zones, else image requests redirect to the 'www' root page for every eimage.

The default in my proxy is to keep the Referer only when the full host names match. For this site I have to allow the Referer when the base domain matches.

So anyway ...

Viewing that site with my proxy today used 18 server connections for 106 requests. 5 of them were blocked because of forbidden domains or URL matching. Each of the 3 referenced brainyquote.com and brainyhistory.com requests responded with connection:close. A total of 15 server connections were for the 98 *.ehowa.com requests that were satisfied, and those connections were re-used 83 times in accordance with normal HTTP/1.1 methods.

Their 'thumbs.' and 'www.' server (Apache) permits 5 seconds of persistence until re-use by giving "Keep-Alive: timeout=5, max=500". Their 'images.' server (lighttpd) didn't specify a timeout.

It took about 6 seconds to retrieve everything and the page to load.
That was with Firefox 2 having specified: network.http.max-persistent-connections-per-proxy = 8
I got almost identical results with Opera 9.

Was the 88 connections with Proxo before today or is that still happening?

(Mar. 25, 2009 12:52 PM)ProxRocks Wrote:  i'm going to just "toggle" (off by default, no "connection" made until toggled "on") any-and-all images coming from a site with "thumb" in the URL...

Like usual, I don't understand. It sounds like you're trying to force a new connection for each of those 'thumbs.' requests. Won't that just slow down your browsing?

[ProxRocks]

(Mar. 25, 2009 08:14 PM)Graycode Wrote:  Was the 88 connections with Proxo before today or is that still happening?

hmmm...
maybe my techno-wording is wrong...

here's what i do - 1) close Proxo, 2) open Proxo, these two steps clears the "what is Proxo thinking" page, 3) open Ernie's site, 4) wait 9.78 seconds, 5) open "Proxo thinking" page, 6) count "connections" in the list...

87 connections "just now", 9.78 seconds to load...
there are times when it takes Ernie's site 12 to 13 seconds to load (i get this with Opera as well, and yes, with Firefox also)...

forgive my impatience, but Proxo should be able to correct that asinine load time...


i'm up-and-running with a toggled-image page load...
page is loading in 1.5 to 2.5 seconds - this i can live with...

basically, i *BLOCK* image thumbnails that are not within "today's" blog post - still a work-in-process... but can "toggle on" any image outside of today's blog post... ie, if i already d/l'd the image thumb while reading the blob YESTERDAY, why should i wait, wait, and wait for it to download again today? (and yeah, i clear my cache at browser exit)...

[sidki3003]

5.6 seconds here.

(Mar. 25, 2009 08:14 PM)Graycode Wrote:  For this site I have to allow the Referer when the base domain matches.

That's the default behavior of this config.

Quote:Each of the 3 referenced brainyquote.com and brainyhistory.com requests responded with connection:close. A total of 15 server connections were for the 98 *.ehowa.com requests that were satisfied, and those connections were re-used 83 times in accordance with normal HTTP/1.1 methods.

About the same stats here.

Quote:Their 'thumbs.' and 'www.' server (Apache) permits 5 seconds of persistence until re-use by giving "Keep-Alive: timeout=5, max=500".

I wonder why i don't see any incoming keep-alive header. Here's a SocketSniff snapshot of Proxomitron's communication (connection keep-alive being implied in HTTP/1.1):

Code:

Send: Return Code: 0x00000000
GET /thumbs/jsexalicia.jpg HTTP/1.1
Host: thumbs.ehowa.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.9.9
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Referer: http://www.ehowa.com/

Receive: Return Code: 0x00000000
HTTP/1.1 200 OK
Date: Wed, 25 Mar 2009 20:52:58 GMT
Server: Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7e PHP/4.4.1 FrontPage/5.0.2.2510
Last-Modified: Wed, 25 Mar 2009 14:39:23 GMT
ETag: "12df0-994-49ca421b"
Accept-Ranges: bytes
Content-Length: 2452
Content-Type: image/jpeg

[Graycode]

Interesting. They're apparently giving different headers based on the request.

Here's the headers received using Opera.

Code:

HTTP/1.1 200 OK
Date: Wed, 25 Mar 2009 21:21:05 GMT
Server: Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7e PHP/4.4.1 FrontPage/5.0.2.2510
Last-Modified: Wed, 25 Mar 2009 14:39:23 GMT
ETag: "12df0-994-49ca421b"
Accept-Ranges: bytes
Content-Length: 2452
Keep-Alive: timeout=5, max=500
Connection: Keep-Alive
Content-Type: image/jpeg

I see now that for Firefox it gave the same headers as you got, without the Keep-Alive and Connection headers.

But if I allow my proxy to let Firefox send a "Connection: Keep-Alive" to the server, then the server gives back:

Code:

HTTP/1.1 200 OK
Date: Wed, 25 Mar 2009 21:41:46 GMT
Server: Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7e PHP/4.4.1 FrontPage/5.0.2.2510
Last-Modified: Wed, 25 Mar 2009 14:39:23 GMT
ETag: "12df0-994-49ca421b"
Accept-Ranges: bytes
Content-Length: 2452
Keep-Alive: timeout=5, max=500
Connection: Keep-Alive
Content-Type: image/jpeg

This Post edited yet again ... sorry about that ...

The request my proxy made for the content is:

Code:

GET /thumbs/jsexalicia.jpg HTTP/1.1
Host: thumbs.ehowa.com
User-Agent: Mozilla/5.0 (Windows; U; Win; en-US; rv:1.8) Gecko/2008 Firefox/2.0
Accept: image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: Keep-Alive
Referer: http://www.ehowa.com/

The User-Agent is forged, it's firefox 2.0.0.20
Firefox originally sent to the proxy these 2 headers:

Code:

Keep-Alive: 25
Proxy-Connection: keep-alive

The value 25 is what I have configured for it in: network.http.keep-alive-timeout the default would be 300. My proxy always drops a "Keep-Alive:" header coming from the user agent, so it doesn't matter anyway. Other browsers won't send a "Keep-Alive:" header.

Since it's not forwarding up to another proxy, it changed the "Proxy-Connection:" into being "Connection:" header.

But even without having received the explicit connection persistence from the server, being an HTTP/1.1 response it should be assumed that there is persistence. Is Proxo assuming otherwise and not re-using the connection?

[sidki3003]

Still nothing here:

Code:

Send: Return Code: 0x00000000
GET /thumbs/jsexalicia.jpg HTTP/1.1
User-Agent: Opera/9.63 (Windows NT 5.1; U)
Host: thumbs.ehowa.com
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en
Accept-Charset: iso-8859-1, utf-8, utf-16, *;q=0.1
Referer: http://www.ehowa.com/

Receive: Return Code: 0x00000000
HTTP/1.1 200 OK
Date: Wed, 25 Mar 2009 21:58:27 GMT
Server: Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7e PHP/4.4.1 FrontPage/5.0.2.2510
Last-Modified: Wed, 25 Mar 2009 14:39:23 GMT
ETag: "12df0-994-49ca421b"
Accept-Ranges: bytes
Content-Length: 2452
Content-Type: image/jpeg

Are you sending a proxy-connection header, or anything that could trigger the different response?


ProxRocks, are you by any chance still stuck with HTTP/1.0?
I remember that being the problem a couple of years ago (either your firewall or company proxy having caused it).


edit: Graycode, just saw your edit. I can't force an outgoing connection: keep alive with "persistent connections" being activated. Okay.

[Graycode]

(Mar. 25, 2009 10:06 PM)sidki3003 Wrote:  edit: Graycode, just saw your edit.

Sorry about the multi-editing of the post. I got a phone call and was trying to do too many things at once, none of them very well.

[ProxRocks]

(Mar. 25, 2009 10:06 PM)sidki3003 Wrote:  ProxRocks, are you by any chance still stuck with HTTP/1.0?
I remember that being the problem a couple of years ago (either your firewall or company proxy having caused it).

my Proxo's HTTP Message Log is cluttered left and right with HTTP/1.1 200 OK's and GET /blah/blah.gif HTTP/1.1's, so i assume that means i'm running 1.1... (as i recall having the discussion when i was on dial-up, which i'm no longer on, but can't recall the "exacts")...

[lnminente]

Sidki, do you have enabled network.http.pipelining and network.http.proxy.pipelining in FF?

Code:

+++GET 1074+++
GET /thumbs/jsexalicia.jpg HTTP/1.1
Host: thumbs.ehowa.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; es-ES; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: es
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Referer: http://www.ehowa.com/
Connection: keep-alive

+++RESP 1074+++
HTTP/1.1 200 OK
Date: Wed, 25 Mar 2009 22:48:14 GMT
Server: Apache/1.3.34 (Unix) mod_ssl/2.8.25 OpenSSL/0.9.7e PHP/4.4.1 FrontPage/5.0.2.2510
Last-Modified: Wed, 25 Mar 2009 14:39:23 GMT
ETag: "12df0-994-49ca421b"
Accept-Ranges: bytes
Content-Length: 2452
Content-Type: image/jpeg

edit: BTW for these 5.6 seconds the FF 3.1b you are using helps a lot hehehe

[sidki3003]

(Mar. 25, 2009 09:35 PM)Graycode Wrote:  But even without having received the explicit connection persistence from the server, being an HTTP/1.1 response it should be assumed that there is persistence. Is Proxo assuming otherwise and not re-using the connection?

It's handling the situation correctly and re-uses the connection.