sidki's config set: 2005-06-09

sidki's config set: 2005-06-09 - Printable Version

+- The Un-Official Proxomitron Forum (https://www.prxbx.com/forums)
+-- Forum: Proxomitron Config Sets (/forumdisplay.php?fid=43)
+--- Forum: Sidki (/forumdisplay.php?fid=44)
+--- Thread: sidki's config set: 2005-06-09 (/showthread.php?tid=358)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

- z12 - May. 09, 2005 12:54 PM

Hi sidki

Here's my caching filters with some comments:

Resources won't cache if the Server sends a Content-Type header that incorrectly identifes the resource as html. To get around this problem, I use a combination of the ETag, IF-None-Match & Last Modifed headers.

The basic idea is to have the browser store a variable that indicates if the cached resource is html or not. We can't really store a "variable" in the browser, so the ETag header is used to simulate a variable.

The first time a resource is requested, the "Cache-Control" header is determined by the "Content-Type", which at this point, may not be correct. After the "Content-Type" filters have had a chance to run, incorrect Mime-Types should have fixed and the ETag filters are applied.

The first ETag filter removes the ETag header if the "Content-Type" is not html. The 2nd ETag Filter adds an ETag header if none is present. If the "Content-Type" is html, an ETag header is added with a field value of prxETag1. Otherwise, an ETag header with field value of prxETag2 is added.

To make all future requests conditional, the Last-Modified header is added if it was missing.

The next time the same resouce is requested, the browser sends an If-Modified-Since and If-None-Match header.

The If-None-Match header contains the value that was in the ETag header. If the value is prxETag2, variable hvINM is set to 2 (not html), otherwise, variable hvINM is set to 1 (html). The header is removed whenever the value is prxEtag* as to not pass a bogus value back to the server.

When the reply is received and the caching filters are enabled, the "Cache-Control:" header defaults to setting for 1 day. This setting is replaced with a value for 1 second if the following conditions are satisfied:

The value of hvINM is not 2 and the Content-Type is html, or, the value of hvINM = 1.

Since this is not the first time this resource has been requested, hvINM will have a value of either 1 or 2, which will determine whether or not the reply should be cached. If hvINM=2 (non-html), the Server's Content-Type is ignored and the reply is cached. If hvINM=1 (html), again, the Server's Content-Type is ignored (it may be missing or invalid on 304 replies) and the reply is not cached (ok it is, but only for 1 second).

Here's the modifications I made:

# modified URL, so 304 replies won't remove Cache-Control: 2 filters replacement value

Code:

In = TRUE

Out = FALSE

Key = "Cache-Control: 3 Kill if 3xx Response: Cache-H!     4.12.23 (cch!) [srl] (d.0) (In)"

URL = "$RESP(3(^04)*)(^$KEYCHK(^C)|$KEYCHK(^S))$TST(keyword=*.i_cache_h:[12].*)"

Match = "?"

# modified URL, so local.ptron html in cache would be reused
# modified Match, so filter would match based on value of variable hvINM

Code:

In = TRUE

Out = FALSE

Key = "Cache-Control: 4 max-age 1s if HTML: Cache-H!     4.12.24 (cch!) [sd] (d.0) (In)"

URL = "(^$KEYCHK(^C)|$KEYCHK(^S)|$RESP([45])|local.ptron)$TST(keyword=*.i_cache_h:[12].*)"

Match = "((^$TST(hvINM=2))$RESP(2*)$IHDR(Content-Type:( ) text/html))|($TST(hvINM=1))"

Replace = "max-age=1"

# added this filter, removes ETag if it's not html

Code:

In = TRUE

Out = FALSE

Key = "ETag: 1 Remove if not Html (cch!) (in)"

URL = "$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "\1&((^$IHDR(content-type:([^;]+&&*(html|xml)*)*))|(^$RESP(2*)))"

# added this filter, adds fake etag if ETag header not present

Code:

In = TRUE

Out = FALSE

Key = "ETag: 2 Add Proxo ETag (cch!) (In)"

URL = "$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "(^?)($RESP(2*) $IHDR(content-type:([^;]+&&*(html|xml)*)*)$SET(1=prxETag1) |$RESP(2*|304)$SET(1=prxETag2) )"

Replace = "\1"

# added this filter, sets variable hvINM, removes header if etag was faked

Code:

In = FALSE

Out = TRUE

Key = "If-None-Match: 1 Remove Prox ETag (cch!) (Out)"

URL = "$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "\1&(prxETag2$SET(hvINM=2)|$SET(hvINM=1))&prxETag*$LOG(CRESP $DTM(c) : removed If-None-Match: \1)"

# modified the existing If-None-Match filter Key so it had a number

Code:

In = FALSE

Out = TRUE

Key = "If-None-Match: 2 Kill: Fresh!     4.08.11 (cch! ^ctrl) [u] (d.0) (Out)"

URL = "(^$KEYCHK(^S))($KEYCHK(^C)|$TST(keyword=*.i_fresh:[12].*))"

Match = "\0&$LOG(CGET $DTM(c) : If-None-Match killed: \0)"

# added this filter, adds header if it's missing, make all future request conditional

Code:

In = TRUE

Out = FALSE

Key = "Last-Modified: 2 Add if Missing (cch!) (In)"

URL = "$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "(^?)$RESP(2*)"

Replace = "$DTM(I)"

You can see the effects of these filters at wired.com

Click on a link to an article, use the browsers back button, then click on the link again. Each time you click on the link, the global.js is reloaded with a max-age=1.

After these tweaks, on the second reply, the javascript is cached with a max-age=86400.

Mike

- sidki3003 - May. 09, 2005 04:32 PM

Hi Mike,

Very interesting! Thanks for the helpful explanations. It will take me some time to get familiar with these filters.

The added filters are just now active for both options, "Always Cache" and "Always Cache except for HTML", but they aren't needed for the first option, right?

A few more questions to get me started:

Did you check if servers that don't send a Last-Modified actually accept an If-Modified-Since?

Along the same line, i would feel a bit uncomfortable with removing all non-html ETags. What's with servers that base their responses on just this header? Or doesn't this happen? Hmm... just had an idea, see below.

Excluding local.ptron: Shouldn't it just get the "i_cache:2" (always cache) keyword?

Re "Cache-Control: 3 Kill if 3xx Response", there was a reason why i added that, but i can't remember which. Do you remember a situation where not killing it for 304s helped? Hmm... maybe i did it because the counter was always reset to 86400 and i never got a fresh copy... gonna check that.

Out of curiousity, what does "hvINM" stand for? Smile!

When hitting back to http://www.pcworld.com/ from another page i still get this:

Code:

RESP 6001 : Last-Modified added

RESP 6001 : Time: 17:08:38::335

+++RESP 6001+++

HTTP/1.1 200 OK

Server: Microsoft-IIS/5.0

Content-Type: image/gif; PrxMsg=Fixed acc. to File Extension: text/html

Content-Length: 70

Date: Mon, 09 May 2005 15:08:38 GMT

Connection: keep-alive

!URL-ID: http://images.pcworld.com/shared/graphics/tabs/new/toolshover.gif

Cache-Control: max-age=1

ETag: prxETag2

Last-Modified: Mon, 09 May 2005 15:08:38 GMT

Maybe it's because those "images" are ASCII files and Firefox doesn't like to send an If-None-Match for them, dunno.

Re using the ETag field as a var stored by the browser, how about making an "array" (a la keyword/flag/volat) out of it? The first field would be the original ETag, if present. All others would get a "prx-" prefix and are stripped on the way out while going thru Prox. That way the old ETag stays intact, and we would have a nice spot for future purposes. Smile!

Later,
sidki

- z12 - May. 09, 2005 10:05 PM

Hi sidki

Quote:The added filters are just now active for both options, "Always Cache" and "Always Cache except for HTML", but they aren't needed for the first option, right?

I've never run "Always Cache" but I can't think on any reason you would need the others. Maybe Last-Modified, if you wanted to send conditional requests after the cache expires.

Quote:Did you check if servers that don't send a Last-Modified actually accept an If-Modified-Since?

Not all do. Sometimes you get the full monty back with a 200 reply, but this only seems to happen when the cache is expired and your requesting the resource again, or, they sent the wrong mime-type and you have to ask for it again.

Quote:Excluding local.ptron: Shouldn't it just get the "i_cache:2" (always cache) keyword?

Well, you got me there. Smile!

The truth is, I'm rather clueless about Keywords & IncludeExclude. I've never used a filter set that had those features, so I need to spend some time reading your documentation and looking at the list to really get a handle on it.

I know one thing for sure, if you browse cnn with these filters you'll eventually request your popup.wav file that I cannot seem to cache.

Code:

BlockList 204: in IncludeExclude, line 64

+++GET 204+++

GET /sidki_h_2005-04-02/PopUp.wav HTTP/1.1

Host: local.ptron

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Referer: http://www.cnn.com/2005/WORLD/europe/05/09/harry.sandhurst/index.html

Connection: keep-alive

+++RESP 204+++

HTTP/1.1 200 Local file

Date: Mon, 09 May 2005 12:07:04 GMT

Server: Proxomitron

Connection: close

Last-Modified: Tue, 07 May 2002 15:48:46 GMT

Content-Length: 1598

Content-type: audio/wav

Cache-Control: public, max-age=86400

ETag: prxETag2

It might be a firefox thing, but I see my User-Agent string changes to the real one on this request, so I dunno.

Quote:Re "Cache-Control: 3 Kill if 3xx Response", there was a reason why i added that, but i can't remember which. Do you remember a situation where not killing it for 304s helped? Hmm... maybe i did it because the counter was always reset to 86400 and i never got a fresh copy... gonna check that.

I know why you did it. If you get a 304 to html it will get cached for a day if they don't send back a html Content-Type header, which occurs often with 304's. Been there, done that.

Quote:Out of curiousity, what does "hvINM" stand for?

header variable If None Match. Not very original. Smile!

Quote:When hitting back to http://www.pcworld.com/ from another page i still get this:

I'm going to check that out after this post.

Quote:Re using the ETag field as a var stored by the browser, how about making an "array" (a la keyword/flag/volat) out of it? The first field would be the original ETag, if present. All others would get a "prx-" prefix and are stripped on the way out while going thru Prox. That way the old ETag stays intact, and we would have a nice spot for future purposes.

I never thought of that. That seems like a good idea, hmm...

Mike

Edit: checked out pcworld, got a couple of images that wouldn't cache. Firefox didn't even send the If-Modified-Since header. curious....

- z12 - May. 10, 2005 03:01 AM

Hi sidki

I made the following changes so the original ETag is preserved.

The previous ETag & If-None-Match header filters should be replaced with the following:

Code:

In = TRUE

Out = FALSE

Key = "ETag: 1 Add Proxo ETag (cch!) (In)"

URL = "$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "\0&( $RESP(2*)$IHDR(content-type:([^;]+&&*(html|xml)*)*)$SET(1=prxETag1;\0)|$RESP(2*|304)$SET(1=prxETag2;\0) )"

Replace = "\1"

In = FALSE

Out = TRUE

Key = "If-None-Match: 1 Remove Prox ETag (cch!) (Out)"

Match = "*prxETag(2$SET(hvINM=2)|$SET(hvINM=1))*&*prxETag*;(\s+(?*)\1|)"

Replace = "\1"

In = FALSE

Out = TRUE

Key = "If-None-Match: 2 Remove if empty (cch!) (Out)"

Match = "(^?)"

In = FALSE

Out = TRUE

Key = "If-None-Match: 3 Kill: Fresh!     4.08.11 (cch! ^ctrl) [u] (d.0) (Out)"

URL = "(^$KEYCHK(^S))($KEYCHK(^C)|$TST(keyword=*.i_fresh:[12].*))"

Match = "\0&$LOG(CGET $DTM(c) : If-None-Match killed: \0)"

I must admit, I like this better, good idea. Smile!

Mike

- z12 - May. 10, 2005 09:13 AM

Hi sidki

simplified the following filter

Code:

In = FALSE

Out = TRUE

Key = "If-None-Match: 1 Remove Prox ETag (cch!) (Out)"

Match = "*prxETag(2$SET(hvINM=2)|$SET(hvINM=1))*;\1"

Replace = "\1"

Mike

- z12 - May. 10, 2005 10:19 AM

Hi sidki

modified match in the following filter to account for xhtml & such, w3.org serves this to firefox

Code:

In = TRUE

Out = FALSE

Key = "Cache-Control: 4 max-age 1s if HTML: Cache-H!     4.12.24 (cch!) [sd] (d.0) (In)"

URL = "(^$KEYCHK(^C)|$KEYCHK(^S)|$RESP([45])|local.ptron)$TST(keyword=*.i_cache_h:[12].*)"

Match = "((^$TST(hvINM=2))$RESP(2*)$IHDR(content-type:*(html|xml)*))|($TST(hvINM=1))"

Replace = "max-age=1"

Also at w3.org, the "Content-Location: Show real Onsite URL " filter kicks in often. When this filter matches, the response won't cache. At w3.org, clicking on the css link loads several css files that won't cache.

Any thoughts on this?

Mike

- sidki3003 - May. 10, 2005 11:14 AM

Quote:I made the following changes so the original ETag is preserved.

Cool! Although appending the Prox field may be better, in case something goes wrong.

Quote:I know why you did it. If you get a 304 to html it will get cached for a day if they don't send back a html Content-Type header, which occurs often with 304's. Been there, done that.

Right! That was it. So "Cache-Control: 3 Kill if 3xx Response" isn't needed while using your added filters?

Why do you add prxETag2 (i.e. no HTML) to all 304 responses (incl. HTML)?

Re "Last-Modified: 2 Add if Missing", i find it a bit confusing to always see a Last-Modified in the title now. I used it as an indicator that the server would send a 304 for the next request.
Do you have a few example links where this filter turns a 200 into a 304, so that i can get an idea about the pros and cons?

Quote:modified match in the following filter to account for xhtml & such, w3.org serves this to firefox

XHTML is fine of course. But there's a bunch of other XML Content-types that i personally wouldn't want to include here.

Quote:Also at w3.org, the "Content-Location: Show real Onsite URL " filter kicks in often. When this filter matches, the response won't cache. At w3.org, clicking on the css link loads several css files that won't cache.

Any thoughts on this?

Gonna check that.

sidki

- sidki3003 - May. 10, 2005 12:19 PM

z12 Wrote:I know one thing for sure, if you browse cnn with these filters you'll eventually request your popup.wav file that I cannot seem to cache.

That works for me - also with your added filters. It's loaded but i don't see it in the log the second time - IOW Cache-Control max-age=86400 is in effect. I didn't manage to get the popup again there on reload - to test for conditional response, but here:
http://www.proxomitron.info/tests/poptest.html

Code:

GET 1549 : If-None-Match: prxETag stripped: oth

GET 1549 : Time: 13:49:17::164

+++GET 1549+++

GET /sidki_h_current/PopUp.wav HTTP/1.1

Host: local.ptron

User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; rv:1.7.7) Gecko/20050414 Firefox/1.0.3

Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip,deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Referer: http://www.proxomitron.info/tests/poptest.html

If-Modified-Since: Tue, 07 May 2002 03:48:46 GMT

Connection: keep-alive

RESP 1549 : Time: 13:49:17::344

+++RESP 1549+++

HTTP/1.1 304 Not Modified

Date: Tue, 10 May 2005 11:49:17 GMT

Server: Proxomitron

Connection: close

Content-Length: 0

!URL-ID: http://local.ptron/sidki_h_current/PopUp.wav

!Z12-Debug: oth

Cache-Control: public, max-age=86400

ETag: prx-CT:oth;

+++CLOSE 1549+++

Quote:It might be a firefox thing, but I see my User-Agent string changes to the real one on this request, so I dunno.

The User-Agent filter (in fact, everything except the Content-Type and caching filters) is bypassed for local.ptron. Smile!

sidki

- sidki3003 - May. 10, 2005 01:08 PM

z12 Wrote:Also at w3.org, the "Content-Location: Show real Onsite URL " filter kicks in often. When this filter matches, the response won't cache. At w3.org, clicking on the css link loads several css files that won't cache.

Any thoughts on this?

The server isn't sending any conditional caching headers, nor does it accept any. Even when unticking the Content-Location filter, i get:

Code:

GET 1827 : Cache-Control killed: max-age=0

GET 1827 : If-None-Match: prxETag stripped: oth

GET 1827 : Time: 14:41:13::160

+++GET 1827+++

GET /StyleSheets/Core/Oldstyle HTTP/1.1

Host: www.w3.org

User-Agent: Mozilla/5.0 (Windows NT 5.0; U; rv:1.7.7) Gecko/20050414 Firefox

Accept: text/css,*/*;q=0.1

Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip, x-gzip, deflate

Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7

Keep-Alive: 300

Referer: http://www.w3.org/Style/CSS/

If-Modified-Since: Tue, 10 May 2005 12:40:46 GMT

Connection: keep-alive

Client-IP: 66.80.139.114

Via: 1.1 none:8016 (Topproxy-2.0/), 1.1 athe736a (NetCache NetApp/5.5R4D2)

X-Forwarded-For: 66.80.139.114

RESP 1827 : Cache-Control replaced: max-age=21600

RESP 1827 : Expires killed: Tue, 10 May 2005 18:41:13 GMT

RESP 1827 : Last-Modified added

RESP 1827 : Vary killed: negotiate

RESP 1827 : Time: 14:41:13::711

+++RESP 1827+++

HTTP/1.1 200 OK

Date: Tue, 10 May 2005 12:41:13 GMT

Server: Apache/1.3.33 (Unix) PHP/4.3.10

Content-Location: Oldstyle.css

TCN: choice

P3P: policyref="http://www.w3.org/2001/05/P3P/p3p.xml"

Cache-Control: public, max-age=86400

Transfer-Encoding: chunked

Content-Type: text/css

!URL-ID: http://www.w3.org/StyleSheets/Core/Oldstyle

!Z12-Debug: oth

ETag: prx-CT:oth;

Last-Modified: Tue, 10 May 2005 12:41:13 GMT

+++CLOSE 1827+++

"max-age=86400" seems to work, with and without the Content-Location filter being active: That CSS doesn't appear in the log on revisit. Hard to be really sure about that tho...

One more thing about XHTML caching. While at W3.org, i remembered that i'm changing the Content-Type with "Content-Type: 3 Sel. XML to text/html" like:

Code:

Content-Type: text/html; charset=utf-8; PrxMsg=Changed - Filterable XML: application/xhtml+xml

I was thinking about what way to choose back then: Changing the Content-Type or testing for XHTML in the web filters. I ended up with former because i'm lazy, but also because the $TYPE(htm) test is much faster than any (let alone wildcarded) $IHDR test. So there is no need to include *xml* in the Content-Type check of the caching filters, or did i miss something here?

edit: There is another way to include XHTML for web filters: Testing for e.g. text/html and application/xhtml+xml just once, setting a var hFilter:1, and let the web filters test for this var instead of $TYPE(htm). But i guess i'll leave that for the moment when it becomes necessary. Smile!

sidki

- z12 - May. 10, 2005 04:53 PM

Hi sidki

Quote:appending the Prox field may be better, in case something goes wrong.

Good point.

Quote:Why do you add prxETag2 (i.e. no HTML) to all 304 responses (incl. HTML)?

doh, now that shouldn't be happening. Looks like when I was fooling around with the ETag filters, I overlooked that.

Quote:So "Cache-Control: 3 Kill if 3xx Response" isn't needed while using your added filters?

Not when its working right.

Quote:Re "Last-Modified: 2 Add if Missing", i find it a bit confusing to always see a Last-Modified in the title now. I used it as an indicator that the server would send a 304 for the next request.

I suppose for html, theres no need to do that, since its only getting a 1 sec cache. My main concern was for non html.

Quote:XHTML is fine of course. But there's a bunch of other XML Content-types that i personally wouldn't want to include here.

I didn't want to cache rss feeds, which might not even be xml for all I remember.

Fixes to follow

Mike

- sidki3003 - May. 11, 2005 03:07 PM

Hi Mike,

z12 Wrote:
Quote:So "Cache-Control: 3 Kill if 3xx Response" isn't needed while using your added filters?
Not when its working right.

Oh right, it all depends on the browser sending an If-None-Match along with the If-Modified-Since.
Mozilla, IE6, and Opera 7 (IIRC) do that. The new Opera 8 isn't sending any conditional headers for the main doc any more on reload. Netscape 4.x is sending an If-Modified-Since, but doesn't know what an ETag is. So in latter i get HTML 304s cached for one day without "Cache-Control: 3 Kill if 3xx Response". Considering that, i wouldn't see a reason to *not* strip Cache-Control from 304s - as a safety measure so to speak.

I couldn't find a file where the added Last-Modified turned a 200 into a 304. Maybe you added it because you where previously removing ETags from non-html docs, in which case it's indeed useful (hmm... i think it is, it's a while back that i tested that). So to me it appears like that filter isn't needed anymore - unless i finally find a working example.

Which brings me to another thing that helps caching - maybe that was why you removed ETags: Removing the If-None-Match, *if* there is an If-Modified-Since (in response to a *real* Last-Modified). That may sound like the opposite of my first statement, but we just rely on the If-None-Match to grab the previously determined content-type, it doesn't necessarily need to leave the machine. See for instance the PayPal logo on this board, or various files (images, CSS, JS) at MSN.

I tested a bit with Firefox/Opera/IE and it looks like adding a Prox ETag field is just needed for 200 responses: The old field doesn't get overwritten by a different ETag value, if it comes from a 304.

Regarding my set, i still think that checking for "*xml*" isn't needed b/c of "Content-Type: 3 Sel. XML to text/html". Wink

Thoughts?
sidki

- z12 - May. 11, 2005 07:12 PM

Hi sidki

Quote: Considering that, i wouldn't see a reason to *not* strip Cache-Control from 304s - as a safety measure so to speak.

I had some new filters to post, but after thinking about that, I think I'll wait.

Are you saying reverse the filter order for these two?

Code:

Cache-Control: 3 Kill if 3xx Response:

Cache-Control: 4 max-age 1s if HTML:

Sounds interesting, I'm going to check that out.

Still having problems caching anything when $RDIR is used, including local.ptron. Don't know what to do about this. I emptied my cache, went to cnn then viewd "about:cache" in another tab.

This caught my eye:

Code:

Key: http://local.ptron/killed.gif

     Data size: 55 bytes

   Fetch count: 1

 Last modified: 05/11/05 14:17:30

       Expires: 05/12/05 14:17:28

  Key: http://local.ptron/killed.gif

     Data size: 832 bytes

   Fetch count: 19

 Last modified: 05/11/05 14:17:30

       Expires: 05/12/05 14:17:28

I'm sure the first one is the result of a $RDIR, but I'm not sure why the two are different.

Mike

- sidki3003 - May. 12, 2005 07:53 AM

z12 Wrote:I had some new filters to post, but after thinking about that, I think I'll wait.

Are you saying reverse the filter order for these two?

Just thoughts, open to discussion. Smile!

I'll try to boil them down to (modified) filters and actually test them, probably won't have enough time today.

Quote:Still having problems caching anything when $RDIR is used, including local.ptron. Don't know what to do about this. I emptied my cache, went to cnn then viewd "about:cache" in another tab.

This caught my eye:

Usually those killed.gif calls aren't $RDIRs but replacements for (in most cases harmless positioning) GIFs by the wegbug filter (red.gif for a real webbug).

I cleared cache, went to CNN.
Got "Fetch count: 1" for the 55 bytes killed.gif, which is the real - compressed - image.
Got "Fetch count: 20" for the 832 bytes killed.gif, which is a memory-only uncompressed copy.
That's how Mozilla handles images.

Reloaded CNN page,"Fetch count: 2" for 55 bytes pic, 304 for killed.gif in Prox' log window.
So 304s are included in fetch count.

Went to some other page with a lot of killed.gif replacements and back to CNN, fetch count still 2.
So "Cache-Control 1 day" is in effect.

sidki

- sidki3003 - May. 12, 2005 12:47 PM

Okay, i did a few tests. That global.js at Wired is answering two of my questions, d'oh!
- Your Last-Modified filter is needed.
- Our Cache-Control header has to be added to 304s as well.

I've no solution to that "browsers not sending If-None-Match" thingy, so i unticked "Cache-Control: 4 Kill if 3xx Response: Cache!" for now.

Here is a - barely tested - WIP set (most log lines are temporary):

Code:

[HTTP headers]

In = TRUE

Out = FALSE

Key = "Cache-Control: 3 max-age 1s if HTML: Cache-H!     5.05.12 (cch!) [sd] (d.0) (In) WIP2"

URL = "(^$KEYCHK(^C)|$KEYCHK(^S)|$RESP([45]))$TST(keyword=*.i_cache_h:[12].*)"

Match = "(^$TST(hFinalCT=oth))$RESP(2)$IHDR(Content-Type:( ) text/html)|$TST(hFinalCT=htm)"

Replace = "max-age=1"

In = TRUE

Out = TRUE

Key = "Cache-Control: 4 no-cache: Fresh!     5.05.12 (cch! ^ctrl) [sd] (d.0) (In+Out) WIP1"

URL = "(^$KEYCHK(^S))($KEYCHK(^C)|$TST(keyword=*.i_fresh:[12].*))"

Replace = "no-store, no-cache, max-age=0"

In = TRUE

Out = FALSE

Key = "ETag: Append Prox ETag Field: Cache-H!     5.05.12 (cch!) [z12] (d.0) (In) WIP1"

URL = "$RESP(2)(^$KEYCHK(^C)|$KEYCHK(^S))$TST(keyword=*.i_cache_h:[12].*)"

Match = "(?$SET(1=\0; )*|)\0&($IHDR(Content-Type:( ) text/html)$SET(2=htm)|$SET(2=oth))"

Replace = "\1prx-CT:\2"

In = FALSE

Out = TRUE

Key = "If-None-Match: 1 Strip Prox ETag Field     5.05.12 [z12] (d.0) (Out) WIP1"

Match = "(\0; |)prx-CT:\1&$SET(hFinalCT=\1)$LOG(CGET $DTM(c) : If-None-Match: Prox ETag stripped: \1)"

Replace = "\0"

In = FALSE

Out = TRUE

Key = "If-None-Match: 2 Kill if empty     5.05.12 [z12] (d.0) (Out) WIP1"

Match = "( )(^?)"

In = FALSE

Out = TRUE

Key = "If-None-Match: 3 Kill if IMS present: Cache!     5.05.12 (cch!) [sd] (d.0) (Out) WIP2"

URL = "(^$KEYCHK(^C)|$KEYCHK(^S))$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "?&\1&$OHDR(If-Modified-Since:(^*PrxMsg)*)$LOG(CGET $DTM(c) : If-None-Match: killed b/c of IMS: \1)"

In = FALSE

Out = TRUE

Key = "If-None-Match: 4 Kill: Fresh!     5.05.12 (cch! ^ctrl) [u] (d.0) (Out) WIP1"

URL = "(^$KEYCHK(^S))($KEYCHK(^C)|$TST(keyword=*.i_fresh:[12].*))"

Match = "\0&$LOG(CGET $DTM(c) : If-None-Match killed: \0)"

In = TRUE

Out = FALSE

Key = "Last-Modified: 2 Add if Missing & no HTML: Cache!     5.05.12 (cch!) [z12] (d.0) (In) WIP3"

URL = "$RESP(2)$TST(keyword=*.(i_cache|i_cache_h):[12].*)"

Match = "(^?)&$TST(hFinalCT=oth)|(^$TST(hFinalCT=htm)|$IHDR(Content-Type:( ) text/html))"

Replace = "$DTM(I)\; PrxMsg=Added$LOG(CRESP $DTM(c) : Last-Modified added)"

I've no more time just now, gonna test them in more detail later...

sidki

- Seikatsu - May. 13, 2005 01:34 AM

Wow, there's some serious conferring going on in this thread! Well, if somebody could stop for a moment and look at why the fields on Weather.com's detail page are empty. I would appreciate it. Thanks.