Post Reply 
Help URL Matching In Exceptions-U
Sep. 29, 2009, 11:21 PM
Post: #1
Help URL Matching In Exceptions-U
A link to DSLR.com was recently posted here in another thread, and upon visiting I noticed that some functionality was missing from the linked page.

The blocked functionality is caused by Header Filter, 'Block All Third Party Javascript', but rather than disable that filter, I prefer to create a specific bypass rule. However, I'm confused by some of the things I'm seeing while testing the bypass.

By the way, see the first post in the linked page below as an example (the box containing the text 'http://google.com/safebrowsing/diagnostic?site=URL' is the block in question):

http://www.dslreports.com/forum/r2306352...tic-Report

Normally, that text area would be formatted as a 'code' box with a link which pops open another window with the code in plain text. When third party scripts are blocked, the text area instead appears similar to an input form with links stripped away.

The blocked j

http://i.dslr.net/css/syntaxhiliter/Scripts/shCore.js

causing the issue originates at 'i.dslr.net', so I attempted to allow all javascript from that domain by adding a bypass rule to Exceptions-U.ptxt, but I can't get much joy.

Here are just some of the combinations I tried:

*dslr* $SET(0=a_js.)
i.dslr.net/ $SET(0=a_js.)
([^/]++.|)dslr.net/ $SET(0=a_js.)
[^.].dslr.net/ $SET(0=a_js.)
i.dslr.net/ $SET(0=a_js.)

All of those matching rules match sample URLs tested in the URL test window, but the only one that works at allowing all scripts (i.e., bypasses the 'Block All Third Party Scripts' filter) from i.dslr.net is the first one in the list:

*dslr* $SET(0=a_js.)

Can you help me understand why this is the case?

Thanks.
Add Thank You Quote this message in a reply
Sep. 30, 2009, 01:03 AM (This post was last modified: Sep. 30, 2009 01:06 AM by JJoe.)
Post: #2
RE: Help URL Matching In Exceptions-U
The header filter is a trigger for a web page filter.

! |||||||||||| 7.2 Block all Third Party Scripts
enables
<script> Block: All Third Party Scripts

Looking at the dbug shows

Code:
<Match: <script> Block: All Third Party Scripts     8.11.18 (cch!) [u sd] (d.0) >
<script type="text/javascript" src="http://i.dslr.net/css/ct2m.js?v3">
</Match>

I think
*dslr* $SET(0=a_js.)
worked because it matched
www.dslreports.com/forum/r23063522-Really-cool-feature-Google-Safe-Browsing-Diagnostic-Report

HTH

Later
Add Thank You Quote this message in a reply
Sep. 30, 2009, 03:46 AM
Post: #3
RE: Help URL Matching In Exceptions-U
The keyword should be set for the page's URL while not the script's URL. That's why the first entry worked while others not.

To make the entry more accurate, you could narrow it down to:
Code:
www.dslreports.com/forum/ $SET(0=a_js.)

Otherwise the *dslr* might match something like http://www.anyhost.com/dslrcamera ...
Add Thank You Quote this message in a reply
Sep. 30, 2009, 12:12 PM (This post was last modified: Sep. 30, 2009 12:15 PM by ProxoDent.)
Post: #4
RE: Help URL Matching In Exceptions-U
Okay.

I must have had a basic misunderstanding (though I did understand that '7.2 Block all Third Party Scripts' was a 'trigger' for web filters).

I was hoping that I could differentiate between third-party scripts allowed and disallowed with finer control while using that filter.

As it stands, if I am now understanding you correctly, I must allow all third-party scripts originating from http://www.dslr.com (or /forum or whatever) with something like this:

Code:
www.dslreports.com/forum/ $SET(0=a_js.)

when I only wanted to allow scripts from i.dslr.net?

What if there were lots of other third-party scripts I didn't want to allow (though in my dlsr example that's really not the case)? Then I would have to resort to adding those to a 'blacklist' (for example, in AdHosts-J.ptxt or AdHosts.ptxt or AdDomains.ptxt).

Basically, I'm looking for a method that would allow me to maintain blocking of all third-party scripts and 'whitelist' very specific scripts that I want to allow rather than having to 'blacklist' the one's I don't want, and to do so with finer-grained control than what you guys are suggesting.

Ultimately, I am trying to gain a better understanding of the capabilities as opposed to trying to simply figure out what works for one specific example.

Thanks for the replies. I appreciate your help.
Add Thank You Quote this message in a reply
Sep. 30, 2009, 03:02 PM
Post: #5
RE: Help URL Matching In Exceptions-U
(Sep. 30, 2009 12:12 PM)ProxoDent Wrote:  Basically, I'm looking for a method that would allow me to maintain blocking of all third-party scripts and 'whitelist' very specific scripts that I want to allow rather than having to 'blacklist' the one's I don't want, and to do so with finer-grained control than what you guys are suggesting.

Ultimately, I am trying to gain a better understanding of the capabilities as opposed to trying to simply figure out what works for one specific example..

I think you may have to add that. Sidki?

To add that,
you could use a 'multi disabled' filter that matched and replaced the code before "<script> Block: All Third Party Scripts" finds it.

Like

Code:
[Patterns]
Name = "<script> Override: Block: All Third Party Scripts"
Active = TRUE
URL = "($TYPE(htm)|$TYPE(js)|$TYPE(vbs))$TST(keyword=*.i_script:11.*)"
Bounds = "<s(\\|" [+,] ')+c[^>]++src=*>|<\" [+,] \"s(" [+,] ')+c[^>]++src=*>|<\' [+,] \'s(" [+,] ')+c[^>]++src=*>"
Limit = 2048
Match = "*src=$AV(http://i.dslr.net/*)"
        "&"
        "\1"
Replace = "\1"

You would probably want to add some tests,
dslr.net would need to be linked to dslreport.
The 'whitelist' could be in the filter or a list somewhere.

HTH
Add Thank You Quote this message in a reply
Sep. 30, 2009, 04:27 PM (This post was last modified: Sep. 30, 2009 04:28 PM by ProxoDent.)
Post: #6
RE: Help URL Matching In Exceptions-U
JJoe,

Excellent post.

The filter you posted works fine.*

As suggested I modified it and created a new list to hold my 'override' items.

Code:
[Patterns]
Name = "<script> Override: Block: All Third Party Scripts"
Active = TRUE
URL = "($TYPE(htm)|$TYPE(js)|$TYPE(vbs))$TST(keyword=*.i_script:11.*)"
Bounds = "<s(\\|" [+,] ')+c[^>]++src=*>|<\" [+,] \"s(" [+,] ')+c[^>]++src=*>|<\' [+,] \'s(" [+,] ')+c[^>]++src=*>"
Limit = 2048
Match = "*src=$AV((http+:)+\\+/\\+/($LST(TP-Override-J)))"
        "&"
        "\1"
Replace = "\1"

Though not addressed yet by me, I guess tying the third-party scripts back to the 'first party' (in this case, dslr.net -> dslr.com) would require a variable with the value of the 'first-party' stored and then a $TST() in the 'override' list?

Thanks again.

* Note: As you mentioned, the filter needs to be placed before the web filter 'Block: All Third Party Scripts 09.07.04 (cch!) [u sd] (d.0)' in order to function. Is it usual (and good form) to design filters that must be placed in a specific order? I was thinking that if the filter also matched

Code:
<script type="text/javascript" src="http://local.ptron/sidki_h_2009-05-24/dummy.js" charset="http://i.dslr.net/css/aj6m.js?v3">

and converted that back to a proper script tag, then order wouldn't matter as far as the 'Block All Third Party Scripts' filter is concerned. I guess I never paid much attention to ordering since the set is so complex, figuring out what order things must go in would be (or is) a monumental task.
Add Thank You Quote this message in a reply
Sep. 30, 2009, 07:42 PM (This post was last modified: Sep. 30, 2009 07:53 PM by ProxoDent.)
Post: #7
RE: Help URL Matching In Exceptions-U
(Sep. 30, 2009 04:27 PM)ProxoDent Wrote:  Though not addressed yet by me, I guess tying the third-party scripts back to the 'first party' (in this case, dslr.net -> dslr.com) would require a variable with the value of the 'first-party' stored and then a $TST() in the 'override' list?

Here's the format I've added to my 'override' list:

Code:
([^/]++.|)dslr.net/*($TST(uDom=dslreports.com))

and it works.

Thanks.

PS:
Not sure why the $TST() has to be wrapped in additional parenthesis, i.e., ($TST()) ... that drove me crazy for a few minutes. Smile! Some examples I saw had them, some didn't.
Add Thank You Quote this message in a reply
Sep. 30, 2009, 10:09 PM (This post was last modified: Sep. 30, 2009 10:13 PM by JJoe.)
Post: #8
RE: Help URL Matching In Exceptions-U
Code:
(http+:)+\\+/\\+/

The plus sign indicates a run of repeating characters or nothing.
So "p+" matches a run of 'p's or no 'p's.
(http+:) would match httppppp: or htt: but not https:.
I suspect you want (https+:) or (http(s|):)

ProxoDent Wrote:  would require a variable with the value of the 'first-party' stored and then a $TST() in the 'override' list?

I see that you found uDom.

ProxoDent Wrote:  Is it usual (and good form) to design filters that must be placed in a specific order?

It depends. Some filters must go before others.
In this case "protecting" the code should be best and has been usual.
To undo 'Block: All Third Party Scripts' Multi would need to be enabled and other filters might unnecessarily processes the unnecessary output.
Order would still be important, imo.

When people request filters, Multi is often enabled and efforts may be made to make the filters 'friendly' regardless of what is best or usual.

Code:
([^/]++.|)dslr.net/*($TST(uDom=dslreports.com))

This may cause Proxo to test the variable after every character after / or match too soon.

I'd try

Code:
([^/]++.|)dslr.net/*&$TST(uDom=dslreports.com)

You are welcome.
Have fun.
Add Thank You Quote this message in a reply
Sep. 30, 2009, 10:43 PM
Post: #9
RE: Help URL Matching In Exceptions-U
(Sep. 30, 2009 10:09 PM)JJoe Wrote:  
Code:
(http+:)+\\+/\\+/

The plus sign indicates a run of repeating characters or nothing.
So "p+" matches a run of 'p's or no 'p's.
(http+Smile! would match httppppp: or htt: but not https:.
I suspect you want (https+Smile! or (http(s|)Smile!

Went with:
Code:
(http(s|)

(Sep. 30, 2009 10:09 PM)JJoe Wrote:  
Code:
([^/]++.|)dslr.net/*($TST(uDom=dslreports.com))

This may cause Proxo to test the variable after every character after / or match too soon.

I'd try

Code:
([^/]++.|)dslr.net/*&$TST(uDom=dslreports.com)

Done. And all is working fine. Just adding items to the 'override' file as I go along. So, far, it doesn't look like I'll need that many as most pages work to my satisfaction even with all third party scripts blocked (keeping my fingers crossed).

Your help has been spot on ... thanks again.
Add Thank You Quote this message in a reply
Oct. 01, 2009, 02:50 PM
Post: #10
RE: Help URL Matching In Exceptions-U
(Sep. 30, 2009 03:02 PM)JJoe Wrote:  I think you may have to add that. Sidki?

Perhaps as a config plugin. Wink

As for whitelist format and invoking, it could be similar to AdHosts-J (with opposite action, of course), which turned out to work well.
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: