Post Reply 
Andrew's Security Filter(s) v5.62 (May 10, 2009)
Jul. 13, 2007, 07:53 AM (This post was last modified: Dec. 23, 2010 11:32 PM by Kye-U.)
Post: #1
Andrew's Security Filter(s) v5.62 (May 10, 2009)
Below version outdated. Newest version can be found here: http://prxbx.com/forums/showthread.php?t...&pid=15168

Tested and working in IE, Firefox, Chromium, Safari, K-Meleon, Opera. Should work well in all browsers that allow the user to use an HTTP proxy. There may be inconsistencies across browsers in terms of appearance.

What it does:

I'd like to think of it as a "NoScript" for all browsers (thanks to Proxomitron of course).

Basically removes all <(no|)script|applet|object|embed)> (and iframe as of Nov. 11, 2007) tags on sites that aren't in the whitelist, and gives you the option of adding an untrusted site to the whitelist - (by two methods: 1) by domain, showing you have great trust in them and, 2) by domain + path, which is ideal for a trusted subdirectory on a domain) - in the very simple yet easy to use "navigation" bar that appears on all filtered sites.

You can easily choose to allow embed or iframe tags (for example) by default by simply going into Header filters and unticking the respective checkbox beside the filter(s). By default, all are set to be removed.

Screenshots:


.png  filter1.png (Size: 12.46 KB / Downloads: 2162)
Buttons are now translucent (when not hovered over)


.png  filter2.png (Size: 15.94 KB / Downloads: 1700)
Counters are shown when your cursor is over the two buttons (and are automatically hidden when your mouse isn't over them). New in v5.60, you see a list of external, third-party scripts that have been removed (if any)


.png  filter3.png (Size: 15.53 KB / Downloads: 1573)
If you click on Allow (the green button), you can choose to allow the whole host or the specific path.


.png  filter4.png (Size: 30.38 KB / Downloads: 1578)
If you click on Advanced, you're able to tick which elements you want to allow for the host/path. The Advanced link acts as a toggle to display the element checklist.

NEW: All-in-one Package

I've put all the essential files in a Zip file that you just need to extract to your Proxomitron folder and merge the "mergeme" file with your config file. Note that you will need to download the "Count.ptxt" file listed below if you're not using Sidki's config pack.

.zip  ASFv5.62.zip (Size: 4.73 KB / Downloads: 3441)

Manual Steps:
  • import the following filters into Proxomitron (All 7 Header filters and both of the 2 Web Filters)
  • Create a new TXT file and add it to Proxomitron's Configuration as a Blockfile named "andrew" (without the quotation marks)
  • Create another new TXT file and add it to Proxomitron's Configuration as a Blockfile named "andrew-src" (without the quotation marks)
  • Download the attached "andrew.js" file and place it in the HTML folder in your Proxomitron installation folder

    .js  andrew.js (Size: 10.87 KB / Downloads: 1436)
  • Download the attached "Count.ptxt" file and add it to Proxomitron's Configuration as a Blockfile named "Count" (without the quotation marks) (this step is not required if you are using Sidki's config pack)

    .ptxt  Count.ptxt (Size: 3.5 KB / Downloads: 3026)

Please bypass Proxomitron and refresh this page as a precaution to ensure no filters have modified the contents of this post! (and of the attachments)

What's New?

May 10, 2009 - Version 5.62
  • if removal of scripts is not whitelisted for a site, then any javascript in the href attribute for <a> tags is removed (e.g.: jalert('hi'); )

NOTE: v5.55+ now uses a new method for its whitelist. You must clear the "andrew" blocklist and start off fresh (or add $SET(0=.ALL.) at the end of each line) in order for this new version to work properly.

Header Filters (7):

Code:
[HTTP headers]
In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: !Load Bypass (Out)"
URL = "$LST(andrew)$SET(a_bypass=\0)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <APPLET> Removal (Out)"
URL = "(^$TST(a_bypass=*.APPLET.*))$SET(a_applet=1)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <EMBED> Removal (Out)"
URL = "(^$TST(a_bypass=*.EMBED.*))$SET(a_embed=1)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <IFRAME> Removal (Out)"
URL = "(^$TST(a_bypass=*.IFRAME.*))$SET(a_iframe=1)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <NOSCRIPT> Removal (Out)"
URL = "(^$TST(a_bypass=*.NOSCRIPT.*))$SET(a_noscript=1)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <OBJECT> Removal (Out)"
URL = "(^$TST(a_bypass=*.OBJECT.*))$SET(a_object=1)"

In = FALSE
Out = TRUE
Key = "! Andrew Security Filter: Enable <SCRIPT> Removal (Out)"
URL = "(^$TST(a_bypass=*.SCRIPT.*))$SET(a_script=1)"

Web Filters (2):

Code:
[Patterns]
Name = "Andrew's Security Filter v5.62 20090510"
Active = TRUE
URL = "$TYPE(htm)\9($TST(\9=\8(\&|\?)trust=bypass)$SET(bypassing=1)$SET(origlink=\8)|)$SET(scriptn=0)$SET(noscriptn=0)$SET(appletn=0)$SET(objectn=0)$SET(embedn=0)$SET(iframen=0)"
Limit = 1500
Match = "(^$TST(bypassing=1))"
        "("
        "(<script([^>]++src="((http(s|)://[^/]++\h/|/)*|[^"':]+)\2"[^>]+>(^$TST(a_bypass=*.ALL.*))$TST(a_script=1)"
        "|[^>]++src="(http(s|)://(^$LST(andrew-src)*)\3)\2"[^>]+>(^$TST(\3=[^/]++\h/*))$SET(scriptsrc=$GET(scriptsrc)\2|)"
        "|(([^>]+)\4|)>(^$TST(\4=*src=*))(^$TST(a_bypass=*.ALL.*))$TST(a_script=1))"
        "$TST(($GET(scriptn)+)=$LST(Count)|*)$SET(scriptn=$GET(i))$SET(1=<script type="foo">)$SET(andrew=yes)"
        ")"
        "|"
        "(^$TST(a_bypass=*.ALL.*))"
        "("
        "(<(applet$TST(a_applet=1)$TST(($GET(appletn)+)=$LST(Count)|*)$SET(appletn=$GET(i))"
        "|object$TST(a_object=1)$TST(($GET(objectn)+)=$LST(Count)|*)$SET(objectn=$GET(i))"
        "|embed$TST(a_embed=1)$TST(($GET(embedn)+)=$LST(Count)|*)$SET(embedn=$GET(i))"
        ")$SET(1=<foo)"
        "|<(noscript$TST(a_noscript=1)$TST(($GET(noscriptn)+)=$LST(Count)|*)$SET(noscriptn=$GET(i))"
        "|iframe$TST(a_iframe=1)$TST(($GET(iframen)+)=$LST(Count)|*)$SET(iframen=$GET(i))"
        ")[^>]+>$SET(1=<textarea style="display: none !important;">)"
        "|</(applet$TST(a_applet=1)"
        "|object$TST(a_object=1)"
        "|embed$TST(a_embed=1)"
        ") >$SET(1=</foo>)"
        "|</(noscript$TST(a_noscript=1)"
        "|iframe$TST(a_iframe=1)"
        ") >$SET(1=</textarea>)"
        ")$SET(andrew=yes)"
        "|[^a-z]on([a-z]+)\5=$TST(a_script=1)$SET(1= on\5foo=)"
        "|[^a-z]href=("|)\6javascript:$TST(a_script=1)$SET(1= href=\6#foo\6 hreffoo=\6javascript:)"
        ")"
        ")"
        "|(^*?)$STOP()"
        "("
        "($TST(andrew=yes)|$TST(bypassing=1))"
        "$SET(1=\r\n\r\n<script type="text/javascript" src="http://local.ptron/andrew.js"></script>\r\n"
        "<script type="text/javascript">\r\n"
        "prx_a_array($GET(scriptn), $GET(noscriptn), $GET(iframen), $GET(appletn), $GET(embedn), $GET(objectn), "$GET(origlink)", "$GET(scriptsrc)");\r\n"
        "</script>)"
        ")"
Replace = "\1"

Name = "Andrew's Security Filter - Whitelist 20090509"
Active = TRUE
URL = "(^$LST(andrew))(^$LST(andrew-src))(\w.|)\7\8.\9/\0(\&(amp;|)|\?)trust=(h|p|ext)\5\&(amp;|)tags=([A-Z.]+)\6$SET(type=\5)($TST(\6=.*)$SET(code=\t\t$SET\(0=\6\))|)$SET(trust=\8.\9)$SET(trustsrc=\7\8.\9/)($TST(\5=p)$SET(trust=$GET(trust)/\0)|)"
Limit = 1
Match = "?($TST(type=(p|h))$SET(4=$ADDLST(andrew, ([^.]+.|)$GET(trust)$GET(code)))"
        "|$TST(type=ext)$SET(4=$ADDLST(andrew-src, $GET(trustsrc))))"
Replace = "\k\4"
          "$ALERT(Success! $GET(trust) has been added to the whitelist.)"
          "<script>window.parent.location.reload();</script>"

Test Page Here:

http://prxbx.com/test/test.html
http://prxbx.com/test/test.php (evil version, click with caution.)

What's Old?

May 9, 2009 - Version 5.61
  • off-domain scripts will now continue to be filtered even if all tags (e.g.: .ALL.) have been allowed for a host/path
  • minor CSS modifications to make interface more consistent across sites
  • simplification/clean-up of filter and JavaScript, fixing minor issues

May 8, 2009 - Version 5.60
  • iframe dynamically generated when whitelisting a path/host (instead of going to a separate page and then redirecting back to the original page)
  • ability to filter external scripts individually
  • there were some sites that had elements with an "onclick" attribute without any actual "offensive" elements (script, iframe, embed, etc), causing the "A B" boxes to appear with a blank counter box
  • interface was improved (to make it more compact, non-intrusive)
  • you no longer need to click exactly on the checkbox for the tags, you can click on the label and it will check the respective checkbox (or uncheck)
  • interface is now more consistent across various pages (less styles "leak" into it)

August 1, 2008 - Version 5.59
  • Refined JavaScript event matching code (Thanks z12!)

July 13, 2008 - Version 5.58
  • Fixed bypassing glitch (due to missing parenthesis)
  • Modified replacement text for iframe and noscript

July 8, 2008 - Version 5.57
  • Removed noapplet, noobject and noembed options (as these tags are pretty much non-existent on the Internet)
  • If the removal of SCRIPTs is enabled, then all Javascript event attributes (onmouseover, etc.) are changed to "foo"
  • Uses new, faster replacement method (Thank you z12!)

June 15, 2008 - Version 5.56
  • Cut down the number of web filters from 4 to 3 (merged the "Trust Path/Host" filters into one)
  • Fixed a typo in the matching expression

June 11, 2008 - Version 5.55
  • Further optimized the filters/javascript
  • Created option to allow the bypassing of certain tags (more than one, if needed). Has a built-in check to make sure there are no duplicates in the allow list
  • Now has only two buttons, "A" for Allow and "B" for Bypass. You can find the links to whitelist the entire host or path when you click on the "A"

June 11, 2008 - Version 5.5
  • Improved counter javascript code (simplified + now it only shows the tags that have been removed. If there is no occurance of a certain tag found on a page (e.g. noembed), then it will not show "0 <NOEMBED>")
  • Made IFRAME more specific (to match SRC parameters that actually have a value)
  • Fixed a minor styling bug with the three injected buttons
  • Not dependent on there being a </body> or </html> tag; it will inject itself on any page with HTML content

June 10, 2008 - Version 5
  • Optimized code (Thank you z12!)
  • Added option to remove noembed, noobject, noapplet tags
  • Fixed the embed match (as the closing embed tag is optional)

June 8, 2008
  • Updated display of buttons + counters
  • Improved "Trust Path" whitelist submission yet again (uses wildcard "([^.]+.|)" as a prefix to the domain name + path)

April 22, 2008
  • Improved "Trust Path" whitelist submission (uses wildcard "[^/]++" as a prefix to the domain name + path)

November 11, 2007
  • drastically modified the coding (uses JS for injected bar, removes all selected code completely, counts number of instances)
  • created two main webfilters, one that counts and one that doesn't
  • created Header filters to enable simple and easy selection of what tags to remove

August 4, 2007
  • Improved matching, uses (more) proper syntax (doesn't use the <andrew> tag Wink)
  • Fixed "overflow" issue

August 2, 2007
  • Improved "Trust Host" whitelist submission (uses wildcard "[^/]++" as a prefix to the domain name)

ALL COMMENTS/FEEDBACK WELCOME
Visit this user's website
Add Thank You Quote this message in a reply
Aug. 02, 2007, 10:23 AM
Post: #2
RE: Andrew's Security Filter(s)
Hi!
I love it, thank you Smile!
I have a q though, some sites keep being filtered even though added to the whitelist?
Quote this message in a reply
Aug. 02, 2007, 11:05 AM
Post: #3
RE: Andrew's Security Filter(s)
Hmm.. That wasnt very informative was it Wink
Ill try to elaborate some(im no proxo expert so please bear with me Smile!):
Ive set up the filter and done some interface editing, dont think ive messed it up as it seems to work fine mostly.
I add an address via either the "security bar" or the white list through the proxomitron tray icon menu.
Usually this works fine and the next time i visit the site its not filtered and the "security bar" doesnt show.
But on some sites it seems not to stick, even though its added the bar shows up and the page is still "crippled"
Itl load correctly once or twice then revert.
fex. isohunt.com. most of the other are private torrent sites so im a bit hesitant to post the urls in public Smile!
Add Thank You Quote this message in a reply
Aug. 02, 2007, 02:01 PM
Post: #4
RE: Andrew's Security Filter(s)
Are you trusting the host or the path? Wink

I've noticed that the filter's been unnecessarily matching pages and "hiding" the entire page; I'll have a look at it soon.
Visit this user's website
Add Thank You Quote this message in a reply
Aug. 02, 2007, 02:07 PM
Post: #5
RE: Andrew's Security Filter(s)
Lol ive tried both Smile!
Cool..much appreciated, friggin awesome filter.
Add Thank You Quote this message in a reply
Aug. 02, 2007, 06:33 PM
Post: #6
RE: Andrew's Security Filter(s)
usr;

Or it could be that you have another filter matching the "inappropriate" code first.

If Filter #1 is matching the code, Kye-U's filter (as Filter #2) won't find it - even it's unchanged by Filter #1. Sad (Normally, Proxo matches only on previously unmatched code, look it up in the help file.) There is a standard workaround of course, we turn on the Multi option. However, if Filter #1 did change the code, then the results won't be found by Filter #1 either, and you're still out of luck. Banging Head

As in most things experimental like this, I suggest that you integrate new filters at the top of your configuration, not in the middle or at the bottom. Once you've made sure it works, then you can try moving it around, should you feel the need. Whistling

(The foregoing presumes you are using Proxo 4.5(m|j). Earlier versions did not guarantee that filters would be loaded in the same order as found in the configuration file.)

HTH


Oddysey

I'm no longer in the rat race - the rats won't have me!
Add Thank You Quote this message in a reply
Aug. 02, 2007, 09:11 PM
Post: #7
RE: Andrew's Security Filter(s)
Hi!
Thanks for the input, much appreciated.

Certainly could be, just noticed the "problem" today so im keeping an eye on it to see if anything reveals itself.
The andrew filters are/were on top and im using proxo 4.5 june.

Cheers Smile!

Edit:
Is it possible there might be a unintentional limit for entries in the whitelist?
Add Thank You Quote this message in a reply
Aug. 02, 2007, 10:43 PM
Post: #8
RE: Andrew's Security Filter(s)
Dear usr,

I've updated the filters (since you've modified it, the filter that I changed is "Andrew's Security Filter - Add Safe Host", the other two filters have not been changed).

What interface changes have you done? Wink

As for ISOHunt.com, I've tested it and it works fine. The whitelist entry "sticks" Smile!

Perhaps there is a limit to the number of entries in the whitelist. I'm not really sure myself; however, the update should help cut down on the number of entries (e.g.: "www.test.com" and "test.com" would both be added to the whitelist since, before, there was no wildcard preceding the domain name)
Visit this user's website
Add Thank You Quote this message in a reply
Aug. 02, 2007, 11:53 PM
Post: #9
RE: Andrew's Security Filter(s)
Hi Kye-U,
Nice, thanks that was fast Smile!
Ill update and test.

Interface changes: nothing much really, i like it to be "blend in" a little, so i changed to white background all around, 1 pixel border, plain font instead of bold and resized text to 6px and the rest to 12px.
Looks a bit odd on some sites but in general i think its ok.

Nice touch with the wildcards, was kinda wondering about that one Wink

Im gonna test more thorough though and see if i can reproduce it with more accuracy as to numbers etc.
Add Thank You Quote this message in a reply
Aug. 04, 2007, 04:26 AM
Post: #10
RE: Andrew's Security Filter(s)
One page that the old filter was "killing" was:

http://www.reuters.com/article/musicNews...4120070803

If you use the updated filters (in the first post), you'll notice that the page now loads in its entirety Smile! (Minus the <((no|)script|applet|object|embed))> tags of course).

If you view the source and the filter is not bypassed, you'll see something like:

Code:
<!-- // Andrew //
[script]  language="JavaScript" type="text/javascript" src="/resources/js/browser-sniff.js"> [/script]
\\ Andrew \\ -->

The code is safely commented out, and if you ever want to know what tag it had originally, just look inside the square brackets!

(All filters have been updated)
Visit this user's website
Add Thank You Quote this message in a reply
Aug. 04, 2007, 10:21 AM
Post: #11
RE: Andrew's Security Filter(s)
Nice, thanks Smile!


Good news!! The only site affected (using the updated filter) are isohunt, the others are now ok.

Short report from me(using the updated filter):
Its not related to my editing.
It reverts to active state if i edit the list manually.
Isohunt is still present in the list though.
It does not occur if isohunt is the only site in the list, if i delete the other sites leaving only isohunt in the list its bypassed proper.
The white list wont accept isohunt after reverting to active state(Even though i add it through the bar its not added to the list)
If i empty the cache, restart opera and re add isohunt it sticks, can be several isohunt entries in the list, but obviously the latter are the one that activates the bypass.

Since its no problem to fix when it occurs and since it only affects isohunt(afaik) its no hassle really, so im not going to pursue this any further, of course if your interested ill be happy to help with testing.

Thanks again for the filter, and all the other filters youve written, thanks also for all the work youve put into the proxo community, that goes for Oddysey and you other experts too Smile!
Much MUCH appreciated, keep up the good work Cheers
Add Thank You Quote this message in a reply
Oct. 22, 2007, 05:15 AM
Post: #12
RE: Andrew's Security Filter(s)
I'll be posting a newer version sometime this week/next week.

New features/goals:

-include removal of IFRAMEs
-improve interface (use Javascript instead of an Absolute-positioned DIV layer)
-possibly add content placeholders (for IFRAMEs?)
-reduce the space taken up by removed <(no|)script|applet|object|embed|iframe)> tags by removing newline, return, tab characters
-optimize coding
Visit this user's website
Add Thank You Quote this message in a reply
Nov. 02, 2007, 08:40 PM
Post: #13
RE: Andrew's Security Filter(s)
Looking foreward to it Smile!
Add Thank You Quote this message in a reply
Nov. 11, 2007, 05:27 AM
Post: #14
RE: Andrew's Security Filter(s) v3 (Nov. 11, 2007)
Updated!

Please disable or remove the previous filters before following the instructions in the first post. Smile!
Visit this user's website
Add Thank You Quote this message in a reply
Nov. 11, 2007, 11:27 AM
Post: #15
RE: Andrew's Security Filter(s) v3 (Nov. 11, 2007)
Pure genius, thanks man Big Teeth
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: