Post Reply 
Setting up Tor with Freecap and Proxomitron
Feb. 04, 2009, 08:15 AM
Post: #61
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 04, 2009 06:23 AM)Kye-U Wrote:  2. I doubt you'd need to modify the config, as I believe the HOSTS file is referenced to on every connection, regardless it's through a proxy server Wink

It's not true. According to http://www.pps.jussieu.fr/~jch/software/...o.html#DNS:
Quote:Polipo usually tries to speak the DNS protocol itself rather than using the system resolver

I just made a test. I can access the entry in HOSTS file under the default configuration.

I didn't try it but I think you can disable Polipo's DNS resolver to make the HOSTS file work:
Code:
# Uncomment this to disable Polipo's DNS resolver and use the system's
# default resolver instead.  If you do that, Polipo will freeze during
# every DNS query:

# dnsUseGethostbyname = yes

What's more, according to http://www.pps.jussieu.fr/~jch/software/...ME.Windows:
Quote:You will probably need to point it at your config file with an explicit ``-c''
command-line argument, and define at least the following configuration variables:

dnsNameServer
diskCacheRoot
forbiddenFile

The forbiddenFile variable is not needed if you are not using the forbidden version.

You can ignore the dnsNameServer variable too if you want to use the system's default resolver.
Add Thank You Quote this message in a reply
Feb. 04, 2009, 03:19 PM
Post: #62
RE: Setting up Tor with Freecap and Proxomitron
Thanks Kye-U and whenever, I have it working now and it does seem that pages load noticeably faster.
I'll fiddle with the other variables enabled in the config (eg. commenting out dnsUseGethostbyname = yes).
Thanks again.

soccerfan
Add Thank You Quote this message in a reply
Feb. 04, 2009, 06:37 PM
Post: #63
RE: Setting up Tor with Freecap and Proxomitron
Kye-U and soccerfan:
May I ask you what browser are you using?

I wonder where that sudden speed up is coming.

FROM: http://en.wikipedia.org/wiki/HTTP_pipelining
Quote:Implementation in web browsers:
Internet Explorer as of version 7 doesn't support pipelining.
Mozilla Firefox 3.0 supports pipelining, but it's disabled by default.
Konqueror 2.0 supports pipelining, but it's disabled by default.
Opera has pipelining enabled by default.
Google Chrome is not believed to support pipelining.

I believe Proxo DOES support persistent conections AND pipelining per se (so no need to use polipo JUST for pipelining - unless your using IE wich I really doubt).

Is there any other pro I'm overlooking?
I can see the benefit of having a LOCAL SHARED cache in a populated small network, but not as an install for use in single computer.
Browsers already have caches, is polipo any better?

Sidki, didnt catch the part of proxo not sending some header. Care to rephrase it? Wink

Maybe someone would find this of use:
Quote:OperaTor is a portable software bundle which allows you to browse the web anonymously. It combines the power of the Opera Browser, Tor and Polipo.
With OperaTor no data will be stored at the computer you plugged your portable memory into.

Link: http://archetwist.com/opera/operator
Add Thank You Quote this message in a reply
Feb. 04, 2009, 07:21 PM
Post: #64
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 04, 2009 06:37 PM)eclipse Wrote:  Kye-U and soccerfan:
May I ask you what browser are you using?

I'm using k-meleon (1.1.3 and 1.5.2). And it is just a perceived speed up. Hard for me to tell if it is really significant.

For anyone interested, I also found a nifty gui wrapper for polipo (called solipo Smile! ) here: http://serennz.sakura.ne.jp/toybox/solipo/
No install required and rready to go. Works on windows XP (which I use) and may work on Vista (author did not test on Vista Kye-U Wink )

soccerfan
Add Thank You Quote this message in a reply
Feb. 04, 2009, 07:43 PM
Post: #65
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 04, 2009 06:37 PM)eclipse Wrote:  Sidki, didnt catch the part of proxo not sending some header. Care to rephrase it? Wink

Common proxies send different headers than un-proxified clients.
Some people emphasize to appear un-proxified.
Just have a look at above linked script.
Add Thank You Quote this message in a reply
Feb. 05, 2009, 02:18 AM
Post: #66
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 04, 2009 07:21 PM)soccerfan Wrote:  For anyone interested, I also found a nifty gui wrapper for polipo (called solipo Smile! ) here: http://serennz.sakura.ne.jp/toybox/solipo/
Thanks for this found. This build could use disk cache under windows, which is what I tried to do with the official build yesterday but without success.
Add Thank You Quote this message in a reply
Feb. 05, 2009, 02:30 AM
Post: #67
RE: Setting up Tor with Freecap and Proxomitron
Quote:some people were upset that Prox isn't sending a "HTTP_CONNECTION=keep-alive" header. Or - when pipelining is turned off - is sending a "HTTP_KEEP_ALIVE=300".

In my case I have always a "HTTP_CONNECTION=keep-alive" header.
BUT, it could be a side effect of my ISP's TRANSPARENT PROXY.
And I thought that was the whole point in using HTTP 1.1, keeping alive connections and having less overhead so how come proxo isn't sending it?


Quote:Some people emphasize to appear un-proxified.
How could anyone tell I'm behind a proxy by those two headers ? ("HTTP_CONNECTION=keep-alive" - "HTTP_KEEP_ALIVE=300")
So chaining proxo > polipo is helping with that?

Quote:Just have a look at above linked script.
I can't see any substantial difference (other than proxo modified headers vs. bypassed normal headers). What should I be looking for?


Mmm too many questions, too late, I need some sleep. Dead Tired
I hope all this makes sense tomorrow Crazy
Add Thank You Quote this message in a reply
Feb. 05, 2009, 03:29 AM
Post: #68
RE: Setting up Tor with Freecap and Proxomitron
Sorry eclipse, i'm not the right person to ask about such things.
Apparently the people who created the ProxyJudge script were analyzing a lot of headers. Smile!

In case it's helpful for someone, here is the crucial subroutine (Perl, Japanese, 2003):
Code:
#-----// Proxy env. value,User-Agent, Keep-Alive check //----#
sub proxyenv_check {
  local( $szEnvValue, $szName, $szValue, $szNameStat, $szValueStat );

  foreach $szEnvValue ( @aszEnvValues ) {
    chop( $szEnvValue ) if( $szEnvValue =~ /\n$/ );
    ( $szName, $szValue, $szNameStat, $szValueStat )
      = split( /\t/, $szEnvValue );

    if( $szName eq 'HTTP_USER_AGENT=' && $szValue =~ /via/i ) {
      $szValueStat =~ s/via/$szColor$szWarn">via$szEndTag/i;
      $szResult    = 'Via a Proxy';
      if( $fszJapanese eq 'yes' ) {
        $szComment  .= ' Via ¤Ã¤Æ¡Ö¡Á·Ðͳ¡×¤Ã¤Æ°ÕÌ£¤Ç¤¹¤è¡£';
      } else {
        $szComment  .= ' USER_AGENT includes "via".';
      }

    } elsif( $szName eq 'HTTP_CONNECTION=' ) {
      if( $szValue !~ /Keep-Alive/i ) {
        $szValueStat = "$szColor$szWarn\">$szValueStat$szEndTag";
        $szResult    = 'Anonymized' if( $szResult ne 'Via a Proxy' );
        if( $fszJapanese eq 'yes' ) {
          $szComment  .= ' °ìÈÌŪ¤Ê¥Ö¥é¥¦¥¶¤À¤È Keep-Alive ¤¬½Ð¤Þ¤¹¡£';
        } else {
          $szComment  .= " CONNECTION doesn't have \"Keep-Alive\".";
        }
      } else {
        $iNoProxy++;
        $fszKeepAlive = 'existed';
      }

    } elsif( $szName !~ /HTTP_(CONNECTION=|USER_AGENT=|HOST=|PRAGMA=|UA_|ACCEPT|REFERER=|MIME=|EXTENSION=|IF_MODIFIED_SINCE=|COOKIE=)/ ) {

      if( $szName !~ /HTTP_(VIA=|.ROXY_.ONNECTION=|X_FORWARDED_FOR=|FORWARDED=|CACHE_CONTROL=|CACHE_INFO=|FROM=|CLIENT_IP=|TE=|SP_HOST=|XONNECTION=)/ ) {
        $szNameStat =~ s/$szName/$szColor$szSuspect">$szName$szEndTag/i;
        $szResult   = 'Via a Proxy';
        if( $fszJapanese eq 'yes' ) {
          $szComment .= ' ̯¤ÊÊÑ¿ô¤¬½Ð¤Æ¤Þ¤¹¡£' if( $szComment !~ /̯¤ÊÊÑ¿ô¤¬½Ð¤Æ¤Þ¤¹¡£/ );
        } else {
          $szComment .= ' Dubious valuable is detected.' if( $szComment !~ /Dubious/ );
        }
      } else {
        $szNameStat =~ s/$&/$szColor$szWarn">$&$szEndTag/i;
        $szResult   = 'Via a Proxy';
        if( $fszJapanese eq 'yes' ) {
          $szComment .= ' Proxy ÊÑ¿ô½Ð¤Æ¤Þ¤¹¡£' if( $szComment !~ /Proxy ÊÑ¿ô½Ð¤Æ¤Þ¤¹¡£/ );
        } else {
          $szComment .= ' Proxy servers valuable is detected.' if( $szComment !~ /Proxy/ );
        }
      }
    } else {
      $iNoProxy++ if( $szName !~ /HTTP_(CONNECTION=|HOST=|PRAGMA=|UA_|ACCEPT|REFERER=|MIME=|EXTENSION=|IF_MODIFIED_SINCE=|COOKIE=)/ );
    }
    $szEnvValue = "$szName\t$szValue\t$szNameStat\t$szValueStat\n";
  }
  return;
}
Add Thank You Quote this message in a reply
Feb. 05, 2009, 07:53 AM
Post: #69
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 05, 2009 02:30 AM)eclipse Wrote:  
Quote:Some people emphasize to appear un-proxified.
How could anyone tell I'm behind a proxy by those two headers ? ("HTTP_CONNECTION=keep-alive" - "HTTP_KEEP_ALIVE=300")
So chaining proxo > polipo is helping with that?

There's some misconception and vague references in the HTTP RFC's. Both the "Connection:" and "Keep-Alive:" headers are hop-by-hop. A proxy sits between the clients and servers, it's up to the proxy to decide how it wants to play ball with communication in either direction.

Many modern browsers do send the "Connection: Keep-Alive" header as explicit indication that they want to be able to re-use an established connection, even if it turns out that server is only HTTP/1.0. Most (not all) browsers send the "Proxy-Connection:" version to a proxy, but servers expect to see "Connection:". If a server sees "Proxy-Connection:" then that's a good indication of a (faulty) proxy in the mixture.

A browser is not likely to send the "Keep-Alive:" header, I've not seen any do that (yet). So if a server sees "Keep-Alive:" then it's probably been put there by a proxy.
Edited: Firefox seems to be sending the "Keep-Alive:" header with a value of 300 seconds of desired persistence.

---

One open question would be for how long should that TCP connection remain open for possible re-use ... a few seconds? 2 minutes? 10 minutes? That's where the "Keep-Alive:" header might come into play.

The detrimental impact of keeping connections open is more on the server's end keeping track of them. Servers will sometimes respond with both a "Connection: Keep-Alive" header, and also something like "Keep-Alive: 300" trying to indicate willingness to wait 300 seconds maximum time. Unfortunately there isn't a real standard for the "Keep-Alive:" header value. RFC 2616 points back to RFC 2068 on the subject, and that one says "HTTP/1.1 does not define any parameters" for the Keep-Alive: header.

Consider this response from cnn.com:
Keep-Alive: timeout=5, max=64
Connection: Keep-Alive


It's trying to indicate willingness to remain connected for 5 seconds and we shouldn't be keeping more than 64 of them. If ours was a proxy servicing many people at a company, then that server is asking for no more than 64 connections be left open for re-use by that company's proxy. Who knows, beyond 64 the server could initiate a denial-of-service defense.

Similarly, a request to ietf.org yields:
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive


This site's server headers contain:
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive


I know how my proxy handles connection management, but know little about how Proxo does it.

The Polipo proxy seems to be intentionally aggressive about keeping server connections open for possible re-use. Doing that in a network environment like Tor might make sense because of the excessive cost of opening those connections. Othersise I'd rather be "kinder and gentler" with regard to the duration of retaining open connections and server resources.
Add Thank You Quote this message in a reply
Feb. 05, 2009, 02:38 PM
Post: #70
RE: Setting up Tor with Freecap and Proxomitron
(Feb. 05, 2009 02:18 AM)whenever Wrote:  
(Feb. 04, 2009 07:21 PM)soccerfan Wrote:  For anyone interested, I also found a nifty gui wrapper for polipo (called solipo Smile! ) here: http://serennz.sakura.ne.jp/toybox/solipo/
Thanks for this found. This build could use disk cache under windows, which is what I tried to do with the official build yesterday but without success.

It does cache to disk (in a default folder named cache). You can change the default location
or default cache limit (200MB) if you right-click on the solipo tray icon and select settings.
The new settings are written to the file solipo.ini (not initially present). Hope this helps.

soccerfan
Add Thank You Quote this message in a reply
Feb. 05, 2009, 02:43 PM
Post: #71
RE: Setting up Tor with Freecap and Proxomitron
Trank you Graycode! That was very enlightening... maybe the sleep helped too Eyes Closed Smile
Add Thank You Quote this message in a reply
Post Reply 


Forum Jump: