ProxHTTPSProxyMII: Development

[GunGunGun]

(Jan. 04, 2015 02:51 AM)whenever Wrote:  

(Dec. 31, 2014 04:30 AM)GunGunGun Wrote:  Yes, that file is an empty file, but its original file size is 32bytes and fill up with space character. But If I use Privoxy and MITM 1.2, the file's file size after decode gzip is 1bytes only.

This is my Privoxy output with ProxHTTPSProxyMII 1.2:

Code:

2015-01-04 10:24:07.192 0000135c Request: www.blogger.com/dyn-css/authorization.css?targetBlogID=14267103&zx=96fd4534-c0b9-42d2-acbc-ccfe659f6435
2015-01-04 10:24:07.472 0000135c Re-Filter: Need to de-chunk first
2015-01-04 10:24:07.472 0000135c Re-Filter: De-chunking successful. Shrunk from 32 to 21
2015-01-04 10:24:07.485 0000135c Re-Filter: Decompression successful. Old size: 21, new size: 1.
2015-01-04 10:24:07.485 0000135c Re-Filter: filtering www.blogger.com/dyn-css/authorization.css?targetBlogID=14267103&zx=96fd4534-c0b9-42d2-acbc-ccfe659f6435 (size 1) with '123' produced 0 hits (new size 1).

It's the Privoxy who does the decompress. It doesn't seem to be a ProxHTTPSProxyMII problem.

The strange thing is if I change to ProxHTTPSProxyMII 1.1 which doesn't stream, Privoxy got more bytes.

Code:

2015-01-04 10:39:44.681 00001b14 Request: www.blogger.com/dyn-css/authorization.css?targetBlogID=14267103&zx=96fd4534-c0b9-42d2-acbc-ccfe659f6435
2015-01-04 10:39:44.985 00001b14 Re-Filter: Decompression successful. Old size: 250, new size: 297.
2015-01-04 10:39:44.986 00001b14 Re-Filter: filtering www.blogger.com/dyn-css/authorization.css?targetBlogID=14267103&zx=96fd4534-c0b9-42d2-acbc-ccfe659f6435 (size 297) with '123' produced 0 hits (new size 297).

I think you can fix that problem by using the old method (fully download and cache and then forward to browser/proxy server) for text file, clearly we can use mimetype (text/css, text/html) to detect text file or just use RegEx something like /.*?(?:\.css|\.js). But Mimetype always better, you can detect Mimetype using Content-Type header or maybe Python will have better method.

That problem is serious because it can make page stop loading, but seem it apply only to css, js, html, text file in general but not video, swf so I think will need a fix.

Also I hope we will have MimeType support for [BYPASS URL], I think better then use Mimetype video/ instead *.mp4, *.avi.. and much reliable.


And can you add a option to disable logging ? I think disable logging will improve performance.

Yeah, I think Privoxy report more bytes because the file get decompressed, so I think no problem. Gzip is a compression method like rar, zip, so probably after content get decompressed it must increase file size.