ProxHTTPSProxyMII: Development

[vlad_s]

After the changes in ProxyTool.py, the above listed sites work without changes. Does Privoxy support WebSocket? The developer’s response is rather vague. Here is what is recorded in the Privoxy log:

Code:

...
2019-07-23 20:14:15.328 7fe55effd700 Header: scan: GET http://tjournal.ru/chan/system HTTP/1.1
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Accept: */*
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Host: tjournal.ru
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:69.0) Gecko/20100101 Firefox/69.0
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Accept-Encoding: gzip, deflate, br
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Sec-WebSocket-Version: 13
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Origin: https://tjournal.ru
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Sec-WebSocket-Extensions: permessage-deflate
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Sec-WebSocket-Key: xLQdEBcAZbVdlHZTmiHG4g==
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Connection: keep-alive, Upgrade
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Cookie: pushVisitsCount=45;fingerprint=f794e8cfdfc384g67de; adblock-state=1; audio_
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Pragma: no-cache
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Cache-Control: no-cache
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Upgrade: websocket
2019-07-23 20:14:15.329 7fe55effd700 Header: scan: Tagged: ProxHTTPSProxyMII FrontProxy/v1.5 Python/3.7.3:397
2019-07-23 20:14:15.329 7fe56d32f700 Header: scan: GET http://static-osnova.gcdn.co/static/build/tjournal.ru/dropdown.7ed182cc2aa8236bff77.min.js HT
2019-07-23 20:14:15.329 7fe56d32f700 Header: scan: Accept: */*
2019-07-23 20:14:15.329 7fe56d32f700 Header: scan: Host: static-osnova.gcdn.co
...
2019-07-23 20:14:15.360 7fe55effd700 Header: Keeping the client header 'Connection: keep-alive, Upgrade' around. The server connection will be kept alive if possible.
...
2019-07-23 20:14:15.400 7fe55effd700 Header: scan: HTTP/1.1 101 Switching Protocols
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Server: nginx
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Date: Tue, 23 Jul 2019 17:14:15 GMT
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Content-Length: 0
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Connection: upgrade
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Sec-WebSocket-Extensions: permessage-deflate
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Access-Control-Allow-Credentials: true
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Access-Control-Allow-Origin: https://tjournal.ru
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Sec-WebSocket-Accept: sGlz46w7kC9gyLQiIHLN6IA81u4=
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Upgrade: websocket
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Strict-Transport-Security: max-age=31536000; preload
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: X-XSS-Protection: 1; mode=block
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: X-Content-Type-Options: nosniff
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Referrer-Policy: origin
2019-07-23 20:14:15.401 7fe55effd700 Header: scan: Content-Security-Policy: default-src * data: 'unsafe-eval' 'unsafe-inline'
2019-07-23 20:14:15.406 7fe55effd700 Header: Adding: Proxy-Connection: keep-alive
...

What I changed in ProxyTool.py to avoid misunderstandings was:

Code:

except (ssl.SSLEOFError, ssl.SSLError) as e:
            logger.error(Fore.RED + Style.BRIGHT + "[SSLError]")
            self.send_error(417, message="Exception %s" % str(e.__class__), explain=str(e))

    def purge_headers(self, headers):
        "Remove hop-by-hop headers that shouldn't pass through a Proxy"
        for name in ["Connection", "Keep-Alive", "Upgrade",
                     "Proxy-Connection", "Proxy-Authenticate"]:
            del headers[name]

    def purge_write_headers(self, headers):
        self.purge_headers(headers)
        for key, value in headers.items():
            self.send_header(key, value)
        self.end_headers()

changes:

Code:

except (ssl.SSLEOFError, ssl.SSLError) as e:
            logger.error(Fore.RED + Style.BRIGHT + "[SSLError]")
            self.send_error(417, message="Exception %s" % str(e.__class__), explain=str(e))

    def purge_headers(self, headers):
        "Remove hop-by-hop headers that shouldn't pass through a Proxy"
        for name in [""]:
            del headers[name]

    def purge_write_headers(self, headers):
        self.purge_headers(headers)
        for key, value in headers.items():
            self.send_header(key, value)
        self.end_headers()

235 line.