bridgedb.https

bridgedb.https

• bridgedb.https
  • bridgedb.https.distributor
  • bridgedb.https.request
  • bridgedb.https.server

Servers for BridgeDB’s HTTPS bridge distributor.

bridgedb.https.distributor

A Distributor that hands out bridges through a web interface.

class HTTPSDistributor(totalSubrings, key, proxies=None, answerParameters=None)

Bases: bridgedb.distribute.Distributor

A Distributor that hands out bridges based on the IP address of an incoming request and the current time period.

Variables:

• proxies – All known proxies, which we treat differently. See :param:`proxies`.
• hashring – A hashring that assigns bridges to subrings with fixed proportions. Used to assign bridges into the subrings of this distributor.

Create a Distributor that decides which bridges to distribute based upon the client’s IP address and the current time.

Parameters:

• totalSubrings (int) – The number of subhashrings to group clients into. Note that if PROXY_LIST_FILES is set in bridgedb.conf, then the actual number of clusters is one higher than totalSubrings, because the set of all known open proxies is given its own subhashring.
• key (bytes) – The master HMAC key for this distributor. All added bridges are HMACed with this key in order to place them into the hashrings.
• proxies (ProxySet) – A bridgedb.proxy.ProxySet containing known Tor Exit relays and other known proxies. These will constitute the extra cluster, and any client requesting bridges from one of these proxies will be distributed bridges from a separate subhashring that is specific to Tor/proxy users.
• answerParameters (bridgedb.Bridges.BridgeRingParameters) – A mechanism for ensuring that the set of bridges that this distributor answers a client with fit certain parameters, i.e. that an answer has “at least two obfsproxy bridges” or “at least one bridge on port 443”, etc.

bridgesPerResponse(hashring=None)

classmethod getSubnet(ip, usingProxy=False, proxySubnets=4)

Map all clients whose **ip**s are within the same subnet to the same arbitrary string.

Hint

For non-proxy IP addresses, any two IPv4 addresses within the same /16 subnet, or any two IPv6 addresses in the same /32 subnet, will get the same string.

Subnets for this distributor are grouped into the number of rings specified by the N_IP_CLUSTERS configuration option, such that Alice (with the address 1.2.3.4 and Bob (with the address 1.2.178.234) are placed within the same cluster, but Carol (with address 1.3.11.33) might end up in a different cluster.

>>> from bridgedb.https.distributor import HTTPSDistributor
>>> HTTPSDistributor.getSubnet('1.2.3.4')
'1.2.0.0/16'
>>> HTTPSDistributor.getSubnet('1.2.211.154')
'1.2.0.0/16'
>>> HTTPSDistributor.getSubnet('2001:f::bc1:b13:2808')
'2001:f::/32'
>>> HTTPSDistributor.getSubnet('2a00:c98:2030:a020:2::42')
'2a00:c98::/32'

Parameters:

• ip (str) – A string representing an IPv4 or IPv6 address.
• usingProxy (bool) – Set to True if the client was using one of the known proxies.
• proxySubnets (int) – Place Tor/proxy users into this number of “subnet” groups. This means that no matter how many different Tor Exits or proxies a client uses, the most they can ever get is proxySubnets different sets of bridge lines (per interval). This parameter only has any effect when usingProxy is True.

Return type:

str

Returns:

The appropriately sized CIDR subnet representation of the ip.

mapSubnetToSubring(subnet, usingProxy=False)

Determine the correct subhashring for a client, based upon the subnet.

Parameters:

• subnet (str) – The subnet which contains the client’s IP. See :staticmethod:`getSubnet`.
• usingProxy (bool) – Set to True if the client was using one of the known proxies.

mapClientToHashringPosition(interval, subnet)

Map the client to a position on a (sub)hashring, based upon the interval which the client’s request occurred within, as well as the subnet of the client’s IP address.

Note

For an explanation of how subnet is determined, see :staticmethod:`getSubnet`.

Parameters:

• interval (str) – The interval which this client’s request for bridges took place within.
• subnet (str) – A string representing the subnet containing the client’s IP address.

Return type:

int

Returns:

The results of keyed HMAC, which should determine the client’s position in a (sub)hashring of bridges (and thus determine which bridges they receive).

prepopulateRings()

Prepopulate this distributor’s hashrings and subhashrings with bridges.

The hashring structure for this distributor is influenced by the N_IP_CLUSTERS configuration option, as well as the number of PROXY_LIST_FILES.

Essentially, totalSubrings is set to the specified N_IP_CLUSTERS. All of the PROXY_LIST_FILES, plus the list of Tor Exit relays (downloaded into memory with :script:`get-tor-exits`), are stored in proxies, and the latter is added as an additional cluster (such that totalSubrings becomes N_IP_CLUSTERS + 1). The number of subhashrings which this Distributor has active in its hashring is then totalSubrings, where the last cluster is reserved for all proxies.

As an example, if BridgeDB was configured with N_IP_CLUSTERS=4 and PROXY_LIST_FILES=["open-socks-proxies.txt"], then the total number of subhashrings is five — four for the “clusters”, and one for the proxies. Thus, the resulting hashring-subhashring structure would look like:

	Directly connecting users				Tor / known proxy users
Clusters	Cluster-1	Cluster-2	Cluster-3	Cluster-4	Cluster-5
Subhashrings (total, assigned)	(5,1)	(5,2)	(5,3)	(5,4)	(5,5)
Filtered Subhashrings bBy requested bridge type)	(5,1)-IPv4	(5,2)-IPv4	(5,3)-IPv4	(5,4)-IPv4	(5,5)-IPv4
	(5,1)-IPv6	(5,2)-IPv6	(5,3)-IPv6	(5,4)-IPv6	(5,5)-IPv6

The “filtered subhashrings” are essentially filtered copies of their respective subhashring, such that they only contain bridges which support IPv4 or IPv6, respectively. Additionally, the contents of (5,1)-IPv4 and (5,1)-IPv6 sets are not disjoint.

Thus, in this example, we end up with 10 total subhashrings.

insert(bridge)

Assign a bridge to this distributor.

getBridges(bridgeRequest, interval)

Return a list of bridges to give to a user.

Parameters:

• bridgeRequest (bridgedb.https.request.HTTPSBridgeRequest) – A BridgeRequestBase with the client attribute set to a string containing the client’s IP address.
• interval (str) – The time period when we got this request. This can be any string, so long as it changes with every period.

Return type:

list

Returns:

A list of getBridgeRequestAnswer for an example of how this is used.

bridgedb.https.request

Classes for parsing and storing information about requests for bridges which are sent to the HTTPS distributor.

bridgedb.https.request
 |
 |_ HTTPSBridgeRequest - A request for bridges which was received through
                         the HTTPS distributor.

TRANSPORT_REGEXP = u'[_a-zA-Z][_a-zA-Z0-9]*'

A regular expression for matching the Pluggable Transport methodname in HTTP GET request parameters.

class HTTPSBridgeRequest(addClientCountryCode=True)

Bases: bridgedb.bridgerequest.BridgeRequestBase

We received a request for bridges through the HTTPS distributor.

Process a new bridge request received through the HTTPSDistributor.

Parameters:addClientCountryCode (bool) – If True, then calling withoutBlockInCountry() will attempt to add the client’s own country code, geolocated from her IP, to the notBlockedIn countries list.

withIPversion(parameters)

Determine if the request parameters were for bridges with IPv6 addresses or not.

Note

If there is an ipv6= parameter with anything non-zero after it, then we assume the client wanted IPv6 bridges.

Parameters:parameters – The twisted.web.http.Request.args.

withoutBlockInCountry(request)

Determine which countries the bridges for this request should not be blocked in.

Note

Currently, a request for unblocked bridges is recognized if it contains an HTTP GET parameter unblocked= whose value is a comma-separater list of two-letter country codes. Any two-letter country code found in the twisted.web.http.Request unblocked= HTTP GET parameter will be added to the notBlockedIn list.

If addClientCountryCode is True, the the client’s own geolocated country code will be added to the to the :data`notBlockedIn` list.

Parameters:request (twisted.web.http.Request) – A Request object containing the HTTP method, full URI, and any URL/POST arguments and headers present.

withPluggableTransportType(parameters)

This request included a specific Pluggable Transport identifier.

Add any Pluggable Transport methodname found in the HTTP GET parameters to the list of transports. Currently, a request for a transport is recognized if the request contains the 'transport=' parameter.

Parameters:parameters – The twisted.web.http.Request.args.

bridgedb.https.server

Servers which interface with clients and distribute bridges over HTTP(S).

getClientIP(request, useForwardedHeader=False)

Get the client’s IP address from the 'X-Forwarded-For:' header, or from the twisted.web.server.Request.

Parameters:

• request (twisted.web.http.Request) – A Request for a twisted.web.resource.Resource.
• useForwardedHeader (bool) – If True, attempt to get the client’s IP address from the 'X-Forwarded-For:' header.

Return type:

None or str

Returns:

The client’s IP address, if it was obtainable.

replaceErrorPage(error, template_name=None)

Create a general error page for displaying in place of tracebacks.

Log the error to BridgeDB’s logger, and then display a very plain “Sorry! Something went wrong!” page to the client.

Parameters:

• error (Exception) – Any exeption which has occurred while attempting to retrieve a template, render a page, or retrieve a resource.
• template_name (str) – A string describing which template/page/resource was being used when the exception occurred, i.e. 'index.html'.

Returns:

A string containing HTML to serve to the client (rather than serving a traceback).

class TranslatedTemplateResource(template=None)

Bases: twisted.web.resource.Resource

A generalised resource which uses gettext translations and Mako templates.

Create a new twisted.web.resource.Resource for a Mako-templated webpage.

isLeaf = True

render_GET(request)

render_POST(request)

class IndexResource

Bases: bridgedb.https.server.TranslatedTemplateResource

The parent resource of all other documents hosted by the webserver.

Create a twisted.web.resource.Resource for the index page.

class OptionsResource

Bases: bridgedb.https.server.TranslatedTemplateResource

A resource with additional options which a client may use to specify the which bridge types should be returned by BridgesResource.

Create a twisted.web.resource.Resource for the options page.

class HowtoResource

Bases: bridgedb.https.server.TranslatedTemplateResource

A resource which explains how to use bridges.

Create a twisted.web.resource.Resource for the HowTo page.

class CaptchaProtectedResource(publicKey=None, secretKey=None, useForwardedHeader=False, protectedResource=None)

Bases: twisted.web.resource.Resource

A general resource protected by some form of CAPTCHA.

isLeaf = True

getClientIP(request)

Get the client’s IP address from the 'X-Forwarded-For:' header, or from the twisted.web.server.Request.

Parameters:request (twisted.web.http.Request) – A Request for a twisted.web.resource.Resource. Return type:None or str Returns:The client’s IP address, if it was obtainable.

getCaptchaImage(request=None)

Get a CAPTCHA image.

Return type:tuple Returns:A 2-tuple of (image, challenge), where image is a binary, JPEG-encoded image, and challenge is a unique string. If unable to retrieve a CAPTCHA, returns a tuple containing two empty strings.

extractClientSolution(request)

Extract the client’s CAPTCHA solution from a POST request.

This is used after receiving a POST request from a client (which should contain their solution to the CAPTCHA), to extract the solution and challenge strings.

Parameters:request (twisted.web.http.Request) – A Request object for ‘bridges.html’. Returns:A redirect for a request for a new CAPTCHA if there was a problem. Otherwise, returns a 2-tuple of strings, the first is the client’s CAPTCHA solution from the text input area, and the second is the challenge string.

checkSolution(request)

Override this method to check a client’s CAPTCHA solution.

Return type:bool Returns:True if the client correctly solved the CAPTCHA; False otherwise.

render_GET(request)

Retrieve a ReCaptcha from the API server and serve it to the client.

Parameters:request (twisted.web.http.Request) – A Request object for a page which should be protected by a CAPTCHA. Return type:str Returns:A rendered HTML page containing a ReCaptcha challenge image for the client to solve.

render_POST(request)

Process a client’s CAPTCHA solution.

If the client’s CAPTCHA solution is valid (according to checkSolution()), process and serve their original request. Otherwise, redirect them back to a new CAPTCHA page.

Parameters:request (twisted.web.http.Request) – A Request object, including POST arguments which should include two key/value pairs: one key being 'captcha_challenge_field', and the other, 'captcha_response_field'. These POST arguments should be obtained from render_GET(). Return type:str Returns:A rendered HTML page containing a ReCaptcha challenge image for the client to solve.

class GimpCaptchaProtectedResource(hmacKey=None, captchaDir='', **kwargs)

Bases: bridgedb.https.server.CaptchaProtectedResource

A web resource which uses a local cache of CAPTCHAs, generated with gimp-captcha, to protect another resource.

Protect a resource via this one, using a local CAPTCHA cache.

Parameters:

• secretkey (str) – A PKCS#1 OAEP-padded, private RSA key, used for verifying the client’s solution to the CAPTCHA. See bridgedb.crypto.getRSAKey() and the GIMP_CAPTCHA_RSA_KEYFILE config setting.
• publickey (str) – A PKCS#1 OAEP-padded, public RSA key, used for creating the captcha_challenge_field string to give to a client.
• hmacKey (bytes) – The master HMAC key, used for validating CAPTCHA challenge strings in captcha.GimpCaptcha.check(). The file where this key is stored can be set via the GIMP_CAPTCHA_HMAC_KEYFILE option in the config file.
• captchaDir (str) – The directory where the cached CAPTCHA images are stored. See the GIMP_CAPTCHA_DIR config setting.
• useForwardedHeader (bool) – If True, obtain the client’s IP address from the X-Forwarded-For HTTP header.
• protectedResource (twisted.web.resource.Resource) – The resource to serve if the client successfully passes the CAPTCHA challenge.

checkSolution(request)

Process a solved CAPTCHA via bridgedb.captcha.GimpCaptcha.check().

Parameters:request (twisted.web.http.Request) – A Request object, including POST arguments which should include two key/value pairs: one key being 'captcha_challenge_field', and the other, 'captcha_response_field'. These POST arguments should be obtained from render_GET(). Rtupe:bool Returns:True, if the CAPTCHA solution was valid; False otherwise.

getCaptchaImage(request)

Get a random CAPTCHA image from our captchaDir.

Creates a GimpCaptcha, and calls its get() method to return a random CAPTCHA and challenge string.

Parameters:request (twisted.web.http.Request) – A client’s initial request for some other resource which is protected by this one (i.e. protected by a CAPTCHA). Returns:A 2-tuple of (image, challenge), where:: - image is a string holding a binary, JPEG-encoded image. - challenge is a unique string associated with the request.

render_GET(request)

Get a random CAPTCHA from our local cache directory and serve it to the client.

Parameters:request (twisted.web.http.Request) – A Request object for a page which should be protected by a CAPTCHA. Return type:str Returns:A rendered HTML page containing a ReCaptcha challenge image for the client to solve.

render_POST(request)

Process a client’s CAPTCHA solution.

If the client’s CAPTCHA solution is valid (according to checkSolution()), process and serve their original request. Otherwise, redirect them back to a new CAPTCHA page.

Parameters:request (twisted.web.http.Request) – A Request object, including POST arguments which should include two key/value pairs: one key being 'captcha_challenge_field', and the other, 'captcha_response_field'. These POST arguments should be obtained from render_GET(). Return type:str Returns:A rendered HTML page containing a ReCaptcha challenge image for the client to solve.

class ReCaptchaProtectedResource(remoteIP=None, **kwargs)

Bases: bridgedb.https.server.CaptchaProtectedResource

A web resource which uses the reCaptcha service.

getCaptchaImage(request)

Get a CAPTCHA image from the remote reCaptcha server.

Parameters:request (twisted.web.http.Request) – A client’s initial request for some other resource which is protected by this one (i.e. protected by a CAPTCHA). Returns:A 2-tuple of (image, challenge), where:: - image is a string holding a binary, JPEG-encoded image. - challenge is a unique string associated with the request.

getRemoteIP()

Mask the client’s real IP address with a faked one.

The fake client IP address is sent to the reCaptcha server, and it is either the public IP address of bridges.torproject.org (if the config option RECAPTCHA_REMOTE_IP is configured), or a random IP.

Return type:str Returns:A fake IP address to report to the reCaptcha API server.

checkSolution(request)

Process a solved CAPTCHA by sending it to the ReCaptcha server.

The client’s IP address is not sent to the ReCaptcha server; instead, a completely random IP is generated and sent instead.

Parameters:request (twisted.web.http.Request) – A Request object, including POST arguments which should include two key/value pairs: one key being 'captcha_challenge_field', and the other, 'captcha_response_field'. These POST arguments should be obtained from render_GET(). Rtupe:twisted.internet.defer.Deferred Returns:A deferred which will callback with a tuple in the following form:

(:type:`bool`, twisted.web.server.Request)

If the CAPTCHA solution was valid, a tuple will contain::

(True, Request)

Otherwise, it will contain::

(False, Request)

render_GET(request)

Retrieve a ReCaptcha from the API server and serve it to the client.

Parameters:request (twisted.web.http.Request) – A Request object for ‘bridges.html’. Return type:str Returns:A rendered HTML page containing a ReCaptcha challenge image for the client to solve.

render_POST(request)

Process a client’s CAPTCHA solution.

If the client’s CAPTCHA solution is valid (according to checkSolution()), process and serve their original request. Otherwise, redirect them back to a new CAPTCHA page.

Parameters:request (twisted.web.http.Request) – A Request object, including POST arguments which should include two key/value pairs: one key being 'captcha_challenge_field', and the other, 'captcha_response_field'. These POST arguments should be obtained from render_GET(). Returns:twisted.web.server.NOT_DONE_YET, in order to handle the Deferred returned from checkSolution(). Eventually, when the Deferred request is done being processed, _renderDeferred() will handle rendering and displaying the HTML to the client.

class BridgesResource(distributor, schedule, N=1, useForwardedHeader=False, includeFingerprints=True)

Bases: twisted.web.resource.Resource

This resource displays bridge lines in response to a request.

Create a new resource for displaying bridges to a client.

Parameters:

• distributor (HTTPSDistributor) – The mechanism to retrieve bridges for this distributor.
• schedule (ScheduledInterval) – The time period used to tweak the bridge selection procedure.
• N (int) – The number of bridges to hand out per query.
• useForwardedHeader (bool) – Whether or not we should use the the X-Forwarded-For header instead of the source IP address.
• includeFingerprints (bool) – Do we include the bridge’s fingerprint in the response?

isLeaf = True

render(request)

Render a response for a client HTTP request.

Presently, this method merely wraps getBridgeRequestAnswer() to catch any unhandled exceptions which occur (otherwise the server will display the traceback to the client). If an unhandled exception does occur, the client will be served the default “No bridges currently available” HTML response page.

Parameters:request (twisted.web.http.Request) – A Request object containing the HTTP method, full URI, and any URL/POST arguments and headers present. Return type:str Returns:A plaintext or HTML response to serve.

getClientIP(request)

Get the client’s IP address from the 'X-Forwarded-For:' header, or from the twisted.web.server.Request.

Parameters:request (twisted.web.http.Request) – A Request object for a twisted.web.resource.Resource. Return type:None or str Returns:The client’s IP address, if it was obtainable.

getBridgeRequestAnswer(request)

Respond to a client HTTP request for bridges.

Parameters:request (twisted.web.http.Request) – A Request object containing the HTTP method, full URI, and any URL/POST arguments and headers present. Return type:str Returns:A plaintext or HTML response to serve.

getResponseFormat(request)

Determine the requested format for the response.

Parameters:request (twisted.web.http.Request) – A Request object containing the HTTP method, full URI, and any URL/POST arguments and headers present. Return type:None or str Returns:The argument of the first occurence of the format= HTTP GET parameter, if any were present. (The only one which currently has any effect is format=plain, see note in renderAnswer().) Otherwise, returns None.

renderAnswer(request, bridgeLines=None)

Generate a response for a client which includes bridgesLines.

Parameters:

• request (twisted.web.http.Request) – A Request object containing the HTTP method, full URI, and any URL/POST arguments and headers present.
• bridgeLines (list or None) – A list of strings used to configure a Tor client to use a bridge. If None, then the returned page will instead explain that there were no bridges of the type they requested, with instructions on how to proceed.

Return type:

str

Returns:

A plaintext or HTML response to serve.

addWebServer(config, distributor)

Set up a web server for HTTP(S)-based bridge distribution.

Parameters:

• config (bridgedb.persistent.Conf) –

  A configuration object from bridgedb.Main. Currently, we use these options:

  HTTP_UNENCRYPTED_PORT
  HTTP_UNENCRYPTED_BIND_IP
  HTTP_USE_IP_FROM_FORWARDED_HEADER
  HTTPS_N_BRIDGES_PER_ANSWER
  HTTPS_INCLUDE_FINGERPRINTS
  HTTPS_KEY_FILE
  HTTPS_CERT_FILE
  HTTPS_PORT
  HTTPS_BIND_IP
  HTTPS_USE_IP_FROM_FORWARDED_HEADER
  HTTPS_ROTATION_PERIOD
  RECAPTCHA_ENABLED
  RECAPTCHA_PUB_KEY
  RECAPTCHA_SEC_KEY
  RECAPTCHA_REMOTEIP
  GIMP_CAPTCHA_ENABLED
  GIMP_CAPTCHA_DIR
  GIMP_CAPTCHA_HMAC_KEYFILE
  GIMP_CAPTCHA_RSA_KEYFILE
  

• distributor (bridgedb.https.distributor.HTTPSDistributor) – A bridge distributor.

Raises SystemExit:  

if the servers cannot be started.

Return type:

twisted.web.server.Site

Returns:

A webserver.