client

Restful json clients.

Use Resource for a connection to a single host. Use Pool for persistent thread-safe connections to a single host. Use Resources for multiple hosts with simple partitioning or replication. Use Shards for horizontally partitioning hosts by different keys. Use Replicas in coordination with automatic host synchronization.

Resources optionally reuse connections, handling request timeouts. Broadcasting to multiple resources is parallelized with asynchronous requests and responses.

The load balancing strategy is randomized, biased by the number of cached connections available. This inherently provides limited failover support, but applications must still handle exceptions as desired. Replicas will automatically retry if host is unreachable.

Note

Caching more client connections than the backend server supports may cause blocking under load. CherryPy’s default thread pool is 10.

Response

class lupyne.client.Response(sock, debuglevel=0, strict=0, method=None, buffering=False)[source]

Bases: httplib.HTTPResponse

A completed response which handles json and caches its body.

status

HTTP status code

reason

HTTP status message

body

string of entire response body

time

server response time

__nonzero__()[source]

Return whether status is successful.

__call__()[source]

Return evaluated response body or raise exception.

Resource

class lupyne.client.Resource(host, port=None, strict=None, timeout=<object object>, source_address=None)[source]

Bases: httplib.HTTPConnection

Synchronous connection which handles json responses.

call(method, path, body=None, params=(), redirect=False)[source]

Send request and return completed response.

delete(path, **params)[source]

Return response body from DELETE request.

download(path, filename)[source]

Download response body from GET request to a file.

get(path, **params)[source]

Return response body from GET request.

getresponse(filename='')[source]

Return completed response, optionally write response body to a file.

multicall(*requests)[source]

Pipeline requests (method, path[, body]) and generate completed responses.

patch(path, body=None, **kwargs)[source]

Return response body from PATCH request.

post(path, body=None, **kwargs)[source]

Return response body from POST request.

put(path, body=None, **kwargs)[source]

Return response body from PUT request.

request(method, path, body=None)[source]

Send request after handling body and headers.

Pool

New in version 1.2.

class lupyne.client.Pool(host, limit=0)[source]

Bases: collections.deque

Thread-safe resource pool for one host.

call(method, path, body=None)[source]

Send request and return completed response.

resource_class

alias of Resource

stream(method, path, body=None)[source]

Generate resource, initial response, and final response while handling timeouts.

Resources

class lupyne.client.Resources(hosts, limit=0)[source]

Bases: dict

Thread-safe mapping of hosts to resource pools.

Parameters:
  • hosts – host[:port] strings
  • limit – maximum number of cached resources per host
broadcast(method, path, body=None, hosts=())[source]

Send requests and return responses from all hosts, optionally from given subset.

choice(hosts)[source]

Return chosen host according to priority.

priority(host)[source]

Return priority for host. None may be used to eliminate from consideration.

unicast(method, path, body=None, hosts=())[source]

Send request and return response from any host, optionally from given subset.

Shards

class lupyne.client.Shards(items=(), limit=0, **multimap)[source]

Bases: dict

Mapping of keys to host clusters, with associated resources.

Parameters:
  • items – host, key pairs
  • limit – maximum number of cached connections per host
  • multimap – mapping of hosts to multiple keys
resources

Resources mapping.

broadcast(key, method, path, body=None)[source]

Send requests and return responses from all hosts for corresponding key.

choice(hosts)

Return chosen host according to priority.

multicast(keys, method, path, body=None)[source]

Send requests and return responses from a minimal subset of hosts which cover all corresponding keys. Response overlap is possible depending on partitioning.

priority(hosts)[source]

Return combined priority for hosts.

unicast(key, method, path, body=None)[source]

Send request and return response from any host for corresponding key.

Replicas

New in version 1.2.

class lupyne.client.Replicas(hosts, limit=0)[source]

Bases: lupyne.client.Resources

Resources which failover assuming the hosts are being automatically synchronized. Writes are dispatched to the first host and sequentially failover. Reads are balanced among all remaining hosts.

call(method, path, body=None, params=(), retry=False)[source]

Send request and return completed response, even if hosts are unreachable.

Parameters:retry – optionally retry request on http errors as well, such as waiting for indexer promotion
delete(path, **params)

Return response body from DELETE request.

get(path, **params)

Return response body from GET request.

post(path, body=None, **kwargs)

Return response body from POST request.

put(path, body=None, **kwargs)

Return response body from PUT request.