rpc Package

rpc Package

spartan.rpc.connect(host, port)[source]
spartan.rpc.listen(host, port)[source]

common Module

Simple RPC library.

The Client and Server classes here work with sockets which should implement the Socket interface.

class spartan.rpc.common.Client(socket)[source]

Bases: object

addr()[source]
close()[source]
handle_read(socket)[source]
class spartan.rpc.common.DummyFuture(base=None)[source]

Bases: object

wait()[source]
class spartan.rpc.common.FnFuture(future, fn)[source]

Bases: object

Chain fn to the given future.

self.wait() return fn(future.wait()).

wait()[source]
class spartan.rpc.common.Future(addr, rpc_id)[source]

Bases: object

on_finished(fn)[source]
timed_out()[source]
wait()[source]
class spartan.rpc.common.FutureGroup[source]

Bases: list

wait()[source]
class spartan.rpc.common.Group[source]

Bases: tuple

class spartan.rpc.common.PendingRequest(socket, rpc_id)[source]

Bases: object

An outstanding RPC request.

Call done(result) when a method is finished processing.

done(result=None)[source]
wait()[source]
class spartan.rpc.common.PickledData(*args, **kw)[source]

Bases: spartan.node.Node

Helper class: indicates that this message has already been pickled, and should be sent as is, rather than being re-pickled.

class spartan.rpc.common.ProxyMethod(client, method)[source]

Bases: object

class spartan.rpc.common.RPCException(*args, **kw)[source]

Bases: spartan.node.Node

exception spartan.rpc.common.RemoteException(tb)[source]

Bases: exceptions.Exception

Wrap a uncaught remote exception.

class spartan.rpc.common.Server(socket)[source]

Bases: object

addr[source]
handle_read(socket)[source]
register_method(name, fn)[source]
register_object(obj)[source]
serve()[source]
serve_nonblock()[source]
shutdown()[source]
class spartan.rpc.common.SocketBase[source]

Bases: object

bind()[source]
close()[source]
connect()[source]
flush()[source]
recv()[source]
register_handler(handler)[source]

A handler() is called in response to read requests.

send(blob)[source]
spartan.rpc.common.capture_exception(exc_info=None)[source]
spartan.rpc.common.forall(clients, method, request)[source]

Invoke method with request for each client in clients

request is only serialized once, so this is more efficient when targeting multiple workers with the same data.

Returns a future wrapping all of the requests.

spartan.rpc.common.pickle(obj)[source]
spartan.rpc.common.pickle_to(obj, writer)[source]
spartan.rpc.common.set_default_timeout(seconds)[source]
spartan.rpc.common.wait_for_all(futures)[source]

core Module

msg Module

Message definitions for RPC services.

class spartan.rpc.msg.DelTable(name)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Empty

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Entry(key, value)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Exception(py_exc)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.GetRequest(table_id, shard_id, keys=None, whole_shard=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.GetResponse(entries=None, shard_data=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.IteratorRequest(table_id, shard_id, iter_id=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.IteratorResponse(iter_id, entries=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.KernelRequest(slices=None, kernel=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.KernelResponse(results=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Message

Bases: object

copy()
static decode(reader)
encode(f=None)
class spartan.rpc.msg.Ping(ping)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Pong(pong)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Reduction

Bases: object

class spartan.rpc.msg.TableDescriptor(name, num_shards, workers=None)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.Update(key, op, value)

Bases: spartan.rpc.msg.Message

class spartan.rpc.msg.UpdateRequest(table_id, shard_id, updates=None, shard_data=None)

Bases: spartan.rpc.msg.Message

spartan.rpc.msg.read(f)
spartan.rpc.msg.write(obj, f)

zeromq Module

ZeroMQ socket implementation.

class spartan.rpc.zeromq.ServerSocket(ctx, sock_type, hostport)[source]

Bases: spartan.rpc.zeromq.Socket

bind()[source]
handle_read(socket)[source]
send(msg)[source]

Send msg to a remote client.

Parameters:msgGroup, with the first element being the destination to send to.
zmq()[source]
class spartan.rpc.zeromq.Socket(ctx, sock_type, hostport)[source]

Bases: spartan.rpc.common.SocketBase

close(*args)[source]
connect()[source]
flush()[source]
handle_close()[source]
handle_read(socket)[source]
handle_write()[source]
host[source]
in_poll_loop()[source]
port[source]
recv()[source]
send(msg)[source]
zmq()[source]
class spartan.rpc.zeromq.StubSocket(source, socket, data)[source]

Bases: spartan.rpc.common.SocketBase

Handles a single read from a client

addr[source]
recv()[source]
send(req)[source]
class spartan.rpc.zeromq.ZMQPoller[source]

Bases: threading.Thread

add(socket, direction)[source]
close(socket)[source]

Execute socket.handle_close() from within the polling thread.

modify(socket, direction)[source]
remove(socket)[source]
stop()[source]
wakeup()[source]
spartan.rpc.zeromq.client_socket(addr)[source]
spartan.rpc.zeromq.poller()[source]
spartan.rpc.zeromq.server_socket(addr)[source]
spartan.rpc.zeromq.shutdown()[source]