7. Features and Guidelines

This chapter discusses some of the more technical features of Pyro, that don't fit in elsewhere. Also it provides some guidelines you must follow when developing with Pyro.

Nested attribute access

As we've already seen in the Pyro Concepts chapter, there is a special Dynamic Proxy that allows direct remote attribute access on Pyro objects. You can use normal Python syntax for this (for instance, print RemoteObj.name prints the attribute 'name' of the remote object 'RemoteObj'). There are a few important points to keep in mind when you're using this: For a better understanding why these rules have to be followed, I'll try to explain exactly what happens when accessing attributes on a Pyro proxy:

When you call object.sub1.sub2 where object is a Pyro proxy to your Pyro server object, Python processes the statement left-to-right. First Python wants to get the sub1 attribute from object. So Pyro intercepts a getattr call from Python and the remote Pyro object returns the sub1 attribute. Unfortunately, this value is an object (i.e. a class instance)! The client receives a class instance of a class it doesn't know about! (that's the reason it crashes with AttributeError: 'module' object has no attribute 'SomeClass', when the class of the attribute object is not available on the client).

The easiest way to solve this is to add a delegation method to your Pyro object (something like getSub2() that returns self.sub1.sub2. Then, don't access the object.sub1.sub2 attribute directly, but get it with object.getSub2(). The other solution is to make the class available to the client process: put it in a separate module and place the module somewhere the client code has access to (i.e. it must be able to import it). An explicit import statement is not necessary.

But there is a serious, confusing issue with nested attributes: you will have a local object instead of the remote object! What's going on: Say you took care of supplying all needed modules and classes to both server and client. Then you want to access object.sub1.sub2. What you get is the sub2 attribute of a LOCAL sub1 object instance! This is no problem when you just want to read the value, but it is a huge problem when you want to perform some actions on this attribute, or change the value! This will be performed in your client, not in the server object!

Explanation: as pointed out above, Python searches for sub1 in object. Because Pyro intercepts the getattr call, you get the remote sub1 object. Its state is pickled on the server (by Pyro) and unpickled on the client (by Pyro), thereby recreating the remote sub1 object on the client. Only after that, Python looks up sub2 in sub1. What happens is that Python itself returns you the sub2 attribute of the local sub1 object! If you call methods on that object, or change it's value, that will only happen on the (temporary) local sub2! All changes are lost because they're not propagated to the server!

Currently the only working solution for this problem is to use extra methods on your original Pyro server object, that manipulate the sub2 object on the server (for instance, you should call object.setSub2(newvalue) to update sub2's value on the server, instead of object.sub1.sub2=newvalue).

Note that this problem only occurs when you're using nested attribute access. Manipulating the attributes of object itself (the Pyro proxy) is no problem, because all attribute accesses (read and write) are intercepted by Pyro and are remoted to the actual server object.

Automatic rebinding

Pyro has an "auto rebind" feature. This means that - with a little help of yourself - your clients can recover from network errors that kill the connection with the server. More specifically, when your client detects a problem with the network (ConnectionClosedError or even ProtocolError) it can use a special 'hidden' method of the internal Pyro protocol adapter object to request it to reconnect to the server:
yourpyroobject.adapter.rebindURI()
You can supply two arguments if desired, tries and wait. The first is the number of retries that is performed (default: sys.maxint), the second is the delay time between each retry in seconds (default: 1).
NOTES:
  1. If your server crashes, and you want to restart it and let the clients reconnect to it, the server has to be prepared for this feature. It must not rely on any transient internal state to function correctly, because that state is lost when your server is restarted.
  2. Your server must register the objects with the connectPersistent method of the PyroDaemon. Otherwise the reconnect feature does not work because the client tries to access an invalid URI. The connectPersistent method reuses the object's URI that is still known in the Name Server. It's no problem to always use connectPersistent instead of the regular connect, but there is a naming lookup overhead per call.
  3. The NS does not have to be the persistent version on disk, as long as it keeps running. All that is needed is that the URI of the objects concerned stay available in the NS, so the server can reuse those URIs when it comes up again. When the NS dies, you're in deep trouble, unless it was the persistent NS, that reloads its naming database when it comes up again.
  4. The client is responsible for detecting a network problem itself. It must call the 'hidden' rebindURI method of the object's protocol adapter itself if this is the case.
  5. The method call that triggered the auto rebind is likely lost, i.e. it almost certainly has not been executed. You have to call it again explicitly if you want to be sure that it has been executed. However, it might be the case that the server received the call and that the connection was lost after the server executed the call but before returning the result. Calling the method again will result in two executions on the server. This is all usual transaction semantics stuff, and Pyro effectively gives you at-most-once semantics (methods are called zero or one times in case of failure).
  6. If a daemon goes down, automatic rebinding only works if the daemon is started again on the same port number. This may be an issue if you start multiple daemons while relying on Pyro's automatic port selection feature. Start the daemons with an explicit port parameter to make sure they reuse the same port every time. The reason this is needed is that the rebinding of the proxy will use the PYRO:// uri that is stored in the proxy. That URI only contains the exact location of the object in a specific daemon. No new naming lookup via the name server is done.
You're probably wondering "Why isn't this transparent? Why no PYRO_AUTOREBIND config item?" The answer is: because you have to have control about it when a network problem occurs. Furthermore, only you can decide if your system needs this feature, and if your system can support this feature (see points 1 and 2 above).

About the difference between the exceptions that you can catch: You get a ConnectionClosedError when Pyro was able to establish a connection initially, but the connection gets interrupted later (because of network problems or something like that). You get a ProtocolError ('connection failed') when Pyro was unable to establish a connection at all, for instance, when the Pyro daemon on the server isn't running or reachable at the time Pyro tries to establish a connection. It is debatable if we should talk about reconnection in the second case, because there has never been a connection to begin with. But, if you want to handle this case too, just catch a ProtocolError instead of ConnectionClosedError.

Examine the example code provided in the "autoreconnect" example, and Pyro's internal adapter code, to see how the rebind feature works.

Mobile code

Pyro supports the concept of mobile code (albeit with a few limitations). What does this mean? Imagine a Pyro object that accepts other objects as arguments in its methods. It will invoke various methods on these objects as they were passed in from the client. The client can pass any object as an argument to a remote method call. What happens on the server is that as soon as the method call arrives, Pyro needs the Python module that contains the code for the objects that were passed in. If this module is not available, you'd normally get an ImportError. However, Pyro intercepts this and returns a NoModuleError.

Unless you enable PYRO_MOBILE_CODE. Now, Pyro internally returns a special code, which makes the client submit the missing Python code to the server. Pyro then proceeds as if nothing was wrong, i.e. it can now load the previously missing module and call those objects! This happens automatically, you only have to enable mobile code on the server by setting the PYRO_MOBILE_CODE config item to 1!

There is one more thing: loading and running arbitrary code is dangerous. Pyro has codeValidators to help you protect your program; see the Security chapter for more info.

Pyro supports 2-way mobile code: your client can also receive code from the server that wasn't available before! This means the server can return objects that are unknown in the client. The module(s) on the server that define those objects will be sent to the client to make them available there, too. Just like enabling mobile code support on the server, you have to enable PYRO_MOBILE_CODE on the client to allow mobile code downloading from the server to the client. The codeValidator on the server checks both directions.

There are a few important limitations with the current mobile code implementation:

It is perfectly ok to put your mobile object module in a Python package. If your codeValidator allows it, your mobile object module can also import other modules that are unknown on the server (but only from within the mobile object class members, not at the module level or from __init__). They will be transported too (if you import them by their fully qualified name). It's easy enough to write an appropriate codevalidator. If you don't want cascaded loading, check for specific module names. If you want to allow cascaded loading, check for module name patterns, for instance allow everything that starts with "agent.". Have a look at the "agent2" example to see how this all works.

Note: if a compiled module (*.pyc or *.pyo) is available, Pyro will use that. If you have old *.pyc files around made with a different Python version, Pyro will crash (with a syntax error) because it can't recognise these compiled files and tries to compile them! Be sure to delete all *.pyc and *.pyo files if you switch Python versions.

Multithreading in Pyro

Pyro has multithreading support in Pyro servers. It is turned on by default. This means that you can have a Pyro server that processes multiple remote object invocations in parallel. This is useful in cases where a single invocation takes a long time to complete. Without multithreading, the next invocation has to wait before your server is finished with the current one. With multithreading, each invocation runs in its own thread, and new invocations can be started while the others are still in progress.

Another case where multithreading is necessary, is when you are using callbacks in your Pyro object (or when you are calling other Pyro objects). Because this call may arrive at the same daemon, it must create a new thread to handle the new incoming call. When running in single threaded mode, Pyro will freeze because the calls are waiting on each other to be processed.

Pyro's use of threads on the server is as follows: the main daemon loop only waits for new connections. If a new connection arrives, a new thread is created that will serve this connection. The thread will process incoming calls sequentially, but that is good ofcourse since they can only arrive sequentially over the socket. The thread keeps running while the connection is still there. Be aware of this: if you ^C a program, you abort the main loop but many other threads might still be running. Pyro makes them 'daemon threads' so they will be terminated when your program exits, but it is preferred to clean up the nice way: call daemon.shutdown() when your program exits.

Of course, a little overhead is introduced. You can see this quite clearly when you are running the "benchmark" example in single- and multithreaded mode. Pyro will default to the multithreaded mode if your system supports it, because usually you'll need Pyro to be able to accept new invocations while others are still in progress. If you want, use the PYRO_MULTITHREADED config item to switch to singlethreaded mode (set it to zero). Pyro will default to single threaded mode if Python threads are not available. Switching to multithreaded mode if Python threads are not available, by setting PYRO_MULTITHREADED to 1, will crash Pyro. Please use Pyro.util.supports_multithreading() to test whether or not your system is able to use multithreading.

Concurrent method invocations, or perhaps not: Pyro.core.SynchronizedObjBase

Be aware that your Pyro objects can be accessed concurrently, i.e. method calls may occur simultaneously from different threads. It can be hard in some cases to create a program that behaves correctly with this. For instance, often a shared data structure may only be accessed by a single thread at the same time. To make it easier, there is the special Pyro.core.SynchronizedObjBase base class that you can use instead of the regular Pyro.core.ObjBase. When you use it, all (remote) method calls are automatically synchronized for you, because a thread lock object is used for your Pyro object. This has a huge negative impact on heavily multithreaded applications, but it saves you from much threading headaches. Note that other Pyro objects may still be accessed concurrently. If you share data over different Pyro objects, you still have to make sure that everything behaves in a thread-safe manner. Note that when you use this object base class, you don't have most of the threading issues mentioned above regarding further calls on proxies stored in your Pyro object, because you are running in a single thread all the time. However, the thread is a different one every call so you still have to transfer thread ownership on the proxy object!

Transferring proxies over remote calls

It's possible to pass proxies as parameters to remote calls. They will be pickled and unpickled in the Pyro object on the other side. Notice that the thread in which the call is executing on the remote side, is automatically taking ownership of the unpickled proxy object. That means that you don't have to worry about any threading issues (mentioned above) when you are using a proxy that you obtained via method parameters! It could be a good idea to just pass callback objects via method parameters on each call, instead of having the server store a callback object reference and the need to deal with all the gory threading details described above.

So when to use multithreading or not?

There are four situations where you don't want multithreading: All other cases will likely benefit from a multithreaded server, or even require one (callbacks/calling other Pyro objects!). Do some tests to find out what suits your needs better in your specific situation!

The "multithread" example shows what's going on, and how multithreading can help to improve performance and response times.

Thread Local Storage in Pyro objects

Sometimes it is required to keep track of some data that should only be used by a single thread. An example could be an open database connection object, because the connection isn't thread safe, or it needs this because of transactions, or some other reason. If you don't want to re-establish the connection for every call you can store it on what is called the Thread Local Storage.

TLS is a container that is local to the current thread. Other threads have no access to that data, and you don't overwrite global data by accident when you use this kind of storage. Pyro's TLS can be accessed from within the remote methods of your Pyro objects ( it exists only when your object is invoked by Pyro, so you cannot access it from your __init__ method for instance).

How do you gain access to the TLS? The ObjBase base class has a method getLocalStorage() that returns an instance of a storage class to put your data in. So you write something like this, to store something on it:

  	 self.getLocalStorage().something=...
  
The TLS object already has one predefined attribute: caller. This attribute contains the Pyro.protocol.TCPConnection object of the client proxy that is performing the current method call. You could use it to identifiy the unique client that is making the method call, but remember that it is just a single proxy that you are pinpointing. Your client application could have many proxies, even many different ones that connect to the same server. It is advised to leave the caller object alone, but you could do nasty things with this if you want to (such as dropping the connection by calling close() on it). You might also use the caller object to access any specific data associated with the connection, that you have stored there yourself (for instance, the authentication id placed as an attribute on it from within a custom connection validator). See the "user_passwd_auth" example. It puts it there to be able to uniquely identify the client, rather than just the calling proxy.

How do you initialize the TLS? Pyro allocates the TLS as soon as it's needed. You cannot initialize it when your objects are initialized because it doesn't exist yet at that time. Instead, you have to set a custom init function in the Daemon using the setInitTLS(initfunc) method of the Daemon. initfunc must be a callable object that takes a single argument: the TLS object to initialize.

For example, this function can make sure that several attributes are created on the TLS object that your methods assume to be there. If you would not do this, you need a check in all your methods for the existence of those attributes, creating them when not yet available, and so on.

Note: When running Pyro in singlethreaded mode the TLS is shared across every Pyro object and every method invocation! This is different than normal, when running in multithreaded mode!

If you use delegation instead of inheriting from ObjBase, you cannot use TLS because you don't have a means of getting to it (the getter is in ObjBase). This isn't bad because objects that use delegation know nothing about Pyro in the first place. (Well, actually, there is a sneaky way of getting to the TLS: use threading.currentThread().localStorage (it's an attribute of the current thread object). But if you do use this, be aware that your delegate object becomes dependent on the Pyro environment, and you probably chose to use delegation approach to avoid this!) Also, in a single-threaded environment, this is not possible because the threading.currentThread() call might not be available. In any case, when Pyro is running in single threaded mode, the current thread (or main thread) does not contain a localStorage attribute. To avoid any problems, it is probably best not to use it at all. Remember that you can use self.getLocalStorage() fine from a Pyro object that is inherited from Pyro.core.ObjBase, even in single threaded mode!

Examples Have a look at the 'user_passwd_auth' and 'sessions' examples to see possible usages of the TLS.

DNS, IP addresses and firewalls

There are some issues to be aware of, depending on your network configuration.

Usually Pyro will locate its servers and objects using fixed IP numbers encoded in the Pyro URIs. This may or may not be appropriate. For instance, when a machine has an IP number that is only temporary, such as DHCP-leased IP numbers. The machine can get a new -different- IP number while Pyro is still running. URIs with the old IP number are now invalid! Therefore it is possible to tell Pyro to use symbolic DNS hostnames in URIs instead of raw IP numbers. Pyro will then use DNS to look up the actual IP number of the specified host (by name). You can enable this by setting the PYRO_DNS_URI config item to 1. However note that each time Pyro has to look up a host, there is a DNS lookup delay.

If your machine has multiple IP addresses (for instance, when it has multiple network cards), you have to decide on what IP address your Pyro servers reside. When you create a Pyro Daemon, use the host argument to specify the hostname/ip address to bind the server on (defaults to '' - the default host). The Name Server can be started with a -n hostname argument, to specify the hostname/ip address to bind on. You can also use the PYRO_HOST config item.

Another issue is when you're using Pyro behind a firewall. There is one specific form of firewalling that is addressed in Pyro: simple Network Address Translating firewalls using port forwarding. Let's say you're at 192.168.0.1 (a private address) behind a NAT gateway that's 192.1.1.1. You have port forwarding on the NAT, so that Pyro requests go to the private box. However, with the way that Pyro is set up by default, the daemon will publish URIs as though they come from 192.168.0.1 -- an address unavailable to the rest of the Net. There is no way to have 192.168.0.1 publish URIs as though it were actually 192.1.1.1. But there is the extra publishhost parameter for the constructor of Pyro.core.Daemon. When constructing a Pyro Daemon, you can give it a special hostname or IP address that it should use when publishing URIs, via the publishhost parameter. The host parameter still is the "real" hostname of the machine the daemon is running on. When publishhost is not specified (it isn't by default) the value for host is taken. If that isn't specified either (it isn't by default) the hostname of the machine is queried and that is used. In our little example, host should be 192.168.0.1 (or just empty/omitted) and publishhost should be 192.1.1.1, the address of the firewall/gateway. By the way, you can also use the PYRO_PUBLISHHOST config item to specify a publish hostname.

Callbacks and Oneway calls

Callbacks are method invocations that are 'reversed'-- the server calls your client. This is useful when the server is something that publishes information, for instance, where the client cannot or doesn't want to poll for new info. Instead, the server calls a method on a callback object on the client to let it know that something happened.

Your client must publish a callback object that is a true Pyro object. In fact, for the callback part, your client must act as a true Pyro server. So you have to code your client as both a client and a server. This may require that your client must run a separate thread to handle Pyro messages (the Pyro daemon loop must be running to accept incoming calls). If your client only sits idle, and only waits for incoming callbacks, you can just run the daemon's requestLoop in the main thread. Have a look at the "callback" example for more details.

Be very aware of threading issues. Callbacks occur in their own thread. A callback may even occur while your client is still busy processing the previous callback. Your server should be even more prepared for callbacks. Usually you have a method that "registers" a client as a callback object. The client will call this method and has to pass a proxy to the callback object, not the object itself! (see usage rules, below).

Possible deadlock: if your objects enter a conversation, deadlock may occur easily. For instance, A calls B, B calls back to A, A calls B again... deadlock! B was still waiting for A to answer the callback, but A invokes a new method instead. Pyro cannot handle this yet. This issue might be addressed in a future Pyro version. In the meantime, please read on about possible alternatives.

Your server usually has a list of callback objects that it should call. Be very careful here: a client can unexpectedly disappear. You have to handle ConnectionClosedError exceptions and you must then remove the defunct callback object from the list. But you have to do this in a thread-safe way (the list must be under a thread-lock), because the server may be multithreaded! The server also has to handle any exceptions that occur in the callback object on the client. Don't trust it. Catch any exception that occurs, otherwise your server dies.
Be aware of a complex issue when your callback object raises an exception: if a callback occured in a remote method, that was called by the client, the exception might travel right back to the client if the server doesn't take precautions!

Not strictly callbacks, but when your Pyro object is calling other Pyro objects, you have to run in multithreaded mode. Because the new call may arrive at the same daemon, it must create a new thread to handle the new incoming call. When running in single threaded mode, Pyro will freeze because the calls are waiting on each other to be processed.

Please consider using the Pyro Event Service, instead of custom callback objects. It will handle all those nasty things for you, at the cost of some control. But it's very easy to use.

Special callback object Pyro.core.CallbackObjBase: Usually any exception that occurs in the callback object is silently transported back to the server, where it is raised again. For many callback objects, this is not exactly what you want, because usually the client is interested in an error within a callback object, and the server often doesn't care. If you use the special Pyro.core.CallbackObjBase as a base class for your callback objects (instead of the regular ObjBase), any exception that occurs in the callback object is not only sent to the server, but also raised again on the cient. You can see this work in one of the "callback" examples.

Oneway calls: These are remote method calls that do not expect any answer, so they return immediately after sending the remote call to the remote object. The call does not wait for the remote method to finish. At a lower level, it doesn't even wait for a protocol reply, so performance is much better too.

Normally, on the server side (the object that executes the oneway call), the call is executed in its own thread so that other calls can be processed in the meantime, if the oneway method takes some time to complete. This only works if multithreading is enabled. This property of oneway calls allows you, for instance, to start a computation and continue your own business while the computation runs. Later on you use another remote call to retrieve the results of the computation (or use a callback). This way, you don't have to worry about programming a multithreaded client, just use a oneway call. You can disable this by setting the PYRO_ONEWAY_THREADED config item to false (0). Oneway calls will then execute in the server's main thread. New method calls on the object will have to wait until the oneway method call has finished processing, as usual.

Note: There is no way to find out what the result of your request was - whether it succeeded or failed. No result nor any exception is ever returned. You're still quite sure that the call is performed though, because the request is sent using the regular PYRO protocol, but there is no guarantee (nor is there with regular calls by the way).

Oneway calls are very nice to have in a callback scenario. The Event Service also makes heavy use of them. Why? Your server is freed from the burden of handling exceptions that may occur in the remote method, and it doesn't block on slow or buggy clients. It just sends out the method invocations and continues on happily while the callback clients process the incoming method call.

You have to specify at runtime in your program which methods of what objects have this Oneway semantics. You do this by calling a special method on the Pyro proxy object:

obj._setOneway(methods)
where obj is your proxy and methods is a list or tuple of method names that have to be called Oneway. It may also be a single method name. Currently there is no way to specify from within the remote object itself, or the creating process, that a method has to be called oneway. The calling party has to set this property. Ofcourse you could build some sort of inquiry method that has to be called first and that tells the caller what methods can have this property, and maybe this will become automatic in a future version, but it's not yet there.

Dealing with exceptions

Pyro objects act just like regular Python objects. Your method calls are performed as if you're calling a regular local object. If the object raises an exception, the calling code will receive that exception and will act just as if the exception occurred in a local object.

Assume the remote Pyro object raises a ValueError exception. Your calling code will receive this and crash wit the same ValueError exception. However, the stacktrace in the traceback info is from your local code, not from the remote code. If you're just catching and processing exceptions, and don't want to deal with stacktrace/traceback info, there is no problem with this. But if you want to print the stacktrace, it is meaningless! It is a stacktrace from within the bowels of the local Pyro code! It provides no clue what piece of remote code caused the problem.

To help you with this, Pyro puts the remote stacktrace inside the exception that travels to your calling code. It is a list of text lines and can be obtained from the special attribute that is defined in Pyro.constants.TRACEBACK_ATTRIBUTE, but it's probably more convenient to use the utility function Pyro.util.getPyroTraceback. You pass the exception object and the function returns a list of lines that contain the remote traceback (if available) and the local traceback. Note that if you set PYRO_DETAILED_TRACEBACK to 1 on the server side, it will not be a normal traceback but a much more elaborated one including local variable values etc. An example:

try:
        print thing.method()            # thing is a Pyro proxy
except Exception,x:
        print ''.join(Pyro.util.getPyroTraceback(x))

This function is safe to call on all exceptions, also normal (local) exceptions. See the "simple" and "exceptions" examples. It is also possible to set a custom excepthook to automatically take care of the remote traceback printing:

    import sys
    import Pyro.util
    
    sys.excepthook = Pyro.util.excepthook
    
    #... call any method ...

Remotely accessing the Daemon

Usually you won't care a bit, but the Pyro Daemon that is running in each Pyro server program, is exposed by a Pyro object itself. That means that you can access a limited set of functions of remote Daemons! Pyro itself uses this for the PYROLOC: direct lookup protocol that bypasses the Name Server; it queries the remote Daemon directly. Because the Daemon is always there, it has a special, fixed GUID: Pyro.constants.INTERNAL_DAEMON_GUID. (this is not a GUID in the true sense, because it is not unique). Here's an example to query all registered objects on a remote Daemon at 192.168.1.50, running on the default port:
import Pyro.core
Pyro.core.initClient()
d=Pyro.core.PyroURI(host='192.168.1.50',objectID=Pyro.constants.INTERNAL_DAEMON_GUID).getProxy()
print d.getRegistered()
The result is a dictionary that maps GUIDs to object names, like this: {'c0a8013208585602469aec911dc92a20': ':Pyro.NameServer', 'c0000000011000001000000010000001': '__PYRO_Internal_Daemon'}.

Currently there is only one other daemon method that is remotely accessible, ResolvePYROLOC. You will probably never call this yourself, it is used for the PYROLOC: protocol. Note: If the daemon is running in SSL mode you have to add an additional prtcol="PYROSSL" argument to the PyroURI constructor call above!

Timeouts and cleaning up unused objects

If network errors occur, you often want to find out quickly. By default, Pyro may take a very long time to notice that a host is unreachable (it relies on the operating system to notice this). That's why it is possible to specify a custom timeout period. It is disabled by default because a default timeout period is very hard to decide on and the timeout logic also slightly decreases performance. But if you need, you can specify a timeout period on data transmission (sends and receives).

Because Pyro's connection protocol requires a handshake at connection time, the timeout also works for connecting to a Pyro daemon. This is nice, because evil clients now cannot eat connections indefinately by just connecting and never finishing the request. Once a connection has been established, it stays there.

How do you set the timeout?

        proxy._setTimeout(20)           # set 20-sec. timeout on proxy object (client)
        daemon.setTimeout(20)           # set 20-sec. timeout on daemon (server)
Clear it again by passing None. If a timeout occurs, Pyro raises a TimeoutError exception, which is subclassed from ConnectionClosedException. The connection has already been dropped by Pyro. (why is it _setTimeout -- with an underscore -- for the proxy? To avoid possible name clash with your own method names)

Note about NS and ES: the Name Server and Event Server have a built-in fixed timeout of 20 seconds. The connection is killed if data transmission takes longer than that, to prevent evil clients of clogging up the servers.

Cleaning up unused objects: reaping transients and passivate proxies

To save resources, Pyro facilitates in cleaning up unused connections or even objects. Your client may decide to release the connection if it doesn't need a proxy for some time. (remember that each proxy opens a network connection). Call proxy._release() to release the connection. Pyro will automatically create the connection again as soon as you start using the proxy again. An additional benefit is that the socket object associated with the proxy is destroyed, so you are able to pickle the proxy object again.

Things are a bit more complex on the server. Usually the Pyro objects you create there must stay active because it is not known when a client comes by to use them. This is especially true for objects that are registered in the Name Server, because that sort of tells the clients "this object is available there-and-there". But for transients, things are different. Remember that transients are objects created on the server, without a name, and usually a proxy is returned to the client to access these new objects. There's no way to tell when the client no longer needs the object! If it 'forgets' to call some sort of "release" method that destroys the object in the server, your server could overflow with all left-over unused transient objects. This is why you can specify an inactivity timeout for transients, by calling daemon.setTransientsCleanupAge(timeout) on your daemon. The argument is in seconds.
Pyro tracks the time an object has not been used, and destroys it when the timeout has expired. You may want to override the _gotReaped() method in the ObjBase to act on the destroy event (for instance, remove the object from a list).
Please be aware that if Pyro runs in single-threaded mode, the reaping of expired objects is only done when a method is invoked!
Also, cleaning up objects on the server doesn't automatically release any network connections and threads that might have been created for these objects. This is because these resources are not necessarily associated with a unique object, so they have to remain active. See also the "Bank2" example, it uses timeouts.

Usage rules and guidelines

You should follow the guidelines below when using Pyro.

  1. The remote class can't have a remote __init__ method. You should use a regular initialization method that you must call explicitly after binding to the remote object. The __init__ method will only be called on the server side when the object is created.
  2. All objects that pass over the wire have to be pickleable. You cannot create a remote method like openFile(filename) that opens a file on the server and returns the file object, because file objects cannot be pickled. This also holds for sockets, and various other object types. As long as you don't access these objects in your client, you're OK though (there's nothing wrong with a Pyro object that has various open files or sockets as attributes -- if you don't access them from the client).
  3. The remote class cannot support 'rich comparison', i.e. object1==object2 for instance. This is because the proxy class needs to hijack the rich comparison mechanism to be able to compare two proxy classes with each other.
  4. You have to choose explicitly for the special proxy that will allow direct attribute access, because it is slower than the regular proxy. Direct attribute access will not work with the regular proxy.
  5. You have to use the built-in proxy for the Name Server, provided in Pyro.naming.NameServerProxy. When you use the Locator, you're safe.
  6. All Python .py source files that contain the code of the objects that are used as parameters in remote method calls, must be available on the client and on the server. Otherwise the server cannot load the implementation code of an object that arrives in a remote method call. This is no longer necessary if you enable the mobile code feature. See the "agent2" example.
  7. All Python .py source files that contain the code of the objects that are used as attributes of Pyro objects on the server, must be available on the client also. Otherwise the client cannot recreate the server object's attributes and will crash if you access these attributes. This cannot be fixed with enabling mobile code yet.
  8. The class that is actually instantiated in the server should inherit from Pyro.core.ObjBase. You could define a new (probably empty) class in the server that inherits both from Pyro.core.ObjBase and the class that is made remotely available.
  9. You could also use the Delegation pattern instead of subclassing from Pyro.core.ObjBase. This works as follows: create an object of your remote class, create a Pyro.core.ObjBase object, and tell the latter to use the former as delegate:
    ...
    impl = MyRemoteClass()
    obj = Pyro.core.ObjBase()
    obj.delegateTo(impl)
    ...
    
    and you then connect obj to the Daemon.
  10. It is preferred to call daemon.shutdown() when your program exits. This will cleanly stop all threads that might still be running.
  11. Note that, because Python doesn't do this automatically, you have to call the __init__ from the base class in the __init__ method -if you have one- of your remote class. You cannot omit this, your Pyro object will not work without the initialization:
    def __init__(self):
        Pyro.core.ObjBase.__init__(self)
        ...
    
  12. If you create new Pyro objects on the server, and you want them to be accessed from the client, or vice versa (such as callback objects), you have to consider some things carefully. Make sure you:
  13. Funny and unexpected things happen if you try to use a Pyro proxy object in various statements such as while obj:. Like the limitation with direct member access, this is also caused by the way the proxy object is currently implemented. It intercepts each and every method call. And testing for zero-ness or nonzero-ness or coercion are also method calls! (__nonzero__, __coerce__ etc.)
  14. Consider subclassing callback objects from Pyro.core.CallbackObjBase instead of the regular ObjBase. (see above at "callbacks")
  15. If you want to use augmented assigment operators (such as +=, *=) and other special things on your Pyro object, please read the chapter of the Pytho/Language Reference, "Emulating numeric types". Also when you're using other special class methods such as __nonzero__. For example, to make obj+=50 work, you need to implement the __iadd__ method in your Pyro object and let it return the object itself (that is, a proxy for the object!) because the method returns the in-place modified object!
    def __iadd__(self,value):
        self.value+=value
        return self.getAttrProxy()
    
  16. You have to make sure that you keep a reference to your Daemon at all time, because if the daemon is no longer referenced, it might be garbage collected (destroyed) by Python. Even if you connected Pyro objects to the daemon. This is recommended anyway because you can then cleanly terminate your Pyro application by calling daemon.shutdown() when it exits. Usually this is not a problem because your program creates a deamon and calls its requestLoop. But a situation might arise where you don't keep a reference to the daemon object, and then things break.
  17. Pyro proxy objects (including the NameServerProxy) can be pickled, and therefore you can pass them around within your Pyro system. Thread ownership is taken by the thread that unpickled the proxy.
  18. If your Pyro object calls other Pyro objects, it is strongly recommended to run the Daemon in multithreaded mode. Failing to do so will probably lock up Pyro, for instance when a "looping" request arrives.
  19. Consider disconnecting objects from the daemon if you don't need them anymore; daemon.disconnect(object) (you can disconnect the object directly or provide its UID)
  20. Be careful when using large data structures. For instance, when you are getting an element from a dictionary using attribute access on the remote object (like this: value=remoteObject.somedict['somekey']) Pyro will transfer the full dictionary to the caller and do the key lookup only after that. If it's a big data structure, you have a big performance hit. Consider writing getter and setter methods on the remote object instead.