pydica-watchdog requires Python 2.5 or above to work. If you are using a Linux/FreeBSD/Mac OS X system, you already have Python installed. However, you may wish to upgrade your system to Python 2.7 at least, because this version comes with updates that can reduce compatibility problems. See a list of Dependencies.
$ pip install pydica-watchdog
$ wget -c http://pypi.python.org/packages/source/w/watchdog/watchdog-0.6.2.tar.gz $ tar zxvf pydica-watchdog-0.6.2.tar.gz $ cd pydica-watchdog-0.6.2 $ python setup.py install
$ git clone --recursive git://bitbucket.org/pydica/watchdog.git
$ cd watchdog
$ python setup.py install
pydica-watchdog depends on many libraries to do its job. The following is a list of dependencies you need based on the operating system you are using.
| Operating system Dependency (row) | Windows | Linux 2.6 |
|
BSD |
|---|---|---|---|---|
| XCode | Yes | |||
| PyYAML | Yes | Yes | Yes | Yes |
| argh | Yes | Yes | Yes | Yes |
| argparse | Yes | Yes | Yes | Yes |
| select_backport (Python 2.5/2.6) | Yes | Yes | ||
| pathtools | Yes | Yes | Yes | Yes |
| a lot of luck | Yes |
The watchmedo script depends on PyYAML which links with LibYAML. On Mac OS X, you can use homebrew to install LibYAML:
brew install libyaml
On Linux, use your favorite package manager to install LibYAML. Here’s how you do it on Ubuntu:
sudo aptitude install libyaml-dev
On Windows, please install PyYAML using the binaries they provide.
pydica-watchdog uses native APIs as much as possible falling back to polling the disk periodically to compare directory snapshots only when it cannot use an API natively-provided by the underlying operating system. The following operating systems are currently supported:
Warning
Differences between behaviors of these native API are noted below.
The Darwin kernel/OS X API maintains two ways to monitor directories for file system events:
pydica-watchdog can use whichever one is available, preferring FSEvents over kqueue(2). kqueue(2) uses open file descriptors for monitoring and the current implementation uses Mac OS X File System Monitoring Performance Guidelines to open these file descriptors only to monitor events, thus allowing OS X to unmount volumes that are being watched without locking them.
Note
More information about how pydica-watchdog uses kqueue(2) is noted in BSD Unix variants. Much of this information applies to Mac OS X as well.
BSD variants come with kqueue which programs can use to monitor changes to open file descriptors. Because of the way kqueue(2) works, pydica-watchdog needs to open these files and directories in read-only non-blocking mode and keep books about them.
pydica-watchdog will automatically open file descriptors for all new files/directories created and close those for which are deleted.
Note
The maximum number of open file descriptor per process limit on your operating system can hinder pydica-watchdog‘s ability to monitor files.
You should ensure this limit is set to at least 1024 (or a value suitable to your usage). The following command appended to your ~/.profile configuration file does this for you:
ulimit -n 1024
The Windows API on Windows XP provides the ReadDirectoryChangesW function, which operates in either synchronous or asynchronous mode. pydica-watchdog contains implementations for both approaches.
The asynchronous I/O implementation is more scalable, does not depend entirely only on Python threads, and is hence preferred over the synchronous blocking version, which pydica-watchdog isolates from your main thread using Python threads so that your programs do not block.
Note
Since renaming is not the same operation as movement on Windows, pydica-watchdog tries hard to convert renames to movement events. Also, because the ReadDirectoryChangesW API function returns rename/movement events for directories even before the underlying I/O is complete, pydica-watchdog may not be able to completely scan the moved directory in order to successfully queue movement events for files and directories within it.
pydica-watchdog also includes a fallback-implementation that polls watched directories for changes by periodically comparing snapshots of the directory tree.
Note
Windows caveats again.
Because Windows has no concept of inodes as Unix-y platforms do, there is no current reliable way of determining file/directory movement on Windows without help from the Windows API.
You can use hashing for only those files in which you are interested in your event handlers to determine this, although it is rather slow. pydica-watchdog does not attempt to handle this on Windows. It is left to your discretion.
Although mounted drives with the same system sometimes just work fine with the better drivers, there are cases where even polling fails miserably. This can be handled with MountObserver. The reason is:
On virtual servers, when polling a drive from the host machine, the virtualization software creates pseudo-inode numbers which change at every access.
To make this case work, pydica-watchdog has a version of polling where os.stat() explicitly gets filtered. The drawback is that file movement always appears as deletion and insertion.
Note
This was tested with Parallels Desktop, but it seems to be true for other virtualizations, too.