Package concurrent_tree_crawler :: Module crawlers_manager :: Class CrawlersManager
[hide private]
[frames] | no frames]

Class CrawlersManager

source code

Starts and stops different crawler threads

Instance Methods [hide private]
 
__init__(self, tree, navigators) source code
 
start(self) source code
 
stop(self) source code
 
wait_until_finish(self, timeout=None)
Wait until all threads finished their jobs and then get rid of them.
source code
Method Details [hide private]

__init__(self, tree, navigators)
(Constructor)

source code 
Parameters:
  • tree (TreeAccessor) - TreeAccessor object to be used by all of the threads
  • navigators (list of NavigatorTreeWrappers) - navigators to be used by the threads. Each thread obtains a single navigator. Number of threads created is the same as the number of navigators.

wait_until_finish(self, timeout=None)

source code 

Wait until all threads finished their jobs and then get rid of them. If timeout seconds pass before the threads are finished, they are stopped.

Parameters:
  • timeout - if the value is not None, the method blocks at most for timeout number of seconds, otherwise the method blocks until all threads are finished.
Returns:
False iff the wait ended because of the timeout