Class Hierarchy

concurrent_tree_crawler.common.activity_schedule.AbstractActivitySchedule: Activity and inactivity time schedule.
- concurrent_tree_crawler.common.activity_schedule.AlwaysActiveSchedule
- concurrent_tree_crawler.common.activity_schedule.DaySchedule
concurrent_tree_crawler.html_multipage_navigator.cmdln.abstract_levels_creator.AbstractCmdLnLevelsCreator
- concurrent_tree_crawler.html_multipage_navigator.cmdln.sample_levels_creator.SampleCmdLnLevelsCreator
concurrent_tree_crawler.abstract_cmdln_navigators_creator.AbstractCmdLnNavigatorsCreator
- concurrent_tree_crawler.html_multipage_navigator.cmdln.navigators_creator.CmdLnNavigatorsCreator
concurrent_tree_crawler.html_multipage_navigator.abstract_page_analyzer.AbstractLevelsCreator: A class responsible for creating a list of Levels which describe structure of the explored web site.
- concurrent_tree_crawler.html_multipage_navigator.sample_page_analyzer.LevelsCreator
concurrent_tree_crawler.abstract_node.AbstractNode: A node representing a single element of the tree traversed by the crawler
- concurrent_tree_crawler.standard_node.StandardNode: A simple in-memory implementation of the AbstractNode.
concurrent_tree_crawler.html_multipage_navigator.abstract_page_analyzer.AbstractPageAnalyzer
- concurrent_tree_crawler.html_multipage_navigator.sample_page_analyzer.ArticlePageAnalyzer: A class that downloads article pages
- concurrent_tree_crawler.html_multipage_navigator.sample_page_analyzer.IssuePageAnalyzer: A class that parses issues-level pages
- concurrent_tree_crawler.html_multipage_navigator.sample_page_analyzer.MagazinePageAnalyzer: A class that parses magazine-level pages
concurrent_tree_crawler.abstract_tree_accessor.AbstractTreeAccessor: An interface for the tree made of AbstractNodes.
- concurrent_tree_crawler.tree_accessor.TreeAccessor: An interface for the tree made of AbstractNodes.
  - concurrent_tree_crawler.rw_lock_tree_accessor.RWLockTreeAccessor: A version of the TreeAccessor where sensitive methods are protected by readers-writers lock.
concurrent_tree_crawler.abstract_tree_navigator.AbstractTreeNavigator: An object that has a direct access to the elements of the domain tree (e.g.
- concurrent_tree_crawler.html_multipage_navigator.tree_navigator.HTMLMultipageNavigator: A web site tree navigator.
concurrent_tree_crawler.html_multipage_navigator.web_browser.AbstractWebBrowser: An abstract functionality of a web browser
- concurrent_tree_crawler.html_multipage_navigator.web_browser.MechanizeBrowser: The default browser.
- concurrent_tree_crawler.html_multipage_navigator.throttled_web_browser.ThrottledWebBrowserWrapper
concurrent_tree_crawler.html_multipage_navigator.web_browser.AbstractWebBrowserCreator: The 'Creator' class from the Factory Method design pattern
- concurrent_tree_crawler.html_multipage_navigator.web_browser.MechanizeBrowserCreator
- concurrent_tree_crawler.html_multipage_navigator.throttled_web_browser.ThrottledWebBrowserCreator
concurrent_tree_crawler.common.activity_schedule.ActivityInfo
SocketServer.BaseRequestHandler: Base class for request handler classes.
- SocketServer.StreamRequestHandler: Define self.rfile and self.wfile for stream sockets.
  - BaseHTTPServer.BaseHTTPRequestHandler: HTTP request handler base class.
    - SimpleHTTPServer.SimpleHTTPRequestHandler: Simple HTTP request handler with GET and HEAD commands.
      - concurrent_tree_crawler.common.delayed_http_files_server._DelayedFilesServerHandler
SocketServer.BaseServer: Base class for server classes.
- SocketServer.TCPServer: Base class for various socket-based server classes.
  - concurrent_tree_crawler.common.delayed_http_files_server._ThreadedTCPServer
concurrent_tree_crawler.cmdln_multithreaded_crawler.CmdLnMultithreadedCrawler: A class that creates the MultithreadedCrawler object based on command-line parameters
concurrent_tree_crawler.crawlers_manager.CrawlersManager: Starts and stops different crawler threads
concurrent_tree_crawler.common.delayed_http_files_server.DelayedHTTPFilesServer: HTTP Files server with a new thread created for each new connection.
concurrent_tree_crawler.html_multipage_navigator.abstract_page_analyzer.Level
concurrent_tree_crawler.common.logger.Logger
rfc822.Message: Represents a single RFC 2822-compliant message.
- mimetools.Message: A derived class of rfc822.Message that knows about MIME headers and contains some hooks for decoding encoded and multipart messages.
concurrent_tree_crawler.multithreaded_crawler.MultithreadedCrawler: Runs several threads to crawl the tree.
concurrent_tree_crawler.navigator_tree_wrapper.NavigatorTreeWrapper: Wrapper for the tree navigator (AbstractTreeNavigator).
concurrent_tree_crawler.abstract_tree_accessor.NodeAction: Enumeration describing actions that can be taken by a crawler when entering a new node
concurrent_tree_crawler.abstract_node.NodeState: Enumeration describing state of processing of a certain node
concurrent_tree_crawler.html_multipage_navigator.abstract_page_analyzer.PageLinks
concurrent_tree_crawler.common.threads.rw_lock.RWLock: Synchronization object used in a solution of so-called second readers-writers problem.
concurrent_tree_crawler.common.resources.Resources
concurrent_tree_crawler.simple_dfs_crawler.SimpleDFSCrawler
concurrent_tree_crawler.common.threads.sleep.Sleep: A counterpart of the time.sleep() function.
concurrent_tree_crawler.common.tempdir.TempDir
SocketServer.ThreadingMixIn: Mix-in class to handle each request in a new thread.
- concurrent_tree_crawler.common.delayed_http_files_server._ThreadedTCPServer
concurrent_tree_crawler.common.threads.token_bucket.TokenBucket: Interface of a token bucket
- concurrent_tree_crawler.common.threads.token_bucket.InfiniteTokenBucket: Token bucket with an infinite supply of tokens
- concurrent_tree_crawler.common.threads.token_bucket.StandardTokenBucket
concurrent_tree_crawler.xml_tree_serialization.XMLTreeReader
concurrent_tree_crawler.xml_tree_serialization.XMLTreeWriter
concurrent_tree_crawler.html_multipage_navigator.tree_navigator._ChildrenHistory: A history of children nodes on consecutive levels of hierarchy.
concurrent_tree_crawler.common.threads.rw_lock._LightSwitch: An auxiliary "light switch"-like object.
concurrent_tree_crawler.cmdln_multithreaded_crawler._TimeParser
object: The most base type
- exceptions.BaseException: Common base class for all exceptions
  - exceptions.Exception: Common base class for all non-exit exceptions.
    - concurrent_tree_crawler.crawlers_manager.CrawlersManagerException
    - concurrent_tree_crawler.abstract_tree_navigator.NavigationException: See the description of AbstractTreeNavigator class for a discussion on the usage of this exception.
      - concurrent_tree_crawler.html_multipage_navigator.sample_page_analyzer.PageAnalyzerException
- threading._Verbose
  - threading.Thread
    - concurrent_tree_crawler.common.threads.ex_thread.ExThread: A thread that can throw an exception to the joining thread.
      - concurrent_tree_crawler.crawler_thread.CrawlerThread
    - concurrent_tree_crawler.common.threads.token_bucket.TokenBucketFiller: This thread fills the token bucket every fill_interval_in_secs seconds with fill_amount number of tokens.
    - concurrent_tree_crawler.tree_saver_thread.TreeSaverThread