auromat.util.url module

exception auromat.util.url.DownloadError

Bases: exceptions.Exception

auromat.util.url.downloadFile(url, path, unifyErrors=True)

Download a single resource and store it as file to disk. On download errors (except 404), the download is retried once, after that an exception is raised.

auromat.util.url.downloadFiles(urls, paths, retFailures=False)

Downloads multiple resources and stores them to disk at the given paths, ignoring already existing files on disk. On download errors (except 404), the download is retried once. If retFailures is False, then True is returned if all files could be downloaded successfully, otherwise False. If retFailures is True, then a tuple (bool, failures) is returned which additionally contains the urls (with exceptions) that couldn’t be downloaded.

auromat.util.url.downloadJSON(url, unifyErrors=True, data=None, **kw)

Parse and return the JSON document at the given URL. Any additional keywords are given to json.load unchanged.

auromat.util.url.downloadResource(url, fn, data=None, unifyErrors=True)

Download a single resource and call fn on it. On download errors (except 404), the download is retried once, after that an exception is raised.

auromat.util.url.urlResponseCode(url)

Return the response code of the server without downloading the actual data.