Package ndg :: Package httpsclient :: Module utils
[hide private]

Module utils

source code

Utilities using NDG HTTPS Client, including a main module that can be used to fetch from a URL.


Date: 09/12/11

Author: R B Wilkinson

Contact: Philip.Kershaw@stfc.ac.uk

Copyright: (C) 2011 Science and Technology Facilities Council

License: BSD - see LICENSE file in top-level directory

Classes [hide private]
  AccumulatingHTTPCookieProcessor
Cookie processor that adds new cookies (instead of replacing the existing ones as HTTPCookieProcessor does)
  URLFetchError
Error fetching content from URL
  Configuration
Connection configuration.
Functions [hide private]
 
fetch_from_url(url, config, data=None, handlers=None)
Returns data retrieved from a URL.
source code
 
fetch_from_url_to_file(url, config, output_file, data=None, handlers=None)
Writes data retrieved from a URL to a file.
source code
file derived type
fetch_stream_from_url(url, config, data=None, handlers=None)
Returns data retrieved from a URL.
source code
 
open_url(url, config, data=None, handlers=None)
Attempts to open a connection to a specified URL.
source code
 
_should_use_proxy(url, no_proxy=None)
Determines whether a proxy should be used to open a connection to the specified URL, based on the value of the no_proxy environment variable.
source code
basestring
_url_as_string(url)
Returns the URL string from a URL value that is either a string or urllib2.Request..
source code
 
main()
Utility to fetch data using HTTP or HTTPS GET from a specified URL.
source code
Variables [hide private]
  __revision__ = '$Id$'
  log = logging.getLogger(__name__)
  __package__ = 'ndg.httpsclient'
Function Details [hide private]

fetch_from_url(url, config, data=None, handlers=None)

source code 

Returns data retrieved from a URL.

Parameters:
  • url (basestring) - URL to attempt to open
  • config (Configuration @return data retrieved from URL or None) - SSL context configuration

fetch_from_url_to_file(url, config, output_file, data=None, handlers=None)

source code 

Writes data retrieved from a URL to a file.

Parameters:
  • url (basestring) - URL to attempt to open
  • config (Configuration) - SSL context configuration
  • output_file (basestring) - output file
Returns:
tuple ( returned HTTP status code or 0 if an error occurred returned message boolean indicating whether access was successful)

fetch_stream_from_url(url, config, data=None, handlers=None)

source code 

Returns data retrieved from a URL.

Parameters:
  • url (basestring) - URL to attempt to open
  • config (Configuration) - SSL context configuration
Returns: file derived type
data retrieved from URL or None

open_url(url, config, data=None, handlers=None)

source code 

Attempts to open a connection to a specified URL.

Parameters:
  • url - URL to attempt to open
  • config (Configuration) - SSL context configuration
  • data (str) - HTTP POST data
  • handlers (iterable) - list of custom urllib2 handlers to add to the request
Returns:
tuple ( returned HTTP status code or 0 if an error occurred returned message or error description response object)

_should_use_proxy(url, no_proxy=None)

source code 

Determines whether a proxy should be used to open a connection to the specified URL, based on the value of the no_proxy environment variable.

Parameters:
  • url (basestring or urllib2.Request) - URL

_url_as_string(url)

source code 

Returns the URL string from a URL value that is either a string or urllib2.Request..

Parameters:
  • url (basestring or urllib2.Request) - URL
Returns: basestring
URL string