Sphinx logo

utils – utility classes and functions


The utils module contains classes and funtions of general utility used in multiple places throughout astropysics. Some of these are astropyhysics-specific algorithms while others are more python tricks.

The utils module is composed of three submodules to make organization clearer. The submodules are fairly different from each other, but the main uniting theme is that all of these submodules are not specific to a particularly astronomical application. Hence, they are available for re-use in astropysics or anywhere else they are deemed useful. The detailed documentation for each of the sub-modules is described below.


The utils module is composed of submodules that can be accessed seperately. However, they are all also included in the base module. Thus, as an example, astropysics.utils.gen.DataObjectRegistry and astropysics.utils.DataObjectRegistry are different names for the same class (DataObjectRegistry). The astropysics.utils.DataObjectRegistry usage is preferred as this allows the internal organization to be changed if it is deemed necessary.

utils.gen – general-purpose (python-specific) utilities

The gen module contains classes and functions of general utility used in various places throughout astropysics. These are python-specific utilities and hacks - general data-processing or numerical operations are in the alg module.

Classes and Inheritance Structure

Inheritance diagram of astropysics.utils.gen

Module API

class astropysics.utils.gen.DataObjectRegistry(dataname='data', datatype=None)

Bases: dict

A class to register data sets used throughout a module and enable easy access using string names.

addToGroup(key, group)

Adds the registered object with the provided name to a group with the given name.

  • key (string) – The object to register
  • group (string) – The group name to apply

Generates and returns list of names of the objects in this registry, extracted from the name attribute of the object if present, otherwise from the registry key. The order is the same as that generated by keys() or values()

Parameters:retdicy – If True, the return value is a dictionary mapping keys from the registry to object names.
Returns:List of names.
getObjects(objectstrs, addmissing=False)

Translates a string, sequence of strings, or mixed sequence of data objects and strings into a sequence of data objects.

  • objectstrs – The requested object names, mixed with objects if desired.
  • addmissing (bool) – If True, data objects in objectstrs will be added to the registry. If the object has a name attribute, it will be used as the name, otherwise a name will be generated of the form “dataname##”.

String, sequence of strings, or mixed sequence of strings and data objects.


A sequence of data objects.

register(objects, groupname=None)

Register a set of objects, possibly in a group

  • objects (dictionary mapping strings to objects with type matching datatype) – map of the datum name to the associated objects to store
  • groupname (string or None) – A common name that applies to all of the objects

Removes the provided group

Parameters:group (string) – name of group to remove
class astropysics.utils.gen.SymmetricMapping(*args)

Bases: _abcoll.MutableMapping

A dict-like object that maps in two directions - the keys for one direction are the value for the other, and vice versa.

Note that this means all values most be unique and non-mutable.


This class is probably not at all thread safe.

initialized same as a dict


This class is a decorator indicating that the decorated object should have part (or all) of its docstring replaced by another object’s docstring.

The arguments are objects with a __name__ and __doc__ attribute. If a string of the form ‘{docstr:Name}’ is present in the decorated object’s docstring, it will be replaced by the docstring from the passed in object with __name__ of ‘Name’. Any arguments without a ‘{docstr:Whatever}’ to replace will be appended to the end of the decorated object’s docstring.


>>> def f1(x):
...     '''Docstring 1'''
...     pass
>>> def f2(x):
...     '''
...     Docstring 2
...     and more!
...     '''
...     pass
>>> @add_docs(f1)
... def f3(x):
...     '''
...     Docstring 3
...     '''
...     pass
>>> @add_docs(f2)
... def f4(x):
...     '''
...     Docstring 3
...     '''
...     pass
>>> @add_docs(f1)
... def f5(x):
...     '''
...     Docstrong 2 {docstr:f1}
...     '''
...     pass
>>> f1.__doc__
'Docstring 1'
>>> f2.__doc__
'\n    Docstring 2\n    and more!\n    '
>>> f3.__doc__
'\n    Docstring 3\n    \n    Docstring 1'
>>> f4.__doc__
'\n    Docstring 3\n    \n    Docstring 2\n    and more!'
>>> f5.__doc__
'\n    Docstrong 2 Docstring 1\n    '

Does the same thing as replace_docs(), but also adds the function signature of the argument function to the replaced (followed by a newline). Note that this requires that the argument object be a function (and not anything with a __name__ and __doc__ attribute). This is typically useful for functions that do f(*args,**kwargs) to wrap some other function.

astropysics.utils.gen.change_indentation(s, ind=0)

Sets the starting indentation of the provided docstring to the given indentation level.

  • s (string) – The string to dedent
  • ind (str or int) – Amount of indentation, either as a string (that will be substituted) or an integer number of spaces.

a string like s with leading indentation removed.

astropysics.utils.gen.check_type(types, val, acceptnone=True)

Call this function to check if the value matches the provided types.

  • types – A single type, a sequence of types, or a callable that accepts one argument and will be called with the value - if it returns True, the value is the correct type.
  • val – The value to type-check.
  • acceptnone (bool) – Always accept the value None regardless of types
  • TypeError – if the type-checking fails
  • ValueError – if the types are invalid


If any of the types are a numpy.dtype, and the value is an numpy.array, the data type will also be checked.

utils.alg – common/repeated algorithms

The alg module contains basic algorithms, numerical tricks, and generic data processing tasks used in more than one place in astropysics.

Module API

astropysics.utils.alg.angle_axis(matrix, degrees=True)

Computes the angle of rotation and the rotation axis for a given rotation matrix.

  • matrix (a 3x3 numpy.ndarray) – the rotation matrix
  • degrees (boolean) – if True, output is in degrees

an (angle,axis) tuple where the angle is in degrees if degrees is True, otherwise in radians

astropysics.utils.alg.centroid(val, axes=None, offset=None)

Convinience function calling moments() to get the first normalized moment (i.e. the centroid).

  • val (array-like) – n-d array for which to compute the centroid.
  • axes (array-like or None) – None to use default 0-based location scheme (see astropysics.utils.stats.moments()) or an array of poisition values for the centriod (must have as many elements as the dimension of val)
  • offset – A fixed offset to subtract from the data before centroiding, or a bgmethod like those accepted by estimate_background().
astropysics.utils.alg.crossmask(x, threshold=0, belowtoabove=True)

Generates a boolean mask for the point where the input crosses or has complted crossing (if between) from below (or above) a threshold.

If belowtoabove is True, the returned masks is for where the input transitions from below to above. Otherwise, from above to below.

  • x (array-like) – input (will be flattened to 1D)
  • threshold (scalar) – the transition value for where the crossings occur
  • belowtoabove (bool) – If True, the returned mask is for where the input transitions from below to above. Otherwise, from above to below.

A mask that is True where crossing occurs, False everywhere else.

Return type:

bool ndarray


>>> from numpy import array,where
>>> xup = [-2,-1,0,1,2]
>>> xdn = [2,1,0,-1,-2]
>>> print crossmask(xup,0,True)
[False False  True False False]
>>> print crossmask(xup,-0.5,True)
[False False  True False False]
>>> print crossmask(xup,0.5,True)
[False False False  True False]
>>> print crossmask(xdn,0,True)
[False False False False False]
>>> print crossmask(xdn,0,False)
[False False  True False False]
>>> print crossmask(xdn,0.5,False)
[False False  True False False]
>>> print crossmask(xdn,-0.5,False)
[False False False  True False]
>>> xupdnup = [-2,-1,0,1,2,1,0,-1,-2,-1,0,1,2]
>>> where(crossmask(xupdnup,0.5,True))
(array([ 3, 11]),)
>>> print array(xupdnup)[crossmask(xupdnup,0,True)]
[0 0]
astropysics.utils.alg.estimate_background(arr, method='median')

Estimates the background of the provided array following a technique specified by the method keyword:

  • ‘median’ : median of all the data
  • ‘mean’ : mean of all the data
  • ‘32’ : 3*median - 2*mean
  • ‘2515’ : 2.5*median - 1.5*mean
  • ‘21’ : 2*median - 1*mean
  • a callable that takes a 1D array and outputs a scalar
  • a scalar : always returns that value
  • None : always returns a 0 background

outputs a scalar background estimate

astropysics.utils.alg.intrinsic_to_observed_ellipticity(ei, i, degrees=True)

Converts intrinsic ellipticity to observed where e^2 = 1-(b/a)^2

  • ei (float or array-like) – intrinsic ellipticity
  • i (float or array-like) – inclination angle
  • degrees – if True, the inclination is assumed to be in degrees

observed ellipticity

astropysics.utils.alg.lin_to_log_rescale(val, lower=1, upper=3, base=10)

Linearly rescales input values onto the range [base^lower,base^upper] and then applies the requested base of logarithm to the rescaled values.

  • val (array-like) – input linear values
  • lower (scalar) – lower value for output values
  • upper (scalar) – upper value for output values
  • base (base of logarithm) – base of logarithm

logarithm of rescaled input


Generates a grid of values given a sequence of 1D arrays. The inputs will be converted to 1-d vectors and the output is an array with dimensions (nvecs,n1,n2,n3,...) varying only on the dimension corresponding to its input order


x = linspace(-1,1,10)
y = x**2+3
z = randn(13)
result = nd_grid(x,y,z)
xg,yg,zg = result

result will be a (3,10,10,13) array, and each of xg,yg, and zg are (10,10,13)

astropysics.utils.alg.nearestsorted(a, val)

Search a sorted sequence for the nearest value.

  • a (array-like) – A sorted sequence to be searched.
  • val – The value(s) at which to find the nearest match in a.

nearest,inds Where nearest are the elements of a that are nearest to the val values, and inds are the indecies into a that give those values.

astropysics.utils.alg.observed_to_intrinsic_ellipticity(eo, i, degrees=True)

Converts observed ellipticity to intrinsic where e^2 = 1-(b/a)^2

  • eo (float or array-like) – observed ellipticity
  • i (float or array-like) – inclination angle
  • degrees – if True, the inclination is assumed to be in degrees

intrinsic ellipticity

astropysics.utils.alg.rotation_matrix(angle, axis='z', degrees=True)

Generate a 3x3 rotation matrix in cartesian coordinates for rotation about the requested axis.

  • axis (string or 3-sequence) – Either ‘x’,’y’, ‘z’, or a (x,y,z) specifying an axis to rotate about. If ‘x’,’y’, or ‘z’, the rotation sense is counterclockwise looking down the + axis (e.g. positive rotations obey left-hand-rule).
  • degrees (boolean) – If True the input angle is degrees, otherwise radians.

A numpy.matrix unitary rotation matrix.

astropysics.utils.alg.rotation_matrix_xy(x, y)

Computes the rotation matrix that moves the +z-axis (pole) to a new vector (x,y,z’) where x and y are specified and z’ is constrained by requiring the vector to be length-1.

  • x (float) – The x-value of the new pole in the starting coordinate system
  • y (float) – The y-value of the new pole in the starting coordinate system

A numpy.matrix unitary rotation matrix.

astropysics.utils.alg.sigma_clip(data, sig=3, iters=1, cenfunc='median', varfunc=<function var at 0x10c747e60>, maout=False)

This performs the sigma clipping algorithm - i.e. the data will be iterated over, each time rejecting points that are more than a specified number of standard deviations discrepant.

  • data (array-like) – input data (will be flattened to 1D)
  • sig (scalar) – The number of standard deviations to use as the clipping limit, or the square root of the variance limit.
  • iters – The number of iterations to perform clipping for, or None to clip until convergence is achieved
  • cenfunc – The technique to compute the center for the clipping - may be any valid input to estimate_background()
  • varfunc (a callable) – The method to compute the variance about the. Should take a 1D array as input and output a scalar. This will be compared to the square of the data as if it is the variance.
  • maout (bool) – If True, return a masked array (see return value for details).

A numpy.ma.Maskedarray with the rejected points masked, if maout is True. If maout is False, a tuple (filtereddata,mask) is returned where the mask is False for rejected points (and matches the shape of the input).

utils.stats – statistcs-related tools

The stats module contains classes and functions for statistics and statistical analysis. These tools in this module are mostly inspecific to astrophysics - the applications are all in the relevant other modules.

See also

scipy.stats - astropysics,utils.stats is intended only to provide utilites and interfaces that are not present in scipy.stats - when possible, scipy.stats should be used.

Module API

class astropysics.utils.stats.Pca(data, names=None)

Bases: object

A class for Principal Component Analysis (PCA).

When mentioned in docstrings, p is the number of dimensions, and N is the number of data points.

  • data – A p X N array-like object that is the data upon which PCA is to be performed.
  • names – A sequence of names for each of the p axes (used in plots).
deproject(A, normed=True)

input is an n X q array, where q <= p

output is p X n


Computes the covariance matrix for the dataset.

Returns:A p x p covariance matrix.

Computes the eigenvalues and eigenvectors of the data set.

Returns:A 2-tuple (eigenvalues,eigenvectors)

Computes and returns a length p array with the eigenvalues.


Computes and returns a p x p array with the eigenvalues.


Computes and returns a length p array with the eigenvalues normalized so that they sum to 1.

plot2d(ix=0, iy=1, clf=True)

Generates a 2-dimensional plot of the data set and principle components using matplotlib.

ix specifies which p-dimension to put on the x-axis of the plot and iy specifies which to put on the y-axis (0-indexed)

plot3d(ix=0, iy=1, iz=2, clf=True)

Generates a 3-dimensional plot of the data set and principle components using mayavi.

ix, iy, and iz specify which of the input p-dimensions to place on each of the x,y,z axes, respectively (0-indexed).

project(vals=None, enthresh=None, nPCs=None, cumen=None)

projects the normalized values onto the components

enthresh, nPCs, and cumen determine how many PCs to use

if vals is None, the normalized data vectors are the values to project. Otherwise, it should be convertable to a p x N array

returns n,p(>threshold) dimension array


clips out all data points that are more than a certain number of standard deviations from the mean.

sigs can be either a single value or a length-p sequence that specifies the number of standard deviations along each of the p dimensions.

subtractPC(pc, vals=None)

pc can be a scalar or any sequence of pc indecies

if vals is None, the source data is self.A, else whatever is in vals (which must be p x m)

astropysics.utils.stats.binned_weights(values, n, log=False)

Produces an array of weights that are generated by subdividing the values into n bins such that each bin has an equal share of the total number of values.

  • values (array-like) – the input values
  • n (int) – number of bins
  • log (bool) – If True, the values are evenly-spaced on logarithmic intervals, otherwise, linear.

An array of weights on [0,1] with shape matching values

astropysics.utils.stats.biweight_midvariance(values, influencescale=9, cenestimator=<function median at 0x10c6ab938>)

Computes the biweight midvariance of a sequence of data points, a robust statistic of scale.

For normal and uniform distributions, it is typically close to, but a bit above the variance.

  • values (array-like, will be treated as 1D) – the values for which to compute the biweight
  • influencescale (int) – The number of MAD units away at which a data point has no weight
  • cenestimator (callable) – A function to estimate the center of the values from a 1D array of values. To be a true standard biweight midvariance, this must be left as the default (median).

biweight,median tuple (both floats)

astropysics.utils.stats.interquartile_range(values, scaletonormal=False)

Computes the interquartile range for the provided sequence of values, a more robust estimator than the variance.

  • values (array-like, will be treated as 1D) – the values for which to compute the interquartile range
  • scaletonormal (bool) – Rescale so that a normal distribution returns 1

the interquartile range as a float

astropysics.utils.stats.median_absolute_deviation(values, scaletonormal=False, cenestimator=<function median at 0x10c6ab938>)

Computes the median_absolute_deviation for the provided sequence of values, a more robust estimator than the variance.

  • values (array-like, will be treated as 1D) – the values for which to compute the MAD
  • scaletonormal (bool) – Rescale the MAD so that a normal distribution is 1
  • cenestimator (callable) – A function to estimate the center of the values from a 1D array of values. To actually be the “median” absolute deviation, this must be left as the default (median).

the MAD as a float

astropysics.utils.stats.moments(arr, ms, axes=None, bgmethod=None, norm=True, std=False)

Compute the moments of the provided n-d array. That is

  • arr – The input array for which the moments are desired.
  • ms

    The desired order of the moment to be returned. Can be:

    • A scalar
      The same order will be used for each dimension, and the return type will be an array of the moment along each dimension.
    • A sequence
      The sequence must match the number of dimensions in arr, and specifies the order along each dimension. e.g. for a 3D cartesian array, ms=[1,3,2] means the moemnt comupted will be x^1 y^3 z^2.
  • axes – If None, the spatial axes are taken to be 0,1,2...,nd-1 for each of the dimenstions in the input. Otherwise, axes must be a seqence of arrays specifying the spatial locations along each axis. This sequence must be of length matching the number of dimensions in the input, and Each element must be of the same length as the corresponding input array dimension.
  • bgmethod – A background-estimation method (see estimate_background() for options), a scalar to subtract from the array, or None to do no subtraction.
  • norm (bool) – If True, the moment will be normalized - i.e. sum(x^m*arr)/sum(arr) instead of just sum(x^m*arr)
  • std (bool) – If True, the output will be standardized (mth moment divided by standard deviation to the mth power)

Either the computed moment if ms is a sequence, or a 1D array of moments for each dimension if ms is a scalar.

utils.io – input/output and various file format tools

The io module contains classes and functions for loading and saving data in various relevant formats used in astronomy, as well as convinience functions for retrieval of built-in data.

Classes and Inheritance Structure

Inheritance diagram of astropysics.utils.io

Module API

class astropysics.utils.io.FixedColumnDataParser(skiprows=0, firstcolindx=1, commentchars='#')

Bases: object

Parses a data file composed of lines with a fixed set of columns with the same number of bytes in each

  • skiprows (int) – The number of rows to skip initially
  • indexing – The value of the first column as used in the definition of the upper and lower edges of fields - that is, if 1, the “1-indexing” convntion is used, or if 0, standard python “0-indexing” is used.
  • commentchars (string (interpreted as sequence of characters)) – The comment character - any lines that start with these characters will be ignored.
addColumn(name, lower, upper, format=None, converters=None)
  • name (string) – Name of the column
  • lower (int) – lowest character index for this column including this one. Convention for the value of first column is controlled by firstcol attribute of the object - default is 1-indexing
  • upper (int) – highest character index for this column including this one. Convention for the value of first column is controlled by firstcol attribute of the object - default is 1-indexing
  • format (input to numpy.dtype or None) – Format for this column, or None to infer from the contents.
  • converters (string dict or None) – A dictionary mapping one string to another - all entries in the parsed data file with a key will be converted to the corresponding value. If None, no conversion will take place.
addColumnsFromFile(fn, columnlinestart=None, sep=None, useskiprows=True, maxcols=100)

Adds columns parsed from a file (typically a data file that will afterwards be read).

The parsed file is expected to have lines that begin with the columnlinestart argument and the rest of the line should be able to be split (using sep) into three or four columns in order name,lower,upper,format .

  • fn (string) – File name of the file to parse
  • columnlinestart (string or None) – String that indicates the line is a column line, or None to just use the first maxcols columns.
  • sep (string or None) – The seperator string to split the line or None for whitespace.
  • useskiprows (bool) – If True, the skiprows attribute of this object will be used to determine how many rows to skip.
  • maxcols (int or None) – Maximum number of columns to parse before quitting (or None to be unlimited).

number of columns added


For a file that starts like:

ID 0 3 int
data 4 10 float
moredata 11 15 int

use addColumnsFromFile(‘filename’,columnlinestart=None,sep=None,maxcols=3,firstcolindex=0)

And for a file with format specifier:

#... ID,1,4,int
#... data,5,11,float
#... moredata,12,16,int

use addColumnsFromFile(‘filename’,columnlinestart=’#... ‘,sep=’,’)


Removes a column by name

Parameters:name (string) – name of the column to remove
parseFile(fn, maskedarray=False)

Parse a file that follows this object’s format.

Parameters:fn (string) – File name of the file to parse
Params maskedarray:
 If True, the function returns a masked array, otherwise a record array.
Returns:If maskedarray is False, a tuple (recarr,masks) where recarr is a numpy.core.records.recarray and masks is a dictionary mapping column names to masks. The masks are True if the value is valid, and False if not. If maskedarray is True, a numpy.ma.core.MaskedArray is returned.
writeFile(fn, data, masks=None, colstart='# ')

Writes a data array out to a data file using this format.

  • fn (string) – The file to write to.
  • data (Structured numpy.array or numpy.core.records.recarray.) – The data array to write. Must match this objects column names and formats.
  • masks (None or dict) – If supplied, maps field names to boolean arrays where True indicates the data value should be written. If None, all data is written.
  • colstart (string) – the string to use to indicate a column specifier
Raises TypeError:

If the data dtype doesn’t match the columns.

class astropysics.utils.io.VOTable(*args, **kwargs)

Bases: astropysics.utils.io.VOTableReader

class astropysics.utils.io.VOTableReader(s, filename=True)

Bases: object

This class represents a VOTable. Currently, it is read-only, and will probably not be enhanced due to the existence of Michael Droettboom’s vo.table package (http://trac6.assembla.com/astrolib).

instantiate a VOTable object from an XML VOTable

If filename is True, the input string will be interpreted as a filename for a VOTable, otherwise s will be interpreted as an XML-formatted string with the VOTable data

dtypemap = {'unsignedByte': 'u1', 'int': 'i4', 'float': 'f4', 'char': 'a', 'boolean': 'b', 'bit': None, 'unicodeChar': 'U', 'short': 'i2', 'double': 'f8', 'floatComplex': 'c8', 'long': 'i8', 'doubleComplex': 'c16'}
astropysics.utils.io.fpickle(object, fileorname, usecPickle=True, protocol=None, append=False)

Pickle an object to a specified file.

  • object – the python object to pickle
  • fileorname (a file name string or a file object) – The file from which to unpickle objects
  • usecPickle (bool) – If True, the cPickle module is to be used in place of pickle (cPickle is faster).
  • protocol (int or None) – Pickle protocol to use - see the pickle module for details on these options. If None, the most recent protocol will be used.
  • append (bool) – If True, the object is appended to the end of the file, otherwise the file will be overwritten (if a file object is given instead of a file name, this has no effect).
astropysics.utils.io.funpickle(fileorname, number=0, usecPickle=True)

Unpickle a pickled object from a specified file and return the contents.

  • fileorname (a file name string or a file object) – The file from which to unpickle objects
  • number (int) – The number of objects to unpickle - if <1, returns a single object.
  • usecPickle (bool) – If True, the cPickle module is to be used in place of pickle (cPickle is faster).

A list of length given by number or a single object if number<1

astropysics.utils.io.get_data(dataurl, asfile=False, localfn=None)

Retrieves a data file from a remote source (usually the internet), and optionally caches that data locally. See set_data_store() for control of caching behavior, and set_data_download_reporter() for control of download progress reporting.

  • dataurl (str) – The URL of the data to be retrieved. If not a URL (i.e. does not have a ‘:’ somewhere), this will be interpreted as a local file name inside the astrpysics data directory.
  • asfile (bool) – If True, a file-like object is returned that can be used to access the data. Otherwise, a string with the full content of the file is returned.
  • localfn – The filename to use for saving (or loading, if the file is present) locally. If it is None, the filename will be inferred from the URL if possible. This file name is always relative to the astropysics data directory (see astropysics.config.get_data_dir()). This has no effect if set_data_store() is set to False.

A file-like object or a string (see asfile)

  • urllib2.URLError – If the dataurl request is a URL and cannot be found.
  • IOError – If the dataurl is requested as a local data file and not found.


The data accessed by this function is distinct from the data accessed via get_package_data(). Package data is crucial basic data included in the astropysics source distribution, while standard data is for larger or optional data files that are downloaded as needed.


Returns the current behavior of the download progress reporter for get_data(). See set_data_download_reporter() for details of the possible values.

Returns:The current value of the data store. Possible values are listed in set_data_download_reporter().

Returns the current behavior of the caching mechanism for get_data(). See set_data_store() for details of the possible values

Returns:The current value of the data store. Possible values are listed in set_data_store().

Use this function to load data files distributed with the astropysics source code.

Parameters:dataname (str) – The name of a file in the package data directory.
Returns:The content of the requested file as a string.


The data accessed by this function is distinct from the data accessed via get_data(). Package data is crucial basic data included in the astropysics source distribution, while standard data is for larger or optional data files that are downloaded as needed.

astropysics.utils.io.load_all_deimos_spectra(dir='.', pattern='spec1d*', extraction='horne', smoothing=None, verbose=True)

loads all deimos spectra found in the specified directory that match the requested pattern and returns a list of the file names and the Spectrum objects.

extraction and smoothing are the same as for load_deimos_spectrum

verbose indicates if information should be printed

returns dictionary mapping file names to Spectrum objects

astropysics.utils.io.load_deimos_spectrum(fn, plot=False, extraction='horne', retdata=False, smoothing=None)

extraction type can ‘horne’ or ‘boxcar’

if smoothing is positive, it is gaussian sigmas, if negative, boxcar pixels

returns Spectrum object with ivar, [bdata,rdata]


This function loads a file in the Tipsy ASCII format (http://www-hpcc.astro.washington.edu/tipsy/man/readascii.html) and outputs a dictionary with entries for grouped data

astropysics.utils.io.load_wcs_spectrum(fn, fluxext=None, errext=None, hdrext=None, errtype='err')

Loads a linear spectrum from a fits file with WCS keywords ‘CD1_1’ or ‘CDELT1’ and ‘CRVAL_1’.

  • fn (string) – File name of fits file with spectrum to load.
  • fluxext (int or None) – Fits extension to use for the spectrum or None to use the first that has the WCS form (i.e. ‘CD1_1’/’CDELT1’ and ‘CRVAL_1’ present).
  • errext (int of None) – Fits extension to use for the errors or None to skip errors.
  • hdrext (int or None) – The fits extension to use to look for the header keywords - if None, this will be the same as fluxext, or ignored if fluxext is None.
  • errtype (str) – Form of error data if present: ‘err’,’ierr’,’var’, or ‘ivar’

An astropysics.spec.Spectrum object

  • ValueError – If the fits file is missing necessary keywords
  • IOError – If fluxext is None and no HDUs have the correct form
astropysics.utils.io.loadtxt_fixed_column_fields(fn, fncol=None, skiprows=0, comments='#', columnlinestart='#:', columnsep=' ', maxcols=None, oneindexed=True, maskedarray=False, updatedict=None)

Loads a fixed column data file using FixedColumnDataParser.

  • fncol – The file from which to load the column data. If None, it will assumed to be the same as fn.
  • updatedict – A dictionary that will be updated with the loaded data such that the keys are the field names and the values are arrays with the data. If None, it will be ignored.

For other arguments see extra documentation below and FixedColumnDataParser.

Additional keywords are passed into numpy.loadtxt().

Returns:recarray or a regular ndarray with data loaded from the text file.

To match the default format, data files to load should look like:

#:col1 1 4 int
#:col2 6 6 bool
#:col3 8 10 S3
#:col4 12 16 float {'------':''}
# could have a comment here if you want
1239 1 abc 12.325
2489 0 zyx ------
9526 1 qwe 89632.

FixedColumnDataParser.parseFile() describes the return types and maskedarray parameter.

For details on arguments controlling how columns are inferred, See FixedColumnDataParser.addColumnsFromFile().

astropysics.utils.io.loadtxt_text_fields(fn, fieldline=1, asrecarray=True, updatedict=None, **kwargs)

Load a text file into a structured numpy array where the field names and types are inferred from a line (typically the first) in the file.

It must begin with a comment (specified by the ‘comments’ keyword) and have fields (sperated by the delimiter, default whitespace) composed of a colon-seperated field name and valid numpy dtype (e.g. ‘f’ for floats or ‘i’ for integers).

If a third colon-seperated component is present, it is a python expression (with no spaces) that can be used to derive the value for that column given values from the other columns. numpy functions can be used with the prefix ‘np’.

An example field line might be:

#data1:f derived:f:data1+data2**2 data2:i

This will result in a 3-field record array with fields ‘data1’, ‘derived’, and ‘data2’, where data1 and data2 are the columns of the input file.

  • fn (string) – The file name to load.
  • fieldline (int) – The line number that has the field information.
  • asrecarray (bool) – If True, returns a numpy.recarray. Otherwise, returns a regular numpy.ndarray with a structured dtype.
  • updatedict (dict or None) – A dictionary that will be updated with the loaded data such that the keys are the field names and the values are arrays with the data. If None, this will be ignored.

extra keywords are passed into numpy.loadtxt()

Returns:A numpy record array or regular array with data from the text file.


The updatedict parameter is seful for injecting the loaded file into the local namespace: loadtxt_text_fields(...,updatedict=locals()).

astropysics.utils.io.open_with_pyfits(*args, **kwargs)

This is a convinience function for pyfits, allowing the following usage in python 2.5 or above (in 2.5, from __future__ import with_statement is needed at the beginning of the file):

from astropysics.utils.io import open_with_pyfits

with open_with_pyfits('filename') as f:
    h = f[0].header
    d = f[0].data

    # ... do something more with the file...

# at this indent level, the pyfits file is now closed. It will also be
# closed if an exception is thrown.

The arguments are the same as for pyfits.open().


Sets the behavior of the download progress reporter for get_data(). The value can subsequently be retrieved via get_data_download_reporter().

Parameters:reportprogress – If False, no progress reports go out while file is downloading. If True, a progress message is printed at the command line. Otherwise, this must be a callable.


If reportprogress is a callable, it will be called as func(progress,started,finished,url), where progress is either a float between 0 and 1 indicating the fraction of the file downloaded (if the file size is known), or the number of bytes downloaded if the file size is unknown (1 or larger). started is True if the download has begun, and finished is True when the download has finished. The function will always be called at the beginning as func(0,False,False) and once at the end as func(?,True,True). url is a string with the target URL.

Raises TypeError:
 If reportprogress is an inappropriate type


If set_data_store() has been set to False, no download progress will be reported, as the data will be retrieved immediately without download to a local file.


Sets the behavior of the caching mechanism when get_data() is called. The value can subsequently be retrieved via get_data_store().

Parameters:store – If True, when get_data() is called the data file will only be downloaded if that URL is accessed for the first time. If False, the data will always be retrieved from the remote source and not saved. If it is the string ‘refresh’, the file is downloaded regardless of whether or not it is present, but the downloaded version will be used in future calls where it is True.
Raises TypeError:
 If store is an inappropriate type