The objcat module contains classes and functions for generating catalogs of objects. This catalog is unique in that it stores source information and can dynamical deriv quantities that are dynamically updated as their sources are changed. It also provides mechanisms for safely saving these object catalogss in a database, and someday soon will include a web interface to interact with the catalog via the internet.
Fundamentally, the catalog should be thought of as a tree (this kind, rather than this one) of FieldNode objects. Normally, a Catalog object acts as the root.
todo:add examples!
Bases: astropysics.objcat.CatalogNode
This object is the superclass for nodes of a catalog that perform an action but do not store data. Thus, they have a parent, but no children.
Subclassing
If an action is desired without inserting the ActionNode into the graph, the following method is typically used:
output = SomeActionClass(None,param1=foo,param2=bar)(targetnode)
| Parameters: |
|
|---|
Bases: astropysics.objcat.StructuredFieldNode
A StructuredFieldNode with the basic fields for an astronomical object.
Bases: astropysics.objcat.CatalogNode
This class represents a catalog of objects or catalogs.
A Catalog is essentially a node in the object tree that is typically (although not necessarily) the root of the catalog tree.
Iterator access is over the children, as is container testing and index access.
Attributes can also be accessed as catobj[attrname] (this allows access to Catalog attributes in the same way as FieldNodes)
Walk through the tree starting from this object and generate an array of the values for the requested fieldname if any FieldNode`s are in this :class:`Catalog.
The other arguments are for FieldNode.extractFieldAtNode() :
Searches the Catalog and finds all field names present in the children
Searches the Catalog and finds all objects with the requested name
Inserts this Catalog into another catalog heirarchy in the place of a node insertnode . This will replace the current parent, if any.
| Parameters: | insertnode (CatalogNode) – The node to insert this Catalog in the place of. This Catalog becomes the parant of insertnode . |
|---|
Searches the Catalog and finds all objects with the requested name
merges the given FieldNode into the catalog by reassigning all of it’s children to this Catalog
testfunc is a function that will be called on the node to be transferred, and if it is True, the node will be moved, or if not it will be left alone. If it is None, this test will be skipped
if skipdup is True, any nodes that already have an object of the same name will not be transferred
returns the number of objects added
Bases: object
This object is the superclass for all elements/nodes of a catalog with both parents and children.
This is an abstract class that must have its initializer overriden.
Adds a node as a child of this node. Note that this will replace the added node’s current parent (if any)
a string uniquely identifying the object in its catalog heirarchy
Load a previously saved CatalogNode.
| Parameters: | file – A file name or file-like object for a file storing a pickled CatalogNode. |
|---|---|
| Raises TypeError: | |
| If file does not contain a CatalogNode. | |
this gives the number of total nodes at this point in the tree (including self - e.g. a leaf in the tree returns 1)
Removes the requested child from this node, leaving it an orphan (i.e. it’s parent is None)
If child is not present, a ValueError will be raised
Change the order of the children
| Parameters: |
|
|---|
save this node as a file with the given name or file-like object
if savechildren is True, the entire subtree will be saved, if False, just this node
Note that the parent and everything above this point will NOT be saved (when reloaded the parent will be None)
extra kwargs are passed into utils.fpickle
This function walks through the object and all its children, executing func(CatalogNode) and returning a list of the return values
| Parameters: |
|
|---|---|
| Returns: | A list with the return values of visitfunc at each visited node. |
Bases: exceptions.Exception
This exception indicates a cycle was detected in some graph-like structure
Bases: exceptions.Warning
This warning indicates a cycle was detected in some graph-like structure
Bases: astropysics.objcat.Source
This class holds weak references to the Field`s that are necessary to generate values such as in :class:`DerivedValue.
This source must know the CatalogNode that it is expected to inhabit to properly interpret string codes for fields. Otherwise, the input fields must be Field objects.
If locator strings are given to depfields, these strings follow a simple mini-language to locate their target field. If they are strings without the ‘-‘ character, they will simply be treated as names within the same node. If a ‘-‘ is present, the text before the ‘-‘ follows the form “^^^.0.st” where a ‘^’ character indicates that the path should go up to the parent of the current node, ”.0” indicates to go to the first child, and ”.st” indicates that a LinkField with the name ‘st’ should be followed. Once the correct node is located, the text after the “-” indicates the name of the field on that node.
Thus,’^^^.0.st-field’ would go three needs up, into the first child, then into the node from a link named ‘st’, and would return the value of the field named “field”.
gets the values of the dependent field.
the return value is a list of the values in the fields if geterrs is False, or (value,upperr,lowerr) if it is True
The CatalogNode for dereferencing source names
this relinks all dead weakrefs using the dependancy strings and returns all of the references or raises a ValueError if any of the strings cannot be dereferenced
Bases: astropysics.objcat.FieldValue
A FieldValue that derives its value from a function of other FieldValues.
The values to use as arguments to the function are initially set through the default values of the function, and can be either references to fields or strings used to locate the dependencies in the catalog tree, using the sourcenode argument as the current source (see DependentSource for details of how dependent values are dereferenced)
Alternatively, the initializer argument flinkdict may be a dictionary of link values that overrides the defaults from the function.
if ferr is True, the return value of f should be a 3-sequence of the form (value,uppererror,lowererror), and the arguments of f will be passed as (depvalue,uerr,lerr) from the source FieldValues.
The class attribute failedvalueaction determines the action to take if a an exception is encountered while deriving the value and can be:
raise an exception (or pass one along)
issue a warning when it occurs, but continue execution with the value returned as None
the value will be returned as None but the value will be left invalid
the value will be returned as None and will be marked valid
f is the function to use for deriving the value - must have default field links for every argument if flinkdict is None.
sourcenode specifies the current node to allow for links to be strings instead of explicit references to Fields.
flinkdict maps argument names to links, overriding the default values of the arguments of f
ensure that the value of this FieldValue is of the requested Type (or None to accept anything). Any mismatches will throw a TypeError
A function like Field.notifyValueChange
This marks this derivedValue as incorrect
The current location in the Catalog tree
Bases: _abcoll.MutableSequence
This class represents an attribute/characteristic/property of the FieldNode it is associated with. It stores the current value as well as all the other possible values.
The values, sources, and default properties will return the actual values contained in the FieldValues, while currentobj and iterating over the Field will return FieldValue objects. Calling the Field (no arguments) will return the current value itself
usedef specified if the default should be set – if True, defaultval will be used, if None, a None defaultval will be ignored but any other will be recognizd, and if False, no default will be set
The field should have a name, and can optionally be given a type. If the name is None, it will be inferred from the node it is placed inside.
The default value is the FieldValue that has a the None Source - this property is the default value itself, not the fieldValue object
returns a list of all the DerivedValue objects
Insert a value into this Field.
| Parameters: |
|
|---|
the node to which this Field belongs
notifies all registered functions that the value in this field has changed
(see registerNotifier)
returns a list of all the ObservedValue objects except the default
this registers a function to be called when the value changes or is otherwise rendered invalid. The notifier will be called as notifier(oldvalobj,newvalobj) BEFORE the value change is finalized.
returns a string with the current value instead of the list of values (the behavior of str(Field_obj)
Selects the type to enforce for this field. if None, no type-checking will be performed if a numpy dtype, the value must be an array matching the dtype can also be a sequence of types (accepts all) or a function that will be called directly on the function that returns True if the type is valid.
Bases: astropysics.objcat.CatalogNode, _abcoll.Sequence
A node in the catalog that has Fields.
Note that for these subclasses, attribute access (e.g. node.fieldname) accesses the Field object, while mapping or sequence-style access (e.g node[‘fieldname’] or node[1]) directly accesses the current value of the field (or None if there is no value). This means that iterating over the object will also give values. To iterate over the Field objects, use the fields() method.
Further, setting an attribute (e.g. node[‘fieldname’] = value) has different behavior depending on the type of the input value:
Create a new fieldNode with the provided parent.
kwargs are assigned as node values as self[kwargkey] = kwargvalue
Walk through the tree starting from this object.
If the kwarg siblings is set to True, this will also extract the siblings (e.g. the parent subtree, except for the parent itself), and this overwrites includeself.
The other arguments are for FieldNode.extractFieldAtNode() :
This will walk through the tree starting from the FieldNode given by the node argument and generate an array of values for the specified fieldname.
| Parameters: |
|
|---|---|
| Returns: | If missing is ‘mask’, the below return values that are wrapped as a (returnval,mask) tuple. If errors are returned, they are (uppererror,lowererror) arrays, i.e. +/- . Can be:
|
this yields an iterator over all of the Field objects (rather than their values, as regular sequence access does)
Generate a string with field:value pairs for all fields using sep as the seperator between the strings
Searches through the tree and returns all nodes for which a particular field name has the requested value
Searches through the tree and returns all nodes for which a particular field name has the requested value
Sets a string or Source object passed in as the src argument as the current source for the requested FieldNode and it’s subtree.
missing can be:
raise a ValueError if there is no value with that source
give a warning if the value is missing
do nothing
Sets a string or Source object passed in as the src argument as the current source for the requested FieldNode and it’s subtree.
missing can be:
raise a ValueError if there is no value with that source
give a warning if the value is missing
do nothing
traversal is the type of traversal to perform on the subtree. See FieldNode.visit() for values.
If siblings is True, the function applies to all the children of this FieldNode's parent.
Sets the given src string or Source object as the current source for this FieldNode and it’s subtree. Based on FieldNode.setToSourceAtNode() :
Bases: object
Superclass for values that belong to a field.
ensure that the value of this FieldValue is of the requested Type (or None to accept anything). Any mismatches will throw a TypeError
The Source for this FieldValue
The value stored by this FieldValue
Bases: astropysics.objcat.ActionNode
This ActionNode will generate a networkx.DiGraph object representing the node and its children. Note that networkx must be installed for this to work.
When this object is called, it returns a networkx.DiGraph object.
If True, matplotlib.pyplot.clf() will be called before the graph is drawn.
A dictionary of keywords that are passed into networkx.draw() when drawing the graph.
Sets the layout used in drawing the graph. If a string, it specified a graphviz program name (e.g. ‘dot’,’neato’,’twopi’) use to position the graph nodes. It can also be a callable, in which case it will be called with the graph as the first argument to get the positions. This is intended to be used with networkx.layout functions, but a user-provided callable should work fine, too assuming it outputs correct positions.
A string specifying the file name to save the graph to when the node is called. If None, the graph will not be saved.
If True, matplotlib.pyplot.show() will be called after the graph is drawn.
Sets the traversal type for generating the graph - is any valid CatalogNode.visit() traversal argument.
Bases: astropysics.objcat.Field
A Field that only has values that point to other nodes.
| Parameters: |
|
|---|
Note
If the current linked node has been removed, calling this will change the current object to the next valid link.
Bases: astropysics.objcat.FieldValue
A FieldValue that points to a node.
Bases: astropysics.objcat.ActionNode
This ActionNode uses matplotlib to generate plots.
This class is an abstract class, requiring subclasses to implement the makePlot() method.
Keyword arguments when this action is called will be used to change the value of the object attribute with the same name as the keyword temporarily for only that call.
The defaultattrs class variable can be set for a class, and it must be a dictionary where the keys are strings. It will be used to initialize class variables, where the defaults are given by the values of defaultattrs .
The following example illustrates the use of the of a MatplotAction. Define the PlotXY by doing:
class PlotXY(MatplotAction):
defaultattrs = dict(logx=False)
def makePlot(self,node):
from numpy import log10
x,y = node.extractField('x,y')
if self.logx:
x = log10(x)
scatter(x,y)
xlabel('x')
ylabel('y')
Then, if c is a Catalog with children that have x and y fields):
PlotXY(c)()
will produce a plot of x vs y, as well as adding the PlotXY object as a child of c. similarly:
PlotXY(logx=True)(c)
Will produce a plot of log(x) vs. y for c‘s chilren, but will not add the PlotXY object as a child of c.
| Parameters: |
|
|---|
Additional keyword can be used to set attributes that have defaults given by defaultattrs.
This method does the actual plotting.
| Parameters: | node – The node this action is associated with. |
|---|
Bases: astropysics.objcat.ObservedValue
This value is an observed value with errorbars
The value is either (value,err) or (value,uperr,lowerr)
ensure that the value and errors are of the requested Type (or None to accept anything). Any mismatches will raise a TypeError
Errors on the measurement as a uppererror,lowererror tuple, or 0,0 if no errors are present.
Bases: astropysics.objcat.FieldValue
This value is an observed or otherwise measured value for the field with the associated Source.
Bases: astropysics.objcat.MatplotAction
Generates 2D plots by extracting fields from the catalog for two specified variables.
Can be used to filter out unwanted values by setting to a callable that takes in a node and reurns True if it should be processed, or False if not. See FieldNode.extractFieldAtNode() for more details.
Determines if a particular axis should be in log (base-10) form. If None, no axes will be log, otherwise, should be either ‘x’, ‘y’, or ‘xy’, specifying which axes should be made logarithmic.
Determines the order of the plotted arrays. If ‘sortx’ or ‘sorty’, the values will be sorted on the x and y axis, respectively. Otherwise, specifies the traversal argument for FieldNode.extractFieldAtNode() .
Additional keywords to be passed into the plotting command, or None for defaults.
A string specifying the type of plot to make. Must be one of:
- ‘plot’ - matplotlib.pyplot.plot()
- ‘errorbar’ - matplotlib.pyplot.errorbar()
- ‘scatter’ - matplotlib.pyplot.scatter()
- ‘hist’ - matplotlib.pyplot.hist() (yaxisname is ignored).
A string specifying the name of the field to use for the x-axis.
A string specifying the name of the field to use for the y-axis.
Bases: astropysics.objcat.Field
This field represents the Spectral Energy Distribution of this object - e.g. a collection of Spectra or Photometric measurements
determines the magnitude or flux in the requested band.
The first photometric measurement in this SEDField that has the band will be used - if not present, it will be computed from the first Spectrum with appropriate overlap. If none of these are found, a ValueError will be raised
if asflux is True, the result will be returned as a flux - otherwise, a magnitude is returned.
asdict returns a dictionary of results - otherwise, a sequence will be returned (or a scalar if only one band is requested)
the generates a astropysics.spec.Spectrum object that represents all the information contained in this SEDField
return a copy of the values masked from the full SED
mask the value (either index or source name) to not appear in the full SED
Generates a plot of the SED of this object.
colors is a tuple of colors as (spec,phot,specerr,photerr,other)
The units to use in the objects of this SED - see astropysics.spec.HasSpecUnits for valid units
unmask the value (either index or source name) to appear in the full SED
unmask all values (all appear in full SED)
Bases: object
A source for an observation/measurement/value. Note that there is always only one instance if a source at a given time - any two Sources with the same source string are the same object
The source can optionally include a URL location to look up metadata like authors, publication date, etc (location property)
the constructor string can be of the form ‘str/loc’ in which case loc will be interpreted as the bibcode (see setBibcode() for valid forms) if it is not specified in the argument. If it is ‘str//loc’, the loc will not be validated (e.g. it is assumed to be a correct ADS abstract code).
The abstract for this Source
The URL for the ADS abstract of this Source
The author list for this Source as a list of strings
Generates and returns BibTeX bibliography for Source objects.
| Parameters: | |
|---|---|
| Returns: | bibstr,fail where bibstr is a string with the BibTeX content and fail is a list of Source objects for which the lookup failed (usually because the Source has no bibcode) |
| Raises TypeError: | |
If sources is not a list of Source objects or ‘all’ |
|
this clears the cache of the specified adscode, or everything, if the adscode is None
The publication date of this Source
Retrieves the BibTeX entry for this Source. This data is retrieved from ADS and thus requires a network connection if it is not already cached.
| Raises SourceDataError: | |
|---|---|
| If no bibcode is defined (see setBibcode()). | |
| Returns: | A string with the BibTeX formatted entry for this source. |
Returns the ADS bibliographic code for this source.
| Returns: | A string with the ADS bibcode or None if no bibcode is defined. |
|---|
The keywords for this source, and the type of the keywords, if present
Opens the ADS abstract for this source in a web browser.
| Parameters: | opentype – The type of web browser window to open. Can be:
|
|---|
Additionaly keyword arguments will be passed into the relevant function.
Sets the ADS bibliographic code for this Source
| Parameters: | val – The location reference for this bibcode. It may be any of the following:
|
|---|---|
| Raises: |
|
The publication title for this Source
This is used to disable or enable the cache for ADS lookups - if the enable argument is True, the cache is enable (or unaltered if it is already active)) and if it is False, it will be disabled
note that if the cache is disabled, all entries are lost
Bases: exceptions.Exception
This exception indicates a problem occured while trying to retrieve external Source-related data
Bases: astropysics.objcat.FieldNode
This class represents a FieldNode in the catalog that follows a particular data structure (i.e. a consistent set of Fields). It is meant to be subclassed to define generic types of objects in the catalog.
The fields and names are inferred from the class definition and hence the class attribute name must match the field name. Any FieldValues present in the class objects will be ignored.
The checkonload class attribute determines the action to take if an unaltered StructuredFieldNode is loaded and found to be inconsistent with the current class definition. It may be:
Do no checking.
show a warning when the loaded object is inconsistent with the definition
raise a ValueError when the loaded object is inconsistent with the definition
silently revert to the current definition if inconsistencies are found
issue a warning, then revert
If True, the object no longer matches the specification given by the class. Note that this will remain True even if the offending fields are returned to their correct state.
This method is to be used as a function decorator to generate a field with a name matching that of the function. Note that the function should NOT have self as the leading argument
Arguments are the same as for the Field constructor, except that leftover kwargs are passed in as links that override the defaults of the function.
The docstring of the function will be taken as the field’s description.
Revert this object back to the standard Fields for the class. Any deleted fields will be populated with the class Default Value any attributes that match the names of deleted Fields will be overwritten
TODO:test
Bases: astropysics.objcat.ActionNode
This ActionNode generates and returns formatted text tables.
Calling this action generates the and returns the table, and saves it if the savefile attribute is not None:
| Parameters: |
|
|---|---|
| Type: | int array or None |
| Returns: | A string with the table data |
A sequence of field names to use to extract the data. If the special name ‘source’ is given, the sources used for each data set are included.
A strings for a table caption or None to have no caption.
A sequence of strings giving more detailed information for for the fields - must match arrnames or be None to ignore.
Include errorbar values in the table, if present.
The formatting specifier for each of the columns. Must be a single string or a sequence that matches arrnames.
Determines tf a particular axis should be in log (base-10) form. If None, no axes will be log, otherwise, must be a sequence of bools matching arrnames that is True if the column should have the log of its data taken.
Determines the order of the plotted arrays. If ‘sort#’, the values will be sorted on the axis specified by #. Otherwise, specifies the traversal argument for FieldNode.extractFieldAtNode() .
A string specifying the file name to save the table to when the node is called. If None, the table will not be saved.
The format for the text table. Can be ‘ascii(sep)’ for a table with sep as the seperator and newlines seperating the rows, ‘ascii’ is the same with space as the seperator, ‘latex’ for a LaTeX-style table, or ‘deluxetable’ for AASTex-style deluxtables.
Generates a catalog of nodes from the array of data.
See arrayToNodes for values,source,fields, and covnerter arguments.
parent is the object to use as the parent for all of the nodes (which will be returned, a string (in which case a Catalog object will be created and returned), or None (return value will be a sequence of nodes)
nodetype is the class to use to create the nodes (usually a subclass of StructuredFieldNode)
filter is a function that will be called on the array row and if it returns True, a node will be created, and if False, that row will be skipped
namefield is the field to use to apply the name of the node from the index of the array row. nameconv can be:
if namefiled is None, no name will be applied
Applies values from an array to CatalogNodes.
values must be a 2-dimensional array or a 1-dimensional structured array. If a 2D array, the first dimension should be the fields, while the second dimension should be the nodes to be created.
source is a Source object or a string that will be converted to a Source object
nodes is either a container of nodes to visit or a CatalogNode in which all FieldNodes within will be used as the nodes to visit
fields is either a sequence of field names (must be at least as long as the array’s second index dimension) or a mapping from indecies to field names or structured array field names to FieldNode field names
if errors is not None, it should either be an object matching values in shape, or a tuple (upper,lower) and will be applied as the errors on the values
matcher is a callable called as matcher(arrayrow,node) - if it returns False, the array will be matched to the next node - if True, the array values will be applied to the node
converters are either a sequence of callables or a mapping from indecies to callables or structured array field names to callables that will be called on each array element before being added to the FieldNode as converter(value). If errors are provided, the converter will be called as converter((value,uerr,lerr)). If converters is None, no converting is performed.
namefield is a field name that will be set to a unique code of the form ‘src-i’ where i is element index of the array row (or None to apply no name). It can also be a 2-tuple (name,converter) where converter is a callable of the form converter(i) that should return the value to apply to the field.
This method is to be used as a function decorator to generate a field with a name matching that of the function. Note that the function should NOT have self as the leading argument
Arguments are the same as for the Field constructor, except that leftover kwargs are passed in as links that override the defaults of the function.
The docstring of the function will be taken as the field’s description.
Load a previously saved CatalogNode.
| Parameters: | file – A file name or file-like object for a file storing a pickled CatalogNode. |
|---|---|
| Raises TypeError: | |
| If file does not contain a CatalogNode. | |
save the specified node as a file with the given name or file-like object
if savechildren is True, the entire subtree will be saved, if False, just this node
Note that the parent and everything above this point will NOT be saved (when reloaded the parent will be None)