tawhiri package

Submodules

tawhiri.dataset module

Open a wind dataset from file by memory-mapping

Datasets downloaded from the NOAA are stored as large binary files that are memmapped into the predictor process and thereby treated like a huge array.

Dataset contains some utility methods to find/list datasets in a directory, and can open (& create) dataset files.

Note: once opened, the dataset is mmaped as Dataset.array, which by itself is not particularly useful. The tawhiri.download creates a numpy.ndarray from this mapping, and :mod:`tawhiri.interpolate casts it (via a memory view) to a pointer in Cython.

class tawhiri.dataset.Dataset[source]

A wind dataset

__init__(ds_time, directory='/srv/tawhiri-datasets', new=False)[source]

Open the dataset file for ds_time, in directory

Parameters:
  • directory (string) – directory containing the dataset
  • ds_time (datetime.datetime) – forecast time
  • new (bool) – should a new (blank) dataset be created (overwriting any file that happened to already be there), or should an existing dataset be opened?

See also

open_latest()

After initalisation, the following attributes are available:

array

A mmap.mmap object; the entire dataset mapped into memory.

ds_time

The forecast time of this dataset (datetime.datetime).

…and this method:

close()[source]

Close the dataset

This deletes array, thereby releasing (a) reference to it. Note that other objects may very well hold a reference to the array, keeping it open.

(The file descriptor is closed as soon as the dataset is mapped.)

The following attributes are class attributes:

shape = (65, 47, 3, 361, 720)

The dimensions of the dataset

Note len(axes[i]) == shape[i].

axes

The values of the points on each axis: a 5-(named)tuple (hour, pressure variable, latitude, longitude).

For example, axes.pressure[4] is 900—points in cells dataset.array[a][4][b][c][d] correspond to data at 900mb.

element_type = 'float32'

The data type of dataset elements

element_size = 4

The size in bytes of element_type

size = 9528667200

The size in bytes of the entire dataset

SUFFIX_GRIBMIRROR = '.gribmirror'

The filename suffix for “grib mirror” files

DEFAULT_DIRECTORY = '/srv/tawhiri-datasets'

The default location of wind data

These “utility” class methods are available:

classmethod filename(ds_time, directory='/srv/tawhiri-datasets', suffix='')[source]

Returns the filename under which we expect to find a dataset

... for forecast time ds_time, in directory with an optional suffix

Parameters:
  • directory (string) – directory in which dataset resides/will reside
  • ds_time (datetime.datetime) – forecast time
Return type:

string

classmethod listdir(directory='/srv/tawhiri-datasets', only_suffices=None)[source]

Scan for datasets in directory

... with filenames matching those generated by filename() and (optionally) filter by only looking for certian suffices.

Parameters:
  • directory (string) – directory to search in
  • only_suffices (set) – if not None, only return results with a suffix contained in this set
Return type:

(named) tuples (dataset time, suffix, filename, full path)

classmethod open_latest(directory='/srv/tawhiri-datasets', persistent=False)[source]

Find the most recent datset in directory, and open it

Parameters:
  • directory (string) – directory to search
  • persistent (bool) – should the latest dataset be cached, and re-used?
Return type:

Dataset

tawhiri.download module

Wind tahwiri.wind.Dataset Downloader

Downloaded data arrives in GRIB format, although three quarters of the records in the downloaded file are ignored. The records that are used can also be written to a new grib file as they are unpacked (which is therefore somewhat smaller, as it is still compressed and only contains the useful bits).

class tawhiri.download.DatasetDownloader(directory, ds_time, timeout=120, first_file_timeout=600, bad_download_retry_limit=3, write_dataset=True, write_gribmirror=True, deadline=None, dataset_host='ftp.ncep.noaa.gov', dataset_path='/pub/data/nccf/com/gfs/prod/gfs.{0}/')[source]

Bases: builtins.object

close(move_files=None)[source]
download()[source]
open()[source]
class tawhiri.download.DownloadDaemon(directory, num_datasets=1)[source]

Bases: builtins.object

clean_directory()[source]
run()[source]
tawhiri.download.main()[source]
tawhiri.download.unpack_grib(filename, dataset=None, checklist=None, gribmirror=None, assert_hour=None, file_checklist=None, callback=None)[source]

Unpack the GRIB file at filename

... into dataset

... setting the cell corresponding to each GRIB record in checklist
to True (see make_checklist())

... copying the GRIB records we care about into gribmirror

... checking that the forecastTime matches assert_hour

... checking that the GRIB records in this file (that we care about)
exactly match the set of (forecast time, level, variable) tuples in file_checklist
... calling callback after processing each record, with arguments
(pass, location indices, location names) (where location is (forecast time, level, variable))

callback must _not_ edit dataset, checklist or gribmirror, or yield to a greenlet that will (hence DownloadDaemon.unpack_lock).

callback is mainly used to yield to other greenlets doing IO (i.e., downloading other files) while we do the CPU intensive task of unpacking GRIB data.

The data is unpacked in two passes; the first

  • checks the shape and forecast time of each record,
  • checks the axes of the first record (i.e., the latitudes and longitudes each point corresponds to) - this is really slow, so is only done once,
  • checks the contents of the file exactly matches file_checklist (excluding records we don’t care about),
  • checks that no elements of checklist that we’re about to unpack are already marked as having been unpacked (i.e., True).

The second pass copies the data in each record to its correct location in dataset, writes a copy to gribmirror and marks the correct place in checklist as True.

ValueError is raised in case of any problems.

tawhiri.interpolate module

tawhiri.interpolate.make_interpolator(dataset)

Produce a function that can get wind data from dataset (a tawhiri.dataset.Dataset).

This function returns a closure:

closure.f(hour, alt, lat, lng)
Returns:delta lat, lon and alt

See also

implementation

See also

wind_data

The interpolation code is not documented here. Please see the source on GitHub.

tawhiri.models module

Provide all the balloon models, termination conditions and functions to combine models and termination conditions.

tawhiri.models.float_profile(ascent_rate, float_altitude, stop_time, dataset)[source]

Make a model chain for the typical floating balloon situation of ascent at constant altitude to a float altitude which persists for some amount of time before stopping. Descent is in general not modelled.

tawhiri.models.make_any_terminator(terminators)[source]

Return a terminator that terminates when any of terminators would terminate.

tawhiri.models.make_burst_termination(burst_altitude)[source]

Return a burst-termination criteria, which terminates integration when the altitude reaches burst_altitude.

tawhiri.models.make_constant_ascent(ascent_rate)[source]

Return a constant-ascent model at ascent_rate (m/s)

tawhiri.models.make_drag_descent(sea_level_descent_rate)[source]

Return a descent-under-parachute model with sea level descent sea_level_descent_rate (m/s). Descent rate at altitude is determined using an altitude model courtesy of NASA: http://www.grc.nasa.gov/WWW/K-12/airplane/atmosmet.html

For a given altitude the air density is computed, a drag coefficient is estimated from the sea level descent rate, and the resulting terminal velocity is computed by the returned model function.

tawhiri.models.make_elevation_data_termination(dataset=None)[source]

A termination criteria which terminates integration when the altitude goes below ground level, using the elevation data in dataset (which should be a ruaumoko.Dataset).

tawhiri.models.make_linear_model(models)[source]

Return a model that returns the sum of all the models in models.

tawhiri.models.make_time_termination(max_time)[source]

A time based termination criteria, which terminates integration when the current time is greater than max_time (a UNIX timestamp).

tawhiri.models.make_wind_velocity(dataset)[source]

Return a wind-velocity model, which gives lateral movement at the wind velocity for the current time, latitude, longitude and altitude. The dataset argument is the wind dataset in use.

tawhiri.models.sea_level_termination(t, lat, lng, alt)[source]

A termination criteria which terminates integration when the altitude is less than (or equal to) zero.

Note that this is not a model factory.

tawhiri.models.standard_profile(ascent_rate, burst_altitude, descent_rate, wind_dataset, elevation_dataset)[source]

Make a model chain for the standard high altitude balloon situation of ascent at a constant rate followed by burst and subsequent descent at terminal velocity under parachute with a predetermined sea level descent rate.

Requires the balloon ascent_rate, burst_altitude and descent_rate, and additionally requires the dataset to use for wind velocities.

Returns a tuple of (model, terminator) pairs.

tawhiri.solver module

tawhiri.solver.solve(t, lat, lng, alt, chain)

Solve from initial conditions t, lat, lng and alt, using models and termination criteria from chain, an iterable of (model, terminator) pairs which make up each stage of the flight.

tawhiri.api module

Provide the HTTP API for Tawhiri.

exception tawhiri.api.APIException[source]

Bases: builtins.Exception

Base API exception.

status_code = 500
exception tawhiri.api.InternalException[source]

Bases: tawhiri.api.APIException

Raised when an internal error occurs.

status_code = 500
exception tawhiri.api.InvalidDatasetException[source]

Bases: tawhiri.api.APIException

Raised if the dataset specified in the request is invalid.

status_code = 404
exception tawhiri.api.NotYetImplementedException[source]

Bases: tawhiri.api.APIException

Raised when the functionality has not yet been implemented.

status_code = 501
exception tawhiri.api.PredictionException[source]

Bases: tawhiri.api.APIException

Raised if the solver raises an exception.

status_code = 500
exception tawhiri.api.RequestException[source]

Bases: tawhiri.api.APIException

Raised if request is invalid.

status_code = 400
tawhiri.api.handle_exception(error)[source]

Return correct error message and HTTP status code for API exceptions.

tawhiri.api.main()[source]

Single API endpoint which accepts GET requests.

tawhiri.api.parse_request(data)[source]

Parse the request.

tawhiri.api.run_prediction(req)[source]

Run the prediction.

Module contents

Tawhiri is trajectory prediction software for high altitude balloons.

See http://www.cusf.co.uk/wiki/tawhiri:start for further details.