********************************* Cubes 0.6 to 0.10.2 Release Notes ********************************* 0.10.2 ====== Summary: * many improvements in handling multiple hierarchies * more support of multiple hierarchies in the slicer server either as parameter or with syntax ``dimension@hierarchy``: * dimension values: ``GET /dimension/date?hierarchy=dqmy`` * cut: get first quarter of 2012 ``?cut=date@dqmy:2012,1`` * drill-down on hierarchy with week on implicit (next) level: ``?drilldown=date@ywd`` * drill-down on hierarchy with week with exlpicitly specified week level: ``?drilldown=date@ywd:week`` * order and order attribute can now be specified for a Level * optional safe column aliases (see docs for more info) for databases that have non-standard requirements for column labels even when quoted Thanks: * Jose Juan Montes (@jjmontesl) * Andrew Zeneski * Reinier Reisy Quevedo Batista (@rquevedo) New Features ------------ * added `order` to Level object - can be ``asc``, ``desc`` or None for unspecified order (will be ignored) * added `order_attribute` to Level object - specifies attribute to be used for ordering according to `order`. If not specified, then first attribute is going to be used. * added hierarchy argument to `AggregationResult.table_rows()` * `str(cube)` returns cube name, useful in functions that can accept both cube name and cube object * added cross table formatter and its HTML variant * ``GET /dimension`` accepts hierarchy parameter * added `create_workspace_from_config()` to simplify workspace creation directly from slicer.ini file (this method might be slightly changed in the future) * `to_dict()` method of model objects now has a flag `create_label` which provides label attribute derived from the object's name, if label is missing * #95: Allow charset to be specified in Content-Type header SQL: * added option to SQL workspace/browser ``safe_labels`` to use safe column labels for databases that do not support characters like ``.`` in column names even when quoted (advanced feature, does not work with denormalization) * browser accepts include_cell_count and include_summary arguments to optionally disable/enable inclusion of respective results in the aggregation result object * added implicit ordering by levels to aggregate and dimension values methods (for list of facts it is not yet decided how this should work) * #97: partially implemented sort_key, available in `aggregate()` and `values()` methods Server: * added comma separator for ``order=`` parameter * reflected multiple search backend support in slicer server Other: * added vim syntax highlighting goodie Changes ------- * AggregationResult.cross_table is depreciated, use cross table formatter instead * `load_model()` loads and applies translations * slicer server uses new localization methods (removed localization code from slicer) * workspace context provides proper list of locales and new key 'translations' * added base class Workspace which backends should subclass; backends should use workspace.localized_model(locale) * `create_model()` accepts list of translations Fixes ----- * browser.set_locale() now correctly changes browser's locale * #97: Dimension values call cartesians when cutting by a different dimension * #99: Dimension "template" does not copy hierarchies 0.10.1 ====== Quick Summary: * multiple hierarchies: * Python: ``cut = PointCut("date", [2010,15], hierarchy='ywd')`` * Server: ``GET /aggregate?cut=date@ywd:2010,15`` * Server drilldown: ``GET /aggregate?drilldown=date@ywd:week`` * added experimental result formatters (API might change) * added experimental pre-aggregations New Features ------------ * added support for multiple hierarchies * added ``dimension_schema`` option to star browser – use this when you have all dimensions grouped in a separate schema than fact table * added `HierarchyError` - used for example when drilling down deeper than possible within that hierarchy * added result formatters: simple_html_table, simple_data_table, text_table * added create_formatter(formatter_type, options ...) * AggregationResult.levels is a new dictionary containing levels that the result was drilled down to. Keys are dimension names, values are levels. * AggregationResult.table_rows() output has a new variable ``is_base`` to denote whether the row is base or not in regard to table_rows dimension. * added ``create_server(config_path)`` to simplify wsgi script * added aggregates: avg, stddev and variance (works only in databases that support those aggregations, such as PostgreSQL) * added preliminary implemenation of pre-aggregation to sql worskspace: * `create_conformed_rollup()` * `create_conformed_rollups()` * `create_cube_aggregate()` Server: * multiple drilldowns can be specified in single argument: ``drilldown=date,product`` * there can be multiple ``cut`` arguments that will be appended into single cell * added requests: ``GET /cubes`` and ``GET /cube/NAME/dimensions`` Changes ------- * **Important:** Changed string representation of a set cut: now using semicolon ';' as a separator instead of a plus symbol '+' * aggregation browser subclasses should now fill result's ``levels`` variable with ``coalesced_drilldown()`` output for requested drill-down levels. * Moved coalesce_drilldown() from star browser to cubes.browser module to be reusable by other browsers. Method might be renamed in the future. * if there is only one level (default) in a dimension, it will have same label as the owning dimension * hierarchy definition errors now raise ModelError instead of generic exception Fixes ----- * order of joins is preserved * fixed ordering bug * fixed bug in generating conditions from range cuts * ``AggregationResult.table_rows`` now works when there is no point cut * get correct reference in ``table_rows`` – now works when simple denormalized table is used * raise model exception when a table is missing due to missing join * search in slicer updated for latest changes * fixed bug that prevented using cells with attributes in aliased joined tables 0.10 ==== Quick Summary ------------- * Dimension defition can have a "template". For example: .. code-block:: javascript { "name": "contract_date", "template": "date" } * added table_rows() and cross_table() * added simple_model(cube_name, dimension_names, measures) *Incompatibilities:* use ``create_model()`` instead of ``Model(**dict)``, if you were using just ``load_model()``, you are fine. New Features ------------ * To address issue #8 create_model(dict) was added as replacement for Model(dict). Model() from now on will expect correctly constructed model objects. ``create_model()`` will be able to handle various simplifications and defaults during the construction process. * added ``info`` attribute to all model objects. It can be used to store custom, application or front-end specific information * preliminary implementation of ``cross_table()`` (interface might be changed) * ``AggregationResult.table_rows()`` - new method that iterates through drill-down rows and returns a tuple with key, label, path, and rest of the fields. * dimension in model description can specify another template dimension – all properties from the template will be inherited in the new dimension. All dimension properties specified in the new dimension completely override the template specification * added `point_cut_for_dimension` * added `simple_model(cube_name, dimensions, measures)` – creates a single-cube model with flat dimensions from a list of dimension names and measures from a list of measure names. For example: .. code-block:: python model = simple_model("contracts", ["year","contractor", "type"], ["amount"]) *Slicer Server:* * ``/cell`` – return cell details (replaces ``/details``) Changes ------- * creation of a model from dictionary through Model(dict) is depreciated, use `create_model(dict)` instead. All initialization code will be moved there. Depreciation warnings were added. Old functionality retained for the time being. (**important**) * Replaced `Attribute.full_name()` with `Attribute.ref()` * Removed `Dimension.attribute_reference()` as same can be achieved with dim(attr).ref() * `AggregationResult.drilldown` renamed to `AggregationResults.cells` Planned Changes: * `str(Attribute)` will return `ref()` instead of attribute name as it is more useful Fixes ----- * order of dimensions is now preserved in the Model 0.9.1 ===== **Summary**: Range cuts, denormalize with slicer tool, cells in ``/report`` query New Features ------------ * `cut_from_string()`: added parsing of range and set cuts from string; introduced requirement for key format: Keys should now have format "alphanumeric character or underscore" if they are going to be converted to strings (for example when using slicer HTTP server) * `cut_from_dict()`: create a cut (of appropriate class) from a dictionary description * `Dimension.attribute(name)`: get attribute instance from name * added exceptions: `CubesError`, `ModelInconsistencyError`, `NoSuchDimensionError`, `NoSuchAttributeError`, `ArgumentError`, `MappingError`, `WorkspaceError` and `BrowserError` *StarBrowser:* * implemented RangeCut conditions *Slicer Server:* * ``/report`` JSON now accepts ``cell`` with full cell description as dictionary, overrides URL parameters *Slicer tool:* * ``denormalize`` option for (bulk) denormalization of cubes (see the the slicer documentation for more information) Changes ------- * all ``/report`` JSON requests should now have queries wrapped in the key ``queries``. This was originally intended way of use, but was not correctly implemented. A descriptive error message is returned from the server if the key ``queries`` is not present. Despite being rather a bug-fix, it is listed here as it requires your attention for possible change of your code. * warn when no backend is specified during slicer context creation Fixes ----- * Better handling of missing optional packages, also fixes #57 (now works without slqalchemy and without werkzeug as expected) * see change above about ``/report`` and ``queries`` * push more errors as JSON responses to the requestor, instead of just failing with an exception Version 0.9 =========== Important Changes ----------------- Summary of most important changes that might affect your code: **Slicer**: Change all your slicer.ini configuration files to have ``[workspace]`` section instead of old ``[db]`` or ``[backend]``. Depreciation warning is issued, will work if not changed. **Model**: Change ``dimensions`` in ``model`` to be an array instead of a dictionary. Same with ``cubes``. Old style: ``"dimensions" = { "date" = ... }`` new style: ``"dimensions" = [ { "name": "date", ... } ]``. Will work if not changed, just be prepared. **Python**: Use Dimension.hierarchy() instead of Dimension.default_hierarchy. New Features ------------ * `slicer_context()` - new method that holds all relevant information from configuration. can be reused when creating tools that work in connected database environment * added `Hierarchy.all_attributes()` and `.key_attributes()` * `Cell.rollup_dim()` - rolls up single dimension to a specified level. this might later replace the Cell.rollup() method * `Cell.drilldown()` - drills down the cell * `create_workspace()` - new top-level method for creating a workspace by name of a backend and a configuration dictionary. Easier to create browsers (from possible browser pool) programmatically. The browser name might be full module name path or relative to the cubes.backends, for example ``sql.browser`` for default SQL denormalized browser. * `get_backend()` - get backend by name * AggregationBrowser.cell_details(): New method returning values of attributes representing the cell. Preliminary implementation, return value might change. * AggregationBrowser.cut_details(): New method returning values of attributes representing a single cut. Preliminary implementation, return value might change. * Dimension.validate() now checks whether there are duplicate attributes * Cube.validate() now checks whether there are duplicate measures or details SQL backend: * new StarBrowser implemented: * StarBrowser supports snowflakes or denormalization (optional) * for snowflake browsing no write permission is required (does not have to be denormalized) * new `DenormalizedMapper` for mapping logical model to denormalized view * new `SnowflakeMapper` for mapping logical model to a snowflake schema * `ddl_for_model()` - get schema DDL as string for model * join finder and attribute mapper are now just Mapper - class responsible for finding appropriate joins and doing logical-to-physical mappings * `coalesce_attribute()` - new method for coalescing multiple ways of describing a physical attribute (just attribute or table+schema+attribute) * dimension argument was removed from all methods working with attributes (the dimension is now required attribute property) * added `create_denormalized_view()` with options: materialize, create_index, keys_only Slicer: * slicer ddl - generate schema DDL from model * slicer test - test configuration and model against database and report list of issues, if any * Backend options are now in [workspace], removed configurability of custom backend section. Warning are issued when old section names [db] and [backend] are used * server responds to /details which is a result of AggregationBrowser.cell_details() Examples: * added simple Flask based web example - dimension aggregation browser Changes ------- * in Model: dimension and cube dictionary specification during model initialization is depreciated, list should be used (with explicitly mentioned attribute "name") -- **important** * **important**: Now all attribute references in the model (dimension attributes, measures, ...) are required to be instances of Attribute() and the attribute knows it's dimension * removed `hierarchy` argument from `Dimension.all_attributes()` and `Dimension.key_attributes()` * renamed builder to denormalizer * Dimension.default_hierarchy is now depreciated in favor of Dimension.hierarchy() which now accepts no arguments or argument None - returning default hierarchy in those two cases * metadata are now reused for each browser within one workspace - speed improvement. Fixes ----- * Slicer version should be same version as Cubes: Original intention was to have separate server, therefore it had its own versioning. Now there is no reason for separate version, moreover it can introduce confusion. * Proper use of database schema in the Mapper Version 0.8 =========== New Features ------------ * Started writing StarBrowser - another SQL aggregation browser with different approach (see code/docs) Slicer Server: * added configuration option ``modules`` under ``[server]`` to load additional modules * added ability to specify backend module * backend configuration is in [backend] by default, for SQL it stays in [db] * added server config option for default ``prettyprint`` value (useful for demontration purposes) Documentation: * Changed license to MIT + small addition. Please refer to the LICENSE file. * Updated documentation - added missing parts, made reference more readable, moved class and function reference docs from descriptive part to reference (API) part. * added backend documentation * Added "Hello World!" example Changed Features ---------------- * removed default SQL backend from the server * moved worskpace creation into the backend module Fixes ----- * Fixed create_view to handle not materialized properly (thanks to deytao) * Slicer tool header now contains #!/usr/bin/env python Version 0.7.1 ============= Added tutorials in tutorials/ with models in tutorials/models/ and data in tutorials/data/: * Tutorial 1: * how to build a model programatically * how to create a model with flat dimensions * how to aggregate whole cube * how to drill-down and aggregate through a dimension * Tutorial 2: * how to create and use a model file * mappings * Tutorial 3: * how hierarhies work * drill-down through a hierarchy * Tutorial 4 (not blogged about it yet): * how to launch slicer server New Features ------------ * New method: `Dimension.attribute_reference`: returns full reference to an attribute * str(cut) will now return constructed string representation of a cut as it can be used by Slicer Slicer server: * added /locales to slicer * added locales key in /model request * added Access-Control-Allow-Origin for JS/jQuery Changes ------- * Allow dimensions in cube to be a list, not only a dictionary (internally it is ordered dictionary) * Allow cubes in model to be a list, not only a dictionary (internally it is ordered dictionary) Slicer server: * slicer does not require default cube to be specified: if no cube is in the request then try default from config or get first from model Fixes ----- * Slicer not serves right localization regardless of what localization was used first after server was launched (changed model localization copy to be deepcopy (as it should be)) * Fixes some remnants that used old Cell.foo based browsing to Browser.foo(cell, ...) only browsing * fixed model localization issues; once localized, original locale was not available * Do not try to add locale if not specified. Fixes #11: https://github.com/Stiivi/cubes/issues/11 Version 0.7 =========== WARNING: Minor backward API incompatibility - Cuboid renamed to Cell. Changes ------- * Class 'Cuboid' was renamed to more correct 'Cell'. 'Cuboid' is a part of cube with subset of dimensions. * all APIs with 'cuboid' in their name/arguments were renamed to use 'cell' instead * Changed initialization of model classes: Model, Cube, Dimension, Hierarchy, Level to be more "pythony": instead of using initialization dictionary, each attribute is listed as parameter, rest is handled from variable list of key word arguments * Improved handling of flat and detail-less dimensions (dimensions represented just by one attribute which is also a key) Model Initialization Defaults: * If no levels are specified during initialization, then dimension name is considered flat, with single attribute. * If no hierarchy is specified and levels are specified, then default hierarchy will be created from order of levels * If no levels are specified, then one level is created, with name ``default`` and dimension will be considered flat Note: This initialization defaults might be moved into a separate utility function/class that will populate incomplete model New features ------------ Slicer server: * changed to handle multiple cubes within model: you have to specify a cube for /aggregate, /facts,... in form: /cube// * reflect change in configuration: removed ``view``, added ``view_prefix`` and ``view_suffix``, the cube view name will be constructed by concatenating ``view prefix`` + ``cube name`` + ``view suffix`` * in aggregate drill-down: explicit dimension can be specified with ``drilldown=dimension:level``, such as: ``date:month`` This change is considered final and therefore we can mark it is as API version 1. Version 0.6 =========== New features ------------ Cubes: * added 'details' to cube - attributes that might contain fact details which are not relevant to aggregation, but might be interesting when displaying facts * added ordering of facts in aggregation browser * SQL denormalizer can now add indexes to key columns, if requested * one detail table can be used more than once in SQL denomralizer (such as an organisation for both - receiver and donor), added key ````alias```` to ````joins```` in model description Slicer server: * added ``log`` a and ``log_level`` configuration options (under ``[server]``) * added ``format=`` parameter to ``/facts``, accepts ``json`` and ``csv`` * added ``fields=`` parameter to ``/facts`` - comma separated list of returned fields in CSV * share single sqlalchemy engine within server thread * limit number of facts returned in JSON (configurable by ``json_record_limit`` in ``[server]`` section) Experimental: (might change or be removed, use with caution) * added cubes searching frontend for separate cubes_search experimenal Sphinx backend (see https://bitbucket.org/Stiivi/cubes-search) Fixes ----- * fixed localization bug in fact(s) - now uses proper attribute name without locale suffix * fixed passing of pagination and ordering parameters from server to aggregation browser when requesting facts * fixed bug when using multiple conditions in SQL aggregator * make host/port optional separately