.. _shabti_microsite: |today| .. _shabti-microsite: **shabti_microsite** -- a kick-the-tyres microsite ================================================== .. rubric :: The ``shabti_microsite`` template generates a *very* small populated web site with an identity model driving the auth'n'auth, complete with an "admin" login (generated by the contents of :file:`websetup.py` and running ``paster setup-app development.ini``), a basic page content model driving a forms-based micro-CMS dashboard. .. note :: shabti_microsite source code is in the `bitbucket code repository `_ About the microsite ------------------- Unlike the standard default Pylons project template, the model, the controllers and the view components are populated, providing a very rough-and-ready design pattern. The site includes examples of forms and validators and provides an integrated set of Genshi and Mako templates implementing a fluid, table-less 1, 2 or 3-column layout solution with a standard framework of CSS stylesheets. Some effort has been devoted to modularising the templates according to broad function (for example there is a component that advertises a standard range of GRDDL transformations). The code is simple and makes a very direct expression of some simple idioms in order to act as a quick sketch of the basic principles. It may possibly have some use as a starting point from which to develop something more contextually specific, for example a customised, in-house proto-framework because the micro-site and the Shabti code together provide a worked example of how to create Paste templates and add commands to paster. The micro-site also offers a worked example of tw.forms, a web forms package based on the Toscawidgets middleware widget rendering and input validation package. A set of the current Pylons official documents is available in Sphinx-rendered "web" format (fpickle) along with a "docs" controller providing a standalone docs browser. The document archive is not distributed with Shabti, it is resident on my server. The microsite docs page will detect its absence and offer to download and install the tar.bz2 archive (to :file:`public/web`). .. warning :: **NB**: Dependency hell. The microsite's main function is to provide a vaguely plausible backdrop for experimentation. There are several dependencies: `Toscawidgets `_, `tw.forms `_ `FormAlchemy `_ and `Babel `_ (all of which are easy_installable). Also, the micro-site is complete with a :file:`log4j.xml` file and can be configured to send logging to Apache chainsaw, in that instance XMLLayout will need to be easy_installed. Information & Software Architecture principles ---------------------------------------------- The microsite explores Pylons' explicit support for following the principles of a `RESTful architecture `_. By providing easily implemented access to web services via parameterized URIs, the REST information architecture / software design pattern presents an aesthetic attraction for developers and end users alike. Ease of implementation means reduced application complexity for the designer and for the user it presents a more readily comprehensible information architecture. **GET** is used for safe operations: ones without any side effects. GETable operations can be completely specified by the URL and can be bookmarked, linked to, prefetched, cached, and otherwise optimized. **POST** is used for non-idempotent operations: ones that cause things to happen. You can’t even use the back button on a post without seeing a warning. The data for a POST request is actually encapsulated in the the HTTP request body, not in the URL as with a GET. Thus POST operations cannot be bookmarked, linked to, prefetched, cached, and otherwise optimized. **PUT** is used to create or update a resource at a specific URL. PUT is not safe (it changes or creates a new resource) but it is idempotent. That means that if you aren’t sure if a request succeeded, try it again. PUTting the same content twice is not a problem. The end result is the same as PUTting it once. **DELETE** simply deletes the resource at a specified URL. It too is idempotent, since deleting a previously deleted resource has no effect. The appeal of REST is that those four verbs are all that is needed. Everything that needs to be done can be done with GET, PUT, POST, and DELETE. It isn't necessary to invent a new verb for every operation that can be imagined. This make HTTP scalable, flexible, and extensible. Simplicity leads to power. One benefit that's obvious with regards to web based applications is that a ReSTful implementation allows a user to bookmark specific "queries" (or requests) and allows those to be conveyed to others across e-mail, instant messages, or to be injected into wikis, etc. Thus this "representation" of a path or entry point into an application state becomes a highly portable feature of the Information Architecture. RESTful Web services ^^^^^^^^^^^^^^^^^^^^ A RESTFul web service is a simple web service implemented using HTTP and the principles of REST. Such a web service can be thought about as a collection of resources. The definition of such a web service can be thought of as comprising three aspects: The URI for the web service such as http://example.com/resources/cars The MIME type of the data supported by the web service. This is often JSON , XML or YAML but can be anything. The set of operations supported by the web service using HTTP methods including but not limited to POST, GET, PUT and DELETE. Members of the collection are addressed by ID using URIs of the form /. The ID can be any unique identifier. For example if a RESTFul web service representing a collection of cars for sale might have the URI http://example.com/resources/cars. If the service uses the car registration number as the ID then a particular car might be present in the collection as http://example.com/resources/cars/yxz123. The following table shows how the HTTP verbs are typically used to implement a web service. .. code-block :: text Method Collection URI, Member URI, e.g. http://x.com/cars/ e.g. http://x.com/cars/yxz123 GET List the members of the collection. Retrieve the addressed member of For example list all the cars for the collection sale. PUT Not generally used. Meaning defined Update the addressed member of the as "replace the entire collection collection or create it with a with another entire collection". defined ID. POST Create a new entry in the collection Not Generally Used. where the ID is assigned by the collection automatically. The ID created is typically returned by this operation. DELETE Not Generally Used. Meaning defined Delete the addressed member of as "delete the entire collection". the collection. .. note :: *Much of the content for the above two sections was adapted from a 2006-10* `blog post `_ *by Elliot Rusty Harold and the relevant* `wikipedia entry for REST `_ - *a wilful extension of the notion of code re-use to include text re-use*. The Routes RESTful mapping ++++++++++++++++++++++++++ To create the appropriate RESTful mapping, add a map statement to the :file:`config/routing.py` file near the top like this: .. code-block :: python map.resource('page', 'pages') To make it easier to setup RESTful web services with Routes, there's a shortcut Mapper method that will setup a batch of routes along with conditions that will restrict them to specific HTTP methods. This is directly styled on the Rails version of :func:`map.resource`, which was based heavily on the `Atom Publishing Protocol `_. .. note :: The Atom Publishing Protocol (AtomPub or APP) is a simple HTTP-based protocol for creating and updating web resources and of which wikipedia observes: "The Atom Publishing Protocol for publishing to blogs is considered a canonical RESTful protocol" Consider the common case where you have a system that deals with pages. In that case operations dealing with the entire group of pages (or perhaps a subset) would be considered collection methods. Operations (or actions) that act on an individual member of that collection are considered member methods. These terms are important to remember as the options to ``map.resource`` rely on a clear understanding of *collection actions* vs. *member actions*. The Routes :func:`map.resource` command creates a set of Routes for common operations on a collection of resources, individually referred to as 'members'. .. code-block :: python map.resource('page', 'pages') Will setup all the routes as if the following map commands had been typed: .. code-block :: python map.connect('pages', controller='pages', action='create', conditions=dict(method=['POST'])) map.connect('pages', 'pages', controller='pages', action='index', conditions=dict(method=['GET'])) map.connect('formatted_pages', 'pages.:(format)', controller='pages', action='index', conditions=dict(method=['GET'])) map.connect('new_page', 'pages/new', controller='pages', action='new', conditions=dict(method=['GET'])) map.connect('formatted_new_page', 'pages/new.:(format)', controller='pages', action='new', conditions=dict(method=['GET'])) map.connect('pages/:id', controller='pages', action='update', conditions=dict(method=['PUT'])) map.connect('pages/:id', controller='pages', action='delete', conditions=dict(method=['DELETE'])) map.connect('edit_page', 'pages/:(id);edit', controller='pages, action='edit', conditions=dict(method=[''GET'])) map.connect('formatted_edit_page', 'pages/:(id).:(format);edit', controller='pages', action='edit', conditions=dict(method=['GET'])) map.connect('page', 'pages/:id', controller='pages', action='show', conditions=dict(method=['GET'])) map.connect('formatted_page', 'pages/:(id).:(format)', controller='pages', action='show', conditions=dict(method=['GET'])) The most important aspects of this is the following mapping that is established: .. code-block :: text GET /pages -> pages.index() -> url_for('pages') POST /pages -> pages.create() -> url_for('pages') GET /pages/new -> pages.new() -> url_for('new_page') PUT /pages/1 -> pages.update(id) -> url_for('page', id=1) DELETE /pages/1 -> pages.delete(id) -> url_for('page', id=1) GET /pages/1 -> pages.show(id) -> url_for('page', id=1) GET /pages/1/edit -> pages.edit(id) -> url_for('edit_page', id=1) Several of these methods map to functions intended to display forms. The :meth:`new` page method should be used to return a form allowing someone to create a new page, while it should ``POST`` to ``/pages``. The :meth:`edit` page function should work similarly returning a form to edit a page, which then posts a ``PUT`` to the ``/pages/1`` resource. The RESTful controller ++++++++++++++++++++++ None of the browser vendors support the ``PUT`` and ``DELETE`` methods, so this functionality must be simulated. In Pylons, hidden form fields are used to carry the type of REST method. The Pylons paster command for creating a RESTful controller produces the output listed below. The docstrings for each action describe the REST method and the URL pattern. **controllers/pages.py** .. code-block :: python from myapp.lib.base import * class PagesController(BaseController): """REST Controller styled on the Atom Publishing Protocol""" # To properly map this controller, ensure your config/routing.py file has # a resource setup: # map.resource('page', 'pages') def index(self, format='html'): """GET /: All items in the collection.""" # url_for('pages') pass def create(self): """POST /: Create a new item.""" # url_for('pages') pass def new(self, format='html'): """GET /new: Form to create a new item.""" # url_for('new_page') pass def update(self, id): """PUT /id: Update an existing item.""" # Forms posted to this method should contain a hidden field: # # Or using helpers: # h.form(h.url('page', id=ID), method='put') # url_for('page', id=ID) pass def delete(self, id): """DELETE /id: Delete an existing item.""" # Forms posted to this method should contain a hidden field: # # Or using helpers: # h.form(h.url('page', id=ID), method='delete') # url_for('page', id=ID) pass def show(self, id, format='html'): """GET /id: Show a specific item.""" # url_for('page', id=ID) pass def edit(self, id, format='html'): """GET /id;edit: Form to edit an existing item.""" # url_for('edit_page', id=ID) pass The Pylons :func:`rest` decorator allows the developer to further simplify the API to the controller by dispatching to alternate controller methods based on the HTTP method. Multiple keyword arguments are passed, with the keyword corresponding to the HTTP method to dispatch on (DELETE, POST, GET, etc.) and the value being the function to call. The value should be a string indicating the name of the function to dispatch to. Example: .. code-block :: python from pylons.decorators import rest class PageController(BaseController): @rest.dispatch_on(POST='create_page') def page(self): # Do something with the page # [ ... ] return render('list.mak') def create_page(self, id): # Do something if its a post to page # [ ... ] return render('result.mak') REST support from toscawidgets ++++++++++++++++++++++++++++++ Toscawidgets Toscawidgets uses a dedicated form fieldname: ``_method`` to carry the REST method. .. code-block :: python class PageForm(ListForm): class fields(WidgetsList): """ The ``_method`` hidden field is required in order to trigger the correct ``REST`` route. """ rest_method = HiddenField( default = "PUT", name = "_method") title = TextField( ... ) MIME type support from Pylons +++++++++++++++++++++++++++++ The previously-detailed list of routes automatically created by ``map.route`` provides four specific format-enabling GET routes. The format value is arbitrary, a value to be supplied by the developer and any choice will require explicit support in the controller action (e.g. the :func:`@jsonify` decorator) but the approach has the advantage of wider flexibility. The example below shows how a specific mime type is passed in to these general-purpose format-carrying routes: .. code-block :: text GET /pages.json -> pages.index() -> url_for('formatted_pages', format='json') GET /pages/new.json -> pages.new() -> url_for('formatted_new_page', format='json') GET /pages/1.json/edit -> pages.edit(id) -> url_for('formatted_edit_page', id=1,format='json') GET /pages/1.json -> pages.show(id) -> url_for('formatted_page', id=1, format='json') :func:`pylons.decorators.jsonify(func)` is an action decorator that formats output for JSON Given a function that will return content, this decorator will turn the result into JSON, with a content-type of ‘application/json’ and output it. Controller actions should be designed to produce whichever additional formats are seen as necessary or desirable for the project. The associated test file :file:`test_rest.py` and its companion example controller file :file:`users_for_testing.py_tmp` contain further examples of MIME-type specific interactions. Using the template ------------------ After successfully installing Shabti, additional paster templates will be available. Simply create a Shabti-configured project by specifying that paster should use the shabti microsite template: .. code-block :: bash $ paster create -t shabti_microsite myproj These are the option dialogue choices appropriate for the Shabti microsite template --- which uses mako templates and SQLAlchemy ... .. code-block :: bash (mako/genshi/jinja/etc: Template language) ['mako']: (True/False: Include SQLAlchemy 0.4 configuration) [False]: True (True/False: Setup default appropriate for Google App Engine) [False]: Once the project has been created, navigate to the project directory and run the test suite: .. code-block :: bash $ nosetests There will be some mixed output that scrolls up the stdout. It should culminate in: .. code-block :: text ---------------------------------------------------------------------- Ran 38 tests in 2.398s <- The number of tests is key, the time will differ. OK After the test suite has successfully the executed 38 tests, initialise the relational store (defaults to sqlite) by running the project setup script: .. code-block :: bash $ paster setup-app development.ini Once the database has been initialised, start the Pylons web app with: .. code-block :: bash $ paster serve --reload development.ini The Shabti microsite template's variant on the standard Pylons welcome screen is browsable at at ``http://localhost:5000/`` ... Notes on the template --------------------- Microsite is mostly about (me) finding some quietly re-usable patterns that can be of broader use when constructing Pylons sites for $$. Microsite's auth'n'auth is the standard elixir-modelled identity from :ref:`shabti-auth`. `Bartosz Radaczyński `_ has forked Shabti and added a more detailed REST controller template and a ``scaffolding`` paster controller creation command. Bartosz' code is gradually being integrated into Shabti ... once I've had a chance to think about the potential for abstraction offered by the current microsite controller code. DRY it isn't --- but sometimes that's just the way it is naturally. The Genshi template set is some months out of synchrony with the Mako set and will require some work to bring up to date. There's a vague plan to add some data in bulk, to give the forms handling packages something to get their teeth into. Screen shots ^^^^^^^^^^^^ Welcome Page ++++++++++++ The standard welcome page (**myapp/public/index.html**) .. image :: images/microsite_welcome.jpg :align: center Home Page +++++++++ Simple elixir-backed page, carries HTML markup in ``Page.content`` .. image :: images/microsite_home.jpg :align: center Installed docs ++++++++++++++ It's a one-click install to replace the holding page with a set of docs to browse locally. .. image :: images/microsite_installed_docs.jpg :align: center Dashboard index +++++++++++++++ Admin dashboard (link in footer). Standard CRUD supported for all elixir entities. Highly unlikely to be working flawlessly, despite the successful tests. .. image :: images/dashboard_index.jpg :align: center Dashboard Add User ++++++++++++++++++ C of CRUD .. image :: images/microsite_add_user.jpg :align: center Some forms grid experiments ^^^^^^^^^^^^^^^^^^^^^^^^^^^ Hand-constructed tw.forms +++++++++++++++++++++++++ .. image :: images/dashboard_users.jpg :align: center tw.forms grid +++++++++++++ .. image :: images/dashboard_pages.jpg :align: center Sprox grid ++++++++++ .. image :: images/sprox_grid_test.jpg :align: center Formalchemy grid ++++++++++++++++ .. image :: images/formalchemy_grid_test.jpg :align: center .. seealso :: :ref:`microsite-shots` for a few more illustrative screenshots. :author: Graham Higgins |today|