September 06, 2010
shabti_microsite – a kick-the-tyres microsite¶
The shabti_microsite template generates a very small populated web site with an identity model driving the auth’n’auth, complete with an “admin” login (generated by the contents of websetup.py and running paster setup-app development.ini), a basic page content model driving a forms-based micro-CMS dashboard.
Note
shabti_microsite source code is in the bitbucket code repository
About the microsite¶
Unlike the standard default Pylons project template, the model, the controllers and the view components are populated, providing a very rough-and-ready design pattern.
The site includes examples of forms and validators and provides an integrated set of Genshi and Mako templates implementing a fluid, table-less 1, 2 or 3-column layout solution with a standard framework of CSS stylesheets.
Some effort has been devoted to modularising the templates according to broad function (for example there is a component that advertises a standard range of GRDDL transformations).
The code is simple and makes a very direct expression of some simple idioms in order to act as a quick sketch of the basic principles.
It may possibly have some use as a starting point from which to develop something more contextually specific, for example a customised, in-house proto-framework because the micro-site and the Shabti code together provide a worked example of how to create Paste templates and add commands to paster.
The micro-site also offers a worked example of tw.forms, a web forms package based on the Toscawidgets middleware widget rendering and input validation package.
A set of the current Pylons official documents is available in Sphinx-rendered “web” format (fpickle) along with a “docs” controller providing a standalone docs browser. The document archive is not distributed with Shabti, it is resident on my server. The microsite docs page will detect its absence and offer to download and install the tar.bz2 archive (to public/web).
Warning
NB: Dependency hell. The microsite’s main function is to provide a vaguely plausible backdrop for experimentation. There are several dependencies: Toscawidgets, tw.forms FormAlchemy and Babel (all of which are easy_installable). Also, the micro-site is complete with a log4j.xml file and can be configured to send logging to Apache chainsaw, in that instance XMLLayout will need to be easy_installed.
Information & Software Architecture principles¶
The microsite explores Pylons’ explicit support for following the principles of a RESTful architecture.
By providing easily implemented access to web services via parameterized URIs, the REST information architecture / software design pattern presents an aesthetic attraction for developers and end users alike. Ease of implementation means reduced application complexity for the designer and for the user it presents a more readily comprehensible information architecture.
GET is used for safe operations: ones without any side effects. GETable operations can be completely specified by the URL and can be bookmarked, linked to, prefetched, cached, and otherwise optimized.
POST is used for non-idempotent operations: ones that cause things to happen. You can’t even use the back button on a post without seeing a warning. The data for a POST request is actually encapsulated in the the HTTP request body, not in the URL as with a GET. Thus POST operations cannot be bookmarked, linked to, prefetched, cached, and otherwise optimized.
PUT is used to create or update a resource at a specific URL. PUT is not safe (it changes or creates a new resource) but it is idempotent. That means that if you aren’t sure if a request succeeded, try it again. PUTting the same content twice is not a problem. The end result is the same as PUTting it once.
DELETE simply deletes the resource at a specified URL. It too is idempotent, since deleting a previously deleted resource has no effect.
The appeal of REST is that those four verbs are all that is needed. Everything that needs to be done can be done with GET, PUT, POST, and DELETE. It isn’t necessary to invent a new verb for every operation that can be imagined. This make HTTP scalable, flexible, and extensible. Simplicity leads to power.
One benefit that’s obvious with regards to web based applications is that a ReSTful implementation allows a user to bookmark specific “queries” (or requests) and allows those to be conveyed to others across e-mail, instant messages, or to be injected into wikis, etc. Thus this “representation” of a path or entry point into an application state becomes a highly portable feature of the Information Architecture.
RESTful Web services¶
A RESTFul web service is a simple web service implemented using HTTP and the principles of REST. Such a web service can be thought about as a collection of resources. The definition of such a web service can be thought of as comprising three aspects:
The URI for the web service such as http://example.com/resources/cars
The MIME type of the data supported by the web service. This is often JSON , XML or YAML but can be anything.
The set of operations supported by the web service using HTTP methods including but not limited to POST, GET, PUT and DELETE.
Members of the collection are addressed by ID using URIs of the form <baseURI>/<ID>. The ID can be any unique identifier. For example if a RESTFul web service representing a collection of cars for sale might have the URI http://example.com/resources/cars. If the service uses the car registration number as the ID then a particular car might be present in the collection as http://example.com/resources/cars/yxz123.
The following table shows how the HTTP verbs are typically used to implement a web service.
Method Collection URI, Member URI,
e.g. http://x.com/cars/ e.g. http://x.com/cars/yxz123
GET List the members of the collection. Retrieve the addressed member of
For example list all the cars for the collection
sale.
PUT Not generally used. Meaning defined Update the addressed member of the
as "replace the entire collection collection or create it with a
with another entire collection". defined ID.
POST Create a new entry in the collection Not Generally Used.
where the ID is assigned by the
collection automatically. The ID
created is typically returned by
this operation.
DELETE Not Generally Used. Meaning defined Delete the addressed member of
as "delete the entire collection". the collection.
Note
Much of the content for the above two sections was adapted from a 2006-10 blog post by Elliot Rusty Harold and the relevant wikipedia entry for REST - a wilful extension of the notion of code re-use to include text re-use.
The Routes RESTful mapping¶
To create the appropriate RESTful mapping, add a map statement to the config/routing.py file near the top like this:
map.resource('page', 'pages')
To make it easier to setup RESTful web services with Routes, there’s a shortcut Mapper method that will setup a batch of routes along with conditions that will restrict them to specific HTTP methods. This is directly styled on the Rails version of map.resource(), which was based heavily on the Atom Publishing Protocol.
Note
The Atom Publishing Protocol (AtomPub or APP) is a simple HTTP-based protocol for creating and updating web resources and of which wikipedia observes: “The Atom Publishing Protocol for publishing to blogs is considered a canonical RESTful protocol”
Consider the common case where you have a system that deals with pages. In that case operations dealing with the entire group of pages (or perhaps a subset) would be considered collection methods. Operations (or actions) that act on an individual member of that collection are considered member methods. These terms are important to remember as the options to map.resource rely on a clear understanding of collection actions vs. member actions.
The Routes map.resource() command creates a set of Routes for common operations on a collection of resources, individually referred to as ‘members’.
map.resource('page', 'pages')
Will setup all the routes as if the following map commands had been typed:
map.connect('pages',
controller='pages', action='create',
conditions=dict(method=['POST']))
map.connect('pages', 'pages',
controller='pages', action='index',
conditions=dict(method=['GET']))
map.connect('formatted_pages', 'pages.:(format)',
controller='pages', action='index',
conditions=dict(method=['GET']))
map.connect('new_page', 'pages/new',
controller='pages', action='new',
conditions=dict(method=['GET']))
map.connect('formatted_new_page', 'pages/new.:(format)',
controller='pages', action='new',
conditions=dict(method=['GET']))
map.connect('pages/:id',
controller='pages', action='update',
conditions=dict(method=['PUT']))
map.connect('pages/:id',
controller='pages', action='delete',
conditions=dict(method=['DELETE']))
map.connect('edit_page', 'pages/:(id);edit',
controller='pages, action='edit',
conditions=dict(method=[''GET']))
map.connect('formatted_edit_page', 'pages/:(id).:(format);edit',
controller='pages', action='edit',
conditions=dict(method=['GET']))
map.connect('page', 'pages/:id',
controller='pages', action='show',
conditions=dict(method=['GET']))
map.connect('formatted_page', 'pages/:(id).:(format)',
controller='pages', action='show',
conditions=dict(method=['GET']))
The most important aspects of this is the following mapping that is established:
GET /pages
-> pages.index() -> url_for('pages')
POST /pages
-> pages.create() -> url_for('pages')
GET /pages/new
-> pages.new() -> url_for('new_page')
PUT /pages/1
-> pages.update(id) -> url_for('page', id=1)
DELETE /pages/1
-> pages.delete(id) -> url_for('page', id=1)
GET /pages/1
-> pages.show(id) -> url_for('page', id=1)
GET /pages/1/edit
-> pages.edit(id) -> url_for('edit_page', id=1)
Several of these methods map to functions intended to display forms. The new() page method should be used to return a form allowing someone to create a new page, while it should POST to /pages. The edit() page function should work similarly returning a form to edit a page, which then posts a PUT to the /pages/1 resource.
The RESTful controller¶
None of the browser vendors support the PUT and DELETE methods, so this functionality must be simulated. In Pylons, hidden form fields are used to carry the type of REST method. The Pylons paster command for creating a RESTful controller produces the output listed below. The docstrings for each action describe the REST method and the URL pattern.
controllers/pages.py
from myapp.lib.base import *
class PagesController(BaseController):
"""REST Controller styled on the Atom Publishing Protocol"""
# To properly map this controller, ensure your config/routing.py file has
# a resource setup:
# map.resource('page', 'pages')
def index(self, format='html'):
"""GET /: All items in the collection."""
# url_for('pages')
pass
def create(self):
"""POST /: Create a new item."""
# url_for('pages')
pass
def new(self, format='html'):
"""GET /new: Form to create a new item."""
# url_for('new_page')
pass
def update(self, id):
"""PUT /id: Update an existing item."""
# Forms posted to this method should contain a hidden field:
# <input type="hidden" name="_method" value="PUT" />
# Or using helpers:
# h.form(h.url('page', id=ID), method='put')
# url_for('page', id=ID)
pass
def delete(self, id):
"""DELETE /id: Delete an existing item."""
# Forms posted to this method should contain a hidden field:
# <input type="hidden" name="_method" value="DELETE" />
# Or using helpers:
# h.form(h.url('page', id=ID), method='delete')
# url_for('page', id=ID)
pass
def show(self, id, format='html'):
"""GET /id: Show a specific item."""
# url_for('page', id=ID)
pass
def edit(self, id, format='html'):
"""GET /id;edit: Form to edit an existing item."""
# url_for('edit_page', id=ID)
pass
The Pylons rest() decorator allows the developer to further simplify the API to the controller by dispatching to alternate controller methods based on the HTTP method.
Multiple keyword arguments are passed, with the keyword corresponding to the HTTP method to dispatch on (DELETE, POST, GET, etc.) and the value being the function to call. The value should be a string indicating the name of the function to dispatch to.
Example:
from pylons.decorators import rest
class PageController(BaseController):
@rest.dispatch_on(POST='create_page')
def page(self):
# Do something with the page
# [ ... ]
return render('list.mak')
def create_page(self, id):
# Do something if its a post to page
# [ ... ]
return render('result.mak')
REST support from toscawidgets¶
Toscawidgets Toscawidgets uses a dedicated form fieldname: _method to carry the REST method.
class PageForm(ListForm):
class fields(WidgetsList):
"""
The ``_method`` hidden field is required
in order to trigger the correct ``REST`` route.
"""
rest_method = HiddenField(
default = "PUT",
name = "_method")
title = TextField( ... )
MIME type support from Pylons¶
The previously-detailed list of routes automatically created by map.route provides four specific format-enabling GET routes. The format value is arbitrary, a value to be supplied by the developer and any choice will require explicit support in the controller action (e.g. the @jsonify() decorator) but the approach has the advantage of wider flexibility. The example below shows how a specific mime type is passed in to these general-purpose format-carrying routes:
GET /pages.json
-> pages.index() -> url_for('formatted_pages',
format='json')
GET /pages/new.json
-> pages.new() -> url_for('formatted_new_page',
format='json')
GET /pages/1.json/edit
-> pages.edit(id) -> url_for('formatted_edit_page',
id=1,format='json')
GET /pages/1.json
-> pages.show(id) -> url_for('formatted_page',
id=1, format='json')
pylons.decorators.jsonify(func)() is an action decorator that formats output for JSON
Given a function that will return content, this decorator will turn the result into JSON, with a content-type of ‘application/json’ and output it.
Controller actions should be designed to produce whichever additional formats are seen as necessary or desirable for the project. The associated test file test_rest.py and its companion example controller file users_for_testing.py_tmp contain further examples of MIME-type specific interactions.
Using the template¶
After successfully installing Shabti, additional paster templates will be available. Simply create a Shabti-configured project by specifying that paster should use the shabti microsite template:
$ paster create -t shabti_microsite myproj
These are the option dialogue choices appropriate for the Shabti microsite template — which uses mako templates and SQLAlchemy ...
(mako/genshi/jinja/etc: Template language) ['mako']:
(True/False: Include SQLAlchemy 0.4 configuration) [False]: True
(True/False: Setup default appropriate for Google App Engine) [False]:
Once the project has been created, navigate to the project directory and run the test suite:
$ nosetests
There will be some mixed output that scrolls up the stdout. It should culminate in:
----------------------------------------------------------------------
Ran 38 tests in 2.398s <- The number of tests is key, the time will differ.
OK
After the test suite has successfully the executed 38 tests, initialise the relational store (defaults to sqlite) by running the project setup script:
$ paster setup-app development.ini
Once the database has been initialised, start the Pylons web app with:
$ paster serve --reload development.ini
The Shabti microsite template’s variant on the standard Pylons welcome screen is browsable at at http://localhost:5000/ ...
Notes on the template¶
Microsite is mostly about (me) finding some quietly re-usable patterns that can be of broader use when constructing Pylons sites for $$.
Microsite’s auth’n’auth is the standard elixir-modelled identity from shabti_auth – boilerplate authentication.
Bartosz Radaczyński has forked Shabti and added a more detailed REST controller template and a scaffolding paster controller creation command. Bartosz’ code is gradually being integrated into Shabti ... once I’ve had a chance to think about the potential for abstraction offered by the current microsite controller code. DRY it isn’t — but sometimes that’s just the way it is naturally.
The Genshi template set is some months out of synchrony with the Mako set and will require some work to bring up to date.
There’s a vague plan to add some data in bulk, to give the forms handling packages something to get their teeth into.
Screen shots¶
Installed docs¶
It’s a one-click install to replace the holding page with a set of docs to browse locally.
Dashboard index¶
Admin dashboard (link in footer). Standard CRUD supported for all elixir entities. Highly unlikely to be working flawlessly, despite the successful tests.
Some forms grid experiments¶
Hand-constructed tw.forms¶
tw.forms grid¶
Sprox grid¶
Formalchemy grid¶
See also
microsite shots – screenshots for a few more illustrative screenshots.
| author: | Graham Higgins <gjh@bel-epa.com> |
|---|
September 06, 2010