.. _shabti_auth_couchdb: Shabti and CouchDB |today| .. _shabti-auth-couchdb: Shabti and CouchDB **shabti_auth_couchdb** -- identity on CouchDB ============================================== .. rubric :: In this template, the now-familiar basic identity model pattern is expressed in `CouchDB `_, a document-oriented, non-relational database recently released from Apache incubator status. The pattern is taken from ``kai``, the Pylons code that currently runs the Pylons HQ web site (as at 2009-02-3), the source of which is available from the `kai bitbucket repository `_. .. note :: There's work still to be done here. However, the authentication is actually functioning, output is coherent and more or less as expected. A good starting position. Sometimes the rows-and-columns architecture of a relational store is not well-suited for the modelling job in hand. Sometimes a hierachically-organised architecture is a better fit --- and sometimes the best fit is a schema-less *ad hoc* store. CouchDB is such a store and in ``kai``, has proved an effective solution for document storage. retrieval and manipulation. Using CouchDB with Pylons is quite straightforward but depends on easy_installing both ``python-couchdb`` and the supporting ``httplib2`` *plus* the availability of a locally running `CouchDB `_ server. It is possible to connect to a remote CouchDB server if you propitiate the gods of ssh tunnelling. I haven't tried it with Python but it is possible to connect Futon to a remote instance of CouchDB via an ssh tunnel. .. note :: shabti_auth_couchdb source code is in the `bitbucket code repository `_ About CouchDB ------------- "Instead of storing data in rows and columns, the database manages a collection of JSON documents" - wikipedia .. note:: **What CouchDB is** * A document database server, accessible via a RESTful JSON API. * Ad-hoc and schema-free with a flat address space. * Distributed, featuring robust, incremental replication with bi-directional conflict detection and management. * Query-able and index-able, featuring a table oriented reporting engine that uses Javascript as a query language. **What it is Not** * A relational database. * A replacement for relational databases. * An object-oriented database. Or more specifically, meant to function as a seamless persistence layer for an OO programming language. *(the above text was lifted from the* `Introduction `_ *page on the couchdb web site.)* .. seealso :: `Getting started with Python `_ on the Apache site. Some understanding of the basics of couchdb is necessary in order to use it effectively. ``Futon`` is a web-based administration tool which gives immediate access for experimentation. The following short series of screen shots showing a demonstration database (available with the PylonsHQ source code) is not just a great saving on an otherwise total of three thousand words but it illustates the degree to which couchdb's schema-less, document-oriented model differs from the probably more familiar relational model ... Futon view on a collection of databases ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image :: images/futon_top.jpg Futon view of a collection of documents ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image :: images/futon_mid.jpg Futon view of a collection of attributes ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image :: images/futon_bottom.jpg Using the template ------------------ After successfully installing Shabti, additional paster templates will be available. Simply create a Shabti-configured project by specifying that paster should use the shabti_couchdb template: .. code-block :: bash $ paster create -t shabti_auth_couchdb myproj These are the option dialogue choices appropriate for the Shabti auth couchdb template --- which uses mako templates and for this template, SQLAlchemy is optional, use just CouchDB or SQLAlchemy as well if you wish ... .. code-block :: bash (mako/genshi/jinja/etc: Template language) ['mako']: (True/False: Include SQLAlchemy 0.4 configuration) [False]: True (True/False: Setup default appropriate for Google App Engine) [False]: Once the project has been created, navigate to the project directory and run the (brief) test suite: .. code-block :: bash $ nosetests 4 tests should executed successfully. If the tests succeeded, the next step is to initialise the couchdb store by running the project setup script: .. code-block :: bash $ paster setup-app development.ini Once the store has been inialised, start the Pylons web app with: .. code-block :: bash $ paster serve --reload development.ini The Shabti couchdb auth'n'auth template's variant on the standard Pylons welcome screen is browsable at at ``http://localhost:5000/`` ... Welcome screen ^^^^^^^^^^^^^^ .. image :: images/shabti_auth_couchdb_welcome.jpg Uauthenticated model view (via browse-db link) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image :: images/shabti_auth_couchdb_model_unauth.jpg sign-in page ^^^^^^^^^^^^ .. image :: images/shabti_auth_couchdb_signin.jpg Authenticated model view (via sign-in link) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. image :: images/shabti_auth_couchdb_model_auth.jpg Futon view of the User entity in the identity model ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Lastly, for completeness, the User identity model as presented by couchdb's GUI ``Futon``: .. image :: images/futon_identity_model.jpg Notes on the template --------------------- Auth'n'auth is not the best use of CouchDB. It would be far more sensible to adopt a standard relational approach to auth'n'auth whilst reserving CouchDB for handling documents. However, (ab)using CouchDB in the context of the standard Shabti auth'n'auth design pattern does allow some of the basic flavour to filter through. CouchDB is used extensively behind the scenes on the `new PylonsHQ web site `_ (2009-01-04 :: source code is currently available on bitbucket - http://www.bitbucket.org/bbangert/kai/). A significantly useful approach to using CouchDB in the context of developing web apps is described in this `introduction to using couchdb with django `_. The example code that is provided can be readily adapted for use with Pylons. The ``shabti_auth_couchdb`` template follows the same approach as the :ref:`shabti-auth` template and re-uses the majority of the code in that template. After the user performs a successful authentication, the ``request.environ`` dict holds the standard ``AUTH_USER`` key, the value of which is the couchdb id key, usable for retrieving the corresponding modelled :class:`User` object, this binding persists until the user explicitly signs off. The ``@authorize`` decorator is inherited from :ref:`shabti-auth` and is immediately available for use in applying authorization constraints on access to controllers and/or actions. Persistence of user, group and permission data is handled in the application's couchdb-mediated model and is amenable to manipulation via the usual REST controller approach. Pylons adjustments for CouchDB ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Note, the examples shown below all assume that ``myproj`` was the project name supplied as arg to ``paster create``. All conversations with CouchDB take place over HTTP and ``development.ini`` specifies the host on which CouchDB is presenting a RESTful API on port 5984 **development.ini** .. code-block :: ini couchdb_server = http://localhost:5984/ couchdb_uri = http://localhost:5984/myproj **config/environment.py** The bindings made in :file:`development.ini` are instantiated into the ``config`` dictionary, as per usual. .. code-block :: python # Setup the database connection config['myproj.server'] = Server(config['couchdb_server']) config['myproj.db'] = Database(config['couchdb_uri']) **model/user.py** The scheme of the declaration of the user model properties runs along reasonably familiar lines but the access methods will be unfamiliar to many. This is an illustrative fragment of the modelling which captures most of the significant points: .. code-block :: python class User(Document): """Represents a human user""" type = TextField(default='User') displayname = TextField() email = TextField() password = TextField() password_check = TextField() created = DateTimeField(default=datetime.utcnow) active = TextField() session_id = TextField() groups = ListField(TextField()) by_displayname = View('user', ''' function(doc) { if (doc.type == 'User') { emit(doc.displayname,doc) } }''') by_email = View('user', ''' function(doc) { if (doc.type == 'User') { emit(doc.email, doc) } }''') Clearly, the approach shows a slight departure from the standard identity model structure used in shabti templates that deal with relational models but, *pace* the :class:`View` methods, the difference is less salient than the similarity. (It's just a standing jump to get from here to there, not a running jump and grab). **lib/base.py** All communication with couchdb is via HTTP on the net, so the standard persistence concepts of server/client or exclusive access to the local filesystem do not apply, nor do issues of database "sessions". Each interaction with the database is RESTful and is therefore less reliable than a firmly-socketed connection to an RDBMS (even if the latter does have a bad habit of "going away"). The read/write function is necessarily GET/PUT respectively, with all of the attendant issues of leading a RESTful life. **Pylons vs Shabti lib/base.py** .. code-block :: diff --- pylons/lib/base.py +++ shabti/lib/base.py @@ -1,11 +1,27 @@ from {{package}}.model import meta +from couchdb import Database class BaseController(WSGIController): + def _setup(self): + """Do basic request setup""" + if pylons.session.get('logged_in', False): + user = User.load(self.db, pylons.session.get('user_id')) + if not user or user.session_id != pylons.session.id: + user = None + pylons.session['logged_in'] = False + pylons.session.delete() + pylons.session.save() + else: + user = None + def __call__(self, environ, start_response): """Invoke the Controller""" # WSGIController.__call__ dispatches to the Controller method # the request is routed to. This routing information is # available in environ['pylons.routes_dict'] + pylons.c.db = self.db = Database(pylons.config['couchdb_uri']) + self._setup() + return WSGIController.__call__(self, environ, start_response){{endif}} **websetup.py** The included :file:`websetup.py` gives an example of using Python to set up a database and add an initial entry. .. code-block :: python server = pylons.config['myproj.server'] # Change teardown to True if the database # exists and is to be torn down and re-created teardown = True if teardown and 'myproj' in server: del server['myproj'] db = server.create('myproj') elif not 'myproj' in server: db = server.create('myproj') else: db = pylons.config['myproj.db'] gadmin = model.user.Group( name = "Administrators", description = u"Administration group", created = datetime.datetime.utcnow(), active = True) gadmin.store(db) gadmin_retrieved = Group.load(db, gadmin.id) # Check the status assert gadmin_retrieved.name == gadmin.name Controller usage ^^^^^^^^^^^^^^^^ Documents stored in CouchDB are typically retrieved with a previously defined and stored ``View`` (of the document collection). In the example shown below, the ``index`` action requests 5 user documents to be retrieved and ordered by the ``by_displayname`` view, the latter being accessed as a property of the :class:`User` entity. The retrieval of :class:`Group` and :class:`Permission` data takes the same approach. .. code-block :: python # controllers/user.py def index(self): c.users = User.by_displayname(c.db, descending=True, count=5) c.groups = Group.by_name(c.db, descending=True, count=5) c.permissions = Permission.by_name(c.db, descending=True, count=5) c.title = 'Test' authusr = req.environ.get('AUTH_USER', False) c.identity = c.db[authusr] if authusr else False return render('test.mak') Listed below is the more familiar SQLAlchemy access approach, included here for ease of comparison. .. code-block :: python # controllers/user.py def index(self): c.users = Session.query(User).all() c.groups = Session.query(Group).all() c.permissions = Session.query(Permission).all() c.title = 'Test' return render('test.mak') Once the database connection parameters have been bound, interactions with couchdb are conducted over the net as per the REST GET/PUT/POST/DELETE architecture, content is expressed in standard JSON. Authentication of the application to the database can be dispensed with if access is purely via the loopback, otherwise the model back-end handles per-request authentication of the application to a remote couchdb server. Couchdb principles ------------------ *(The following is lifted directly from* `Getting started with python-couchdb `_ *and shows fragments of the API for a Document class)* Getting Started ^^^^^^^^^^^^^^^ This tutorial exposes key features of this library through mainly code examples. For in-depth description of the modules, you'll want to read the API documentation. Create a CouchDB document: .. code-block:: python from simplecouchdb import schema class Greeting(schema.Document): author = schema.StringPropertty() content = schema.StringProperty() date = schema.DateTimePropertty() Store the submitted Greetings +++++++++++++++++++++++++++++ .. code-block:: python import datetime from simplecouchdb import Server db = Server() greet = Greeting( author="Benoit", content="Welcome to simplecouchdb world", date=datetime.datetime.utcnow() ) greet.save(db) Updating documents in the database ++++++++++++++++++++++++++++++++++ .. code-block:: python greet.content = "Welcome to CouchDB" greet.save(db) Adding a property to a document +++++++++++++++++++++++++++++++ .. code-block:: python greet.homepage = "http://www.e-engura.org" greet.save(db) Get all instances ^^^^^^^^^^^^^^^^^ Create a ``view`` and save it in the db, add a :class:`simplecouchdb.schema.ViewProperty` to the Document schema: .. code-block:: python class Greeting(schema.Document): ... all = schema.ViewProperty("greeting", map_fun="""function(doc) { if (doc.doc_type == "Greeting") emit(doc._id, doc); }""") greets = Greeting.all.get(db) That's it you now get all greets. :author: Graham Higgins |today|