This step-by-step tutorial will make you create, document and use webservices on a basic TurboGears application with TGWebServices.
The services we will create allow a third-party application to do a few operations on the user definitions, so it can, for example, synchronise the user definitions with its own one.
Python : This tutorial was tested with python 2.5 on a machine with multiple versions of python installed, so when needed the interpreter is explicitely given in the command lines so that another version does not interfere.
setuptools (or distribute) for the version of python you will use (preferably python 2.5).
virtualenv for the version of python you will use:
easy_install-2.5 virtualenv
Choose a workspace directory name, for example ~/workspace or C:workspace (we assert you use one of these in this tutorial) and create it.
Create the virtualenv and activate it :
Unix:
mkdir ~/workspace
cd ~/workspace
virtualenv -p python2.5 tutoenv
source tutoenv/bin/activate
Windows
c:
mdir c:\workspace
cd \workspace
C:\Python25\Scripts\virtualenv tutoenv
tutoenv\Scripts\activate.bat
Your shell prompt should now look like :
(tutoenv)cdevienne@tryo:~/workspace$
Install TurboGears >= 1.1, < 1.1.99:
easy_install -f http://files.turbogears.org/eggs/ "TurboGears >= 1.1, < 1.1.99"
Quickstart a TG project named WSTuto with identity activated
tg-admin quickstart --sqlalchemy --identity WSTuto -p wstuto
cd WSTuto
Note
From now, for all of the commands we assume the current directory is your project root, WSTuto
Add the required dependencies in setup.py (around line 57), and also specify where to look for the packages:
install_requires=[
"TurboGears >= 1.1.1",
"WebTest",
"SQLAlchemy >= 0.6.3",
"TGWebServices >= 1.2.4rc1, < 1.2.99",
"Sphinx >= 0.6.5, < 0.6.99",
],
dependency_links=['http://files.turbogears.org/eggs/'],
Note
We depend on Sphinx only to make this tutorial easier to execute. It is not really a dependency of your application, but more a tool you need when you work on the documentation that you can install manually.
Make sure your application has all its dependencies:
python setup.py develop
Create your database
tg-admin sql create
You should now be able to start your turbogears application:
start-wstuto
And one of the last line of logs on your console should soon be:
2010-08-25 22:52:31,397 cherrypy.msg INFO HTTP: Serving HTTP on http://0.0.0.0:8080/
Point your brower on http://localhost:8080/ to see what it looks like for a minute !
First we need to add a webservices root controller somewhere in our controllers tree.
Adding it as “/ws” or “/api” is generally a good idea. Here we’ll use “/ws”.
Another thing we can determine now is the soap-namespace we will use in the wsdl for types and xxx. Here we will use http://wstuto.com/ and http://wstuto.com/types/.
Create a new directory in wstuto for storing all the webservice controllers and name it “ws”:
mkdir wstuto/ws
Add a new file called “__init__.py” in the wstuto/ws/ directory, with the following content:
import logging
from turbogears import config
from tgwebservices.controllers import WebServicesRoot, wsexpose, wsvalidate
log = logging.getLogger(__name__)
class WSController(WebServicesRoot):
def __init__(self):
super(WSController, self).__init__('http://localhost:%s/ws/' % config.get('server.socket_port'),
'http://wstuto.com/',
'http://wstuto.com/types/')
In wstuto/controllers.py, import your new controller and mount it:
# After all the other imports
from wstuto.ws import WSController
# ...
# In the root controller
class Root(controllers.RootController):
"""The root controller of the application."""
ws = WSController()
# ...
We now have a “/ws” controller that is the root of all the webservices we will expose.
You can already see the wsdl definition of your empty services at http://localhost:8080/ws/soap/api.wsdl.
Now we will create a dedicated controller for the user management functions, so that all the related calls we be done on URLs starting with “/ws/user”.
Create a file wstuto/ws/user.py with the following content:
# coding: utf-8
import logging
from tgwebservices.controllers import WebServicesController, wsexpose, wsvalidate
log = logging.getLogger(__name__)
class UserWSController(WebServicesController):
pass
Mount this controller on your webservice root by adding the following code to wstuto/ws/__init__.py:
# After all the other imports
from wstuto.ws.user import UserWSController
# ...
# In the webservice root controller definition :
class WSController(WebServicesRoot):
user = UserWSController()
# ...
For now the webservice still does nothing, but our controllers tree is ready.
Time to think a bit !
The apis we want to provide should allow a third-party application to manager our users, groups and permissions.
First we will need to define complex type to manipulate the user and group definitions at least.
Also, we will certainly manipulate user or group references. Since they have integer ids in the database, we can directly use integers. But they will be untyped references. To make things more strict, we will create custom types for references.
Most importantly, we will document the types right away, so that sphinx can extract the docstrings for the final documentation.
Add the following code to wstuto/ws/types.py:
import datetime
class UserRef(object):
"""A user reference"""
#: The user unique id
id = int
def __init__(self, obj=None):
self.id = None
if obj:
self.id = obj.user_id
class User(object):
"""A user data"""
#: The user unique id
id = int
#: The user name, used for login.
name = unicode
#: The user email address
email_address = unicode
#: The user display name
display_name = unicode
#: When was this user created
created = datetime.datetime
def __init__(self, obj=None):
self.id = None
self.name = None
self.email_address = None
self.display_name = None
self.created = None
if obj:
self.id = obj.user_id
self.name = obj.user_name
self.email_address = obj.email_address
self.display_name = obj.display_name
class UserList(object):
totalcount = int
offset = int
users = [User]
def __init__(self, **kw):
self.totalcount = kw.get('totalcount')
self.offset = kw.get('offset')
self.users = kw.get('users')
class GroupRef(object):
"""A user group reference"""
#: The group unique id
id = int
def __init__(self, id=None):
self.id = id
class Group(object):
"""A user group data"""
#: The group unique id
id = int
#: The group name
name = unicode
#: The group display name
display_name = unicode
#: When was this group created
created = datetime.datetime
def __init__(self):
self.id = None
self.name = None
self.display_name = None
self.created = None
class PermissionRef(object):
"""A permission reference"""
#: The permission unique id
id = int
def __init__(self, id=None):
self.id = id
class Permission(object):
"""A permission datas"""
#: The permission unique id
id = int
#: The permission name
name = unicode
#: The permission description
description = unicode
def __init__(self):
self.id = None
self.name = None
self.description = None
Note
It is possible to auto-create these types from the sqlalchemy mapped classes. This is not the aim of this tutorial though.
Although we created custom types for user, group and permission handling, we will concentrate on the user related functions only.
What do we need to provide ? In one ‘word’: CRUD (Create, Read, Update and Delete).
This can be done by a few functions:
In addition, we can have functions to (re)set a user password:
Let’s translate these ideas into tgws definitions. All we need to do is to expose functions on the UserWSController in wstuto/ws/user.py. We’ll also import the needed types from wstuto.ws.types, and document immediately the api:
# after all the other imports:
from wstuto.ws.types import User, UserRef, UserList
# complete the '/ws/user' controller :
class UserWSController(WebServicesController):
@wsexpose(User)
@wsvalidate(User)
def create(self, datas):
"""Create a new user.
:param datas: The initial datas of the user. The will be ignored.
"""
pass
@wsexpose(UserList)
@wsvalidate(int, int)
def list(self, limit=20, offset=0):
"""List all the users, with pagination.
:param limit: How many users should be returned at most.
:param offset: From which user should the list start.
"""
pass
@wsexpose(UserList)
@wsvalidate(str, str, str, int, int)
def query(self, user_name=None, email_address=None, display_name=None,
limit=20, offset=0):
"""Returns the users having the given values in their datas.
The applied operator is "in".
:param user_name: If non-null, a string that should be found in
the user name.
:param email_address: If non-null, a string that should be found in
the user email address.
:param display_name: If non-null, a string that should be found in
the user display name.
"""
pass
@wsexpose(User)
@wsvalidate(User)
def update(self, data):
"""Update a user datas and returns its new datas
:param data: The new values. null values will be ignored. The id
must exist in the database.
"""
pass
@wsexpose()
@wsvalidate(UserRef)
def delete(self, ref):
"""Delete a user.
:param ref: An existing user reference.
"""
pass
@wsexpose()
@wsvalidate(UserRef, str, str)
def change_password(self, ref, old_password, new_password):
"""Change a user password.
:param ref: The reference of the user.
:param old_password: The user old password.
:param new_password: The user new password.
"""
pass
@wsexpose()
@wsvalidate(UserRef, str)
def reset_password(self, ref, password):
"""Reset a user password.
:param ref: The reference of the user.
:param password: The new password.
"""
pass
Have another look to http://localhost:8080/ws/soap/api.wsdl, you should see a much more complex xml content than before that describe your types and functions.
Good news, the most difficult part is done ! Now that we properly defined the types, the functions and the documentation of our web services, the next steps are only technical formalities.
The documentation is actually already written in the code. All we need to do is to properly create a sphinx documentation project, install the tgwsdoc extension and write few directives to reference our code.
mkdir doc
cd doc
sphinx-quickstart
Sphinx quickstart will ask a few questions. Leave the defaults answers when there are some. For the others:
Modify a bit the html target in the generated Makefile (at line 33):
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) ../wstuto/static/doc
@echo
@echo "Build finished. The HTML pages are in ../wstuto/static/doc"
and make.bat (line 36) if your using windows:
if "%1" == "html" (
%SPHINXBUILD% -b html %ALLSPHINXOPTS% ../wstuto/static/doc
echo.
echo.Build finished. The HTML pages are in ../wstuto/static/doc.
goto end
)
Now build this quickstarted project to make sure its working properly:
cd doc
make html
Have a look at http://localhost:8080/static/doc/index.html, and admire your new (empty) embedded documentation.
In conf.py:
Add “tgwsdoc” to the extensions lists
Add “.” to sys.path.
Add the following line somewhere:
html_style = 'wstuto.css'
Prepare a few files :
Download http://tgws.googlecode.com/hg/contrib/tgwsdoc/tgwsdoc.py to the doc directory
Download http://tgws.googlecode.com/hg/contrib/tgwsdoc/toggle.css and http://tgws.googlecode.com/hg/contrib/tgwsdoc/toggle.js to doc/_static.
Create the file doc/_static/wstuto.css with the following content:
@import 'default.css';
@import 'toggle.css';
For more details, see tgwsdoc (Installing the extension).
Note
The tgwsdoc extension will be more easy to be installed when ported to Sphinx 1.0.
Add a doc/ws.rst file, in which we’ll document our application webservices:
WebServices
===========
Our great application provides webservices thanks to TGWebServices.
Types
-----
.. autotgwstype:: wstuto.ws.types.UserRef
.. autotgwstype:: wstuto.ws.types.User
.. autotgwstype:: wstuto.ws.types.UserList
.. autotgwstype:: wstuto.ws.types.GroupRef
.. autotgwstype:: wstuto.ws.types.Group
.. autotgwstype:: wstuto.ws.types.PermissionRef
.. autotgwstype:: wstuto.ws.types.Permission
Functions
---------
.. tgwsrootcontroller:: wstuto.ws.WSController
.. autotgwscontroller:: wstuto.ws.user.UserWSController
:members:
Now add this file to your index.rst toctree:
.. toctree::
:maxdepth: 2
ws
That’s it !
cd doc
make html
If everything goes well, you will obtain a nice documentation of your web services at http://localhost:8080/static/doc/ws.html.
We have a lot of functions defined, but we will concentrate on 2 of them : create, and change_password. The other ones are let as an exercise.
from turbogears.database import session
from wstuto import model
#...
class UserWSController(Controller):
@wsexpose(User)
@wsvalidate(User)
def create(self, datas):
"""Create a new user.
:param datas: The initial datas of the user. The will be ignored.
"""
user = model.User()
user.user_name = datas.name
user.email_address = datas.email_address
user.display_name = datas.display_name
session.add(user)
session.flush()
return User(user)
# ...
@wsexpose()
@wsvalidate(UserRef, str)
def reset_password(self, ref, password):
"""Reset a user password.
:param ref: The reference of the user.
:param password: The new password.
"""
user = session.query(model.User).get(ref.id)
user.password = password
session.flush()
The webservices we just wrote are accessible to anybody on the network, which might be a security problem.
To address this problem, the solution is very easy, we just need to use what TurboGears provides, identity.
Let’s say we add two permissions: “userread” for read-only functions, and “userwrite” for all modifying functions. Give this permission to a user, and protect your functions with the identity.requires decorator.
For example, the create and query functions definitions will be:
@wsexpose(User)
@wsvalidate(User)
@identity.requires(identity.has_permission("userwrite"))
def create(self, datas):
"""Create a new user
"""
@wsexpose(UserList)
@wsvalidate(str, str, str, int, int)
@identity.requires(identity.has_permission("userread"))
def query(self, user_name=None, email_address=None, display_name=None,
limit=20, offset=0):
"""Returns the users having the given values in their datas.
"""
For more information on this aspect, please refer to the TurboGears documentation.
Now that we have web services, we will see how they can be accessed with different protocols and languages.
First you will obviously need to run your application:
start-wstuto
...
Listening on http://localhost:8080/
The functions are callable doing a GET or POST to ws/user/<functionname>, giving parameters.
For example, for calling user/create, post this code to /ws/user/create:
<parameters>
<datas>
<name>newuser</name>
<email_address>newuser@example.com</email_address>
<display_name>New User</display_name>
</datas>
</parameters>
Same as REST+XML, except that json is used instead of xml. The post or get should add a “Accept = application/json” header to the request.
For example, to call user/delete, post this code to /ws/user/delete:
{
'ref': {
'id': 4
}
}
A wsdl file is available at /ws/soap/api.wsdl. Any decent soap client will be able to use this file to auto-create a mapping in the target language.
The functions names will be: userCreate, userQuery etc.
import simplejson
import liburl2
url = 'http://localhost:8080/ws/'
def call(fname, params):
req = urllib2.Request(url=url+fname, data=simplejson.dumps(params))
# If authentication is required, the easiest way is to use http
# authorization :
#
# req.add_header('Authorization',
# 'Basic %s' % base64.encodestring(
# '%s:%s' % (username, password))[:-1])
# Set the type of result we want
req.add_header('Accept', 'text/javascript')
# Set the type of data we send
req.add_header('Content-Type', 'application/json')
# Call the function
response = urllib2.urlopen(req)
return simplejson.loads(response)
call('user/create',
{
'datas': {
'name': 'newuser',
'email_address': 'newuser@example.com',
'display_name': 'New User'
}
})
With suds:
import datetime
from suds.client import Client
import suds
url = 'http://localhost:8080/ws/soap/api.wsdl'
client = Client(url, cache=None)
userdata = client.factory.create('{http://wstuto.com/types/}User')
userdata.id = 0
userdata.created = datetime.now()
userdata.name = 'newuser'
userdata.email_address = 'newuser@example.com'
userdata.display_name = 'New User'
userdata = client.service.userCreate(userdata)
print "The new user id is:", userdata.id
userref = client.factory.create('{http://wstuto.com/types/}UserRef')
userref.id = userdata.id
client.service.userReset_password(userref, 'test')