pyramid_storage is a simple file upload manager for the Pyramid framework. It currently supports uploads to the local file system and to the Amazon S3 cloud storage service.
Install with pip install pyramid_storage. To install from source, unzip/tar, cd and python setup.py install.
Please report bugs and issues (and better still, pull requests) on the Github repo.
There are a number of ways to configure pyramid_storage with your Pyramid app.
The easiest way is to add pyramid_storage to the pyramid.includes directive in your configuration file(s):
pyramid.includes =
pyramid_storage
Alternatively you can use pyramid.config.Configurator.include() in your app setup:
config.include('pyramid_storage')
Either setup will add an instance of pyramid_storage.storage.FileStorage to your app registry. The instance will also be available as a property of your request as request.storage.
To use S3 file storage instead of storing files locally on your server (the default assumption):
pyramid.includes =
pyramid_storage.s3
alternatively:
config.include('pyramid_storage.s3')
Local file storage (default)
The available settings are listed below:
| Setting | Default | Description |
|---|---|---|
| base_path | required | Absolute location for storing uploads |
| base_url | Relative or absolute base URL for uploads; must end in slash (“/”) | |
| extensions | default | List of extensions or extension groups (see below) |
| name | storage | Name of property added to request, e.g. request.storage |
S3 file storage
| Setting | Default | Description |
|---|---|---|
| aws.access_key | required | AWS access key |
| aws.secret_key | required | AWS secret key |
| aws.bucket_name | required | AWS bucket |
| aws.acl | public-read | AWS ACL permissions |
| base_url | Relative or absolute base URL for uploads; must end in slash (“/”) | |
| extensions | default | List of extensions or extension groups (see below) |
| name | storage | Name of property added to request, e.g. request.storage |
Configuring extensions: extensions are given as a list of space-separated extensions or groups of extensions. These groups provide a convenient shortcut for including a large number of extensions. Each group must be separated by a plus-sign “+”. Some examples:
The extension groups are listed below:
| Group | Extensions |
|---|---|
| any | all extensions (including no extensions) |
| text | txt |
| documents | pdf rtf odf ods gnumeric abw doc docx xls xls |
| images | jpg jpe jpeg png gif svg bmp tiff |
| audio | wav mp3 aac ogg oga flac |
| video | mpeg 3gp avi divx dvr flv mp4 wmv |
| data | csv ini json plist xml yaml yml |
| scripts | js php pl py rb sh |
| archives | gz bz2 zip tar tgz txz 7z |
| executables | so exe dll |
| default | documents+images+text+data |
Warning
It is the responsibility of the deployment team to ensure that target directories used in file uploads have the appropriate read/write permissions.
When uploading a file in a view, call pyramid_storage.storage.FileStorage.save() to save the file to your file system:
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPSeeOther
@view_config(route_name='upload',
request_method='POST')
def upload(request):
request.storage.save(request.POST['my_file'])
return HTTPSeeOther(request.route_url('home'))
This operation will save the file to your file system under the top directory specified by the base_path setting.
If the file does not have the correct file extension, a pyramid_storage.exceptions.FileNotAllowed exception is raised. A more secure way of writing the above would be:
from pyramid_storage.exceptions import FileNotAllowed
@view_config(route_name='upload',
request_method='POST')
def upload(request):
try:
request.storage.save(request.POST['my_file'])
except FileNotAllowed:
request.session.flash('Sorry, this file is not allowed')
return HTTPSeeOther(request.route_url('home'))
You can override the default extensions in the method call:
from pyramid_storage.exceptions import FileNotAllowed
@view_config(route_name='upload',
request_method='POST')
def upload(request):
try:
request.storage.save(request.POST['my_file'],
extensions=('jpg', 'png', 'txt'))
except FileNotAllowed:
request.session.flash('Sorry, this file is not allowed')
return HTTPSeeOther(request.route_url('home'))
You may also wish to obfuscate or randomize the filename. The randomize argument will generate a random filename, preserving the extension:
filename = request.storage.save(request.POST['my_file'], randomize=True)
So for example if your filename is test.jpg the new filename will be something like 235a344c-8d70-498a-af0a-151afdfcd803.jpg.
If there is a filename clash (i.e. another file with the same name is in the target directory) a numerical suffix is added to the new filename. For example, if you have an existing file test.jpg then the next file with that name will be renamed test-1.jpg and so on.
Warning
Remember to ensure your forms include the attribute enctype=”multipart/form-data” or your uploaded files will be empty.
If you pass in the folder argument this will be used to add subfolder(s):
request.storage.save(request.POST['my_file'], folder="photos")
The above call will store the contents of my_file under the directory photos under your base path.
If you want to check in advance that the extension is permitted (for example, in the form validation stage) you can use pyramid_storage.storage.FileStorage.file_allowed():
request.storage.file_allowed(request.POST['my_file'])
To access the URL of the file, for example in your Python code or templates, use the pyramid_storage.storage.FileStorage.url() method:
request.storage.url(filename)
You may not wish to provide public access to files - for example users may upload to private directories. In that case you can simply serve the file in your views:
from pyramid.response import FileResponse
@view_config(route_name='download')
def download(request):
filename = request.params['filename']
return FileResponse(request.storage.path(filename))
Warning
S3 support requires you install the Boto library separately (e.g. pip install boto). As of writing this library is not Python 3 compatible.
Warning
It is the responsibility of the deployment team to ensure that the application has the correct AWS settings and permissions.
Basic usage is similar to LocalFileStorage:
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPSeeOther
@view_config(route_name='upload',
request_method='POST')
def upload(request):
request.storage.save(request.POST['my_file'])
return HTTPSeeOther(request.route_url('home'))
One difference is that filenames are not resolved with a numeric suffix as with local files, to prevent network round-trips. Instead you can pass the replace argument to replace the file on s3 (default is False):
from pyramid.view import view_config
from pyramid.httpexceptions import HTTPSeeOther
@view_config(route_name='upload',
request_method='POST')
def upload(request):
request.storage.save(request.POST['my_file'], replace=True)
return HTTPSeeOther(request.route_url('home'))
Alternatively you can use the randomize argument to ensure a (near) unique filename.
The storage.base_url setting should be set to //s3amazonaws.com/<my-bucket-name>/ unless you want to serve the file behind a proxy or through your Pyramid application.
It’s easier to run unit tests on your upload views without actually handling real files. The class pyramid_storage.storage.DummyFileStorage provides a convenient way to mock these operations.
This class stores the names of the files internally for your assertions in the saved attribute:
import mock
from pyramid.testing import DummyRequest
from pyramid_storage.storage import DummyFileStorage
def test_my_upload():
from .views import my_upload_view
req = DummyRequest()
req.storage = DummyFileStorage()
my_file = mock.Mock()
my_file.filename = 'test.jpg'
req.POST['my_file'] = my_file
res = my_upload_view(req)
assert 'test.jpg' in req.storage.saved
Not that DummyFileStorage only provides one or two convenience methods. You may wish to extend this class for your own specific needs.
Manages storage and retrieval of file uploads to local filesystem on server.
| Parameters: |
|
|---|
Deletes the filename. Filename is resolved with the absolute path based on base_path. If file does not exist, returns False, otherwise True
| Parameters: | filename – base name of file |
|---|
Checks if file exists. Resolves filename’s absolute path based on base_path.
| Parameters: | filename – base name of file |
|---|
Checks if an extension is permitted. Both e.g. ”.jpg” and “jpg” can be passed in. Extension lookup is case-insensitive.
| Parameters: | extensions – iterable of extensions (or self.extensions) |
|---|
Checks if a file can be saved, based on extensions
| Parameters: |
|
|---|
Checks if a filename has an allowed extension
| Parameters: |
|
|---|
Returns a new instance from config settings.
| Parameters: |
|
|---|
Returns absolute file path of the filename, joined to the base_path.
| Parameters: | filename – base name of file |
|---|
Resolves a unique name and the correct path. If a filename for that path already exists then a numeric prefix will be added, for example test.jpg -> test-1.jpg etc.
| Parameters: |
|
|---|
Saves contents of a cgi.FieldStorage object to the file system. Returns modified filename(including folder). If there is a clash with an existing filename then filename will be resolved accordingly. If path directories do not exist they will be created.
Returns the resolved filename, i.e. the folder + the (randomized/incremented) base name.
| Parameters: |
|
|---|---|
| Returns: | modified filename |
Saves a file object to the uploads location. Returns the resolved filename, i.e. the folder + the (randomized/incremented) base name.
| Parameters: |
|
|---|---|
| Returns: | modified filename |
Saves a filename in local filesystem to the uploads location.
Returns the resolved filename, i.e. the folder + the (randomized/incremented) base name.
| Parameters: |
|
|---|---|
| Returns: | modified filename |
Deletes the filename. Filename is resolved with the absolute path based on base_path. If file does not exist, returns False, otherwise True
| Parameters: | filename – base name of file |
|---|
Checks if an extension is permitted. Both e.g. ”.jpg” and “jpg” can be passed in. Extension lookup is case-insensitive.
| Parameters: | extensions – iterable of extensions (or self.extensions) |
|---|
Checks if a file can be saved, based on extensions
| Parameters: |
|
|---|
Checks if a filename has an allowed extension
| Parameters: |
|
|---|
Saves contents of a cgi.FieldStorage object to the file system. Returns modified filename(including folder).
Returns the resolved filename, i.e. the folder + (modified/randomized) filename.
| Parameters: |
|
|---|---|
| Returns: | modified filename |
| Parameters: |
|
|---|---|
| Returns: | modified filename |
Saves a filename in local filesystem to the uploads location.
Returns the resolved filename, i.e. the folder + the (randomized/incremented) base name.
| Parameters: |
|
|---|---|
| Returns: | modified filename |