https://github.com/garethr/dumper

A static file generator for read only web services. Currently supports MySQL backend only.
https://github.com/garethr/dumper

Last synced: 10 days ago
JSON representation

A static file generator for read only web services. Currently supports MySQL backend only.

• Host: GitHub
• URL: https://github.com/garethr/dumper
• Owner: garethr
• Created: 2010-03-13T18:22:42.000Z (almost 15 years ago)
• Default Branch: master
• Last Pushed: 2010-03-19T21:13:57.000Z (almost 15 years ago)
• Last Synced: 2024-12-11T15:56:13.529Z (12 days ago)
• Language: Python
• Homepage:
• Size: 117 KB
• Stars: 4
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.textile

Awesome Lists containing this project

README

        
Dumper is a static file generator for building simple read only web services. Think "Jekyll":http://github.com/mojombo/jekyll or "Nanoc":http://nanoc.stoneship.org/ but for XML and JSON.

Still to do:

* bundling commands for deploying changes

* build in web server for testing

* versioning

* date based snapshots

* support for more output formats than JSON and XML

* support for more backends than just MySQL

* logging other than command line output

h2. Trying Dumper

First you'll need a MySQL database. I've provided a simple table definition and some data to get you started. See the tests folder for an example sql file.

mysqladmin create dumper -u root

mysql dumper < people.sql -u root



You can now install dumper using the provided setup.py file. This should install all the requirements. I'd recommend using virtualenv to sandbox your python environment.

python setup.py install


You should now have the dumper command line application available on your path. Dumper uses a configuration file to determine which backend to use. An example configuration file is shown below:

[Dumper]

path: people

index: id

backend: mysql


[Database]

sql: SELECT id, name FROM people

host: localhost

username: root

password:

database: dumper


Using the example file from the tests directory like so:

dumper -c people.truck dump


This should give you something like this output to the console:

[Created] output/people

[Created] output/people/1.json

[Created] output/people/1.xml

[Created] output/people/2.json

[Created] output/people/2.xml

[Created] output/people.json

[Created] output/people.xml


Run dumper help for a full list of options.

h2. Processors

Processors allow you to massage the data taken from the SQL query or to modify the XML output to match your preferred schema. You specify processors in the ini file like so:

[Dumper]

path: people

index: id

backend: mysql

post_processors: examplepre

pre_processors: examplepost


These are simply python modules on your python path. Note that the directory where you run dumper is added to the python path to make it easier. Processors run one after the other, passing the output of the first to the second and so on. Preprocessors take the python list or dictionary with the results of the SQL query. Postprocessors take the XML string before it is written to disk. A simple processor might just delete all XML output like so:

def processor(input):

    return ""


More likely you'll want to do validation on data based on an external, non-sql source, or you'll want to transform the XML to match a specific output format.

h2. Subclassing

But what if you want complete control over how things are serialized. Or even how things are output? You can register your own subclasses as backends and use those.

The following is just a noddy example. With this class available you can use the backend 'mysql2' and it will work just the same as the 'mysql' backend.

from dumper import BaseDumper, MySQLDumper


class MySQLDumper2(MySQLDumper, BaseDumper):

    @classmethod

    def is_dumper_for(cls, type):

	"The string passed in to the factory to get this class"

	return type == 'mysql2'


In order for the backend detection to work you need to get these classes imported. For that we have the import parameter in the config file which allows for a comma seperated list of import commands.