https://github.com/datadotworld/ckanext-datadotworld
CKAN extension for data.world
https://github.com/datadotworld/ckanext-datadotworld
ckan ckan-extension ckanext datasets dwstruct-t50-public-projects open-data reference-implementation
Last synced: about 2 months ago
JSON representation
CKAN extension for data.world
- Host: GitHub
- URL: https://github.com/datadotworld/ckanext-datadotworld
- Owner: datadotworld
- License: apache-2.0
- Created: 2017-04-05T18:59:02.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2023-12-05T05:20:35.000Z (over 1 year ago)
- Last Synced: 2025-05-08T21:14:32.496Z (about 2 months ago)
- Topics: ckan, ckan-extension, ckanext, datasets, dwstruct-t50-public-projects, open-data, reference-implementation
- Language: Python
- Homepage: https://data.world/integrations/ckan
- Size: 161 KB
- Stars: 12
- Watchers: 41
- Forks: 9
- Open Issues: 3
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
README
=============
ckanext-datadotworld
=============
With this extension enabled, the manage view for organizations is provided with the additional tab
`data.world`. Within the data.world tab organization admins can specify syncronization options that will apply for that organization.
------------------
Supported versions
------------------
CKAN version 2.4 or greater (including 2.7).
All versions support celery backend, but version 2.7 will use RQ.
There are no changes required to use new backend - just start
it using::
paster --plugin=ckan jobs worker -c /config.ini
instead of::
paster --plugin=ckan celeryd run -c /config.ini
Details at http://docs.ckan.org/en/latest/maintaining/background-tasks.html
------------
Installation
------------
To install ckanext-datadotworld:
1. Activate your CKAN virtual environment, for example::
. /usr/lib/ckan/default/bin/activate
2. If you already have an older version of this extension, remove it first::
pip uninstall -y ckanext-datadotworld
Install the ckanext-datadotworld Python package into your virtual environment::
pip install git+https://github.com/datadotworld/ckanext-datadotworld
3. Add ``datadotworld`` to the ``ckan.plugins`` setting in your CKAN config file (by default the config file is located at ``/etc/ckan/default/production.ini``).
4. Create DB tables::
paster --plugin=ckanext-datadotworld datadotworld init -c /config.ini
paster --plugin=ckanext-datadotworld datadotworld upgrade -c /config.ini
5. Start celery daemon either with suprevisor or using paster::
paster --plugin=ckan celeryd run -c /config.ini
---------------
Config Settings
---------------
Attempts to push failed datasets can be scheduled by adding the following line to cron::
* 8 * * * paster --plugin=ckanext-datadotworld datadotworld push_failed -c /config.ini
A similar solution enables syncronization with remote (i.e. not uploaded) resources with data.world::
* 8 * * * paster --plugin=ckanext-datadotworld datadotworld sync_resources -c /config.ini
**Delay option**
There is a 1 second delay configured by default. This delay period can be controlled by modifying the "ckan.datadotworld.request_delay" configuration variable within the CKAN ini file.
For example:
ckan.datadotworld.request_delay = 1
To ensure that the delay will work correctly, you also need to configure Celery to work in single thread mode. To do this, add the following flag to the Celery start command:
--concurrency=1
Details at http://celery.readthedocs.io/en/latest/userguide/workers.html#concurrency.
-----------------
Template snippets
-----------------
In order to add data.world banner on dataset page(currently it seats at the top of `package_resources` block)
you may add next snippet to template with `datadotworld_extras` variable that contains object(model) with
currently viewed package's datadotworld extras and `org_id` - owner organization of viewed packaged::
{% snippet 'snippets/datadotworld/banner.html', org_id=pkg.owner_org, datadotworld_extras=c.pkg.datadotworld_extras %}
Sidebar label may be added by placing next snippet to your template(`org_id` is ID of viewed organization)::
{% snippet 'snippets/datadotworld/label.html', org_id=organization.id %}
------------------------
Development Installation
------------------------
To install ckanext-datadotworld for development, activate your CKAN virtualenv and
do the following::
git clone https://github.com/datadotworld/ckanext-datadotworld.git
cd ckanext-datadotworld
python setup.py develop
paster datadotworld init -c /config.ini
-----------------
Running the Tests
-----------------
Make sure you follow the CKAN testing guide (http://docs.ckan.org/en/latest/contributing/test.html).
To run the tests, do the following::
nosetests --ckan --nologcapture --with-pylons=test.ini
To run the tests and produce a coverage report, first make sure you have coverage installed in your virtualenv (``pip install coverage``) then run::
nosetests --ckan --nologcapture --with-pylons=test.ini --with-coverage --cover-package=ckanext.datadotworld --cover-inclusive --cover-erase --cover-tests