https://github.com/pwwang/remotedata

Accessing and caching remote data.
https://github.com/pwwang/remotedata

Last synced: 14 days ago
JSON representation

Accessing and caching remote data.

• Host: GitHub
• URL: https://github.com/pwwang/remotedata
• Owner: pwwang
• Created: 2019-07-02T19:30:43.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2020-04-22T19:18:57.000Z (over 4 years ago)
• Last Synced: 2024-12-10T04:27:01.073Z (24 days ago)
• Language: Python
• Size: 1.08 MB
• Stars: 1
• Watchers: 2
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# remotedata

Accessing and caching remote data for python.

May be used in the cases that:

1. The remote data is being updated frequently

2. You don't want to sync all the data but just per your request

3. You want to cache the data locally for some time

4. Especially, when the files are used for testing

## Installation

```shell

pip install remotedata

```

## Usage

Currently, data from `github` and `dropbox` are supported

### Github

```python

from remotedata import remotedata

rdata = remotedata(dict(

	source = 'github',

	cachedir = '/tmp/cached/',

	## if branch is not master: pwwang/remotedata/branch

	repos  = 'pwwang/remotedata',

	## optional, default is first part of repos

	# user = 'pwwang',

	## github token, in case you have > 60 requests per hours to github API

	# token = 'xxx',

))

readme = rdata.get('README.md')

# README.md is downloaded to /tmp/cache/github/pwwang.remotedata@master/README.md

# Now you can use it as a local file

# readme will be cached, we don't have to download it again,

# until it has been changed remotely.

# remove cached file

rdata.remove('README.md')

# clear up all caches

rdata.clear()

```

### Dropbox

```python

from remotedata import remotedata

rdata = remotedata(dict(

	source = 'dropbox',

	cachedir = '/tmp/cached/',

	dropbox_token = 'xxx'

))

rdata.get('/somefile') # or

rdata.get('somefile')

```