https://github.com/cldellow/url-cache
Read a URL from the Internet, fetching from local cache if available.
https://github.com/cldellow/url-cache
Last synced: 9 months ago
JSON representation
Read a URL from the Internet, fetching from local cache if available.
- Host: GitHub
- URL: https://github.com/cldellow/url-cache
- Owner: cldellow
- Created: 2019-03-09T21:36:30.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2019-03-09T22:10:11.000Z (over 7 years ago)
- Last Synced: 2025-03-29T12:13:21.004Z (over 1 year ago)
- Language: Shell
- Size: 12.7 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# url-cache
Read a file from the Internet, unless we've already fetched it,
in which case, read it from the local cache.
This is minimalist. Known issues:
- Doesn't do connection pooling
- Multiple requests for the same file from different threads may race
and result in a corrupt file
- The naming schema for `fetchNamed` can result in conflicts
...in other words, it's not bullet proof. But it's probably suitable
for most purposes.
```scala
import com.cldellow.urlcache.Cache
// Caches to /tmp/cache/d6/57/d657127c40d4ecfe29c74c366b3198bd
Cache.fetch("https://google.com/robots.txt")
// Caches to /tmp/mydir/d6/57/d657127c40d4ecfe29c74c366b3198bd
new Cache("/tmp/mydir").fetch("https://google.com/robots.txt")
// Cache in /tmp/cache/mydir/d6/57/d657127c40d4ecfe29c74c366b3198bd
Cache.fetch("https://google.com/robots.txt", prefix = "mydir"))
// Cache to /tmp/cache/misc/https_google.com_robots.txt
Cache.fetchNamed("https://google.com/robots.txt")
// Cache first 100 bytes to /tmp/cache/d6/57/d657127c40d4ecfe29c74c366b3198bd-0-100
Cache.fetch("https://google.com/robots.txt", range = (0, 100))
```