https://github.com/wragge/asio-files

Code to harvest and process publicly accessible ASIO files from the National Archives of Australia.
https://github.com/wragge/asio-files

archives code4lib recordsearch

Last synced: about 2 months ago
JSON representation

Code to harvest and process publicly accessible ASIO files from the National Archives of Australia.

• Host: GitHub
• URL: https://github.com/wragge/asio-files
• Owner: wragge
• License: cc0-1.0
• Created: 2015-08-29T02:28:28.000Z (over 9 years ago)
• Default Branch: master
• Last Pushed: 2015-09-05T13:34:49.000Z (over 9 years ago)
• Last Synced: 2024-10-12T15:56:27.138Z (3 months ago)
• Topics: archives, code4lib, recordsearch
• Language: Python
• Size: 364 KB
• Stars: 3
• Watchers: 3
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# asio-files

Code to harvest and process publicly accessible ASIO files from the National Archives of Australia.

## DIY harvesting

If you want to harvest the data yourself you'll need to clone my [recordsearch-tools](https://github.com/wragge/recordsearch_tools) repository into a direct called 'rstools'. Then in Python you can just:

````

import harvest

# Initiate harvester with a series id

harvester = harvest.SeriesClient(series='A6119')

# Harvest item metadata

harvester.do_harvest()

# Harvest ALL the digitised images in this series

harvest.harvest_images()

````

Note that harvest_images() is set up to create derivatives of every image. Just comment that out if it's not what you want.

Note too that this will work with any series in the National Archives of Australia, not just the ASIO ones!

## Here's a harvest I prepared earlier

Unless you're very keen and very patient you probably don't want to harvest everything yourself. I'm happy to make the results of my own harvests available for research.

Here's [a summary](https://github.com/wragge/asio-files/blob/master/data/series_summary.csv) of the series I've harvested.

### Metadata

Here's the basic metadata for every item saved as a CSV file:

* [A6119](https://github.com/wragge/asio-files/blob/master/data/A6119.csv)

* [A6122](https://github.com/wragge/asio-files/blob/master/data/A6122.csv)

* [A6126](https://github.com/wragge/asio-files/blob/master/data/A6126.csv)

* [A9626](https://github.com/wragge/asio-files/blob/master/data/A9626.csv)

### Images

If you'd like copies of all the images I've harvested, let me know. I'm keen to encourage further research using these files. Here are the README files for each of the image dumps:

* [README-A6119](https://github.com/wragge/asio-files/blob/master/docs/README-A6119.md)

* [README-A6122](https://github.com/wragge/asio-files/blob/master/docs/README-A6122.md)

* [README-A6126](https://github.com/wragge/asio-files/blob/master/docs/README-A6126.md)

* [README-A9626](https://github.com/wragge/asio-files/blob/master/docs/README-A9626.md)

All together there are about 300,000 images, comprising nearly 70gb of data.

Here's a [sample file for each of the series](https://www.dropbox.com/sh/e6x65z73h4kmhty/AAAlub1KKQ_rsEhXwGC557cOa?dl=0) on DropBox.