{"id":15568637,"url":"https://github.com/jayvdb/era_data","last_synced_at":"2026-01-11T23:00:00.264Z","repository":{"id":71577558,"uuid":"54857757","full_name":"jayvdb/era_data","owner":"jayvdb","description":"ERA ( Excellence in Research for Australia ) reference data","archived":false,"fork":false,"pushed_at":"2021-10-27T00:57:40.000Z","size":42,"stargazers_count":0,"open_issues_count":8,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-02T08:48:40.330Z","etag":null,"topics":["analysis","appveyor","australian","continuous-integration","dataset","era","era-data","field-of-research","journal-list","journal-rank","msys2","mysql-database","oracle-ci","oracle-database","pgsql","reference-data","sqlite3","travis-ci","travis-oracle","unix"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jayvdb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-28T01:51:03.000Z","updated_at":"2021-10-27T00:57:42.000Z","dependencies_parsed_at":"2023-02-27T20:15:38.888Z","dependency_job_id":null,"html_url":"https://github.com/jayvdb/era_data","commit_stats":{"total_commits":22,"total_committers":2,"mean_commits":11.0,"dds":"0.045454545454545414","last_synced_commit":"048fd6d168f4f6c02cc8aeca7fa06574c23ff1c4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jayvdb/era_data","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jayvdb%2Fera_data","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jayvdb%2Fera_data/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jayvdb%2Fera_data/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jayvdb%2Fera_data/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jayvdb","download_url":"https://codeload.github.com/jayvdb/era_data/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jayvdb%2Fera_data/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28326166,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-11T22:11:01.104Z","status":"ssl_error","status_checked_at":"2026-01-11T22:10:58.990Z","response_time":60,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analysis","appveyor","australian","continuous-integration","dataset","era","era-data","field-of-research","journal-list","journal-rank","msys2","mysql-database","oracle-ci","oracle-database","pgsql","reference-data","sqlite3","travis-ci","travis-oracle","unix"],"created_at":"2024-10-02T17:19:59.602Z","updated_at":"2026-01-11T23:00:00.249Z","avatar_url":"https://github.com/jayvdb.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"ERA data\n========\n\nThis repository contains tools to fetch and load data from the\n[Excellence in Research for Australia](https://en.wikipedia.org/wiki/Excellence_in_Research_for_Australia)\nprogram (ERA) into a database.\n\nThe motivation is the reference data for 2010 and 2012 ERA came in slightly different formats,\nand the data had unusual copyright protections when it was released from the Australian Research Council (ARC),\nin part due to [the Australian Crown copyright](https://en.wikipedia.org/wiki/Crown_copyright#Australia).\n\nCopyright\n---------\nThe [ARC website copyright notice](http://www.arc.gov.au/copyright.htm) mentions\n[Creative Commons (CC) BY 3.0](http://creativecommons.org/licenses/by/3.0/legalcode)\nhowever it also has an other \"Other Copyright\" section referring to restrictive\nCrown copyright, and no statement regarding when CC or Crown applies.\n\nIn addition, the datasets for ERA 2010 and ERA 2012 can no longer be found on\ntheir website, and the copyright notice did not mention CC licenses when these\ndatasets were available from their website.\n\nTo avoid this copyright problem, this repository does not contain any ERA data.\nInstead it provides tools to fetch and transform the datasets into other formats,\nand will do analysis on the dataset in the cloud.\n\nAll files stored in this repository are available under the MIT license.\n\nData sources\n------------\nThis repository has a script `fetch.sh` that downloads the following two files from [Australian Government Web Archive](http://content.webarchive.nla.gov.au/):\n* [ERA 2010 tech pack](http://content.webarchive.nla.gov.au/gov/wayback/20120317002747/http://www.arc.gov.au/zip/ERA2010_tech_pack.zip)\n* [ERA 2012 Journal List](http://content.webarchive.nla.gov.au/gov/wayback/20140212052430/http://www.arc.gov.au/xls/era12/ERA2012JournalList.xlsx)\n\nUsage\n-----\nThe datasets are processed automatically after each checkin on the following two services:\n* [Travis-CI](https://travis-ci.org/jayvdb/era_data/builds) - four Linux jobs, loading the data into Sqlite3, PosgresSQL, Mysql and Oracle databases.  The databases are destroyed at the end of the build.\n* [Appveyor CI](https://ci.appveyor.com/project/jayvdb/era-data) - one Microsoft Windows job, producing [artifacts](https://ci.appveyor.com/project/jayvdb/era-data/build/artifacts) from the transformations.\n\nOne way to use this repository is to fork the repository on Github, and\nadd your own transforms or analysis to the scripts in your fork.\nThey can be run automatically using your own Travis-CI and/or Appveyor CI account.\n\nAlternatively you can recreate the steps performed by Travis-CI or Appveyor on your local machine,\nusing a private database.\n\nUnix\n----\nSee [.travis.yml](https://github.com/jayvdb/era_data/blob/master/.travis.yml)\n\n`fetch.sh` only requires either the Unix `unzip` or [`7-zip`](http://www.7-zip.org/) in the path,\nand either the Unix `wget` or the [Python `wget` package](https://pypi.python.org/pypi/wget)\nwhich can be installed using the pip requirements file `win-requirements.txt`.\n\nOther necessary executables include bzip2, sed, perl, and python.\n\nWindows\n-------\nThe Windows build depend on most of the same commands as the Unix build.\n\nThe most common way to achieve that on Windows is to set up\n[Cygwin](https://cygwin.com/) or [MSYS](http://www.mingw.org/wiki/msys).\n\nThe scripts in this repository have been tested on Cygwin and MSYS 1.0 and MSYS 2.0.\n\nThe `7z` executable may be located in a directory containing a space.\nAll other executables must be in paths that do not include spaces.\n\nSee [appveyor.yml](https://github.com/jayvdb/era_data/blob/master/appveyor.yml)\n\nTravis CI configuration\n-----------------------\nIn order to run SQL against a Oracle database, this repository downloads and installs Oracle XE.\nTo download Oracle XE, the Travis CI settings must include environment variables `ORACLE_LOGIN_ssousername` and `ORACLE_LOGIN_password`.\nSee [travis-oracle](https://github.com/cbandy/travis-oracle) for more information on how this works.\n\nContributing\n------------\nFeel free to submit pull requests for additional transforms or analysis.\nDont worry if you have only tested the changes against the databases that you have access to.\ni.e. Open door policy. The more the merrier. Submit pull requests early.\nIt is my job to figure out how to incorporate any new code into the repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjayvdb%2Fera_data","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjayvdb%2Fera_data","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjayvdb%2Fera_data/lists"}