{"id":22621435,"url":"https://github.com/cca/scrape-portal-img","last_synced_at":"2025-03-29T02:24:29.962Z","repository":{"id":84610441,"uuid":"62913117","full_name":"cca/scrape-portal-img","owner":"cca","description":"grab images for Koha patron records by scraping from the CCA Portal","archived":false,"fork":false,"pushed_at":"2016-07-08T19:48:38.000Z","size":8,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-02-03T12:35:18.000Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cca.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-07-08T19:42:33.000Z","updated_at":"2016-07-08T19:43:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"613ecbdb-e390-4110-9e43-5a6c07e68f35","html_url":"https://github.com/cca/scrape-portal-img","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cca%2Fscrape-portal-img","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cca%2Fscrape-portal-img/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cca%2Fscrape-portal-img/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cca%2Fscrape-portal-img/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cca","download_url":"https://codeload.github.com/cca/scrape-portal-img/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246127759,"owners_count":20727783,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-08T23:08:53.860Z","updated_at":"2025-03-29T02:24:29.956Z","avatar_url":"https://github.com/cca.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scraping Portal Profile Images\n\nA small project to grab images for Koha patron records by scraping them from the [CCA Portal](https://portal.cca.edu). See [the Koha Manual page](http://manual.koha-community.org/3.22/en/tools.html#uploadpatronimages) on bulk uploading patron images. The basic process is:\n\n- download a list of patrons from Koha\n    + need at least barcode \u0026 username\n    + can filter out people where we don't have their CCA username since we won't be able to look them up on the portal\n- convert the download to a JSON file named \"patrons.json\" formatted like so:\n\n```json\n[\n    [ \"PCAC0000000000\", \"booradley\" ],\n    [ \"PCAC0000000001\", \"hlewis\" ],\n    [ \"barcode\", \"username\" ]\n]\n```\n\nThat can be done with a single search-and-replace usually.\n\n- run `node index \u003e urls.csv` to write profile image URLs to a file\n    + stdout gets a CSV-like row with barcode, CCA username, \u0026 URL\n    + stderr receives notices about users whose images we couldn't find\n- download all the images into the \"img\" folder with the included \"dl.sh\" script\n    + the \"img\" folder and \"urls.csv\" filenames are hard-coded into this script\n- dl.sh also creates the necessary IDLINK.txt file\n    + note: since dl.sh names the images using the patron's barcode, barcodes ending in a forward slash \"/\" result in an invalid file name \u0026 the download does not complete successfully, this is a bug that I should fix if I ever need to use this tool again\n- zip \u0026 upload the \"img\" folder (containing IDLINK.txt) to Koha https://library-staff.cca.edu/cgi-bin/koha/tools/picture-upload.pl\n\n# Requirements\n\n- node \u0026 npm\n- run `npm install` to get project dependencies\n- python csvkit (`pip install csvkit`) is needed for the \"dl.sh\" script\n\n# LICENSE\n\n[ECL Version 2.0](https://opensource.org/licenses/ECL-2.0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcca%2Fscrape-portal-img","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcca%2Fscrape-portal-img","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcca%2Fscrape-portal-img/lists"}