https://github.com/egorsmkv/dspace-checker
A simple Go app that collects scientific articles from Dspace repos using Geziyor
https://github.com/egorsmkv/dspace-checker
dspace scientific-publications
Last synced: 8 months ago
JSON representation
A simple Go app that collects scientific articles from Dspace repos using Geziyor
- Host: GitHub
- URL: https://github.com/egorsmkv/dspace-checker
- Owner: egorsmkv
- Created: 2022-11-10T16:47:51.000Z (about 3 years ago)
- Default Branch: master
- Last Pushed: 2022-11-10T16:50:06.000Z (about 3 years ago)
- Last Synced: 2025-02-02T10:23:07.541Z (10 months ago)
- Topics: dspace, scientific-publications
- Language: Go
- Homepage:
- Size: 35.2 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Dspace checker
## Overview
A simple Go app that collects scientific articles from Dspace repos using Geziyor
## Install
```bash
go build -o dspace-checker
```
## Usage
Add sources into the `sources.txt` file and then run the app:
```bash
./dspace-checker
```
It will generate the file `out.jsonl` with all results.
Program's output will be like the following:
```text
Scraping Started
URL already visited http://dspace.idgu.edu.ua/jspui/browse?type=dateissued&sort_by=2&order=DESC&rpp=20&etal=0&submit_browse=%D0%9E%D0%BD%D0%BE%D0%B2%D0%B8%D1%82%D0%B8
Crawled: (200)
Crawled: (200)
Crawled: (200)
Crawled: (200)
Crawled: (200)
Crawled: (200)
Scraping Finished
```