https://github.com/inspirehep/hepcrawl

Scrapy project for feeds into INSPIRE-HEP
https://github.com/inspirehep/hepcrawl

crawler harvest-data publishing python

Last synced: about 4 hours ago
JSON representation

Scrapy project for feeds into INSPIRE-HEP

• Host: GitHub
• URL: https://github.com/inspirehep/hepcrawl
• Owner: inspirehep
• License: other
• Created: 2015-10-26T15:55:58.000Z (about 9 years ago)
• Default Branch: master
• Last Pushed: 2024-07-18T17:44:26.000Z (4 months ago)
• Last Synced: 2024-10-13T22:35:57.753Z (25 days ago)
• Topics: crawler, harvest-data, publishing, python
• Language: Python
• Homepage: http://inspirehep.net
• Size: 3.97 MB
• Stars: 17
• Watchers: 20
• Forks: 30
• Open Issues: 37
• Metadata Files:
  • Readme: README.rst
  • Contributing: docs/contributing.rst
  • License: LICENSE

Awesome Lists containing this project

README

        
..

    This file is part of hepcrawl.

    Copyright (C) 2015, 2016, 2017 CERN.

    hepcrawl is a free software; you can redistribute it and/or modify it

    under the terms of the Revised BSD License; see LICENSE file for

    more details.

==========

 HEPcrawl

==========

.. image:: https://img.shields.io/travis/inspirehep/hepcrawl.svg

        :target: https://travis-ci.org/inspirehep/hepcrawl

.. image:: https://img.shields.io/github/tag/inspirehep/hepcrawl.svg

        :target: https://github.com/inspirehep/hepcrawl/releases

.. image:: https://img.shields.io/pypi/dm/hepcrawl.svg

        :target: https://pypi.python.org/pypi/hepcrawl

.. image:: https://img.shields.io/github/license/inspirehep/hepcrawl.svg

        :target: https://github.com/inspirehep/hepcrawl/blob/master/LICENSE

HEPcrawl is a harvesting library based on Scrapy (http://scrapy.org) for INSPIRE-HEP

(http://inspirehep.net) that focuses on automatic and semi-automatic retrieval of

new content from all the sources the site aggregates. In particular content from

major and minor publishers in the field of High-Energy Physics.

The project is currently in early stage of development.

See full documentation at http://pythonhosted.org/hepcrawl