{"id":15444673,"url":"https://github.com/andreoliwa/scrapy-tegenaria","last_synced_at":"2025-04-13T05:40:30.589Z","repository":{"id":35920218,"uuid":"40207952","full_name":"andreoliwa/scrapy-tegenaria","owner":"andreoliwa","description":"🕷🕸 Spiders to crawl ads of houses and apartments. 🏠 🏢","archived":false,"fork":false,"pushed_at":"2025-03-17T19:19:20.000Z","size":1465,"stargazers_count":2,"open_issues_count":45,"forks_count":5,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-03-26T22:21:34.542Z","etag":null,"topics":["crawler","flask","postgresql","python","python3","scrapy"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/andreoliwa.png","metadata":{"files":{"readme":"README.rst","changelog":"HISTORY.rst","contributing":"CONTRIBUTING.rst","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS.rst","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-08-04T20:40:20.000Z","updated_at":"2020-07-27T21:45:23.000Z","dependencies_parsed_at":"2024-02-20T00:28:24.118Z","dependency_job_id":"642e415c-5b96-4723-9665-44c3f47049bd","html_url":"https://github.com/andreoliwa/scrapy-tegenaria","commit_stats":{"total_commits":146,"total_committers":3,"mean_commits":"48.666666666666664","dds":0.5821917808219178,"last_synced_commit":"8f16f6e96b0f8ad8f1ac5d964934b71333565416"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreoliwa%2Fscrapy-tegenaria","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreoliwa%2Fscrapy-tegenaria/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreoliwa%2Fscrapy-tegenaria/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/andreoliwa%2Fscrapy-tegenaria/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/andreoliwa","download_url":"https://codeload.github.com/andreoliwa/scrapy-tegenaria/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248670514,"owners_count":21142898,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["crawler","flask","postgresql","python","python3","scrapy"],"created_at":"2024-10-01T19:42:15.082Z","updated_at":"2025-04-13T05:40:29.968Z","avatar_url":"https://github.com/andreoliwa.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"=========\nTegenaria\n=========\n\n.. image:: https://img.shields.io/travis/andreoliwa/scrapy-tegenaria.svg\n        :target: https://travis-ci.org/andreoliwa/scrapy-tegenaria\n\nScrapy spiders to collect and browse different items (so far, apartment ads).\n\n* Free software: BSD license\n* Documentation: https://tegenaria.readthedocs.org.\n\nHow to setup the dev environment\n--------------------------------\n\nTODO\n\nMetrics\n-------\n\n.. image:: https://badge.waffle.io/andreoliwa/scrapy-tegenaria.svg?label=ready\u0026title=Ready\n   :alt: Stories in Ready\n   :target: http://waffle.io/andreoliwa/scrapy-tegenaria\n\n.. image:: https://graphs.waffle.io/andreoliwa/scrapy-tegenaria/throughput.svg\n   :alt: Throughput Graph\n   :target: https://waffle.io/andreoliwa/scrapy-tegenaria/metrics\n\nQuickstart\n----------\n\nFirst, set your app's secret key as an environment variable. For example, example add the following to ``.bashrc`` or ``.bash_profile``.\n\n.. code-block:: bash\n\n    export TEGENARIA_SECRET='something-really-secret'\n\n\nThen run the following commands to bootstrap your environment.\n\n\n::\n\n    git clone https://github.com/andreoliwa/scrapy-tegenaria\n    cd tegenaria\n    pip install -r requirements/dev.txt\n    flask run\n\nYou will see a pretty welcome screen.\n\nOnce you have installed your DBMS, run the following to create your app's database tables and perform the initial migration:\n\n::\n\n    flask db init\n    flask db migrate\n    flask db upgrade\n    flask run\n\n\n\nDeployment\n----------\n\nIn your production environment, make sure the ``TEGENARIA_ENV`` environment variable is set to ``\"prod\"``.\n\n\nShell\n-----\n\nTo open the interactive shell, run ::\n\n    flask shell\n\nBy default, you will have access to ``app``, ``db``, and the ``User`` model.\n\n\nRunning Tests\n-------------\n\nTo run all tests, run ::\n\n    flask test\n\n\nMigrations\n----------\n\nWhenever a database migration needs to be made. Run the following commands:\n::\n\n    flask db migrate\n\nThis will generate a new migration script. Then run:\n::\n\n    flask db upgrade\n\nTo apply the migration.\n\nFor a full migration command reference, run ``flask db --help``.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreoliwa%2Fscrapy-tegenaria","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandreoliwa%2Fscrapy-tegenaria","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandreoliwa%2Fscrapy-tegenaria/lists"}