{"id":19681280,"url":"https://github.com/marfullsen/books-scraper","last_synced_at":"2025-07-31T06:38:40.729Z","repository":{"id":139327047,"uuid":"421919027","full_name":"Marfullsen/books-scraper","owner":"Marfullsen","description":"Scraping al sitio books.ToScrape.com y generar CSV con los datos recopilados.","archived":false,"fork":false,"pushed_at":"2021-10-27T17:53:52.000Z","size":706,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-20T15:56:32.589Z","etag":null,"topics":["python","python3","requests","scraping","scrapping","webscraper","webscraping","webscrapping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Marfullsen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-27T17:48:57.000Z","updated_at":"2021-10-29T05:14:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"15240f99-f7af-4e42-a63a-83cd3add24a4","html_url":"https://github.com/Marfullsen/books-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Marfullsen/books-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marfullsen%2Fbooks-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marfullsen%2Fbooks-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marfullsen%2Fbooks-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marfullsen%2Fbooks-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Marfullsen","download_url":"https://codeload.github.com/Marfullsen/books-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Marfullsen%2Fbooks-scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267999933,"owners_count":24178882,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-31T02:00:08.723Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","python3","requests","scraping","scrapping","webscraper","webscraping","webscrapping"],"created_at":"2024-11-11T18:07:25.151Z","updated_at":"2025-07-31T06:38:40.700Z","avatar_url":"https://github.com/Marfullsen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# BooKs ScRaPeR\n[![python3](https://img.shields.io/badge/Python-3.9.4-blue.svg)](https://www.python.org/)\n[![beautifulsoup](https://img.shields.io/badge/Beautifulsoup-4.10-orange.svg)](https://www.crummy.com/software/BeautifulSoup/)\n[![requests](https://img.shields.io/badge/Requests-2.26-yellow.svg)](https://docs.python-requests.org/)\n\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./docs/img/scraping-logo.png\" width=\"400\" alt=\"Scraping-logo\"\u003e\n\u003c/p\u003e\n\n## Contenidos\n\n- [¿Qué es Books Scraper?](#about)\n- [Echa un vistazo](#getting_started)\n- [Uso](#usage)\n- [Referencias](#refs)\n\n## ¿Qué es Books Scraper? \u003ca name = \"about\"\u003e\u003c/a\u003e\n\nPrueba de postulación a Tech-K, el proyecto consiste en hacer _scraping_ al sitio [books.ToScrape.com](https://books.toscrape.com/), obtener la información de mil libros y entregar un archivo CSV con los datos recopilados.\n\n## Archivo original\n\nEl archivo de instrucciones original puede encontrarse [haciendo click aquí](./INSTRUCTIONS.md)\n\n## Echa un vistazo \u003ca name = \"getting_started\"\u003e\u003c/a\u003e\n\nLo que hace el script es pedir la página y analizar el catálogo, recorriendo hoja a hoja, libro a libro.\n\n[![catalogo](https://github.com/Marfullsen/webscraper-challenge/blob/master/docs/img/screenshot-website-books-toscrape.png \"catalogo\")]()\n\nHay que tener en cuenta que el catálogo tiene enlaces que llevan a una página especial para cada detalle del libro seleccionado.\n\n[![catalogo](./docs/img/book-1.png \"Libro 1\")]()\n\nEl script __main.py__ iniciará el proceso de recolección de datos, esto generará un archivo __CSV__,\\\nsi lo deseas puedes ver el ejemplar obtenido con los datos recopilados: [thousend_scraped_books.csv](https://github.com/Marfullsen/webscraper-challenge/blob/master/docs/csv/thousend_scraped_books.csv)\n\n[![csv_generado](https://github.com/Marfullsen/webscraper-challenge/blob/master/docs/img/thousend_books.png \"CSV_generado\")]()\n\nPara ejecutar el script solo necesitas seguir los pasos que se explican a continuación:\n\n### Requisitos Previos\n\n- Python3\n- Dependencias externas para Python3\n\nEl archivo de requisitos puede ser usado con el comando `python -m pip install -r requirements.txt`\n\nDetalladamente, las bibliotecas a instalar son:\n\n```\npip install requests\npip install beautifulsoup4\npip install lxml\npip install pytest\n```\n\n## Uso \u003ca name = \"usage\"\u003e\u003c/a\u003e\n\nAl tener una conexión a internet y corroborar que el sitio se encuentre estable, bastará con ejecutar el archivo `main.py`\n\nCada página cuenta con 20 libros, revisar cada uno de esos libros demora alrededor de un segundo **por libro**.\n\n```\nPágina 1                    \nPágina 2                    \nPágina 3                    \n\n...                  \n\nPágina 50                   \nSe procesarion 1000 libros. \n```\n\n**Nota:** Una vez iniciado el proceso se irá rellenando el archivo CSV, en caso de problemas con la ejecución, el archivo quedará con la cantidad de datos que haya logrado procesar en aquel momento.\n\n### Pruebas unitarias\n\nLas pruebas unitarias se pueden realizar con el comando `pytest -xv tests.py`\n\nLas pruebas a realizarse son:\n- test_exists, comprueba que el archivo principal exista.\n- test_create_csv_file, corrobora que se haya creado un archivo CSV **en blanco** (sólo con las cabeceras).\n- test_write_to_csv, corrobora que se escriban __datos challa__.\n- test_get_page_1_returns_200, verifica que un sitio existente devuelva el código http que indique que todo salió bien.\n- test_get_page_bookname_returns_200, verifica que al entrar al enlance del libro del catálogo, este existe.\n- test_get_page_0_returns_404, verifica que la página no existe o no se encuentra disponible.\n\n## Referencias \u003ca name = \"refs\"\u003e\u003c/a\u003e\n\n- Documentación de BS - [https://www.crummy.com/software/BeautifulSoup/bs4/doc/](https://www.crummy.com/software/BeautifulSoup/bs4/doc/)\n- Documentación de CSV - [https://docs.python.org/3/library/csv.html](https://docs.python.org/3/library/csv.html)\n- Primer libro del siito - [https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html](https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html)\n- Reading and Writing CSV Files in Python - [https://realpython.com/python-csv/](https://realpython.com/python-csv/)\n- Python docs, CSV Library - [https://docs.python.org/3/library/csv.html](https://docs.python.org/3/library/csv.html)\n- Enumerate Python [https://www.geeksforgeeks.org/enumerate-in-python/](https://www.geeksforgeeks.org/enumerate-in-python/)\n- Fatal bad revision GIT - [stackoverflow.com/what-does-fatal-bad-revision-mean](https://stackoverflow.com/questions/14550802/what-does-fatal-bad-revision-mean)\n- Snippet encoding - [https://www.codegrepper.com/code-examples/python/csv+writerow+without+newline](https://www.codegrepper.com/code-examples/python/csv+writerow+without+newline)\n- Decoding - [stackoverflow.com/how-to-decode-scrambled-character-encoding-special-character-encoding](https://stackoverflow.com/questions/8706107/how-to-decode-scrambled-character-encoding-special-character-encoding)\n- Pounds Charcode problem - [https://stackoverflow.com/questions/55737316/python-selenium-text-returns-%C3%A2%E2%82%AC-instead-of-apostrophe](https://stackoverflow.com/questions/55737316/python-selenium-text-returns-%C3%A2%E2%82%AC-instead-of-apostrophe)\n- Encoding - [https://stackoverflow.com/questions/27092833/unicodeencodeerror-charmap-codec-cant-encode-characters](https://stackoverflow.com/questions/27092833/unicodeencodeerror-charmap-codec-cant-encode-characters)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarfullsen%2Fbooks-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmarfullsen%2Fbooks-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmarfullsen%2Fbooks-scraper/lists"}