{"id":28541366,"url":"https://github.com/arlovy/scrapingmeli","last_synced_at":"2026-04-30T06:39:58.476Z","repository":{"id":292244345,"uuid":"968347468","full_name":"arlovy/ScrapingMELI","owner":"arlovy","description":"Extraccion y carga de datos de inmuebles en venta de MercadoLibre dentro de una BD en PostgreSQL","archived":false,"fork":false,"pushed_at":"2025-05-29T00:48:41.000Z","size":37,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-30T06:39:51.355Z","etag":null,"topics":["automation","beautifulsoup","etl","etl-pipeline","multithreading","postgresql","python","scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/arlovy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-17T23:43:34.000Z","updated_at":"2025-05-29T00:48:44.000Z","dependencies_parsed_at":"2025-05-08T22:28:52.524Z","dependency_job_id":"a74b32ad-2dab-4f74-947c-4983e31b4a3e","html_url":"https://github.com/arlovy/ScrapingMELI","commit_stats":null,"previous_names":["arlovy/re-pricedata","arlovy/scrapingmeli"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/arlovy/ScrapingMELI","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arlovy%2FScrapingMELI","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arlovy%2FScrapingMELI/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arlovy%2FScrapingMELI/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arlovy%2FScrapingMELI/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/arlovy","download_url":"https://codeload.github.com/arlovy/ScrapingMELI/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/arlovy%2FScrapingMELI/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32457110,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-29T22:27:22.272Z","status":"online","status_checked_at":"2026-04-30T02:00:05.929Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","beautifulsoup","etl","etl-pipeline","multithreading","postgresql","python","scraping"],"created_at":"2025-06-09T20:08:14.097Z","updated_at":"2026-04-30T06:39:58.448Z","avatar_url":"https://github.com/arlovy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ScrapingMELI\nEste es un proyecto de webscraping  en Python, que extrae datos de inmuebles en venta de MercadoLibre. Si bien MercadoLibre tiene una API para realizar este tipo de consultas de manera más eficiente, estaba interesado en desarrollar este programa para experimentar con la librería BeautifulSoup y manejo de consultas a bases de datos usando Python. \n\n### Tecnologías\n- Python. \n    - Librería requests para la descarga del HTML. \n    - Librería BeautifulSoup4 para el parseo de los archivos.\n    - Librería psycopg3 para la conexión y ejecución de consultas a la base de datos.\n- PostgreSQL.\n\n### Funcionamiento\nEl programa hace consultas a la página de MercadoLibre, solo trayendo el contenido de 42 páginas debido al límite de navegación del sitio. Se le pueden pasar proxies al programa, para evitar bloqueos por exceso de solicitudes. Para hacer esto de forma más rápida, el programa hace uso de multihilado con ThreadPoolExecutor.\n\n## Modo de uso\n1. Se debe tener PostgreSQL instalado de forma local, y levantar el archivo ```db.sql```, a través del siguiente comando.\n\n```\npsql -U [USUARIO DE LA BASE DE DATOS] -d [NOMBRE DE LA BASE DE DATOS] -f db.sql\n```\n\n2. Instalar los requerimientos definidos en ```requirements.txt```.\n\n```\npip install requirements.txt\n```\n\n3. Una vez levantada la base de datos, con la tabla 'properties' dentro de ella, ejecutar el archivo ```main.py``` a través del siguiente comando.\n\n```\npython main.py [NOMBRE DE LA BASE DE DATOS] [USUARIO] [CONTRASEÑA] [(OPCIONAL) ruta del archivo de texto con proxies.]\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farlovy%2Fscrapingmeli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Farlovy%2Fscrapingmeli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Farlovy%2Fscrapingmeli/lists"}