{"id":24211353,"url":"https://github.com/tylershin/uri-luda","last_synced_at":"2025-09-06T11:43:02.104Z","repository":{"id":50205427,"uuid":"137973634","full_name":"TylerShin/uri-luda","owner":"TylerShin","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-08T02:15:00.000Z","size":5,"stargazers_count":1,"open_issues_count":2,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-03T16:21:30.143Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TylerShin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-06-20T03:09:00.000Z","updated_at":"2023-03-05T01:18:38.000Z","dependencies_parsed_at":"2023-01-24T05:10:15.282Z","dependency_job_id":null,"html_url":"https://github.com/TylerShin/uri-luda","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TylerShin/uri-luda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TylerShin%2Furi-luda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TylerShin%2Furi-luda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TylerShin%2Furi-luda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TylerShin%2Furi-luda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TylerShin","download_url":"https://codeload.github.com/TylerShin/uri-luda/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TylerShin%2Furi-luda/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":273898812,"owners_count":25187732,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-06T02:00:13.247Z","response_time":2576,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-14T02:34:57.124Z","updated_at":"2025-09-06T11:43:02.023Z","avatar_url":"https://github.com/TylerShin.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Luda Project\n\nCollect and archive Luda's photos.\n\n## User scenario\n\n- Crawl Luda's photo's from seed url page.\n- seed url page should be easily changed by an user.\n- Archive the photos to local drive or S3.\n- There is admin page that shows the photos and statistics about the crawling.\n\n## Design specs\n\n### Crawler\n\n- Crawler should parse and execute Javascript to read SPA websites.\n- There should be another crawler also parse only HTML because of the speed issue.\n- Crawler should handle a blocking logic of the target webpage. So, it shouldn't be super fast and has too many parallel instances.\n\n### Detector\n\n- Detector should find and grab images on the webpage.\n- Detector should know whether the photo's main character is Luda or not.\n\n## Archiver\n\n- Archiver should know whether there was already same photo in local(s3) drive or not.\n- To perform upper spec, Archiver should discriminate which photo is same photo. Not just by a file name.\n- If same photo exists, Archiver will save a better one. (normally bigger size)\n- If same photo doesn't exist, Archiver just save the photo.\n\n## Admin page\n\n- WIP\n\n## What we've done\n\n- Make README.md\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftylershin%2Furi-luda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftylershin%2Furi-luda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftylershin%2Furi-luda/lists"}