{"id":25177917,"url":"https://github.com/axelgard/seek-ipns","last_synced_at":"2026-04-12T04:32:50.799Z","repository":{"id":154621294,"uuid":"631852488","full_name":"AxelGard/seek-ipns","owner":"AxelGard","description":"a search engine for IPNS","archived":false,"fork":false,"pushed_at":"2023-11-12T07:20:29.000Z","size":56481,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-04T03:24:33.400Z","etag":null,"topics":["golang","ipfs","ipns","ipns-ipfs","python","scikit-learn","search","search-engine"],"latest_commit_sha":null,"homepage":"https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-198457","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AxelGard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-24T07:43:38.000Z","updated_at":"2023-11-21T18:15:48.000Z","dependencies_parsed_at":"2023-10-15T08:05:23.742Z","dependency_job_id":null,"html_url":"https://github.com/AxelGard/seek-ipns","commit_stats":null,"previous_names":["axelgard/seek-ipns"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/AxelGard/seek-ipns","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxelGard%2Fseek-ipns","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxelGard%2Fseek-ipns/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxelGard%2Fseek-ipns/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxelGard%2Fseek-ipns/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AxelGard","download_url":"https://codeload.github.com/AxelGard/seek-ipns/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AxelGard%2Fseek-ipns/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31704492,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-11T21:17:31.016Z","status":"online","status_checked_at":"2026-04-12T02:00:06.763Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","ipfs","ipns","ipns-ipfs","python","scikit-learn","search","search-engine"],"created_at":"2025-02-09T14:49:56.383Z","updated_at":"2026-04-12T04:32:50.768Z","avatar_url":"https://github.com/AxelGard.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# seekIPNS \nA search engine for IPNS records. \n\n## Disclaimer\nThis is a part of my master thesis, so PR will not be accepted since this is for the thesis. \nHowever feel free to fork and use as you wish. \n\n## The thesis\n\nThe full thesis can be found **[HERE](https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-198457)**.\n\nMy presentation for the thesis can be found **[HERE](./doc/TQDT33%20-%20Final%20presentation.pdf)**\n\n\n### Search\n\nA part of the thesis was building a search engine, which can also be found in this repo, [here](./server/).\nThe search model used [TF-IDF](https://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html).\n\nI made a super simple frontend for it as well which can be found [here](./frontend/).\n\n![](./doc/Screenshot_seekIPNS_result.png)\n\n### Some results\n\nFor each IPNS record found \n* Peer ID\n* CID\n* All file names\n* All file path\n* All file sizes\n* All file formats\n* Number of files\n* Number of peers hosting the CID\n* Time when peer was found\n\nwas eveluated \n\n\n\nThe records some of the records found on the network was saved and used for search. \n\n35K unique peers were crawled, and of those, 333 were hosting a nonempty record\nin their default IPNS record.\n\n![peer crawled](./charts/peers_time.png)\n\n83099 files was found in the crawling.\n![number of files](./charts/files_per_peer_log.png)\n\nThe decentralization of the content was also looked at: \n\n![decentralization](./charts/peers_hosting_cids.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxelgard%2Fseek-ipns","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faxelgard%2Fseek-ipns","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faxelgard%2Fseek-ipns/lists"}