{"id":20052510,"url":"https://github.com/subins2000/wikipeer","last_synced_at":"2025-05-05T11:32:29.288Z","repository":{"id":43915551,"uuid":"284490772","full_name":"subins2000/wikipeer","owner":"subins2000","description":"Wikipedia over WebRTC \u0026 WebTorrent. Decentralized P2P proxy to access Wikipedia circumventing internet censorship.","archived":false,"fork":false,"pushed_at":"2022-02-13T19:12:06.000Z","size":15766,"stargazers_count":11,"open_issues_count":18,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-08T22:41:36.115Z","etag":null,"topics":["p2p","p2pt","webrtc","webtorrent","wikipedia","wikipedia-api"],"latest_commit_sha":null,"homepage":"https://wikipeer.subinsb.com/","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/subins2000.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-08-02T15:40:41.000Z","updated_at":"2024-08-11T19:45:43.000Z","dependencies_parsed_at":"2022-09-23T11:05:15.757Z","dependency_job_id":null,"html_url":"https://github.com/subins2000/wikipeer","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subins2000%2Fwikipeer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subins2000%2Fwikipeer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subins2000%2Fwikipeer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subins2000%2Fwikipeer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/subins2000","download_url":"https://codeload.github.com/subins2000/wikipeer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252489205,"owners_count":21756285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["p2p","p2pt","webrtc","webtorrent","wikipedia","wikipedia-api"],"created_at":"2024-11-13T12:17:59.457Z","updated_at":"2025-05-05T11:32:28.846Z","avatar_url":"https://github.com/subins2000.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Wikipeer\n\nDecentralized P2P proxy in browser to access Wikipedia. Works entirely on [WebTorrent](https://webtorrent.io/) ecosystem with [P2PT](https://github.com/subins2000/p2pt). No installation needed, direct use from browser!\n\nWikipeer is available on the following domains, all static sites :\n\n* [wikipeer.subinsb.com](https://wikipeer.subinsb.com)\n* [wikipeer.herokuapp.com](https://wikipeer.herokuapp.com/)\n* [dazzling-nightingale-9995ef.netlify.app](https://dazzling-nightingale-9995ef.netlify.app/)\n\nThis project can be also called (aka) :\n\n* Access Wikipedia over WebRTC \u0026 Torrents\n* Website proxy over WebRTC\n\n## Architecture\n\nTerms used :\n\n* Client -\u003e A user that can't access Wikipedia by HTTP because it's blocked.\n* Proxy -\u003e A user that can access Wikipedia by HTTP.\n* Compile (verb) -\u003e Collect the text, images, media of a Wikipedia page and make a torrent.\n* Seed (verb) -\u003e Same as the seeding in bittorrent.\n* Info Hash -\u003e Info Hash of torrent made by compiling a page.\n* Feed -\u003e Wikipedia homepage\n* Article -\u003e A Wikipedia article\n\nWikipeer uses [Wikivue](https://github.com/santhoshtr/wikivue) for the user interface. Wikipeer will continue to pull changes from Wikivue. Because of this, only the really necessary changes are made in the source code to avoid future merge conflicts. If you'd like to make improvements to the user interface, please send a patch to [Wikivue](https://github.com/santhoshtr/wikivue).\n\nThe Wikipeer implementation to Wikivue is mostly made by these files (ordered according to importance HIGH to LOW). Take a peek at them for understanding this document better.\n\n* src/wiki/P2WikiClass.js\n* src/wiki/api/*\n* `src/components/ArticleContent.vue` \u0026 `src/components/ArticleSectionContent.vue` - These should hopefully be merged into Wikivue\n* src/views/Home.vue\n\n### Development\n\nWikipeer has two components :\n\n* Browser: Any user opening  Wikipeer via browser can either be a client or proxy, but not both at the same time\n* Node: Dedicated proxy service that runs in the terminal\n\n`npm serve` - Runs Vue development server (browser part).\n\n`npm proxy` - Runs the dedicated proxy.\n\n`npm start` - Runs both proxy \u0026 Vue dev server.\n\nBoth browser \u0026 node can communicate with each other through WebRTC. The Node component is basically [WebTorrent-hybrid](https://github.com/webtorrent/webtorrent-hybrid).\n\n### P2PT\n\nRead about [P2PT here](https://github.com/subins2000/p2pt).\n\nThe app identifier is \"p2wiki\". Since both clients and proxies will be in the same swarm, a proxy is identified in the intial message. Client will send `c` and proxies will send `p`. All other communication between client \u0026 proxy is in JSON format.\n\nTODO: Maintain a balanced list of clients and proxies.\n\n#### Consensus\n\nSince we can't really trust a proxy, a consensus need to be reached to make a trust. A response from a proxy is trusted when different proxies return the same response. This is done by equating checksum of response of each proxy.\n\n* Consensus value (`const PROXY_TRUST_CONSENSUS_COUNT`) -\u003e How many proxies should return the same response for it to be trusted and start downloading ?\n\nSince Wikipeer is at its beginning, the consensus value is set to **1** that is all proxies are trusted and considered honest. This should be increased as Wikipeer grows with more proxies.\n\n### Content\n\n#### Feed\n\nThe Wikipedia feed (or homepage) is identified by the language \u0026 the present date. Every proxy will by default seed the feed. Once a client has the feed, they will also seed it (See [#torrent](#torrent))\n\nWhen Wikipeer is visited, the client will request proxies for the \"feed\" :\n\n```javascript\n{\n  get: \"feed\",\n  lang: \"en\" // English\n}\n```\n\nThe proxies will respond back the info hash :\n\n```javascript\n{\n  hash: \"dd8255ecdc7ca55fb0bbf81323d87062db1f6d1c\"\n}\n```\n\nIf all the proxies return the same info hash (see [#consensus](#consensus)), the torrent of feed is downloaded and displayed. The clients will store this homepage torrent and start seeding it.\n\n#### Article\n\nA Wikipedia article is identified by language and title. Title may vary according to language.\n\nLike the [#feed](#feed), proxies are requested for an article :\n\n```javascript\n{\n  get: \"article\",\n  lang: \"ml\" // Malayalam\n  title: \"കേരളം\"\n}\n```\n\nResponse from proxy :\n\n```javascript\n{\n  hash: \"209c8226b299b308beaf2b9cd3fb49212dbd13ec\"\n}\n```\n\nThis info hash reflects the latest revision of the article at the time of torrent creation.\n\n* Torrent remove timeout -\u003e A torrent is kept seeded by proxy for a particular `timePeriod`. If an article torrent is inactive (no downloads) for `timePeriod`, it's destroyed. This ensures proxies are not keeping less visited articles forever and save resources. This `timePeriod` is stored as minutes in `TORRENT_REMOVE_TIMEOUT`.\n\n* Problem: If a frequently visited article is kept seeding, proxies may not give the latest revision, because proxies need to wait for `timePeriod` to complete for the torrent to get destroyed. Only after this will the latest revision be fetched, and a new torrent of the latest revision. This also cause honest (trusted) proxies to return different info hashes. \u003cbr/\u003e\n  Solution: The old revision torrent will be destroyed even though it might take time. This problem is not that big of a concern. Still it's a problem\n\n#### Search\n\nResult of search queries are not communicated by torrent, but instead directly sent by proxies. This is to speed up auto suggestion on entering search query.\n\n```javascript\n{\n  get: \"search\",\n  q: \"query\"\n}\n```\n\nThe proxies would respond with :\n\n```javascript\n{\n  pages: {}, // results\n  hash: \"168fc8ca6cdb8c98502428dd0f9e5113\" // MD5 hash of the previous pages object\n}\n```\n\nThe [#consensus](#consensus) rule applies here too using the `hash` value.\n\n### Torrent\n\nAny torrent client that supports WebTorrent (Seeding to web peers over WebRTC) can help in sharing articles to Wikipeer clients. The torrent needs to be made in a specific way so that the info hash made by different seeders/proxies will be the same.\n\nWhen a client downloads a torrent, they will also seed it increasing the availability of that feed/article. There is no time limit like in proxy for destroying the torrent.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubins2000%2Fwikipeer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsubins2000%2Fwikipeer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubins2000%2Fwikipeer/lists"}