{"id":18645494,"url":"https://github.com/cscott/nell-wikipedia","last_synced_at":"2025-11-05T03:30:31.982Z","repository":{"id":6723933,"uuid":"7969768","full_name":"cscott/nell-wikipedia","owner":"cscott","description":"Nell's Wikipedia: an offline wikipedia browser for learning literacy","archived":false,"fork":false,"pushed_at":"2016-06-22T08:23:51.000Z","size":10491,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-12-27T11:42:28.019Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cscott.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-02-02T01:08:11.000Z","updated_at":"2016-02-24T15:19:11.000Z","dependencies_parsed_at":"2022-09-12T00:40:34.285Z","dependency_job_id":null,"html_url":"https://github.com/cscott/nell-wikipedia","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cscott%2Fnell-wikipedia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cscott%2Fnell-wikipedia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cscott%2Fnell-wikipedia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cscott%2Fnell-wikipedia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cscott","download_url":"https://codeload.github.com/cscott/nell-wikipedia/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239449567,"owners_count":19640533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T06:16:11.742Z","updated_at":"2025-02-18T09:46:34.698Z","avatar_url":"https://github.com/cscott.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Nell's Wikipedia\n\nThis is an offline browser for Wikipedia.  Unlike other offline readers,\nit doesn't require a preprocessing step: it reads article text (in\n[parsoid](http://www.mediawiki.org/wiki/Parsoid/About) format)\ndirectly from Wikipedia. (At the moment it uses an external parsoid\nserver to translate from mediawiki to parsoid format on the fly,\nhowever Wikipedia's\n[long-term plan](http://www.mediawiki.org/wiki/Future/Parser_plan)\nis to support parsoid format directly from its\n[API](https://www.mediawiki.org/wiki/API:Main_page).)\nWe then store the data in a local\n[IndexedDB](http://en.wikipedia.org/wiki/Indexed_Database_API) for future\noffline use.  Any page you visit while online can then be accessed\noffline as well.\n\nYou can also seed the app with a collection of useful pages, or pages\nlinked from pages you've visited before, to make it more likely that\nthe information you want will be ready when you need it.\n\nNell's Wikipedia also downloads images used on pages, although it shrinks\nand compresses them heavily to save space.\n\nWe hope to also support editing pages off-line, peer-to-peer sharing page\nchanges with friends, and uploading changes back to Wikipedia the next\ntime the user is online.  We hope to use the\n[Visual Editor](http://www.mediawiki.org/wiki/VisualEditor) codebase\nfor editing and page diffs.\n\n## Under the hood\n\nPages are [LZMA](http://en.wikipedia.org/wiki/LZMA)-compressed in\nmoderately-sized batches, since compression\nworks better when pages are compressed together.\n\u003c!--\n  Each batch is stored in\na [B-tree](http://en.wikipedia.org/wiki/B-tree) in order to support dynamic\ninsertion/deletion of pages into the compressed store.\n--\u003e\n\nThe current implementation of the parsoid format results in output a\nfactor of 10 larger than mediawiki markup.  After LZMA compression the\nparsoid is only 3x larger.  We hope that further improvements will be\nmade upstream.\n\nImages are resized using an HTML5 canvas before being compressed and stored\nlike pages. (XXX: jpeg compression should actually be done w/ canvas; check\ncurrent browser support for this).\n\n## Related libraries\n\nNell's Wikipedia was inspired by Kevin Kwow (antimatter15)'s\n[offline wiki](https://code.google.com/p/offline-wiki/) and\nthe Sugar [Wikipedia](http://wiki.laptop.org/go/WikiBrowse) activity.\nIt uses the following libraries:\n* [lzma-purejs](http://github.com/cscott/lzma-purejs) by Gary Linscott\n* [parsoid](http://www.mediawiki.org/wiki/Parsoid) to convert mediawiki markup\n\nPrevious versions of the code used the following libraries:\n* [Instaview](http://github.com/cscott/instaview) to render mediawiki markup\n* [seek-bzip](http://github.com/cscott/seek-bzip) to do random seeks inside a bzip-compressed file\n* [html5slider](http://frankyan.com/labs/html5slider/) by Frank Yan\n* [HTML5 Progress Polyfill](http://github.com/LeaVerou/HTML5-Progress-polyfill) by Lea Verou\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcscott%2Fnell-wikipedia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcscott%2Fnell-wikipedia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcscott%2Fnell-wikipedia/lists"}