{"id":13572425,"url":"https://github.com/TransparencyToolkit/LookingGlass","last_synced_at":"2025-04-04T10:30:30.937Z","repository":{"id":20000243,"uuid":"23267671","full_name":"TransparencyToolkit/LookingGlass","owner":"TransparencyToolkit","description":"Intuitive and configurable search interface for document archives.","archived":false,"fork":false,"pushed_at":"2019-11-11T23:33:56.000Z","size":412347,"stargazers_count":198,"open_issues_count":19,"forks_count":36,"subscribers_count":30,"default_branch":"master","last_synced_at":"2024-11-05T05:34:16.884Z","etag":null,"topics":["archive","document","interface","search"],"latest_commit_sha":null,"homepage":"https://transparencytoolkit.org","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TransparencyToolkit.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2014-08-23T22:29:31.000Z","updated_at":"2024-09-19T23:05:00.000Z","dependencies_parsed_at":"2022-08-07T09:15:48.656Z","dependency_job_id":null,"html_url":"https://github.com/TransparencyToolkit/LookingGlass","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TransparencyToolkit%2FLookingGlass","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TransparencyToolkit%2FLookingGlass/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TransparencyToolkit%2FLookingGlass/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TransparencyToolkit%2FLookingGlass/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TransparencyToolkit","download_url":"https://codeload.github.com/TransparencyToolkit/LookingGlass/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247160161,"owners_count":20893780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archive","document","interface","search"],"created_at":"2024-08-01T14:01:22.853Z","updated_at":"2025-04-04T10:30:30.078Z","avatar_url":"https://github.com/TransparencyToolkit.png","language":"Ruby","readme":"LookingGlass\n============\n\nSearch, filter, and browse any set of documents. LookingGlass includes full\ntext search, category filters, and date queries all through a nice search\ninterface with an **Elasticsearch** backend. LookingGlass also supports\ncustomizable\n[themes](https://github.com/TransparencyToolkit/LookingGlass/blob/master/THEMES.md)\nand flexible document view pages for browsing and embedding a variety of\ndocument types.\n\nLookingGlass requires\n[DocManager](https://github.com/TransparencyToolkit/DocManager) so that it can\ninteract with Elasticsearch. LookingGlass can be used in combination with\n[Harvester](https://github.com/TransparencyToolkit/Harvester) for crawling,\nparsing, and loading documents and automatically turning them into a\nsearchable archive. However, it also works well as a standalone archiving\ntool.\n\n\n# Installation\n\n## Dependencies\n\n* [DocManager](https://github.com/TransparencyToolkit/DocManager) and all of\n  its dependencies\n* ruby 2.4.1\n* rails 5\n* (optionally) [Harvester](https://github.com/TransparencyToolkit/Harvester)\n* libmagic-dev\n\n## Setup Instructions\n\n1. Install the dependencies\n\n* Download elasticsearch (https://www.elastic.co/downloads/elasticsearch)\n* Download rvm (https://rvm.io/rvm/install)\n* Install Ruby: Run `rvm install 2.4.1` and `rvm use 2.4.1`\n* Install Rails: `gem install rails`\n* Follow the installation instructions for [DocManager](https://github.com/TransparencyToolkit/DocManager)\n\n2. Get LookingGlass\n\n* Clone repo: `git clone --recursive git@github.com:TransparencyToolkit/LookingGlass.git`\n* Go into the LookingGlass directory: `cd LookingGlass`\n* Install the Rubygems LookingGlass uses: `bundle install`\n* Generate simple form data: `rails generate simple_form:install --bootstrap`\n* Precompile assets: `rake assets:precompile`\n\n3. Run LookingGlass\n\n* Start DocManager: Follow the instructions on the\n  [DocManager](https://github.com/TransparencyToolkit/DocManager) repo\n* Configure Project: Edit the file in `config/initializers/project_config` so\n  that the PROJECT_INDEX value is the name of the index in the\n  [DocManager](https://github.com/TransparencyToolkit/DocManager) project\n  config LookingGlass should use\n* Start LookingGlass: Run `rails server -p 3001`\n* Use LookingGlass: Go to [http://0.0.0.0:3001](http://0.0.0.0:3001) in your\n  browser\n \n\n# Features\n\nLookingGlass is a frontend for searchable document archives. Previously, it\nalso included the backend that interacted with Elasticsearch, but this has\nsince been split out into\n[DocManager](https://github.com/TransparencyToolkit/DocManager). The key\nfeatures are described below.\n\n## Display of Document Sets\n\nLookingGlass shows document sets from multiple data sources. It displays a\nlist of documents on the main page. The fields displayed for each document on\nthe index page and the order the documents are displayed in (sorted by date or\nanother numerical field) are customizable in\n[DocManager](https://github.com/TransparencyToolkit/DocManager)'s data source\nconfig files.\n\nEach individual document set is then displayed on its own page for easy\nreading. The document page includes a sidebar with the document's categorical\nfield and a customizable set of tabs that can display the document text, embed\nthe document itself (which is stored remotely, locally, or on document cloud),\noffer document downloads, or load links.\n\n## Search\n\nLookingGlass allows full text of document sets using the Elasticsearch\nbackend. It can be used to search documents in most languages. LookingGlass\nsupports searching all fields or individual fields, and a variety of non-text\nfields like dates. Results are sorted by relevance with text matching the\nquery highlighted.\n\n## Categorical Filters\n\nMany document sets have categorical fields that are common across documents,\neither in the original data or that can be extracted with a tool like\n[Catalyst](https://github.com/transparencytoolkit/catalyst). For example,\ncountries mentioned in a document, file format, hashtags, and topic-specific\nkeywords are common types of categories. LookingGlass allows filtering\ndocument sets by one or more categories by clicking links on the sidebar to\nget, say, all the documents that are about a particular country.\n\nThe category sidebar also displays the number of documents for each value in\neach category that matches the current query. This is great for getting an\noverview of the content in the document set.\n\n## Document View Templates\n\nOn both the search results/document index and individual document pages, the\nway the document is displayed is highly customizable. It is possible to add\nnew templates to display different types of data sources however you want and\neven thread together multiple documents when needed (in email datasets, for\nexample).\n\nThese view templates are defined in app/views/docs/show/tabs/panes (for the\ndocument view page) and app/views/docs/index/results/result_templates (for the\nindex/result view). The fields to use as a thread ID and view templates to\nused are specified per-source in the\n[DocManager](https://github.com/TransparencyToolkit/DocManager) data source\nconfig files.\n\n## Version Tracking\n\nLookingGlass can be used to track which documents change over time and\nhow. Documents that are changed are specified in categories on the sidebar and\nthe document view page has a tool that allows users to view the exact\ndifference between two documents over time.\n\nThe fields used to check if a document has changed are specified per-source in\nthe [DocManager](https://github.com/TransparencyToolkit/DocManager) data\nsource config files.\n\n## Custom Themes\n\nLookingGlass supports custom theming. The color scheme, fonts, logo, text, and\nlinks are all entirely customizable.\n\nSome of these settings, like the theme used, project title, and logo are defined in the\n[DocManager](https://github.com/TransparencyToolkit/DocManager) project config\nfile. The colors and fonts can then be set by [creating a\ntheme](https://github.com/TransparencyToolkit/LookingGlass/blob/master/THEMES.md).\n","funding_links":[],"categories":["Ruby"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTransparencyToolkit%2FLookingGlass","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTransparencyToolkit%2FLookingGlass","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTransparencyToolkit%2FLookingGlass/lists"}