{"id":16337811,"url":"https://github.com/gvolpe/fts","last_synced_at":"2025-03-22T23:32:37.311Z","repository":{"id":47676806,"uuid":"394400520","full_name":"gvolpe/fts","owner":"gvolpe","description":":mag: Postgres full-text search (fts)","archived":false,"fork":false,"pushed_at":"2022-01-06T10:01:19.000Z","size":22547,"stargazers_count":28,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-17T18:13:03.966Z","etag":null,"topics":["desktop-application","fs2","fs2-data","full-text-search","gui-application","haskell","nix","postgres","postgresql","scala","skunk"],"latest_commit_sha":null,"homepage":"","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gvolpe.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-09T18:34:58.000Z","updated_at":"2022-11-02T15:05:04.000Z","dependencies_parsed_at":"2022-09-23T15:50:35.909Z","dependency_job_id":null,"html_url":"https://github.com/gvolpe/fts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvolpe%2Ffts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvolpe%2Ffts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvolpe%2Ffts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gvolpe%2Ffts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gvolpe","download_url":"https://codeload.github.com/gvolpe/fts/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244250940,"owners_count":20423248,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["desktop-application","fs2","fs2-data","full-text-search","gui-application","haskell","nix","postgres","postgresql","scala","skunk"],"created_at":"2024-10-10T23:48:06.945Z","updated_at":"2025-03-22T23:32:33.391Z","avatar_url":"https://github.com/gvolpe.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"fts (full-text search)\n======================\n\n[![CI Status](https://github.com/gvolpe/fts/workflows/Haskell%20CI/badge.svg)](https://github.com/gvolpe/fts/actions)\n\nFull-text search demo powered by PostgreSQL, inspired by [this article](https://blog.crunchydata.com/blog/postgres-full-text-search-a-search-engine-in-a-database).\n\n## Overview\n\nThere are three components in this project: the dataset loader and both full-text search applications: a console app and a GUI app, described below.\n\nThe idea behind this demo project is to showcase the full-text search feature offered by PostgreSQL, as well as another example of a native multi-platform desktop GUI application written in Haskell.\n\nIt also showcases the power of functional effectful streams on the Scala side.\n\nOverall, it was just a fun pet project :nerd_face:\n\n### System requirements\n\nBefore anything else, we need a PostgreSQL instance up and running. You can use the supplied [docker-compose.yml](./docker-compose.yml) file, or have your own instance running.\n\nIf you care about reproducibility, then [Nix](https://nixos.org/) is your best friend. If not, you can try and build the project via `cabal new-build`, good luck with that :wink:\n\n```shell\n$ cachix use fts\n$ nix-build\n...\n$ ls result/bin/\nfts fts-ui\n```\n\nThe first command is optional but recommended, if you don't want to compile the world! Once the build is complete, you'll find two executables corresponding the two applications described below.\n\n### Full-text search console app\n\nIn the console app, you can search movies by title (hit `Ctrl + C` to exit).\n\n![console-app](assets/img/fts.png)\n\n**NOTE**: Your terminal needs to support the rendering of emojis and Unicode characters. Make sure you have a modern font installed in your system. In my case, I use [Alacritty](https://github.com/alacritty/alacritty) and [JetBrainsMono](https://www.jetbrains.com/lp/mono/).\n\n### Full-text search GUI app\n\nThe GUI application is richer in features, powered by the [monomer](https://hackage.haskell.org/package/monomer) package, and adapted from the [books example](https://github.com/fjvallarino/monomer/blob/main/docs/examples/02-books.md).\n\n![ui-app](assets/img/gui.jpg)\n\nIn addition to searching movies in Postgres, it tries to fetch the movie poster from [TMDB](https://www.themoviedb.org/), if the `TMDB_API_KEY` environment variable is set. To get one, have a look at the [instructions](https://developers.themoviedb.org/3/getting-started/introduction).\n\n**NOTE**: Non-NixOS users need to run `fts-ui` via [nixGL](https://github.com/guibou/nixGL), due to some complicated OpenGL linking issues. TL;DR:\n\n```shell\n$ nix-channel --add https://github.com/guibou/nixGL/archive/main.tar.gz nixgl\n$ nix-channel --update\n$ nix-env -iA nixgl.auto.nixGLDefault\n```\n\nThen run the program as `nixGL result/bin/fts-ui` instead.\n\n### Dataset loader\n\nThe loader is a nix shell script interpreted by [Ammonite](http://ammonite.io/), written in Scala, which reads a movies CSV file, parses its content, and it stores them in Postgres. These tasks are made easy by [fs2](https://fs2.io), [fs2-data](https://github.com/satabin/fs2-data), and [skunk](https://github.com/tpolecat/skunk).\n\n```shell\n$ cd data\n$ ./Loader.sc\n```\n\nIf you wish to change the Postgres connection details, have a look at the [configuration file](data/DB.sc).\n\n### Technical details\n\nEvery time a title is entered, a full-text search is performed against Postgres via the following query, which orders the results by the corresponding ranking (`ts_rank`).\n\n```sql\nSELECT title_id, title, genre, year, language, description, actors\nFROM movies\nWHERE ts @@ to_tsquery('english', ?)\nORDER BY ts_rank(ts, to_tsquery('english', ?)) DESC\n```\n\nThe console application only displays the title together with a link to the movie in [imdb](https://www.imdb.com/) but anything should be possible with a bit of customization, as done in the GUI application.\n\n## Dataset License\n\nThe movies dataset is licensed under [CC0 1.0](https://creativecommons.org/publicdomain/zero/1.0/), downloaded from [here](https://www.kaggle.com/stefanoleone992/imdb-extensive-dataset).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgvolpe%2Ffts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgvolpe%2Ffts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgvolpe%2Ffts/lists"}