{"id":16780041,"url":"https://github.com/ekzhu/josie","last_synced_at":"2025-10-19T02:27:16.094Z","repository":{"id":52934192,"uuid":"114835965","full_name":"ekzhu/josie","owner":"ekzhu","description":"Code and Benchmarks for JOSIE (SIGMOD 2019)","archived":false,"fork":false,"pushed_at":"2023-04-13T15:57:22.000Z","size":51,"stargazers_count":18,"open_issues_count":3,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T21:15:16.289Z","etag":null,"topics":["josie","set-similarity-search","sigmod"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ekzhu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-20T02:56:51.000Z","updated_at":"2024-07-24T19:03:33.000Z","dependencies_parsed_at":"2024-06-19T01:49:04.028Z","dependency_job_id":"7c66ec1f-c0e4-4d4e-9684-e2316a0bc5f9","html_url":"https://github.com/ekzhu/josie","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhu%2Fjosie","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhu%2Fjosie/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhu%2Fjosie/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ekzhu%2Fjosie/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ekzhu","download_url":"https://codeload.github.com/ekzhu/josie/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248298312,"owners_count":21080320,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["josie","set-similarity-search","sigmod"],"created_at":"2024-10-13T07:33:48.640Z","updated_at":"2025-10-19T02:27:11.072Z","avatar_url":"https://github.com/ekzhu.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JOSIE: Overlap Set Similarity Search\n\nThis repository contains the code and benchmarks for the SIGMOD 2019 paper:\n[*JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data \nLakes*](https://dl.acm.org/doi/pdf/10.1145/3299869.3300065).\nFollow the steps here to run the experiments.\n\n## Requirements\n\nPostgreSQL and Go are required to run the experiments.\n\n### Postgres\n\n1. [Download](https://www.postgresql.org/ftp/source/v10.0/) and \n[install](https://www.postgresql.org/docs/10/static/install-procedure.html) \nfrom source. Make sure to use `--prefix=$HOME` to install to user home directory,\nand `--with-pgport=5442` to set the port for both server and client.\n2. Initialize a database directory: `initdb -D pg_data`.\n3. Use the configuration file in `conf/postgresql.conf` to start a server:\n`postgres -D pg_data -c config_file=conf/postgresql.conf`.\n4. Create a new database same as your Unix user name: `createdb \u003cdbname\u003e`.\n5. Test the client-server connection using `psql -p 5442`.\n\n### Go\n\n1. Download and install the [Go programming language](https://golang.org/dl/)\n2. Create a directory under your home directory `mkdir ~/go`, this will be your\n   go path\n3. Make sure you have [set up `$GOPATH`](https://golang.org/doc/install) \nin your bash environment by adding the following lines to your bash profile, then restart your bash session\n\n```\nexport GOPATH=$HOME/go\nexport GOBIN=$GOPATH/bin\nexport PATH=$GOBIN:$PATH\n```\n\n4. **Important:** check out this repository under your go path: \n```\nmkdir -p ~/go/src/github.com/ekzhu/josie\ngit clone git@github.com:ekzhu/josie.git ~/go/src/github.com/ekzhu/josie\n```\n\n## Run the benchmarks in the original paper\n\nNow go into the project directory at `~/go/src/github.com/ekzhu/josie`.\n\n### Prepare benchmarks\n\nFirst download the benchmarks in the form of Postgres dumps.\n\n* [Canada-US-UK Open Data](https://storage.googleapis.com/josie-benchmark/canada_us_uk_benchmark.sql.gz)\n* [WDC Web Table 2015, English Relational Only](https://storage.googleapis.com/josie-benchmark/webtable_benchmark.sql.gz)\n\nUncompress the dump files (use `gzip -d`) \nand run the SQL files (or use `pg_restore`)\nto load the benchmarks into Postgres. \nMake sure to use the port setting you used when installing \nPostgres earlier, so the dump files get imported into the \nright database.\n\nThen, run the SQL script `create_indexes.sql` to create indexes for the \nsets and posting lists tables.\n\n### Run experiments\n\nWe use the targets defined in `Makefile` to run experiments.\nFirst you need to generate a cost sample table to compute the read\ncost of sets and posting lists.\n\n```\nmake sample_cost_canada_us_uk\nmake sample_cost_webtable\n```\n\nTo run experiments using the Open Data benchmark:\n\n```\nmake canada_us_uk\n```\n\nWeb Table benchmark:\n\n```\nmake webtable\n```\n\n**Notice:** the experiments can take many hours or even days depending on your\nhardware environment (SSD will be much faster than HDD). \nTo fine tune which experiments to run, you can modify\n`exp.go`.\n\n### Plot results\n\nResults are located in the `results` directory. Use the targets defined\nin the `Makefile` to plot results:\n\n```\nmake plot\n```\n\nThe output plots are located in the `plots` directory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhu%2Fjosie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fekzhu%2Fjosie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fekzhu%2Fjosie/lists"}