{"id":19653601,"url":"https://github.com/sematext/related-searches","last_synced_at":"2025-08-08T08:39:41.188Z","repository":{"id":140122760,"uuid":"65059695","full_name":"sematext/related-searches","owner":"sematext","description":"Related Searches - get queries related or similar to a given query ","archived":false,"fork":false,"pushed_at":"2016-08-23T09:29:29.000Z","size":49,"stargazers_count":8,"open_issues_count":0,"forks_count":3,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-04-28T17:45:32.819Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sematext.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-08-06T01:47:02.000Z","updated_at":"2022-01-10T19:20:29.000Z","dependencies_parsed_at":"2024-03-31T00:15:53.314Z","dependency_job_id":null,"html_url":"https://github.com/sematext/related-searches","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sematext/related-searches","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sematext%2Frelated-searches","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sematext%2Frelated-searches/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sematext%2Frelated-searches/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sematext%2Frelated-searches/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sematext","download_url":"https://codeload.github.com/sematext/related-searches/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sematext%2Frelated-searches/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266241294,"owners_count":23898066,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T15:14:38.498Z","updated_at":"2025-07-21T04:36:30.899Z","avatar_url":"https://github.com/sematext.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Related Searches\n\nGiven a user query Related Searches returns a list of, you guessed it - related searches, much like related searches you see on Google search results pages.\n\nThe related searches are derived from query logs structured like the infamous AOL search data leak - [AOL search data leak](https://en.wikipedia.org/wiki/). See sample further below.\n\nRelatedness is based on query similarity, as well as click data stored in the query logs.\n\n## Getting Started\n\nTo run Related Searches you will need to clone the repository and \n\n### Requirements\n\nYou need to have a recent version of Java installed, Apache Maven to build the jar files and [Redis](http://redis.io) to help with data processing.  \n\n#### Redis role ####\n\nRedis is used during calculation as a fast key-value store. \n\n#### Elasticsearch role ####\n\nElasticsearch is one of data stores related searches can be written to.\n\n### Building from Source\n\nRelated Searches uses [Maven](https://maven.apache.org) for its build system. You'll need to have a modern version of Maven installed, any version from the 3.x branch should work.\n\nTo create a distribution, simply run `maven clean install` command in the cloned directory. The runnable jar files will be created in the @target@ directory and will be called: `BasicRelatedSearches.jar` and `TimeAndClickRelatedSearches.jar`.\n\n### Running \n\nTo run Related Searches you need to have Redis running. Once you build Related Searches you have two options:\n\n#### Basic Related Searches\n\nThe most basic related searches calculation, which doesn't take click count into consideration. To run it and generate file output one has to run the following command:\n\n`java -jar BasicRelatedSearches.jar REDIS_HOST REDIS_PORT TIME_BETWEEN_QUERIES_IN_SECONDS SUGGESTION_THRESHOLD QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD QUERY_SIMILARITY_MORE_SIMILAR QUERIES_FILE OUTPUT_FILE $INDEX_NAME \u003e /dev/null 2\u003e\u00261`\n\nTo run basic related searches calculation and get the result in Elasticsearch index, one should run the following command:\n\n`java -jar -DuseElasticSearch=true BasicRelatedSearches.jar REDIS_HOST REDIS_PORT TIME_BETWEEN_QUERIES_IN_SECONDS SUGGESTION_THRESHOLD QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD QUERY_SIMILARITY_MORE_SIMILAR QUERIES_FILE ES_HOST INDEX_NAME \u003e /dev/null 2\u003e\u00261`\n\nAnd the parameters are:\n- REDIS_HOST - Redis host, e.g., _localhost_\n- REDIS_PORT - Redis port, e.g., _6379_\n- TIME_BETWEEN_QUERIES_IN_SECONDS - maximum time between queries in seconds, e.g., _20_\n- SUGGESTION_THRESHOLD - suggestion threshold, e.g., _0.2_\n- QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD - query similarity threshold, e.g., _0.9_\n- QUERY_SIMILARITY_MORE_SIMILAR - boolean, set to _true_ will result in suggesting similar queries according to the given _QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD_\n- QUERIES_FILE - queries log file\n- OUTPUT_FILE - output file with results\n- ES_HOST - Elasticsearch address\n- INDEX_NAME - Elasticsearch index name\n\n#### Time and Click-based Related Searches\n\nTo run related searches calculation that takes into consideration time and click information from the query log and output the results to file, one has to run the following command:\n\n`java -jar TimeAndClickRelatedSearches.jar REDIS_HOST REDIS_PORT TIME_BETWEEN_QUERIES_IN_SECONDS SUGGESTION_THRESHOLD QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD QUERY_SIMILARITY_MORE_SIMILAR TEXT_SIMILARITY_BOOST TIME_AND_CLICK_BOOST SUGGEST_NON_ZERO_HITS_ONLY QUERIES_FILE OUTPUT_FILE \u003e /dev/null 2\u003e\u00261`\n\nTo run time and click using related searches calculation and get the result in Elasticsearch index, one should run the following command:\n\n`java -jar -DuseElasticSearch=true TimeAndClickRelatedSearches.jar REDIS_HOST REDIS_PORT TIME_BETWEEN_QUERIES_IN_SECONDS SUGGESTION_THRESHOLD QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD QUERY_SIMILARITY_MORE_SIMILAR TEXT_SIMILARITY_BOOST TIME_AND_CLICK_BOOST SUGGEST_NON_ZERO_HITS_ONLY QUERIES_FILE ES_HOST ES_INDEX_NAME \u003e /dev/null 2\u003e\u00261`\n\nAnd the parameters are:\n- REDIS_HOST - Redis host, e.g., _localhost_\n- REDIS_PORT - Redis port, e.g., _6379_\n- TIME_BETWEEN_QUERIES_IN_SECONDS - maximum time between queries in seconds, e.g., _20_ \n- SUGGESTION_THRESHOLD - suggestion threshold, e.g., _0.2_\n- QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD - query similarity threshold, e.g., _0.9_\n- QUERY_SIMILARITY_MORE_SIMILAR - boolean, set to _true_ will result in suggesting similar queries according to the given _QUERY_SIMILARITY_ACCEPTANCE_THRESHOLD_\n- TEXT_SIMILARITY_BOOST - boost related to text similarity, e.g., _0.2_\n- TIME_AND_CLICK_BOOST - boost related to clicks, e.g., _2.0_\n- SUGGEST_NON_ZERO_HITS_ONLY - boolean value, when set to _true_ only non-zero hits queries will be suggested\n- QUERIES_FILE - queries log file\n- OUTPUT_FILE - output file with results\n- ES_HOST - Elasticsearch address\n- ES_INDEX_NAME - Elasticsearch index name\n\n#### Query Log Format\n\nRelated Searches expects a given log format out of the box, the popular [AOL](https://en.wikipedia.org/wiki/AOL_search_data_leak) log file format. Each query log line contained tabulate separated entries:\n- session identifier\n- query \n- query time\n- click information (optional)\n- was search successful (optional)\n\nFor example:\n```\n12345\tsematext training\tFri, 19 Aug 2016 13:30:00 GMT\n12345\tsematext solr training\tFri, 19 Aug 2016 13:31:00 GMT\n56789\tsematext elasticsearch\tFri, 19 Aug 2016 13:35:00 GMT\t1\ttrue\t\n56789\tsematext training\tFri, 19 Aug 2016 13:35:20 GMT\t0\tfalse\n```\n\n# License\n\n```\nThis software is licensed under the Apache License, version 2 (\"ALv2\"), quoted below.\n\nCopyright 2009-2016 Sematext Group, Inc \u003chttp://sematext.com\u003e\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not\nuse this file except in compliance with the License. You may obtain a copy of\nthe License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT\nWARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the\nLicense for the specific language governing permissions and limitations under\nthe License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsematext%2Frelated-searches","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsematext%2Frelated-searches","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsematext%2Frelated-searches/lists"}