{"id":19016027,"url":"https://github.com/varunu28/prefix-autocomplete","last_synced_at":"2025-09-06T17:31:07.970Z","repository":{"id":77611548,"uuid":"152357648","full_name":"varunu28/Prefix-Autocomplete","owner":"varunu28","description":"A command line autocomplete suggestion application using Java","archived":false,"fork":false,"pushed_at":"2021-03-28T19:42:11.000Z","size":27,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-11-08T19:49:52.079Z","etag":null,"topics":["autocomplete","java","tries"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/varunu28.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-10-10T03:27:42.000Z","updated_at":"2024-10-29T11:53:45.000Z","dependencies_parsed_at":"2023-03-12T01:15:09.652Z","dependency_job_id":null,"html_url":"https://github.com/varunu28/Prefix-Autocomplete","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunu28%2FPrefix-Autocomplete","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunu28%2FPrefix-Autocomplete/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunu28%2FPrefix-Autocomplete/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/varunu28%2FPrefix-Autocomplete/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/varunu28","download_url":"https://codeload.github.com/varunu28/Prefix-Autocomplete/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232133899,"owners_count":18477299,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autocomplete","java","tries"],"created_at":"2024-11-08T19:40:38.642Z","updated_at":"2025-01-01T23:10:17.714Z","avatar_url":"https://github.com/varunu28.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Prefix-Autocomplete\nA command line autocomplete application using Java\n\n## What it does?\nA prefix autocomplete which can match words associated with a prefix typical to an SQL editor.\nSo for a list of words such as below:\n- reversal\n- reverse\n- reverent\n- first_name_reversal\n\nIf we provide a prefix of ```rev```, it would pick up all the four words while performing an autocomplete.\n\n## How to run it?\n\n - Open the project in any IDE (Preferably IntelliJ)\n - I have a sample file in src/resources called ```data.txt``` containing 2000 words with associated weights. If you want to change it, then upload your file with same name. If you change the filename then you would have to make changes in ```InputDataProcessor.java```\n - Run the ```Runner.java``` and start adding prefixes. You will be getting top 10 suggestions for the prefix (If there are at least 10 words starting with that prefix or else you will get less number of suggestions)\n - For changing cache size(```CACHE_SIZE```) or suggestions limit (```SIZE```) change the corresponding values in ```Autocomplete.java```\n - Open an issue if you find any difficulty while running the code\n  \n## Design\nWhen we hear autocomplete, the first thing that comes to our mind is ```trie```. That one data structure which gets overshadowed by complex tree traversals :stuck_out_tongue: \n\nSo when I started working on this project, I had a slight idea on how to approach this problem as there is a specific data structure assigned for this task in computer science. I created a a POC and then moved on to optimize my approach in order to make the task of getting top suggestions as fast as possible. \n\nI have divided my approach for this project below in form of multiple stages this project went through and tried to explain that the end result was not the first solution that I designed but an iteration of improvements which I implemented on the code base.\n\n#### Initial POC\n\nI started by fulfilling the basic criteria an autocomplete should fulfil. So if there are words given like the ones below:\n- reversal\n- reverse\n- reverent\n- first_name_reversal\n- hello\n\nAnd the user adds a prefix of ```rev``` then they should get back the first three words which start with the prefix. I am right now not considering any weight associated to these words which actually happen in real life searches eg: Once we type ```do``` in Google search, chances of getting ```Donald Trump``` are higher than ```dollar```. I used a ```trie``` data structure for this where the ```trie``` node looked like below:\n```\n    private static final class Node {\n        String prefix;\n        Map\u003cCharacter, Node\u003e childrens;\n        boolean isWord;\n        \n         public Node(String prefix) {\n            this.prefix = prefix;\n            this.childrens = new HashMap\u003c\u003e();\n        }\n    }\n```\n\nHere my approach was to create a data structure of ```trie``` nodes with the given database of words with each character of the word and children of that node as the remaining characters of that word.\nThis gives me the expected results as when a user types a prefix of ```rev```, I look up through the existing nodes which I created using the input database of words and start narrowing down my search character-by-character of the given prefix. If I reach a point where a character from the prefix does not have any children in the stored data structure then that means that there is no word present starting with that prefix and we return an empty result.\n\n#### Weights associated with words\nNow that I was getting the results for an input prefix, my next goal was to bring weights associated with the word into play and for that I needed to narrow down the suggestion list to 10 results based on the weights of the words. For this I used two different data structures. First being a ```HashMap``` where I stored the word and its associated weight. I populate this ```HashMap``` when I read the input file of words which contains all the words and their associated weights. \n\nNow that I had all the suggestions for a prefix and weights associated with these words, I had to choose top 10 words based on their weights. A data structure that does it in an efficient manner is a ```PriorityQueue``` in which I use a comparision based on the weights of words which I get from the ```HashMap```\n```$xslt\nPriorityQueue\u003cString\u003e results =\n                new PriorityQueue\u003c\u003e(SIZE, Comparator.comparingInt(s -\u003e words.get(s)));\n```\nLimiting the size of ```PriorityQueue``` to 10 gives the top 10 suggestions for a prefix.\n\n#### Caching \nA basic LRU caching really helps to optimize the operations of getting the suggestions, if user repeats certain prefix query frequently. The code right now checks if the prefix entered by user is already present in cache and returns the result associated with prefix directly from cache rather than doing the complete computation. Cache gets modified in a ```LRU``` manner and the size of cache can be increased based on the use case.\n\n#### Pre-computations\nCurrently, for a given prefix, the code goes down every character of prefix and creates a list of words. An optimization to that is to pre-compute a list of completions on each prefix so that we don't have to traverse the complete word list again.\n\nFor this, a ```list``` of completions is added to the ```trie``` node to store completions while creating initial data structure from input database. Now for each prefix, we just check if that is a valid prefix and if that is then we just return a list of associated words back as a list which is converted to a ```PriorityQueue```.\n\nA trade-off which we are making here is to increase storage space as compared to computation time.\n\n\n#### Autocomplete with underscore\nFor this autocomplete to be used in an SQL editor, it has to pass the test of matching words where prefix is not just in the starting of word but also in case the prefix is after an underscore.\n\n - first_name_appoint\n - apple\n - first_name_class_appoint_verb\n\nIn the above example, a prefix of ```app``` should match all the three words as the word ```appoint``` is present immediately after a prefix. \n\nTo solve this problem, I updated the logic of adding the word. So if I get the word ```first_name_appoint```, while inserting the word into database, I would insert three separate words i.e. ```first, name, appoint``` and put ```first_name_appoint``` into the list of their completions so that I can get it later. I add a check while putting words for a given prefix into the ```PriorityQueue``` that if that word was part of the ```HashMap``` which I created while reading the input word list. This is to avoid adding words which are not part of the actual word list to the suggestions.\n\n#### Optimization for searching in O(1) time\nRight now the code does a computation till the length of prefix for list of words associated with it. I creared a ```prefixMap``` in order to store all the computations associated with a prefix into a ```HashMap``` so that now we don't need to do go down the complete prefix character-by-character as we have pre-stored the completions which can be directly fed to ```PriorityQueue```.\n\n*A further optimization to this can be storing top K suggestions in a map so that we no longer need to do the PriorityQueue operations.*\n\n#### Serialization\nAs we have stored the prefix and associated words in a map, it becomes a ```Key-Value``` structure which can be transferred to a database such as ```Redis``` and then we can query the database to get the list associated with a prefix to be used in a ```PriorityQueue```. \n\n## What can be improved?\n\n - A fuzzy search functionality so that it can pick up partial words without an underscore\n - Code for updating and writing to a ```Redis``` database and wrapper around database operations to make ```CRUD``` operations on database easier.\n - Code to wipe out existing database of words and upload new word-list using a text file or an S3 bucket\n - Dockerize the application\n\n## TODO \n - [ ] Add unit tests\n - [ ] Add end to end tests\n - [ ] Clean up code\n    - [ ] Add interfaces for policies\n    - [ ] Make code in `Autocomplete.java` functional\n    - [ ] Add exception handling for data-processor with tests","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarunu28%2Fprefix-autocomplete","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvarunu28%2Fprefix-autocomplete","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvarunu28%2Fprefix-autocomplete/lists"}