{"id":18778758,"url":"https://github.com/neulab/knn-transformers","last_synced_at":"2025-04-03T02:11:07.156Z","repository":{"id":45662868,"uuid":"511270617","full_name":"neulab/knn-transformers","owner":"neulab","description":"PyTorch + HuggingFace code for RetoMaton: \"Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval\" (ICML 2022), including an implementation of kNN-LM and kNN-MT","archived":false,"fork":false,"pushed_at":"2022-10-20T14:34:50.000Z","size":1696,"stargazers_count":270,"open_issues_count":7,"forks_count":24,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-24T08:07:06.561Z","etag":null,"topics":["huggingface","knn","knn-lm","knn-mt","knn-transformers","knnlm","knnmt","language","language-models","machine","models","nearest","nearest-neighbor","neighbor","neuro-symbolic","pytorch","retomaton","transformers","translation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/neulab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null}},"created_at":"2022-07-06T19:41:53.000Z","updated_at":"2025-03-16T13:19:35.000Z","dependencies_parsed_at":"2023-01-20T16:00:57.562Z","dependency_job_id":null,"html_url":"https://github.com/neulab/knn-transformers","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neulab%2Fknn-transformers","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neulab%2Fknn-transformers/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neulab%2Fknn-transformers/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/neulab%2Fknn-transformers/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/neulab","download_url":"https://codeload.github.com/neulab/knn-transformers/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246922247,"owners_count":20855345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["huggingface","knn","knn-lm","knn-mt","knn-transformers","knnlm","knnmt","language","language-models","machine","models","nearest","nearest-neighbor","neighbor","neuro-symbolic","pytorch","retomaton","transformers","translation"],"created_at":"2024-11-07T20:17:12.157Z","updated_at":"2025-04-03T02:11:07.131Z","avatar_url":"https://github.com/neulab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kNN-transformers: Nearest-Neighbor Language and Machine Translation Models based on Hugging Face's 🤗 `transformers` library\n\nThis is a Hugging Face's 🤗 `transformers` implementation of k-nearest-neighbor-based language models and machine translation models,\ndesigned to be easy and useful in research, and for experimenting with new ideas in kNN-based models. \n\nAll previous kNN-LM based implementations are implemented in the `fairseq` library, and **they forked/duplicated the library's entire codebase** to implement their modification.\nThese include the official kNN-LM repository [https://github.com/urvashik/knnlm](https://github.com/urvashik/knnlm), the kNN-MT repository [https://github.com/urvashik/knnmt](https://github.com/urvashik/knnmt), the official RetoMaton repository [https://github.com/neulab/retomaton](https://github.com/neulab/retomaton), and others.\n\n\nWe implement [k-nearest-neighbor language model (kNN-LM)](https://arxiv.org/pdf/1911.00172.pdf) (Khandelwal et al., ICLR'2020), [k-nearest-neighbor machine translation (kNN-MT)](https://arxiv.org/pdf/2010.00710) (Khandelwal et al., ICLR'2021) and this is also\nan official implementation of the RetoMaton model described in:\n[Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval](https://arxiv.org/pdf/2201.12431.pdf) (ICML'2022). Most importantly, we implement these models in 🤗 `transformers`, and without modifying the `transformers` library itself.\n\nTo use this repository, all you need to do is copy its `*.py` files into your project.\nYou can load any language model from Hugging Face's hub (such as `gpt2`), by `model = AutoModelForCausalLM.from_pretrained(...)`, build a datastore or download ours (need to be performed only once), and then:\n```python\nknn_wrapper = KNNWrapper(...) # or: RetomatonWrapper(...)\nknn_wrapper.break_into(model)\n```\n\nThat's it! The model now internally uses kNN-LM or RetoMaton (see a concrete example at [run_clm.py](run_clm.py#L427-L438))\n\nThe files `knnlm.py` and `retomaton.py` are standalone and can be copied to any project. The file `run_clm.py` is a modified version of [this example by huggingface](https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_clm.py) which shows an example of how to load and run kNN-LM and RetoMaton.\nThe file `run_translation.py` is a modified version of [this translation example by huggingface](https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation), which shows how to use our code for kNN-MT and RetoMaton.\n\n\nThis repository is maintained by [Uri Alon](https://urialon.ml).\nPlease let us know if anything is not working as expected, and feel free to create [new issues](https://github.com/neulab/knn-transformers/issues) or email [ualon@cs.cmu.edu](ualon@cs.cmu.edu) with any questions.\nContributions are welcome!\n\n\nTable of Contents\n=================\n  * [Background](#background)\n  * [Available Models](#available-models)\n  * [Results](#results)\n  * [Quickstart](#quickstart)\n    * [Requirements](#requirements)\n    * [Step 1: Evaluating the base Language Model](#step-1-evaluating-the-base-language-model)\n    * [Step 2: Saving a Datastore](#step-2-saving-a-datastore)\n    * [Step 3: Building the FAISS index](#step-3-building-the-faiss-index)\n    * [Step 4: Evaluating Models](#step-4-evaluating-models)\n    * [Step 5: Adding clustering](#step-5-adding-clustering)\n  * [Machine Translation (kNN-MT)](#machine-translation-knn-mt)\n  * [All files](#all-files)\n  * [Differences from the kNN-LM implementation](#differences-from-the-knn-lm-implementation)\n  * [Citation](#citation)\n\n## Background\n\n### kNN-LM and kNN-MT\nThe k-nearest neighbor language model takes an already-trained model, performs a single forward pass over the entire training set, and creates a datastore of `(key,value)` pairs, where `key` is a hidden representation of the trained model after reading a training example, and `value` is the token that should be predicted next.\n\nAt test time, for every predicted token, the model performs a k-nearest neighbors search in the datastore, retrieves the `(key,value)` pairs that are closest to the test hidden representation, and normalizes their distances using softmax. Finally, the model interpolates the base LM's probability with the probability formed by the retrieved nearest neighbors and their normalized distances.\nFor more details, see the [paper by Khandelwal et al., ICLR'2020](https://arxiv.org/pdf/1911.00172.pdf)\n\nkNN-MT is similar to kNN-LM, but it addresses seq2seq/encoder-decoder models, mainly for machine translation. For more details, see the [paper by Khandelwal et al., ICLR'2021](https://arxiv.org/pdf/2010.00710.pdf)\n\n\n### RetoMaton\nRetoMaton extends kNN-LM and kNN-MT, by (1) saving a *pointer* in every datastore entry; and (2) clustering entries according to their keys. That is, every datastore entry is now a tuple `(key,value,pointer)`, and it belongs to a cluster. \n\nThese two changes create an automaton from the datastore, where states are clusters, edges are pointers (shared among examples in the same cluster), and transition weights are the normalized distances between the test representation and each key.\n\nAt test time, the model traverses the automaton, and follows the edges according to the token that was predicted.\nThis allows to save up to 80% of the kNN searches by following pointers instead of performing the expensive search, or reducing perplexity without saving searches.\n\nFor more details, see the [paper by Alon et al., ICML'2022](https://arxiv.org/pdf/2201.12431.pdf)\n\n\u003ccenter style=\"padding: 40px\"\u003e\u003cimg width=\"60%\" src=\"images/overview.jpeg\" /\u003e\u003c/center\u003e\n\n## Available Models\n\nkNN-LM and RetoMaton datastores depend on the LM that was used to create them. We fine-tuned a few `gpt2`-based models on the training set of Wikitext-103 (because Wikitext-103 was not included in GPT2's pretraining data):\n* `neulab/distilgpt2-finetuned-wikitext103`\n* `neulab/gpt2-finetuned-wikitext103`\n* `neulab/gpt2-med-finetuned-wikitext103`\n* `neulab/gpt2-large-finetuned-wikitext103`\n\nAll these models are available at the Hugging Face Hub and can be loaded by (for example):\n```python\nfrom transformers import AutoTokenizer, AutoModelForCausalLM\n\ntokenizer = AutoTokenizer.from_pretrained('neulab/gpt2-finetuned-wikitext103')\nmodel = AutoModelForCausalLM.from_pretrained('neulab/gpt2-finetuned-wikitext103')\n```\n\nThis project is not limited to these models, and can work with any language model or seq2seq model.\n\nWe fine-tuned all language models using:\n```bash\npython run_clm.py --model_name_or_path \u003cbase_model_name\u003e \\\n    --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n    --do_train --do_eval --output_dir finetune_gpt2_wikitext103/ \\\n    --save_total_limit 5 --per_device_train_batch_size 2\n```\nWhere `\u003cbase_model_name\u003e` is, for example, `gpt2`, `distilgpt2`, `gpt2-med`, `gpt2-large`, or `gpt2-xl`.\n\nWe have not yet released finetuned machine translation models, but the code in this repository works for machine translation as well, using the `run_translation.py` script.\n\n## Results - **Wikitext-103**\nThe exact results from the RetoMaton papers can be reproduced using the code at [https://github.com/neulab/retomaton](https://github.com/neulab/retomaton) (based on `fairseq`).\n\nThe following results were obtained using the code in this repository:\n\n\n\n| Base LM:        | `distilgpt2` | `gpt2` | `gpt2-medium` |\n| :---            |    ----:   |     ---: | ---:          |\n| base perplexity | 18.25      | 14.84    | 11.55         |\n| kNN-LM          |  15.03     |   12.57  |  **10.59**      |\n| RetoMaton       | **14.71**  | **12.44**| **10.59**     |\n\nAnd when varying the fraction of saved searches:\n\n\u003ccenter style=\"padding: 40px\"\u003e\u003cimg width=\"60%\" src=\"images/wiki_distilgpt2.png\" /\u003e\u003c/center\u003e\n\u003ccenter style=\"padding: 40px\"\u003e\u003cimg width=\"60%\" src=\"images/wiki_gpt2.png\" /\u003e\u003c/center\u003e\nThese are the results from the RetoMaton paper, on a model that was trained on Wikitext-103 from scratch:\n\u003ccenter style=\"padding: 40px\"\u003e\u003cimg width=\"60%\" src=\"images/wiki.png\" /\u003e\u003c/center\u003e\n\n## Results - **Translation**\nOn the validation set of `--dataset_name wmt16 --dataset_config_name ro-en`.\n\n| Base model:     | `t5-small` | `t5-base` | \n| :---            |    ----:   |     ---: | \n| base BLEU       | 26.15      | 27.70    | \n| + kNN-MT          |  **26.42**     |  **27.92**  | \n|                   | `--knn_temp=50 --k=32 --lmbda=0.25` | `--knn_temp=200 --k=512 --lmbda=0.2`\n\n**If you perform additional experiments with our code, we would love to learn more about your results and share them here!**\n\n## Quickstart - Language Modeling\n\n### Step 0: Clone this repository:\n```bash\ngit clone https://github.com/neulab/knn-transformers\ncd knn-transformers\n```\n\n#### Requirements \nRun:\n```bash\npip install requirements.txt`\n```\n\n* The project also depends on the `faiss` library. In MacOS, use the Anaconda installation instead:\n```\nconda install -c conda-forge faiss-cpu\n```\n\n### Step 1: Evaluating the base Language Model\n\nTo evaluate the fine-tuned model (for example, `neulab/gpt2-finetuned-wikitext103`) on the validation set (without any retrieval):\n\n```bash\nMODEL=neulab/gpt2-finetuned-wikitext103\n\npython -u run_clm.py \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n  --output_dir checkpoints/${MODEL} \\\n  --do_eval --eval_subset validation\n```\n\n### Step 2: Saving a Datastore\n\nYou can either download our preprocessed Wikitext-103 datastores, or preprocess them yourself.\n\nTo download a datastore for Wikitext-103 that we created for the finetuned `gpt2` model (`neulab/gpt2-finetuned-wikitext103`):\n```bash\nwget -P checkpoints/gpt2/ https://knn-transformers.s3.amazonaws.com/gpt2/dstore_gpt2_116988150_768_vals.npy\n```\n\nSimilarly, we created datastores using the `distilgpt2-finetuned-wikitext103`, `gpt2-med-finetuned-wikitext103` and `gpt2-large-finetuned-wikitext103`.\nFor all available datastores, see: [https://knn-transformers.s3.amazonaws.com/index.html](https://knn-transformers.s3.amazonaws.com/index.html)\n\nTo save a datastore, run:\n```bash\nMODEL=neulab/gpt2-finetuned-wikitext103\n\npython -u run_clm.py \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n  --do_eval --eval_subset train \\\n  --output_dir checkpoints/${MODEL} \\\n  --dstore_dir checkpoints/${MODEL} \\\n  --save_knnlm_dstore\n```\n\n\n### Step 3: Building the FAISS index\n\nThe FAISS index requires a training phase where it learns an index for accessing the keys quickly. This step does not require a GPU.\n\nTo download an index for the finetuned `gpt2` model (`neulab/gpt2-finetuned-wikitext103`):\n```\nwget -P checkpoints/gpt2/ https://knn-transformers.s3.amazonaws.com/gpt2/index_gpt2_116988150_768.indexed\n```\n\nSimilarly, we trained `faiss` indexes for  `distilgpt2-finetuned-wikitext103`, `gpt2-med-finetuned-wikitext103` and `gpt2-large-finetuned-wikitext103`, see: [https://knn-transformers.s3.amazonaws.com/index.html](https://knn-transformers.s3.amazonaws.com/index.html)\n\nTo build the FAISS index yourself (not needed if you already downloaded ours):\n```bash\nMODEL=neulab/gpt2-finetuned-wikitext103\n\npython -u run_clm.py \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n  --output_dir checkpoints/${MODEL} \\\n  --dstore_dir checkpoints/${MODEL} \\\n  --build_index\n```\n\n\n### Step 4: Evaluating Models\n\nTo evaluate kNN-LM and RetoMaton on the validation set:\n\n```bash\nMODEL=neulab/gpt2-finetuned-wikitext103\n\npython -u run_clm.py \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n  --output_dir checkpoints/${MODEL} \\\n  --do_eval --eval_subset validation \\\n  --dstore_dir checkpoints/${MODEL} --retomaton\n```\n\nTo use kNN-LM, use the `--knn` flag instead of `--retomaton`.\n\nTo encourage the RetoMaton model to perform a full kNN search more frequently and thus increase accuracy and reduce perplexity, use a larger value of `--min-knns` such as `100`. Using `--min-knns 9999999` makes the model perform kNN search at every step (`FoSS = 0` in Figure 3 of the paper), and achieves the best results at the cost of slower speed.\n\nAdditional possible test-time tunable hyperparameters are `--lmbda` (the interpolation factor between the datastore and the base LM), `--k` (the number of retrieved nearest neighbors) and `--knn_temp` (the softmax temperature when converting the nearest-neighbor distances into a probability distribution).\n\n### Step 5: Adding clustering\n\nRetoMaton can work without clusters, in which is utilizes its pointers only.\nUsing clustering allows it to save more nearest-neighbor searches and further reduce perplexity.\n\nTo download our processed clusters the finetuned `gpt2` model (`neulab/gpt2-finetuned-wikitext103`):\n```\nwget -P checkpoints/gpt2/ https://knn-transformers.s3.amazonaws.com/index.htmlgpt2/members_gpt2_116988150_768_20000000_500000.pkl\n```\n\nSimilarly, we also provide clusters for the `distilgpt2` model (`neulab/distilgpt2-finetuned-wikitext103`) at [https://knn-transformers.s3.amazonaws.com/index.html](https://knn-transformers.s3.amazonaws.com/index.html).\n\nTo cluster similar keys for RetoMaton yourself:\n```bash\nMODEL=neulab/gpt2-finetuned-wikitext103\n\npython -u run_clm.py \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wikitext --dataset_config_name wikitext-103-raw-v1 \\\n  --output_dir checkpoints/${MODEL} \\\n  --dstore_dir checkpoints/${MODEL}/ \\\n  --cluster_dstore --num_clusters 500000 --sample_size 20000000\n```\n\nOnce the clustering file is saved in the directory pointed to by `--dstore_dir`, it will automatically be picked up when running evaluation ([as in the previous step](#step-4-evaluating-models))\n\nOptional clustering hyperparameters are `--num_clusters` (typically `1/100` or `1/200` of the datastore size) and `--sample_size`  (ideally as high as possible, but higher values consume more memory and take longer to run).\n\n## Machine Translation (kNN-MT)\nUsing our code for machine translation and kNN-MT is very similar to language modeling, using the file `run_translation.py` instead of `run_clm.py`, and following the example instructions from huggingface: [https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation](https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation).\n\nImportantly, the `--knn_temp` flag should be used and tuned for kNN-MT. As shown in [the kNN-MT paper](https://arxiv.org/pdf/2010.00710), the optimal temperature for kNN-MT can be `10` to `100`.\n\nThe `--lmbda` interpolation factor is also typically larger in kNN-MT, and can be `0.4`-`0.8`.\n\n### Evaluating the base MT model \n```bash\nMODEL=t5-small\n\npython -u run_translation.py  \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wmt16 --dataset_config_name ro-en \\\n  --per_device_eval_batch_size=4 \\\n  --output_dir checkpoints-translation/${MODEL} \\\n  --source_lang en --target_lang ro \\\n  --do_eval \\\n  --predict_with_generate \\\n  --source_prefix \"translate English to Romanian: \"\n```\n\n**Note** that the flag `--source_prefix \"translate English to Romanian: \"` is slightly different for every model and task, and may be unneeded for other models, as detailed at [https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation](https://github.com/huggingface/transformers/tree/main/examples/pytorch/translation).\n\n\n\n### Saving a datastore for kNN-MT\n\nExamples datastores for `t5-small` and `t5-base` on `wmt16 en-ro` are available at [https://knn-transformers.s3.amazonaws.com/index.html](https://knn-transformers.s3.amazonaws.com/index.html).\n\n```bash\nMODEL=t5-small\n\npython -u run_translation.py  \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wmt16 --dataset_config_name ro-en \\\n  --per_device_train_batch_size 4 --per_device_eval_batch_size=4 \\\n  --output_dir checkpoints-translation/${MODEL} \\\n  --source_lang en --target_lang ro \\\n  --dstore_size 26565876 \\\n  --dstore_dir checkpoints-translation/${MODEL} \\\n   --save_knnlm_dstore --do_eval --eval_subset train \\\n   --source_prefix \"translate English to Romanian: \"\n```\n\n### Building the FAISS index for kNN-MT\n```bash\nMODEL=t5-small\n\npython -u run_translation.py  \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wmt16 --dataset_config_name ro-en \\\n  --per_device_train_batch_size 4 --per_device_eval_batch_size=4 \\\n  --output_dir checkpoints-translation/${MODEL} \\\n  --source_lang en --target_lang ro \\\n  --dstore_size 26565876 \\\n  --dstore_dir checkpoints-translation/${MODEL} \\\n  --build_index\n```\n\n### Evaluating kNN-MT\n```bash\nMODEL=t5-small\n\npython -u run_translation.py  \\\n  --model_name_or_path ${MODEL} \\\n  --dataset_name wmt16 --dataset_config_name ro-en \\\n  --per_device_eval_batch_size=4 \\\n  --output_dir checkpoints-translation/${MODEL} \\\n  --source_lang en --target_lang ro \\\n  --do_eval \\\n  --predict_with_generate \\\n  --source_prefix \"translate English to Romanian: \" \\\n  --dstore_size 26565876 \\\n  --dstore_dir checkpoints-translation/${MODEL} \\\n  --knn_temp 50 --k 32 --lmbda 0.25 \\\n  --retomaton\n```\n\nTo use kNN-MT, use the `--knn` flag instead of `--retomaton`.\n\n## All files: \nDatastores and indexes can be downloaded from\n [https://knn-transformers.s3.amazonaws.com/index.html](https://knn-transformers.s3.amazonaws.com/index.html)\n\nAll fine-tuned models are available on Hugging Face Hub: [https://huggingface.co/neulab](https://huggingface.co/neulab)\n\n\n## Differences from the kNN-LM implementation\n* The original [kNN-LM](https://github.com/urvashik/knnlm) repository uses `faiss` CPU to perform retrieval. However, we added the flag `--knnlm-gpu` that allows performing retrieval much faster on the GPU.\n* After each retrieval, the original [kNN-LM](https://github.com/urvashik/knnlm) repository loads the found keys and re-computes the distance from the query to each nearest neighbor. This is much more time consuming, unless loading all the keys (200GB) into memory.\nWe thus use the distances returned by `faiss` when performing search, or reconstructing the vectors from their index in RetoMaton, without loading the huge `keys.npy` file into memory.\n\n\n## Citation\nIf you use our code for research, please cite:\n\n[Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval](https://arxiv.org/pdf/2201.12431.pdf)\n\n```\n@inproceedings{alon2022neuro,\n  title={Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval},\n  author={Alon, Uri and Xu, Frank and He, Junxian and Sengupta, Sudipta and Roth, Dan and Neubig, Graham},\n  booktitle={International Conference on Machine Learning},\n  pages={468--485},\n  year={2022},\n  organization={PMLR}\n}\n```\n\nThis repository also implements:\n[Generalization through Memorization: Nearest Neighbor Language Models](https://arxiv.org/pdf/1911.00172)\n```\n@inproceedings{khandelwal20generalization,\n  title={{Generalization through Memorization: Nearest Neighbor Language Models}},\n  author={Khandelwal, Urvashi and Levy, Omer and Jurafsky, Dan and Zettlemoyer, Luke and Lewis, Mike},\n  booktitle={International Conference on Learning Representations (ICLR)},\n  year={2020}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneulab%2Fknn-transformers","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneulab%2Fknn-transformers","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneulab%2Fknn-transformers/lists"}