{"id":13679286,"url":"https://github.com/AdeDZY/K-NRM","last_synced_at":"2025-04-29T18:31:57.099Z","repository":{"id":72284427,"uuid":"109185433","full_name":"AdeDZY/K-NRM","owner":"AdeDZY","description":"K-NRM: End-to-End Neural Ad-hoc Ranking with Kernel Pooling","archived":false,"fork":false,"pushed_at":"2017-11-02T21:17:50.000Z","size":274,"stargazers_count":201,"open_issues_count":13,"forks_count":43,"subscribers_count":18,"default_branch":"master","last_synced_at":"2024-11-11T22:35:13.478Z","etag":null,"topics":["deep-learning","information-retrieval","neural-network"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AdeDZY.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-11-01T21:22:30.000Z","updated_at":"2024-10-30T07:34:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"713961be-348d-4c2d-8128-2d86c4484d16","html_url":"https://github.com/AdeDZY/K-NRM","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AdeDZY%2FK-NRM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AdeDZY%2FK-NRM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AdeDZY%2FK-NRM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AdeDZY%2FK-NRM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AdeDZY","download_url":"https://codeload.github.com/AdeDZY/K-NRM/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251560040,"owners_count":21609129,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","information-retrieval","neural-network"],"created_at":"2024-08-02T13:01:03.967Z","updated_at":"2025-04-29T18:31:56.825Z","avatar_url":"https://github.com/AdeDZY.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# K-NRM\nThis is the implementation of the Kernel-based Neural Ranking Model (K-NRM) model from paper [End-to-End Neural Ad-hoc Ranking with Kernel Pooling](http://www.cs.cmu.edu/~zhuyund/papers/end-end-neural.pdf).\n\n\u003cp align=\"center\"\u003e \n\u003cimg src=\"https://github.com/AdeDZY/K-NRM/blob/master/model_simplified-1.png\" width=\"400\" align=\"center\"\u003e\n\u003c/p\u003e\n\nIf you use this code for your scientific work, please cite it as ([bibtex](#cite-the-paper)):\n\n```\nC. Xiong, Z. Dai, J. Callan, Z. Liu, and R. Power. End-to-end neural ad-hoc ranking with kernel pooling. \nIn Proceedings of the 40th International ACM SIGIR Conference on Research \u0026 Development in Information Retrieval. \nACM. 2017.\n```\n\n\n\n### Requirements\n---\n- Tensorflow 0.12 \n- Numpy\n- traitlets\n\nComing soon: K-NRM with Tensorflow 1.0\n\n### Guide To Use\n---\n**Configure**: first, configure the model through the config file. Configurable parameters are listed [here](#configurations)\n\n[sample.config](https://github.com/AdeDZY/K-NRM/blob/master/sample.config)\n\n**Training** : pass the config file, training data and validation data as\n```ruby\npython ./knrm/model/model_knrm.py config-file\\\n    --train \\\n    --train_file: path to training data\\\n    --validation_file: path to validation data\\\n    --train_size: size of training data (number of training samples)\\\n    --checkpoint_dir: directory to store/load model checkpoints\\ \n    --load_model: True or False. Start with a new model or continue training\n```\n\n[sample-train.sh](https://github.com/AdeDZY/K-NRM/blob/master/sample-train.sh)\n\n**Testing**: pass the config file and testing data as\n```ruby\npython ./knrm/model/model_knrm.py config-file\\\n    --test \\\n    --test_file: path to testing data\\\n    --test_size: size of testing data (number of testing samples)\\\n    --checkpoint_dir: directory to load trained model\\\n    --output_score_file: file to output documents score\\\n\n```\nRelevance scores will be output to output_score_file, one score per line, in the same order as test_file.\nWe provide a script to convert scores into trec format.\n```ruby\n./knrm/tools/gen_trec_from_score.py\n```\n\n### Data Preperation\n---\nAll queries and documents must be mapped into sequences of integer term ids. Term id starts with 1.\n-1 indicates OOV or non-existence. Term ids are sepereated by `,`\n\n**Training Data Format**\n\nEach training sample is a tuple of (query, postive document, negative document)\n\n`query   \\t postive_document   \\t negative_document  \\t score_difference `\n\nExample: `177,705,632   \\t  177,705,632,-1,2452,6,98   \\t  177,705,632,3,25,14,37,2,146,159, -1   \\t    0.119048`\n\nIf `score_difference \u003c 0`, the data generator will swap postive docment and negative document.\n\nIf `score_difference \u003c lickDataGenerator.min_score_diff`, this training sample will be omitted.\n\nWe recommend shuffling the training samples to ease model convergence. \n\n**Testing Data Format**\n\nEach testing sample is a tuple of (query, document)\n\n`q   \\t document`\n\nExample: `177,705,632  \\t   177,705,632,-1,2452,6,98`\n\n\n\n### Configurations \n---\n\n**Model Configurations**\n- \u003ccode\u003eBaseNN.n_bins\u003c/code\u003e: number of kernels (soft bins) (default: 11. One exact match kernel and 10 soft kernels)\n- \u003ccode\u003eKnrm.lamb\u003c/code\u003e: defines the guassian kernels' sigma value. sigma = lamb * bin_size (default:0.5 -\u003e sigma=0.1)\n- \u003ccode\u003eBaseNN.embedding_size\u003c/code\u003e: embedding dimension (default: 300)\n- \u003ccode\u003eBaseNN.max_q_len\u003c/code\u003e: max query length (default: 10)\n- \u003ccode\u003eBaseNN.max_d_len\u003c/code\u003e: max document length (default: 50)\n- \u003ccode\u003eDataGenerator.max_q_len\u003c/code\u003e: max query length. Should be the same as \u003ccode\u003eBaseNN.max_q_len\u003c/code\u003e (default: 10)\n- \u003ccode\u003eDataGenerator.max_d_len\u003c/code\u003e: max query length. Should be the same as \u003ccode\u003eBaseNN.max_d_len\u003c/code\u003e (default: 50)\n- \u003ccode\u003eBaseNN.vocabulary_size\u003c/code\u003e: vocabulary size.\n- \u003ccode\u003eDataGenerator.vocabulary_size\u003c/code\u003e: vocabulary size.\n\n\n\n**Data**\n- \u003ccode\u003eKnrm.emb_in\u003c/code\u003e: initial embeddings\n- \u003ccode\u003eDataGenerator.min_score_diff\u003c/code\u003e: \nminimum score differences between postive documents and negative ones (default: 0)\n\n**Training Parameters**\n- \u003ccode\u003eBaseNN.bath_size\u003c/code\u003e: batch size (default: 16)\n- \u003ccode\u003eBaseNN.max_epochs\u003c/code\u003e: max number of epochs to train\n- \u003ccode\u003eBaseNN.eval_frequency\u003c/code\u003e: evaluate model on validation set very this steps (default: 1000)\n- \u003ccode\u003eBaseNN.checkpoint_steps\u003c/code\u003e: save model very this steps (default: 10000)\n- \u003ccode\u003eKnrm.learning_rate\u003c/code\u003e: learning rate for Adam Opitmizer (default: 0.001)\n- \u003ccode\u003eKnrm.epsilon\u003c/code\u003e: epsilon for Adam Optimizer (default: 0.00001)\n\nEfficiency\n---\nDuring training, it takes about 60ms to process one batch on a single-GPU machine with the following settings:\n- batch size: 16\n- max_q_len: 10\n- max_d_len: 50\n- vocabulary_size: 300K\n\nSmaller vocabulary and shorter documents accelerate the training.\n\n### Click2Vec\n---\nWe also provide the click2vec model as described in our paper.\n- \u003ccode\u003e./knrm/click2vec/generate_click_term_pair.py\u003c/code\u003e: generate \u003cquery_term, clicked_title_term\u003e pairs\n- \u003ccode\u003e./knrm/click2vec/run_word2vec.sh\u003c/code\u003e: call Google's word2vec tool to train click2vec.\n\n### Cite the paper\n---\nIf you use this code for your scientific work, please cite it as:\n\n```\nC. Xiong, Z. Dai, J. Callan, Z. Liu, and R. Power. End-to-end neural ad-hoc ranking with kernel pooling. \nIn Proceedings of the 40th International ACM SIGIR Conference on Research \u0026 Development in Information Retrieval. \nACM. 2017.\n```\n\n```\n@inproceedings{xiong2017neural,\n  author          = {{Xiong}, Chenyan and {Dai}, Zhuyun and {Callan}, Jamie and {Liu}, Zhiyuan and {Power}, Russell},\n  title           = \"{End-to-End Neural Ad-hoc Ranking with Kernel Pooling}\",\n  booktitle       = {Proceedings of the 40th International ACM SIGIR Conference on Research \u0026 Development in Information Retrieval},\n  organization    = {ACM},\n  year            = 2017,\n}\n```\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAdeDZY%2FK-NRM","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAdeDZY%2FK-NRM","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAdeDZY%2FK-NRM/lists"}