{"id":18384860,"url":"https://github.com/allegro/allrank","last_synced_at":"2025-10-07T04:48:38.340Z","repository":{"id":37502391,"uuid":"214192712","full_name":"allegro/allRank","owner":"allegro","description":"allRank is a framework for training learning-to-rank neural models based on PyTorch.","archived":false,"fork":false,"pushed_at":"2024-08-06T19:11:59.000Z","size":129,"stargazers_count":962,"open_issues_count":18,"forks_count":127,"subscribers_count":26,"default_branch":"master","last_synced_at":"2025-10-05T19:47:54.330Z","etag":null,"topics":["click-model","deep-learning","information-retrieval","learning-to-rank","machine-learning","ndcg","python","pytorch","ranking","transformer"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allegro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-10T13:39:33.000Z","updated_at":"2025-09-26T19:30:41.000Z","dependencies_parsed_at":"2024-11-13T02:00:28.859Z","dependency_job_id":"7b3f6e92-426d-41f9-8f40-f80fabac6d1f","html_url":"https://github.com/allegro/allRank","commit_stats":{"total_commits":62,"total_committers":7,"mean_commits":8.857142857142858,"dds":0.5806451612903225,"last_synced_commit":"2923985135c8afcdae6392e5b810b62da7851276"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/allegro/allRank","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2FallRank","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2FallRank/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2FallRank/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2FallRank/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allegro","download_url":"https://codeload.github.com/allegro/allRank/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allegro%2FallRank/sbom","scorecard":{"id":185194,"data":{"date":"2025-08-11","repo":{"name":"github.com/allegro/allRank","commit":"c88475661cb72db292d13283fdbc4f2ae6498ee4"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":4,"checks":[{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Code-Review","score":10,"reason":"all changesets reviewed","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: containerImage not pinned by hash: Dockerfile:3","Warn: containerImage not pinned by hash: Dockerfile:14","Warn: containerImage not pinned by hash: Dockerfile:17","Warn: containerImage not pinned by hash: Dockerfile:20","Warn: pipCommand not pinned by hash: Dockerfile:15","Warn: pipCommand not pinned by hash: Dockerfile:18","Info:   0 out of   4 containerImage dependencies pinned","Info:   0 out of   2 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 30 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T19:40:21.598Z","repository_id":37502391,"created_at":"2025-08-16T19:40:21.598Z","updated_at":"2025-08-16T19:40:21.598Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278722765,"owners_count":26034461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-07T02:00:06.786Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["click-model","deep-learning","information-retrieval","learning-to-rank","machine-learning","ndcg","python","pytorch","ranking","transformer"],"created_at":"2024-11-06T01:15:41.421Z","updated_at":"2025-10-07T04:48:38.285Z","avatar_url":"https://github.com/allegro.png","language":"Python","readme":"# allRank : Learning to Rank in PyTorch\n\n## About\n\nallRank is a PyTorch-based framework for training neural Learning-to-Rank (LTR) models, featuring implementations of:\n* common pointwise, pairwise and listwise loss functions\n* fully connected and Transformer-like scoring functions\n* commonly used evaluation metrics like Normalized Discounted Cumulative Gain (NDCG) and Mean Reciprocal Rank (MRR)\n* click-models for experiments on simulated click-through data\n\n### Motivation\n\nallRank provides an easy and flexible way to experiment with various LTR neural network models and loss functions.\nIt is easy to add a custom loss, and to configure the model and the training procedure. \nWe hope that allRank will facilitate both research in neural LTR and its industrial applications.\n\n## Features\n\n### Implemented loss functions:\n 1. ListNet (for binary and graded relevance)\n 2. ListMLE\n 3. RankNet\n 4. Ordinal loss\n 5. LambdaRank\n 6. LambdaLoss\n 7. ApproxNDCG\n 8. RMSE\n 9. NeuralNDCG (introduced in https://arxiv.org/pdf/2102.07831)\n\n### Getting started guide\n\nTo help you get started, we provide a ```run_example.sh``` script which generates dummy ranking data in libsvm format and trains\n a Transformer model on the data using provided example ```config.json``` config file. Once you run the script, the dummy data can be found in `dummy_data` directory\n and the results of the experiment in `test_run` directory. To run the example, Docker is required.\n\n### Getting the right architecture version (GPU vs CPU-only)\n\nSince torch binaries are different for GPU and CPU and GPU version doesn't work on CPU - one must select \u0026 build appropriate docker image version.\n\nTo do so pass `gpu` or `cpu` as `arch_version` build-arg in \n\n```docker build --build-arg arch_version=${ARCH_VERSION}```\n\nWhen calling `run_example.sh` you can select the proper version by a first cmd line argument e.g. \n\n```run_example.sh gpu ...```\n\nwith `cpu` being the default if not specified.\n\n### Configuring your model \u0026 training\n\nTo train your own model, configure your experiment in ```config.json``` file and run  \n\n```python allrank/main.py --config_file_name allrank/config.json --run_id \u003cthe_name_of_your_experiment\u003e --job_dir \u003cthe_place_to_save_results\u003e```\n\nAll the hyperparameters of the training procedure: i.e. model defintion, data location, loss and metrics used, training hyperparametrs etc. are controlled\nby the ```config.json``` file. We provide a template file ```config_template.json``` where supported attributes, their meaning and possible values are explained.\n Note that following MSLR-WEB30K convention, your libsvm file with training data should be named `train.txt`. You can specify the name of the validation dataset \n (eg. valid or test) in the config. Results will be saved under the path ```\u003cjob_dir\u003e/results/\u003crun_id\u003e```\n \nGoogle Cloud Storage is supported in allRank as a place for data and job results.\n\n\n### Implementing custom loss functions\n\nTo experiment with your own custom loss, you need to implement a function that takes two tensors (model prediction and ground truth) as input\n and put it in the `losses` package, making sure it is exposed on a package level.\nTo use it in training, simply pass the name (and args, if your loss method has some hyperparameters) of your function in the correct place in the config file:\n\n```\n\"loss\": {\n    \"name\": \"yourLoss\",\n    \"args\": {\n        \"arg1\": val1,\n        \"arg2: val2\n    }\n  }\n```\n\n### Applying click-model\n\nTo apply a click model you need to first have an allRank model trained.\nNext, run:\n\n```python allrank/rank_and_click.py --input-model-path \u003cpath_to_the_model_weights_file\u003e --roles \u003ccomma_separated_list_of_ds_roles_to_process e.g. train,valid\u003e --config_file_name allrank/config.json --run_id \u003cthe_name_of_your_experiment\u003e --job_dir \u003cthe_place_to_save_results\u003e``` \n\nThe model will be used to rank all slates from the dataset specified in config. Next - a click model configured in config will be applied and the resulting click-through dataset will be written under ```\u003cjob_dir\u003e/results/\u003crun_id\u003e``` in a libSVM format.\nThe path to the results directory may then be used as an input for another allRank model training.\n\n## Continuous integration\n\nYou should run `scripts/ci.sh` to verify that code passes style guidelines and unit tests.\n\n## Research\n\nThis framework was developed to support the research project [Context-Aware Learning to Rank with Self-Attention](https://arxiv.org/abs/2005.10084). If you use allRank in your research, please cite:\n```\n@article{Pobrotyn2020ContextAwareLT,\n  title={Context-Aware Learning to Rank with Self-Attention},\n  author={Przemyslaw Pobrotyn and Tomasz Bartczak and Mikolaj Synowiec and Radoslaw Bialobrzeski and Jaroslaw Bojar},\n  journal={ArXiv},\n  year={2020},\n  volume={abs/2005.10084}\n}\n```\nAdditionally, if you use the NeuralNDCG loss function, please cite the corresponding work, [NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting](https://arxiv.org/abs/2102.07831):\n```\n@article{Pobrotyn2021NeuralNDCG,\n  title={NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting},\n  author={Przemyslaw Pobrotyn and Radoslaw Bialobrzeski},\n  journal={ArXiv},\n  year={2021},\n  volume={abs/2102.07831}\n}\n```\n\n## License\n\nApache 2 License\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallegro%2Fallrank","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallegro%2Fallrank","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallegro%2Fallrank/lists"}