{"id":17891406,"url":"https://github.com/amanpriyanshu/linearcosine","last_synced_at":"2025-10-07T01:52:12.141Z","repository":{"id":259078246,"uuid":"875463565","full_name":"AmanPriyanshu/LinearCosine","owner":"AmanPriyanshu","description":"LinearCosine: Adding beats multiplying for lower-precision efficient cosine similarity","archived":false,"fork":false,"pushed_at":"2024-10-21T16:17:28.000Z","size":1694,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-03T00:04:26.454Z","etag":null,"topics":["algorithms","artificial-intelligence","benchmarking","computation","computer-vision","cosine-similarity","cpp","deep-learning","energy-efficiency","floating-point","linear-algebra","low-precision","machine-learning","matrix-multiplication","neural-networks","nlp","optimization","performance-optimization","quantization"],"latest_commit_sha":null,"homepage":"https://amanpriyanshu.github.io/LinearCosine/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AmanPriyanshu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-20T03:39:35.000Z","updated_at":"2024-10-21T16:22:28.000Z","dependencies_parsed_at":"2024-10-22T11:17:31.496Z","dependency_job_id":null,"html_url":"https://github.com/AmanPriyanshu/LinearCosine","commit_stats":null,"previous_names":["amanpriyanshu/linearcosine"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmanPriyanshu%2FLinearCosine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmanPriyanshu%2FLinearCosine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmanPriyanshu%2FLinearCosine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AmanPriyanshu%2FLinearCosine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AmanPriyanshu","download_url":"https://codeload.github.com/AmanPriyanshu/LinearCosine/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246928861,"owners_count":20856362,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","artificial-intelligence","benchmarking","computation","computer-vision","cosine-similarity","cpp","deep-learning","energy-efficiency","floating-point","linear-algebra","low-precision","machine-learning","matrix-multiplication","neural-networks","nlp","optimization","performance-optimization","quantization"],"created_at":"2024-10-28T14:17:27.302Z","updated_at":"2025-10-07T01:52:07.106Z","avatar_url":"https://github.com/AmanPriyanshu.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LinearCosine\n## Adding beats multiplying for lower-precision efficient cosine similarity\n\nThis repository contains a simple C++ benchmarking experiment inspired by the paper [\"Addition is All You Need for Energy-efficient Language Models\" by Hongyin Luo and Wei Sun (2024)](https://arxiv.org/abs/2410.00907). The experiment explores the performance of the proposed Linear-complexity Multiplication (L-Mul) algorithm compared to standard low-precision multiplication in the context of cosine similarity calculations.\n\n## TLDR: Technical Explanation\n\nThe L-Mul algorithm proposes an efficient approximation of floating-point multiplication using primarily integer addition operations. Here's a quick breakdown:\n\n### Standard Floating-Point Multiplication\n\nThe traditional method multiplies two floating-point numbers as follows:\n\n$$\\text{Mul}(x,y) = (1 + x_m) \\cdot 2^{x_e} \\cdot (1 + y_m) \\cdot 2^{y_e} = (1 + x_m + y_m + x_m \\cdot y_m) \\cdot 2^{x_e + y_e}$$\n\nWhere $x_m$ and $y_m$ are mantissas, and $x_e$ and $y_e$ are exponents.\n\n### L-Mul Algorithm\n\nThe L-Mul method approximates this multiplication:\n\n$$\\mathcal{L}\\text{-Mul}(x,y) = (1 + x_m + y_m + 2^{-l(m)}) \\cdot 2^{x_e + y_e}$$\n\nWhere $l(m)$ is defined as:\n\n$$l(m) = \\begin{cases} \nm \u0026 \\text{if } m \\leq 3, \\\\ \n3 \u0026 \\text{if } m = 4, \\\\ \n4 \u0026 \\text{if } m \u003e 4. \n\\end{cases}$$\n\nHere, $m$ represents the number of mantissa bits.\n\n### Key Differences\n\n- L-Mul replaces the $x_m \\cdot y_m$ term with a constant $2^{-l(m)}$.\n- This substitution allows the operation to be performed primarily with integer addition.\n- The $l(m)$ function adjusts the precision based on the number of mantissa bits used.\n\nThis approximation significantly reduces computational complexity and energy consumption while maintaining competitive accuracy, especially for lower-precision operations common in many AI tasks.\n\n## Background\n\nLarge neural networks, especially language models, consume significant computational resources and energy. The paper proposes L-Mul, an algorithm that approximates floating-point multiplication using integer addition operations. This approach aims to reduce energy consumption and potentially increase computational efficiency. While implementing the full L-Mul algorithm in neural networks was beyond the scope of my experiment, I decided to focus on optimizing cosine similarity calculations, which are crucial for many information retrieval and RAG (Retrieval-Augmented Generation) applications that are already widespread in the AI industry.\n\n## Experiment Overview\n\nOur benchmarking experiment implements L-Mul and standard multiplication for cosine similarity calculations with varying mantissa bit precisions. We compare the performance in terms of computation time and accuracy (measured by Mean Squared Error) for three scenarios:\n\n1. 1D to 1D vector cosine similarity\n2. 1D to 2D vector cosine similarity\n3. 2D to 2D matrix cosine similarity\n\n## Repository Structure\n\n```\nREADME.md\nbenchmark/\n├── cosine_1d_to_1d.cpp\n├── cosine_1d_to_2d.cpp\n├── cosine_2d_to_2d.cpp\n└── README.md\n```\n\n## Key Findings\n\n- L-Mul consistently achieves time reductions compared to standard multiplication across all scenarios.\n- The average time reduction for 1D to 1D cosine similarity is approximately 24.55%.\n- For 1D to 2D and 2D to 2D scenarios, time reductions range from about 16.8% to 23.12%.\n- L-Mul maintains acceptable levels of accuracy (measured by MSE) for most practical applications.\n\n## Limitations and Notes\n\n- This experiment focuses on raw C++ implementations and may not directly reflect hardware-level efficiency.\n- Time efficiency gains observed here don't necessarily equate to compute or energy efficiency in real-world scenarios.\n- The implementation is a simplified version of the concepts presented in the original paper and is intended for educational purposes.\n\n## Running the Benchmarks\n\nTo run the benchmarks, compile and execute each C++ file separately. For example:\n\n```bash\ng++ -O3 -std=c++11 cosine_1d_to_1d.cpp -o cosine_1d_to_1d\n./cosine_1d_to_1d\n```\n\nRepeat for the other benchmark files.\n\n## Acknowledgments\n\nThis experiment is based on the concepts presented in:\n\n```\n@misc{luo2024additionneedenergyefficientlanguage,\n      title={Addition is All You Need for Energy-efficient Language Models}, \n      author={Hongyin Luo and Wei Sun},\n      year={2024},\n      eprint={2410.00907},\n      archivePrefix={arXiv},\n      primaryClass={cs.CL},\n      url={https://arxiv.org/abs/2410.00907}, \n}\n```\n\n## Disclaimer\n\nThis is an educational project and a simplified implementation of the concepts presented in the original paper. It is not intended for production use and may not accurately represent the full potential of the L-Mul algorithm in real-world scenarios.\n\n## Future Work\n\nPotential areas for future exploration include:\n- Implementing L-Mul in actual neural network architectures\n- Exploring hardware-level implementations for more accurate efficiency measurements\n- Investigating the impact of L-Mul on model accuracy for various NLP tasks\n\nFeel free to contribute, suggest improvements, or use this as a starting point for further research!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famanpriyanshu%2Flinearcosine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Famanpriyanshu%2Flinearcosine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Famanpriyanshu%2Flinearcosine/lists"}