{"id":16271312,"url":"https://github.com/ikergarcia1996/matrix-benchmark","last_synced_at":"2025-10-05T03:31:24.543Z","repository":{"id":133848203,"uuid":"297667387","full_name":"ikergarcia1996/Matrix-Benchmark","owner":"ikergarcia1996","description":"A cupy (GPU) / numpy benchmark to measure how fast different hardware can perform matrix operations.","archived":false,"fork":false,"pushed_at":"2021-10-04T12:28:57.000Z","size":64,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-01-10T23:45:29.611Z","etag":null,"topics":["benchmark","cuda","cupy","embedding","gpu","matrix","numpy","python","word-embeddings"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ikergarcia1996.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-22T14:02:58.000Z","updated_at":"2024-12-06T19:54:59.000Z","dependencies_parsed_at":null,"dependency_job_id":"4eff8642-4cce-4260-a7be-7f7fe048d90f","html_url":"https://github.com/ikergarcia1996/Matrix-Benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikergarcia1996%2FMatrix-Benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikergarcia1996%2FMatrix-Benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikergarcia1996%2FMatrix-Benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ikergarcia1996%2FMatrix-Benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ikergarcia1996","download_url":"https://codeload.github.com/ikergarcia1996/Matrix-Benchmark/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235360760,"owners_count":18977566,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","cuda","cupy","embedding","gpu","matrix","numpy","python","word-embeddings"],"created_at":"2024-10-10T18:13:15.830Z","updated_at":"2025-10-05T03:31:18.788Z","avatar_url":"https://github.com/ikergarcia1996.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Matrix Benchmark\n\nA cupy (GPU) / numpy (Numpy) benchmark to measure how fast different hardware can perform matrix operations. The benchmark tests operations commonly used in the word embedding research field. \n\n## Tests\n* Matrix dot\n* Squared Distance\n* Euclidean distance\n* K-nearest neighbours (dot)\n* K-nearest neighbours (euclidean distance)\n\n## Requeriments\n* python3\n* numpy\n* cupy (GPU support)\n* tqdm \n\n## Usage\n```\npython3 run_benchmark.py\n```\nYou can test different batch sizes with the --gpy_batch_sizes parameter\n```\npython3 run_benchmark.py --gpu_batch_sizes 100 500 1000 2000\n```\n\nYou can test different matrix sizes with the --matrix_size parameter\n\n```\npython3 run_benchmark.py --gpu_batch_sizes 100 500 1000 2000 --matrix_size 10000\n```\n\nTo run the kkn / euclian distance / squared distance benchmarks use the --full_benchmark flag\n\n```\npython3 run_benchmark.py --full_benchmark\n```\n\nYou can run the benchmark in FP16, Fp32 or FP64 (FP16 not supported for knn in current cupy version)\n\n```\npython3 run_benchmark.py --fp16\npython3 run_benchmark.py --fp32\npython3 run_benchmark.py --fp64\n```\n\n## Sample Output\n```\n---\u003e Running benchmark \u003c---\nDevice: GeForce RTX 2080 SUPER. FP32. Matrix size: 10000 x 300\n\nRunning dot task. Batch size: 1000. fp32. Time: 0.3407488663991292 seconds.\nRunning squared_distance task. Batch size: 1000. fp32. Time: 3.1240174770355225 seconds.\nRunning euclidean_distance task. Batch size: 1000. fp32. Time: 3.0890189011891684 seconds.\nRunning knn_dot task. Batch size: 1000. fp32. Time: 1.4538000424702961 seconds.\nRunning knn_euclidean_distance task. Batch size: 1000. fp32. Time: OUT OF MEMORY\n```\n\n## Benchmarks\n\n![alt text](Results.png \"Benchmarks\")\n\nTo reproduce results:\n```\npython3 run_benchmark.py --gpu_batch_sizes 2000 --matrix_size 50000 --fp 16\npython3 run_benchmark.py --gpu_batch_sizes 2000 --matrix_size 50000 --fp 32\npython3 run_benchmark.py --gpu_batch_sizes 2000 --matrix_size 50000 --fp 64\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikergarcia1996%2Fmatrix-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fikergarcia1996%2Fmatrix-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fikergarcia1996%2Fmatrix-benchmark/lists"}