{"id":13785795,"url":"https://github.com/xqgex/NUMA_Black-Box","last_synced_at":"2025-05-11T21:31:50.735Z","repository":{"id":69317272,"uuid":"102409237","full_name":"xqgex/NUMA_Black-Box","owner":"xqgex","description":"Black-box Concurrent Data Structures for NUMA Architectures","archived":false,"fork":false,"pushed_at":"2017-09-04T23:04:18.000Z","size":19,"stargazers_count":7,"open_issues_count":1,"forks_count":9,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-11-17T22:35:43.245Z","etag":null,"topics":["brown-university","c","numa","tau","tel-aviv-university"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xqgex.png","metadata":{"files":{"readme":"readme.txt","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-09-04T22:55:43.000Z","updated_at":"2022-11-23T17:12:41.000Z","dependencies_parsed_at":"2023-06-11T23:00:36.456Z","dependency_job_id":null,"html_url":"https://github.com/xqgex/NUMA_Black-Box","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xqgex%2FNUMA_Black-Box","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xqgex%2FNUMA_Black-Box/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xqgex%2FNUMA_Black-Box/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xqgex%2FNUMA_Black-Box/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xqgex","download_url":"https://codeload.github.com/xqgex/NUMA_Black-Box/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253638938,"owners_count":21940434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["brown-university","c","numa","tau","tel-aviv-university"],"created_at":"2024-08-03T19:01:04.549Z","updated_at":"2025-05-11T21:31:50.290Z","avatar_url":"https://github.com/xqgex.png","language":"C","funding_links":[],"categories":["NUMA-aware data structures"],"sub_categories":[],"readme":"Black-box Concurrent Data Structures for NUMA Architectures\n\nBased on: http://cs.brown.edu/~irina/papers/asplos2017-final.pdf\nBy: Irina Calciu, Siddhartha Sen, Mahesh Balakrishnan, Marcos K. Aguilera\n\n             Node #1                              Node #2             \n╔═══════╦═══════╦═══════╦═══════╗    ╔═══════╦═══════╦═══════╦═══════╗\n║ Core  ║ Core  ║ Core  ║ Core  ┣━━━━┫ Core  ║ Core  ║ Core  ║ Core  ║\n╟───────╫───────╫───────╫───────╊━━━━╉───────╫───────╫───────╫───────╢\n║ Cache ║ Cache ║ Cache ║ Cache ┣━━━━┫ Cache ║ Cache ║ Cache ║ Cache ║\n╟───────╨───────╨───────╨───────╢    ╟───────╨───────╨───────╨───────╢\n║             Cache             ║    ║             Cache             ║\n╚═════════════┳━┳━┳═════════════╝    ╚═════════════┳━┳━┳═════════════╝\n              ┃ ┃ ┃                                ┃ ┃ ┃              \n┌─────────────┺━┻━┹─────────────┐    ┌─────────────┺━┻━┹─────────────┐\n│            Memory             │    │            Memory             │\n└───────────────────────────────┘    └───────────────────────────────┘\n\n/************************************************************/\n/************\tMy implementation\t*********************/\n/************************************************************/\nThis is the general scheme of my implementation:\n                                              ┌────────────────────────┐\n                                              │      libgenericDS      │\n                                              ├────────────────────────┤\n                                              │ genericDataStructure.h │\n                                              │ genericDataStructure.c │\n                                              └────────────────────────┘\n                                                          ▲             \n                                                          ║             \n                               Shared Memory              ▼             \n┌──────────────────────┐    ┌──────────────────┐    ┌────────────┐      \n│ libgenerateCommands  │═══▶│ gds_commands.bin │═══▶│ libNUMAbb  │      \n├──────────────────────┤    └──────────────────┘    ├────────────┤      \n│  generateCommands.c  │                            │  NUMAbb.h  │      \n│                      │                            │  NUMAbb.c  │      \n└──────────────────────┘                            └────────────┘      \n            ▲                                             ▲             \n            ║                  ┌────────────┐             ║             \n            ╚══════════════════│ myprogram  │═════════════╝             \n                               ├────────────┤                           \n                               │   main.c   │                           \n                               └────────────┘                           \nThe code was designed to work on unix system and was written in C.\nIn order to run the program there is only one file that need to be run,\nBut the files are still acting like a separate libraries.\nThere is more work that need to be done, Feel free to fork my project.\n\nRun the program:\n\tBuild and run the program by running the command: \"sh makefile.sh\"\n\nLimitation from the current implementation:\n\t1) The mmap size is a limitaion on the amount of commands.\n\t2) Variable MAX_CORES limit the program to work with up-to 512 cores.\n\t3) Maximum 'op' and 'args' size is 1024 chars each.\n\nProblems:\n\tUnfortunately, My code isn't perfect yet, \n\nMore upgrades that can be done:\n\tSection 5.2\n\t\tTo avoid small inefficient batches, the combiner in NR waits\n\t\tif the batch size is smaller than a parameter MIN BATCH. Rather\n\t\tthan idle waiting, the combiner refreshes the local replica from\n\t\tthe log, though it might need to refresh again after finally\n\t\tadding the batch to the log\n\tSection 5.5 - Better readers-writer lock\n\t\tThe distributed readers-writer lock uses a per-reader\n\t\tlock to reduce reader overhead; the writer must acquire the\n\t\tlocks from all readers. We modify this algorithm to reduce\n\t\twriter overhead as well, by adding an additional writer lock.\n\t\tTo enter the critical section, the writer must acquire the\n\t\twriter lock and wait for all the readers locks to be released,\n\t\twithout acquiring them; to exit, it releases its lock. A reader\n\t\twaits if the writer lock is taken, then acquires its local lock,\n\t\tand checks the writer lock again; if this lock is taken, the\n\t\treader releases its local lock and restarts; otherwise, it enters\n\t\tthe critical section; to exit, it releases the local lock. With\n\t\tthis scheme, the writer and readers incur just one atomic\n\t\twrite each on distinct cache lines to enter the critical sec-\n\t\ttion. Readers may starve if writers keep coming, but this is\n\t\tunlikely with NR , as often only one thread wishes to be a\n\t\twriter at a time (the combiner) and that thread has signif-\n\t\ticant work outside the critical section.\n\tSection 5.7\n\t\tPadded the data and aligned the cache to avoid false sharing.\n\tGeneral\n\t\tWhat prevent two combiners (on separate nodes) to write the log at the same time?\n\t\tI think there is need to create something that will prevent it.\n\nBuild errors:\n\tIn case of compilation errors, Make sure you have all the dependency packages on your computer,\n\tUse \"apt-get install\" / \"yum install\" / \"dnf install\" To install: (On Ubunto devel =\u003e dev)\n\t\t1) libnuma-devel\tDevelopment files for libnuma\n\t\t2) numactl\t\tLibrary for tuning for Non Uniform Memory Access machines\n\t\t3) numactl-libs\t\tlibnuma libraries\n\t\t4) numactl-devel\tDevelopment package for building Applications that use numa\n\t\t5) glibc-devel\t\tObject files for development using standard C libraries.\n\t\t6) glibc-static\t\tC library static libraries for -static linking.\n\t\t7) libstdc++-devel\tHeader files and libraries for C++ development\n\t\t8) libstdc++-static\tStatic libraries for the GNU standard C++ library\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxqgex%2FNUMA_Black-Box","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxqgex%2FNUMA_Black-Box","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxqgex%2FNUMA_Black-Box/lists"}