{"id":13642169,"url":"https://github.com/ucb-bar/ccbench","last_synced_at":"2025-04-13T08:42:59.692Z","repository":{"id":14476207,"uuid":"17188454","full_name":"ucb-bar/ccbench","owner":"ucb-bar","description":"Memory System Microbenchmarks","archived":false,"fork":false,"pushed_at":"2023-02-09T05:09:46.000Z","size":3573,"stargazers_count":62,"open_issues_count":0,"forks_count":24,"subscribers_count":41,"default_branch":"master","last_synced_at":"2025-03-27T00:22:14.619Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ucb-bar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2014-02-25T21:16:15.000Z","updated_at":"2024-12-10T06:35:04.000Z","dependencies_parsed_at":"2024-01-14T09:11:13.018Z","dependency_job_id":"0f6e2b25-1f26-4b03-8341-9b9cb0ecc6d6","html_url":"https://github.com/ucb-bar/ccbench","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucb-bar%2Fccbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucb-bar%2Fccbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucb-bar%2Fccbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucb-bar%2Fccbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ucb-bar","download_url":"https://codeload.github.com/ucb-bar/ccbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248687565,"owners_count":21145736,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:28.143Z","updated_at":"2025-04-13T08:42:59.670Z","avatar_url":"https://github.com/ucb-bar.png","language":"C","readme":"##############################################################################\n* Christopher Celio\n* celio@eecs.berkeley.edu\n* UC Berkeley, Parlab\n* 2010 Nov\n##############################################################################\n\n\nThis repository contains a collection of small micro-kernels to stress-test\nmemory systems and processors. See **./docs** for more in-depth material on\nthe theory behind these micro-benchmarks.\n\nThe [github wiki](https://github.com/ucb-bar/ccbench/wiki/_pages) shows some of\nthe latest results on some interesting machines (examples include the RISC-V\nRocket processor and the ARM Cortex A9).\n\n------------------------------------------------------------------------------\n## Quick Way to Run Tests\n------------------------------------------------------------------------------\n\nSee what happens! (works best if matplotlib is installed)\n\n    $ ./runall.sh\n\nThe \"runall.sh\" script is the preferred way to compile benchmarks, run the\ntests, collect the data, and plot the results. \n \nYou will need matplotlib installed to graph results. The 'open' command is also\nused, which only works on OSX (comment it out otherwise). \n \nSet the desired tests to use through the $TEST variable in the script. If\n$RUN_LOCAL is set to 1, the tests will run on the local machine.  Otherwise,\nyou can provide the IP addresses in $remote_hosts to run the tests remotely\n(set up loginless login for best results).  You will also need to get the\ncorresponding $architecture(s) variable to specify the target ISA.\n\n\n------------------------------------------------------------------------------\n## More Controlled Way to Run Tests\n------------------------------------------------------------------------------\n\nHow to Run the \"caches\" benchmark: \n\n    $ cd $CCBENCH/caches\n    $ make clean; make\n    $ ./run_test.py -i large\n    $ open plots/caches.pdf\n\nThis runs the \"caches\" benchmark using the \"large\" input set (see input.txt for\nother set types). Data is written to the ./report/report.txt file (actually, a\ngenerated report file based on the current timestamp is used). If matplotlib is\ninstalled, ./run_test.py will also plot the results to a *.pdf file.\n\n\nIf the data is collected on a machine without matplotlib, you can plot the data\nby copying the report.txt file to a machine with matplotlib, and then running\n./run_test.py using the \"-n\" flag (\"no-run\"). \n\n    laptop$ cd $CCBENCH/caches\n    laptop$ scp $LINUX_BOX:~/ccbench/caches/report/report.txt ./report/report.txt\n    labtop$ ./run_test.py -n\n    laptop$ open plots/caches.pdf\n\nYou can feed it a specific report file using \"-r my_report_name.txt\". \n\nUse the \"-h\" flag to ./run_test.py to learn about some of its other options,\nwhich includes specialized annotations for some processors. \n                                                             \n------------------------------------------------------------------------------\n## Building and Executing ccbench on non-x86 ISAs\n------------------------------------------------------------------------------\n\nThe ccbench suite currently supports benchmarking processors (or emulators)\nrunning the following ISAs: x86, ARM, Tilera's TILE64, and UCB's RISC-V. \n\nThe easiest method is to use the \"runall.sh\" top-level script and specify the\nmachine under the $architecture(s) variable (x86, arm, tile64, riscv). This\ndrives the Makefile to compile the benchmark as appropriate and supplies the\nrun_test.py script with the appropriate \"-a\" flag to invoke the proper target\nmachine. \n\nTo add your own, new architecture (or to specialize your compiler/invocation\nsettings), make of a copy of one of the directories in $CCBENCH/arch and\nchange as needed. For example, you may want to use a different compiler with\nits own set of compiler flags, even when running on a particular x86 machine.\nSecond, modify $CCBENCH/common/Makefile.tests to invoke your new architecture's\nMakefile fragment. No modifications are required to the python run_test.py\nscripts.\n\nBy default, a \"generic\" architecture setting is used in which gcc is called to\ncompile the benchmark.  The benchmark is then executed directly on the machine.\nThis will work for both x86 and ARM. However, to get more specific compiler\nflags (as well as for targets that require cross-compiling), it is recommended\nthat you specify exactly the desired target architecture.\n  \n------------------------------------------------------------------------------\n## Running Tests Remotely\n------------------------------------------------------------------------------\n\nAn expected use-case is wanting to easily benchmark a zoo of processors that\nreside remotely. Typically, these processors will not have matplotlib\ninstalled. Using \"runall.sh\", one can easily specify a list of IP addresses,\nprocessor names, and architecture types of each of these target machines. \n\nYou will need to copy the ccbench directory on each remote host machine and\npoint to this directory using the $HOST_CC_DIR variable in \"runall.sh\".  Python\nis required, but matplotlib is not.\n\nThe \"runall.sh\" script can be run on your personal machine, which then ssh's\ninto the remote host machine, compiles the benchmark, and invokes the benchmark\non the target processor (either the same as the remote host machine, or could\nbe an attached accelerator, etc.).  The resulting data is dumped to a report\nfile, which is then copied back to your local machine. Then, the data can be\ngraphed locally.\n\nTo make life easier, it is recommended that you have passwordless login.\n\n------------------------------------------------------------------------------\n## Important Files\n------------------------------------------------------------------------------\n\n  - run_tests.py        - main CLI program to run and plot tests. Python \n                            code that invokes C code through bash. Found inside\n                            each micro-benchmark directory, and customized to\n                            plot data for the given micro-benchmark. \n  - common/             - Contains code shared by all tests.\n                            Includes a pthreads barrier implementation for\n                            platforms that do not natively support\n                            pthreads_barrier_t (i.e., OS X), and an abstracted\n                            clock interface.\n  - common/Makefile.tests - Main makefile used by all ukernels.\n  - reports/report.txt  - Auto-generated by run_tests.py. Stores the\n                            results of all of the runs from its latest\n                            invocation (actual name is auto-generated and \n                            involves the current timestamp).\n  - plots/              - Contains plots generated by run_tests.py\n  - input.txt           - Contains input sets used by run_tests.py\n\n \n------------------------------------------------------------------------------\n## Current Micro-benchmarks And What They Measure\n------------------------------------------------------------------------------\n\n  - caches       - cache sizes, access latencies (pointer chase)\n  - cache2cache  - cache-to-cache latency, bandwidth (ping pong arrays \n                    between threads)\n  - band_req     - number of outstanding requests (pointer chase with \n                    multiple streams).\n  - band_req_mc  - machine total bandwidth (pointer chase with \n                    multiple streams, with multiple threads)\n  - strided      - strided memory acccesses: cache sizes, access latencies. \n                    Mostly ineffectual due to prefetching.\n  - peakflops    - Prints out max flops of the machine. Does NOT plot \n                    anything (and so doesn't fit within the normal flow \n                    of runall.sh, etc.). Saves results to results.txt file.\n  - incluexclu   - figure out if a LLC is inclusive (one thread runs out of \n                    L1, the other thread attempts to blow out the LLC). \n                    (Very experimental, not recommended for use).\n  - mem_interleaving- preliminary attempt to measure best interleaving of \n                    multiple threads with different memory distances between \n                    threads (very experimental, not recommended for use).\n\n  \n------------------------------------------------------------------------------\n## Running benchmarks on the RISC-V emulator\n------------------------------------------------------------------------------\n\nBy setting the \"architecture\" variable as RISC-V (e.g., runall.sh), ccbench\nwill run on the RISC-V processor \"emulator\" binary.*\n\n**Requirements:**\n\n   - Your RISC-V C++ emulator binary must be named \"emulator\".\n   - The \"emulator\" binary must be in your path.\n   - The \"dramsim2_ini\" directory should be located in the same directory as\n      the benchmark binary (dramsim2 is used to simulate the off-chip memory).\n      This means a copy must exist in $CCBENCH/caches, $$CCBENCH/band_req, etc.\n      Hopefully this restriction will be addressed at a later date.\n   - patience. You CANNOT run RISC-V emulator using the same input sets you use\n      with the other processors. We're talking a ~10,000x difference in run-time.\n\n\n*Unless you specify the \"proc\" is \"spike\", in which case the RISC-V ISA\nsimulator spike will be used.  It is recommended that you begin all tests with\n\"spike\" first, as it is ~1000x faster than \"emulator\" and can suss out any bugs\nor issues that may arise. \n\n\n------------------------------------------------------------------------------\n## A note about RISC-V support\n------------------------------------------------------------------------------\n \nCurrently, this suite provides no support for multi-thread RISC-V operations.\nFor example, the barrier code is completely ifdef'ed out. Currently, ccbench\nruns on top of the RISC-V proxy kernel (pk).\n\n \n------------------------------------------------------------------------------\n## LICENSE\n------------------------------------------------------------------------------\n \nSee LICENSE for details. \n\n\n------------------------------------------------------------------------------\n## Additional Information \n------------------------------------------------------------------------------\n\nBuyer beware. Feel free to provide feedback as well as contribution.\n\nThe CS267 report provided in $CCBENCH/docs is out-of-date, so do not completely\ntrust the graphs/results provided in it (but the theory behind the benchmarks\nis still relevant). \n\n\n------------------------------------------------------------------------------\n## TODO\n------------------------------------------------------------------------------\n\n  - make easier to use, extend\n  - improve RISC-V code performance (currently too many instructions in inner\n        loops, etc.).\n  - improve RISC-V support (multi-thread, dramsim2 installation, etc.)\n  - modify peakflops to excercise FMA unit\n  - modify peakflops to better fit with other benchmarks, more capable of\n        compiling to all architectures\n  - serial flag for c2c bandwidth (see if matchs intel's)\n  - make c2c pick between lat and band (or run and report both numbers?)\n  - different strides for c2c\n  - more work on mem_interleaving\n  - measure cache2cache with MSR prefetching turned off\n\n","funding_links":[],"categories":["C"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fucb-bar%2Fccbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fucb-bar%2Fccbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fucb-bar%2Fccbench/lists"}