{"id":25776984,"url":"https://github.com/facebookarchive/linkbench","last_synced_at":"2025-02-27T06:07:05.112Z","repository":{"id":3357769,"uuid":"4403338","full_name":"facebookarchive/linkbench","owner":"facebookarchive","description":"Facebook Graph Benchmark","archived":true,"fork":false,"pushed_at":"2020-02-11T05:12:23.000Z","size":15923,"stargazers_count":415,"open_issues_count":3,"forks_count":142,"subscribers_count":65,"default_branch":"master","last_synced_at":"2024-04-08T00:14:53.664Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/facebookarchive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-05-22T06:58:52.000Z","updated_at":"2024-03-26T14:02:04.000Z","dependencies_parsed_at":"2022-08-26T13:12:22.203Z","dependency_job_id":null,"html_url":"https://github.com/facebookarchive/linkbench","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookarchive%2Flinkbench","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookarchive%2Flinkbench/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookarchive%2Flinkbench/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/facebookarchive%2Flinkbench/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/facebookarchive","download_url":"https://codeload.github.com/facebookarchive/linkbench/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240987435,"owners_count":19889335,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-27T06:01:33.852Z","updated_at":"2025-02-27T06:07:05.105Z","avatar_url":"https://github.com/facebookarchive.png","language":"Java","funding_links":[],"categories":["测试"],"sub_categories":[],"readme":"- - -\n\n**_This project is not actively maintained. Proceed at your own risk!_**\n\n- - -  \n\nLinkBench Overview\n====================\nLinkBench is a database benchmark developed to evaluate database\nperformance for workloads similar to those of Facebook's production MySQL deployment.\nLinkBench is highly configurable and extensible.  It can be reconfigured\nto simulate a variety of workloads and plugins can be written for\nbenchmarking additional database systems.\n\nLinkBench is released under the Apache License, Version 2.0.\n\nBackground\n----------\nOne way of modeling social network data is as a *social graph*, where\nentities or *nodes* such as people, posts, comments and pages are connected\nby *links* which model different relationships between the nodes.  Different\ntypes of links can represent friendship between two users, a user liking another object,\nownership of a post, or any relationship you like.  These nodes and links carry metadata such as\ntheir type, timestamps and version numbers, along with arbitrary *payload data*.\n\nFacebook represents much of its data in this way, with the data stored in MySQL\ndatabases.  The goal of LinkBench is to emulate the social graph database workload\nand provide a realistic benchmark for database performance on social workloads.\nLinkBench's data model is based on the social graph, and LinkBench has the ability to generate\na large synthetic social graph with key properties similar to the real graph.\nThe workload of database operations is based on Facebook's production workload, and is also\ngenerated in such a way that key properties of the workload match the production workload.\n\nLinkBench Architecture\n----------------------\n\u003cpre\u003e\n++====================================++\n||          LinkBench Driver          ||\n++====================================++\n||   +---------------------------+    ||\n||   | Graph      | Workload     |    ||      Open connections  +=======+\n||   | Generator  | Generator    |    ||   /------------------\u003e | Graph |\n||   +---------------------------+    ||  /-------------------\u003e | Store |\n||   |                           |\u0026lt;======---------------------\u003e | Shard |\n||   |   Graph Store Adapter     |\u0026lt;======---------------------\u003e |       |\n||   |   (e.g. MySQL adapter)    |\u0026lt;======---------------------\u003e | e.g.  |\n||   +---------------------------+    ||  \\-------------------\u003e | MySQL |\n||                                    ||   \\------------------\u003e | Server|\n||   ~~~~~~~~~~~~    ~~~~~~~~~~~~     ||                        +=======+\n||   ~~~~~~~~~~~~    ~~~~~~~~~~~~     ||\n||   ~~~~~~~~~~~~    ~~~~~~~~~~~~     ||\n||   ~~~~~~~~~~~~    ~~~~~~~~~~~~     ||\n||     Requester Threads              ||\n++====================================++\n\u003c/pre\u003e\n\nThe main software component of LinkBench is the driver, which acts as the\nclient to the database being benchmarked.\nLinkBench is designed to support benchmarking of any database system that can support\nall of the require graph operations through a *Graph Store Adapter*.\n\nThe LinkBench benchmark typically proceeds in two phases.\n\nThe first is the *load phase*,\nwhere an initial graph is generated using the\n*graph generator* and loaded into the graph store in bulk.\nOn a large benchmark run, this graph might have a billion nodes, and occupy over a terabyte\non disk.  The generated graph is designed to have similar properties to the Facebook\nsocial graph.  For example, the number of links out from each node follows a power-law\ndistribution, where most nodes have at most a few links, but a few nodes have many\nmore links.\n\nThe second is the *request phase*, where the actual benchmarking occurs.  In\nthe request phase, the benchmark driver spawns many request threads, which make\nconcurrent requests to the database.  The *workload generator* is used by each\nrequest thread to generate a series of database operations that mimics the\nFacebook production workload in many aspects.  For example, the mix of\ndifferent varieties of read and write operations is the same, and the\naccess patterns create a similar pattern of hot (frequently access)\nand cold nodes in the graph.  At the end of the request phase LinkBench\nwill report a range of statistics such as latency and throughput.\n\nGetting Started\n===============\nIn this README we'll walk you through compiling LinkBench and running\na MySQL benchmark.\n\nPrerequisites:\n--------------\nThese instructions assume you are using a UNIX-like system such as a Linux distribution\nor Mac OS X.\n\n**Java**: You will need a Java 7+ runtime environment.  LinkBench by default\n      uses the version of Java on your path.  You can override this by setting the\n      JAVA\\_HOME environment variable to the directory of the desired\n      Java runtime version.  You will also need a Java JDK to compile from source.\n\n**Maven**: To build LinkBench, you will need the Apache Maven build tool. If\n    you do not have it already, it is available from http://maven.apache.org .\n\n**MySQL Connector**:  To benchmark MySQL with LinkBench, you need MySQL\n    Connector/J, A version of the MySQL connector is bundled with\n    LinkBench.  If you wish to use a more recent version, replace the\n    mysql jar under lib/.  See http://dev.mysql.com/downloads/connector/j/\n\n**MySQL Server**: To benchmark MySQL you will need a running MySQL\n    server with free disk space.\n\nGetting and Building LinkBench\n----------------------------\nFirst get the source code\n\n    git clone git@github.com:facebook/linkbench.git\n\nThen enter the directory and build LinkBench\n\n    cd linkbench\n    mvn clean package\n\nIn order to skip slower tests (some run quite long), type\n\n    mvn clean package -P fast-test\n\nTo skip all tests\n\n    mvn clean package -DskipTests\n\nIf the build is successful, you should get a message like this at the end of the output:\n\n    BUILD SUCCESSFUL\n    Total time: 3 seconds\n\nIf the build fails while downloading required files, you may need to configure Maven,\nfor example to use a proxy.  Example Maven proxy configuration is shown here:\nhttp://maven.apache.org/guides/mini/guide-proxies.html\n\nNow you can run the LinkBench command line tool:\n\n    ./bin/linkbench\n\nRunning it without arguments will show a brief help message:\n\n    Did not select benchmark mode\n    usage: linkbench [-c \u003cfile\u003e] [-csvstats \u003cfile\u003e] [-csvstream \u003cfile\u003e] [-D\n           \u003cproperty=value\u003e] [-L \u003cfile\u003e] [-l] [-r]\n     -c \u003cfile\u003e                       Linkbench config file\n     -csvstats,--csvstats \u003cfile\u003e     CSV stats output\n     -csvstream,--csvstream \u003cfile\u003e   CSV streaming stats output\n     -D \u003cproperty=value\u003e             Override a config setting\n     -L \u003cfile\u003e                       Log to this file\n     -l                              Execute loading stage of benchmark\n     -r                              Execute request stage of benchmark\n\nRunning a Benchmark with MySQL\n==============================\nIn this section we will document the process of setting\nup a new MySQL database and running a benchmark with LinkBench.\n\nMySQL Setup\n-----------\nWe need to create a new database and tables on the MySQL server.\nWe'll create a new database called `linkdb` and\nthe needed tables to store graph nodes, links and link counts.\nRun the following commands in the MySQL console:\n\n    create database linkdb;\n    use linkdb;\n\n    CREATE TABLE `linktable` (\n      `id1` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `id2` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `link_type` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `visibility` tinyint(3) NOT NULL DEFAULT '0',\n      `data` varchar(255) NOT NULL DEFAULT '',\n      `time` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `version` int(11) unsigned NOT NULL DEFAULT '0',\n      PRIMARY KEY (link_type, `id1`,`id2`),\n      KEY `id1_type` (`id1`,`link_type`,`visibility`,`time`,`id2`,`version`,`data`)\n    ) ENGINE=InnoDB DEFAULT CHARSET=latin1 PARTITION BY key(id1) PARTITIONS 16;\n\n    CREATE TABLE `counttable` (\n      `id` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `link_type` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `count` int(10) unsigned NOT NULL DEFAULT '0',\n      `time` bigint(20) unsigned NOT NULL DEFAULT '0',\n      `version` bigint(20) unsigned NOT NULL DEFAULT '0',\n      PRIMARY KEY (`id`,`link_type`)\n    ) ENGINE=InnoDB DEFAULT CHARSET=latin1;\n\n    CREATE TABLE `nodetable` (\n      `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,\n      `type` int(10) unsigned NOT NULL,\n      `version` bigint(20) unsigned NOT NULL,\n      `time` int(10) unsigned NOT NULL,\n      `data` mediumtext NOT NULL,\n      PRIMARY KEY(`id`)\n    ) ENGINE=InnoDB DEFAULT CHARSET=latin1;\n\nYou may want to set up a special database user account for benchmarking:\n\n    -- Note: replace 'linkbench'@'localhost' with 'linkbench'@'%' to allow remote connections\n    CREATE USER 'linkbench'@'localhost' IDENTIFIED BY 'mypassword';\n    -- Grant all privileges on linkdb to this user\n    GRANT ALL ON linkdb TO 'linkbench'@'localhost'\n\nIf you want to obtain representative benchmark results, we highly\nrecommend that you invest some time configuring and tuning MySQL.\nMySQL performance tuning can be complex and a comprehensive guide\nis beyond the scope of this readme, but here are a few basic guidelines:\n* Read the [Optimization section of the MySQL user manual](http://dev.mysql.com/doc/refman/5.6/en/optimization.html).\n* Make sure you have a sensible size setting for the [InnoDB buffer pool size](http://dev.mysql.com/doc/refman/5.6/en/optimizing-innodb-diskio.html),\n  so as to reduce disk I/O.\n* Table partitioning (as shown above) can eliminate some bottlenecks\n  that occur with LinkBench where the linktable is heavily accessed.\n\nConfiguration Files\n-------------------\nLinkBench requires several configuration files that specify the\nbenchmark setup, the parameters of the graph to be generated, etc.\nBefore benchmarking you will want to make a copy of the example config file:\n\n    cp config/LinkConfigMysql.properties config/MyConfig.properties\n\nOpen MyConfig.properties.  At a minimum you will need to fill in the\nsettings under *MySQL Connection Information* to match the server, user\nand database you set up earlier. E.g.\n\n    # MySQL connection information\n    host = localhost\n    user = linkbench\n    password = your_password\n    port = 3306\n    dbid = linkdb\n\nYou can read through the settings in this file.  There are a lot of settings\nthat control the benchmark itself, and the output of the LinkBench\ncommand link tool.  Notice that MyConfig.properties\nreferences another file in this line:\n\n    workload_file = config/FBWorkload.properties\n\nThis workload file defines how the social\ngraph should be generated and what mix of operations should make\nup the benchmark.  The included workload file has been tuned to\nmatch our production workload in query mix.  If you want to change\nthe scale of the benchmark (the default graph is quite small for\nbenchmarking purposes), you should look at the maxid1 setting.  This\ncontrols the number of nodes in the initial graph created in the load\nphase: increase it to get a larger database.\n\n      # start node id (inclusive)\n      startid1 = 1\n\n      # end node id for initial load (exclusive)\n      # With default config and MySQL/InnoDB, 1M ids ~= 1GB\n      maxid1 = 10000001\n\nLoading Data\n------------\nFirst we need to do an initial load of data using our new config file:\n\n    ./bin/linkbench -c config/MyConfig.properties -l\n\nThis will take a while to load, and you should get frequent progress updates.\nOnce loading is finished you should see a notification like:\n\n    LOAD PHASE COMPLETED.  Loaded 10000000 nodes (Expected 10000000).\n      Loaded 47423071 links (4.74 links per node).  Took 620.4 seconds.\n      Links/second = 76435\n\nAt the end LinkBench reports a range of statistics on load time that are\nof limited interest at this stage.\n\n\nYou can significantly speed up the LinkBench load phase by making these\ntemporary changes in the MySQL command shell before loading:\n\n    alter table linktable drop key `id1_type`;\n    set global innodb_flush_log_at_trx_commit = 2;\n    set global sync_binlog = 0;\n\nAfter loading you should revert the changes:\n\n    set global innodb_flush_log_at_trx_commit = 1;\n    set global sync_binlog = 1;\n    alter table linktable add key `id1_type`\n      (`id1`,`link_type`,`visibility`,`time`,`id2`,`version`,`data`);\n\nRequest Phase\n-------------\nNow you can do some benchmarking.\nRun the request phase using the below command:\n\n    ./bin/linkbench -c config/MyConfig.properties -r\n\nLinkBench will log progress to the console, along with statistics.\nOnce all requests have been sent, or the time limit has elapsed, LinkBench\nwill notify you of completion:\n\n    REQUEST PHASE COMPLETED. 25000000 requests done in 2266 seconds.\n      Requests/second = 11029\n\nYou can also inspect the latency statistics. For example, the following line tells us the mean latency\nfor link range scan operations, along with latency ranges for median (p50), 99th percentile (p99) and\nso on.\n\n    GET_LINKS_LIST count = 12678653  p25 = [0.7,0.8]ms  p50 = [1,2]ms\n                   p75 = [1,2]ms  p95 = [10,11]ms  p99 = [15,16]ms\n                   max = 2064.476ms  mean = 2.427ms\n\nAdvanced LinkBench Command Line Usage\n-------------------------------------\nHere are some further examples of how to use the LinkBench command link utility.\n\nYou can override any properties from the configuration file from the\ncommand line with -D key=value.  For example, this runs the benchmark\nwith a 10 minute warmup before collecting statistics:\n\n    ./bin/linkbench -c config/MyConfig.properties -D warmup_time=600 -r\n\nThis runs the benchmark with more detailed logging, and all output going to the file linkbench.log:\n\n    ./bin/linkbench -c config/MyConfig.properties -D debuglevel=DEBUG -L linkbench.log -r\n\nLinkBench supports output of statistics in csv format for easier analysis.\nThere are two categories of statistic: the final summary and per-thread statistics\noutput periodically through the benchmark.  -csvstats controls the former and -csvstream the latter:\n\n    ./bin/linkbench -c config/MyConfig.properties -csvstats final-stats.csv -csvstreams streaming-stats.csv -r\n\n\nBenchmark Guidelines\n====================\nBenchmarks are often controversial and are challenging to do well.\nHere are some guidelines for avoiding common pitfalls with LinkBench.\n\nDatabase Tuning\n---------------\nTo remove confounding factors in database setup, there are several steps you can take\nto obtain better results:\n\n* Warm up the databases before collecting statistics.  LinkBench has a\n  *warmup_time* setting that sends requests for a period before starting to\n  collect statistics.\n* Run benchmarks for long periods of time (hours rather than minutes)\n    to reduce impact of random variation and to allow the database to\n    reach a steady state.\n* If at all possible, get expert help tuning the database for your\n  hardware and workload.\n* Benchmarks where the database fits mostly or entirely in RAM are interesting\n  but aren't comparable to benchmarks where\n  the database is much larger than RAM.  Typically for MySQL benchmarks our databases\n  are 10-15x larger than the buffer pool.\n* Databases should be benchmarked in comparable configurations.  We\n  always run LinkBench with durable writes (i.e. so that\n  after an operation returns, the data is written to persistent storage and can be\n  recovered in the event of a system crash).\n  Similarly, our LinkBench MySQL implementation provides serializable consistency of\n  operations.  Weaker durability or consistency properties should be\n  disclosed alongside benchmark results.\n\nUnderstanding Performance Profile Under Varying Load\n----------------------------------------------------\nDifferent systems can behave different when heavily or lightly loaded.\nThe default benchmark settings simulate a heavily loaded database,\nwith 100 concurrent request threads each sending requests as quickly as they can.\nSome database systems perform better than others with many concurrent\nclients or heavy load, so performance under heavy load does not\ngive a complete picture of performance.\nTypically databases are not fully loaded all of the time,\nso latency of requests under moderate load is also an important\nmeasure of database performance.\n\nTo get a better understanding of database performance under varying load it\ncan be helpful to:\n* Modify the *requesters* parameter to test database performance with varying\n  numbers of clients.\n* Modify the *requestrate* config setting so that requests are throttled.\n  Request latency vs. throughput curves help with understanding the full\n  performance profile of a database system.\n\nUnderstanding Resource Utilization\n-------------------------\nIf you are doing a benchmark exercise, it is often a good idea to collect\nadditional information about system resource utilization, particularly\nfor CPU and I/O.  This can aid a lot in understanding and comparing\nbenchmark results beyond headline performance numbers.  It is\neasiest to make use of collected data if you can match up timestamps\nto your benchmark logs, so the examples here will append\ntimestamps to each line of output.\n\nvmstat reports useful summary information on CPU and memory:\n\n    vmstat 1 | gawk '{now=strftime(\"%Y-%m-%d %T \"); print now $0}' \u003e linkbench.run.1/vmstat.out\n\niostat reports some useful I/O statistics:\n\n    iostat -d -x 1 |  gawk '{now=strftime(\"%Y-%m-%d %T \"); print now $0}' \u003e linkbench.run.1/iostat.out\n\n\nExtending/Customizing LinkBench\n===============================\nYou can customize LinkBench in several ways.\n\nReconfiguring Workload\n---------------------\nWe have already introduced you to the LinkBench configuration files.\nAll settings in these files are documented and a great deal can be changed\nsimply through these configuration files.  For example:\n\n* You can experiment with read-intensive or write-intensive workloads by\n  modifying the mix of operations.\n* You can alter the mix of hot/cold rows by modifying the shape\n  parameter for ZipfDistribution.  If you set it close to 1, there will be only\n  a few very hot nodes in the database, or if you set if close to 0, accesses\n  will be spread evenly across all nodes.\n\nAdditional Workload Generators\n------------------------------\nIt is possible to further customize the data and workload by providing\nnew implementations of some key classes:\n* ProbabilityDistribution: which can be used to control the distribution of\n    out-edges in the graph, or the access patterns for requests.\n* DataGenerator: which can be used to generate data in different ways for requests.\n\nAdditional Database Systems\n---------------------------\nYou can write plugins to benchmark additional database systems\nsimply by writing a Java class implementing a small set of graph operations.\nAny classes implementing the `com.facebook.LinkBench.LinkStore`\nand `com.facebook.LinkBench.NodeStore` interfaces can be loaded\nthrough the *linkstore* and *nodestore* configuration file keys.\n\nThere are several steps you will have to go through to add a\nnew plugin .\nFirst you need to choose you will represent LinkBench\nnodes and links.  Several factors play a role in the design, but\nspeed of range scans and atomicity of updates are particularly important.\nThe MySQL schema from earlier in this README serves as a reference\nimplementation.\n\nNext you need to create a new Java class, such as `public class MyStore\nextends GraphStore`, and implement all of the required methods of\n`LinkStore` and `NodeStore`.  Two reference implementations are provided:\n`LinkStoreMysql`, a fully-fledged implementation,  and `MemoryLinkStore`,\na toy in-memory implementation.\n\nLinkBench provides some tests to validate your implementation that you\ncan use during development.  If you extend any of the test classes\n`LinkStoreTestBase`, `NodeStoreTestBase` and `GraphStoreTestBase` with\nthe required methods that set up your database, then a range of tests\nwill be run against it.  These tests are sanity checks rather than\ncomprehensive verification of your implementation.  In particular,\nthey do not try to verify the atomicity, consistency or durability\nproperties of the implementation.\n\nDatabase-specific tests are not run by default.  You can enable them\nwith Maven profiles.  For example, to run the MySQL tests you can\nrun:\n\n    mvn test -P mysql-test\n\nThe MySQL related unit tests are run against a test database that needs\nsetting up before running the unit tests. The default settings for this\ntest database are hardcoded in src/test/java/com/facebook/LinkBench/MySqlTestConfig.java.\nThe default settings uses localhost:3306 to connect to the database and\nuses username \"linkbench\" and password \"linkbench\".  The unit test code \ncreates all the required tables, so the developer needs to setup a \nMySql database called \"linkbench_unittestdb\" to which the linkbench user \nhas permissions to create and drop tables.\n\n**If you implement a plugin for a new database, please consider contributing\nit back to the main LinkBench distribution with a pull request.**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookarchive%2Flinkbench","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffacebookarchive%2Flinkbench","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffacebookarchive%2Flinkbench/lists"}