{"id":26096372,"url":"https://github.com/stuttter/ludicrousdb","last_synced_at":"2025-05-15T15:01:50.847Z","repository":{"id":22925803,"uuid":"26274823","full_name":"stuttter/ludicrousdb","owner":"stuttter","description":"LudicrousDB is an advanced database interface for WordPress that supports replication, failover, load balancing, \u0026 partitioning","archived":false,"fork":false,"pushed_at":"2024-06-23T03:45:07.000Z","size":246,"stargazers_count":529,"open_issues_count":19,"forks_count":77,"subscribers_count":30,"default_branch":"master","last_synced_at":"2025-05-08T06:05:28.750Z","etag":null,"topics":["database","master-slave","read-write","wordpress"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stuttter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-11-06T14:59:39.000Z","updated_at":"2025-05-05T17:41:37.000Z","dependencies_parsed_at":"2023-02-12T18:45:47.112Z","dependency_job_id":"3e5225d4-020a-4469-83e9-ad4cc9ba84e8","html_url":"https://github.com/stuttter/ludicrousdb","commit_stats":{"total_commits":108,"total_committers":15,"mean_commits":7.2,"dds":0.5092592592592593,"last_synced_commit":"145e3b5e3efe7e7ed7434037ac4c5f39045a520d"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stuttter%2Fludicrousdb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stuttter%2Fludicrousdb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stuttter%2Fludicrousdb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stuttter%2Fludicrousdb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stuttter","download_url":"https://codeload.github.com/stuttter/ludicrousdb/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254364264,"owners_count":22058877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["database","master-slave","read-write","wordpress"],"created_at":"2025-03-09T14:45:49.803Z","updated_at":"2025-05-15T15:01:50.780Z","avatar_url":"https://github.com/stuttter.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LudicrousDB\n\nLudicrousDB is an advanced database interface for WordPress that supports replication, fail-over, load balancing, and partitioning, based on Automattic's HyperDB drop-in.\n\n## 0. Installation\n\n### Files\n\nCopy the main `ludicrousdb` plugin folder \u0026 its contents to either:\n\n* `wp-content/plugins/ludicrousdb/`\n* `wp-content/mu-plugins/ludicrousdb/`\n\nIt does not matter which one; LudicrousDB will figure it out. The folder name should be exactly `ludicrousdb`. Be careful when you do \"Download ZIP\" from github and unzip.\n\n### Drop-ins\n\nWordPress supports a few \"drop-in\" style plugins, used for advanced overriding of a few specific pieces of functionality.\n\nLudicrousDB includes 3 basic database drop-ins:\n\n* `db.php` \u003c-\u003e `wp-content/db.php` - Bootstrap for replacement `$wpdb` object\n* `db-error.php` \u003c-\u003e `wp-content/db-error.php` - Endpoint for fatal database error output to users\n* `db-config.php` \u003c-\u003e `ABSPATH/db-config.php` - For configuring your database environment\n\nYou'll probably want to copy these files to their respective locations, and modify them once you're comfortable with what they do and how they work.\n\n## 1. Configuration\n\nLudicrousDB can manage connections to a large number of databases. Queries are distributed to appropriate servers by mapping table names to datasets.\n\nA dataset is defined as a group of tables that are located in the same database. There may be similarly-named databases containing different tables on different servers. There may also be many replicas of a database on different servers. The term \"dataset\" removes any ambiguity. Consider a dataset as a group of tables that can be mirrored on many servers.\n\nConfiguring LudicrousDB involves defining databases and datasets. Defining a database involves specifying the server connection details, the dataset it contains, and its capabilities and priorities for reading and writing. Defining a dataset involves specifying its exact table names or registering one or more callback functions that translate table names to datasets.\n\n### Sample Configuration 1: Default Server\n\nThis is the most basic way to add a server to LudicrousDB using only the required parameters: host, user, password, name. This adds the DB defined in wp-config.php as a read/write server for the 'global' dataset. (Every table is in 'global' by default.)\n\n```php\n$wpdb-\u003eadd_database( array(\n    'host'     =\u003e DB_HOST,     // If port is other than 3306, use host:port.\n    'user'     =\u003e DB_USER,\n    'password' =\u003e DB_PASSWORD,\n    'name'     =\u003e DB_NAME,\n) );\n```\n\nThis adds the same server again, only this time it is configured as a replica. The last three parameters are set to the defaults but are shown for clarity.\n\n```php\n$wpdb-\u003eadd_database( array(\n    'host'     =\u003e DB_HOST,     // If port is other than 3306, use host:port.\n    'user'     =\u003e DB_USER,\n    'password' =\u003e DB_PASSWORD,\n    'name'     =\u003e DB_NAME,\n    'write'    =\u003e 0,\n    'read'     =\u003e 1,\n    'dataset'  =\u003e 'global',\n    'timeout'  =\u003e 0.2,\n) );\n```\n\n### Sample Configuration 2: Partitioning\n\nThis example shows a setup where the multisite blog tables have been separated from the global dataset.\n\n```php\n$wpdb-\u003eadd_database( array(\n    'host'     =\u003e 'global.db.example.com',\n    'user'     =\u003e 'globaluser',\n    'password' =\u003e 'globalpassword',\n    'name'     =\u003e 'globaldb',\n) );\n\n$wpdb-\u003eadd_database( array(\n    'host'     =\u003e 'blog.db.example.com',\n    'user'     =\u003e 'bloguser',\n    'password' =\u003e 'blogpassword',\n    'name'     =\u003e 'blogdb',\n    'dataset'  =\u003e 'blog',\n) );\n\n$wpdb-\u003eadd_callback( 'my_db_callback' );\n\n// Multisite blog tables are \"{$base_prefix}{$blog_id}_*\"\nfunction my_db_callback( $query, $wpdb ) {\n    if ( preg_match(\"/^{$wpdb-\u003ebase_prefix}\\d+_/i\", $wpdb-\u003etable) ) {\n        return 'blog';\n    }\n}\n```\n\n### Configuration Functions\n\n#### add_database()\n\n```php\n$wpdb-\u003eadd_database( $database );\n```\n\n`$database` is an associative array with these parameters:\n\n```php\nhost          (required) Hostname with optional :port. Default port is 3306.\nuser          (required) MySQL user name.\npassword      (required) MySQL user password.\nname          (required) MySQL database name.\nread          (optional) Whether server is readable. Default is 1 (readable).\n                         Also used to assign preference. See \"Network topology\".\nwrite         (optional) Whether server is writable. Default is 1 (writable).\n                         Also used to assign preference in multi-primary mode.\ndataset       (optional) Name of dataset. Default is 'global'.\ntimeout       (optional) Seconds to wait for TCP responsiveness. Default is 0.2\nlag_threshold (optional) The minimum lag on a replica in seconds before we consider it lagged.\n                         Set null to disable. When not set, the value of $wpdb-\u003edefault_lag_threshold is used.\n```\n\n#### add_table()\n\n```php\n$wpdb-\u003eadd_table( $dataset, $table );\n```\n\n`$dataset` and `$table` are strings.\n\n#### add_callback()\n\n```php\n$wpdb-\u003eadd_callback( $callback, $callback_group = 'dataset' );\n```\n\n`$callback` is a callable function or method. `$callback_group` is the group of callbacks, this `$callback` belongs to.\n\nCallbacks are executed in the order in which they are registered until one of them returns something other than null.\n\nThe default `$callback_group` is 'dataset'. Callback in this group  will be called with two arguments and expected to compute a dataset or return null.\n\n```php\n$dataset = $callback($table, \u0026$wpdb);\n```\n\nAnything evaluating to false will cause the query to be aborted.\n\nFor more complex setups, the callback may be used to overwrite properties of `$wpdb` or variables within `LudicrousDB::connect_db()`. If a callback returns an array, LudicrousDB will extract the array. It should be an associative array and it should include a `$dataset` value corresponding to a database added with `$wpdb-\u003eadd_database()`. It may also include `$server`, which will be extracted to overwrite the parameters of each randomly selected database server prior to connection. This allows you to dynamically vary parameters such as the host, user, password, database name, lag_threshold and TCP check timeout.\n\n## 2. Primary \u0026 Replica Databases\n\nA database definition can include 'read' and 'write' parameters. These operate as boolean switches but they are typically specified as integers. They allow or disallow use of the database for reading or writing.\n\nA primary database might be configured to allow reading and writing:\n\n```php\n'write' =\u003e 1,\n'read'  =\u003e 1,\n```\n\nwhile a replica would be allowed only to read:\n\n```php\n'write' =\u003e 0,\n'read'  =\u003e 1,\n```\n\nIt might be advantageous to disallow reading from the primary, such as when there are many replicas available and the primary is very busy with writes.\n\n```php\n'write' =\u003e 1,\n'read'  =\u003e 0,\n```\n\nLudicrousDB tracks the tables that it has written since instantiation and sending subsequent read queries to the same server that received the write query. Thus a primary set up this way will still receive read queries, but only subsequent to writes.\n\n## 3. Network topology / Datacenter awareness\n\nWhen your databases are located in separate physical locations there is typically an advantage to connecting to a nearby server instead of a more distant one. The read and write parameters can be used to place servers into logical groups of more or less preferred connections. Lower numbers indicate greater preference.\n\nThis configuration instructs LudicrousDB to try reading from one of the local replicas at random. If that replica is unreachable or refuses the connection, the other replica will be tried, followed by the primary, and finally the remote replicas in random order.\n\n```php\nLocal replica 1:   'write' =\u003e 0, 'read' =\u003e 1,\nLocal replica 2:   'write' =\u003e 0, 'read' =\u003e 1,\nLocal primary:     'write' =\u003e 1, 'read' =\u003e 2,\nRemote replica 1:  'write' =\u003e 0, 'read' =\u003e 3,\nRemote replica 2:  'write' =\u003e 0, 'read' =\u003e 3,\n```\n\nIn the other datacenter, the primary would be remote. We would take that into account while deciding where to send reads. Writes would always be sent to the primary, regardless of proximity.\n\n```php\nLocal replica 1:   'write' =\u003e 0, 'read' =\u003e 1,\nLocal replica 2:   'write' =\u003e 0, 'read' =\u003e 1,\nRemote replica 1:  'write' =\u003e 0, 'read' =\u003e 2,\nRemote replica 2:  'write' =\u003e 0, 'read' =\u003e 2,\nRemote primary:    'write' =\u003e 1, 'read' =\u003e 3,\n```\n\nThere are many ways to achieve different configurations in different locations. You can deploy different config files. You can write code to discover the web server's location, such as by inspecting `$_SERVER` or `php_uname()`, and compute the read/write parameters accordingly.\n\n## 4. Replication Lag\n\nLudicrousDB accommodates replica lag by making decisions, based on the defined lag threshold. If the lag threshold is not set, it will ignore the replica lag. Otherwise, it will try to find a non-lagged replica, before connecting to a lagged one.\n\nA replica is considered lagged, if its lag is bigger than the lag threshold you have defined in `$wpdb-\u003edefault_lag_threshold` or in the per-database settings. You can also rewrite the lag threshold, by returning `$server['lag_threshold']` variable with the 'dataset' group callbacks.\n\nLudicrousDB does not check the lag on the replicas. You have to define two callbacks callbacks to do that:\n\n```php\n$wpdb-\u003eadd_callback( $callback, 'get_lag_cache' );\n```\n\nand\n\n```php\n$wpdb-\u003eadd_callback( $callback, 'get_lag' );\n```\n\nThe first one is called before connecting to a replica, and should return either: the replication lag in seconds, or false if unknown (based on `$wpdb-\u003elag_cache_key`).\n\nThe second callback is called after a connection to a replica is established. It should return either: it's replication lag, or false if unknown (based on the connection in `$wpdb-\u003edbhs[ $wpdb-\u003edbhname ]`).\n\n## Sample replication lag detection configuration\n\nTo detect replication lag, try [mk-heartbeat](http://www.maatkit.org/doc/mk-heartbeat.html) or pt-heartbeat from Percona Toolkit. These tools insert a timestamp into a table on the primary and then check the lag on the replicas. The lag is the difference in seconds between the current time and the timestamp on the replica.\n\nThis implementation requires the database user to have read access to the heartbeat table.\n\nThe cache uses shared memory for portability. Can be modified to work with Memcached, APC and etc.\n\n```php\n$wpdb-\u003elag_cache_ttl = 30;\n$wpdb-\u003eshmem_key     = ftok( __FILE__, \"Y\" );\n$wpdb-\u003eshmem_size    = 128 * 1024;\n\n$wpdb-\u003eadd_callback( 'get_lag_cache', 'get_lag_cache' );\n$wpdb-\u003eadd_callback( 'get_lag',       'get_lag' );\n\nfunction get_lag_cache( $wpdb ) {\n    $segment  = shm_attach( $wpdb-\u003eshmem_key, $wpdb-\u003eshmem_size, 0600 );\n    $lag_data = @shm_get_var( $segment, 0 );\n\n    shm_detach( $segment );\n\n    if ( ! is_array( $lag_data ) || !is_array( $lag_data[ $wpdb-\u003elag_cache_key ] ) ) {\n        return false;\n    }\n\n    if ( $wpdb-\u003elag_cache_ttl \u003c time() - $lag_data[ $wpdb-\u003elag_cache_key ][ 'timestamp' ] ) {\n        return false;\n    }\n\n    return $lag_data[ $wpdb-\u003elag_cache_key ][ 'lag' ];\n}\n\nfunction get_lag( $wpdb ) {\n    $dbh = $wpdb-\u003edbhs[ $wpdb-\u003edbhname ];\n\n    if ( ! mysql_select_db( 'heartbeat', $dbh ) ) {\n        return false;\n    }\n\n    $result = mysql_query( \"SELECT UNIX_TIMESTAMP() - UNIX_TIMESTAMP(ts) AS lag FROM heartbeat LIMIT 1\", $dbh );\n\n    if ( ! $result || false === $row = mysql_fetch_assoc( $result ) ) {\n        return false;\n    }\n\n    // Cache the result in shared memory with timestamp\n    $sem_id = sem_get( $wpdb-\u003eshmem_key, 1, 0600, 1 );\n    sem_acquire( $sem_id );\n    $segment = shm_attach( $wpdb-\u003eshmem_key, $wpdb-\u003eshmem_size, 0600 );\n    $lag_data = @shm_get_var( $segment, 0 );\n\n    if ( ! is_array( $lag_data ) ) {\n        $lag_data = array();\n    }\n\n    $lag_data[ $wpdb-\u003elag_cache_key ] = array( 'timestamp' =\u003e time(), 'lag' =\u003e $row[ 'lag' ] );\n\n    shm_put_var( $segment, 0, $lag_data );\n    shm_detach( $segment );\n    sem_release( $sem_id );\n\n    return $row[ 'lag' ];\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstuttter%2Fludicrousdb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstuttter%2Fludicrousdb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstuttter%2Fludicrousdb/lists"}