{"id":27643304,"url":"https://github.com/matrixorigin/mo-chaos","last_synced_at":"2025-04-24T00:12:25.056Z","repository":{"id":204433809,"uuid":"671301216","full_name":"matrixorigin/mo-chaos","owner":"matrixorigin","description":null,"archived":false,"fork":false,"pushed_at":"2024-10-28T03:01:11.000Z","size":770,"stargazers_count":0,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-24T00:12:05.086Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matrixorigin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-27T02:37:05.000Z","updated_at":"2024-10-28T03:01:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"426f2a1d-ec7f-442e-80b4-b6852ac67224","html_url":"https://github.com/matrixorigin/mo-chaos","commit_stats":null,"previous_names":["matrixorigin/mo-chaos-driller","matrixorigin/mo-chaos"],"tags_count":43,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fmo-chaos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fmo-chaos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fmo-chaos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matrixorigin%2Fmo-chaos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matrixorigin","download_url":"https://codeload.github.com/matrixorigin/mo-chaos/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250535106,"owners_count":21446508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-24T00:12:24.439Z","updated_at":"2025-04-24T00:12:25.046Z","avatar_url":"https://github.com/matrixorigin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# mo-chaos工具简介：\n## 主要目录文件介绍：\n     chaos-mesh/ : 存放chaos-mesh的helm chart目录\n     thread/ : chaos线程、test线程、controller线程\n     config/ : 存放配置chaos 和 test 的配置yaml文件，用户设计好对应的chaos故障配置文件保存在里面\n     mo_chaos_test.py : 执行chaos 的主文件\n     logs/ : 程序运行时自动创建，产生log文件的目录\n     test-tool/ : 测试工具目录存放目录\n     test-report/ : 测试工具产生的log 存放的目录\n\n## 部署 chaos-mesh\n     下载chaos-mesh的helm chart到执行机器上面\n     创建ns kubectl create ns \u003cnamespace\u003e\n     之后在ns 创建下载镜像用户密码的 secret，如果没有权限联系吴叶磊开通权限 \n     ps：机房机器无法下载海外镜像，目前将镜像转存到公司阿里云镜像repo里面\n     kubectl create secret docker-registry aliyun --docker-server=registry.cn-hangzhou.aliyuncs.com/mocloud --docker-username= --docker-password= -n \u003cnamespace\u003e\n     helm部署： helm 部署命令 helm install chaos-mesh ./chaos-mesh -n=\u003cnamespace\u003e --set chaosDaemon.runtime=containerd --set chaosDaemon.socketPath=/run/containerd/containerd.sock\n   \n## 使用命令\n   python mo_chaos_test.py -c \u003cchaos yaml文件名称\u003e  -t  \u003ctest yaml文件名称\u003e\n    \n## chaos yaml文件结构介绍\n样例\n```yaml\nchaos:\n  cm-chaos:\n    - name: task_a1\n      kubectl_yaml: |\n        apiVersion: chaos-mesh.org/v1alpha1\n        kind: PodChaos\n        metadata:\n          name: pod-kill-01\n          namespace: mocluster2\n        spec:\n          action: pod-kill\n          mode: one\n          selector:\n            namespaces:\n              - mocluster2\n            labelSelectors:\n              'matrixorigin.io/component': 'CNSet'\n      times: 2\n      interval: 10\n      is_delete_after_apply: true\n    - name: task_a2\n      kubectl_yaml: |\n        kind: NetworkChaos\n        apiVersion: chaos-mesh.org/v1alpha1\n        metadata:\n          namespace: mocluster2\n          name: network-delay-pod\n        spec:\n          selector:\n            namespaces:\n              - mocluster2\n            labelSelectors:\n              'matrixorigin.io/component': 'CNSet'\n          mode: all\n          action: delay # 网络故障类型，延迟\n          duration: 20m # 持续时间, 这个为了持续观察，所以配置的时间较长\n          delay:\n            latency: 100ms # 延迟时间\n            correlation: '0.5' # 表示延迟时间的时间长度与前一次延迟时长的相关性\n            jitter: 20ms # 表示延迟时间的变化范围\n      times: 3\n      interval: 10\n      is_delete_after_apply: true\n  sql-chaos:\n    - name: sql-chaos01\n      type: database_flush_chaos\n      dbname: tpcc\n      interval: 10\n      times: 2\n    - name: sql-chaos02\n      type: table_flush_chaos\n      dbname: tpcc\n      tablename: bmsql_new_order\n      interval: 10\n      times: 2\n    - name: sql-chaos03\n      type: database_merge_chaos\n      dbname: tpcc\n      interval: 10\n      times: 2\n    - name: sql-chaos04\n      type: table_merge_chaos\n      dbname: tpcc\n      tablename: bmsql_new_order\n      interval: 10\n      times: 2\n    - name: sql-chaos05\n      type: checkpoint_chaos\n      interval: 10\n      times: 2\n  chaos_combination:\n    mode: \"in-turn\"  # or \"random-turn\"\n  mo-env:\n      host: \"10.222.4.14\"\n      user: \"dump\"\n      port: 31429\n      password: \"111\"\n  namespace: \"mocluster2\"\n```\n    1、节点cm-chaos： 配置chaos-mesh类型故障\n           name：代表节点名称\n           kubectl_yaml： chaos-mesh故障 kubectl apply -f 对应的配置文件 ，\n                          可以参考[chaos-mesh 官方文档](https://chaos-mesh.org/zh/docs/) pod-kill、network-delay、io-delay等类型故障\n           times: 代表这个故障执行多少次\n           interval：每次执行完sleep多少秒\n           is_delete_after_apply： 执行完 之后sleep interval 之后 是否删除 默认填true 删除\n    2、节点sql-chaos： 配置sql类型故障\n           name： 代表节点名称\n           type： 目前有5种类型 \n           dbname： database \u003c对于checkpoint_chaos 不需要\u003e\n           tablename： 表名称 \u003c对于checkpoint_chaos 、database_flush_chaos、database_merge_chaos 不需要\u003e\n           times:代表这个故障执行多少次\n           interval：每次执行完sleep 多少秒\n    3、节点chaos_combination： 配置chaos 运行方式\n           mode：in-turn （轮流执行cm-chaos 和 sql-chaos）\n                 random-turn（随机选择cm-chaos 和 sql-chaos）\n    4、节点mo-env: 配置连接mo的基本配置 （host、port、user、password）\n    5、节点namespace: 代表服务部署对应的ns\n\n\n## test yaml 文件配置介绍\n```yaml\ntasks:\n- name: mo-tpcc\n  work-path: mo-tpcc\n  run-steps:\n    - command: ./runBenchmark.sh props.mo\n  verify:\n    - command: ./runVerify.sh props.mo\n  verify-mode: parallel\n  log-paths:\n    - path: benchmarksql-info.log\n    - path: benchmarksql-error-10-2.log\n- name: mo-tpcc-2\n  work-path: mo-tpcc-2\n  run-steps:\n    - command: ./runBenchmark.sh props.mo\n  verify:\n    - command: ./runVerify.sh props.mo\n  verify-mode: after\n  log-paths:\n    - path: benchmarksql-info.log\n    - path: benchmarksql-error-10-2.log\n```\n       tasks: 下面挂载多个task节点，并行运行\n         name： 代表task name\n         work-path: 代表测试工具工作目录\n         run-steps: 配置 test 任务的执行命令\n         verify: 配置验证测试结果的执行命令\n         verify-mode： parallel代表和测试任务一起运行循环执行，after代表执行完test任务命令之后再执行\n         log-paths: 配置测试工具生成的log文件，跑完任务会cp 到 test-report/ 目录下\n\n## yaml文件配置路径\n   放入config 目录下，执行的时候 python mo_chaos_test.py -c \u003cchaos yaml文件名称\u003e  -t  \u003ctest yaml文件名称\u003e 带文件名称不要带路径，程序只会到config找同名文件\n\n## 使用TIPS\n     A、在配置chaos-mesh单个故障时不要配置schedule类型故障，把周期性任务通过交给程序（times、interval）来做，\n        配置单个chaos故障，只配置基础的不要配置组合场景，我们执行的过程都有log记录\n     B、logs下有对应log文件产生可以通过观测log确认运行情况\n     C、使用之前确认chaos-mesh 和mo 服务都部署在 k8s集群，执行程序的机器的kubectl 能访问对应的k8s集群\n     D、使用之前通过pip install -r requirements.txt\n     E、使用python版本需要3.11\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixorigin%2Fmo-chaos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatrixorigin%2Fmo-chaos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatrixorigin%2Fmo-chaos/lists"}