{"id":19867706,"url":"https://github.com/cwsmith/commkernel","last_synced_at":"2025-06-21T05:02:51.335Z","repository":{"id":30403972,"uuid":"33956752","full_name":"cwsmith/commKernel","owner":"cwsmith","description":"communication kernel for performance testing on Stampede","archived":false,"fork":false,"pushed_at":"2015-06-23T16:41:37.000Z","size":304,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-01T00:39:13.974Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cwsmith.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-04-14T20:59:51.000Z","updated_at":"2020-09-02T17:54:34.000Z","dependencies_parsed_at":"2022-09-22T06:52:08.759Z","dependency_job_id":null,"html_url":"https://github.com/cwsmith/commKernel","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/cwsmith/commKernel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwsmith%2FcommKernel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwsmith%2FcommKernel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwsmith%2FcommKernel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwsmith%2FcommKernel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cwsmith","download_url":"https://codeload.github.com/cwsmith/commKernel/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cwsmith%2FcommKernel/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261065252,"owners_count":23104762,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T15:30:03.921Z","updated_at":"2025-06-21T05:02:46.324Z","avatar_url":"https://github.com/cwsmith.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"## commKernel\nMPI left-right communication kernel for performance testing on Stampede.  Three binaries are used for testing:\n\nthreadWorkComm\n\nOnly 1 MPI task is ran and the number of workers corresponds to the number of pthreads that are created. Data is exchanged between pthreads. This code is using MPI_THREAD_MULTIPLE\n\nmpiWork\n\nHere workers are representing MPI tasks. Data is exhchanged between MPI tasks. The code is using MPI_INIT.\n\nmpiWorkThreadMult\n\nHere workers are representing MPI tasks. Data is exhchanged between MPI tasks. The code is using MPI_THREAD_MULTIPLE.\n\n## tl;dr\n\nMPI_THREAD_MULTIPLE peformance with pthreads is bad on the Phi.  \n\nThe following presentation and paper discuss the use of a library that implements the proposed MPI endpoints apis to increase bandwidth at the cost of increased latency.  Results show net application speedup despite the latency penalty.\n\nhttp://meetings.mpi-forum.org/secretary/2014/12/slides/Hybrid-Plenary--EP-Lib.pdf\n\nhttp://pcl.intel-research.net/publications/sridharan-sc14.pdf\n\n## contents\n- envImpi.sh - environment file for Intel MPI 4\n- envImpi5.sh - environment file for Intel MPI 5\n- envImpi51.sh - environment file for Intel 16 and Intel MPI 5.1 (beta)\n- envMvapich.sh - environment file for Mvapich2-mic\n- envMvapich2.sh - environment file for Mvapich2-mic v2.0 - edit the install path\n- envMpich3.sh - environment file for MPICH 3.2b2 - edit the install path\n- build.sh - build script \n- build.x86.sh - build script for stampede host processors\n- getTimeImpi.sh - run script for Intel MPI 4 and 5\n- getTimeImpi.x86.sh - run script for Intel MPI 4 and 5 on stampede host processors\n- getTimeImpi51.sh - run script for Intel MPI 5.1\n- getTimeMvapich.sh - run script Mvapich2-mic\n- getTimeMvapich2.sh - run script Mvapich2-mic\n- getTimeMpich3.sh - run script MPICH 3.2b2\n- kernelComm.c - communication kernel\n- kernelComm.x86.c - communication kernel with affinity on stampede host processors\n- kernel.h - kernel header\n- mpiWork.c - mpi driver\n- README.md - this file \n- threadWork.c - threaded mpi driver\n\n## build\n    source env\u003cImpi|Impi51|Mvapich|Mvapich2|Mpich3\u003e.sh\n    for i in 1 2 4; do ./build[.x86].sh $i; done\n\n## run\nFrom an interactive idev session:\n\n    ./getTime\u003cImpi|Mvapich|Mvapich2|Mpich3\u003e[.x86].sh\n\n## Performance Results\n\nThe following tests used MPSS 3.3 as installed on Stampede.\nsee performance.csv\n\n## Mvapich2-MIC v2.0\nMvapich2-MIC v2.0 is not installed on Stampede.  To install it follow the instructions below:\n\n### download\n    wget http://mvapich.cse.ohio-state.edu/download/mvapich/mic/mvapich2-mic-2.0_mpss-3.3.run\n\n### run installer\n    ./mvapich2-mic-2.0_mpss-3.3.run\n    #set the install path to something in your home/work directory\n    #accept defaults for library paths \nedit the 'ins' variable in the environment file 'envMvapichMic2.sh' to point at the chosen install path\n\n### sanity check the install by running the latency benchmark\n\n    source envMvapich2.sh\ncreate file 'config'\n\n    echo \"-n 2 : $MV2_MIC_INSTALL_PATH/libexec/mvapich2/osu_latency\" \u003e config\ncreate file 'hosts'\n\n    echo 'mic0:2' \u003e\u003e hosts\nrun\n\n    mpirun_rsh -config ./config -hostfile ./hosts\n\n## MPICH3 \nMPICH 3.2b is not installed on Stampede.  To install it follow the instructions below:\n\n### download \n    wget http://www.mpich.org/static/downloads/3.2b2/mpich-3.2b2.tar.gz\n\n### unpack, configure, and build\n    tar xzf mpich-3.2b2.tar.gz\n    cd mpich-3.2b2\n    mkdir buildIntel15\n    cd buildIntel15\n    ./doConfigIntel15.sh \u003cinstall prefix absolute path\u003e\n    make \n    make install\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcwsmith%2Fcommkernel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcwsmith%2Fcommkernel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcwsmith%2Fcommkernel/lists"}