{"id":17175041,"url":"https://github.com/jdblischak/htslib-test-builds","last_synced_at":"2025-03-24T21:17:18.540Z","repository":{"id":65676293,"uuid":"596177901","full_name":"jdblischak/htslib-test-builds","owner":"jdblischak","description":null,"archived":false,"fork":false,"pushed_at":"2023-02-10T21:39:46.000Z","size":46,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-30T01:28:02.892Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jdblischak.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-01T16:24:14.000Z","updated_at":"2023-02-02T20:32:45.000Z","dependencies_parsed_at":"2023-02-18T11:32:07.112Z","dependency_job_id":null,"html_url":"https://github.com/jdblischak/htslib-test-builds","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jdblischak%2Fhtslib-test-builds","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jdblischak%2Fhtslib-test-builds/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jdblischak%2Fhtslib-test-builds/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jdblischak%2Fhtslib-test-builds/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jdblischak","download_url":"https://codeload.github.com/jdblischak/htslib-test-builds/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245352091,"owners_count":20601093,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T23:55:33.853Z","updated_at":"2025-03-24T21:17:18.516Z","avatar_url":"https://github.com/jdblischak.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# htslib test builds\n\nThis repository contains various GitHub Actions workflows that I used to\nexperiment with building [htslib][].\n\n[htslib]: https://github.com/samtools/htslib\n\n## Build latest htslib release with msys2\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/setup-msys2-action.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/setup-msys2-action.yml)\n\nI created the msys2 environment on GitHub Actions using [msys2/setup-msys2][].\nWhile msys2 is already pre-installed on the GitHub Actions Windows runner image,\nit's not on the PATH, and I didn't want to deal with the headache of modifying\nWindows paths.\n\n[msys2/setup-msys2]: https://github.com/msys2/setup-msys2\n\nI mostly followed the [official installation instructions][install]. I got the\npacman flags from the [AppVeyor build][appveyor].\n\n[install]: https://github.com/samtools/htslib/blob/463830bf7de8c4ab731c4d67c49ddc446f498f50/INSTALL#L276\n[appveyor]: https://github.com/samtools/htslib/blob/develop/.appveyor.yml\n\n* [setup-msys2-action.yml](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/setup-msys2-action.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/actions/workflows/setup-msys2-action.yml)\n\n## Build a statically-linked, dependency-free htslib.dll with current msys2\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-static-linking.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-static-linking.yml)\n\nI modified my existing msys2 build above to implement David's instructions for\nbuilding a statically-linked, dependency-free htslib.dll. The specific changes are:\n\n1. Passed `update: true` to msys2/setup-msys2 to replicate `pacman -Syuu`\n1. Installed htslib 1.15.1 instead of the latest release (1.16)\n1. Passed `CFLAGS=-DCURL_STATICLIB` to `./configure` and downloaded David's\n   [patches][patches]\n1. Used the action [ilammy/msvc-dev-cmd][] to setup MSVC. Again, this is\n   pre-installed, but it's not in the PATH\n\n[patches]: https://github.com/TileDB-Inc/TileDB-VCF/blob/dlh/sc-24772-msys2-htslib-msvc-tdbvcf-WIP1/libtiledbvcf/cmake/patches/\n[ilammy/msvc-dev-cmd]: https://github.com/ilammy/msvc-dev-cmd\n\n* [msys2-static-linking.yml](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/msys2-static-linking.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-static-linking.yml)\n\n## Prototype to build a statically linked htslib.dll under msys2\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-htslib-prototype.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-htslib-prototype.yml)\n\nThis is a continuation of the above job, which had confirmed I could implement\nDavid's instructions in a GitHub Actions run. This workflow is a prototype of\nwhat I eventually want to create as the source repository for building these\nstatically linked builds.\n\nI made the following updates:\n\n* Moved code into standalone scripts\n* Created and applied patches to `Makefile` and `config.mk` (as opposed to\n  downloading complete replacement files)\n* Used environment variables to avoid hard-coding versions\n* Upload all the required files as an artifact\n* Create a release and upload the tarball as a release artifact\n\n* [msys2-htslib-prototype.yml](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/msys2-htslib-prototype.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/actions/workflows/msys2-htslib-prototype.yml)\n\n## Build m2w64-htslib from statically-linked .dll\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64-dll.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64-dll.yml)\n\nInstead of trying to build htslib from source in a conda-installed msys2\nenvironment (which was attempted in the job described in the section below),\nthis job builds a conda binary from a recipe that simply downloads the release\ntarball of the pre-built binary and unpacks it into a conda environment. It\ndownloads the release tarball from the releases in this repository created by\nthe job\n[msys2-htslib-prototype.yml](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/msys2-htslib-prototype.yml).\nIt uploads the conda binary as a workflow artifact.\n\n* [conda-m2w64-dll.yml](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64-dll.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/conda-m2w64-dll.yml)\n\n## Build m2w64-htslib\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64.yml)\n\nI attempted to build a m2w64 version of htslib from a conda recipe. I started\nfrom the [bioconda recipe][htslib-bioconda], and replaced the dependencies with\neither posix (`m2-`) or native (`m2w64-`) alternatives. I was mainly following\nother example recipes, eg this one for [m2w64-icu][].\n\n[m2w64-icu]: https://github.com/conda-forge/msys2-recipes/blob/c34a184e6d198d5dc9286cf5a0ebcac241481fe6/msys2/m2w64-icu/meta.yaml\n\nKnown limitations:\n\n* No m2-libdeflate or m2w64-libdeflate\n* No m2w64-libcurl\n* conda-forge has clearly [converged on\n  MSVC](https://conda-forge.org/docs/maintainer/knowledge_base.html#compilers).\n  Most of the msys2 packages are [over 2 years\n  old](https://github.com/conda-forge/msys2-recipes). Even if we can get a msys2\n  conda build to work, this infrastructure will continue to bit rot without\n  substantial maintenance intervention\n\nI tried various compilers. Nothing is currently working.\n\n* `{{ compiler('m2w64_c') }}` this fails to even solve the conda environment.\n  This is baffling to me since this compiler is used in many existing, current\n  conda-forge recipes, especially for R packages\n  ```sh\n  conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform win-64: {'m2w64_c_win-64'}\n  ```\n\n* `m2w64-gcc` I've had the most luck by explicitly setting the appropriate\n  compiler. Note that `conda build` does not like this. However, it suddenly\n  fails without much indication as to what went wrong\n  ```sh\n  (%BUILD_PREFIX%) %SRC_DIR%\u003emake\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o kfunc.o kfunc.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o kstring.o kstring.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o bcf_sr_sort.o bcf_sr_sort.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o bgzf.o bgzf.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o errmod.o errmod.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o faidx.o faidx.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o header.o header.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o hfile.o hfile.c\n  echo '#define HTS_VERSION_TEXT \"1.16\"' \u003e version.h\n  echo '#define HTS_CC \"gcc\"' \u003e config_vars.h\n  echo '#define HTS_CPPFLAGS \"-D_XOPEN_SOURCE=600\"' \u003e\u003e config_vars.h\n  echo '#define HTS_CFLAGS \"-Wall -g -O2 -fvisibility=hidden\"' \u003e\u003e config_vars.h\n  echo '#define HTS_LDFLAGS \"-fvisibility=hidden \"' \u003e\u003e config_vars.h\n  echo '#define HTS_LIBS \"-lbz2 -lws2_32 -lz  \"' \u003e\u003e config_vars.h\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o hts.o hts.c\n  gcc -Wall -g -O2 -fvisibility=hidden  -I. -D_XOPEN_SOURCE=600 -c -o hts_expr.o hts_expr.c\n  Makefile:173: recipe for target 'hts_expr.o' failed\n  ```\n\n* `{{ compiler('c') }}` - this install MSVC, and then fails to find zlib\n\n* `{{ compiler('r_clang') }}` as seen in the recipe for\n  [r-arrow](https://github.com/conda-forge/r-arrow-feedstock/blob/494d0e66ffab16cd8a6b56a31ddecc592e596e28/recipe/meta.yaml#L31).\n  Just as above, it\n  [fails](https://github.com/jdblischak/htslib-test-builds/actions/runs/4075803294/jobs/7022692556)\n  to find zlib\n\n* [conda-m2w64.yml](https://github.com/jdblischak/htslib-test-builds/actions/workflows/conda-m2w64.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/conda-m2w64.yml)\n\n## Build bioconda recipe\n\n[![status](https://github.com/jdblischak/htslib-test-builds/actions/workflows/bioconda.yml/badge.svg)](https://github.com/jdblischak/htslib-test-builds/actions/workflows/bioconda.yml)\n\nThis was mostly a sanity check. I wanted to confirm that I could build the\nexisting  [bioconda recipe for htslib][htslib-bioconda] in a GitHub Actions\nenvironment for both Ubuntu and macOS. It was relatively straightforward. I used\n[mamba-org/provision-with-micromamba][] to install [bioconda-utils][]\n\n[htslib-bioconda]: https://github.com/bioconda/bioconda-recipes/tree/master/recipes/htslib\n[mamba-org/provision-with-micromamba]: https://github.com/mamba-org/provision-with-micromamba\n[bioconda-utils]: https://github.com/bioconda/bioconda-utils\n\n* [bioconda.yml](https://github.com/jdblischak/htslib-test-builds/blob/main/.github/workflows/bioconda.yml)\n* [build logs](https://github.com/jdblischak/htslib-test-builds/actions/workflows/bioconda.yml)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjdblischak%2Fhtslib-test-builds","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjdblischak%2Fhtslib-test-builds","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjdblischak%2Fhtslib-test-builds/lists"}