{"id":16339106,"url":"https://github.com/phadej/git-badc0de","last_synced_at":"2025-03-22T23:33:01.237Z","repository":{"id":66242017,"uuid":"284541703","full_name":"phadej/git-badc0de","owner":"phadej","description":"Make git commits with pretty hashes","archived":false,"fork":false,"pushed_at":"2020-08-04T09:36:22.000Z","size":32,"stargazers_count":28,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-18T15:49:01.089Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/phadej.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-02T21:06:20.000Z","updated_at":"2024-05-08T08:24:21.000Z","dependencies_parsed_at":"2023-02-20T16:46:11.150Z","dependency_job_id":null,"html_url":"https://github.com/phadej/git-badc0de","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phadej%2Fgit-badc0de","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phadej%2Fgit-badc0de/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phadej%2Fgit-badc0de/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/phadej%2Fgit-badc0de/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/phadej","download_url":"https://codeload.github.com/phadej/git-badc0de/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245036127,"owners_count":20550662,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T23:53:30.123Z","updated_at":"2025-03-22T23:33:01.214Z","avatar_url":"https://github.com/phadej.png","language":"C","funding_links":[],"categories":["C"],"sub_categories":[],"readme":"# git-badc0de - a tool to improve git commits you receive\n\nThere are various practices of authoring patches or commits in version control systems.\nIf you are, like me, annoyed by *fix typo* fix up commits in pull or merge requests\nyou get at work or in a open source project,\nor if you simply get too much contributions (which is good place to be, tell me how get there),\nthen `git-badc0de` is the tool for you.\n\nThe problem is clearly that it is too easy to make new commits.\nThe solution is make creating commits harder.\nGit developers make `git` interface saner with each release,\nand there are various tools (like GitHub web editing),\nwhich make writing *fix typo* commits child play easy.\n\n`git-badc0de` (GitHub: [phadej/git-badc0de](https://github.com/phadej/git-badc0de))\ntakes an out-of-the-box approach.\nInstead of trying to encourage (or force) humans to put more effort\ninto each commit, it makes their machines do the work.\n\n`git-badc0de` takes the `HEAD` commit, and creates an altered copy,\nsuch that the commit hash starts with some (by default `badc0de`) prefix.\nObviously, I use `git-badc0de` in the development process of `git-badc0de` itself.\nCheck the tail of `git log`:\n\n```plain\nbadc0dea Add CONTRIBUTING.md\nbadc0ded Make prefix configurable.\nbadc0de4 Add message to do git reset\nbadc0de6 Comment out some debug output\nbadc0de5 Initial commit\n```\n\nIt's up to the project owners to decide how long prefix you want to have.\nSeven base16 characters (i.e. 28 bits out of 160) is doable on modern multi-core hardware in a minute,\nwith good luck in less[^eightchars].\nThese seconds are important.\nIt is an opportunity to reflect,\nmaybe even notice a typo in the commit message.\nModern machines are so fast, and even some compilers too[^compilers],\nthat we don't pause and think of what we have just done.\n\n[^eightchars]: Eight base16 characters (i.e. 4 bytes) took one and half hour of CPU time or \"just\" 5.5 minutes of wall clock time. I run that experiment only few times. Take a look at [deadc0de branch](https://github.com/phadej/git-badc0de/commits/deadc0de).\n\n[^compilers]: If your programming language of choice is a compiled one.\n\nHow git-badc0de works\n---------------------\n\n*Git is content-addressable file system* is how\n[a chapter in Pro Git book on Git objects](https://git-scm.com/book/en/v2/Git-Internals-Git-Objects) starts.\nVery nice model, very easy to tinker with.\nYou can try out to `git cat-file -p HEAD` in your current Git project to see\nthe `HEAD` commit object data. In `git-badc0de` one commit looks like:\n\n```plain\ntree 91aaad77e68aa7bf94219a5b9cea97f26e2cce2b\nparent badc0dea0106987c4edfb1d169b5a43d95845569\nauthor Oleg Grenrus \u003coleg.grenrus@iki.fi\u003e 1596481157 +0300\ncommitter Oleg Grenrus \u003coleg.grenrus@iki.fi\u003e 1596481157 +0300\n\nRewrite README.md\n\nPoW: HRYsAAAAAAF\n```\n\nGit commit hash is a hash of a header and these contents.\nA header for commit object looks like\n\n```plain\ncommit \u003ccontent length as ASCII number\u003e\u003cNUL\u003e\n```\n\n`git-badc0de` takes the most recent commit data,\nand by adding some  `PoW: DECAFC0FFEE` *salts* to the end,\ntries to find one which makes commit hash with the correct prefix.\nIt takes 11 characters to encode 64 bits in base64.\nWhy base64, no particular reason.\nBased on [this StackOverflow answer](https://stackoverflow.com/a/57701124/1308058)\nwe could put salts into commit headers,\nto hide them from `git log`.\nSomething to do in the future.\n\nWhen a valid salt is found, `git-badc0de` writes the new commit object\nto the Git object store.\nAt this point nothing is changed, only a new dangling object inside `.git` directory.\nYou can reset your branch to point to the new commit with\n`git reset`, and `git-badc0de` invites you to do so.\n\nAre you serious\n---------------\n\nYes, I'm dead serious (*No*). But I had fun implementing `git-badc0de`.\nI was surprised that getting seven characters \"right\" in a hash is an easy job.\nThat causes nice artifacts in GitHub web interface.\n\nThe top commit shown on [the project main page](https://github.com/phadej/git-badc0de) is always `badc0de`...\n\n![first GitHub screenshot](https://oleg.fi/gists/images/git-badc0de-github-1.png)\n\n... and in fact [all commits](https://github.com/phadej/git-badc0de/commits/master) seem to have the same hash (prefix)...\n\n![second GitHub screenshot](https://oleg.fi/gists/images/git-badc0de-github-2.png)\n\nNote how command line `git log` is smart to show enough characters to make\nprefixes unambiguous. It is deliberate, check on some of your smaller projects,\nthere `git log --oneline` probably prints seven character abbreviations.\nIn GHC (Haskell compiler) `git log --oneline` prints ten characters for me\n(GitHub still shows just seven, so I assume it is hardcoded).\n\nWe can also use `git-badc0de` to produce commits with ordered hashes!\nThe downside is that you have to decide the maximum commit count at the start.\nYet $16^7 = 268435456$ should be enough for about any project.\nSee [ordered branch](https://github.com/phadej/git-badc0de/commits/ordered),\nisn't that cool!?\n\n![third GitHub screenshot](https://oleg.fi/gists/images/git-badc0de-github-3.png)\n\nImplementation comments\n-----------------------\n\nHow `git-badc0de` is implemented?\nI have to confess: I started with a Python prototype.\nPython comes with all pieces needed, though I essentially only needed `hashlib`.\n\n[The Haskell implementation](https://github.com/phadej/git-badc0de) has eleven dependencies at the moment of writing.\nFive of them are bundled with compiler, the rest six are not.\nEven for some basic tasks you have to go package shopping:\n\n- [`async`](https://hackage.haskell.org/package/async) to parallelize computations\n- [`base16-bytestring`](https://hackage.haskell.org/package/base16-bytestring)\n- [`cryptohash-sha1`](https://hackage.haskell.org/package/cryptohash-sha1)\n- [`primitive`](https://hackage.haskell.org/package/primitive) to write some low level code\n- [`utf8-string`](https://hackage.haskell.org/package/utf8-string) to convert from byte UTF-8 `ByteString` representation to `String` and back.\n- [`zlib`](https://hackage.haskell.org/package/zlib) to compress git objects, as they are stored compressed.\n\nMy first Haskell implementation was noticeably faster than Python3 version.\nI suspect that is because Haskell is simply better at gluing bytes together.\n\nThe motivation to use Haskell had two parts:\n\n- I just use Haskell for everything. (Except for prototyping silly ideas). This is the most important reason.\n- Haskell is good for writing parallel programs. This is a bonus.\n\nTo my surprise, my first Haskell parallelization attempt didn't work at all.\nAn idea is to spawn multiple workers, which would try different salts.\nAnd then make them race, until one worker finds a valid salt.\nAdding more workers should not slowdown the overall program,\nminus maybe some small managerial overhead.\n\nThe overhead turned out to be quite large.\nParallelism in Haskell works well when you deal with Haskell \"native\" data.\n`git-badc0de` use case is however gluing bytes (`ByteString`s) together\nand calling out to C implementation of SHA1 algorithm.\n\nThe nasty detail of, I guess any, higher level languages is that\nforeign function interface has culprits.\nI run into `foreign import unsafe` issue.\nYou may read about `foreign import unsafe` in [the excellent GHC User Guide](https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ffi-chap.html#guaranteed-call-safety).\n\n\u003cblockquote\u003e\nGHC \u003cb\u003eguarantees\u003c/b\u003e that garbage collection will never occur during an \u003ccode\u003eunsafe\u003c/code\u003e call, ...\n\u003c/blockquote\u003e\n\nWith many threads generating some amount of garbage,\nbut also calling `unsafe` foreign functions in a tight loop caused problems.\nSurprisingly, both `cryptohash-sha1` and `bytestring` use plenty of `unsafe` calls\n([`cryptonite`](https://hackage.haskell.org/package/cryptonite-0.27/docs/src/Crypto.Hash.SHA1.html#line-26) uses too).\n\nMy solution was to redo the loop. Less garbage generation and less foreign calls.\n\n`cryptohash-sha1` (and `cryptonite`) import `_init`, `_update` and `_finalize`\nC functions. The hashing context is allocated and plumbing done in Haskell.\nHowever, we can setup things in way such that we pass a single continuous block of memory to be hashed.\nThis functionality is missing from the library, so I copied `cbits/` from\n`cryptohash-sha1` and added small C function, to do C plumbing in C:\n\n```c\nvoid\nhs_cryptohash_sha1(const uint8_t *data, size_t len, uint8_t *out)\n{\n  struct sha1_ctx ctx;\n  hs_cryptohash_sha1_init(\u0026ctx);\n  hs_cryptohash_sha1_update(\u0026ctx, data, len);\n  hs_cryptohash_sha1_finalize(\u0026ctx, out);\n}\n```\n\nThis way we can have one `safe` foreign call to calculate the hash.\nWe have to make sure that pointers are pointing at pinned memory,\ni.e. memory which garbage collector won't move.\n\nNext, the same problem is in the `bytestring` library,\nI was naive to think that as byte data I work with is so small,\nthat concatenating it (and thus `memcpy`ing) won't be noticeable,\nhashing should dominate.\nUsually it isn't a problem, but as copying was done on each loop\niteration and `memcpy` is `foreign import unsafe` in `bytestring`\nlibrary, that also contributed to slowdown.\nThat was my hypothesis.\n\nFiguring out how to do it better with `bytestring` seemed difficult,\nso I opted for a different solution. Write some C-in-Haskell.\nNow each worker creates own *mutable* template, which is updated with new salt\non each loop iteration.\nSalt length is fixed, so we don't need to change the commit object header.\nAs a bonus, the loop become essentially non-allocating (I didn't check though).\n\nAfter that change, `git-badc0de` started to use all the cores, and not just spin in GC locks.\nThe runtime system statistics are nice to look at\n\n```plain\n                                Tot time (elapsed)  Avg pause  Max pause\nGen  0      0 colls,     0 par    0.000s   0.000s     0.0000s    0.0000s\nGen  1      1 colls,     0 par    0.000s   0.000s     0.0004s    0.0004s\n\n...\n\nProductivity 100.0% of total user, 99.9% of total elapsed\n```\n\nNo time is spent in garbage collection. Productivity is an amount of time\nused to do actual work and not collecting garbage.\nDisclaimer: it seems that waiting for GC locks is not counted towards GC time,\nbut as there was only a single collection, that doesn't matter.\n\nI could optimize further:\nas the salt is at the end of the content it is silly\nto rehash whole commit object every time.\nYet, `git-badc0de` is silly project to begin with,\nand I am satisfied with the current state.\n\nThe lesson here is that foreign function interface (FFI) is not easy,\nyou have to think and test.\n\n\"Luckily\" I learned about the `unsafe` issue recently\nin [`postgresql-libpq`](https://github.com/phadej/postgresql-libpq/pull/9),\nso was able to think about it causing my problems.\nIn this case, `unsafe` doesn't mean that \"I know what I'm doing\" (as e.g. with `unsafePerformIO`),\nbut rather the opposite.\n\nAlso, I don't think that we (= Haskell ecosystem) have a good tooling to benchmark how code behaves in highly parallel environments.\nI *hope* that `Data.ByteString.Builder`, for example, doesn't use any `unsafe` foreign calls,\nEcosystem relies on that module for constructing JSON (in `aeson`) and HTML (both `blaze-markup` and `lucid`).\nSomething for someone to test, maybe fix and document.\n\nLearnings\n---------\n\nDoes this mean that Haskell is crap, and the promise for easy parallel\nand concurrent programming is a lie, and we should all use Rust instead?\n\nWell, no.\nIn this isolated example, Rust would probably shine.\nThere are, however, also other parts than hashing loop even in this simple program,\nand they have to be written as well.\nThere Haskell feels a lot like Python,\nin a sense that I can just write consice code which works.\n\nPython was quite nice in the very early prototyping stage,\nas it happened to have all needed functionality available in the repl.\nThe \"early prototyping stage\" lasted for maybe 10 or 15 minutes,\nthat was enough to verify that basic idea might work.\nWith Haskell, you would need to restart repl to add a new library, losing all the context, which would killed the flow.\nFor some other \"problem\"I might start to prototype directly in Haskell.\nI have no experience with how nice repl experience Rust has.\n\nIf `git-badc0de` project were to develop further, I would rewrite\nthe hashing loop in C, instead of writing C-in-Haskell. Maybe.\nOr in Rust, if I that was easier to setup.\n(GHC knows how to invoke C compiler).\n\nHaskell is a great glue language, among many other great properties it has.\nDon't believe anyone who tells you otherwise.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphadej%2Fgit-badc0de","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fphadej%2Fgit-badc0de","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fphadej%2Fgit-badc0de/lists"}