{"id":30389452,"url":"https://github.com/chrismo/superkit","last_synced_at":"2025-08-21T08:21:39.604Z","repository":{"id":280831467,"uuid":"902623077","full_name":"chrismo/superkit","owner":"chrismo","description":"superdb library of useful functions and operations","archived":false,"fork":false,"pushed_at":"2025-08-07T21:58:56.000Z","size":13398,"stargazers_count":2,"open_issues_count":51,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-15T05:56:30.036Z","etag":null,"topics":["superdb","supersql","zq"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chrismo.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.jsup","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-12T23:54:39.000Z","updated_at":"2025-08-07T21:59:00.000Z","dependencies_parsed_at":"2025-03-23T23:23:37.369Z","dependency_job_id":"f3fd3368-c673-4fc9-a602-289515ea0156","html_url":"https://github.com/chrismo/superkit","commit_stats":null,"previous_names":["chrismo/superkit"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/chrismo/superkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrismo%2Fsuperkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrismo%2Fsuperkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrismo%2Fsuperkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrismo%2Fsuperkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chrismo","download_url":"https://codeload.github.com/chrismo/superkit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chrismo%2Fsuperkit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271448405,"owners_count":24761441,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-21T02:00:08.990Z","response_time":74,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["superdb","supersql","zq"],"created_at":"2025-08-21T08:21:27.708Z","updated_at":"2025-08-21T08:21:39.599Z","avatar_url":"https://github.com/chrismo.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SuperKit\n\nThe SuperKit library is a collection of common Functions and Operators for\n[SuperDB](https://superdb.org/). These will also work with the last released\nversion of `zq` and `zed`, version 1.18.0.\n                            \nSuperKit also contains additional docs and descriptive tests as a supplement to\nthe official documentation.\n\nhttps://github.com/chrismo/superkit/issues\n\n## Installation\n\nTo install SuperKit, run the following command:\n\n```sh\ncurl -fsS https://raw.githubusercontent.com/chrismo/superkit/main/install.sh | bash\n```\n\nThis will install the SuperKit shell scripts (`sk`, `skdoc`, `skgrok`, `skops`)\ninto the XDG bin path, which is usually `~/.local/bin`, and the remaining files\ninto the XDG data path, which is usually `~/.local/share/superkit` if you\nhaven't redefined the XDG env vars. For more info, see the [XDG Base Directory\nSpecification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).\n               \n### sk\n\n`sk` is a shell script that wraps `super` calls automatically including the\nSuperKit library of funcs and ops, of which there are few currently, but\nwe're working on it. :) \n\n```shell\n❯ sk -c '[1,20,300] | over this | sk_format_bytes(this*10000000)'  \n\"9 MB\"\n\"190 MB\"\n\"2 GB\"\n```\n\n### skdoc\n\n`skdoc` is a shell script that searches installed .md documentation for glob\nmatches of the search term (more options to come). This includes not only the\nstandard docs (`super` only at the moment, `zq` likely to be added in the\nfuture), but supplemental SuperKit docs on a (small) variety of topics.\n\n`skdoc` will look for a few recommended tools for viewing markdown files in your\nterminal, starting with\n[glow](https://github.com/charmbracelet/glow?tab=readme-ov-file#glow), then\nfalling back to\n[bat](https://github.com/sharkdp/bat?tab=readme-ov-file#installation), and\nfinally checking the `PAGER` env var or using `less` if `PAGER` isn't set.\n                                                                          \n`glow` is preferred as it has great built-in features to browse, search, and\nview the entire installed directory of documents.\n\nHere's an example to show the SuperKit doc on Subqueries:\n\n```shell\n❯ skdoc subqueries\n```\n\nWhen an argument is passed to `skdoc`, the text is globbed with asterisks on\nboth sides, and opens the first file found with the preferred viewer (again,\n`glow`, then `bat`, then `PAGER` if set, then `less`).\n\nRunning `skdoc` without any arguments will show the list of available docs in\nvarious ways, depending on the installed markdown viewer.\n\n```shell\n❯ skdoc\n```\n\nIf `glow` is installed, `glow` opens the root docs dir, and allows you to browse\nand search all the docs from that root. Otherwise, if `fzf` is installed, then\nall docs are listed in a `fzf` menu for selection, and the selected file will be\nopened with the preferred reader (`bat`, `PAGER`, or `less`). Otherwise, without\n`fzf`, all the .md files are listed in the terminal.\n\n### skgrok\n                                                         \n`skgrok` is a small, focused tool, that allows you to search through the built-in\nSuperDB grok patterns. It's best used with `fzf` for interactive searching, but\nwill also work with `less` or the `PAGER` env var set to another pager.\n\n`skgrok` takes no arguments currently, and without `fzf` you can, of course,\npass the output to `grep` or other tools.\n                \n### skops\n                 \n`skops` is a simple tool that lists a compact view of all the examples of funcs\nand ops from both of the standard SuperDB docs and the SuperKit library. This is\nuseful as a quick reference and is best used with `fzf` installed to quickly\nsearch through all examples as a reminder of how to use a particular func or op.\nIf `fzf` isn't present, the contents are paged out with `PAGER` or `less`.\n\nIf you have version 0.56.0 of `fzf` or later, `skops` will take advantage of its\nmulti-line support to show the examples in a more readable format.\n\n![screenshot of fzf multi-line](img/skops-fzf-multi-line.png)\n\n`skops` takes no arguments currently, and without `fzf` you can, of course, pass\nthe output to `grep` or other tools.\n\n# Developing SuperKit\n   \n## Repo Structure\n\nFunctions and ops can be in any number of different files, with test files\nside-by-side.\n\nThe release process produces a single combined file that will be installed as a\nsingle file to be included automatically by the `sk` script.\n\n### Includes between spq files\n\nCurrently, for distribution, it's fine to bundle all the .spq files together\ninto a single lib file that can be included once. superkit isn't too big yet,\nthere are no practical limitations.\n\nThis isn't scalable in the long haul, but there's nothing in SuperDB to manage\nincludes in .spq files, we can worry about that down the road, if SuperDB ever\nadds tooling for this.\n\nI like keeping things separate in the repo, to help prevent circular\ndependencies, but it's a minor pain to have to include the dependencies in each\nunit test script. As soon as there's a dependency introduced between, e.g.,\nstring.spq -\u003e integer.spq, then all the unit tests need that inclusion added,\neven pre-existing ones that would (in theory) still fine without it.\n\nFor now, the test script supports `// sk_include` comments in the .spq files and\nwill include these referenced files in the test run. It doesn't yet support\nrecursive `sk_includes`, but that may be added soon. Just itching one scratch at\na time right now.\n              \n## Prefer using functions over operators\n\nOperators cannot be executed in places required to be expressions, so if the\ncontents of the op or func can work as a func, make it a func.\n\nIf you can still return a single value, you can use lateral expressions (or the \n\"parenthetical ops hack\") inside a func.\n\nSee also [this draft of a doc from\nPhil](https://gist.github.com/philrz/b93f65538eb9e82e178a787988bfad5c) on User\nDefined Functions and Operators.\n                    \n## Prefer referencing `this` instead of argument in operators\n\n?? or the opposite?\n\n## All custom funcs/ops start with `sk_`\n\nThere's nothing like namespacing in SuperDB, so we prefix all custom funcs/ops\nthis way. If we didn't do this, then more commonly named ones could possibly\ncollide with additions made in SuperDB itself in the future. Having some without\nthis prefix and some with would likely be confusing, and I'd rather avoid\nwasting time deciding which items should get the prefix or not. It is a little\naesthetically unpleasing in some cases to have the prefix, but ¯\\\\_(ツ)\\_/¯\n\n## Multiple Versions\n\nEspecially with the supplemental documentation, there are variances between\n`super` and `zq`, but the goal is to encapsulate these into one set of docs,\nrather than versioning SuperKit separately for `super` and `zq`. In the future\nwe may need to do this, but for now, we'll try to keep it simple.\n\n## Testing Installation Changes on a Branch\n\nUse this command-line instead of the main installation script to test changes on\na branch:\n\n```sh\ncurl -fsS https://raw.githubusercontent.com/chrismo/superkit/refs/heads/\u003cbranch-name\u003e/install.sh |\n  RELEASE=\u003cpre-release-version\u003e bash\n```\n\n## Release Process\n\n### Prep Release\n\nIf it's a big enough release, or especially if it will have installation script\nchanges, make a new branch first to work on.\n\nOnce work is ready to be considered for release, fill out the changelog.jsup\nrecord for the new release.\n\nDo a fresh build! and then publish a new _pre_-release from the branch:\n\n```shell\nbuild  \n./release.sh pre-release\n```\n                                 \nIf you have previous pre-releases, you may want to delete them manually in\nGitHub.\n\nTo test this release, after util/launch-ubuntu.sh, install the pre-release with:\n\n```shell\n./install-sk.sh [branch name] [release version]\n```\ne.g.\n```shell\n./install-sk.sh rel 0.2.1-1f718a7\n```\n                             \nOr install it locally with util/dcp-install.sh, which will copy over the tarball\nfrom the dist folder, as well as the install.sh script, and then executes\ninstall.sh with LOCAL_INSTALL set to true.\n\n```shell\n\n_NOTE: there is a download option for a Draft release, but the download links\nare private and you have to authenticate to download them via the GitHub API\nurls, which more-or-less defeats the purpose of a full install.sh test. So you\ncan docker cp superkit.tar.gz into the container if you don't really need to\ntest the install script, or publish the release, then delete the release later\nif you need._\n          \n### Make Release\n\nOnce the pre-release is tested and ready to go, we're ready to Do It Live.\n\n- Merge the PR to main\n\n- Update the install.sh script to default to the version number of this release\n  (or do this on branch?)\n\n- Commit and push to `main`.\n\n- Re-run the build and release script.\n\n```shell\nbuild \u0026\u0026 ./release.sh release\n```\n\nReview the draft release, add the Full Changelog link to the release notes, then\npublish the release.\n\n### After Release\n\nTo prime changelog.jsup and the version string in the library, run this command.\nThis won't change the version in the install.sh script, we don't want that until\nthe next version is actually released.\n                                      \n```shell\n./release.sh bump_next_version\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrismo%2Fsuperkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchrismo%2Fsuperkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchrismo%2Fsuperkit/lists"}