{"id":13434578,"url":"https://github.com/attractivechaos/klib","last_synced_at":"2025-05-13T19:13:34.265Z","repository":{"id":1308768,"uuid":"1251393","full_name":"attractivechaos/klib","owner":"attractivechaos","description":"A standalone and lightweight C library","archived":false,"fork":false,"pushed_at":"2025-02-05T04:16:43.000Z","size":1275,"stargazers_count":4369,"open_issues_count":91,"forks_count":574,"subscribers_count":190,"default_branch":"master","last_synced_at":"2025-04-27T20:00:06.324Z","etag":null,"topics":["algorithm","avl-tree","b-tree","c","generic","hashtable","library","sort"],"latest_commit_sha":null,"homepage":"http://attractivechaos.github.io/klib/","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/attractivechaos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2011-01-13T17:06:02.000Z","updated_at":"2025-04-26T17:49:45.000Z","dependencies_parsed_at":"2023-07-06T21:46:52.636Z","dependency_job_id":"ee37c1c2-2f8b-4f80-bac1-233354d9a3e6","html_url":"https://github.com/attractivechaos/klib","commit_stats":{"total_commits":348,"total_committers":21,"mean_commits":"16.571428571428573","dds":0.4683908045977011,"last_synced_commit":"9a063b33efd841fcc42d4b9f68cb78bb528bf75b"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attractivechaos%2Fklib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attractivechaos%2Fklib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attractivechaos%2Fklib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attractivechaos%2Fklib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/attractivechaos","download_url":"https://codeload.github.com/attractivechaos/klib/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254010813,"owners_count":21998995,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithm","avl-tree","b-tree","c","generic","hashtable","library","sort"],"created_at":"2024-07-31T03:00:18.068Z","updated_at":"2025-05-13T19:13:34.214Z","avatar_url":"https://github.com/attractivechaos.png","language":"C","funding_links":[],"categories":["Xposed","Miscellaneous","C","C/C++ 程序设计","排序","Standard Libraries \u0026 Utility Libraries"],"sub_categories":["Utility","网络服务_其他","多项混杂"],"readme":"# Klib: a Generic Library in C\n\n## \u003ca name=\"overview\"\u003e\u003c/a\u003eOverview\n\nKlib is a standalone and lightweight C library distributed under [MIT/X11\nlicense][1]. Most components are independent of external libraries, except the\nstandard C library, and independent of each other. To use a component of this\nlibrary, you only need to copy a couple of files to your source code tree\nwithout worrying about library dependencies.\n\nKlib strives for efficiency and a small memory footprint. Some components, such\nas khash.h, kbtree.h, ksort.h and kvec.h, are among the most efficient\nimplementations of similar algorithms or data structures in all programming\nlanguages, in terms of both speed and memory use.\n\nA new documentation is available [here](http://attractivechaos.github.io/klib/)\nwhich includes most information in this README file.\n\n#### Common components\n\n* [khash.h][khash]: generic [hash table][2] with open addressing.\n* [kbtree.h][kbtree]: generic search tree based on [B-tree][3].\n* [kavl.h][kavl]: generic intrusive [AVL tree][wiki-avl].\n* [ksort.h][ksort]: generic sort, including [introsort][4], [merge sort][5], [heap sort][6], [comb sort][7], [Knuth shuffle][8] and the [k-small][9] algorithm.\n* [kseq.h][kseq]: generic stream buffer and a [FASTA][10]/[FASTQ][11] format parser.\n* kvec.h: generic dynamic array.\n* klist.h: generic single-linked list and [memory pool][12].\n* kstring.{h,c}: basic string library.\n* kmath.{h,c}: numerical routines including [MT19937-64][13] [pseudorandom generator][14], basic [nonlinear programming][15] and a few special math functions.\n* [ketopt.h][ketopt]: portable command-line argument parser with getopt\\_long-like API.\n\n#### Components for more specific use cases\n\n* ksa.c: constructing [suffix arrays][16] for strings with multiple sentinels, based on a revised [SAIS algorithm][17].\n* knetfile.{h,c}: random access to remote files on HTTP or FTP.\n* kopen.c: smart stream opening.\n* khmm.{h,c}: basic [HMM][18] library.\n* ksw.(h,c}: Striped [Smith-Waterman algorithm][19].\n* knhx.{h,c}: [Newick tree format][20] parser.\n\n\n## \u003ca name=\"methodology\"\u003e\u003c/a\u003eMethodology\n\nFor the implementation of generic [containers][21], klib extensively uses C\nmacros. To use these data structures, we usually need to instantiate methods by\nexpanding a long macro. This makes the source code look unusual or even ugly\nand adds difficulty to debugging. Unfortunately, for efficient generic\nprogramming in C that lacks [template][22], using macros is the only\nsolution. Only with macros, we can write a generic container which, once\ninstantiated, compete with a type-specific container in efficiency. Some\ngeneric libraries in C, such as [Glib][23], use the `void*` type to implement\ncontainers. These implementations are usually slower and use more memory than\nklib (see [this benchmark][31]).\n\nTo effectively use klib, it is important to understand how it achieves generic\nprogramming. We will use the hash table library as an example:\n\n    #include \"khash.h\"\n    KHASH_MAP_INIT_INT(m32, char)        // instantiate structs and methods\n    int main() {\n        int ret, is_missing;\n        khint_t k;\n        khash_t(m32) *h = kh_init(m32);  // allocate a hash table\n        k = kh_put(m32, h, 5, \u0026ret);     // insert a key to the hash table\n        if (!ret) kh_del(m32, h, k);\n        kh_value(h, k) = 10;             // set the value\n        k = kh_get(m32, h, 10);          // query the hash table\n        is_missing = (k == kh_end(h));   // test if the key is present\n        k = kh_get(m32, h, 5);\n        kh_del(m32, h, k);               // remove a key-value pair\n        for (k = kh_begin(h); k != kh_end(h); ++k)  // traverse\n            if (kh_exist(h, k))          // test if a bucket contains data\n    \t\t\tkh_value(h, k) = 1;\n        kh_destroy(m32, h);              // deallocate the hash table\n        return 0;\n    }\n\nIn this example, the second line instantiates a hash table with `unsigned` as\nthe key type and `char` as the value type. `m32` names such a type of hash table.\nAll types and functions associated with this name are macros, which will be\nexplained later. Macro `kh_init()` initiates a hash table and `kh_destroy()`\nfrees it. `kh_put()` inserts a key and returns the iterator (or the position)\nin the hash table. `kh_get()` and `kh_del()` get a key and delete an element,\nrespectively. Macro `kh_exist()` tests if an iterator (or a position) is filled\nwith data.\n\nAn immediate question is this piece of code does not look like a valid C\nprogram (e.g. lacking semicolon, assignment to an _apparent_ function call and\n_apparent_ undefined `m32` 'variable'). To understand why the code is correct,\nlet's go a bit further into the source code of `khash.h`, whose skeleton looks\nlike:\n\n    #define KHASH_INIT(name, SCOPE, key_t, val_t, is_map, _hashf, _hasheq) \\\n      typedef struct { \\\n        int n_buckets, size, n_occupied, upper_bound; \\\n        unsigned *flags; \\\n        key_t *keys; \\\n        val_t *vals; \\\n      } kh_##name##_t; \\\n      SCOPE inline kh_##name##_t *init_##name() { \\\n        return (kh_##name##_t*)calloc(1, sizeof(kh_##name##_t)); \\\n      } \\\n      SCOPE inline int get_##name(kh_##name##_t *h, key_t k) \\\n      ... \\\n      SCOPE inline void destroy_##name(kh_##name##_t *h) { \\\n        if (h) { \\\n          free(h-\u003ekeys); free(h-\u003eflags); free(h-\u003evals); free(h); \\\n        } \\\n      }\n    \n    #define _int_hf(key) (unsigned)(key)\n    #define _int_heq(a, b) (a == b)\n    #define khash_t(name) kh_##name##_t\n    #define kh_value(h, k) ((h)-\u003evals[k])\n    #define kh_begin(h, k) 0\n    #define kh_end(h) ((h)-\u003en_buckets)\n    #define kh_init(name) init_##name()\n    #define kh_get(name, h, k) get_##name(h, k)\n    #define kh_destroy(name, h) destroy_##name(h)\n    ...\n    #define KHASH_MAP_INIT_INT(name, val_t) \\\n    \tKHASH_INIT(name, static, unsigned, val_t, is_map, _int_hf, _int_heq)\n\n`KHASH_INIT()` is a huge macro defining all the structs and methods. When this\nmacro is called, all the code inside it will be inserted by the [C\npreprocess][37] to the place where it is called. If the macro is called\nmultiple times, multiple copies of the code will be inserted. To avoid naming\nconflict of hash tables with different key-value types, the library uses [token\nconcatenation][36], which is a preprocessor feature whereby we can substitute\npart of a symbol based on the parameter of the macro. In the end, the C\npreprocessor will generate the following code and feed it to the compiler\n(macro `kh_exist(h,k)` is a little complex and not expanded for simplicity):\n\n    typedef struct {\n      int n_buckets, size, n_occupied, upper_bound;\n      unsigned *flags;\n      unsigned *keys;\n      char *vals;\n    } kh_m32_t;\n    static inline kh_m32_t *init_m32() {\n      return (kh_m32_t*)calloc(1, sizeof(kh_m32_t));\n    }\n    static inline int get_m32(kh_m32_t *h, unsigned k)\n    ...\n    static inline void destroy_m32(kh_m32_t *h) {\n      if (h) {\n        free(h-\u003ekeys); free(h-\u003eflags); free(h-\u003evals); free(h);\n      }\n    }\n\n\tint main() {\n\t\tint ret, is_missing;\n\t\tkhint_t k;\n\t\tkh_m32_t *h = init_m32();\n\t\tk = put_m32(h, 5, \u0026ret);\n\t\tif (!ret) del_m32(h, k);\n\t\th-\u003evals[k] = 10;\n\t\tk = get_m32(h, 10);\n\t\tis_missing = (k == h-\u003en_buckets);\n\t\tk = get_m32(h, 5);\n\t\tdel_m32(h, k);\n\t\tfor (k = 0; k != h-\u003en_buckets; ++k)\n\t\t\tif (kh_exist(h, k)) h-\u003evals[k] = 1;\n\t\tdestroy_m32(h);\n\t\treturn 0;\n\t}\n\nThis is the C program we know.\n\nFrom this example, we can see that macros and the C preprocessor plays a key\nrole in klib. Klib is fast partly because the compiler knows the key-value\ntype at the compile time and is able to optimize the code to the same level\nas type-specific code. A generic library written with `void*` will not get such\nperformance boost.\n\nMassively inserting code upon instantiation may remind us of C++'s slow\ncompiling speed and huge binary size when STL/boost is in use. Klib is much\nbetter in this respect due to its small code size and component independency.\nInserting several hundreds lines of code won't make compiling obviously slower.\n\n## \u003ca name=\"resources\"\u003e\u003c/a\u003eResources\n\n* Library documentation, if present, is available in the header files. Examples\ncan be found in the [test/][24] directory.\n* **Obsolete** documentation of the hash table library can be found at\n[SourceForge][25]. This README is partly adapted from the old documentation.\n* [Blog post][26] describing the hash table library.\n* [Blog post][27] on why using `void*` for generic programming may be inefficient.\n* [Blog post][28] on the generic stream buffer.\n* [Blog post][29] evaluating the performance of `kvec.h`.\n* [Blog post][30] arguing B-tree may be a better data structure than a binary search tree.\n* [Blog post][31] evaluating the performance of `khash.h` and `kbtree.h` among many other implementations.\n[An older version][33] of the benchmark is also available.\n* [Blog post][34] benchmarking internal sorting algorithms and implementations.\n* [Blog post][32] on the k-small algorithm.\n* [Blog post][35] on the Hooke-Jeeve's algorithm for nonlinear programming.\n\n[1]: http://en.wikipedia.org/wiki/MIT_License\n[2]: https://en.wikipedia.org/wiki/Hash_table\n[3]: http://en.wikipedia.org/wiki/B-tree\n[4]: http://en.wikipedia.org/wiki/Introsort\n[5]: http://en.wikipedia.org/wiki/Merge_sort\n[6]: http://en.wikipedia.org/wiki/Heapsort\n[7]: http://en.wikipedia.org/wiki/Comb_sort\n[8]: http://en.wikipedia.org/wiki/Fisher-Yates_shuffle\n[9]: http://en.wikipedia.org/wiki/Selection_algorithm\n[10]: http://en.wikipedia.org/wiki/FASTA_format\n[11]: http://en.wikipedia.org/wiki/FASTQ_format\n[12]: http://en.wikipedia.org/wiki/Memory_pool\n[13]: http://en.wikipedia.org/wiki/Mersenne_twister\n[14]: http://en.wikipedia.org/wiki/Pseudorandom_generator\n[15]: http://en.wikipedia.org/wiki/Nonlinear_programming\n[16]: http://en.wikipedia.org/wiki/Suffix_array\n[17]: https://sites.google.com/site/yuta256/sais\n[18]: http://en.wikipedia.org/wiki/Hidden_Markov_model\n[19]: http://en.wikipedia.org/wiki/Smith-Waterman_algorithm\n[20]: http://en.wikipedia.org/wiki/Newick_format\n[21]: http://en.wikipedia.org/wiki/Container_(abstract_data_type)\n[22]: http://en.wikipedia.org/wiki/Template_(C%2B%2B)\n[23]: http://en.wikipedia.org/wiki/GLib\n[24]: https://github.com/attractivechaos/klib/tree/master/test\n[25]: http://klib.sourceforge.net/\n[26]: http://attractivechaos.wordpress.com/2008/09/02/implementing-generic-hash-library-in-c/\n[27]: http://attractivechaos.wordpress.com/2008/10/02/using-void-in-generic-c-programming-may-be-inefficient/\n[28]: http://attractivechaos.wordpress.com/2008/10/11/a-generic-buffered-stream-wrapper/\n[29]: http://attractivechaos.wordpress.com/2008/09/19/c-array-vs-c-vector/\n[30]: http://attractivechaos.wordpress.com/2008/09/24/b-tree-vs-binary-search-tree/\n[31]: http://attractivechaos.wordpress.com/2008/10/07/another-look-at-my-old-benchmark/\n[32]: http://attractivechaos.wordpress.com/2008/09/13/calculating-median/\n[33]: http://attractivechaos.wordpress.com/2008/08/28/comparison-of-hash-table-libraries/\n[34]: http://attractivechaos.wordpress.com/2008/08/28/comparison-of-internal-sorting-algorithms/\n[35]: http://attractivechaos.wordpress.com/2008/08/24/derivative-free-optimization-dfo/\n[36]: http://en.wikipedia.org/wiki/C_preprocessor#Token_concatenation\n[37]: http://en.wikipedia.org/wiki/C_preprocessor\n\n[wiki-avl]: https://en.wikipedia.org/wiki/AVL_tree\n\n[kbtree]: http://attractivechaos.github.io/klib/#KBtree%3A%20generic%20ordered%20map:%5B%5BKBtree%3A%20generic%20ordered%20map%5D%5D\n[khash]: http://attractivechaos.github.io/klib/#Khash%3A%20generic%20hash%20table:%5B%5BKhash%3A%20generic%20hash%20table%5D%5D\n[kseq]: http://attractivechaos.github.io/klib/#Kseq%3A%20stream%20buffer%20and%20FASTA%2FQ%20parser:%5B%5BKseq%3A%20stream%20buffer%20and%20FASTA%2FQ%20parser%5D%5D\n[ksort]: http://attractivechaos.github.io/klib/#Ksort%3A%20sorting%2C%20shuffling%2C%20heap%20and%20k-small:%5B%5BKsort%3A%20sorting%2C%20shuffling%2C%20heap%20and%20k-small%5D%5D\n[kavl]: http://attractivechaos.github.io/klib/#KAVL%3A%20generic%20intrusive%20AVL%20tree\n[ketopt]: http://attractivechaos.github.io/klib/#Ketopt%3A%20parsing%20command-line%20arguments\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattractivechaos%2Fklib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fattractivechaos%2Fklib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fattractivechaos%2Fklib/lists"}