{"id":21429585,"url":"https://github.com/mayank-02/msort","last_synced_at":"2025-07-14T11:30:42.490Z","repository":{"id":53556669,"uuid":"272980869","full_name":"mayank-02/msort","owner":"mayank-02","description":"Sort lines of text files","archived":false,"fork":false,"pushed_at":"2021-03-28T06:09:03.000Z","size":2852,"stargazers_count":4,"open_issues_count":5,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2023-08-15T06:38:34.401Z","etag":null,"topics":["automake","external-sorting","linux","linux-sort","merge-sort","msort","sort-lines","utility"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mayank-02.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-17T13:26:39.000Z","updated_at":"2023-08-15T06:38:34.402Z","dependencies_parsed_at":"2022-09-21T10:13:22.234Z","dependency_job_id":null,"html_url":"https://github.com/mayank-02/msort","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fmsort","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fmsort/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fmsort/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mayank-02%2Fmsort/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mayank-02","download_url":"https://codeload.github.com/mayank-02/msort/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225971250,"owners_count":17553461,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automake","external-sorting","linux","linux-sort","merge-sort","msort","sort-lines","utility"],"created_at":"2024-11-22T22:18:23.145Z","updated_at":"2024-11-22T22:18:23.948Z","avatar_url":"https://github.com/mayank-02.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\u003cimg width=12.5% src=\"./doc/logo.png\"\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\u003cimg width=25% src=\"./doc/logo-msort.png\"\u003e\u003c/p\u003e\n\n\u003cp align=\"center\u003e\n\n[![GitHub issues](https://img.shields.io/github/issues/mayank-02/msort)](https://github.com/mayank-02/msort)\n![Contributions welcome](https://img.shields.io/badge/contributions-welcome-green.svg)\n[![License](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT)\n\n\u003c/p\u003e\n\nmsort is a command line utility which can :\n\n1. Sort lines of all the named files together and write the result to specified output.\n2. Merge lines of all presorted files together and write the result to specified output.\n\nIt uses external sorting mechanism to sort files of **any** size.\n\n## Installation\n\nFollow following steps to install msort in your system\n\n```bash\n# Run all autotools in the right order\n\u003e  autoreconf --install\n\n# Generate Makefile from Makefile.in\n\u003e ./configure\n\n# Use Makefile to build the program\n\u003e make\n\n# Use Makefile to install the program\n\u003e sudo make install\n```\n\nYou can uninstall msort using\n\n```bash\n# Use Makefile to uninstall the program\n\u003e sudo make uninstall\n```\n\n## Manual Page\n\nOnce installed, to check out the manual page for msort, do\n\n```bash\n\u003e man msort\n```\n\n## Options supported\n\n```\n-m\nMerge only; the input file shall be assumed to be already sorted.\n\n-o output\nSpecify the name of an output file to be used instead of the standard output.\n\n-d\nSpecify that only \u003cblank\u003e characters and alphanumeric characters, shall be\nsignificant in comparisons. The behavior is undefined for a sort key to which\n-i or -n also applies.\n\n-f\nConsider all uppercase characters that have lowercase equivalents, to be the\nlowercase equivalent for the purposes of comparison.\n\n-i\nIgnore all characters that are non-printable.\n\n-n\nRestrict the sort key to an initial numeric string, consisting of optional\n\u003cblank\u003e characters, optional \u003chyphen-minus\u003e character, and zero or more digits\nwith an optional radix character, which shall be sorted by arithmetic value.\nAn empty digit string shall be treated as zero. Leading zeros and signs on\nzeros shall not affect ordering.\n\n-r\nReverse the sense of comparisons.\n\n-b\nIgnore leading \u003cblank\u003e characters.\n\n-t  char\nUse char as the field separator character. Each occurrence of char shall be\nsignificant (for example, \u003cchar\u003e\u003cchar\u003e delimits an empty field). If -t is not\nspecified, \u003cblank\u003e character shall be used as default field separators.\n\n-k  field_start[,field_end]\nThe keydef argument is a restricted to field_start and field_end which define a\nkey field restricted to a portion of the line. If no -k option is specified,\na default sort key of the entire line shall be used.\n\nNote : All byte sequences conform to POSIX locale.\n```\n\nOperands :\n\n```\nfile\nA pathname of a file(text files only) to be sorted or merged. If no file operands are specified, the standard input shall be used. If sort encounters an error when opening or reading a file operand, it may exit by writing an error to standard output without processing later operands.\n```\n\n## Usage\n\n```bash\n\u003e msort -bdf infile.txt\n\nSorts infile.txt ignoring leading spaces and case, according to dictionary order and then outputs the result on the screen\n```\n\n```bash\n\u003e msort -nru -k2,2 infile.txt -o outfile.txt\n\nSorts infile.txt according to 2nd column, numerically and in reverse order.\nAlso, all duplicate lines are supressed. Result is stored in outfile.txt.\n```\n\n```bash\n\u003e msort -k4.2,4.8 -t, infile.txt\n\nSorts infile.txt according contents between 2nd character of 4th \u003c,\u003e separated\nfield and 8th character of 4th \u003c,\u003e separated field.\n```\n\n```bash\n\u003e msort -m infile1.txt infile2.txt\n\nOnly merges both the files and outputs to stdout.\n```\n\nNote: Use LC_ALL = C before using linux sort to match outputs\n\n```bash\n\u003e LC_ALL = C sort -k4.2,4.8 -t, infile.txt\n```\n\n## Implementation\n\nTo sort user input or files of any size, **external sorting** was implemented. It\nwas done in 3 stages :\n\n1. Making Runs : Breaking the input into small blocks having size less than size of internal memory using a single buffer.\n2. Sorting the Runs : Using quicksort sorting algorithm to sort the contents of the buffer depending on the data type and options. The\nbuffer was then output to a file stored temporarily on the hard\ndisk.\n3. Merging Sorted Runs : A merge utility was used which uses 2-way merge. Sorted runs created after Step 2 were passed as arguments and the output files at each stage were later again passed to it until only one file remained.\n\n**Handling options :**\n\n+ To handle the various options, data was classified as string or numeric.\n+ Options were handled using the getopt() function.\n+ Two comparator functions were made, namely strcompare() and numcompare().\n+ Each function returned an integer based on which qsort() ordered the strings.\n+ The original strings weren't modified at all.\n+ Comparisons were made based on key specified (entire line if not). They were\ndone byte by byte using the collating sequence for POSIX locale (LC_ALL = C)\nto avoid ambiguity of results on different machines.\n\n**Undestanding automake:**\n\n![Understanding automake](./doc/automake.png)\n\nOn the maintainer’s system:\n\n```bash\n# Set up an m4 environment\naclocal\n\n# Generate configure from configure.ac\nautoconf\n\n# Generate Makefile.in from Makefile.am\nautomake --add-missing\n\n# Generate Makefile from Makefile.in\n./configure\n\n# Use Makefile to build and test a tarball to distribute\nmake distcheck\n```\n\nOn the end-user’s system:\n\n```bash\n # Generate Makefile from Makefile.in\n./configure\n\n# Use Makefile to build the program\nmake\n\n# Use Makefile to install the program\nsudo make install\n```\n\n## References\n\n1. To understand how sort and it's various options work :\n   + [GNU sort](https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html)\n   + [Open Group Base Specifications sort](http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html)\n   + [Man page](http://man7.org/linux/man-pages/man1/sort.1.html)\n   + ['sort' command - 2 minute tutorial](http://2min2code.com/articles/sort_command_intro/)\n   + [Linux sort command](https://www.computerhope.com/unix/usort.htm)\n\n2. Understanding External Sorting :\n   + [External Sorting](https://cis.stvincent.edu/html/tutorials/swd/extsort/extsort.html)\n   + [CS302 --- External Sorting](http://web.eecs.utk.edu/~leparker/Courses/CS302-Fall06/Notes/external-sorting2.html)\n\n3. [How locale affects sort?](http://pubs.opengroup.org/onlinepubs/9699919799/nframe.html)\n\n4. [Using getopt() function](https://www.cs.rutgers.edu/~pxk/416/notes/c-tutorials/getopt.html)\n\n5. [Miscellaneous help](https://stackoverflow.com/)\n\n6. [Automake](http://mij.oltrelinux.com/devel/autoconf-automake/)\n\n7. [Autotools Tutorial](http://mij.oltrelinux.com/devel/autoconf-automake/)\n\n8. Books :\n   + The C Programming Language by Kernighan and Ritchie :\n   Referred to sorting algorithms like quicksort and also on how to write comparator functions for qsort().\n   + Fundamentals of Data Structures by Ellis Horowitz and Sartaj Sahni : Referred for external sorting and various sorting algorithms.\n\n## Authors\n\n[Mayank Jain](https://github.com/mayank-02)\n\n## Contributing\n\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.\n\nPlease make sure to update tests as appropriate.\n\nTo test msort, you can run the tests as follows\n\n```bash\n# Make sure msort has been installed first\ncd tests/\n./tests.sh\n```\n\n## License\n\n[MIT](https://choosealicense.com/licenses/mit/)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayank-02%2Fmsort","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmayank-02%2Fmsort","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmayank-02%2Fmsort/lists"}