{"id":31913455,"url":"https://github.com/alejandrogzi/gtfsort","last_synced_at":"2025-10-13T18:50:06.763Z","repository":{"id":190454733,"uuid":"682664986","full_name":"alejandrogzi/gtfsort","owner":"alejandrogzi","description":"a lexicographically-based GTF/GFF sorter ","archived":false,"fork":false,"pushed_at":"2025-04-27T13:10:47.000Z","size":61494,"stargazers_count":36,"open_issues_count":1,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-05T16:51:29.802Z","etag":null,"topics":["algorithms","bioinformatics","gtf","sorting","sorting-algorithms"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alejandrogzi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-08-24T16:55:38.000Z","updated_at":"2025-07-21T17:08:03.000Z","dependencies_parsed_at":"2023-08-24T19:00:50.708Z","dependency_job_id":"aec4d13c-7bc6-40dc-bbbb-79f448701f85","html_url":"https://github.com/alejandrogzi/gtfsort","commit_stats":{"total_commits":55,"total_committers":1,"mean_commits":55.0,"dds":0.0,"last_synced_commit":"bde73132e446eb0f3ada102aaa9a1df8fb120440"},"previous_names":["alejandrogzi/gtfsort"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/alejandrogzi/gtfsort","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrogzi%2Fgtfsort","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrogzi%2Fgtfsort/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrogzi%2Fgtfsort/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrogzi%2Fgtfsort/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alejandrogzi","download_url":"https://codeload.github.com/alejandrogzi/gtfsort/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alejandrogzi%2Fgtfsort/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279016622,"owners_count":26085853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["algorithms","bioinformatics","gtf","sorting","sorting-algorithms"],"created_at":"2025-10-13T18:49:28.921Z","updated_at":"2025-10-13T18:50:06.758Z","avatar_url":"https://github.com/alejandrogzi.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003e\n    gtfsort\n  \u003c/h1\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://img.shields.io/badge/version-0.1.0dev-green\" target=\"_blank\"\u003e\n      \u003cimg alt=\"Version Badge\" src=\"https://img.shields.io/badge/version-0.3.0-green\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://crates.io/crates/chaintools\" target=\"_blank\"\u003e\n      \u003cimg alt=\"Crates.io Version\" src=\"https://img.shields.io/crates/v/gtfsort\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://github.com/alejandrogzi/chaintools\" target=\"_blank\"\u003e\n      \u003cimg alt=\"GitHub License\" src=\"https://img.shields.io/github/license/alejandrogzi/gtfsort?color=blue\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://crates.io/crates/chaintools\" target=\"_blank\"\u003e\n      \u003cimg alt=\"Crates.io Total Downloads\" src=\"https://img.shields.io/crates/d/gtfsort\"\u003e\n    \u003c/a\u003e\n  \u003c/p\u003e\n\n\n  \u003cp align=\"center\"\u003e\n    An optimized chr/pos/feature GTF/GFF sorter using a lexicographically-based index ordering algorithm written in Rust.\n  \u003c/p\u003e\n\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n    \u003cimg width=700 align=\"center\" src=\"./supp/overview.png\"\u003e\n\u003c/p\u003e\n\nWhile current tools (most of them GFF3-focused) have been recommended for sorting GTF files, none are directed towards chr/pos/feature ordering. This approach ensures custom sorting directionality, which is useful for reducing computation times in tools that work with sorted GTF files. Furthermore, it provides a friendly and organized visualization of gene structures (gene -\u003e transcript -\u003e CDS/exon -\u003e start/stop -\u003e UTR/Sel), allowing users to search for features more efficiently.\n\n\u003e[!NOTE]\n\u003e \n\u003e If you use gtfsort in your work, please cite:\n\u003e \n\u003e Gonzales-Irribarren A and Fu. A. gtfsort: a tool to efficiently sort GTF files. bioRxiv 2023.10.21.563454; doi: https://doi.org/10.1101/2023.10.21.563454\n\n## Usage\n### Binary\n``` rust\nUsage: gtfsort -i \u003cGTF\u003e -o \u003cOUTPUT\u003e [-t \u003cTHREADS\u003e]\n\nArguments:\n    -i, --input \u003cGTF\u003e: unsorted GTF file\n    -o, --output \u003cOUTPUT\u003e: sorted GTF file\n\nOptions:\n    -t, --threads \u003cTHREADS\u003e: number of threads [default: your max ncpus]\n    --help: print help\n    --version: print version\n```\n\n### Library\n``` rust\nuse gtfsort::sort_annotations;\nuse num_cpus;\n\nfn main() {\n    let input = PathBuf::new(\"/path/to/unsorted.gtf\");\n    let output = PathBuf::new(\"/path/to/sorted.gtf\");\n\n    let gtfsort_job_info = sort_annotations(\n            \u0026input, \n            \u0026output,\n            num_cpus::get()\n            );\n}\n```\n### Python\nbuild the port to install it as a pkg:\n```\ngit clone https://github.com/alejandrogzi/gtfsort.git \u0026\u0026 cd gtfsort/py-gtfsort\nhatch shell\nmaturin develop --release\n```\nuse it:\n``` python3\nfrom gtfsortpy import sort\n\ninput = \"/path/to/unsorted.gtf\"\noutput = \"/path/to/sorted.gtf\"\ngtfsort_job_info = sort(input, output)\n```\n\n#### crate: [https://crates.io/crates/gtfsort](https://crates.io/crates/gtfsort)\n\n\n\u003cdetails\u003e\n\u003csummary\u003eclick for detailed formats\u003c/summary\u003e\n\u003cp\u003e\n\nGTF stands for Gene Transfer Format. The GTF format is a 9-column text format used to describe and represent genomic features. Each of the columns in a GTF file represent useful information [1]:\n\n``` plaintext\n\u003cseqname\u003e\n\n    The \u003cseqname\u003e field contains the name of the sequence which this gene is on.\n\n\u003csource\u003e\n\n    The \u003csource\u003e field should be a unique label indicating where the annotations came from – typically the name of either a prediction program or a public database.\n\n\u003cfeature\u003e\n\n    The \u003cfeature\u003e field can take 4 values: \"CDS\", \"start_codon\", \"stop_codon\" and \"exon\". The “CDS” feature represents the coding sequence starting with the first translated codon and proceeding to the last translated codon. Unlike Genbank annotation, the stop codon is not included in the “CDS” feature for the terminal exon. The “exon” feature is used to annotate all exons, including non-coding exons. The “start_codon” and “stop_codon” features should have a total length of three for any transcript but may be split onto more than one line in the rare case where an intron falls inside the codon.\n\n\u003cstart\u003e, \u003cend\u003e\n    Integer start and end coordinates of the feature relative to the beginning of the sequence named in \u003cseqname\u003e. \u003cstart\u003e must be less than or equal to \u003cend\u003e. Sequence numbering starts at 1. Values of \u003cstart\u003e and \u003cend\u003e must fall inside the sequence on which this feature resides.\n\n\u003cscore\u003e\n\n    The \u003cscore\u003e field is used to store some score for the feature. This can be any numerical value, or can be left out and replaced with a period.\n\n\u003cstrand\u003e\n\n    '+' or '-'.\n\n\u003cframe\u003e\n\n   A value of 0 indicates that the first whole codon of the reading frame is located at 5'-most base. 1 means that there is one extra base before the first whole codon and 2 means that there are two extra bases before the first whole codon. Note that the frame is not the length of the CDS mod 3. If the strand is '-', then the first base of the region is value of \u003cend\u003e, because the corresponding coding region will run from \u003cend\u003e to \u003cstart\u003e on the reverse strand.\n\n\u003cattributes\u003e\n\n    Each attribute in the \u003cattribute\u003e field should have the form: attribute_name “attribute_value”;\n    Attributes must end in a semicolon which must then be separated from the start of any subsequent attribute by exactly one space character (NOT a tab character). Attributes’ values should be surrounded by double quotes.  \n```\n\nThe GTF format has different versions, the most used ones are GTF2.5 and GTF3 (Ensembl-based structure). Each version difference from the other mainly by the feature ordering within attributes. gtfsort is designed to work with both GTF2.5 and GTF3.\n\n\n| format | ... | feature | ... | attributes |\n| --- | --- | --- | --- | --- |\n| GTF2.5 | ... | gene, transcript, exon, CDS, UTR, start_codon, stop_codon, Selenocysteine | ... |  attribute_name “attribute_value”; attribute_name “attribute_value”; |\n|  GTF3 | ... | gene, transcript, exon, CDS, Selenocysteine, start_codon, stop_codon, three_prime_utr and five_prime_utr| ... | attribute_name “attribute_value”; attribute_name “attribute_value”; | \n\n\u003c/p\u003e\n\u003c/details\u003e\n\n## Installation\nto install gtfsort on your system follow this steps:\n1. get rust: `curl https://sh.rustup.rs -sSf | sh` on unix, or go [here](https://www.rust-lang.org/tools/install) for other options\n2. run `cargo install gtfsort` (make sure `~/.cargo/bin` is in your `$PATH` before running it)\n4. use `gtfsort` with the required arguments\n\n## Build\nto build gtfsort from this repo, do:\n\n1. get rust (as described above)\n2. run `git clone https://github.com/alejandrogzi/gtfsort.git \u0026\u0026 cd gtfsort`\n3. run `cargo run --release -- -i \u003cGTF\u003e -o \u003cOUTPUT\u003e`\n\n## Container image\nto build the development container image:\n1. run `git clone https://github.com/alejandrogzi/gtfsort.git \u0026\u0026 cd gtfsort`\n2. initialize docker with `start docker` or `systemctl start docker`\n3. build the image `docker image build --tag gtfsort .`\n4. run `docker run --rm -v \"[dir_where_your_gtf_is]:/dir\" gtfsort -i /dir/\u003cINPUT\u003e -o /dir/\u003cOUTPUT\u003e`\n\n## Conda\nto use gtfsort through Conda just:\n1. `conda install gtfsort -c bioconda` or `conda create -n gtfsort -c bioconda gtfsort`\n\n\n## Benchmark\n\nTo assess the efficiency and results of gtfsort, two main benchmarks were conducted. First, I ran gtfsort over the whole Ensembl Animalia GTF3 dataset (110 release; 306 species) [2]. Here, gtfsort demonstrated both of their main attributes: speed and efficiency. **This tool is able to sort a 1.9 GB GTF file (*Cyprinus carpio carpio*) in 3 seconds with high accuracy using less than 1 GB of RAM**. Species of common interest are highlighted. \n\nSecondly, I conducted a comparative analysis of gtfsort in relation to several existing software tools: GNU v.8.25 (both in single and multi-core configurations), AGAT (utilizing the --gff flag for both complete and partial parsing phases) [3], gff3sort (with specific options, including --precise and --chr_order natural) [4], and rsort (an unpublished multi-core Rust implementation with nested data structures). This benchmark included different biological domains, spanning bacteria, fungi, insects, mammals, and more. To ensure a robust assessment, I employed nine common reference species: *Homo sapiens*, *Mus musculus*, *Canis lupus familiaris*, *Gallus gallus*, *Danio rerio*, *Salmo salar*, *Crocodylus porosus*, *Drosophila melanogaster* and *Saccharomyces cerevisiae*. \n\nIn this comparative analysis, gtfsort demonstrated remarkable efficiency, producing the shortest computation time along with the GNU software (in both single and multi-core modes). It is worth noting, however, that GNU software fails to consistently maintain a stable chromosome/position/feature order and encounters difficulties when sorting commented lines (e.g., lines commencing with \"#\" at the beginning of the file). The remaining tools exhibited substantially longer processing times, with some employing parallel processing approaches (like rsort, which utilized 16 cores).\n\nFurthermore, it is noteworthy that the memory allocation required for sorting files remained conservative in three of the tools evaluated: GNU (both single and multi-core), gff3sort, and gtfsort. The memory utilization for the largest file did not exceed 2.3 Gbs, even when handling substantial datasets (up to 1.6 Gbs in size).\n\n\u003cp align=\"center\"\u003e\n    \u003cimg align=center width=600 src=\"./supp/benchmark.png\"\u003e\n\u003c/p\u003e\n\nFrom the suite of tools employed in the preceding step, only three assert to incorporate a feature sorting step [5]: gff3sort, AGAT, and gtfsort. Gff3sort, a Perl-based program tailored for sorting GFF3/GTF files, is adept at generating results compatible with tabix tools [4]. It employs a topological algorithm to sequence features after an initial two-block sorting phase (first by chromosome, then by position). AGAT, an analysis toolkit also scripted in Perl, features a GFF3/GTF sorting tool within the `agat_convert_sp_gxf2gxf.pl` script [3], likewise employing a topological sorting approach.\n\nTo assess the performance of these three tools, we subjected them to the GRCh38 *Homo sapiens* GTF file from the latest Ensembl release (110 release). Among the software tested, gtfsort emerged as the fastest, with a processing time of **1.71 seconds**, followed by gff3sort at 16.3970 seconds, and AGAT, which required approximately 900 seconds to complete the sorting operation. The notorious difference with the extensive computation time of AGAT is due to the fact that `agat_convert_sp_gxf2gxf.pl` does not only sort a GTF file but inspects some controversial lines and fixes/adds corrected/missing lines.\n\nAlthough computation time is an important feature, the actual sorting output would be the key variable to compare. I choose a random gene (including all its transcripts/features) and tested whether the output ordering demonstrated a coherent and accurate layout.\n\n- Chromosomal ordering: Only gtfsort and gff3sort presented an intuitive ordering (starting with chromosome 1 and ending with chromosome X). AGAT fails here, locating MT and sex chromosomes at the beginning.\n- Feature ordering: gff3sort (--precise --chr_order natural) completely fails to present an ordered structure of features (something that is quickly perceived by the exon 5 of the first transcript at the beginning of the block). AGAT and gtfsort, conversely, do exhibit an intuitive structure order: gene -\u003e transcript -\u003e features. AGAT presents 2 blocks per transcript, all CDS after all exons with start/stop codons and UTRs at the end. gtfsort, on the other hand, adopted a distinct approach presenting pairs or triplets of features in conjunction with their respective exon numbers, sorted in descending order, even for sequences on negative strands. UTRs were consistently positioned at the conclusion of the sequence, enabling a natural and rapid comprehension of the information associated with a given exon (exon/CDS/start/stop).\n\n\u003cp align=\"center\"\u003e\n    \u003cimg align=center height=750 src=\"./supp/order.png\"\u003e\n\u003c/p\u003e\n\n- All the values presented herein represent the average of ten consecutive iterations for each species, encompassing both time and memory usage.\n\n- In light of the notably extended computation times associated with AGAT-complete and AGAT-parse, we have expressed the time values for these tools in their decimal form (divided by 10) for enhanced clarity during visualization.\n\n- All benchmark assessments were conducted on an AMD Ryzen 7 5700X with 128 GB of RAM.\n\n## References\n1. https://agat.readthedocs.io/en/latest/gxf.html\n2. https://www.ensembl.org/index.html\n3. https://github.com/NBISweden/AGAT\n4. https://github.com/billzt/gff3sort\n5. https://www.biostars.org/p/306859/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falejandrogzi%2Fgtfsort","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falejandrogzi%2Fgtfsort","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falejandrogzi%2Fgtfsort/lists"}