{"id":16884328,"url":"https://github.com/simolus3/tar","last_synced_at":"2025-05-08T21:06:21.276Z","repository":{"id":43776140,"uuid":"324171835","full_name":"simolus3/tar","owner":"simolus3","description":"Memory-efficient, streaming implementation of the tar archive format in Dart","archived":false,"fork":false,"pushed_at":"2024-10-05T11:33:59.000Z","size":1369,"stargazers_count":27,"open_issues_count":2,"forks_count":9,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-02T02:54:22.792Z","etag":null,"topics":["archive","dart","io","tar"],"latest_commit_sha":null,"homepage":"","language":"Dart","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simolus3.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-24T14:17:14.000Z","updated_at":"2024-10-05T11:34:03.000Z","dependencies_parsed_at":"2024-05-14T13:43:39.453Z","dependency_job_id":"8410727b-c12a-460e-8959-08eefd7f21ab","html_url":"https://github.com/simolus3/tar","commit_stats":{"total_commits":99,"total_committers":5,"mean_commits":19.8,"dds":0.06060606060606055,"last_synced_commit":"b62573f39a4de28f69d9ed82b02fbd96b12b9633"},"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simolus3%2Ftar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simolus3%2Ftar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simolus3%2Ftar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simolus3%2Ftar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simolus3","download_url":"https://codeload.github.com/simolus3/tar/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238044093,"owners_count":19407128,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archive","dart","io","tar"],"created_at":"2024-10-13T16:17:21.679Z","updated_at":"2025-02-10T02:04:23.019Z","avatar_url":"https://github.com/simolus3.png","language":"Dart","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tar\n\n![Build status](https://github.com/simolus3/tar/workflows/build/badge.svg)\n\nThis package provides stream-based readers and writers for tar files.\n\nWhen working with large tar files, this library consumes considerably less memory\nthan [package:archive](https://pub.dev/packages/archive), although it is slightly slower due to the async overhead.\n\n## Reading\n\nTo read entries from a tar file, use a `TarReader` with a `Stream` emitting bytes (as `List\u003cint\u003e`):\n\n```dart\nimport 'dart:convert';\nimport 'dart:io';\nimport 'package:tar/tar.dart';\n\nFuture\u003cvoid\u003e main() async {\n  final reader = TarReader(File('file.tar').openRead());\n\n  while (await reader.moveNext()) {\n    final entry = reader.current;\n    // Use reader.header to see the header of the current tar entry\n    print(entry.header.name);\n    // And reader.contents to read the content of the current entry as a stream\n    print(await entry.contents.transform(utf8.decoder).first);\n  }\n  // Note that the reader will automatically close if moveNext() returns false or\n  // throws. If you want to close a tar stream before that happens, use\n  // reader.cancel();\n}\n```\n\nTo read `.tar.gz` files, transform the stream with `gzip.decoder` before\npassing it to the `TarReader`.\n\nTo easily go through all entries in a tar file, use `TarReader.forEach`:\n\n```dart\nFuture\u003cvoid\u003e main() async {\n  final inputStream = File('file.tar').openRead();\n\n  await TarReader.forEach(inputStream, (entry) {\n    print(header.name);\n    print(await entry.contents.transform(utf8.decoder).first);\n  });\n}\n```\n\n__Warning__: Since the reader is backed by a single stream, concurrent calls to\n`read` are not allowed! Similarly, if you're reading from an entry's `contents`,\nmake sure to fully drain the stream before calling `read()` again.\n_Not_ subscribing to `contents` before calling `moveNext()` is acceptable too.\nIn this case, the reader will implicitly drain the stream.\nThe reader detects concurrency misuses and will throw an error when they occur,\nthere's no risk of reading faulty data.\n\n## Writing\n\nWhen writing archives, `package:tar` expects a `Stream` of tar entries to include in\nthe archive.\nThis stream can then be converted into a stream of byte-array chunks forming the\nencoded tar archive.\n\nTo write a tar stream into a `StreamSink\u003cList\u003cint\u003e\u003e`, such as an `IOSink` returned by\n`File.openWrite`, use `tarWritingSink`:\n\n```dart\nimport 'dart:convert';\nimport 'dart:io';\nimport 'package:tar/tar.dart';\n\nFuture\u003cvoid\u003e main() async {\n  final output = File('test.tar').openWrite();\n  final tarEntries = Stream\u003cTarEntry\u003e.value(\n    TarEntry.data(\n      TarHeader(\n        name: 'hello.txt',\n        mode: int.parse('644', radix: 8),\n      ),\n      utf8.encode('Hello world'),\n    ),\n  );\n\n  await tarEntries.pipe(tarWritingSink(output));\n}\n```\n\nFor more complex stream transformations, `tarWriter` can be used as a stream\ntransformer converting a stream of tar entries into archive bytes.\n\nTogether with the `gzip.encoder` transformer from `dart:io`, this can be used\nto write a `.tar.gz` file:\n\n```dart\nimport 'dart:io';\nimport 'package:tar/tar.dart';\n\nFuture\u003cvoid\u003e write(Stream\u003cTarEntry\u003e entries) {\n  return entries\n      .transform(tarWriter) // convert entries into a .tar stream\n      .transform(gzip.encoder) // convert the .tar stream into a .tar.gz stream\n      .pipe(File('output.tar.gz').openWrite());\n}\n```\n\nA more complex example for writing files can be found in [`example/archive_self.dart`](example/archive_self.dart).\n\n### Encoding options\n\nBy default, tar files are  written in the pax format defined by the\nPOSIX.1-2001 specification (`--format=posix` in GNU tar).\nWhen all entries have file names shorter than 100 chars and a size smaller\nthan 8 GB, this is equivalent to the `ustar` format. This library won't write\nPAX headers when there is no reason to do so.\nIf you prefer writing GNU-style long filenames instead, you can use the\n`format` option:\n\n```dart\nFuture\u003cvoid\u003e write(Stream\u003cTarEntry\u003e entries) {\n  return entries\n      .pipe(\n        tarWritingSink(\n          File('output.tar').openWrite(),\n          format: OutputFormat.gnuLongName,\n      ));\n}\n```\n\nTo change the output format on the `tarWriter` transformer, use\n`tarWriterWith`.\n\n### Synchronous writing\n\nAs the content of tar entries is defined as an asynchronous stream, the tar encoder is asynchronous too.\nThe more specific `SynchronousTarEntry` class stores tar content as a list of bytes, meaning that it can be\nwritten synchronously.\n\nTo synchronously write tar files, use `tarConverter` (or `tarConverterWith` for options):\n\n```dart\nList\u003cint\u003e createTarArchive(Iterable\u003cSynchronousTarEntry\u003e entries) {\n  late List\u003cint\u003e result;\n  final sink = ByteConversionSink.withCallback((data) =\u003e result = data);\n\n  final output = tarConverter.startChunkedConversion(sink);\n  entries.forEach(output.add);\n  output.close();\n\n  return result;\n}\n```\n\n## Features\n\n- Supports v7, ustar, pax, gnu and star archives\n- Supports extended pax headers for long file or link names\n- Supports long file and link names generated by GNU-tar\n- Hardened against denial-of-service attacks with invalid tar files\n- Supports being compiled to JavaScript, tested on Node.js\n\n## Security considerations\n\nInternally, this package contains checks to guard against some invalid tar files.\nIn particular,\n\n- The reader doesn't allocate memory based on values in a tar file (so there's\n  a guard against DoS attacks with tar files containing huge headers).\n- When encountering malformed tar files, the reader will throw a `TarException`.\n  Any other exception thrown indicates a bug in `package:tar` or how it's used.\n  The reader should never crash.\n- Reading a tar file can be cancelled mid-stream without leaking resources.\n\nHowever, the tar reader __does not__ throw exceptions for well-formed archives\nwith suspicious contents, such as\n\n- File names beginning with `../`, `/` or names pointing out of the archive by\n  other means.\n- Link references to files outside of the archive.\n- Paths not using forward slashes.\n- Gzip + tar bombs.\n- Invalid permission bits in entries.\n- ...\n\nWhen reading or extracting untrusted tar files, it is your responsibility to\ndetect and handle these cases.\nFor instance, this naive extraction function is susceptible to invalid tar\nfiles containing paths outside of the target directory:\n\n```dart\nFuture\u003cvoid\u003e extractTarGz(File tarGz, Directory target) async {\n  final input = tarGz.openRead().transform(gzip.decoder);\n\n  await TarReader.forEach(input, (entry) async {\n    final destination =\n        // DON'T DO THIS! If `entry.name` contained `../`, this may escape the\n        // target directory.\n        path.joinAll([target.path, ...path.posix.split(entry.name)]);\n\n    final f = File(destination);\n    await f.create(recursive: true);\n    await entry.contents.pipe(f.openWrite());\n  });\n}\n```\n\nFor an idea on how to guard against this, see the [extraction logic](https://github.com/dart-lang/pub/blob/3082796f8ba9b3f509265ac3a223312fb5033988/lib/src/io.dart#L904-L991)\nused by the pub client.\n\n-----\n\nBig thanks to [Garett Tok Ern Liang](https://github.com/walnutdust) for writing the initial\nDart tar reader that this library is based on.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimolus3%2Ftar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimolus3%2Ftar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimolus3%2Ftar/lists"}