{"id":13458408,"url":"https://github.com/siudesu/BinaryArchive","last_synced_at":"2025-03-24T15:31:17.199Z","repository":{"id":59790122,"uuid":"535789194","full_name":"siudesu/BinaryArchive","owner":"siudesu","description":"A very simplistic approach to bundling assets/files/data in an archive fashion. Originally meant for images, now functions as an all-purpose archive module.","archived":false,"fork":false,"pushed_at":"2024-04-09T04:16:40.000Z","size":848,"stargazers_count":10,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-10-29T03:33:02.865Z","etag":null,"topics":["lua","solar2d"],"latest_commit_sha":null,"homepage":"","language":"Lua","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/siudesu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-12T18:00:21.000Z","updated_at":"2024-08-28T01:31:52.000Z","dependencies_parsed_at":"2024-04-08T10:42:13.744Z","dependency_job_id":"4a8e977f-60cd-4490-8a08-d453959f3b64","html_url":"https://github.com/siudesu/BinaryArchive","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siudesu%2FBinaryArchive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siudesu%2FBinaryArchive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siudesu%2FBinaryArchive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/siudesu%2FBinaryArchive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/siudesu","download_url":"https://codeload.github.com/siudesu/BinaryArchive/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245298000,"owners_count":20592514,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lua","solar2d"],"created_at":"2024-07-31T09:00:52.261Z","updated_at":"2025-03-24T15:31:12.804Z","avatar_url":"https://github.com/siudesu.png","language":"Lua","funding_links":[],"categories":["Files \u0026 Data"],"sub_categories":[],"readme":"# Binary Archive Module\n### A [Solar2D](https://solar2d.com) Lua module for reading and writing binary archive files for storing resources.\n\n\n\u003c/br\u003e\n\n## Features\n - Support for multiple archive files in same project.\n - Allows caching (optional) of fetched data, minimizing disk access.\n - Provides a set of wrapper functions to facilitate the creation of certain Solar2D objects using assets stored in archive, see [DOCS](https://github.com/siudesu/BinaryArchive/blob/main/DOCUMENTATION.md).\n - In compliance with Solar2D's [dynamic image selection](https://docs.coronalabs.com/guide/basics/configSettings/index.html#dynamic-image-selection) when creating [newImageRect](https://docs.coronalabs.com/api/library/display/newImageRect.html) and [newImageSheet](https://docs.coronalabs.com/api/library/graphics/newImageSheet.html) objects.\n - Uses AES-256 for data encryption; *now optional by default.*\n - Not limited to files, any data in form of [String](https://docs.coronalabs.com/api/type/String.html) can be easily appended, and retrieved, such as data encoded in [JSON](https://docs.coronalabs.com/api/library/json/index.html) or any possible API or license keys you may be using with certain Solar2D plugins, etc.\n\n\n\u003c/br\u003e\n\n## Limitations\n - While the purpose of using this type of archive is, in part, loading assets without disk extraction, in this fashion it is currently limited to `png`, `jpg`, and `jpeg` files, and common data.\n\t\u003eAny file can be appended, but would need to be extracted before it can be used. Please see [FAQ](#FAQ) for more details.\n\n- Size limit per file to append:\n\t- `~200 MB` If not using encryption. (Limitation on Lua IO read(), currently processed in a single pass.)\n\t- `~100 MB` If using encryption. (Limitation on ciphering without using chunks.)\n\n\u003c/br\u003e\n\n## Benchmark Numbers:\nTests performed on a PC with HDD (not SSD), 1.6 GB archive with 18,383 files, measured in seconds:\n- Creating the archive:\n\t- `28s` with debug info enabled\n\t- `22s` without debug\n- Loading the archive:\n\t- `3.0s` with debug info enabled\n\t- `0.2s` without debug\n- Creating 50 objects using 2 image files:\n\t- `0.012s` loading directly from disk (not using archive)\n\t- `0.075s` loading from archive without cache enabled\n\t- `0.002s` loading from archive with cache enabled\n\n## Requirements\n- [Bytemap plugin](https://github.com/solar2d/com.xibalbastudios-plugin.Bytemap) by [Steven Johnson](https://github.com/ggcrunchy), must be added in `build.settings`.\n- (Optional) [OpenSSL plugin](https://docs.coronalabs.com/plugin/openssl/index.html), must be added in `build.settings` ONLY if encryption is desired.\n\n### build.settings (Plugins section)\n```lua\n\tplugins =\n\t{\n\t\t[\"plugin.Bytemap\"] = { publisherId = \"com.xibalbastudios\" },\n\t\t[\"plugin.openssl\"] = { publisherId = \"com.coronalabs\" }, -- optional\n\t}\n```\n\n\u003c/br\u003e\n\n## Sample Code\n### Creating a new archive with default values:\n```lua\n-- load module\nlocal bin = require( \"m_binary_archive\" )\n\n-- specify full path where assets are located (all files will be appended, includes sub-directories)\nlocal options = { baseDir = \"D:/Projects/Solar2D/AwesomeProject/assets/graphics\" }\n\n-- create a new archive, output will be saved at baseDir\nbin.new(options)\n```\n### Loading and using an archive:\n```lua\n-- load module\nlocal bin = require( \"m_binary_archive\" )\n\n-- specify file to load, file path is relative to project where main.lua resides\nlocal options = { file = \"assets/data.bin\" }\n\n-- load binary file\nbin.load(options)\n\n-- create newImageRect using wrapper function; parameters are the same as display.newImageRect()\nlocal balloon = bin.newImageRect( \"SnapshotEraser/balloon.jpg\", 200, 240 )\n\n-- create new rectangle and apply object fill using custom function\nlocal rect = display.newRect( 150, 150, 50, 50 )\n\tbin.imagePaint( rect, \"Fishies/fish.small.red.png\" )\n\n-- create newImageRect and apply mask\nlocal bg = bin.newImageRect( \"FilterGraph/image.jpg\", 480, 320 )\n\tbin.setMask( bg, \"SnapshotEraser/mask.png\" )\n\tbg.x = 240\n\tbg.y = 160\n```\n\n\u003c/br\u003e\n\n## Wrapper Functions\nThese are designed to work in place of Solar2D API functions by the same name:\n\u003c/br\u003e\n\u003e Note: Parameters and requirements are the same in all cases except for `baseDir`, this variable is neither used nor parsed as assets are stored in archive.\n\n\u003c/br\u003e\n\n### From `display.*` API:\n```lua\n\tMODULE.newEmitter( emitterParams )\n\tMODULE.newImage( [parent,] filename, [x, y] )\n\tMODULE.newImageRect( [parent,] filename, width, height )\n```\n\n### From `graphics.*` API:\n```lua\n\tMODULE.newImageSheet( filename, options )\n\tMODULE.newMask( filename )\n\tMODULE.newOutline( coarsenessInTexels, imageFileName )\n\tMODULE.newTexture( params ) -- this returns a Bytemap texture used as a replacement for [graphics.newTexure](https://docs.coronalabs.com/api/library/graphics/newTexture.html)\n```\n### Custom:\n```lua\n\tMODULE.compositePaint( object, filename1,  filename2 )\t-- wrapper for [CompositePaint](https://docs.coronalabs.com/api/type/CompositePaint/index.html)\n\tMODULE.imagePaint( object, filename )\t\t\t\t\t-- wrapper for [BitmapPaint] (https://docs.coronalabs.com/api/type/BitmapPaint/index.html)\n\tMODULE.setMask( object, filename )\t\t\t\t\t\t-- wrapper for [graphics.setMask](https://docs.coronalabs.com/api/type/DisplayObject/setMask.html)\n```\n\n\u003c/br\u003e\n\n## For more details on usage and samples please see [DOCS](https://github.com/siudesu/BinaryArchive/blob/main/DOCUMENTATION.md)\n\n\u003c/br\u003e\n\n## TODO\n- [ ] [Dynamically-Selected Mask](https://docs.coronalabs.com/api/library/graphics/newMask.html#dynamically-selected-mask) can be tentatively implemented in module so that no additional code is required as shown in Solar2D docs.\n\n\u003c/br\u003e\n\n## FAQ\n1. `What's the purpose of using an archive file?`\n\n   1. Hide project assets from plain sight.\n   2. Implement a method of using bundled assets without extracting them to disk, such as a zip file. This has other implications, see below `FAQ #7`.\n\n2. `How are archives created?`\n\n   Archives are created using Lua libraries (LFS and I/O) to read and append files into a single binary file. \n   \u003c/br\u003eAdditional information is added to help fetch files quickly.\n\n3. `How many files can be stored in an archive?`\n\n   An arbitrary number of 4,294,967,295 .... that's probably more than enough. :smile:\n\n4. `Is there any overhead in disk space?`\n\n\tTesting with 1.6 GB worth of data, +18k files, resulted in a 1.6 GB archive with about 1.5 MB of overhead.\n\t\n5. `Are the files secured in the archive?`\n\n   All appended files and data can be optionally encrypted using AES-256, please see [DOCS](https://github.com/siudesu/BinaryArchive/blob/main/DOCUMENTATION.md).\n\n6. `What files can I store in an archive?`\n\n   You can append any type of file from disk, and any data in form of a [String](https://docs.coronalabs.com/api/type/String.html), but usage may vary, see below `FAQ #7`.\n\n7. `What are the implications of using an archive for assets?`\n   \n   The implication is that Solar2D API loads asset from disk, in particular [these four directories](https://docs.coronalabs.com/guide/data/readWriteFiles/index.html#system-directories).\n   \u003c/br\u003eThe loading of textures is currently supported by [Bytemap plugin](https://github.com/solar2d/com.xibalbastudios-plugin.Bytemap) providing the ability to create external textures from Memory, these are not extracted anywhere on disk.\n   \u003c/br\u003eFor any other file types, you may need to extract them to [any of the write-access directories](https://docs.coronalabs.com/guide/data/readWriteFiles/index.html#system-directories) before using them.\n\n8. `Is there future plans for using other files in an archive without extracting them to disk?`\n\n   I think fonts and audio files are desirable to load from an archive, this would allow for most common applications to have assets fully bundled.\n   \u003c/br\u003eHowever, a technical implementation in the engine, or by way of a plugin, is first required, and both are out of my scope-- I'm just doing the easy \"packaging\" part. :slightly_smiling_face:\n   \u003c/br\u003eIt's worth noting that audio files will soon be usable from an archive; the same [creator](https://github.com/ggcrunchy) of the [Bytemap plugin](https://github.com/solar2d/com.xibalbastudios-plugin.Bytemap) has a [WIP project](https://discord.com/channels/721785436195782677/721785737258860544/1013963898589823056) which also allows loading audio files from Memory.\n\n\u003c/br\u003e\n\n---\n\n## License\nDistributed under the MIT License. See [LICENSE](https://github.com/siudesu/BinaryArchive/blob/main/LICENSE) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiudesu%2FBinaryArchive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsiudesu%2FBinaryArchive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsiudesu%2FBinaryArchive/lists"}