{"id":13794236,"url":"https://github.com/CalebEverett/arloader","last_synced_at":"2025-05-12T20:31:41.500Z","repository":{"id":42635945,"uuid":"423087035","full_name":"CalebEverett/arloader","owner":"CalebEverett","description":"Rust command line application and client for uploading files to Arweave.","archived":false,"fork":false,"pushed_at":"2024-02-09T11:46:38.000Z","size":4253,"stargazers_count":83,"open_issues_count":10,"forks_count":23,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-06T05:04:59.449Z","etag":null,"topics":["arweave","nft","rust","solana"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CalebEverett.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-31T07:56:53.000Z","updated_at":"2024-10-14T15:09:55.000Z","dependencies_parsed_at":"2024-08-03T23:03:27.366Z","dependency_job_id":"2f94f987-abbd-4e57-b388-10ae159f3d43","html_url":"https://github.com/CalebEverett/arloader","commit_stats":{"total_commits":173,"total_committers":6,"mean_commits":"28.833333333333332","dds":"0.040462427745664775","last_synced_commit":"414cc82e0a5c4c0219ee48019ca31c3056e6ede8"},"previous_names":["calebeverett/arload"],"tags_count":65,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalebEverett%2Farloader","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalebEverett%2Farloader/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalebEverett%2Farloader/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CalebEverett%2Farloader/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CalebEverett","download_url":"https://codeload.github.com/CalebEverett/arloader/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253816776,"owners_count":21968879,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arweave","nft","rust","solana"],"created_at":"2024-08-03T23:00:37.567Z","updated_at":"2025-05-12T20:31:38.230Z","avatar_url":"https://github.com/CalebEverett.png","language":"Rust","funding_links":[],"categories":["Tools ⚙️"],"sub_categories":[],"readme":"[![crates.io](https://img.shields.io/crates/v/arloader.svg)](https://crates.io/crates/arloader)\n[![build status](https://github.com/CalebEverett/arloader/actions/workflows/build.yml/badge.svg)](https://github.com/CalebEverett/arloader/actions/workflows/build.yml)\n[![docs.rs](https://img.shields.io/docsrs/arloader)](https://docs.rs/arloader)\n![Crates.io](https://img.shields.io/crates/l/arloader)\n\n# arloader\n\nCommand line application and client for uploading files to [Arweave](https://www.arweave.org/). Arweave stores documents and applications forever.\n\nUpload gigabytes of files with one command. Files are read and posted to [arweave.net](https://arweave.net) asynchronously with computationally intensive bundle preparation performed in parallel on multiple threads.\n\n## Contents\n* [Installation](#installation)\n* [NFT Usage](#nft-usage)\n* [General Usage](#general-usage)\n* [Usage with SOL](#usage-with-sol)\n* [Reward Multiplier](#reward-multiplier)\n* [Usage without Bundles](#usage-without-bundles)\n* [Benchmarks](#benchmarks)\n* [Pricing Comparison](#pricing-comparison)\n* [Roadmap](#roadmap)\n* [Transactions Prior to v0.1.51](#potential-issue-with-transactions-uploaded-prior-to-version-0.1.51)\n\n## Installation\n\n1. The easiest way to use arloader is to download the binary for your system (Linux or Mac) from the [releases on github](https://github.com/CalebEverett/arloader/releases).\n\nYou can also install from [crates.io](https://crates.io) once you have [rust installed](https://www.rust-lang.org/tools/install) with the nightly toolchain.\n\n```\nrustup default nightly\ncargo install arloader\n```\n\n2. Get an Arweave wallet json file [here](https://faucet.arweave.net/).\n\n3. If you're going to use AR to pay for transactions, [get AR tokens](https://arweave.news/how-to-buy-arweave-token/).\n\n4. If you're going to use SOL, get a [Solana wallet](https://docs.solana.com/wallet-guide/cli) json file and transfer some SOL to it.\n\n## NFT Usage\nThe single `upload-nfts` command below assumes you have a pair of image and metadata files for each of your NFTs. To learn more about each of the steps to upload your files, check out [upload_nfts_steps](docs/upload_nfts_steps.md). See [multiple_asset_files](docs/multiple_asset_files.md) for an example of how to upload multiple media files.\n\nFor a beginner's step-by-step guide of how to create an NFT in Solana (from creating a wallet, to uploading assets to arweave using arloader, to listing it in a marketplace) check out [this medium article](https://medium.com/@murki/the-ultimate-dev-guide-to-manually-minting-a-brand-new-nft-in-solana-fb5af9771688).\n\n### Create Upload Folder\n Put your assets and associated metadata files with `.json` extension in a folder by themselves. You can use any kind of file you want. Arloader automatically adds a content type tag to your upload so that browsers will handle it correctly when accessed from Arweave.\n ```\n├── 0.json\n├── 0.png\n├── 1.json\n├── 1.png\n├── 2.json\n├── 2.png\n├── 3.json\n├── 3.png\n├── 4.json\n├── 4.png\n├── 5.json\n├── 5.png\n```\n\n### Upload Assets\nIf you want to fund transactions with SOL, run the command below where `\u003cFILE_PATHS\u003e` matches your asset files.\n```\narloader upload-nfts \u003cFILE_PATHS\u003e --with-sol --sol-keypair-path \u003cSOL_KEYPAIR_PATH\u003e --ar-default-keypair\n```\n\nFor example, if you were uploading mp4 files as your assets and they were in a folder called `path/to/my/assets` and the path to your SOL keypair was `path/to/my/solkeypair.json`, you would enter:\n\n```\narloader upload-nfts path/to/my/assets/*.mp4 --with-sol --sol-keypair-path path/to/my/solkeypair.json --ar-default-keypair\n```\n\nTo fund transactions with AR, instead run:\n```\narloader upload-nfts \u003cFILE_PATHS\u003e --ar-keypair-path \u003cAR_KEYPAIR_PATH\u003e\n```\n\nThis will first upload your assets, logging statuses to a newly created directory named `arloader_\u003cRANDOM_CHARS\u003e` in the folder where the assets are located.\n\nThen a manifest file will be created from the logged statuses and uploaded. A manifest is a special file that Arweave uses to access your files by their names, relative to the id of the manifest transaction: `https://arweave.net/\u003cMANIFEST_ID\u003e/\u003cFILE_PATH\u003e`. You'll still be able to access your files by their id at `https://arweave.net/\u003cBUNDLE_ITEM_ID\u003e`, but creating and uploading a manifest gives you the option of using either. Once uploaded, the manifest file itself can be accessed online at `https://arweave.net/tx/\u003cMANIFEST_ID\u003e/data.json`.\n\n#### Update Metadata and Upload \nNext your metadata files will be updated with links to the uploaded assets. For the `upload-nfts` command arloader will replace the `image` key with the newyly created link and append the new link(s) to `files` key in your metadata `.json`. It defaults to using the id link, `https://arweave.net/\u003cBUNDLE_ITEM_ID\u003e`, but if you prefer to use the file path based link, `https://arweave.net/\u003cMANIFEST_ID\u003e/\u003cFILE_PATH\u003e`, you can pass the `--link-file` flag.\n\nAfter your metadata files have been updated, they will be uploaded, followed by the creation and upload of a manifest file for your metadata  files.\n\n### Get Links to Uploaded Metadata\n\nOnce everything has been uploaded, the links to your uploaded metadata files, to be included in your on chain token metadata, can be found in `arloader_\u003cRAND_CHAR\u003e/metadata/manifest_\u003cTXID\u003e.json`.\n\n```json\n{\n    \"0.json\": {\n        \"id\": \"ScU9mEuKBbPX5o5nv8DZkDnZuJbzf84lyLk-uLVDqNk\",\n        \"files\": [\n            {\n                \"uri\": \"https://arweave.net/ScU9mEuKBbPX5o5nv8DZkDnZuJbzf84lyLk-uLVDqNk\",\n                \"type\": \"application/json\"\n            },\n            {\n                \"uri\": \"https://arweave.net/fo9P3OOq78REajk48vFWbKfIhw6mDzgjANQIh3L7Njs/0.json\",\n                \"type\": \"application/json\"\n            }\n        ]\n    },\n    \"1.json\": {\n        \"id\": \"8APeQ5lW0-csTcBaGdPBDLAL2ci2AT9pTn2tppGPU_8\",\n        \"files\": [\n            {\n                \"uri\": \"https://arweave.net/8APeQ5lW0-csTcBaGdPBDLAL2ci2AT9pTn2tppGPU_8\",\n                \"type\": \"application/json\"\n            },\n            {\n                \"uri\": \"https://arweave.net/fo9P3OOq78REajk48vFWbKfIhw6mDzgjANQIh3L7Njs/1.json\",\n                \"type\": \"application/json\"\n            }\n        ]\n    },\n```\n\nIf you are creating your NFTs with the [Metaplex Candy Machine](https://docs.metaplex.com/candy-machine-v2/introduction), you can use the file named `metaplex_items_\u003cMANIFIEST_ID\u003e.json` in the same directory as `\u003cMANIFEST_PATH\u003e`. As with updating metadata files, Arloader defaults to using the id based link, `https://arweave.net/\u003cBUNDLE_ITEM_ID\u003e`, but you can use the file based link, `https://arweave.net/\u003cMANIFEST_ID\u003e/\u003cFILE_PATH\u003e`, by passing the `--link-file` flag.\n\n```json\n{\n        \"0\": {\n            \"link\": \"uri link\",\n            \"name\": \"name\",\n            \"onChain\": false\n        },\n        \"1\": {\n            \"link\": \"uri link\",\n            \"name\": \"name\",\n            \"onChain\": false\n        },\n```\n### Confirm All Transactions\n\nBefore you create your tokens, make sure that all of your transactions have been confirmed at least 25 times. Run the command below where `\u003cLOG_DIR\u003e` refers to the automatically created directory in your assets folder that begins with `arloader_`.\n\nThe primary reason for transactions having a status of `NotFound` is that they were rejected by miners for not having a big enough reward or for having to big of a reward. See [Reward Multiplier](#reward-multiplier) for instructions on increasing the reward by passing the optional `--reward-multiplier` argument.\n\n```\narloader update-nft-status \u003cLOG_DIR\u003e\n```\n\n```\nUpdating asset bundle statuses...\n\n bundle txid                                   items      KB  status       confirms\n------------------------------------------------------------------------------------\n kmgLCgV-dB-DGML8cvFwuP3a-ZKedz7nyDuEsqYPTis      10     980  Confirmed          60\nUpdated 1 statuses.\n\n\nUpdating metadata bundle statuses...\n\n bundle txid                                   items      KB  status       confirms\n------------------------------------------------------------------------------------\n kmgLCgV-dB-DGML8cvFwuP3a-ZKedz7nyDuEsqYPTis      10     980  Confirmed          60\nUpdated 1 statuses.\n\n\nUpdating asset manifest status...\n\n id                                           status     confirms\n------------------------------------------------------------------\n URwQtoqrbYlc5183STNy3ZPwSCRY4o8goaF7MJay3xY  Confirmed        60\n\n\nUpdating metadata manifest status...\n\n id                                           status     confirms\n------------------------------------------------------------------\n fo9P3OOq78REajk48vFWbKfIhw6mDzgjANQIh3L7Njs  Confirmed        57\n ```\n\n## General Usage\n\nIf you're uploading more than one file, you should pretty much always be using bundles. Bundles take multiple files and packages them together in a single transaction. This is better than uploading multiple individual files because you only have to wait for one transaction to be confirmed. Once the bundle transaction is confirmed, all of your files will be available. Larger transactions with larger rewards are more attractive to miners, which means a larger bundled transaction is more likely to get written quickly than a bunch of smaller individual ones.\n\nArloader will create as many bundles as necessary to upload all of your files. Your files are read asynchronously, bundled in parallel across multiple threads and then posted to [arweave.net](https://arweave.net). Arloader supports bundle sizes up to 200 MB, with a default of 100 MB. This should work fine for individual files up to 100 MB. If your files sizes are bigger than 100 MB (but smaller than 200 MB), you can specify a larger bundle size with the `--bundles-size` argument - `--bundle-size 200` to specify a size of 200 MB, for example. If your file sizes are bigger than 200 MB, you can upload them as individual files by passing the `--no-bundle` flag.\n\n### Estimate Cost\nTo get an estimate of the cost of uploading your files run\n\n```\narloader estimate \u003cFILE_PATHS\u003e\n```\n\n`\u003cFILE_PATHS\u003e` can be a glob, like `path/to/my/files/*.png`, or one or more files separated by spacees, like `path/to/my/files/2.mp4 path/to/my/files/0.mp path/to/my/files/2.mp`.\n\n### Upload\nTo upload your files run\n\n```\narloader upload \u003cFILE_PATHS\u003e\n```\n\nThis kicks off the process of uploading a stream of bundles created from your files. The example output below had a bundle size of 5000 bytes.\n\n```\nbundle txid                                   items      KB  status       confirms\n------------------------------------------------------------------------------------\n QGPFcZq91lQgmmz2l7rQHkSQpgfJi-Vhv47oTqIYLm4       2       3  Submitted           0\n _-bhdsi4irDEWz8R9wXT-1c06WVQVSMAmQxVF9OkW94       2       3  Submitted           0\n -OAWdFiGS4NKOZXVJG3yZ0yN4xydGOhfQGX2FCdlG88       2       3  Submitted           0\n UBWGFKyTrUVaCa7wi_181FjAd545vdoHmBQEdlaVdA4       2       3  Submitted           0\n qzQlASZrQXNF9HYIOTPjEZL9uy1U9Ou086kCkQWqld0       2       3  Submitted           0\n ```\n\nA status object gets written to a json file named `\u003cTXID\u003e.json` in a newly created sub directory in the parent folder of the first file in `\u003cFILE_PATHS\u003e`. The folder will be named `arloader_\u003cRAND_CHAR\u003e`. You can specify an existing folder to write statuses to by passing the `--log-dir` argument.\n\n```json\n{\n    \"id\": \"_-bhdsi4irDEWz8R9wXT-1c06WVQVSMAmQxVF9OkW94\",\n    \"status\": \"Submitted\",\n    \"file_paths\": {\n        \"tests/fixtures/8.png\": {\n            \"id\": \"0jd-NTQUZhmnKRY-kMt2vEcmSqgzKOLX_P3QYw6CaNE\"\n        },\n        \"tests/fixtures/9.png\": {\n            \"id\": \"1XdiLkoZ5POHsNx7eLyRgisjnxTLzW8SxGsRcb22j84\"\n        }\n    },\n    \"number_of_files\": 2,\n    \"data_size\": 3546,\n    \"created_at\": \"2021-11-23T05:47:41.948103600Z\",\n    \"last_modified\": \"2021-11-23T05:47:41.948107100Z\",\n    \"reward\": 50947968\n}\n```\n\n### Check Status\nAfter uploading your files, you'll want to check on their status to make sure the have been uploaded successfully and that they ultimately are confirmed at least 25 times before you can be absolutely certain they have been permanently uploaded.\n\n```\narloader update-status \u003cLOG_DIR\u003e\n```\n\nThis will read the files in `\u003cLOG_DIR\u003e`, looking for a valid transaction id as a file stem, and then go out to the network to update the status of each. The example below contained two sets of bundles, one still pending and one with 45 confirmations.\n\n```\nbundle txid                                   items      KB  status       confirms\n------------------------------------------------------------------------------------\n -OAWdFiGS4NKOZXVJG3yZ0yN4xydGOhfQGX2FCdlG88       2       3  Pending             0\n _-bhdsi4irDEWz8R9wXT-1c06WVQVSMAmQxVF9OkW94       2       3  Pending             0\n qzQlASZrQXNF9HYIOTPjEZL9uy1U9Ou086kCkQWqld0       2       3  Pending             0\n QGPFcZq91lQgmmz2l7rQHkSQpgfJi-Vhv47oTqIYLm4       2       3  Pending             0\n UBWGFKyTrUVaCa7wi_181FjAd545vdoHmBQEdlaVdA4       2       3  Pending             0\n KuuEZpbfCbw6izMeN3knWlpzmaFhnrDL9dUKCW2LQHw       2       3  Confirmed          45\n IRToYYvsftCiR71sfW5qt8XCzBFotwoDFBoEMEtrMrU       2       3  Confirmed          45\n M2QZYxUqw3ZJ2KXzU4pfw9fFIkVOSrJbSpE7NAvHLvo       2       3  Confirmed          45\n qvci4i6Mfr-5_NHI1bL-Omv16QEUw3iiirzv4fXefnM       2       3  Confirmed          45\n NAP2vTKQdMG_eKyKBYz3876T4yBFl4oYFYqwwwnHbFA       2       3  Confirmed          45\n ```\n\n### Re-Upload\nIf you find that not all of your transactions have a status of `Confirmed` or that the number of confirmations is below 25 after some period of time, you will want to re-upload your transactions with the following command:\n\n```\narloader reupload \u003cFILE_PATHS\u003e --log-dir \u003cLOG_DIR\u003e --statuses \u003cSTATUSES\u003e --max-confirmations \u003cMAX_CONFIRMATIONS\u003e\n```\nThis will first check to make sure that all of the files in `\u003cFILE_PATHS\u003e` are included in the status objects in `\u003cLOG_DIR\u003e`, adding them to the list of files to be reuploaded. Then it will filter the paths in the status objects based on `\u003cSTATUSES\u003e` and `\u003cMAX_CONFIRM\u003e`. You can provide multiple statuses and max confirmations will only re-upload transactions with fewer than `\u003cMAX_CONFIRM\u003e` confirmations.\n\nFor example, if had uploaded a bunch of jpegs that were in `my/images` and statuses had been logged to `my/images/arloader_hehQJu-RJpo`, if you wanted to re-upload transactions with either a status of `NotFound` or `Pending`, you would run:\n\n```\narloader reupload my/images/*.jpeg --log-dir my/images/arloader_hehQJu-RJpo --statuses NotFound Pending\n```\n\nIf you wanted to reupload anything with less than 25 confirmations, you would run:\n\n```\narloader reupload my/images/*.jpeg --log-dir my/images/arloader_hehQJu-RJpo --max-confirms 25\n```\n\n\n### Create Manifest\nOnce you have a sufficient number of confirmations of your files, you may want to create a manifest file, which is used by the Arweave gateways to provide relative paths to your files. In order to do that, you run\n\n```\narloader upload-manifest \u003cLOG_DIR\u003e\n```\nwhere `\u003cLOG_DIR\u003e` is the directory containing your bundle status json files. This will go through and consolidate the paths from each of the bundles, create a consolidated manifest, upload it to Arweave and then write a file named `manifest_\u003cTXID\u003e.json`to `\u003cLOG_DIR\u003e`. Once the transaction uploading the manifest has been confirmed, you will be able to access your files and both `https://arweave.net/\u003cBUNDLE_ITEM_ID\u003e` and `https://arweave.net/\u003cMANIFEST_ID\u003e/\u003cFILE_PATH\u003e`  where `MANIFEST_ID` is the id of the manifest transaction and `FILE_PATH` is the relative path of the file included with the `upload` command.\n\n```json\n{\n    \"tests/fixtures/0.png\": {\n        \"files\": [\n            {\n                \"type\": \"image/png\",\n                \"uri\": \"https://arweave.net/BSvIAiwthQu_xwQBHn9FcgACaZ8ko4py5mqMNP4r-jM/tests/fixtures/0.png\"\n            },\n            {\n                \"type\": \"image/png\",\n                \"uri\": \"https://arweave.net/JQbz5py065lqaS_8R7NCtLcK2b-pSkkG6Je0OT8379c\"\n            }\n        ],\n        \"id\": \"JQbz5py065lqaS_8R7NCtLcK2b-pSkkG6Je0OT8379c\"\n    },\n    \"tests/fixtures/1.png\": {\n        \"files\": [\n            {\n                \"type\": \"image/png\",\n                \"uri\": \"https://arweave.net/BSvIAiwthQu_xwQBHn9FcgACaZ8ko4py5mqMNP4r-jM/tests/fixtures/1.png\"\n            },\n            {\n                \"type\": \"image/png\",\n                \"uri\": \"https://arweave.net/Os-tEyRqdjwwyNo1mpLaPGu8_r3KbV-iNRH-aPtJFOw\"\n            }\n        ],\n        \"id\": \"Os-tEyRqdjwwyNo1mpLaPGu8_r3KbV-iNRH-aPtJFOw\"\n    },\n```\n\nYou can run the following command to get an update on the status of your manifest transaction.\n```\narloader get-status \u003cMANIFEST_ID\u003e\n```\n\n## Usage with SOL\n\nYou can use SOL to pay for your transactions without going through the hassle of procuring AR tokens.\n\nArloader usage is pretty much exactly the same as above, with the addition of the `--with-sol` flag.\n\n1. To get an estimate of the cost of uploading your files run\n\n```\narloader estimate \u003cFILE_PATHS\u003e --with-sol\n```\n\n2. To upload your files run\n\n```\narloader upload \u003cFILE_PATHS\u003e --with sol --ar-default-keypair\n```\n\nThis will create the same stream of bundles that gets created without using SOL and then goes out to an api to get your transactions signed. Once the SOL payment transaction has gone through, the signature comes back from the api and gets added to your bundle transaction. Then the transaction gets uploaded directly to the [arweave.net](https:://arweave.net) gateway from your computer.\n\n## Reward Multiplier\n\nArweave is limited to approximately 1,000 transactions every two minutes so if you happen to submit your transaction at a time when there are a lot of pending transactions, it may take longer to get written, or if there are enough more attractive transaction, i.e, with higher rewards, it may not get written at all. To check the current number of pending transactions, run \n\n```\narloader pending\n```\nand that will print the number of pending transactions every second for one minute.\n\n```\n pending tx\n-------------------------------------------------------------------------------------------------\n  118 | ▥▥▥\n  123 | ▥▥▥\n  124 | ▥▥▥\n  224 | ▥▥▥▥▥\n  125 | ▥▥▥\n  326 | ▥▥▥▥▥▥▥\n  128 | ▥▥▥\n  ```\n\nGiven that Arloader bundles by default, your transaction is hopefully relatively attractive and you don't need to increase the reward to get it written in a timely fashion. However, if you see that there are a lot of transactions pending and you want to be sure your transaction goes through quickly, you can adjust the reward with `--reward-multiplier` followed by something tha can be parsed as a float between `0.0` and `3.0`. The reward included in your transaction will then be multiplied by this factor when it gets submitted. Similar to the `--with-sol` flag, you can add `--reward-multiplier` to both `estimate` and `upload` commands.\n\n## Usage without Bundles\n\nYou can add the `--no-bundle` flag if for some reason you want to create individual transactions. This works with both `estimate` and `upload` commands. In that case individual status objects are written to `\u003cLOG_DIR\u003e` and you can run `update-status` to update them from the network and `status-report` for a count of transactions by status.\n\n## Benchmarks\n\nThe table below shows the average duration required to create transactions across a range of file sizes and numbers of files. Detailed statistical analyses and charts can be found [here](https://calebeverett.github.io/arloader/) (numbers may vary slightly from those below).\n\nFor an NFT project with 10,000 tokens it would take 20 seconds to process the images if they were 256 KB. If the images were 4 MB, it would take approximately two minutes.\n\n\n| File Size | Num Files | Total Size | Data Item | Data Items | Bundle | Transaction | Total | Per 1,000 |\n| --------: | --------: | ---------: | --------: | ---------: | -----: | ----------: | ----: | --------: |\n|     32 KB |       500 |         16 |         4 |        430 |     30 |          40 |   0.5 |       1.0 |\n|    256 KB |       500 |        128 |         4 |        493 |    179 |         326 |   1.0 |       2.0 |\n|      1 MB |       500 |        512 |         5 |        616 |    903 |        1033 |   2.6 |       4.1 |\n|      4 MB |       150 |        614 |        11 |        360 |   1050 |        1554 |   3.0 |      10.4 |\n|     16 MB |       50  |        819 |        35 |        393 |   1403 |        2058 |   3.9 |      77.1 |\n\n\nBenchmarks include only processing activity and exclude reading files from disk and uploading them to the network. Benchmarks were performed on an Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz processor with 6 cores.\n\n| Column | Description |\n| --- | --- |\n| Total Size | File size x number of files in megabytes.|\n| Data Item | Time in milliseconds required to create a single data item of the file size. The entails creating a merkle tree data root, generating an id from the deep hash algorithm and signing it.|\n| Data Items | Time in milliseconds required to create data items for the number of files. Data items are processed in parallel using all available cores.|\n| Bundle | Time in milliseconds required to create a single bundle from the data items. This entails serializing each of the data items and packing them together.|\n| Transaction |Time in milliseconds required to create a transaction from the bundle. This entails creating a merkle tree data root, generating an id from the deep hash algorithm and signing it.|\n| Total | Sum of the time required to create data items, bundle and transaction in seconds.|\n| Per 1,000 | Extrapolation of total to 1,000 files.|\n\n## Pricing Comparison\n\nThe table below compares the cost to upload 256 MB of files to Arweave as individual files versus in a single bundle and includes a comparison to the bundlr network as well. This was run on 2021-12-17. You can run `cargo run --example pricing` to print a table with current prices.\n\n```\nPrice in USD to upload 256 MB of files of various sizes in KB ($48.33 USD per AR):\n\nfile size | num files | arweave | bundlr | arweave total | bundlr total | arweave bundle\n-----------------------------------------------------------------------------------------\n        1      262144    0.0019   0.0000        508.3164         5.1784           1.9616\n       32        8192    0.0019   0.0003         15.8849         2.5892           1.9616\n      256        1024    0.0019   0.0025          1.9856         2.5892           1.9616\n     1024         256    0.0077   0.0101          1.9676         2.5892           1.9616\n     4096          64    0.0307   0.0405          1.9631         2.5892           1.9616\n    16384          16    0.1226   0.1618          1.9620         2.5892           1.9616\n```\n\n## Roadmap\n\n- [x] Bundle size unit in MB\n- [x] Handle error on pricing look up\n- [x] Buffer chunks post stream\n- [x] Add upload nft project example\n- [x] Point at folder of assets and json and get back links to uploaded metadata\n- [x] Clean up handling of paths\n- [x] Re upload bundles\n- [ ] Add super simple single upload, return link\n- [ ] Progress indicators for longer running processes\n- [ ] Output in metaboss format, or include in metaplex cli\n- [ ] Implement bundlr\n- [ ] Async benchmarking, including reading files from disk\n- [ ] Bundlr benchmarking\n- [ ] Report on missing files in `list-status` and `update-status` commands\n- [ ] Include duration in completion output.\n\n## Potential Issue with Transactions Uploaded Prior to Version 0.1.51\nThe way arloader was formatting transactions for upload was not entirely compatible with the Arweave protocol prior to version 1.51. For transactions bigger than 256 KB it is possible that even though your transactions are visible and are showing more than 25 confirmations that they were not written to the Arweave blockchain. If you would like assistance determining whether your transactions were impacted, please open an issue and I will be happy to help, including paying for any necessary re-uploading.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCalebEverett%2Farloader","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCalebEverett%2Farloader","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCalebEverett%2Farloader/lists"}