{"id":21870016,"url":"https://github.com/solana-developers/anchor-zero-copy-example","last_synced_at":"2025-04-14T23:42:36.643Z","repository":{"id":160419529,"uuid":"614383874","full_name":"solana-developers/anchor-zero-copy-example","owner":"solana-developers","description":"Explaining on some examples heap, stack and account size limits and zero copy.","archived":false,"fork":false,"pushed_at":"2024-07-25T15:25:24.000Z","size":171,"stargazers_count":55,"open_issues_count":3,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-14T23:42:32.735Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/solana-developers.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-15T13:32:56.000Z","updated_at":"2025-04-10T17:21:05.000Z","dependencies_parsed_at":null,"dependency_job_id":"f926e9f9-a6c3-4178-954e-029ccdc9adab","html_url":"https://github.com/solana-developers/anchor-zero-copy-example","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solana-developers%2Fanchor-zero-copy-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solana-developers%2Fanchor-zero-copy-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solana-developers%2Fanchor-zero-copy-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solana-developers%2Fanchor-zero-copy-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/solana-developers","download_url":"https://codeload.github.com/solana-developers/anchor-zero-copy-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248981259,"owners_count":21193143,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-28T06:09:58.270Z","updated_at":"2025-04-14T23:42:36.625Z","avatar_url":"https://github.com/solana-developers.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# How to handle big accounts\n\n## How to run \nTo run the tests:\n1. Install Solana CLI: https://docs.solana.com/de/cli/install-solana-cli-tools\n2. Open Terminal: solana-test-validator to start a local validator\n3. In vs code in the terminal run: \"npm install\" to install the node packages needed to run the tests.\n4. Then: anchor build \u0026\u0026 anchor deploy \n5. Copy the deployed program id from the terminal and paste it into the lib.rs and the anchor.toml file\n6. Then: anchor test or anchor test --skip-local-validator depending on your node version\n\nMaybe you also need to install Anchor or Rust: https://www.anchor-lang.com/docs/installation\n\n[Video walkthrough](https://www.youtube.com/watch?v=zs_yU0IuJxc\u0026ab_channel=Solana)\n\n## Explanation of Solana Memory and Zero Copy \n\nThe heap and stack memory in the Solana runtime are very limited. We have 4Kb to work with on the stack and 32Kb on the heap.\nThe stack increased by 10Kb per loaded account. These limits are quickly reached when writing a program. \nBy default in Anchor all account structs are being loaded will be on the stack. If you reach the stack limit you will an error similar to this: \n\n```js\nStack offset of -30728 exceeded max offset of -4096 by 26632 bytes, please minimize large stack variables\n```\n(See test stacksize.ts)\n\nTo prevent this to a certain degree you can Box your account. What this means is that the account will move to the heap and there will only a pointer of 16 Bytes will be saved on the Stack.\nThis can be done like this: \n\n```js\n#[derive(Accounts)]\npub struct Example {\n    pub my_acc: Box\u003cAccount\u003c'info, MyData\u003e\u003e\n}\n```\n\nIf your account gets bigger it gets a bit more complicated. Solana does not allow Cross Program Invocations with accounts bigger than 10Kb.\nAnchor does use a CPI to initialize all new accounts. So it calls the System Program internally to create a new account.\nYou can allocate more memory to your account like this with an extra transaction: \n\n```js\nProgram: \n\n    #[derive(Accounts)]\n    #[instruction(len: u16)]\n    pub struct IncreaseAccoutSize\u003c'info\u003e {\n        #[account(mut, \n            realloc = len as usize, \n            realloc::zero = true, \n            realloc::payer=signer)]\n        pub data_holder: Account\u003c'info, DataHolderNoZeroCopy\u003e,\n        #[account(mut)]\n        pub signer: Signer\u003c'info\u003e,\n        #[account(address = system_program::ID)]\n        pub system_program: Program\u003c'info, System\u003e,\n    }\n\nTs: \n    let txRealloc = await program.methods\n    .increaseAccountData(20480)\n    .accounts({\n    signer: signer.publicKey,\n    dataHolder: pdaNoZeroCopy,\n    systemProgram: anchor.web3.SystemProgram.programId\n    })\n    .signers([signer])\n    .rpc();\n```\n\nYou can then call this multiple times adding 10240 in each transaction. \nWhen loading an account which is bigger than the 10240 bytes though you will get an out of memory exception.\n(See test: withoutzerocopy)\n\nIf you need an even bigger account size you need to look into Zero Copy serialization. \nYou should only use zero copy for large accounts that can not be Borsh/Anchor serialized without hitting the heap or stack limits. \nWith zero copy deserialization, all bytes from the account's backing `RefCell\u003c\u0026mut [u8]\u003e` are simply re-interpreted as a reference to the data structure. No allocations or copies necessary. This is how we can get around the stack and heap limitations.\n\nFor the account you want to serialize with zero copy you need to add this zero_copy to the account: \n\n```js\n#[account(zero_copy)]\n```\n\nThen you can define the repr which definies how the data will be packed. By default repr[c] will be used, so the C serialization. \nThis will by default break options and enums in your structs because the C serialization is different from the Borsh serialization.\nYou can also use:  \n\n```js\n#[repr(C)]\nor\n#[repr(packed)]\n```\n\nwhich should remove all the extra space that the C serialization adds.\n\nHere is a list of different repr types \u003cbr/\u003e\n[Repr types](https://doc.rust-lang.org/nomicon/other-reprs.html)\u003cbr/\u003e\n[Space needed for different data types](https://book.anchor-lang.com/anchor_references/space.html)\u003cbr/\u003e\n\nKeep in mind that when you use different repr types the data may not deserialize as expected in the client because for example options and enums may be packed differently.\n\nNext you replace Account with AccountLoader and then in you anchor program you can access the data using .load_mut()?\nLike this you can interact with the data of the account using copy_from_slice or mem copy without loading the whole account into memory.\n\n```js\n\n    pub fn set_data(ctx: Context\u003cSetData\u003e, string_to_set: String, index: u64) -\u003e Result\u003c()\u003e {\n        let text_to_add_to_the_account = str::from_utf8(string_to_set.as_bytes()).unwrap();\n        msg!(text_to_add_to_the_account);\n\n        // Since the account is bigger that the heap space as soon as we access the whole account we will get a out of memory error        \n        // let string = \u0026ctx.accounts.data_holder.load_mut()?.long_string;\n        // let complete_string = str::from_utf8(string).unwrap(); \n        // msg!(\"DataLength: {}\", string.len());\n        // msg!(\"CompleteString: {}\", complete_string);\n\n        // So the solution is use copy_from_slice and mem copy when we want to access data in the big account\n        ctx.accounts\n            .data_holder\n            .load_mut()?\n            .long_string[((index) as usize)..((index +912) as usize)]\n            .copy_from_slice(string_to_set.as_bytes());\n\n        Ok(())\n    }\n\n    // This will initialize the PDA with the maximum possible size of 10 Kb\n    #[derive(Accounts)]\n    pub struct Initialize\u003c'info\u003e {\n        #[account(init, seeds = [b\"data_holder_zero_copy_v0\", \n        signer.key().as_ref()], \n        bump, \n        payer=signer, \n        space= 10 * 1024 as usize)]\n        pub data_holder: AccountLoader\u003c'info, DataHolder\u003e,\n        #[account(mut)]\n        pub signer: Signer\u003c'info\u003e,\n        #[account(address = system_program::ID)]\n        pub system_program: Program\u003c'info, System\u003e,\n    }\n\n    #[account(zero_copy)]\n    #[repr(packed)]\n    pub struct DataHolder {\n        // 40952 = 40960 - 8 (account desciminator)\n        pub long_string: [u8; 40952],\n    }\n\n    #[derive(Accounts)]\n    #[instruction(len: u16)]\n    pub struct IncreaseZeroCopy\u003c'info\u003e {\n        #[account(mut, \n            realloc = len as usize, \n            realloc::zero = true, \n            realloc::payer=signer)]\n        pub data_holder: AccountLoader\u003c'info, DataHolder\u003e,\n        #[account(mut)]\n        pub signer: Signer\u003c'info\u003e,\n        #[account(address = system_program::ID)]\n        pub system_program: Program\u003c'info, System\u003e,\n    }\n}\n```\n\n[Great Blog article about size Anchor differences](https://www.sec3.dev/blog/all-about-anchor-account-size)\n\nHere is a game anchor program that uses Zero Copy for a game grid: \u003cbr/\u003e \n[Anchor Program Game](https://github.com/Woody4618/SolPlay_Unity_SDK/blob/main/Assets/SolPlay/Examples/SolHunter/AnchorProgram/src/state/game.rs)\nAnd here another example saving items in a zero copy account: \n[Anchor Program Items](https://github.com/coral-xyz/anchor/issues/651)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolana-developers%2Fanchor-zero-copy-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolana-developers%2Fanchor-zero-copy-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolana-developers%2Fanchor-zero-copy-example/lists"}