{"id":17113015,"url":"https://github.com/emmt/arraytools.jl","last_synced_at":"2025-10-14T14:45:54.141Z","repository":{"id":61800857,"uuid":"216584603","full_name":"emmt/ArrayTools.jl","owner":"emmt","description":"Tools for basic array manipulation and help dealing with the different flavors of arrays in Julia","archived":false,"fork":false,"pushed_at":"2024-09-24T15:16:16.000Z","size":607,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-09-20T15:02:10.831Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Julia","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/emmt.png","metadata":{"files":{"readme":"README.md","changelog":"NEWS.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-21T14:09:59.000Z","updated_at":"2024-09-24T14:58:52.000Z","dependencies_parsed_at":"2024-05-16T00:11:39.782Z","dependency_job_id":"5807eb5c-4e7c-403e-806d-ae4e2ef834f0","html_url":"https://github.com/emmt/ArrayTools.jl","commit_stats":{"total_commits":138,"total_committers":1,"mean_commits":138.0,"dds":0.0,"last_synced_commit":"aefbb11eeddf26ccc92a7a434b85371c82f40eec"},"previous_names":[],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/emmt/ArrayTools.jl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emmt%2FArrayTools.jl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emmt%2FArrayTools.jl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emmt%2FArrayTools.jl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emmt%2FArrayTools.jl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/emmt","download_url":"https://codeload.github.com/emmt/ArrayTools.jl/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/emmt%2FArrayTools.jl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279019161,"owners_count":26086685,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-14T17:02:18.752Z","updated_at":"2025-10-14T14:45:54.123Z","avatar_url":"https://github.com/emmt.png","language":"Julia","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Utilities for coding with Julia arrays\n\n[![License][license-img]][license-url]\n[![Stable][doc-stable-img]][doc-stable-url]\n[![Dev][doc-dev-img]][doc-dev-url]\n[![Build Status][github-ci-img]][github-ci-url]\n[![Build Status][appveyor-img]][appveyor-url]\n[![Coverage][codecov-img]][codecov-url]\n\nThis [Julia][julia-url] package provides a number of methods and types to deal\nwith the variety of array types (sub-types of `AbstractArray`) that exist in\nJulia and to help building custom array-like types without sacrificing\nperformances.\n\nThese are useful to implement methods to process arrays in a generic way.\n\n## Rubber indices\n\nThe constants `..` and `…` (type `\\dots` and hit the `tab` key) can be used in\narray indexation to left or right justify the other indices. For instance,\nassuming `A` is a `3×4×5×6` array, then all the following equalities hold:\n\n```julia\nA[..]          === A # the two are the same object\nA[..,3]         == A[:,:,:,3]\nA[2,..]         == A[2,:,:,:]\nA[..,2:4,5]     == A[:,:,2:4,5]\nA[:,2:3,..]     == A[:,2:3,:,:]\nA[2:3,..,1,2:4] == A[2:3,:,1,2:4]\n```\n\nAs can be seen, the advantage of the *rubber index* `..` is that it\nautomatically expands as the list of colons needed to have the correct number\nof indices. The expressions are also more readable. The idea comes from the\n[`Yorick`](http://github.com/LLNL/yorick/) language by Dave Munro.\n\nThe rubber index may also be used for setting values. For instance:\n\n```julia\nA[..] .= 1         # to fill A with ones\nA[..,3] = A[..,2]  # to copy A[:,:,:,2] in A[:,:,:,3]\nA[..,3] .= A[..,2] # idem but faster\nA[2,..] = A[3,..]  # to copy A[3,:,:,:] in A[2,:,:,:]\nA[..,2:4,5] .= 7   # to set all elements in A[:,:,2:4,5] to 7\n```\n\nLeading/trailing indices may also be specified as Cartesian indices (of type\n`CartesianIndex`).\n\nTechnically, the constant `..` is defined as `RubberIndex()` where\n`RubberIndex` is the singleton type that represents any number of indices.\n\nCall `colons(n)` if you need a `n`-tuple of colons `:`. When `n` is known at\ncompile time, it is faster to call `colons(Val(n))`.\n\n:warning: **Warning.** A current limitation of the rubber index is that it will\nconfuse the interpretation of the `end` token appearing in the same index list\n*after* the rubber index. This is beacuse the parser wrongly assumes that the\nrubber index counts for a single dimension. The `end` token may however appears\n*before* the rubber index. Example:\n\n```.julia\nA = rand(5,10,4,3);\nA[:,5:end,..] == A[:,5:end,:,:] # ok\nA[..,5:end,:] == A[:,:,5:end,:] # throws a BoundsError\n```\n\n\n## Array-like objects\n\n### Defining custom array-like objects\n\nJulia array interface is very powerful and flexible, it is therefore tempting\nto define custom array-like types, that is Julia types that behave like arrays,\nwithout sacrificing efficiency. The `ArrayTools` package provides simple means\nto define such array-like types if the values to be accessed as if in an array\nare stored in an array (of any concrete type) embedded in the object instance.\n\nThis is as simple as:\n\n1. Make your type inherit from `LinearArray{T,N}` or `CartesianArray{T,N}`\n   depending whether the index style of the embedded array is `IndexLinear()`\n   or `IndexCartesian()`.\n\n2. Extend the `Base.parent(A)` method for your custom type so that it returns\n   the embedded array of an instance `A`.\n\nFor instance (of course replacing the ellipsis `...`):\n\n```julia\nusing ArrayTools.PseudoArrays\nstruct CustomArray{T,N,...} \u003c: LinearArray{T,N}\n    arr::Array{T,N} # can be any array type with linear index style\n    ...             # another member\n    ...             # yet another member\n    ...             # etc.\nend\n\n@inline Base.parent(A::CustomArray) = A.arr\n```\n\nAs a result, instances of your `CustomArray{T,N}` will be also seen as\ninstances of `AbstractArray{T,N}` and will behave as if they implement linear\nindexing. Apart from the needs to extend the `Base.parent` method, the\ninterface to `LinearArray{T,N}` should provide any necessary methods for\nindexation, getting the dimensions, the element type, *etc.* for the derived\ncustom type. You may however override these definitions by more optimized or\nmore suitable methods specialized for your custom array-like type.\n\nIf your custom array-like type is based on an array whose index style is\n`IndexCartesian()` (instead of `IndexLinear()` in the above example), just make\nyour custom type derived from `CartesianArray{T,N}` (instead of\n`LinearArray{T,N}`). For such array-like object, index checking requires an\nefficient implementation of the `Base.axes()` method which you may have to\nspecialize. The default implementation is:\n\n```julia\n@inline Base.axes(A::CartesianArray) = axes(parent(A))\n```\n\n\n### Array-like objects with properties\n\nAs a working example of custom array-like objects, the `ArrayTools` package\nprovides `AnnotatedArray{T,N,P}` objects which store values like arrays but\nalso have properties stored in a dictionary or a named tuple (of type `P`).\nHere the parameters are the element type `T` of the values in the array part,\nthe number `N` of dimensions of the array part and the type `P` of the object\nstoring the properties.\n\nBuilding annotated arrays is easy:\n\n```julia\nusing ArrayTools.AnnotatedArrays\ndims = (100, 50)\nT = Float32\nA = AnnotatedArray(zeros(T, dims), units = \"photons\", Δx = 0.20, Δy = 0.15)\nB = AnnotatedArray{T}(undef, dims, units = \"µm\", Δx = 0.10, Δy = 0.20)\n```\n\nHere the initial properties of `A` and `B` are specified by the keywords in the\ncall to the constructor; their properties will have symbolic names with any\nkind of value. The array contents of `A` is an array of zeros, while the array\ncontents of `B` is created by the constructor with undefined values. Indexing\n`A` or `B` with integers of Cartesian indices is the same as accessing the\nvalues of their array contents while indexing `A` or `B` by symbols is the same\nas accessing their properties. For example:\n\n```julia\nA.Δx             # yields 0.2\nA[:Δx]           # idem\nA.units          # yields \"photons\"\nA[:units]        # idem\nA[:,3] .= 3.14   # set some values in the array contents of A\nsum(A)           # yields the sum of the values of A\nA[:gizmo] = π    # set a property\nA.gizmo = π      # idem\npop!(A, :gizmo)  # yields property value and delete it\n```\n\nCreating an annotated array is summarized by:\n\n```julia\nusing ArrayTools.AnnotatedArrays\nA = AnnotatedArray(arr, prop)\nB = AnnotatedArray{T}(init, dims, prop)\n```\n\nwhere `arr` is an existing array or an expression whose result is an array,\n`prop` specifies the initial properties (more on this below), `T` is the type\nof array element, `init` is usually `undef` and `dims` is a tuple of array\ndimensions. If `arr` is an existing array, the object `A` created above will\nreference this array and hence share its contents with the caller (call\n`copy(arr)` to avoid that). The same applies if the initial properties are\nspecified by a dictionary.\n\nThe properties `prop` can be specified by keywords, by key-value pairs, as a\ndictionary or as a named tuple. To avoid ambiguities, these different styles\ncannot be mixed. Below are a few examples:\n\n```julia\nusing ArrayTools.AnnotatedArrays\narr = zeros(3,4,5)\nA = AnnotatedArray(arr,  units  =  \"µm\",  Δx  =  0.1,  Δy  =  0.2)\nB = AnnotatedArray(arr, :units  =\u003e \"µm\", :Δx  =\u003e 0.1, :Δy  =\u003e 0.2)\nC = AnnotatedArray(arr, \"units\" =\u003e \"µm\", \"Δx\" =\u003e 0.1, \"Δy\" =\u003e 0.2)\nD = AnnotatedArray(arr, (units  =  \"µm\",  Δx  =  0.1,  Δy  =  0.2))\n```\n\nThe two first examples (`A` and `B`) both yield an annotated array whose\nproperties have symbolic keys and can have any type of value. The third example\n(`C`) yields an annotated array whose properties have string keys and can have\nany type of value. The properties of `A`, `B` and `C` are *dynamic*: they can\nbe modified, deleted and new properties can be inserted. The fourth example\n(`D`) yields an annotated array whose properties are stored by a *named tuple*,\nthey are *immutable* and have symbolic keys.\n\nAccessing a property is possible via the syntax `obj[key]` or, for symbolic and\ntextual keys, via the syntax `obj.key`. Accessing *immutable* properties is the\nfastest while accessing textual properties as `obj.key` is the slowest (because\nit involves converting a symbol into a string).\n\nWhen initially specified by keywords or as key-value pairs, the properties are\nstored in a dictionary whose key type is specialized if possible (for\nefficiency) but with value type `Any` (for flexibility). If one wants specific\nproperties key and value types, it is always possible to explicitly specify a\ndictionary in the call to `AnnotatedArray`. For instance:\n\n```julia\nE = AnnotatedArray(arr, Dict{Symbol,Int32}(:a =\u003e 1, :b =\u003e 2))\n```\n\nyields an annotated array whose properties have symbolic keys and integer\nvalues of type `Int32`.\n\nProperty key types are not limited to `Symbol` or `String`, but, to avoid\nambiguities, key types must be more specialized than `Any` and must not inherit\nfrom types like `Integer` or `CartesianIndex` which are reserved for indexing\nthe array contents of annotated arrays.\n\nIf the dictionary is unspecified, the properties are stored in a, initially\nempty, dictionary with symbolic keys and value of any type, *i.e.*\n`Dict{Symbol,Any}()`.\n\nIterating on an annotated array is iterating on its array values. To iterate on\nits properties, call the `properties` method which returns the object storing\nthe properties:\n\n```julia\ndims = (100, 50)\nT = Float32\nN = length(dims)\nA = AnnotatedArray(zeros(T, dims), units = \"µm\", Δx = 0.2, Δy = 0.1)\nfor (k,v) in properties(A)\n    println(k, \" =\u003e \", v)\nend\n```\n\nSimilar types are provided by\n[MetaArrays](https://github.com/haberdashPI/MetaArrays.jl),\n[MetadataArrays](https://github.com/piever/MetadataArrays.jl) and\n[ImageMetadata](https://github.com/JuliaImages/ImageMetadata.jl).\n\n\n## General tools\n\n### Array indexing\n\nThe `all_indices` method takes any number of array arguments and yields an\nefficient iterator for visiting all indices each index of the arguments. Its\nbehavior is similar to that of `eachindex` method except that `all_indices`\nthrows a `DimensionMismatch` exception if the arrays have different axes. It is\nalways safe to specify `@inbounds` (and `@simd`) for a loop like:\n\n```julia\nfor i in all_indices(A, B, C, D)\n   A[i] = B[i]*C[i] + D[i]\nend\n```\n\nAn alternative is to call the `@assert_same_axes` macro which throws a\n`DimensionMismatch` exception if the provided arguments are not arrays with the\nsame axes. For example:\n\n```julia\n@assert_same_axes A B C D\n@inbounds for i in eachindex(A, B, C, D)\n   A[i] = B[i]*C[i] + D[i]\nend\n```\n\nwhere the macro call amounts to:\n\n```julia\naxes(A) == axes(B) == axes(C) == axes(D) ? nothing : throw(DimensionMismatch(\"...\"))\n```\n\nThe `eachindex` and `all_indices` methods are very useful when writing loops\nover array elements so as to be agnostic to which specific indexing rule is the\nmost suitable. Some algorithms are however more efficient or easier to write if\nall involved arrays are indexed by a single 1-based index. In that case, `using\nArrayTools` provides:\n\n```julia\nto_fast_array(A)\n```\n\nwhich checks whether array `A` is suitable for fast indexing (by a single\ninteger starting at 1); if it does, `A` is returned to the caller; otherwise,\nthe contents of `A` is converted to a suitable array type implementing fast\nindexing and is returned to the caller.\n\nTo just check whether array `A` is suitable for fast indexing, call:\n\n```julia\nis_fast_array(A) -\u003e bool\n```\n\nMultiple arguments can be checked at the same time: `is_fast_array(A,B,...)` is\nthe same as `is_fast_array(A) \u0026\u0026 is_fast_array(B) \u0026\u0026 is_flat_array(...)`.\n\n\n### Array storage\n\nWhen calling (with `ccall`) a compiled function coded in another language (C,\nFORTRAN, etc.), you have to make sure that array arguments have the same\nstorage as assumed by these languages so that it is safe to pass the pointer of\nthe array to the compiled function.\n\nTypically, you want to ensure that the elements are stored in memory\ncontiguously and in column-major order. This can be checked by calling:\n\n```julia\nis_flat_array(A) -\u003e bool\n```\n\nor, with several arguments:\n\n```julia\nis_flat_array(A, B, C, ...)  -\u003e bool\n```\n\nIn order to get an array with such *flat* storage and possibly with a given\nelement type `T`, call:\n\n```julia\nto_flat_array([T = eltype(A),] A)\n```\n\nwhich just returns `A` if the requirements hold or converts `A` to a suitable\narray form.\n\n\n## FAQ\n\n* What is the difference between `IndexStyle` (defined in base Julia) and\n  `IndexingTrait` (defined in `ArrayTools`)?\n\n  If `IndexStyle(A) === IndexLinear()`, then array `A` can be efficiently\n  indexed by one integer (even if `A` is multidimensional) and column-major\n  ordering is used to access the elements of `A`. The only (known) other\n  possibility is `IndexStyle(A) === IndexCartesian()`.\n\n  If `IndexingTrait(A) === FastIndexing()`, then `IndexStyle(A) ===\n  IndexLinear()` also holds (see above) **and** array `A` has standard 1-based\n  indices.\n\n* What is the difference between `Base.has_offset_axes` (provided by Julia) and\n  `has_standard_indexing` (provided by `ArrayTools`)?\n\n  For the caller, `has_standard_indexing(args...)` yields the opposite result\n  as `Base.has_offset_axes(args...)`. Furthermore, `has_standard_indexing` is a\n  bit faster.\n\n\n## Installation\n\n`ArrayTools` is an [official Julia package][julia-pkgs-url] and is easy to\ninstall. In Julia, hit the `]` key to switch to the package manager REPL (you\nshould get a `... pkg\u003e` prompt) and type:\n\n```julia\n... pkg\u003e add ArrayTools\n```\n\n[doc-stable-img]: https://img.shields.io/badge/docs-stable-blue.svg\n[doc-stable-url]: https://emmt.github.io/ArrayTools.jl/stable\n\n[doc-dev-img]: https://img.shields.io/badge/docs-dev-blue.svg\n[doc-dev-url]: https://emmt.github.io/ArrayTools.jl/dev\n\n[license-url]: ./LICENSE.md\n[license-img]: http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat\n\n[github-ci-img]: https://github.com/emmt/ArrayTools.jl/actions/workflows/CI.yml/badge.svg?branch=master\n[github-ci-url]: https://github.com/emmt/ArrayTools.jl/actions/workflows/CI.yml?query=branch%3Amaster\n\n[appveyor-img]: https://ci.appveyor.com/api/projects/status/github/emmt/ArrayTools.jl?branch=master\n[appveyor-url]: https://ci.appveyor.com/project/emmt/ArrayTools-jl/branch/master\n\n[codecov-img]: http://codecov.io/github/emmt/ArrayTools.jl/coverage.svg?branch=master\n[codecov-url]: http://codecov.io/github/emmt/ArrayTools.jl?branch=master\n\n[julia-url]: https://julialang.org/\n[julia-pkgs-url]: https://pkg.julialang.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femmt%2Farraytools.jl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Femmt%2Farraytools.jl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Femmt%2Farraytools.jl/lists"}