{"id":22731782,"url":"https://github.com/fenhl/lazyjson","last_synced_at":"2025-04-14T00:45:12.233Z","repository":{"id":12733281,"uuid":"15406213","full_name":"fenhl/lazyjson","owner":"fenhl","description":"lazy JSON I/O in Python","archived":false,"fork":false,"pushed_at":"2022-07-29T23:54:15.000Z","size":40,"stargazers_count":21,"open_issues_count":1,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-27T14:55:06.362Z","etag":null,"topics":["json","python-library","python3"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fenhl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-12-23T22:35:21.000Z","updated_at":"2025-02-28T20:50:58.000Z","dependencies_parsed_at":"2022-09-23T09:53:31.102Z","dependency_job_id":null,"html_url":"https://github.com/fenhl/lazyjson","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fenhl%2Flazyjson","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fenhl%2Flazyjson/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fenhl%2Flazyjson/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fenhl%2Flazyjson/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fenhl","download_url":"https://codeload.github.com/fenhl/lazyjson/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248804721,"owners_count":21164127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","python-library","python3"],"created_at":"2024-12-10T19:30:41.029Z","updated_at":"2025-04-14T00:45:12.212Z","avatar_url":"https://github.com/fenhl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"**lazyjson** is a module for Python 3.2 or higher that provides lazy JSON I/O.\n\nThis is `lazyjson` version 3.0.2 ([semver](https://semver.org/)). The versioned API is described below, in the section *API*.\n\n# Usage\n\nFirst you need a lazyjson object, which can represent a JSON formatted file on the file system:\n\n```python\n\u003e\u003e\u003e import lazyjson\n\u003e\u003e\u003e f = lazyjson.File('example-file.json')\n```\n\nor even a remote file:\n\n```python\n\u003e\u003e\u003e import lazyjson\n\u003e\u003e\u003e f = lazyjson.SFTPFile('example.com', 22, '/foo/bar/example-file.json', username='me')\n```\n\nYou can then use the `File` object like a regular `dict`:\n\n```python\n\u003e\u003e\u003e print(f)\n{'someKey': ['a', 'b', 'c']}\n```\n\nOkay, so far so good. But why not just use [`json.load`](https://docs.python.org/3/library/json.html#json.load)? Well, this is where the “lazy” part comes in. Let's say some other program modifies our `example-file.json`. Let's do the same call again, still on the same `File` object:\n\n```python\n\u003e\u003e\u003e print(f['someKey'])\n['a', 'b', 'c', 'd']\n```\n\nThe result has magically changed, because the file is actually read each time you get data from it. If you have write permission, you can also modify the file simply by changing the values from the `File` object:\n\n```python\n\u003e\u003e\u003e f['someKey'] += ['e', 'f']\n\u003e\u003e\u003e with open('example-file.json') as text_file:\n...     print(text_file.read())\n... \n{\n    \"someKey\": [\n        \"a\",\n        \"b\",\n        \"c\",\n        \"d\",\n        \"e\",\n        \"f\"\n    ]\n}\n```\n\n# API\n\nLazyjson 2 provides the `BaseFile` ABC and the concrete subclasses `File`, `CachedFile`, `HTTPFile`, `MultiFile`, `PythonFile`, and `SFTPFile`.\n\n## BaseFile\n\n`BaseFile` inherits from `Node`, and represents its own root node (see below).\n\nIt has 4 abstract methods:\n\n* `__eq__`\n* `__hash__`\n* `set`: write the JSON value passed as argument to the file.\n* `value`: read and return the JSON value from the file.\n\nBoth methods should handle native Python objects, as used in the [`json`](docs.python.org/3/library/json.html) module.\n\nIt also has an `__init__` method that takes no arguments and must be called from subclasses' `__init__`.\n\n## File\n\nWhen instantiating a `File`, the first constructor argument must be one of the following:\n\n* a valid single argument to the built-in function [`open`](https://docs.python.org/3/library/functions.html#open),\n* an open [file object](https://docs.python.org/3/glossary.html#term-file-object),\n* or an [instance](https://docs.python.org/3/library/functions.html#isinstance) of [`pathlib.Path`](https://docs.python.org/3/library/pathlib.html#pathlib.Path).\n\nThe optional `file_is_open` argument can be used to force appropriate behavior for a file that is already open, or one that will be opened on each read or write access. By default, behavior depends on whether the file argument inherits from `io.IOBase`.\n\nIf a `json.decoder.JSONDecodeError` is encountered while reading the file and the `File` isn't in `file_is_open` mode, another attempt is made after 1 second. This avoids intermittent errors when the file is accessed while also in the middle of being written to disk. The optional `tries` argument specifies how many read attempts should be made before reraising the `JSONDecodeError`. The default value for this is `10`.\n\nIf the optional `init` argument is given and the file does not exist, it will be created and the argument is encoded and written to the file. For an open file object, this parameter is ignored.\n\nAny other keyword arguments, such as `encoding`, will be passed to the `open` calls.\n\nNote that constructing a `File` from a file object may result in unexpected behavior, as lazyjson uses `.read` on the file every time a value is accessed, and `.write` every time one is changed. The file object must also support [`str`](https://docs.python.org/3/library/stdtypes.html#str) input for changes to succeed.\n\n## CachedFile\n\nThe `CachedFile` class takes a mutable mapping `cache` and another `BaseFile`. Any access of the `CachedFile`'s value will be retrieved from the cache if present, otherwise the inner `BaseFile`'s value is stored in the cache and returned.\n\nThis class performs *no* cache invalidation whatsoever except when the `CachedFile`'s value is modified.\n\n## HTTPFile\n\nThe `HTTPFile` class uses [requests](http://python-requests.org/) to represent a JSON file accessed via [HTTP](https://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol).\n\nThe constructor takes the request URL as a required positional-only argument. An optional `post_url` argument may also be given, which will then be used as the URL for POST requests when mutating the file. By default, the same request URL will be used.\n\nAny other keyword arguments will be passed to the request as [parameters](http://docs.python-requests.org/en/latest/api/#requests.request) (except for `json` which will be overwritten for POST requests).\n\n## MultiFile\n\nA `MultiFile` represents a stack of JSON files, with values higher up on the stack extending or overwriting those below them.\n\nThe constructor takes a variable number of positional arguments, which should all be instances of `BaseFile` subclasses. These will become the file stack, listed from top to bottom.\n\nWhen reading a `MultiFile`, a single file representation is created by recursively merging/overriding the values in the file stack. Two objects are merged into one, and all other types of JSON values, as well as an object and a different value, overwrite each other.\n\nWhen writing to a `MultiFile`, only the topmost file is ever modified. It will be modified in such a way that using the reading algorithm on the multifile will have the intended effect. The only exception is deleting pairs from a JSON object, which is undefined behavior.\n\n**Note:** the exact writing behavior of `MultiFile` is undefined and may change at any point without requiring a major release.\n\n## PythonFile\n\nThis class makes a lazyjson file out of a native Python object, as defined by the `json` module. It can be used in a `MultiFile` to provide a fallback value.\n\n## SFTPFile\n\nThe `SFTPFile` class uses [paramiko](https://github.com/paramiko/paramiko) to represent a JSON file accessed via [SFTP](https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol).\n\nThe constructor takes the following required positional-only arguments:\n\n* the hostname\n* the port (usually 22)\n* the remote path/filename\n\nAny keyword arguments are passed to [`connect`](https://docs.paramiko.org/en/1.15/api/transport.html#paramiko.transport.Transport.connect) on the [`paramiko.Transport`](https://docs.paramiko.org/en/1.15/api/transport.html#paramiko.transport.Transport) object.\n\nIf not passed to the constructor, the keyword argument `pkey` is initialized from the file `~/.ssh/id_rsa`, and `hostkey` from `~/.ssh/known_hosts`.\n\nThe file is fetched from the SFTP connection on each read, no caching is performed.\n\n## Node\n\nA node represents a JSON value, such as an entire file (the root node), or a value inside an array inside an object inside the root node.\n\nNodes representing JSON arrays or objects can mostly be used like Python `list`s and `dict`s, respectively: they can be indexed, sliced, and iterated over as usual. Some of these operations may return nodes, or succeed even for missing keys. Trying this on primitive nodes (numbers, strings, booleans, or null) is undefined behavior.\n\nUnder the hood, the `Node` object only holds a reference to its file (`BaseFile` subclass instance), and its key path. All data is lazily read from, and immediately written to the file each time it is accessed. This means that you can have `Node` objects representing nonexistent nodes. This will become apparent when calling `value` on this node raises an exception.\n\nSome methods to note:\n\n* `__iter__` takes a snapshot of the keys/indices at the time of being called, and always yields nodes: for object nodes, it behaves similar to the `values` method.\n* `get` is overridden to always return a native Python object. It returns the value at the specified key, index, or slice if it exists, or the default value provided otherwise.\n* `set` can be used to directly change the value of this node.\n* `value` returns the JSON value of the node as a native Python object, similar to [`json.load`](https://docs.python.org/3/library/json.html#json.load).\n\nAnd the properties:\n\n* `key` returns the last element in the `key_path` (see below), or `None` for the root node.\n* `key_path` returns a list of keys (strings, integers, or slices) which lead from the root node to this node. For example, in `{\"one\": \"eins\", \"two\": [\"dos\", \"deux\"]}`, the `\"dos\"` would have a key path of `[\"two\", 0]`. The root node's key path is `[]`.\n* `parent` returns the parent node, or `None` for the root node.\n* `root` returns the root node of this file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffenhl%2Flazyjson","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffenhl%2Flazyjson","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffenhl%2Flazyjson/lists"}