{"id":18420585,"url":"https://github.com/openworm/owmeta-core","last_synced_at":"2025-04-07T13:32:09.260Z","repository":{"id":37091743,"uuid":"238332665","full_name":"openworm/owmeta-core","owner":"openworm","description":"Core library for owmeta","archived":false,"fork":false,"pushed_at":"2025-03-18T00:12:04.000Z","size":4841,"stargazers_count":3,"open_issues_count":41,"forks_count":2,"subscribers_count":11,"default_branch":"develop","last_synced_at":"2025-03-18T00:29:54.820Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openworm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-05T00:03:39.000Z","updated_at":"2025-03-18T00:12:04.000Z","dependencies_parsed_at":"2024-08-03T11:28:58.801Z","dependency_job_id":"6fd4be4c-2517-4a8b-8689-ce547e0e9ed5","html_url":"https://github.com/openworm/owmeta-core","commit_stats":null,"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openworm%2Fowmeta-core","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openworm%2Fowmeta-core/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openworm%2Fowmeta-core/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openworm%2Fowmeta-core/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openworm","download_url":"https://codeload.github.com/openworm/owmeta-core/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247661879,"owners_count":20975140,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T04:22:18.699Z","updated_at":"2025-04-07T13:32:05.460Z","avatar_url":"https://github.com/openworm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build develop](https://github.com/openworm/owmeta-core/actions/workflows/scheduled-dev-build.yml/badge.svg)](https://github.com/openworm/owmeta-core/actions/workflows/scheduled-dev-build.yml)[![Docs](https://readthedocs.org/projects/owmeta-core/badge/?version=latest)](https://owmeta-core.readthedocs.io/en/latest)\n[![Coverage Status](https://coveralls.io/repos/github/openworm/owmeta-core/badge.svg?branch=develop)](https://coveralls.io/github/openworm/owmeta-core?branch=develop)\n\nowmeta-core\n===========\nowmeta-core helps with sharing relational data over the Internet and keeping\ntrack of where that data comes from. Exactly *how* that is achieved is best\nunderstood through demonstration, which you can find in the \"Usage\" section\nbelow.\n\nInstall\n-------\nTo install owmeta-core, you can use pip:\n\n    pip install owmeta-core\n\nUsage\n-----\nFirst of all, owmeta-core wraps [RDFLib][rdflib]:\n\n    \u003e\u003e\u003e from owmeta_core import connect\n    \u003e\u003e\u003e with connect() as conn: # creates an in-memory graph\n    ...     conn.rdf\n    \u003cGraph identifier=... (\u003cclass 'owmeta_core.data._Dataset'\u003e)\u003e\n\nThat means you can do several of the same things as you would do with RDFLib\nalone.[^1] You can configure different backing stores as well:\n\n    \u003e\u003e\u003e with connect({\"rdf.store\": \"FileStorageZODB\",\n    ...               \"rdf.store_conf\": \"./example.db\"}) as conn:\n    ...     pass\n\n(This will make a few files, named like `example.db*`, in the current working\ndirectory.)\n\nAssuming you've added some data to your graph, you may want to share it. In\nowmeta-core, the primary way to share things is via \"Bundles\". Bundles are,\nessentially, serialized collections of named graphs with additional attributes\nfor identifying them. To create a bundle, you \"install\" it, collecting the\nnamed graphs you want to include in the bundle, serializing them, and putting\nthem in a particular file structure. Here's how you can do that:\n\n    \u003e\u003e\u003e from owmeta_core.bundle import Installer, Descriptor\n    \u003e\u003e\u003e from rdflib.term import URIRef\n\n    \u003e\u003e\u003e with connect().transaction() as conn:\n    ...     # add some stuff to http://example.org/ctx ...\n    ...     g = conn.rdf.graph(URIRef('http://example.org/ctx'))\n    ...     _ = g.add((URIRef('http://example.org/s'),\n    ...            URIRef('http://example.org/p'),\n    ...            URIRef('http://example.org/o')))\n    ...     inst = Installer('.', './bundles', conn.rdf)\n    ...     desc = Descriptor(\"a-bundle\",\n    ...             version=1,\n    ...             includes=['http://example.org/ctx'])\n    ...     bundle_directory = inst.install(desc)\n\nSo, let's unpack that a little. First we add some things to named graphs. How\nyou do this is up to you, but above is a trivial example of adding a statement\nto an in-memory graph that we'll ultimately include in the bundle. Then, we\ncreate an `Installer` that will install bundles to the directory \"bundles\" in\nthe current working directory. Installers get instructed on what to install\nthrough `Descriptor` objects. We create a bundle descriptor that says what the\nbundle identifier is (\"a-bundle\") what version of the bundle we're installing\n(1) and which contexts we're including in the bundle (just\n`http://example.org/ctx`).  We pass the descriptor to the installer's `install`\nmethod and it does creates the bundle file structure under `bundles`.\n\nWe haven't *shared* the bundle with anyone yet with the above. You may choose\nto package the bundle into an archive and share that somehow (E-mail, shared\nfile storage, etc.), and `owmeta_core.bundle.archive.Archiver` can help with\nthat. This code creates a bundle archive named 'a-bundle.tar.xz' in the current\ndirectory:\n\n    \u003e\u003e\u003e from owmeta_core.bundle.archive import Archiver\n\n    \u003e\u003e\u003e # Save a bundle a-bundle.tar.xz in the current working directory\n    \u003e\u003e\u003e Archiver('.').pack(\n    ...        bundle_directory=bundle_directory,\n    ...        target_file_name='a-bundle.tar.xz')\n    './a-bundle.tar.xz'\n\nThere are, however, facilities for uploading and downloading bundles through\nowmeta-core. owmeta-core has the concept of \"Remotes\" which are places bundles\ncan be sent to or retrieved from. A remote has a set of \"accessor configs\"\n(ACs) which provide information for getting bundles. An AC may include a URL\nor other pieces of information, like authentication credentials. A remote may\nbe set up as the upload target or download source for a single bundle or\nmultiple bundles, and it may have multiple ACs for different protocols or for\nthe same protocol (e.g., for download mirrors). Here's how we can define a\nremote:\n\n    \u003e\u003e\u003e from owmeta_core.bundle import Remote, URLConfig\n\n    \u003e\u003e\u003e a_remote = Remote('a-bundle-server',\n    ...    [URLConfig('https://example.org/bundle-server')])\n\nHere's how we can upload to a remote:\n\n    \u003e\u003e\u003e from owmeta_core.bundle import Deployer\n\n    \u003e\u003e\u003e deployer = Deployer([a_remote])\n    \u003e\u003e\u003e deployer.deploy(bundle_directory)  # doctest: +SKIP\n\nAssuming that `a_remote` reflects an actual HTTP server that can accept POST\nrequests with bundle archives in the request body, this would upload the\nbundle. (As written, this sends the bundle to Downloading bundles is similar:\n\n    \u003e\u003e\u003e from owmeta_core.bundle import Fetcher\n\n    # Fetch version 3 of \"another-bundle\" and install it to \"./bundles\"\n    \u003e\u003e\u003e fetcher = Fetcher('./bundles', [a_remote])\n    \u003e\u003e\u003e fetcher.fetch('another-bundle', bundle_version=3)  # doctest: +SKIP\n\nOf course, once we can fetch bundles from remotes, we might want to actually\nuse them. While it is possible to parse the serialized graphs from the\n\"./bundles\" directory, that would discard some useful information about the\nbundle, including dependencies between bundles (not discussed above).[^2]\nInstead, it's recommended to use `Bundle` objects to access bundle data. Here's\nan example:\n\n    \u003e\u003e\u003e from owmeta_core.bundle import Bundle\n\n    \u003e\u003e\u003e with Bundle('a-bundle', version=1, bundles_directory='./bundles') as bnd:\n    ...    g = bnd.rdf.get_context('http://example.org/ctx')\n    ...    assert (URIRef('http://example.org/s'),\n    ...            URIRef('http://example.org/p'),\n    ...            URIRef('http://example.org/o')) in g\n\n\nHere we take `a-bundle` version 1, cached underneath the `./bundles` directory,\nand get an RDFLib `Dataset` with the data from the bundle.\n\n[rdflib]: https://rdflib.readthedocs.io/en/stable/\n\n### RDF \u003c-\u003e Python object mapping\n\nWhen sharing data we have the problem of data independence: dealing with schema\nchanges of the underlying RDF. owmeta-core tries to deal with this problem by\nproviding tools for constructing adaptable two-way mappings between RDF and\nPython objects. These mappings relate [RDF classes][rdf_class] to Python\nclasses, which classes are constrained to be sub-classes of `BaseDataObject`.\nThe mappings are stored in what we call the \"class registry\" which is itself a\nconstruct described using RDF.\n\nAdding classes to the mapping proceeds in phases. First, you define sub-classes\nof BaseDataObject for the entities you want to represent in RDF. Then, you save\ndescriptions of the classes to a local database. After you've released or\npublished the classes in software, you can publish the RDF descriptions with\nreference to the published software so that others can find the code from the\npublished data.\n\nMaking the classes is easy enough. For basic classes, you can define them like\nthis:\n\n    \u003e\u003e\u003e from owmeta_core.dataobject import BaseDataObject, DatatypeProperty\n\n    \u003e\u003e\u003e class Jar(BaseDataObject):\n    ...     content_type = DatatypeProperty()\n    ...     content_amount = DatatypeProperty()\n    ...     volume = DatatypeProperty()\n\nFor more information, check out the [owmeta-core docs][owcdocs].\n\n[owcdocs]: https://owmeta-core.readthedocs.io/en/latest/making_dataObjects.html\n\n\u003c!-- TODO: Make it easier to save classes without a project --\u003e\nIt's easiest to manage the RDF/Python mapping in the context of an owmeta-core\n\"project\".  You can create a project in the current directory like this:\n\n    owm init .\n\nThis creates a `.owm` directory in the current directory that keeps the project\nconfiguration. You can then save the classes to the local database, located\nunder the project directory with this command, assuming you've saved the `Jar`\nclass module as `glue_factory.containers`:\n\n    owm save glue_factory.containers\n\nIt's recommended to include \"module accessor\" descriptions for the \"mapped\"\nclasses. Module accessors describe how to get the source code module in which\nthe class is defined. For modules in PyPI, owmeta-core provides some short-cuts\nfor declaring module-accessors. For example, say you have `BaseDataObject`\nclasses defined in a package called \"glue-factory\": you can link that package\nand the means of accessing it to those classes in the class registry with a\ncommand like this:\n\n    owm registry module-access declare python-pip glue-factory\n\nAssuming you have `glue-factory` installed in the same Python environment[^3] as\n`owm`, the installed version of the package will be associated with all of the\nmodules previously added to the project database.\n\nTools to support evolution of these mappings are still being developed, but\nhere are a few recommendations:\n\n - In order to keep old code working through upgrades of your package, try to\n   create a new Python class for each new version of the corresponding RDF\n   class and publish both in new versions of your software package. If\n   instances of the old class are valid instances of the new one, then you can\n   add the `rdf:type` triples pointing to the new class from those instances. A\n   similar thing can be done if instances of the new RDF class can be handled\n   with the old Python class.\n - Indicate the required software packages in the `description` field of your\n   bundle.\n\n[rdf_class]: https://www.w3.org/TR/rdf-schema/#ch_classes\n\n\nNotes\n-----\n[^1]: You can also create an `rdflib.graph.Graph` rather than a `Dataset` by\n   defining a new `owmeta_core.data.RDFSource` and assigning it to\n   `conn.conf[\"rdf.graph\"]`. This turns out to not be especially useful in\n   owmeta-core, but it is possible.\n[^2]: Bundle dependency information isn't stored as RDF. It likely will be\n   eventually, to allow other software to query bundle relationships without\n   needing to understand the particular format of bundle manifests and the\n   bundle cache file tree.\n[^3]: Environment here means either the system Python installation, a \"virtual\n   environment\", a Jupyter notebook, etc. Basically, it's whatever\n   `importlib.metadata` looks for for distributions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenworm%2Fowmeta-core","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenworm%2Fowmeta-core","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenworm%2Fowmeta-core/lists"}