{"id":19484952,"url":"https://github.com/opencog/atomspace-bridge","last_synced_at":"2026-01-25T07:03:54.854Z","repository":{"id":138151283,"uuid":"579515435","full_name":"opencog/atomspace-bridge","owner":"opencog","description":"Read/write interfaces between AtomSpace and SQL databases.","archived":false,"fork":false,"pushed_at":"2025-08-15T20:36:05.000Z","size":222,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-08-15T22:42:57.807Z","etag":null,"topics":["atomspace","database","postgresql","sql"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/opencog.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-18T00:01:30.000Z","updated_at":"2025-08-15T20:36:09.000Z","dependencies_parsed_at":"2024-04-27T19:28:46.726Z","dependency_job_id":null,"html_url":"https://github.com/opencog/atomspace-bridge","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/opencog/atomspace-bridge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Fatomspace-bridge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Fatomspace-bridge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Fatomspace-bridge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Fatomspace-bridge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/opencog","download_url":"https://codeload.github.com/opencog/atomspace-bridge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/opencog%2Fatomspace-bridge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28747308,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T05:12:38.112Z","status":"ssl_error","status_checked_at":"2026-01-25T05:04:50.338Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atomspace","database","postgresql","sql"],"created_at":"2024-11-10T20:24:51.621Z","updated_at":"2026-01-25T07:03:54.837Z","avatar_url":"https://github.com/opencog.png","language":"C++","readme":"AtomSpace-SQL DB Bridge\n=======================\nThe bridge connects pre-existing SQL DB's to the AtomSpace, so that\ndata can move between the two, in both directions.  It is meant to allow\nthe AtomSpace to access foreign, external data, and manipulate it using\nthe AtomSpace software stack.  This is *not* intended for generic\nsave/restore of AtomSpace contents. For that, see the native\n[atomspace-rocks](https://github.com/opencog/atomspace-rocks) RocksDB\n[StorageNode](https://wiki.opencog.org/w/StorageNode) and the\n[atomspace-cog](https://github.com/opencog/atomspace-cog) network\nStorageNode.\n\nThe following are desired features:\n* Automatic creation of maps from the SQL tables to AtomSpace\n  structures. Fully automated, by importing pre-existing SQL table\n  schemas. Optionally, this can be overloaded with custom templates.\n* Custom mappings or templates would allow more \"natural\"\n  [Atomese](https://wiki.opencog.org/w/Atomese) representations.\n  Complex SQL systems tend to have ugly, convoluted table structures\n  with complex primary-key/foreign-key interactions. It would be nice\n  to hide this in the AtomSpace mapping, so that we don't import the\n  grunge and cruft of the SQL table designs.\n* The bridge should allow both the read and the update of the contents\n  (rows) in the SQL tables (while maintaining all foreign-key\n  constraints during writing.)\n* The bridge loads data incrementally. The bridge does not require\n  a bulk import/export, but rather will access individual rows, columns\n  and tables on an as-needed basis. This would allow the AtomSpace to\n  work with datasets that are too big to fit in RAM.\n* Adherence to the [AtomSpace Sensory](https://github.com/opencog/sensory)\n  perception-action API for general agents.\n\nAt this time, only Postgres is supported.\n\nStatus\n------\n***Version 0.2.1*** -- The current implementation can load table\ndescriptions and table contents from Postgres DB's. It can also\nload related rows and columns (rows and columns joined by a common\ncolumn name). Two demos: a basic demo, showing the basic idea, and\nan ASCII table browser, allowing you to pilot around, bouncing from\ntable to table, along joins. It's an ASCII browser because all of\nthe graphical browsers for the AtomSpace have been neglected.\n(These: [atomspace-explorer](https://github.com/opencog/atomspace-explorer),\n[atomspace-typescript](https://github.com/opencog/atomspace-typescript) and\n[cogprotolab](https://github.com/opencog/cogprotolab).)\n\nHistory\n-------\nPer request of Mike Duncan, Dec 2022. Map the\n[FlyBase Drosophila Genome Database](http://flybase.org)\n(current release\n[here](https://ftp.flybase.net/releases/FB2022_06/psql/FB2022_06.sql.gz))\ninto the AtomSpace, using the\n[Chado](http://gmod.org/wiki/Chado)\nschema given [here](http://gmod.org/wiki/Chado_Tables).\n\nGeneric Mapping\n---------------\nHere's a sketch for a generic mapping. This could work for *any*\nSQL database.\n\nFor each row in some table `tablename`, create a conventional\nAtomese [EdgeLink](https://wiki.opencog.org/w/EdgeLink).\n```\n   Edge                      ; A single row in the table.\n      Predicate \"tablename\"  ; Replace by actual name of table.\n      List                   ; This list is one row in the table.\n         Concept \"thing\"     ; Entry in that column, if its a string.\n         NumberNode 42       ; If that column type is a number.\n         ...\n         Concept \"stuffs\"    ; Last column in the table.\n```\n\nPrimary and Foreign Keys\n------------------------\nThe biggest design question is what to do with primary and foreign keys.\nThe simplest solution is to \"do nothing\" and just import rows from\nSQL straight-up. Whatever primary/foreign keys are in the SQL tables,\nthey will also appear in the AtomSpace. They will join automatically,\ndue to fundamental AtomSpace architecture. With this solution, writing\ncomplex queries appears to be straight-forward (and easier than writing\nSQL queries).  This works and is surprisingly flexible.\n\nOther mappings are possible; however, there is no natural way of asking\nPostgres which table columns are foreign keys, and which other tables\nthey might reference. If this info was available, then we could have a\nan Atomese representation that does NOT keep any keys at all in the\nAtomSpace, and instead just makes direct links between table rows.\nSo, for example:\n\n```\n   Edge\n      Predicate \"some join relation\"\n      Set\n         Edge   ;;; row in the host table\n            Predicate \"some tablename\"\n            List ...\n\n         Edge   ;; row in the target table.\n            Predicate \"another tablename\"\n            List ...\n\n         Edge   ;; if more than two tables are joined.\n            Predicate \"yet another tablename\"\n            List ...\n```\nSee the OpenCog wiki:\n* [EdgeLink](https://wiki.opencog.org/w/EdgeLink)\n* [PredicateNode](https://wiki.opencog.org/w/PredicateNode)\n\nFrom what I can tell, the above alternate form uses about the same\namount of RAM as having explicit keys in each row. It also takes\njust about the same amount of time to query over. So it does not\nseem to offer any size or performance advantage over brute-force\nprimary/foreign keys.\n\nIt does seem to make things more readable, more \"natural\" in Atomese.\nIt provides more flexibility: you can create and destroy joins at any\ntime. You can join some rows but not others. All the usual stuff that\nmakes the AtomSpace more flexible than SQL.\n\nFor just right now, we punt, and store the naked PRIMARY/FOREIGN KEY\nvalues as Atoms in the AtomSpace. It feels ugly, but it works.\n\nTable Schemas\n-------------\nSQL table definitions provide a definition of the columns of that\ntable.  These are imported into the AtomSpace, where they are used\nto find the names of the columns, and the types. (Note that the\n`EdgeLink` example above does not contain the column names.)\nBelow is the current mapping.\n\nThe SQL column names are recorded as\n[VariableNode](https://wiki.opencog.org/w/VariableNode) names.\nThe type of each variable *aka* column is given.\n```\n    Signature\n       Predicate \"tablename\"    ;; Replace by actual table name.\n       List\n          TypedVariable\n              Variable \"name of column 1\"\n              Type 'ConceptNode ;; For SQL text/varchar\n          TypedVariable\n              Variable \"name of column 2\"\n              Type 'NumberNode  ;; For SQL numbers\n          ...\n```\n\nSee the OpenCog wiki:\n* [SignatureLink](https://wiki.opencog.org/w/SignatureLink)\n* [VariableNode](https://wiki.opencog.org/w/VariableNode)\n* [TypedVariable](https://wiki.opencog.org/w/TypedVariable)\n* [TypeNode](https://wiki.opencog.org/w/TypeNode)\n\nThe base AtomSpace supports only a few primitive types; there's no\ndirect analog to the varied primitive types that SQL has. So far,\nit doesn't seem to be needed.  Perhaps the set of primitive types\ncould be enriched, e.g. by creating:\n* `StringNode` for SQL `TEXT` and `VARCHAR` (instead of ConceptNode)\n* `DateNode` for SQL dates and times.\n* `IntegerNode` (the existing\n  [NumberNode](https://wiki.opencog.org/w/NumberNode) is a vector of\n  floats. It works great, but some people just love ints.)\n\nSpecific databases might benefit from custom types:\n* `GeneNode`\n* `ProteinNode`\n* `URLNode`\n* `HumanReadableDescriptionNode`\n* `YourFavoriteIdeaHereNode`\n\nThe [agi-bio](https://github.com/opencog/agi-bio) project provides\nthe first two types. The\n[cheminformatics](https://github.com/opencog/cheminformatics) project\nprovides types for atomic elements and molecular binding.\n\nUsing\n-----\nBelow is a sketch of how things could work. For actual examples that\nactually run and actually work correctly, see the\n[examples directory](./examples).\n\nExamples of accessing data in a foreign database.\n\n```\n; Describe where it is located.\n(define foreign-db\n   (BridgeStorageNode \"postgres://example.com/foo?user=foo\u0026passwd=bar\"))\n\n; Open it.\n(cog-open foreign-db)\n\n; Load the *entire* table `gene.allele`. Optional; only if you\n; actually want the whole table in RAM. The table name is known\n; in advance, and is given directly.\n(fetch-incoming-set (Predicate \"gene.allele\"))\n\n; Instead of loading entire tables, perhaps we only want all\n; rows of all tables that mention gene CG7069.\n(fetch-incoming-set (Concept \"CG7069\"))\n\n; Perhaps we plan to do a join. So, load *all* tables that have\n; a given column name. In this case, all tables having a column\n; called \"genotype\".\n(fetch-incoming-set (Variable \"genotype\"))\n\n; StorageNodes do have the ability to run generic queries, and\n; we could, in principle, translate at least some of the simpler\n; Atomese queries into SQL, and run those.\n(fetch-query (Meet (Edge (Predicate \"gene.allele\")\n    (List (Concept \"CG7069\") (Glob \"rest of the row\")))))\n\n; The fetch-query function is already built into the base, core\n; AtomSpace. We could also create custom functions:\n(cog-bridge-get-row \"gene.allele\" (Concept \"CG7069\"))\n\n; Custom functions are not appealing, since they don't work\n; with the rest of the StorageNode and ProxNode infrastructure.\n```\n\nIt could be convenient to introduce special-purpose\n[Atom types](https://wiki.opencog.org/w/Atom_types), such as\n`GeneNode` from the agi-bio project. This would allow queries such\nas\n```\n(fetch-incoming-set (Gene \"CG7069\"))\n```\nwhich could be marginally more efficient, presuming that the\n`gene.allele` table schema was properly declared:\n```\n    Signature\n       Predicate \"gene.allele\"\n       VariableList\n          TypedVariable\n              Variable \"symbol\"\n              Type 'GeneNode\n          TypedVariable\n              Variable \"is_alleleof\"\n              Type 'NumberNode\n          TypedVariable\n              Variable \"propagate_transgenic_uses\"\n              Type 'NumberNode\n          TypedVariable\n              Variable \"gene_is_regulatory_region\"\n              Type 'NumberNode\n          ...\n```\n\nBuilding and Installing\n-----------------------\nThis module works. It can load tables, it can load joining columns,\nand it can join rows, all \"automatically\".  The build process is\nidentical to that of other modules in OpenCog.\n\n### Prerequisites\n\n##### AtomSpace\n* The AtomSpace, of course.\n* https://github.com/opencog/atomspace\n* It uses exactly the same build procedure as this package. Be sure\n  to `sudo make install` at the end.\n\n##### Postgres\n* SQL Database\n* https://postgres.org | `apt install postgresql postgresql-client libpq-dev`\n\n### Building this module\n\nBe sure to install the pre-requisites first!\nPerform the following steps at the shell prompt:\n```\n    cd to project root dir\n    mkdir build\n    cd build\n    cmake ..\n    make -j4\n    sudo make install\n    make -j4 check\n```\n\nOpen Questions\n--------------\nSome issues to ponder:\n* What's the right solution to the foreign-key dilemma?\n\n* The flybase dataset seems to have a large number of extremely complex\n  SQL stored procedures. What are they? What do they do? What should we\n  do with them?\n\n* What to do with views?\n\n* What if the same table appears in multiple schemas?\n\n-----------------------------------------------------------------------\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Fatomspace-bridge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencog%2Fatomspace-bridge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencog%2Fatomspace-bridge/lists"}