{"id":29639210,"url":"https://github.com/neotomadb/neotoma_sql","last_synced_at":"2025-08-31T16:34:11.459Z","repository":{"id":48474091,"uuid":"92864137","full_name":"NeotomaDB/Neotoma_SQL","owner":"NeotomaDB","description":"Repository for functions associated with Neotoma's Postgres DB","archived":false,"fork":false,"pushed_at":"2025-02-05T04:18:10.000Z","size":5741,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":6,"default_branch":"production","last_synced_at":"2025-02-05T05:22:40.191Z","etag":null,"topics":["neotoma","paleoecology","postgres","sql"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NeotomaDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-30T18:33:18.000Z","updated_at":"2025-02-05T04:18:14.000Z","dependencies_parsed_at":"2025-02-05T05:31:55.280Z","dependency_job_id":null,"html_url":"https://github.com/NeotomaDB/Neotoma_SQL","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NeotomaDB/Neotoma_SQL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotoma_SQL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotoma_SQL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotoma_SQL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotoma_SQL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NeotomaDB","download_url":"https://codeload.github.com/NeotomaDB/Neotoma_SQL/tar.gz/refs/heads/production","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotoma_SQL/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266371486,"owners_count":23918862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-21T11:47:31.412Z","response_time":64,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["neotoma","paleoecology","postgres","sql"],"created_at":"2025-07-21T20:08:30.566Z","updated_at":"2025-07-21T20:08:30.783Z","avatar_url":"https://github.com/NeotomaDB.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neotoma_SQL\n\nThis is a repository to store the functions used within the Neotoma Database for data access and updates.  The Neotoma is a Postgres-based system, migrated from a SQL Server database.  As such, at the current time, this repository largely reflects the state of transition between the two systems.\n\n## Contributors\n\nAll individuals are welcome to contribute to this repository.  Contributions are subject to the [Code of Conduct](https://github.com/neotomadb/Neotoma_SQL/blob/master/code_of_conduct.md) for this repository.\n\n* Steve Crawford - [Penn State](http://www.ems.psu.edu/node/147)\n* Simon Goring   - [University of Wisconsin](http://goring.org)\n* Mike Stryker   - [Penn State](http://www.ems.psu.edu/node/2892)\n* Anna George    - University of Wisconsin\n* Jack Williams  - University of Wisconsin\n\n## Description\n\nThe repository is divided into two main folders, `legacy` and `function`:\n\n* `legacy` folder is intended to act as the store for the now deprecated SQL Server Stored Procedures.\n* `function` folder represents the newer Postgres functions.  This folder is then divided into folders for each individual database schema.\n\nThis structure is not necessary, but has been implemented to help manage the workflow of rewriting the large number of functions associated with the original database.\n\n### Neotoma Postgres Databases\n\nNeotoma maintains five separate databases within the PSU Postgres server.  Of these five, three are significant, each has a specific function:\n\n* `neotomadev`: Main development database.  Currently this is the parent database and should be the most accurate/up to date version of the database.  This is the database that the API points to.\n* `neotoma`: A database for experimentation.  This version of the database may be out of sync with the SQL Server and `neotomadev` as it is used to build and prototype new tools.\n* `neotomatilia`: A database for testing and managing Tilia services.\n\n### Associated Files\n\nThere are several files that have been used in the transition and are included here for posterity.\n\n* [`rewrite_funs.R`](https://github.com/NeotomaDB/Neotoma_SQL/blob/master/rewrite_funs.R) is a short R script that was intended to rewrite any function in the legacy `TI` schema that used the format `SELECT * FROM table`.\n* [`tilia_check.py`](https://github.com/NeotomaDB/Neotoma_SQL/blob/master/tilia_check.py) was intended to check against existing functions in the `ti` namespace to help ensure alignment with the Tilia software package designed to work with the Neotoma Paleoecology Database.\n\n### Maintaining the Repository\n\nThis repository is currently intended to be **read-only** with respect to the databases (`neotoma`, `neotomadev` and `neotomatilia`).  The stored postgres functions can be updated using the Python3 script `connect_remote.py` by calling:\n\n```python\npython3 connect_remote.py\n```\n\nThis program checks the `pg_catalog` for the stated database and pulls each function within a defined set of namespaces, and returns each function as its own `sql` file to a folder in the `function` directory.\n\n#### Using `connect_remote.py`\n\n`connect_remote.py` has built in help that can be accessed using:\n\n```python3\nconnect_remote.py -h\n```\n\n```\nusage: connect_remote.py [-h] [-dev] [-push] [-g [PULLGIT]] [-tilia]\n\nCheck Neotoma SQL functions against functions in the online database servers\n(`neotoma` and `neotomadev`).\n\noptional arguments:\n  -h, --help    show this help message and exit\n  -dev          Use the `dev` database? (`False` without the flag)\n  -push         Assume that SQL functions in the repository are newer, push to\n                the db server.\n  -g [PULLGIT]  Pull from the remote git server before running?.\n  -tilia        Use the `dev` database? (`False` without the flag)\n```\n\nA user should make sure that local changes are committed and pushed before using the script, but in the case that changes were made directly to the GitHub repository online, they may use the `-g` flag.\n\nExpected use should `-push` to each of the three databases.\n\n### Standards For New Functions\n\nFunctions are expected to follow (as much as possible) [Simon Holywell's SQL Style Guide](http://www.sqlstyle.guide/) and the [Postgres Coding Convention](https://www.postgresql.org/docs/current/static/source.html).\n\n* Functions should be generated in standalone files\n* Functions should be named using verbs\n* Functions should refrain from using vendor specific functions\n* Use C style commenting: /* . . . */\n\n### Existing Database Schema\n\nFor more insight into the structure of the Neotoma Database (specifically the `ndb` schema, which contains the core data of the database), visit the [Neotoma Database Documentation](http://neotomadb.github.io/dbschema/index.html).  Documentation was generated using [the SchemaSpy software](http://schemaspy.org/).\n\n## Issues and Bugs\n\nPlease feel free to raise issues using the issue tracker on this repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneotomadb%2Fneotoma_sql","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneotomadb%2Fneotoma_sql","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneotomadb%2Fneotoma_sql/lists"}