{"id":29639192,"url":"https://github.com/neotomadb/neotomapostgres","last_synced_at":"2026-02-23T01:31:35.730Z","repository":{"id":45466858,"uuid":"513654415","full_name":"NeotomaDB/NeotomaPostgres","owner":"NeotomaDB","description":"Docker image of the Neotoma SQL database","archived":false,"fork":false,"pushed_at":"2025-04-18T04:18:34.000Z","size":93,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-07-21T20:16:35.464Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NeotomaDB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"code_of_conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-07-13T19:57:48.000Z","updated_at":"2025-04-18T04:18:28.000Z","dependencies_parsed_at":"2025-04-20T08:30:40.412Z","dependency_job_id":null,"html_url":"https://github.com/NeotomaDB/NeotomaPostgres","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NeotomaDB/NeotomaPostgres","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotomaPostgres","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotomaPostgres/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotomaPostgres/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotomaPostgres/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NeotomaDB","download_url":"https://codeload.github.com/NeotomaDB/NeotomaPostgres/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NeotomaDB%2FNeotomaPostgres/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29734468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T20:09:16.275Z","status":"ssl_error","status_checked_at":"2026-02-22T20:09:13.750Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-21T20:08:22.547Z","updated_at":"2026-02-23T01:31:35.710Z","avatar_url":"https://github.com/NeotomaDB.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- badges: start --\u003e\n\n[![lifecycle](https://img.shields.io/badge/lifecycle-stable-green.svg)](https://www.tidyverse.org/lifecycle/#stable) [![NSF-1948926](https://img.shields.io/badge/NSF-1948926-blue.svg)](https://nsf.gov/awardsearch/showAward?AWD_ID=1948926)\n\u003c!-- badges: end --\u003e\n\n# Neotoma Database Postgres Container\n\nThis repository contains the instructions to build and download a Docker container provides the most recent snapshot of the [Neotoma Paleoecology Database](https://neotomadb.org) in a Posgres RMDS. This container will allow the user to use Postgres and PostGIS capabilities without having to directly install the software on their own computer.\n\nThe intended user is a \"power user\" of the Neotoma Database, who has some familiarity with the database and the database structure, and has taken time to examine the updated [Neotoma Database Manual](). Individuals can connect directly to the container using the commandline `psql` application if they have a local installation of Postgres, using a database GUI such as pgAdmin or DBeaver, or using R or Python.\n\nIn each case, the Docker container exposes a port (in the examples below we use port `2022`) on the `localhost` for the database `neotoma`.\n\nThe main Docker image is stored in DockerHub (and supported by this public git repository). Associated utilites perform commandline or `psql` operations that are intended to occur periodically. By associating them with this repository and the Docker container we can ensure that any operations do not affect the main Neotoma database, or any of its associated services.\n\n## Contributors\n\nThis project is an open project, and contributions are welcome from any individual.  All contributors to this project are bound by a [code of conduct](CODE_OF_CONDUCT.md).  Please review and follow this code of conduct as part of your contribution.\n\n* [![orcid](https://img.shields.io/badge/orcid-0000--0002--7926--4935-brightgreen.svg)](https://orcid.org/0000-0002-7926-4935) [Socorro Dominguez Vidana](https://sedv8808.github.io/)\n* [![orcid](https://img.shields.io/badge/orcid-0000--0002--2700--4605-brightgreen.svg)](https://orcid.org/0000-0002-2700-4605) [Simon Goring](http://goring.org)\n\n### Tips for Contributing\n\nIssues and bug reports are always welcome.  Code clean-up, and feature additions can be done either through pull requests to [project forks](https://github.com/NeotomaDB/neotoma2/network/members) or [project branches](https://github.com/NeotomaDB/neotoma2/branches).\n\nAll products of the Neotoma Paleoecology Database are licensed under an [MIT License](LICENSE) unless otherwise noted.\n\n## Container Use\n\nUsers can either build the container locally, or pull a pre-compiled image from DockerHub.  Assuming that Docker is currently installed and running on the user's computer, the easiest way to run the container is to use `docker run`. If you do not have an account on DockerHub (and choose not to create one), or if you wish to build the image from scratch locally, you can use the instructions for Building the Image to create and use the container.\n\n### Using `docker run`\n\nIf using the DockerHub image, you will need a [Docker account](https://hub.docker.com/) with a username and password.  The `docker run` command will begin by asking for these credentials. The image is hosted on Socorro Dominguez's DockerHub account, [`sedv8808`](https://hub.docker.com/u/sedv8808), and the following commandline arguments will set up the container locally, with Docker running the Postgres server, and connecting the container port `5432` (the default Postgres port) to a localhost port `2022`. *Please continue reading below for a note about the total build time.*\n\n```bash\ndocker run --name test -d -p 2022:5432 -v $PWD/out:/tmp -e POSTGRES_PASSWORD=postgres sedv8808/neotoma_postgres postgres\n```\n\nInformation about `docker run` options and tags can be found in the [documentation for `docker run`](https://docs.docker.com/engine/reference/commandline/run/). Here we describe the specific tags used in the above command:\n\n* `-- name` : This is the Docker image name, here we are setting the name to `test`, but this can be changed as needed.\n* `-d` : Run the container in the background (detached). This frees up the commandline terminal once the container is started.\n* `-p` : Define the localhost port that open container ports will point to. Postgres, by default, uses port `5432`. We use the format `USER`:`CONTAINER`, so this is saying that the user's `localhost:2022` is connected to the container's `localhost:5432`.  You can use `5432:5432` if you wish, but this may cause a conflict if there is a local version of Postgres running.\n* `-v` : Link external folders to a folder within the container.  Again we use the `USER`:`CONTAINER` format. Here, the `$PDW/out` folder (`$PWD` representing the current working directory) is linked to the folder `tmp` within the container. We use the root `/tmp` folder so that any user (e.g., `postgres` user) can have access to the connected folder.\n* `-e POSTGRES_PASSWORD` : Within the container set the environment variable `POSTGRES_PASSWORD`, which will be used for the default `postgres` user.\n\nThe docker image itself lives at `sedv8808/neotoma_postgres` on DockerHub, and the term `postgres` at the end of the `docker run` command initializes Postgres within the container.\n\nOnce you're done with the image you can stop it from running in memory using `docker stop test` (depending on the name you give it), and, if you want to start it again you can use `docker start test`.  This will preserve any changes you have made to the container so that (for example) if you've deleted records in the container, or made modifications to the database structure, those changes will persist when you open it again.\n\nIf you made changes to the container that you do not want to persist, you will stop the container, delete it, and then, you can rebuild it:\n\n```bash\ndocker stop test\ndocker rm test\ndocker run --name test -d -p 2022:5432 -v $PWD/out:/tmp -e POSTGRES_PASSWORD=postgres sedv8808/neotoma_postgres postgres\n```\n\nIn this case we won't see the files downloading, but the container will take several minutes to initialize as it downloads the latest snapshot of Neotoma and restores it locally. Running `psql` too early will result in the error:\n\n```bash\n$ psql -h localhost -p 2022 -U postgres -d neotoma\npsql: error: connection to server at \"localhost\" (127.0.0.1), port 2022 failed: server closed the connection unexpectedly\n        This probably means the server terminated abnormally\n        before or while processing the request.\n```\n\nTiming with the `timetesting.sh` script (included) seems to indicate an average of between 90 -- 120s before the container is fully ready for interaction through `psql` or other DB tools.\n\n#### In Plain English\n\nWe take the image at `sedv8808/neotoma_postgres` and build it locally, naming it `test`. Within the container we execute the command `postgres`, which starts the Postgres server. Traffic through the container's port `5432` is connected to the user port `2022`, and all files in the user's `out` folder, are connected to the folder called `/tmp` inside the container, which allows us to move files out of the container if needed.\n\n### Building the Image Locally\n\nThis GitHub repository includes a Dockerfile. If you would rather build the image in your local machine:\n\n* Clone the repository locally using `git clone https://github.com/NeotomaDB/NeotomaPostgres`\n* Navigate to the folder using the commandline, and run:\n\n```bash\ndocker build -t neotoma_postgres .\n```\n\nWith the container now built and stored in your system you can start running Postgres. To start the image, first run:\n\n```bash\ndocker run --name test -d -p 2022:5432 -v $PWD/out:/tmp --rm -e POSTGRES_PASSWORD=postgres neotoma_postgres postgres\n```\n\nThere is likely a delay required to wait while the `sql` dump is restored locally. Running `psql` too early will result in the error:\n\n```bash\n$ psql -h localhost -p 2022 -U postgres -d neotoma\npsql: error: connection to server at \"localhost\" (127.0.0.1), port 2022 failed: server closed the connection unexpectedly\n        This probably means the server terminated abnormally\n        before or while processing the request.\n```\n\nLocal testing by SJG suggests that this can take ~ 1.5 minutes.\n\n#### Pushing to DockerHub\n\nIf you make changes to the Docker container and wish to save the modified image for future use on other computers/servers, you can create a DockerHub account, and create the image remotely using your `\u003cusername\u003e` (*e.g.*, `sjgoring`) and a `\u003ctag\u003e` for the image (for example, `mybuild`):\n\n```bash\ndocker tag neotoma_postgres:latest \u003cusername\u003e/neotoma_postgres:\u003ctag\u003e\ndocker push \u003cusername\u003e/neotoma_postgres:\u003ctag\u003e\n```\n\nFor example:\n\n```bash\ndocker tag neotoma_postgres:latest sjgoring/neotoma_postgres:latest\ndocker push sjgoring/neotoma_postgres:latest\n```\n\n## Working with the Container\n\n### With Postgres Installed locally\n\nAssuming you have Postgres install locally you can work directly with the Neotoma image using `psql`:\n\n#### Querying Neotoma Directly\n\nHere we will run the Docker container, open `psql` and export the results of a query to file:\n\n```bash\n$ docker run --name test -d -p 2022:5432 -v $PWD/out:/tmp -e POSTGRES_PASSWORD=postgres sedv8808/neotoma_postgres postgres\n$ psql -h localhost -p 2022 -d neotoma -U postgres\n\nneotoma=# \\copy (SELECT * FROM ndb.sites WHERE sitename LIKE 'Lake%') TO '/tmp/lakesofneotoma.csv' csv;\n```\n\nAfter opening up `psql` at the appropriate port and with the appropriate user name, we use the `\\copy` command for `psql` to define where the output should go.  In this case we're sending it to our `tmp` folder.  We then query for all sites in Neotoma that start with the word *Lake*, and make sure we define the output as a csv. When we exit the container using `docker stop test`, the data remains in the `tmp` folder.\n\n## Connecting with R\n\nGiven that you've started the Docker container before starting R (and waited the required time, as indicated above), you can start up R and use the `RPostgreSQL` package:\n\n```R\nlibrary(RPostgreSQL)\n\nconn \u003c- dbConnect(dbDriver('PostgreSQL'), \n                 dbname = 'neotoma',\n                 host = 'localhost', \n                 port = 2022,\n                 user = 'postgres', \n                 password = 'postgres')\n\nsites \u003c- dbGetQuery(conn, \"SELECT * FROM ndb.sites WHERE sitename LIKE 'Lake%'\")\nplot(sites$latitudenorth ~ sites$longitudeeast)\n```\n\n![A plot of the sites selected using the query showing the global coverage of Neotoma data.](img/neotomasiteimages.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneotomadb%2Fneotomapostgres","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneotomadb%2Fneotomapostgres","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneotomadb%2Fneotomapostgres/lists"}