{"id":15662146,"url":"https://github.com/jgdavey/clj-pgcopy","last_synced_at":"2025-05-05T23:07:30.076Z","repository":{"id":62431479,"uuid":"197096401","full_name":"jgdavey/clj-pgcopy","owner":"jgdavey","description":"Import data into postgres quickly from Clojure","archived":false,"fork":false,"pushed_at":"2025-03-23T19:02:25.000Z","size":54,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-05T23:07:16.384Z","etag":null,"topics":["clojure","postgres","postgresql"],"latest_commit_sha":null,"homepage":"","language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jgdavey.png","metadata":{"files":{"readme":"README.org","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-16T01:14:58.000Z","updated_at":"2025-04-23T13:16:15.000Z","dependencies_parsed_at":"2022-11-01T21:00:35.274Z","dependency_job_id":null,"html_url":"https://github.com/jgdavey/clj-pgcopy","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgdavey%2Fclj-pgcopy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgdavey%2Fclj-pgcopy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgdavey%2Fclj-pgcopy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jgdavey%2Fclj-pgcopy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jgdavey","download_url":"https://codeload.github.com/jgdavey/clj-pgcopy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252590609,"owners_count":21772937,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clojure","postgres","postgresql"],"created_at":"2024-10-03T13:30:22.469Z","updated_at":"2025-05-05T23:07:30.060Z","avatar_url":"https://github.com/jgdavey.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"#+TITLE: clj-pgcopy\n\n#+html: \u003ca href=\"https://clojars.org/clj-pgcopy\"\u003e\u003cimg src=\"https://img.shields.io/clojars/v/clj-pgcopy.svg\" alt=\"Clojars\" /\u003e\u003c/a\u003e\n\nImport data into postgres quickly, implemented using postgresql's\n=COPY= in binary format.\n\nBecause sometimes =jdbc/insert!= and friends aren't fast enough.\n\nThis library uses type-based dispatch for determining the correct\npostgres binary format. See the [[Input Type mapping][mapping]] section for more info.\n\n* Usage\n\nAdd the library to your project. See [[https://clojars.org/clj-pgcopy][description on Clojars]] for more\ninformation.\n\n#+BEGIN_SRC clojure :exports none\n(require '[clojure.java.jdbc :as jdbc])\n\n(def conn-spec \"jdbc:postgresql://localhost:5432/test_pgcopy\")\n#+END_SRC\n\n#+RESULTS:\n: nil#'user/conn-spec\n\nGiven a table and some data:\n\n#+BEGIN_SRC clojure :exports code\n(require '[clj-pgcopy.core :as pgcopy])\n\n(jdbc/with-db-connection [conn conn-spec]\n  (jdbc/db-do-commands conn\n                       [\"drop table if exists example\"\n                        \"create table example(\n  internal_id bigint primary key,\n  external_id uuid,\n  height float8,\n  title varchar(64) not null,\n  description text,\n  values bytea,\n  dob date,\n  created_at timestamptz\n)\"]))\n\n(def data\n  [{:internal_id 201223123\n    :external_id #uuid \"1902c205-2bc6-40b8-943b-f5b199241316\"\n    :height nil\n    :title \"Mr. Sandman\"\n    :description nil\n    :values (.getBytes \"not very secret\" \"UTF-8\")\n    :dob (java.time.LocalDate/of 1954 8 20)\n    :created_at (java.util.Date.)}\n   {:internal_id 2012391238\n    :external_id nil\n    :height 160.2\n    :title \"Prince\"\n    :description \"Tonight we're gonna party\"\n    :values (.getBytes \"2000 Two Zero\" \"UTF-8\")\n    :dob (java.time.LocalDate/of 1999 12 31)\n    :created_at (java.util.Date.)}])\n\n#+END_SRC\n\n#+RESULTS:\n: nil(0 0)#'user/data\n\nWith =clojure.java.jdbc=, open a connection, prepare data rows (as\ntuples, not maps), and call =clj-pgcopy.core/copy-into!=:\n\n#+BEGIN_SRC clojure :exports both\n(let [columns [:internal_id :external_id :height\n               :title :description :values :dob :created_at]]\n  (jdbc/with-db-connection [conn conn-spec]\n    (pgcopy/copy-into! (:connection conn)\n                       :example\n                       columns\n                       (map (apply juxt columns) data))))\n#+END_SRC\n\n#+RESULTS:\n: 2\n\nThe table has been populated with the data:\n\n#+BEGIN_SRC clojure :exports both :results pp\n(jdbc/with-db-connection [conn conn-spec]\n  (jdbc/query conn \"table example\"))\n#+END_SRC\n\n#+RESULTS:\n#+begin_example\n({:internal_id 201223123,\n  :external_id #uuid \"1902c205-2bc6-40b8-943b-f5b199241316\",\n  :height nil,\n  :title \"Mr. Sandman\",\n  :description nil,\n  :values\n  [110, 111, 116, 32, 118, 101, 114, 121, 32, 115, 101, 99, 114, 101,\n   116],\n  :dob #inst \"1954-08-20T04:00:00.000-00:00\",\n  :created_at #inst \"2019-07-23T01:24:38.466000000-00:00\"}\n {:internal_id 2012391238,\n  :external_id nil,\n  :height 160.2,\n  :title \"Prince\",\n  :description \"Tonight we're gonna party\",\n  :values [50, 48, 48, 48, 32, 84, 119, 111, 32, 90, 101, 114, 111],\n  :dob #inst \"1999-12-31T05:00:00.000-00:00\",\n  :created_at #inst \"2019-07-23T01:24:38.466000000-00:00\"})\n#+end_example\n\nNote: depending on how you've set up =clojure.java.jdbc= and its\n=IResultSetReadColumn= protocol, the types that come back on query may\ndiffer from the above.\n\n* Input Type mapping\n\n** Basic type mapping\n\n| JVM type       | Postgres type                 |\n|----------------+-------------------------------|\n| Short          | int2 (aka smallint)           |\n| Integer        | int4 (aka integer)            |\n| Long           | int8 (aka bigint)             |\n| Float          | float4 (aka real)             |\n| Double         | float8 (aka double presicion) |\n| BigDecimal     | numeric/decimal               |\n| Boolean        | boolean                       |\n| String         | text/varchar/char             |\n| java.util.UUID | uuid                          |\n\n** Date-related mappings\n\n| JVM type                     | Postgres type |\n|------------------------------+---------------|\n| java.sql.Date                | date          |\n| java.time.LocalDate          | date          |\n| java.util.Date               | timestamp[tz] |\n| java.sql.Timestamp           | timestamp[tz] |\n| java.time.Instant            | timestamp[tz] |\n| java.time.ZonedDateTime      | timestamp[tz] |\n| java.time.OffsetDatetime     | timestamp[tz] |\n| org.postgres.util.PGInterval | interval      |\n\n** Other JVM native types\n\n| JVM type              | Postgres type                 |\n|-----------------------+-------------------------------|\n| java.net.Inet4Address | inet                          |\n| java.net.Inet6Address | inet                          |\n\n** Geometric mappings\n\n| JVM type                         | Postgres type |\n|----------------------------------+---------------|\n| org.postgres.geometric.PGpoint   | point         |\n| org.postgres.geometric.PGline    | line          |\n| org.postgres.geometric.PGpath    | path          |\n| org.postgres.geometric.PGbox     | box           |\n| org.postgres.geometric.PGcircle  | circle        |\n| org.postgres.geometric.PGpolygon | polygon       |\n\nThings that are String-like, or serialized in string form, should work\nusing the String -\u003e text mapping. An exception is the =jsonb= type,\nbecause the binary format requires a version signifier. Wrapping a\nJSON string in a =JsonB= handles that, which is provided by the\nlibrary.\n\n** Arrays\n\nImpemented for the following JVM-typed arrays for:\n\n| JVM type         | Postgres type                     |\n|------------------+-----------------------------------|\n| int[]            | int4[] (aka integer[])            |\n| long[]           | int8[] (aka bigint[])             |\n| float[]          | float4[] (aka real[])             |\n| double[]         | float8[] (aka double precision[]) |\n| byte[]           | bytea                             |\n| String[]         | text[] (or varchar[])             |\n| java.util.UUID[] | uuid[]                            |\n\n\nCurrently, only 1-dimensional Postgres arrays are supported.\n\n** Planned But Not Yet Implemented\n\n- more array types (date, timestamp, etc)\n- range types\n\n** Unplanned\n\n- hstore\n- cidr, macaddr, macaddr8\n- bit strings\n- composite types / records\n- multi-dimensional arrays\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjgdavey%2Fclj-pgcopy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjgdavey%2Fclj-pgcopy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjgdavey%2Fclj-pgcopy/lists"}