Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/svenvc/p3
A lean and mean PostgreSQL client for Pharo
https://github.com/svenvc/p3
database-driver glorp pharo postgresql smalltalk
Last synced: 2 months ago
JSON representation
A lean and mean PostgreSQL client for Pharo
- Host: GitHub
- URL: https://github.com/svenvc/p3
- Owner: svenvc
- License: mit
- Created: 2017-06-27T08:14:32.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2024-04-18T08:58:35.000Z (8 months ago)
- Last Synced: 2024-10-09T13:20:15.954Z (2 months ago)
- Topics: database-driver, glorp, pharo, postgresql, smalltalk
- Language: Smalltalk
- Size: 435 KB
- Stars: 72
- Watchers: 15
- Forks: 21
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# P3
P3 is a modern, lean and mean PostgreSQL client for Pharo.
[](https://github.com/svenvc/P3/actions/workflows/CI.yml)
**P3Client** uses frontend/backend protocol 3.0 (PostgreSQL version 7.4 [2003] and later),
implementing the simple and extended query cycles.
It supports plaintext, md5 and scram-sha-256 password authentication.
When SQL queries return row data, it efficiently converts incoming data to objects.
P3Client supports most common PostgreSQL types.
P3Client can be configured manually or through a URL.
```smalltalk
P3Client new url: 'psql://username:password@localhost:5432/databasename'.
```
Not all properties need to be specified, the minimum is the following URL.
```smalltalk
P3Client new url: 'psql://user@localhost'.
```
P3Client has a minimal public protocol, basically #query: (#execute: is an alias).
Opening a connection to the server (#open) and running the authentication
and startup protocols (#connect) are done automatically when needed from #query.
P3Client also supports SSL connections. Use #connectSSL to initiate such a connection.
Alternatively you can add `sslmode=require` to the connection URL, as in
`'psql://username:password@localhost:5432/databasename?sslmode=require'`.
Through the #prepare: message, you can ask P3Client to prepare/parse an SQL statement or
query with parameters. This will give you a P3PreparedStatement instance than you can then
execute with specific parameters. Polymorphic to this there is also P3FormattedStatement
which you create using the #format: message. These work at the textual, client side level.
## Basic Usage
Here is the simplest test that does an actual query, it should return true.
```smalltalk
(P3Client new url: 'psql://sven@localhost') in: [ :client |
[ client isWorking ] ensure: [ client close ] ].
```
This is how to create a simple table with some rows in it.
```smalltalk
(P3Client new url: 'psql://sven@localhost') in: [ :client |
client execute: 'DROP TABLE IF EXISTS table1'.
client execute: 'CREATE TABLE table1 (id INTEGER, name TEXT, enabled BOOLEAN)'.
client execute: 'INSERT INTO table1 (id, name, enabled) VALUES (1, ''foo'', true)'.
client execute: 'INSERT INTO table1 (id, name, enabled) VALUES (2, ''bar'', false)'.
client close ].
```
Now we can query the contents of the simple table we just created.
```smalltalk
(P3Client new url: 'psql://sven@localhost') in: [ :client |
[ client query: 'SELECT * FROM table1' ] ensure: [ client close ] ].
```
The result is an instance of P3Result
```smalltalk
a P3Result('SELECT 2' 2 records 3 columns)
```
P3Result contains 3 elements, results, descriptions & data:
- Results is a string (collection of strings for multiple embedded queries) indicating successful execution.
- Descriptions is a collection of row field description objects.
- Data is a collection of rows with fully converted field values as objects.
The data itself is an array with 2 sub arrays, one for each record.
```smalltalk
#( #(1 'foo' true) #(2 'bar' false) )
```
Finally we can clean up.
```smalltalk
(P3Client new url: 'psql://sven@localhost') in: [ :client |
[ client execute: 'DROP TABLE table1' ] ensure: [ client close ] ].
```
## References
- https://postgresql.org
- https://en.wikipedia.org/wiki/PostgreSQL
- https://www.postgresql.org/docs/9.6/static/protocol.html
## Using Prepared and Formatted Statements
Although you are free to create your SQL statements in any way you see fit,
feeding them to #execute: and #query:,
inserting arguments in SQL statements can be hard (because you have to know the correct syntax),
error-prone (because you might violate syntax rules) and dangerous (due to SQL injection attacks).
P3 can help here with two mechanisms: prepared and formatted statements.
They are mostly polymorphic and use the same template notation.
They allow you to create a statement once, specifying placeholders with $n,
and execute it once or multiple times with concrete arguments,
with the necessary conversions happening automatically.
The difference between the two is that formatted statements are implemented
using simple textual substitution on the client side, while
prepared statements are evaluated on the server side with full syntax checking,
and are executed with more type checks.
Prepared statements are more efficient since the server can do part of its optimization
in the prepare phase, saving time on each execution.
Here is a transcript of how to use them. First we set up a client and create a test table.
```smalltalk
client := P3Client new url: 'psql://sven@localhost'.
client execute: 'DROP TABLE IF EXISTS table1'.
client execute: 'CREATE TABLE table1 (id INTEGER, name TEXT, weight REAL, enabled BOOLEAN)'.
```
Next we insert some data and then query it using prepared statements.
```smalltalk
statement := client prepare: 'INSERT INTO table1 (id, name, weight, enabled) VALUES ($1, $2, $3, $4)'.
statement execute: { 1. 'foo'. 75.5. true }.
statement executeBatch: { { 2. 'bar'. 80.25. true }. { 3. 'foobar'. 10.75. false } }.
statement close.
statement := client prepare: 'SELECT id, name, weight FROM table1 WHERE id = $1 AND enabled = $2'.
statement query: { 1. true }.
statement close.
```
Note that prepared statements are server side resources that need to be closed when no longer needed.
Prepared statements exist in the scope of a single session/connection.
Next we start over and do the same insert and query using formatted statements.
```smalltalk
client execute: 'TRUNCATE TABLE table1'.
statement := client format: 'INSERT INTO table1 (id, name, weight, enabled) VALUES ($1, $2, $3, $4)'.
statement execute: { 1. 'foo'. 75.5. true }.
statement executeBatch: { { 2. 'bar'. 80.25. true }. { 3. 'foobar'. 10.75. false } }.
statement := client format: 'SELECT id, name, weight FROM table1 WHERE id = $1 AND enabled = $2'.
statement query: { 1. true }.
```
And finally we clean up.
```smalltalk
client execute: 'DROP TABLE table1'.
client close.
```
## Supported Data Types
P3 supports most common PostgreSQL types. Here are some tables with the details.
As of PostgreSQL 9.6, there are 41 general purpose data types of which 32 are currently implemented.
These are the 32 general purpose data type currently implemented,
with the Pharo class they map to.
Name | Alias | Description | Oid | Class
-----|-------|-------------|-----|------
bigint | int8 | signed eight-byte integer | 20 | Integer
bigserial | serial8 | autoincrementing eight-byte integer | 20 | Integer
bit [n] | | fixed-length bit string | 1560 | P3FixedBitString
bit varying | varbit | variable-length bit string | 1562 | P3BitString
boolean | bool | logical boolean (true/false) | 16 | Boolean
box | | rectangular box on a plane (upperright, lowerleft) | 603 | P3Box
bytea | | binary data (byte array) | 17 | ByteArray
character [n] | char | fixed-length character string | 1042 | String
character varying | varchar | variable-length character string | 1043 | String
circle | | circle on a plane (center, radius) | 718 | P3Circle
date | | calendar date (year,month,day) | 1082 | Date
double precision | float8 | double precision floating point number (8 bytes) | 701 | Float
integer | int, int4 | signed four-byte integer | 23 | Integer
interval | | time span | 114 | P3Interval
json | | textual JSON data | 114 | NeoJSONObject
jsonb | | binary JSON data, decomposed | 3802 | NeoJSONObject
line | | infinite line on a plane (ax+by+c=0) | 628 | P3Line
lseg | | line segment on a plane (start,stop) | 601 | P3LineSegment
numeric | decimal | exact number of selectable precision | 1700 | ScaledDecimal
path | | geometric path on a plane (points) | 602 | P3Path
point | | geometric point on a plane (x, y) | 600 | P3Point
polygon | | closed geometric path on a plane (points) | 604 | P3Polygon
real | float4 | single-precision floating point number (4-bytes) | 700 | Float
smallint | int2 | signed two-byte integer | 21 | Integer
smallserial | serial2 | autoincrementing two-byte integer | 21 | Integer
serial | serial4 | autoincrementing four-byte integer | 23 | Integer
text | | variable-length character string | 25 | String
time [ without time zone ] | | time of day (no time zone) | 1083 | Time
time with time zone | timetz | time of day including time zone | 1266 | Time
timestamp [ without time zone ] | | date and time (no time zone) | 1114 | DateAndTime
timestamp with time zone | timestamptz | date and time includig time zone | 1184 | DateAndTime
uuid | | universal unique identifier | 2950 | UUID
Here are the 9 general purpose data types that are not yet implemented.
Name | Description | Oid
-----|-------------|----
cidr | IPv4 or IPv6 network address | 650
inet | IPv4 or IPv6 host address | 869
macaddr | MAC (Media Access Control) address | 829
money | currency amount | 790
pg_lsn | PostgreSQL Log Sequence Number | 3220
tsquery | text search query | 3615
tsvector | text search document | 3614
txid_snapshot | user-level transaction ID snapshot | 2970
xml | XML data | 142
Additionally, the following 10 common types are also implemented,
with the Pharo class they map to.
Name | Description | Oid | Class
-----|-------------|-----|------
oid | object identifier | 26 | Integer
name | name | 19 | String
bpchar | text | 1042 | String
void | void | 2278 | UndefinedObject
_bool | boolean array | 1000 | Array\
_int4 | integer array | 1007 | Array\
_oid | oid array | 1028 | Array\
_text | string array | 1009 | Array\
_varchar | string array | 1015 | Array\
_float8 | float array | 1022 | Array\
P3 also supports enums. Each enum definition creates a new type.
You can send #loadEnums to P3Client to create mappings for all visible enums.
When you do a query that results in data of an unknown type you will get an error,
P3 cannot convert typeOid XXX, where XXX is the oid in the pg_type table.
## Connection and Authentication
P3 connects over the network (TCP) to PostgreSQL and
supports plain (#connect) and TLS/SSL encrypted (#connectSSL) connections.
It is out of the scope of this README to explain how to install and configure
an advanced database like PostgreSQL. There is extensive high quality documentation
available covering all aspect of PostgreSQL, see https://postgresql.org
Out of the box, most PostgreSQL installations do not allow for network connections
from other machines, only for local connections.
Check the listen_addresses directive in postgresql.conf
As for authentication, CleartextPassword, MD5Password and SCRAM-SHA-256 are supported.
This means that SCMCredential, GSS, SSPI are currently not (yet) supported.
An error will be signalled when the server requests an unsupported authentication.
You have to create database users, called roles and give them a password.
In SQL you can do this with CREATE|ALTER ROLE user1 LOGIN PASSWORD 'secret'
Next you have to tell PostgreSQL how network users should authenticate themselves.
This is done by editing pg_hba.conf choosing specific methods,
trust (no password, no authentication), password, md5 and scram-sha-256 work with P3.
Note that for SCRAM-SHA-256 to work, you need to change the password_encryption
directive in postgresql.conf to scam-sha-256, restart and reenter all user passwords.
## Glorp
Included is **P3DatabaseDriver**, an interface between Glorp, an advanced object-relational mapper, and P3Client.
To install this driver (after loading Glorp itself), do
```smalltalk
PharoDatabaseAccessor DefaultDriver: P3DatabaseDriver.
```
Configure your session using a Glorp Login object
```smalltalk
Login new
database: PostgreSQLPlatform new;
username: 'username';
password: 'password';
connectString: 'host:5432_databasename';
encodingStrategy: #utf8;
yourself.
```
## Code loading
The default group loads P3Client and its basic dependencies NeoJSON and ZTimestamp
```smalltalk
Metacello new
baseline: 'P3';
repository: 'github://svenvc/P3';
load.
```
The glorp group loads P3DatabaseDriver and the whole of Glorp (warning: large download)
```smalltalk
Metacello new
baseline: 'P3';
repository: 'github://svenvc/P3';
load: 'glorp'.
```
## Unit tests
**P3ClientTest** holds unit tests for the P3 PSQL client.
Configure it by setting its class side's connection URL.
```smalltalk
P3ClientTest url: 'psql://sven:secret@localhost:5432/database'.
```
The minimal being the following:
```smalltalk
P3ClientTest url: 'psql://sven@localhost'.
```
## Logging
P3 uses object logging, an advanced form of code instrumentation.
This means that during execution instances of subclasses of P3LogEvent
are created (some including timing information)
and sent to an Announcer (accessible via P3LogEvent announcer).
Interested parties can subscribe to these log events
and use the information contained in them
to learn about P3 code execution.
The standard print method of a P3LogEvent can be used to generate textual output.
The following expression enables logging to the Transcript.
```smalltalk
P3LogEvent logToTranscript.
```
Executing the four expressions of the Basic Usage section yields the following output.
```bash
2020-09-21 16:27:57 001 [P3] 63731 #Connect sven@localhost:5432 Trust
2020-09-21 16:27:57 002 [P3] 63731 #Query SELECT 565 AS N
2020-09-21 16:27:57 003 [P3] 63731 #Result SELECT 1, 1 record, 1 colum, 4 ms
2020-09-21 16:27:57 004 [P3] 63731 #Close
2020-09-21 16:28:07 005 [P3] 63733 #Connect sven@localhost:5432 Trust
2020-09-21 16:28:07 006 [P3] 63733 #Query DROP TABLE IF EXISTS table1
2020-09-21 16:28:07 007 [P3] 63733 #Error P3Notification PostgreSQL table "table1" does not exist, skipping
2020-09-21 16:28:07 008 [P3] 63733 #Result DROP TABLE, 6 ms
2020-09-21 16:28:07 009 [P3] 63733 #Query CREATE TABLE table1 (id INTEGER, name TEXT, enabled BOOLEAN)
2020-09-21 16:28:07 010 [P3] 63733 #Result CREATE TABLE, 50 ms
2020-09-21 16:28:07 011 [P3] 63733 #Query INSERT INTO table1 (id, name, enabled) VALUES (1, 'foo', true)
2020-09-21 16:28:07 012 [P3] 63733 #Result INSERT 0 1, 4 ms
2020-09-21 16:28:07 013 [P3] 63733 #Query INSERT INTO table1 (id, name, enabled) VALUES (2, 'bar', false)
2020-09-21 16:28:07 014 [P3] 63733 #Result INSERT 0 1, 0 ms
2020-09-21 16:28:07 015 [P3] 63733 #Close
2020-09-21 16:28:20 016 [P3] 63737 #Connect sven@localhost:5432 Trust
2020-09-21 16:28:20 017 [P3] 63737 #Query SELECT * FROM table1
2020-09-21 16:28:20 018 [P3] 63737 #Result SELECT 2, 2 records, 3 columns, 2 ms
2020-09-21 16:28:20 019 [P3] 63737 #Close
2020-09-21 16:39:52 020 [P3] 63801 #Connect sven@localhost:5432 Trust
2020-09-21 16:39:52 021 [P3] 63801 #Query DROP TABLE table1
2020-09-21 16:39:52 022 [P3] 63801 #Result DROP TABLE, 13 ms
2020-09-21 16:39:52 023 [P3] 63801 #Close
```
Remember that the information inside the log events can be used to build other applications.
## Development, Goals, Contributing
The main goal of P3 is to be a modern, lean and mean PostgreSQL client for Pharo.
Right now, P3 is functional and usable.
The quality of open source software is determined by it being alive, supported and maintained.
The first way to help is to simply use P3 in your projects and tells us about
your successes and the issues that you encounter.
You can ask questions on the Pharo mailing lists.
Development happens on GitHub, where you can create issues.
Contributions should be done with pull requests solving specific issues.