https://github.com/darold/external_file
Allow access to "external files" from PostgreSQL server file systems.
https://github.com/darold/external_file
Last synced: about 1 year ago
JSON representation
Allow access to "external files" from PostgreSQL server file systems.
- Host: GitHub
- URL: https://github.com/darold/external_file
- Owner: darold
- License: other
- Created: 2015-01-22T19:08:39.000Z (over 11 years ago)
- Default Branch: master
- Last Pushed: 2023-01-14T14:52:47.000Z (over 3 years ago)
- Last Synced: 2025-03-27T10:38:14.466Z (about 1 year ago)
- Language: Raku
- Size: 27.3 KB
- Stars: 24
- Watchers: 8
- Forks: 8
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG
- License: LICENSE
Awesome Lists containing this project
README
External file access extension
==============================
Allow access to "external files" from PostgreSQL server file systems.
This extension adds the same functionalities given by the Oracle's BFILE data
type that stores unstructured binary data in flat files outside the database.
A BFILE column stores a file locator that points to an external file containing
the data: (DIRECTORY, FILENAME). Here the data type is called EFILE.
The extension access to external files using secure call to the server side `lo_*`
functions and not by directly reading/writing to files.
Installation requirements
-------------------------
PostgreSQL 9.1 or better are required.
User with PostgreSQL superuser role for creating extension.
Installation
------------
external_file has been written as a PostgreSQL extension and uses the Extension
Building Infrastructure "PGXS".
You will need PostgreSQL headers and PGXS installed (if your PostgreSQL was
installed with packages, install the development package).
Get/Unpack the source code in a fresh directory Then the software installation
should be as simple as
$ make (In these version do nothing)
$ make install
To install the extension in a database, connect as superuser and
CREATE EXTENSION external_file;
By default all objects of the extension are created in the `external_file` schema.
If you want to change the schema name you must edit the `external_file.control`
file. Note that this schema must not be writable by normal user to not allow
bypassing of the search path set with the security definer.
When using schema with extension, it's better to include this schema in the
default search path. For example:
ALTER DATABASE SET search_path="$user",public,external_file;
Also you can restrict USAGE grant on `external_file` schema to specific user and
change the default search path at user level too.
GRANT USAGE ON SCHEMA external_file TO ;
Please refer to the PostgreSQL documentation for more information.
Usage
-----
External file are accessed using two values, an alias for the path of the
directory where the file is, and the file name.
So, first, alias must be defined for the path. This definition is performed
using the "directories" table. For security reason, only superuser can insert,
update, delete directory definition. It's possible, with GRANT command, to
change this but it's NOT recommended.
Example:
INSERT INTO directories(directory_name,directory_path) VALUES ('temporary','/tmp/');
ATTENTION:
* the directory path must use the terminal file system separator!
* the system user running PostgreSQL server (generally postgres) must have the
system rights to read and/or write files. See `pg_read_server_files` and
`pg_write_server_files` privileges introduced in PostgreSQL 11.
* the filename don't include any / or \ character for security reason
Second, rights for user and/or role are defined using the "directory_access"
table.
Example:
INSERT INTO directory_roles(directory_name,directory_role,directory_read,directory_write) VALUES ('temporary','a_role',true,false);
Now standard user can use external files.
Example:
-- Store a new external file blahblah.txt into the directory
SELECT writeEfile('\x48656c6c6f2c0a0a596f75206172652072656164696e67206120746578742066696c652e0a0a526567617264732c0a', ('temporary', 'blahblah.txt'));
ls -la /tmp/blahblah.txt
-rw-r--r-- 1 postgres postgres 47 janv. 22 19:16 /tmp/blahblah.txt
-- Create a table that will use external files
CREATE TABLE efile_test ( id smallint primary key, the_file efile);
-- Insert a row to access the external file called blahblah.txt
INSERT INTO efile_test VALUES (1,('temporary','blahblah.txt'));
-- Assuming user has right to read, and the file exists
SELECT id, readefile(the_file) FROM efile_test;
-- Make a physical copy of the external file assuming user has right to read AND write
SELECT copyefile(('temporary','blahblah.txt'),('temporary','copy_blahblah.txt'));
INSERT INTO efile_test VALUES (2, ('temporary','copy_blahblah.txt'):::efile);
-- or the equivalent using efilename()
INSERT INTO efile_test VALUES (3, efilename('temporary','copy_blahblah.txt'));
ls /tmp/*blahblah.txt
-rw-r--r-- 1 postgres postgres 47 janv. 22 19:16 /tmp/blahblah.txt
-rw-r--r-- 1 postgres postgres 47 janv. 22 19:24 /tmp/copy_blahblah.txt
file=# SELECT id, readefile(the_file) FROM efile_test;
id | readefile
----+--------------------------------------------------------------------------------------------------
1 | \x48656c6c6f2c0a0a596f75206172652072656164696e67206120746578742066696c652e0a0a526567617264732c0a
2 | \x48656c6c6f2c0a0a596f75206172652072656164696e67206120746578742066696c652e0a0a526567617264732c0a
(2 lines)
Function reference
------------------
* **efilename(directory in name, filename in varchar(256)) returns efile**
Returns an EFILE data type that is referencing the external file on the server filesystem.
Returns NULL on null imput.
* **readEfile(e_file in efile) returns bytea**
copy the external file into a bytea.
Error will be generated if something wrong.
* **writeEfile(buffer in bytea, e_file in efile) returns void**
copy a bytea into a external file.
Error will be generated if something wrong.
* **copyEfile(src in efile, dest in efile) returns void**
duplicate file defined by src into file dest
Error will be generated if something wrong.
* **getEfilePath(e_file efile, need_read in boolean, need_write in boolean) returns text**
giving an efile and booleans, one for read and one for write need, return the
full path for the file, otherwise an error is generated
useful to check if session user has access to this external file
Authors
-------
* Dominique Legendre
* Gilles Darold
License
-------
external_file is free software distributed under the PostgreSQL Licence.
* Copyright (c) 2012-2018 Brgm - All rights reserved.
* Copyright (c) 2022 MigOps Inc - All rights reserved.
See LICENSE file.