Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/yahoojapan/presto-audit
THIS REPOSITORY IS DEPRECATED
https://github.com/yahoojapan/presto-audit
eventlistener plugin presto presto-plugin
Last synced: about 2 months ago
JSON representation
THIS REPOSITORY IS DEPRECATED
- Host: GitHub
- URL: https://github.com/yahoojapan/presto-audit
- Owner: yahoojapan
- Archived: true
- Created: 2017-09-13T23:30:26.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2023-07-06T01:38:06.000Z (about 1 year ago)
- Last Synced: 2024-07-20T23:44:23.236Z (2 months ago)
- Topics: eventlistener, plugin, presto, presto-plugin
- Language: Java
- Homepage:
- Size: 195 KB
- Stars: 19
- Watchers: 8
- Forks: 14
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ⛔️ THIS REPOSITORY IS DEPRECATED ⛔️
This repository is no longer maintained. Please refer to the following developer guides:
* https://prestodb.io/docs/current/develop/event-listener.html
* https://trino.io/docs/current/develop/event-listener.html
# Presto Query Audit Log(log history)
Plugin for Presto to save queries and metrics into files.
[](https://travis-ci.org/yahoojapan/presto-audit)
## Requirement
* Presto 0.193 or later
# Build
```bash
% ./mvnw clean compile package
```
## packaging - rpm
```bash
% ./mvnw clean compile package rpm:rpm
```
# Usage
## install
Copy jar file to target directory.
```bash
% export PLUGIN_VER=1.0
% cp target/presto-audit-plugin-${PLUGIN_VER}-SNAPSHOT-all.jar /usr/lib/presto/lib/plugin/yj-audit/
```
## run
create a event-listener.properties file under /etc/presto/ .
eg.
/etc/presto/event-listener.properties
```text
event-listener.name=presto-audit-log
event-listener.audit-log-path=/var/log/presto/
event-listener.audit-log-filename=presto-auditlog.log
```
## full log
To enable full log, add the following line to the file
```text
event-listener.audit-log-full-filename=presto-auditlog-full.log
```
To add a filter for full log, add the following line to the file.
The filter should be a regex. If the filter match with `--source` of a query, this query will not be
logged in full log.
```text
# Format:
event-listener.audit-log-full-filter=
# Example:
event-listener.audit-log-full-filter=SRE_SYSTEM
event-listener.audit-log-full-filter=(SRE_SYSTEM|Presto-team)
```
## Send message to Pulsar broker
Send the log to Pulsar broker immediately after a query is completed.
Config:
```text
event-listener.pulsar.simple-log-topic=persistent://namespace/global/test/topic1
event-listener.pulsar.full-log-topic=persistent://namespace/global/test/topic2
event-listener.pulsar.pulsar-url=pulsar+ssl://pulsar.cluster.com:6651
event-listener.pulsar.pulsar-cert-path=/etc/pki/tls/certs/ca-bundle.crt
event-listener.pulsar.use-TLS=true # Optional
event-listener.pulsar.compression-type=ZLIB # Optional (default: NONE)
event-listener.athenz.private-key-path=/etc/presto/athenz/private.key
event-listener.athenz.tenant-domain=tenant.pulsar.tenant
event-listener.athenz.tenant-service=mq
event-listener.athenz.provider-domain=provider.plusar.tenant
event-listener.athenz.config-path=/usr/local/etc/pulsar-athenz-config/athenz.conf
event-listener.athenz.principal-header=Athenz-Principal-Auth
event-listener.athenz.role-header=Athenz-Role-Auth
```
## Analyze SQL samples
Table DDL can be found in src/sql/ddl.sql
```sql
SELECT
date_parse(executionStartTime, '%Y%m%d%H%i%S.%f') AS EXEC_START,
state,
date_diff('millisecond', date_parse(executionStartTime, '%Y%m%d%H%i%S.%f'), date_parse(endTime, '%Y%m%d%H%i%S.%f')) AS PRESTO_EXEC_TIME_MS
FROM
presto_audit
WHERE
ymd = '20170912' AND createTime > '20170912233000.572'
ORDER BY
createTime;
```