Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/statisticsnorway/pseudo-service
(De/)pseudonymization endpoints
https://github.com/statisticsnorway/pseudo-service
azure-pipeline dapla dlp pseudo statistikktjenester
Last synced: 6 days ago
JSON representation
(De/)pseudonymization endpoints
- Host: GitHub
- URL: https://github.com/statisticsnorway/pseudo-service
- Owner: statisticsnorway
- License: mit
- Created: 2020-06-17T11:47:21.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-12-10T07:25:29.000Z (13 days ago)
- Last Synced: 2024-12-10T07:41:03.068Z (13 days ago)
- Topics: azure-pipeline, dapla, dlp, pseudo, statistikktjenester
- Language: Java
- Homepage:
- Size: 31 MB
- Stars: 1
- Watchers: 14
- Forks: 0
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
# Dapla Pseudo Service
(De/)pseudonymization and export endpoints.
Browse the API docs as:
* [Redoc](https://dapla-pseudo-service.staging-bip-app.ssb.no/api-docs/redoc) (recommended)
* [Rapidoc](https://dapla-pseudo-service.staging-bip-app.ssb.no/api-docs/rapidoc)
* [Swagger UI](https://dapla-pseudo-service.staging-bip-app.ssb.no/api-docs/swagger-ui)
... or parse [the Open API specs](https://dapla-pseudo-service.staging-bip-app.ssb.no/api-docs/dapla-pseudo-service-1.0.yml) for yourself 🤓
## Examples
### Pseudonymize JSON file and stream back the result
```sh
curl "${root_url}/pseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k.json' \
--form 'request={
"targetContentType": "application/json",
"pseudoConfig": {
"rules": [
{
"name": "allthenumbers",
"pattern": "**/*nummer",
"func": "fpe-anychar(secret1)"
}
]
}
}'
```
### Depseudonymize JSON file and stream back the result as CSV
```sh
curl "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k-pseudonymized.json' \
--form 'request={
"targetContentType": "text/csv",
"pseudoConfig": {
"rules": [
{
"name": "allthenumbers",
"pattern": "**/*nummer",
"func": "fpe-anychar(secret1)"
}
]
}
}'
```
### Depseudonymize JSON file and download a zipped CSV-file
```sh
curl "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/15k-pseudonymized.json' \
--form 'request={
"targetContentType": "text/csv",
"pseudoConfig": {
"rules": [
{
"name": "allthenumbers",
"pattern": "**/*nummer",
"func": "fpe-anychar(secret1)"
}
]
},
"compression": {
"password": "kensentme"
}
}'
```
### Depseudonymize archive with multiple JSON files and download a zipped CSV-file
```sh
curl --output depseudonymized.zip "${root_url}/depseudonymize/file" \
--header "Authorization: Bearer ${dapla_auth_token}" \
--form 'data=@src/test/resources/data/multiple-json-files.zip' \
--form 'request={
"targetContentType": "text/csv",
"pseudoConfig": {
"rules": [
{
"name": "id",
"pattern": "**/*identifikator*",
"func": "fpe-fnr(secret1)"
}
]
},
"compression": {
"password": "kensentme"
}
}'
```
## A note regarding encrypted archives
Standard zip encryption is weak. Thus, for enhanced security, all compressed archives are password encrypted
using AES256. You might need to use a non-standard unzip utility in order to decompress these files. A good
alternative is 7zip.
To unzip using 7zip:
```sh
7z x
```
## Pseudo rules
Pseudo rules are defined by:
* _name_ (used only for logging purposes)
* _pattern_ - [glob pattern](https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob) that matches fields
to be (de)/pseudonymized.
* _func_ - references a pseudo function (such as `fpe-anychar`, `fpe-fnr` or `fpe-digits`). The function references the
pseudo secret to be used.
## Development
See `Makefile` for details/examples of common dev tasks.
```
build-all Build all and create docker image
build-mvn Build project and install to you local maven repo
build-docker Build dev docker image
init-local-config Creates configuration files in the local directory
run-local Run the app locally (without docker)
release-dryrun Simulate a release in order to detect any issues
release Release a new version. Update POMs and tag the new version in git
```