Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/creatiwity/siren
Siren API to serve INSEE v3 data
https://github.com/creatiwity/siren
api docker-image insee rust siren siren-api sirene
Last synced: about 18 hours ago
JSON representation
Siren API to serve INSEE v3 data
- Host: GitHub
- URL: https://github.com/creatiwity/siren
- Owner: Creatiwity
- License: mit
- Created: 2019-10-11T09:36:54.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2024-05-07T12:37:37.000Z (6 months ago)
- Last Synced: 2024-05-07T13:38:17.348Z (6 months ago)
- Topics: api, docker-image, insee, rust, siren, siren-api, sirene
- Language: Rust
- Homepage: https://creatiwity.net/realisations/api-siren/23
- Size: 691 KB
- Stars: 15
- Watchers: 6
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# SIREN API
[](https://github.com/Creatiwity/siren/actions?query=workflow%3ABuild)
[](https://hub.docker.com/r/creatiwity/siren/)
[](http://microbadger.com/images/creatiwity/siren)
[](https://hub.docker.com/r/creatiwity/siren/tags/)
REST API for serving INSEE files v3.
## Getting started
To have a working copy of this project, follow the instructions.
### Installation
Setup [Rust](https://www.rust-lang.org).
Define your environment variables as defined in `.env.sample`. You can either manually define these environment variables or use a `.env` file.
Setup a postgresql database (macOS commands).
```
brew install postgresql
createuser --pwprompt sirene # set password to sirenepw for instance
createdb --owner=sirene sirene
```
## Documentation
### Configuration
Recommended configuration for production with docker:
```
RUST_LOG=sirene=warn
SIRENE_ENV=production
BASE_URL=[Your base URL, needed to update asynchronously]
API_KEY=[Any randomized string, needed to use the HTTP admin endpoint]
DATABASE_URL=postgresql://[USER]:[PASSWORD]@[PG_HOST]:[PG_PORT]/[PG_DATABASE]
DATABASE_POOL_SIZE=100
INSEE_CREDENTIALS=[Base64(consumer-key:consumer-secret)]
```
**How to generate INSEE_CREDENTIALS**
This variable is only needed if you want to have the daily updates.
1. Go to https://api.insee.fr/catalogue/
2. Create an account or sign in
3. Create an application on this portal
4. Subscribe this application to the _Sirene - V3_ API
5. Generate a key pair in the application details
6. Copy the key from the `curl` example and paste it in `.env`: `Authorization: Basic [INSEE_CREDENTIALS]`
### CLI
**> sirene --help**
```
sirene 2.0.0
Julien Blatecky
Sirene service used to update data in database and serve it through a HTTP REST API
USAGE:
sirene [OPTIONS]
FLAGS:
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
--db-folder Path to the file storage folder for the database, you can set
in environment variable as DB_FOLDER. Could be the same as
FILE_FOLDER if this app and the database are on the same file
system. Files copied by this app inside FILE_FOLDER must be
visible by the database in DB_FOLDER
--file-folder Path to the file storage folder for this app, you can set in
environment variable as FILE_FOLDER
--temp-folder Path to the temp folder, you can set in environment variable
as TEMP_FOLDER
SUBCOMMANDS:
help Prints this message or the help of the given subcommand(s)
serve Serve data from database to /unites_legales/ and /etablissements/
update Update data from CSV source files
```
**> sirene serve --help**
```
sirene-serve
Serve data from database to /unites_legales/ and /etablissements/
USAGE:
sirene serve [OPTIONS]
FLAGS:
--help Prints help information
-V, --version Prints version information
OPTIONS:
-k, --api-key API key needed to allow maintenance operation from HTTP, you can
set in environment variable as API_KEY
-b, --base-url Base URL needed to configure asynchronous polling for updates, you
can set in environment variable as BASE_URL
--env Configure log level, you can set in environment variable as
SIRENE_ENV [possible values: development, staging, production]
-h, --host Listen this host, you can set in environment variable as HOST
-p, --port Listen this port, you can set in environment variable as PORT
```
**> sirene update --help**
```
sirene-update
Update data from CSV source files
USAGE:
sirene update [FLAGS] [SUBCOMMAND]
ARGS:
Configure which part will be updated [possible values: unites-legales,
etablissements, all]
FLAGS:
--data-only Use an existing CSV file already present in FILE_FOLDER and does not delete
it
--force Force update even if the source data where not updated
-h, --help Prints help information
-V, --version Prints version information
SUBCOMMANDS:
clean-file Clean files from FILE_FOLDER
download-file Download file in TEMP_FOLDER
finish-error Set a staled update process to error, use only if the process is really
stopped
help Prints this message or the help of the given subcommand(s)
insert-data Load CSV file in database in loader-table from DB_FOLDER
swap-data Swap loader-table to production
sync-insee Synchronise daily data from INSEE since the last modification
unzip-file Unzip file from TEMP_FOLDER, and move it to the FILE_FOLDER
```
### HTTP API
```
GET /v3/unites_legales/
GET /v3/etablissements/
```
**Maintenance**
_This API is enabled only if you have provided an API_KEY when starting the `serve` process._
```
POST /admin/update
{
api_key: string,
group_type: "UnitesLegales" | "Etablissements" | "All",
force: bool,
asynchronous: bool,
}
```
If `asynchronous` is set to `true`, the update endpoint will immediately return the following:
```
Status: 202 Accepted
Location: /admin/update/status?api_key=string
Retry-After: 10
[Initial status for the started update]
```
```
GET /admin/update/status?api_key=string
```
If an update is in progress, the status code will be 202, otherwise 200.
```
POST /admin/update/status/error
{
api_key: string,
}
```
### Basic usage
Serve:
```
cargo run serve
```
Update:
```
cargo run update all
```
Help:
```
cargo run help
```
## Tests
```
cargo test
```
## Deployment
A docker image is built and a sample `docker-compose.yml` with its `docker` folder are usable to test it.
## Authors
- **Julien Blatecky** - [Julien1619](https://twitter.com/Julien1619)
## License
[MIT](LICENSE.md)