Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/johnlonganecker/libpostal-rest

Last synced: 18 days ago
JSON representation

Host: GitHub
URL: https://github.com/johnlonganecker/libpostal-rest
Owner: johnlonganecker
License: mit
Created: 2016-05-20T02:06:01.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2022-06-17T21:21:08.000Z (over 2 years ago)
Last Synced: 2024-08-01T00:43:21.543Z (3 months ago)
Language: Go
Homepage:
Size: 36.1 KB
Stars: 86
Watchers: 3
Forks: 47
Open Issues: 5
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

# Libpostal REST

## Build

Go 1.13+. Build steps for older go versions may vary.

```
go build .
ls ./libpostal-rest
```

## Running

### Environment Variables

This service can be configured with the following environment variables:

`LISTEN_HOST` - hostname or IP address to listen on - default: 0.0.0.0 (all interfaces)

`LISTEN_PORT` - port the server is listening on - default: 8080

`LOG_LEVEL` - setting the global log level - default `info` (more on that further below)

`LOG_STRUCTURED` - if set to `true`, the logger generates structured JSON log output, otherwise it will create "pretty" output - default: `false`

`PROMETHEUS_ENABLED` - set to `true` enables Prometheus metrics collector and endpoint, 'false' or missing disables it - default: `false`

`PROMETHEUS_PORT` - port number where Prometheus exposes the `/metrics` endpoint - default: 9090

### Logging

Logging has been implemented using [zerolog](https://github.com/rs/zerolog) which is a super-fast structured logger outputting JSON. However, `zerolog` also allows outputting non-structured ("pretty printed") log lines, which is the default setting. `zerolog` also supports different log levels which are as follows:

* `panic`
* `fatal`
* `error`
* `warn`
* `info`
* `debug`
* `trace`

For example, to change the default log level from `info` to `debug` one would set the following environment variable:
```
export LOG_LEVEL='debug'
```

> Please note, if the string set in the environment variable is not one of the valid ones listed above, the service will log a warning and used the default level `info`.

### Metrics Generation

The service has been instrumented with the [Prometheus client library](https://github.com/prometheus/client_golang). By default metrics collection and the Prometheus endpoint is disabled. It can be enabled by setting the environment variable `PROMETHEUS_ENABLED` to `true`. Once enabled, the metrics can be scraped at the following endpoint:
```
http://:/metrics
```

The metrics endpoint exposes the default [golang metrics](https://github.com/prometheus/client_golang/blob/v1.12.2/prometheus/collectors/go_collector_latest.go#L25) as well as the following custom metrics of the service:
* `libpostal_expand_reqs_total` - The total number of processed expand requests
* `libpostal_expand_durations_historgram_ms` - Latency distributions of expand calls in milliseconds
* `libpostal_parse_reqs_total` - The total number of processed parse requests
* `libpostal_parse_durations_historgram_ms` - Latency distributions of parse calls in milliseconds
* `libpostal_expandparse_reqs_total` - The total number of processed expandparse requests
* `libpostal_expandparse_durations_historgram_ms` - Latency distributions of expandparse calls in milliseconds

## API Example

Replace `` with your host, e.g. `localhost`

### Parser
**Request**
```
curl -X POST -d '{"query": "100 main st buffalo ny"}' http://:8080/parser
```

**Response**
```
[
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main st"
},
{
"label": "city",
"value": "buffalo"
},
{
"label": "state",
"value": "ny"
}
]
```

### Expand without language options
**Request**

```
curl -X POST -d '{"query": "100 main st buffalo ny"}' http://:8080/expand
```

**Response**

```
[
"100 main saint buffalo new york",
"100 main saint buffalo ny",
"100 main street buffalo new york",
"100 main street buffalo ny"
]
```

### Expand **with** language options
**Request**

> **IMPORTANT NOTE:** if the `langs` array contains an invalid language specifier, e.g. `"xyz"`, the string will **not** be expanded for that specified language input and instead the original query string will be returned.

```
curl -X POST -d "{\"query\": \"100 main st buffalo ny\", \"langs\": [\"de\"]}" http://:8080/expand
```

**Response**

```
[
""100 main sankt buffalo ny""
]
```

Incorrect language specified

**Request**
```
curl -X POST -d "{\"query\": \"100 main st buffalo ny\", \"langs\": [\"de\", \"xyz\"]}" http://:8080/expand
```

**Response**

```
[
"100 main sankt buffalo ny",
"100 main st buffalo ny"
]
```

### Expand and Parse without language option
**Request**

```
curl -X POST -d '{"query": "100 main st buffalo ny"}' http://:8080/expandparser
```

Original query is parsed and added with `"type": "query"`.
All query expansions are parsed and added with `"type": "expansion"`

**Response**

```
[
{
"data": "100 main st buffalo ny",
"parsed": [
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main st"
},
{
"label": "city",
"value": "buffalo"
},
{
"label": "state",
"value": "ny"
}
],
"type": "query"
},
{
"data": "100 main saint buffalo ny",
"parsed": [
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main"
},
{
"label": "city",
"value": "saint buffalo"
},
{
"label": "state",
"value": "ny"
}
],
"type": "expansion"
},
{
"data": "100 main saint buffalo new york",
"parsed": [
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main"
},
{
"label": "city",
"value": "saint buffalo"
},
{
"label": "state",
"value": "new york"
}
],
"type": "expansion"
},
{
"data": "100 main street buffalo ny",
"parsed": [
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main street"
},
{
"label": "city",
"value": "buffalo"
},
{
"label": "state",
"value": "ny"
}
],
"type": "expansion"
},
{
"data": "100 main street buffalo new york",
"parsed": [
{
"label": "house_number",
"value": "100"
},
{
"label": "road",
"value": "main street"
},
{
"label": "city",
"value": "buffalo"
},
{
"label": "state",
"value": "new york"
}
],
"type": "expansion"
}
]
```

### Expand and Parse **with** language option
**Request**
```
curl -X POST -d "{\"query\": \"100 main st buffalo ny\", \"langs\": [\"fr\"]}" http://:8080/expandparser
```

**Response**