https://github.com/regadas/scio-contextual
Collection of additional string interpolators to use in your scio pipeline
https://github.com/regadas/scio-contextual
gcp scala scio
Last synced: 11 months ago
JSON representation
Collection of additional string interpolators to use in your scio pipeline
- Host: GitHub
- URL: https://github.com/regadas/scio-contextual
- Owner: regadas
- License: apache-2.0
- Created: 2020-04-04T21:28:37.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2024-08-19T19:19:14.000Z (almost 2 years ago)
- Last Synced: 2025-03-26T21:37:24.624Z (about 1 year ago)
- Topics: gcp, scala, scio
- Language: Scala
- Homepage:
- Size: 95.7 KB
- Stars: 2
- Watchers: 2
- Forks: 3
- Open Issues: 12
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# scio-contextual
[](./LICENSE)
[](https://maven-badges.herokuapp.com/maven-central/io.regadas/scio-contextual_2.12)
Small library that adds some potencially useful string interpolators to use in your [scio](https://github.com/spotify/scio) pipeline when you need to have things interpreted at compile-time.
```scala
libraryDependencies ++= Seq(
"io.regadas" %% "scio-contextual" % ""
)
```
## Compatibility table
| scio-contextual | scio |
|-----------------|-------|
| 0.1.1 | 0.8.3 |
| 0.1.0 | 0.8.3 |
## Google BigQuery
```scala
import io.regadas.scio.contextual.bigquery._
```
### Valid
```scala
spec"projectid:datasetid.tableid"
// res0: com.spotify.scio.bigquery.Table.Spec = Spec(
// "projectid:datasetid.tableid"
// )
ref"""
{
"datasetId": "dataset",
"projectId": "project",
"tableId": "table"
}
"""
// res1: com.spotify.scio.bigquery.Table.Ref = Ref(
// {"datasetId":"dataset","projectId":"project","tableId":"table"}
// )
```
### Invalid
```scala
// project id needs to be at least 6 chars
spec"proj:datasetid.tableid"
// error: Table reference is not in [project_id]:[dataset_id].[table_id] format: proj:datasetid.tableid
// spec"proj:datasetid.tableid"
// ^^^^^^^^^^^^^^^^^^^^^^^
```
## Google Cloud Pub/Sub
```scala
import io.regadas.scio.contextual.pubsub._
```
### Valid
```scala
subscription"projects/project-id/subscriptions/subName"
// res3: String = "projects/project-id/subscriptions/subName"
topic"projects/project-id/topics/name"
// res4: String = "projects/project-id/topics/name"
```
### Invalid
```scala
// invalid project id
subscription"projects/proj/subscriptions/subName"
topic"projects/proj/topics/name"
// error: Illegal project name: needs to be [a-z][-a-z0-9:.]{4,61}[a-z0-9]
// subscription"projects/proj/subscriptions/subName"
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// error: Illegal project name: needs to be [a-z][-a-z0-9:.]{4,61}[a-z0-9]
// topic"projects/proj/topics/name"
// ^^^^^^^^^^^^^^^^^^^^^^^^^^
```
## Google Cloud Storage
```scala
import io.regadas.scio.contextual.gcs._
```
### Valid
```scala
gcs"gs://bucket/scio-contextual"
// res6: String = "gs://bucket/scio-contextual"
```
### Invalid
```scala
// invalid bucket
gcs"gs://bucket_/scio-contextual"
gcs"gs://bu/scio-contextual"
// wrong schema
gcs"gcs://bucket/scio-contextual"
// error: invalid uri format: gs://[a-z0-9][-_a-z0-9.]+[a-z0-9](/.*)?
// gcs"gs://bucket_/scio-contextual"
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// error: invalid uri format: gs://[a-z0-9][-_a-z0-9.]+[a-z0-9](/.*)?
// gcs"gs://bu/scio-contextual"
// ^^^^^^^^^^^^^^^^^^^^^^^^
// error: invalid uri format: gs://[a-z0-9][-_a-z0-9.]+[a-z0-9](/.*)?
// gcs"gcs://bucket/scio-contextual"
// ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```