https://github.com/ngageoint/seed-silo
Rest API for discovering Seed images
https://github.com/ngageoint/seed-silo
Last synced: 9 months ago
JSON representation
Rest API for discovering Seed images
- Host: GitHub
- URL: https://github.com/ngageoint/seed-silo
- Owner: ngageoint
- License: apache-2.0
- Created: 2017-10-31T16:10:47.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2022-02-07T15:32:26.000Z (almost 4 years ago)
- Last Synced: 2025-03-27T16:48:58.470Z (10 months ago)
- Language: Go
- Homepage:
- Size: 4.27 MB
- Stars: 5
- Watchers: 18
- Forks: 1
- Open Issues: 13
-
Metadata Files:
- Readme: readme.adoc
- License: LICENSE
Awesome Lists containing this project
README
= SILO API
image:https://badges.gitter.im/ngageoint/seed.svg[link="https://gitter.im/ngageoint/seed?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge"]
image:https://travis-ci.org/ngageoint/seed-silo.svg?branch=master[link="https://travis-ci.org/ngageoint/seed-silo"]
SILO (Seed Images Location Operation) is a REST API provided by the Seed team for discovering Seed images. The API
allows users to scan one or more repositories for seed images and then search the resulting images and their manifests
by keywords. Frameworks such as Scale can use these endpoints to assist users in finding algorithm images and creating
jobs out of them.
== Install
To build/install the SILO webapp on linux:
----
Install go, docker and httpd packages
yum install go
yum install docker
yum install httpd
go get github.com/ngageoint/seed-silo
~/go/src/github.com/ngageoint/seed-silo/install-silo.sh
----
Or even better, run SILO from a https://hub.docker.com/r/geoint/seed-silo[seed-silo container]:
----
docker run -p 9000:9000 docker.io/geoint/seed-silo:1.2.1
----
SILO runs on port 9000 so that port needs to be opened to the docker host. It could be changed to port 80 if desired
with the flag '-p 80:9000'. The command above is all that is needed, but if customization is desired, the following
environment variables can be passed in:
.Environment Variables
[cols="1,4"]
|===
|Name |Description
|SILO_ADMIN
|Specifies the username of the default admin. If not specified, this defaults to admin.
|SILO_ADMIN_PASSWORD
|Specifies the password of the default admin. If not specified, the default can be found in the codebase.
|DATABASE_URL
|Specifies the connection parameters for an existing database to use. This database should have a silo schema.
If such a schema does not exist, it will be created. The environment variable follows the syntax outlined at
https://github.com/kennethreitz/dj-database-url#url-schema[dj-database-url project]. In short, it follows the form:
postgis://user:password@host[:port]/name
|SILO_LITE_PATH
|Specifies the path to use for a SQLite database if a postgres url is not specified via DATABASE_URL. If
neither it nor DATABASE_URL is set the path /usr/silo/seed-silo.db is used. A database will be created inside the container
if needed or a database can be mounted into the container at the path for persistence between runs.
|===
== Usage
=== Registry
Registries can be added, deleted and scanned. A registry consists of a name, url, organization (optional), username (optional),
and password (optional).
==== Get Registry
Retrieves a registry
[cols="h,5a"]
|===
| URL
| /registries/{id}
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: {"ID":1,"Name":"dockerhub","Url":"https://hub.docker.com","Org":"geointseed","Username":"","Password":""}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No registry found with that ID" }
|Sample Call
| curl -X "GET" http://localhost:9000/registries/1
|===
==== Add Registry
Adds a registry to the list of registries to be scanned. An error will be returned and the registry won't be added if
the daemon is unable to connect to the registry.
[cols="h,5a"]
|===
| URL
| /registries/add
| Method
| POST
| URL Params
| None
| Data Params
| {"name":"localhost", "url":"https://localhost:5000", "org":"", "username":"testuser", "password": "testpassword"}
| Success Response
| Code: 201 +
Content: {}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Unable to connect to registry" } +
Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" }
Code: 422 Unprocessable Entity +
Content: { error : "Error unmarshalling json. " }
|Sample Call
| curl -H "Authorization: Token " -H "Content-Type: application/json" -d '{"name":"localhost", "url":"https://localhost:5000", "org":"", "username":"testuser", "password": "testpassword"}' http://localhost:9000/registries/add
|===
==== Delete Registry
Removes a registry from the list of registries along with all images associated with that registry.
[cols="h,5a"]
|===
| URL
| /registries/delete/{id}
| Method
| DELETE
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: { }
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" }
|Sample Call
| curl -H "Authorization: Token " -X "DELETE" http://localhost:9000/registries/delete/1
|===
==== Scan Registries
Removes all existing image entries, scans all registries for seed images and adds them to the database.
Requires admin authorization token
[cols="h,5a"]
|===
| URL
| /registries/scan
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 202 +
Content: { } +
Code: 202 +
Content: {"message":"Scanning Registries"}
|Error Response
| Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" }
|Sample Call
| curl -H "Authorization: Token " "https://localhost:9000/registries/scan"
|===
==== Scan Registry
Removes all existing image entries, scans all registries for seed images and adds them to the database.
Requires admin authorization token
[cols="h,5a"]
|===
| URL
| /registries/{id}/scan
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 202 +
Content: { } +
Code: 202 +
Content: {"message":"Scanning Registries"}
|Error Response
| Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" }
|Sample Call
| curl -H "Authorization: Token " "https://localhost:9000/registries/1/scan"
|===
==== List Registries
Retrieves all of the registries that have been successfully added
[cols="h,5a"]
|===
| URL
| /registries
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"Name": "localhost", +
"Url": "https://localhost:5000", +
"Org": "", +
"Username": "", +
"Password": "" +
} +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/registries"
|===
=== Image
Images are added/removed by scanning registries. An image consists of a name, registry, organization (optional), and the
Seed manifest.
==== List Images
Retrieves all of the Seed images that have been scanned from registries
[cols="h,5a"]
|===
| URL
| /images
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"RegistryId": 1, +
"Name": "my-job-0.1.0-seed:0.1.0", +
"Registry": "docker.io", +
"Org": "geointseed", +
"JobName": "my-job", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"JobVersion": "0.1.0", +
"PackageVersion": "0.1.0" +
}, +
{ +
"ID": 2, +
"RegistryId": 1, +
"Name": "extractor-0.1.0-seed:0.1.0", +
"Registry": "docker.io", +
"Org": "geointseed", +
"JobName": "extractor", +
"Title": "Extractor", +
"Maintainer": "John Tobe", +
"Email": "jtobe@example.com", +
"MaintOrg": "", +
"Description": "Read's a zip file and extracts the contents", +
"JobVersion": "0.1.0", +
"PackageVersion": "0.1.0" +
}, +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/images"
|===
==== Search Images
Searches the Seed images that have been scanned from registries and returns images matching the given query. Images are
returned if the name, organization or manifest strings match the given query.
[cols="h,5a"]
|===
| URL
| /images/search/{query}
| Method
| GET
| URL Params
| query = string
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"RegistryId": 1, +
"Name": "my-job-0.1.0-seed:0.1.0", +
"Registry": "docker.io", +
"Org": "geointseed", +
"JobName": "my-job", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"JobVersion": "0.1.0", +
"PackageVersion": "0.1.0" +
}, +
{ +
"ID": 2, +
"RegistryId": 1, +
"Name": "extractor-0.1.0-seed:0.1.0", +
"Registry": "docker.io", +
"Org": "geointseed", +
"JobName": "extractor", +
"Title": "Extractor", +
"Maintainer": "John Tobe", +
"Email": "jtobe@example.com", +
"MaintOrg": "", +
"Description": "Read's a zip file and extracts the contents", +
"JobVersion": "0.1.0", +
"PackageVersion": "0.1.0" +
}, +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/images/search/test"
|===
==== Get Image
Retrieves an image
[cols="h,5a"]
|===
| URL
| /images/{id}
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: +
{ +
"ID": 1, +
"RegistryId": 1, +
"JobId": 1, +
"JobVersionId": 1, +
"FullName": "my-job-0.1.0-seed:0.1.0", +
"ShortName": "my-job", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"JobVersion": "0.1.0", +
"PackageVersion": "0.1.0", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"Registry": "docker.io", +
"Org": "geointseed", +
"Manifest": "{\"seedVersion\":\"0.1.0\",\"job\":{\"name\":\"my-job\",...}}" +
link:seed.manifest.json[sample manifest] +
}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No image found with that ID" }
|Sample Call
| curl -X "GET" http://localhost:9000/images/1
|===
==== Image Manifest
Returns the Seed manifest json for the given image id.
[cols="h,5a"]
|===
| URL
| /images/{id}/manifest
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: link:seed.manifest.json[sample manifest]
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No image found with that ID" }
|Sample Call
| curl "https://localhost:9000/images/1/manifest"
|===
=== Job
Jobs are groups of images with the same job name. A job has a name, title, maintainer, email, organization, description,
latest job version and latest package version. It also has a list of images and job versions.
==== List Jobs
Retrieves all of the Jobs that have been scanned from registries
[cols="h,5a"]
|===
| URL
| /jobs
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"Name": "my-job", +
"LatestJobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"ImageIDs": [0, 1, 2], +
"JobVersions": [{JobVersion struct}, {JobVersion struct}...] +
}, +
{ +
"ID": 2, +
"Name": "another-job", +
"LatestJobVersion": "1.0.0", +
"LatestPackageVersion": "1.0.0", +
"Title": "My second job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "blah blah blah", +
"ImageIDs": [3], +
"JobVersions": [{JobVersion struct}] +
}, +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/jobs"
|===
==== Search Jobs
Searches the Seed images that have been scanned from registries and returns jobs for the images matching the given query. Images are
returned if the name, organization or seed manifest match the given query. Images/job versions that are irrelevant to the query
are omitted from the ImageIDs and JobVersions structures.
[cols="h,5a"]
|===
| URL
| /jobs/search/{query}
| Method
| GET
| URL Params
| query = string
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"Name": "my-job", +
"LatestJobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"ImageIDs": [0, 2], +
"JobVersions": [{JobVersion struct}, {JobVersion struct}...] +
}, +
{ +
"ID": 2, +
"Name": "another-job", +
"LatestJobVersion": "1.0.0", +
"LatestPackageVersion": "1.0.0", +
"Title": "My second job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "blah blah blah", +
"ImageIDs": [3], +
"JobVersions": [{JobVersion struct}] +
}, +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/jobs/search/test"
|===
==== Get Job
Retrieves a job
[cols="h,5a"]
|===
| URL
| /jobs/{id}
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: +
{ +
"ID": 1, +
"Name": "my-job", +
"LatestJobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Title": "My first job", +
"Maintainer": "John Doe", +
"Email": "jdoe@example.com", +
"MaintOrg": "E-corp", +
"Description": "Reads an HDF5 file and outputs two TIFF images, a CSV and manifest containing cell_count", +
"ImageIDs": [0, 2], +
"JobVersions": [{JobVersion struct}, {JobVersion struct}...] +
}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No job found with that ID" }
|Sample Call
| curl -X "GET" http://localhost:9000/jobs/1
|===
=== Job Version
Job Versions are groups of images with the same job name and the same job version. A job version has a job name, job id,
job version, latest package version and a list of images.
==== List Job Versions
Retrieves all of the Job Versions that have been scanned from registries
[cols="h,5a"]
|===
| URL
| /job-versions
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"JobName": "my-job", +
"JobId": "1", +
"JobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Images": [{Image struct}, {Image struct}...] +
}, +
{ +
"ID": 2, +
"JobName": "my-job", +
"JobId": "1", +
"JobVersion": "0.2.0", +
"LatestPackageVersion": "0.2.0", +
"Images": [{Image struct}] +
}, +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/job-versions"
|===
==== Get Job Versions
Returns the job versions for a specific job
[cols="h,5a"]
|===
| URL
| /jobs/{id}/job-versions
| Method
| GET
| URL Params
| id = int
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"JobName": "my-job", +
"JobId": "1", +
"JobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Images": [{Image struct}, {Image struct}...] +
}, +
{ +
"ID": 2, +
"JobName": "my-job", +
"JobId": "1", +
"JobVersion": "0.2.0", +
"LatestPackageVersion": "0.2.0", +
"Images": [{Image struct}] +
}, +
]
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No job found with that ID" }
|Sample Call
| curl "https://localhost:9000/jobs/1/job-versions"
|===
==== Get Job Version
Retrieves a job version
[cols="h,5a"]
|===
| URL
| /job-versions/{id}
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: +
{ +
"ID": 1, +
"JobName": "my-job", +
"JobId": "1", +
"JobVersion": "0.1.0", +
"LatestPackageVersion": "0.1.0", +
"Images": [{Image struct}, {Image struct}...] +
}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No job version found with that ID" }
|Sample Call
| curl -X "GET" http://localhost:9000/jobs/1
|===
=== User
Users can be added, deleted, listed and used to login. A user consists of a username, password, and a role.
==== Get User
Retrieves a user
[cols="h,5a"]
|===
| URL
| /users/{id}
| Method
| GET
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: {"ID":1,"username":"admin","role":"admin"}
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 404 File not found +
Content: { error : "No user found with that ID" }
|Sample Call
| curl -X "GET" http://localhost:9000/user/1
|===
==== Add User
Adds a user to the system. Requires a valid token from an admin user.
[cols="h,5a"]
|===
| URL
| /users/add
| Method
| POST
| URL Params
| None
| Data Params
| {"username":"admin", "password": "hunter17", "role": "admin"}
| Success Response
| Code: 201 +
Content: {"username":"admin", "password": "hunter17", "role": "admin"}
|Error Response
| Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" } +
Code: 422 Unprocessable Entity +
Content: { error : "Error unmarshalling json. " }
|Sample Call
|curl -H "Content-Type: application/json" -d '{"username":"admin", "password": "hunter17", "role": "admin"}' -H "Authorization: Token " http://localhost:9000/users/add
|===
==== Delete User
Removes a user from the system. Requires a valid token from an admin user.
[cols="h,5a"]
|===
| URL
| /users/delete/{id}
| Method
| DELETE
| URL Params
| id = integer
| Data Params
| None
| Success Response
| Code: 200 +
Content: { }
|Error Response
| Code: 400 Bad Request +
Content: { error : "Invalid ID" } +
Code: 401 Unauthorized +
Content: { error : "Invalid authorization token" } or { error : "Missing authorization token" } +
Code: 403 Forbidden +
Content: { error : "User does not have permission to perform this action" }
|Sample Call
| curl -X "DELETE" -H "Authorization: Token " http://localhost:9000/users/delete/1
|===
==== List Users
Retrieves all of the users in the system
[cols="h,5a"]
|===
| URL
| /users
| Method
| GET
| URL Params
| None
| Data Params
| None
| Success Response
| Code: 200 +
Content: [ +
{ +
"ID": 1, +
"username": "admin", +
"role": "admin" +
}, +
{ +
"ID": 2, +
"username": "user", +
"role": "user" +
} +
]
|Error Response
| None
|Sample Call
| curl "https://localhost:9000/users"
|===
==== Login
Authenticates a user and returns a token to be used in subsequent api calls
[cols="h,5a"]
|===
| URL
| /login
| Method
| GET
| URL Params
| None
| Data Params
| {"username":"admin", "password": "password"}
| Success Response
| Code: 200 +
Content: {"token":""}
|Error Response
| Code: 401 Unauthorized +
Content: { error : "Invalid login" } +
Code: 422 Unprocessable Entity +
Content: { error : "Error unmarshalling json. " }
|Sample Call
| curl -H "Content-Type: application/json" -d '{"username":"admin", "password": "password"}' "http://localhost:9000/login"
|===