{"id":21192162,"url":"https://github.com/mu-semtech/file-service","last_synced_at":"2025-10-11T01:39:58.165Z","repository":{"id":44806356,"uuid":"33834201","full_name":"mu-semtech/file-service","owner":"mu-semtech","description":"File manager microservice to store files and their file-specific metadata","archived":false,"fork":false,"pushed_at":"2025-01-16T10:07:46.000Z","size":93,"stargazers_count":10,"open_issues_count":4,"forks_count":12,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-09-15T06:33:56.048Z","etag":null,"topics":["file-upload","microservice","mu-service","musemtech"],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mu-semtech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2015-04-12T21:14:56.000Z","updated_at":"2025-01-16T10:07:47.000Z","dependencies_parsed_at":"2024-04-08T08:49:50.930Z","dependency_job_id":"c4535d96-a0ce-4be3-9f2f-6de687da58c7","html_url":"https://github.com/mu-semtech/file-service","commit_stats":null,"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"purl":"pkg:github/mu-semtech/file-service","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Ffile-service","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Ffile-service/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Ffile-service/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Ffile-service/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mu-semtech","download_url":"https://codeload.github.com/mu-semtech/file-service/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mu-semtech%2Ffile-service/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279005928,"owners_count":26083986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["file-upload","microservice","mu-service","musemtech"],"created_at":"2024-11-20T19:07:46.817Z","updated_at":"2025-10-11T01:39:58.121Z","avatar_url":"https://github.com/mu-semtech.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# file-service\n\nMicroservice to upload and download files and store their file-specific metadata based on [mu-ruby-template](https://github.com/mu-semtech/mu-ruby-template).\n\n## Tutorials\n\n### Add the file-service to a stack\n\nAdd the following snippet to your `docker-compose.yml` to include the file service in your project.\n\n```yaml\nfile:\n  image: semtech/mu-file-service:3.4.0\n  links:\n    - database:database\n  volumes:\n    - ./data/files:/share\n```\n\nStart the service in your stack using `docker-compose up -d file`. The file service will be created.\n\nNext, add rules to `./config/dispatcher/dispatcher.ex` to dispatch all relevant requests starting with `/files/` to the file service. E.g.\n\n```elixir\n  define_accept_types [\n    json: [ \"application/vnd.api+json\" ],\n  ]\n\n  ...\n\n  get \"/files/:id/download\", %{ layer: :services } do\n    Proxy.forward conn, [], \"http://file/files/\" \u003c\u003e id \u003c\u003e \"/download\"\n  end\n\n  post \"/files/*path\", %{ layer: :services } do\n    Proxy.forward conn, path, \"http://file/files/\"\n  end\n\n  delete \"/files/*path\", %{ accept: [ :json ], layer: :services } do\n    Proxy.forward conn, path, \"http://file/files/\"\n  end\n```\n\nThe host `file` in the forward URL reflects the name of the file service in the `docker-compose.yml` file.\n\nFinally update the authorization configuration `config/authorization/config.ex` to make sure the user has appropriate read/write access on the resource type `nfo:FileDataObject`. E.g.\n\n```elixir\n    ...\n    constraint: %ResourceConstraint{\n      resource_types: [\n        \"http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject\",\n        ...\n      ]\n    }\n    ...\n```\n\nRestart the dispatcher and authorization database to pick up the new configuration\n\n```bash\ndocker-compose restart dispatcher database\n```\n\nMore information how to setup a mu.semte.ch project can be found in [mu-project](https://github.com/mu-semtech/mu-project).\n\n## How-to guides\n\n### How to configure file resources in mu-cl-resources\n\nIf you want to model the files of the file service in the domain of your [mu-cl-resources](https://github.com/mu-semtech/mu-cl-resources) service, add the following snippet to your resource configuration.\n\nIf you use the Lisp configuration format add the following to your `domain.lisp`:\n\n```lisp\n(define-resource file ()\n  :class (s-prefix \"nfo:FileDataObject\")\n  :properties `((:name :string ,(s-prefix \"nfo:fileName\"))\n                (:format :string ,(s-prefix \"dct:format\"))\n                (:size :number ,(s-prefix \"nfo:fileSize\"))\n                (:extension :string ,(s-prefix \"dbpedia:fileExtension\"))\n                (:created :datetime ,(s-prefix \"dct:created\")))\n  :has-one `((file :via ,(s-prefix \"nie:dataSource\")\n                   :inverse t\n                   :as \"download\"))\n  :resource-base (s-url \"http://data.example.com/files/\")\n  :features `(include-uri)\n  :on-path \"files\")\n```\n\nAnd configure these prefixes in your `repository.lisp`:\n\n```lisp\n(add-prefix \"nfo\" \"http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#\")\n(add-prefix \"nie\" \"http://www.semanticdesktop.org/ontologies/2007/01/19/nie#\")\n(add-prefix \"dct\" \"http://purl.org/dc/terms/\")\n(add-prefix \"dbpedia\" \"http://dbpedia.org/ontology/\")\n```\n\nIf you use the JSON configuration format add the following to your `domain.json`:\n\n```javascript\n{\n  \"version\": \"0.1\",\n  \"prefixes\": {\n    \"nfo\": \"http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#\",\n    \"nie\": \"http://www.semanticdesktop.org/ontologies/2007/01/19/nie#\",\n    \"dct\": \"http://purl.org/dc/terms/\",\n    \"dbpedia\": \"http://dbpedia.org/resource/\"\n  },\n  \"resources\": {\n    \"files\": {\n      \"name\": \"file\",\n      \"class\": \"nfo:FileDataObject\",\n      \"attributes\": {\n        \"name\": {\n          \"type\": \"string\",\n          \"predicate\": \"nfo:fileName\"\n        },\n        \"format\": {\n          \"type\": \"string\",\n          \"predicate\": \"dct:format\"\n        },\n        \"size\": {\n          \"type\": \"integer\",\n          \"predicate\": \"nfo:fileSize\"\n        },\n        \"extension\": {\n          \"type\": \"string\",\n          \"predicate\": \"dbpedia:fileExtension\"\n        },\n        \"created\": {\n          \"type\": \"datetime\",\n          \"predicate\": \"dct:created\"\n        }\n      },\n      \"relationships\": {\n        \"download\": {\n          \"predicate\": \"nie:dataSource\",\n          \"target\": \"file\",\n          \"cardinality\": \"one\",\n          \"inverse\": true\n        },\n      },\n      \"new-resource-base\": \"http://data.example.com/files/\",\n      \"features\": [\"include-uri\"]\n    }\n  }\n}\n```\n\nNext, add the following rule to `./config/dispatcher/dispatcher.ex`. Make sure to add it somewhere after the rule forwarding `/files/:id/download`.\n\n```elixir\n  define_accept_types [\n    json: [ \"application/vnd.api+json\" ],\n  ]\n\n  ...\n\n  get \"/files/*path\", %{ accept: [ :json ], layer: :services } do\n    Proxy.forward conn, path, \"http://resource/files/\"\n  end\n```\n\nFinally, restart the services to pick up the configuration changes:\n\n```bash\ndocker-compose restart resource dispatcher\n```\n\n### How to upload a file using a curl command\n\nAssuming mu-dispatcher is running on localhost:80 a file upload can be executed using\n\n```bash\ncurl -i -X POST -H \"Content-Type: multipart/form-data\" -F \"file=@/a/file.somewhere\" http://localhost/files\n```\n\n### How to upgrade the file service from 2.x to 3.x\n\nTo upgrade the file service from 2.x to 3.x a migration must be executed in the form of a SPARQL query.\n\nIf you use [mu-migrations-service](https://github.com/mu-semtech/mu-migrations-service) add the following SPARQL query in a `*.sparql` file in your migrations folder. Else directly execute the SPARQL query against the datastore. _Note: this will break support for file service 2.x!_\n\n```\nPREFIX mu: \u003chttp://mu.semte.ch/vocabularies/core/\u003e\nPREFIX nfo: \u003chttp://www.semanticdesktop.org/ontologies/2007/03/22/nfo#\u003e\nPREFIX nie: \u003chttp://www.semanticdesktop.org/ontologies/2007/01/19/nie#\u003e\nPREFIX dct: \u003chttp://purl.org/dc/terms/\u003e\nPREFIX dbpedia: \u003chttp://dbpedia.org/ontology/\u003e\n\nWITH \u003chttp://mu.semte.ch/application\u003e\nDELETE {\n  ?uploadUri nfo:fileUrl ?fileUrl .\n} INSERT {\n  ?uploadUri dbpedia:fileExtension ?extension .\n  ?fileUri a nfo:FileDataObject ;\n    mu:uuid ?fileUuid ;\n    nfo:fileName ?fileName ;\n    dct:format ?format ;\n    nfo:fileSize ?fileSize ;\n    dbpedia:fileExtension ?extension ;\n    dct:created ?created ;\n    dct:modified ?modified ;\n    nie:dataSource ?uploadUri .\n} WHERE {\n  ?uploadUri a nfo:FileDataObject ;\n    nfo:fileName ?fileName ;\n    dct:format ?format ;\n    nfo:fileSize ?fileSize ;\n    nfo:fileUrl ?fileUrl ;\n    dct:created ?created ;\n    dct:modified ?modified .\n\n  OPTIONAL { ?fileUrl mu:uuid ?id }\n  BIND(IF(BOUND(?id), ?id, STRUUID()) as ?fileUuid)\n\n  BIND(IRI(REPLACE(STR(?fileUrl), \"file:///data/\", \"share://\")) as ?fileUri)\n\n  BIND(STRAFTER(?fileName, \".\") as ?extension)\n}\n```\n\n## Reference\n\n### Model\n\n![Data model](docs/data-model.svg)\n\n#### Ontologies and prefixes\n\nThe file service is mainly build around the [Nepomuk File Ontology](https://www.semanticdesktop.org/ontologies/2007/03/22/nfo/).\n\n| Prefix  | URI                                                       |\n| ------- | --------------------------------------------------------- |\n| nfo     | http://www.semanticdesktop.org/ontologies/2007/03/22/nfo# |\n| nie     | http://www.semanticdesktop.org/ontologies/2007/01/19/nie# |\n| dct     | http://purl.org/dc/terms/                                 |\n| dbpedia | http://dbpedia.org/ontology/                              |\n\n#### Files\n\n##### Description\n\nThe file service represents an uploaded file as 2 resources in the triplestore: a resource reflecting the (virtual) uploaded file and another resource reflecting the (physical) resulting file stored on disk.\n\nThe URI of the stored file uses the `share://` protocol and reflects the location where the file resides as a relative path to the share folder. E.g. `share://uploads/my-file.pdf` means the file is stored at `/share/uploads/my-file.pdf`.\n\n##### Class\n\n`nfo:FileDataObject`\n\n##### Properties\n\n| Name       | Predicate               | Range                | Definition                                                        |\n| ---------- | ----------------------- | -------------------- | ----------------------------------------------------------------- |\n| name       | `nfo:fileName`          | `xsd:string`         | Name of the uploaded file                                         |\n| format     | `dct:format`            | `xsd:string`         | MIME-type of the file                                             |\n| size       | `nfo:fileSize`          | `xsd:integer`        | Size of the file in bytes                                         |\n| extension  | `dbpedia:fileExtension` | `xsd:string`         | Extension of the file                                             |\n| created    | `dct:created`           | `xsd:dateTime`       | Upload datetime                                                   |\n| dataSource | `nie:dataSource`        | `nfo:FileDataObject` | Uploaded file this file originates from (only set on stored file) |\n\n### Configuration\n\n#### Environment variables\n\nThe following settings can be configured via environment variables:\n\n| ENV                              | Description                                                                                                           | default                                         | required |\n|----------------------------------|-----------------------------------------------------------------------------------------------------------------------|-------------------------------------------------|----------|\n| FILE_RESOURCE_BASE               | Base URI for a new upload-file resource. Must end with a trailing `/`. It will be concatenated with a uuid            | http://mu.semte.ch/services/file-service/files/ |          |\n| MU_APPLICATION_FILE_STORAGE_PATH | Mounted subfolder where you want to store your files. It must be a relative path to `/share/` in the Docker container | None                                            |          |\n| VALIDATE_READABLE_METADATA       | Whether metadata of files must be readable on upload                                                                  | false                                           |          |\n| MU_SPARQL_ENDPOINT               | SPARQL read endpoint URL                                                                                              | http://database:8890/sparql                     |          |\n| MU_SPARQL_TIMEOUT                | Timeout (in seconds) for SPARQL queries                                                                               | 60                                              |          |\n| LOG_LEVEL                        | The level of logging. Options: debug, info, warn, error, fatal                                                        | info                                            |          |\n\n#### File storage location\n\nBy default the file service stores the files in the root of the mounted volume `/share/`. You can configure the service to store the files in a mounted subfolder through the `MU_APPLICATION_FILE_STORAGE_PATH` environment variable. It must be a relative path to `/share/` in the Docker container.\n\nE.g.\n\n```yaml\nfile:\n  image: semtech/mu-file-service:3.4.0\n  links:\n    - database:database\n  environment:\n    MU_APPLICATION_FILE_STORAGE_PATH: \"my-project/uploads/\"\n  volumes:\n    - ./data/my-project/uploads:/share/my-project/uploads\n```\n\nThe subfolder will be taken into account when generating the file URI. A URI for a file stored using the file service configured above will look like `share://my-project/uploads/example.pdf`.\n\n#### Database connection\n\nThe triple store used in the backend is linked to the file service container as `database`. If you configure another SPARQL endpoint URL through `MU_SPARQL_ENDPOINT` update the link name accordingly. Make sure the file service is able to execute update queries against this store.\n\n### REST API\n\n#### POST /files\n\nUpload a file. Accepts a `multipart/form-data` with a `file` parameter containing the uploaded file.\n\n##### Response\n\n###### 201 Created\n\nOn successful upload with the newly created file in the response body:\n\n```javascript\n{\n  \"links\": {\n    \"self\": \"files/b178ba66-206e-4551-b41e-4a46983912c0\"\n  },\n  \"data\": {\n    \"type\": \"files\",\n    \"id\": \"b178ba66-206e-4551-b41e-4a46983912c0\",\n    \"attributes\": {\n        \"name\": \"upload-name.pdf\",\n        \"format\": \"application/pdf\",\n        \"size\": 1930\n        \"extension\": \"pdf\"\n    }\n  }\n}\n```\n\n###### 400 Bad Request\n\n- if file param is missing.\n\n#### GET /files/:id\n\nGet metadata of the file with the given id.\n\n##### Response\n\n###### 200 OK\n\nReturns the metadata of the file with the given id.\n\n```javascript\n{\n  \"links\": {\n    \"self\": \"files/b178ba66-206e-4551-b41e-4a46983912c0\"\n  },\n  \"data\": {\n    \"type\": \"files\",\n    \"id\": \"b178ba66-206e-4551-b41e-4a46983912c0\",\n    \"attributes\": {\n        \"name\": \"upload-name.pdf\",\n        \"format\": \"application/pdf\",\n        \"size\": 1930\n        \"extension\": \"pdf\"\n    }\n  }\n}\n```\n\n##### 404 Bad Request\n\nIf a file with the given id cannot be found.\n\n#### GET /files/:id/download\n\nDownload the content of the file with the given id.\n\n##### Query paramaters\n\n- `name` (optional): name for the downloaded file (e.g. `/files/1/download?name=report.pdf`)\n- `content-disposition` (optional): specify with which [Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition) header-value you want the service to respond. Defaults to `attachment`.\n\n##### Response\n\n###### 200 Ok\n\nExpected response, the file is returned.\n\n###### 404 Bad Request\n\nNo file could be found with the given id.\n\n###### 500 Server error\n\nA file with the given id could be found in the database but not on disk. This is most likely due to configuration issue on the server.\n\n#### DELETE /files/:id\n\nDelete the file (metadata and content) with the given id.\n\n##### Response\n\n###### 204 No Content\n\nOn successful delete.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmu-semtech%2Ffile-service","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmu-semtech%2Ffile-service","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmu-semtech%2Ffile-service/lists"}