{"id":20631226,"url":"https://github.com/solidlabresearch/google-sheet-sync","last_synced_at":"2025-07-18T04:36:13.647Z","repository":{"id":186715626,"uuid":"664992092","full_name":"SolidLabResearch/google-sheet-sync","owner":"SolidLabResearch","description":"Syncing Google Sheets and pod resources","archived":false,"fork":false,"pushed_at":"2023-09-06T11:20:18.000Z","size":348,"stargazers_count":2,"open_issues_count":3,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-15T18:47:30.792Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SolidLabResearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-11T07:47:15.000Z","updated_at":"2023-08-07T10:37:33.000Z","dependencies_parsed_at":null,"dependency_job_id":"7084940a-e534-4442-b873-981678afcdb3","html_url":"https://github.com/SolidLabResearch/google-sheet-sync","commit_stats":null,"previous_names":["solidlabresearch/google-sheet-sync"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/SolidLabResearch/google-sheet-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fgoogle-sheet-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fgoogle-sheet-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fgoogle-sheet-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fgoogle-sheet-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SolidLabResearch","download_url":"https://codeload.github.com/SolidLabResearch/google-sheet-sync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SolidLabResearch%2Fgoogle-sheet-sync/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265701124,"owners_count":23813751,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-16T14:11:36.249Z","updated_at":"2025-07-18T04:36:13.623Z","avatar_url":"https://github.com/SolidLabResearch.png","language":"JavaScript","readme":"# Google Sheet Sync\n\nGoogle Sheet Sync is an agent that allows a user to convert and synchronise data between a Google Sheet and a Solid pod.\nIt is based off of [this challenge](https://github.com/SolidLabResearch/Challenges/issues/120).\nYou find the latest screencast of the agent [here](https://cloud.ilabt.imec.be/index.php/s/SXyQ3986x3GfYyx).\nYou find a screencast of the first version [here](https://cloud.ilabt.imec.be/index.php/s/eFrEKF2YCkSx22j).\n\n## Pods\n\nTo set up the CSS instances with pod data, run\n\n```shell\nnpm run prepare:pods\n```\n\nTo start the server, run\n\n```shell\nnpm run start:pods\n```\n\n## Google Sheet API\n\nTo read and alter Google Sheets, we use the [Google Sheet API](https://developers.google.com/sheets/api/guides/concepts).\nTo access and use the Google Sheets API, one has to create/use a Google Cloud project.\n\nThese steps should be ignored if Google Sheets API OAuth2 client credentials (ID/secret pair) is supplied by an administrator.\n\n1) Navigate to the [Google Cloud console](https://console.cloud.google.com/) and create a new project.\n2) Go to \"APIs \u0026 services\" and press \"Enable APIs and services\".\n3) Search for the Google Sheets API and press \"Enable\".\n4) Before we can create any OAuth2 credentials,\n   one has to set up the [OAuth consent screen](https://developers.google.com/workspace/guides/configure-oauth-consent)\n   by pressing \"OAuth consent screen\" on the left side of the projects API's \u0026 services interface.\n5) Select external user type.\n6) Fill out the required fields. App logo and domain can be left empty. No scopes have to be specified either.\n7) If the project is not yet published and still has the \"Testing\" status, the user should add themselves or be added\n   as a test user, even if the user is the administrator/creator of the project.\n8) Navigate to the Google Sheet API's Credentials interface by first navigating to the Google Sheets API that is listed under\n   \"Enabled APIs \u0026 services\" and navigating to the \"Credentials\" tab.\n9) Press \"Create credentials\" and select \"OAuth client ID\".\n10) Select application type \"Desktop app\" and create the credentials.\n\nAfter creating or receiving OAuth2 credentials, this ID/secret pair should be pasted to the `.env` file.\nAn example on how this should be done is present in `.example.env`.\n\n## OAuth2 tokens\n\nWhen a valid OAuth2 ID and secret is supplied, one still has to create an access token and refresh token to use the API.\nTo create these, follow these steps using the authentication app:\n\n1) Make sure all dependencies have been installed by running `npm i`.\n2) Run `npm run auth` to start the authentication web app.\n3) Navigate to `http://localhost:8081/` (or another port if changed) in a browser.\n4) Press \"Authenticate\" under the \"Google\" section.\n5) Log in/select a Google account that has access to the Google Cloud project and/or is added as a test user\n   if the project is not published yet.\n6) When successful, the correct tokens have now been written to `credentials.json`.\n   You find an example in `credentials.example.json`.\n\nThe synchronisation app can now read and use these tokes to access the Google Sheet with the Google Sheets API.\n\n## Solid authentication\n\nTo set up authentication for Solid pods, you use the same authentication server as the OAuth2 setup.\n\n1) Make sure all dependencies have been installed by running `npm i`.\n2) Run `npm run auth` to start the authentication web app.\n3) Navigate to `http://localhost:8081/` (or another port if changed) in a browser.\n4) Fill in all the necessary information:\n    - host server (Url of your [Community Solid Server](https://github.com/CommunitySolidServer/CommunitySolidServer)\n    where your pod is located)\n    - email for your pod\n    - password for your pod\n  \n    The Solid instance that comes with this program is seeded with 1 pod by default. The default pod name is `example`.\nThe email is `hello@example.com` with password `abc123`. You can use these values to authenticate.\n5) Press \"Authenticate\" under the \"Solid Pod\" section.\n6) When successful, the correct tokens have now been written to `solid-credentials.json`.\n\n## Configuration\n\nThe synchronisation application is configured through the `config.yml` file.\n\n### Single resource mode\n\n#### resource (string)\n\nThis parameter allows a user to specify a resource.\nThis resource should be represented as a URI to a Solid pod from which the data will be fetched.\n\n#### host (string)\n\nThis parameter allows a user to specify the host of a resource.\nThis is required to use the websocket protocol to listen for changes on the resource.\n\nexample:\n\n```yaml\nresource: \"http://localhost:3000/testing/software\"\nhost: \"http://localhost:3000\"\n```\n\n### Multi resource mode\n\nWhen querying multiple resources at the same time, use the following structure.\n\n```yaml\nresources:\n   - resource: \"http://localhost:3000/testing/ratings\"\n     host: \"http://localhost:3000\"\n   - resource: \"http://localhost:3000/testing/tv-shows\"\n     host: \"http://localhost:3000\"\n```\n\nMake sure that for every resource, you provide a host value. Because each resource could be on a different host.\n\nYou can find an example for multiple resources in the files `config.query-multiple.example.yml` and `rules-multiple.example.yml`\n\n### query (string)\n\nThis parameter allows a user to define a SPARQL query that will be used to retrieve data from the specified data sources.\n\nexample:\n\n```yaml\nquery: \u003e\n SELECT DISTINCT * WHERE {\n     ?s \u003chttp://schema.org/name\u003e ?name .\n     OPTIONAL {?s \u003chttp://schema.org/description\u003e ?description} .\n     OPTIONAL {?s \u003chttp://schema.org/logo\u003e ?logo} .\n }\n```\n\n### Google Sheet configuration\n\nThe `sheet` section of the configuration file contains settings related to a specific sheet.\n\n#### id (string)\n\nThis parameter allows you to specify an id for the Google sheet that should be read and/or altered.\n\nTo find the id of your Google sheet, look at the URL of the Google Sheet in the address bar of your web browser.\nThe URL should look something like this:\n\n```text\nhttps://docs.google.com/spreadsheets/d/DOCUMENT_ID/edit#gid=0\n```\n\nHere, \"DOCUMENT_ID\" will be a long string of characters, letters, and numbers.\nThis is the unique identifier for the Google Sheet.\n\n#### name (string)\n\nThis parameter allows you to specify a name for the Google sheet that should be read and/or altered. \\\nThis is the name of the tab on the bottom left that you want to sync.\n\n#### interval (int)\n\nThis parameter allows you to specify the number of milliseconds between polls.\nThe code will poll the sheet for changes after the specified number of milliseconds.\nThe code will also poll the pod after this amount of milliseconds when websockets aren't used.\n\nexample:\n\n```yaml\nsheet:\n  id: \"ABCD1234\"\n  name: \"Sheet1\"\n  interval: 1000\n```\n\n### Using Fields for Data Retrieving\n\nInstead of using a single, user defined SPARQL query as in the previous method, the user can use the `fields` option\nto specify the specific fields you want to retrieve from the data source.\nThis method provides a more structured way of fetching data.\n\n#### required (list)\n\nThis parameter allows you to specify a list of fields that must/should be present in\nthe retrieved RDF data on the resource.\nEach field is represented as a key-value pair,\nwhere the key is the field name and the value is the corresponding SPARQL predicate or URI.\n\nexample:\n\n```yaml\nfields:\n required:\n   - name: \"\u003chttp://schema.org/name\u003e\"\n```\n\n#### optional (list)\n\nThis parameter allows you to specify a list of fields that are optional in the retrieved RDF data on the resource(s).\nSimilar to required, each field is represented as a key-value pair.\n\nexample:\n\n```yaml\nfields:\n optional:\n   - description: \"\u003chttp://schema.org/description\u003e\"\n   - logo: \"\u003chttp://schema.org/logo\u003e\"\n```\n\n### Debug configurations\n\n#### websockets\n\nThis parameter allows you to turn off websockets when you want explicit polling every 5 seconds.\nThe `interval` option from the Google Sheet configuration changes this value.\n\nexample:\n\n```yaml\nwebsockets: \"false\"\n```\n\n### Full examples\n\nFull configuration examples that incorporate either the query or fields method are present in\n`config.query.example.yml` and `config.fields.example.yml` respectively.\n\n## Rules (YARRRML)\n\nTo convert and write back changes from the Google Sheet back to the resource, the synchronisation agent\nuses the [RMLMapper](https://rml.io/). This mapper relies on declarative rules\nthat define how the RDF data should be generated from the data on the Google Sheet.\nWrite these rules in the form of [YARRRML](https://rml.io/yarrrml/) in the `rules.yml` file.\n\nYou are responsible that the Sheet data that is fed to the [RMLMapper](https://rml.io/)\ncontains enough information to be converted back to triples. The program itself keeps no track of\nthe origin of the separate pieces of data nor the entity to which they belong.\n\nYou find an example in `rules.example.yml`.\n\n## Start synchronisation app\n\nTo set up and use the synchronisation agent, first make sure all the necessary dependencies have been installed by running\n\n```shell\nnpm i\n```\n\nAfterward, start the agent by running\n\n```shell\nnpm start\n```\n\n## Linting\n\nESLint linter set up for this project. To run the linter, execute the following command:\n\n```shell\nnpm run lint:js\n```\n\nThere is also a markdown linter set up for this project. To run, execute the following command:\n\n```shell\nnpm run lint:markdown\n```\n\nor\n\n```shell\nnpm run lint:markdown:fix\n```\n\nto apply automatic fixes.\n\n## Technical assumptions\n\n### No 2 applications write at the same time\n\nCurrently, it is not handled when the sheet and\nanother application try to update the resource in the pod at exactly the same time.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolidlabresearch%2Fgoogle-sheet-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolidlabresearch%2Fgoogle-sheet-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolidlabresearch%2Fgoogle-sheet-sync/lists"}