{"id":17875824,"url":"https://github.com/constraintautomaton/comunica-link-queue-parser-rs","last_synced_at":"2025-08-10T16:45:13.992Z","repository":{"id":247353514,"uuid":"825617152","full_name":"constraintAutomaton/comunica-link-queue-parser-rs","owner":"constraintAutomaton","description":null,"archived":false,"fork":false,"pushed_at":"2024-08-07T12:20:08.000Z","size":43,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-08T14:31:05.485Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/constraintAutomaton.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-08T07:31:42.000Z","updated_at":"2024-08-07T12:20:12.000Z","dependencies_parsed_at":"2024-08-07T11:07:51.592Z","dependency_job_id":"2d173c5b-24be-4635-b526-de43e377163d","html_url":"https://github.com/constraintAutomaton/comunica-link-queue-parser-rs","commit_stats":null,"previous_names":["constraintautomaton/comunica-link-queue-parser-rs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constraintAutomaton%2Fcomunica-link-queue-parser-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constraintAutomaton%2Fcomunica-link-queue-parser-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constraintAutomaton%2Fcomunica-link-queue-parser-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/constraintAutomaton%2Fcomunica-link-queue-parser-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/constraintAutomaton","download_url":"https://codeload.github.com/constraintAutomaton/comunica-link-queue-parser-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246912856,"owners_count":20853937,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-28T11:25:28.463Z","updated_at":"2025-04-03T00:23:42.144Z","avatar_url":"https://github.com/constraintAutomaton.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# comunica-link-queue-parser-rs\n\nA small CLI tool to parse the occupancy of the link queue of the [comunica link traversal query engine](https://github.com/comunica/comunica-feature-link-traversal).\n\n## Dependencies\n- [Rust toolkit](https://www.rust-lang.org/fr)\n\n## Installation\n\n```bash\ncargo build --release\n```\n\n## Usage\nThe binary is located at `./target/release/comunica-link-queue-parser-rs` and can be executed.\n\n```\nA small CLI tool to parse the occupancy of the link queue of the comunica link traversal query engine\n\nUsage: comunica-link-queue-parser-rs [OPTIONS]\n\nOptions:\n  -i, --input \u003cINPUT\u003e    Path of logging file of comunica By default the value is [default: ./info]\n  -o, --output \u003cOUTPUT\u003e  Path of the output occupancy file By default the value is [default: ./occupancy.json]\n  -p, --print            Print the occupancy object [default: false]\n  -h, --help             Print help\n  -V, --version          Print version\n```\n## Example of an execution\n\n- Compile the code (see the [installation section](#installation))\n\n- Run a query using [Comunica feature link traversal](https://github.com/comunica/comunica-feature-link-traversal) with the [Comunica Wrapper Info Occupancy RDF Resolve Hypermedia Links Queue Actor](https://github.com/comunica/comunica-feature-link-traversal/tree/master/packages/actor-rdf-resolve-hypermedia-links-queue-wrapper-info-occupancy) and a [`@comunica/logger-bunyan`](https://github.com/comunica/comunica/tree/master/packages/logger-bunyan) and pipe the log into a file.\n\n```js\nimport { LoggerBunyan, BunyanStreamProviderStdout } from '@comunica/logger-bunyan';\nimport { QueryEngineFactory } from '@comunica/query-sparql-link-traversal-solid';\n\nconst query = `\nPREFIX foat: \u003chttp://xmlns.com/foaf/0.1/\u003e\nPREFIX ruben: \u003chttps://ruben.verborgh.org/profile/#\u003e\nPREFIX rubent: \u003chttps://www.rubensworks.net/#\u003e\nSELECT DISTINCT * WHERE {\n    rubent:me foaf:knows ?person.\n    ruben:me foaf:knows ?person.\n    ?person foaf:name ?name.\n}`;\n\nconst source = 'https://www.rubensworks.net/';\nconst configPath = './config.json';\nconst engine = await new QueryEngineFactory().create({ configPath });\n\nconst streamProvider = new BunyanStreamProviderStdout({ level: 'trace' });\nconst loggerParams = {\n  name: 'comunica',\n  level: 'trace',\n  streamProviders: [ streamProvider ],\n};\nconst logger = new LoggerBunyan(loggerParams);\n\nconst bindingsStream = await engine.queryBindings(query, {\n  sources:[source],\n  lenient: true,\n  log: logger,\n});\n\nconst res = await bindingsStream.toArray();\n```\nwhere the `./config` is defined as follow.\n\n```json\n{\n    \"@context\": [\n      \"https://linkedsoftwaredependencies.org/bundles/npm/@comunica/config-query-sparql/^3.0.0/components/context.jsonld\",\n      \"https://linkedsoftwaredependencies.org/bundles/npm/@comunica/config-query-sparql-link-traversal/^0.0.0/components/context.jsonld\"\n    ],\n    \"import\": [\n      \"ccqslt:config/config-base.json\",\n      \"ccqslt:config/extract-links/actors/content-policies-conditional.json\",\n      \"ccqslt:config/extract-links/actors/quad-pattern-query.json\",\n      \"ccqslt:config/rdf-resolve-hypermedia-links/actors/traverse-replace-conditional.json\",\n      \"ccqslt:config/rdf-resolve-hypermedia-links-queue/actors/wrapper-limit-count.json\",\n      \"ccqslt:config/rdf-resolve-hypermedia-links-queue/actors/wrapper-info-occupancy.json\"\n    ]\n  }\n  \n```\n\n The resulting file should look like the example below but much longer.\n\n```json\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":30,\"headers\":{\"accept\":\"application/n-quads,application/trig;q=0.95,application/ld+json;q=0.9,application/n-triples;q=0.8,text/turtle;q=0.6,application/rdf+xml;q=0.5,text/n3;q=0.35,application/xml;q=0.3,image/svg+xml;q=0.3,text/xml;q=0.3,text/html;q=0.2,application/xhtml+xml;q=0.18,application/json;q=0.135,text/shaclc;q=0.1,text/shaclc-ext;q=0.05\",\"user-agent\":\"Comunica/actor-http-fetch (Node.js v20.13.1; linux)\"},\"method\":\"GET\",\"actor\":\"urn:comunica:default:http/actors#fetch\",\"msg\":\"Requesting https://www.rubensworks.net/\",\"time\":\"2024-08-07T08:21:44.158Z\",\"v\":0}\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":30,\"actor\":\"urn:comunica:default:query-source-identify-hypermedia/actors#none\",\"msg\":\"Identified as file source: https://www.rubensworks.net/\",\"time\":\"2024-08-07T08:21:44.408Z\",\"v\":0}\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":20,\"entries\":3,\"variables\":[[\"person\"],[\"person\"],[\"person\",\"name\"]],\"costs\":{\"inner-bind\":2423100.896815718,\"inner-multi-smallest\":59100},\"coefficients\":{\"inner-bind\":{\"iterations\":0.08968157181571816,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":24231},\"inner-multi-smallest\":{\"iterations\":0,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":591}},\"msg\":\"Determined physical join operator 'inner-multi-smallest'\",\"time\":\"2024-08-07T08:21:44.417Z\",\"v\":0}\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":20,\"entries\":2,\"variables\":[[\"person\"],[\"person\"]],\"costs\":{\"inner-hash\":40193,\"inner-symmetric-hash\":40071,\"inner-nested-loop\":39400,\"inner-bind\":1221400},\"coefficients\":{\"inner-hash\":{\"iterations\":61,\"persistedItems\":61,\"blockingItems\":61,\"requestTime\":394},\"inner-symmetric-hash\":{\"iterations\":61,\"persistedItems\":61,\"blockingItems\":0,\"requestTime\":394},\"inner-nested-loop\":{\"iterations\":0,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":394},\"inner-bind\":{\"iterations\":0,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":12214}},\"msg\":\"Determined physical join operator 'inner-nested-loop'\",\"time\":\"2024-08-07T08:21:44.418Z\",\"v\":0}\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":20,\"entries\":2,\"variables\":[[\"person\",\"name\"],[\"person\"]],\"costs\":{\"inner-hash\":20506,\"inner-symmetric-hash\":20382,\"inner-nested-loop\":19700,\"inner-bind\":19700},\"coefficients\":{\"inner-hash\":{\"iterations\":62,\"persistedItems\":62,\"blockingItems\":62,\"requestTime\":197},\"inner-symmetric-hash\":{\"iterations\":62,\"persistedItems\":62,\"blockingItems\":0,\"requestTime\":197},\"inner-nested-loop\":{\"iterations\":0,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":197},\"inner-bind\":{\"iterations\":0,\"persistedItems\":0,\"blockingItems\":0,\"requestTime\":197}},\"msg\":\"Determined physical join operator 'inner-nested-loop'\",\"time\":\"2024-08-07T08:21:44.419Z\",\"v\":0}\n{\"name\":\"comunica\",\"streamProviders\":[{\"level\":\"trace\"}],\"hostname\":\"bryanelliott-latitude5530\",\"pid\":22297,\"level\":10,\"data\":{\"type\":\"push\",\"link\":{\"url\":\"https://data.verborgh.org/people/anastasia_dimou\",\"producedByActor\":{\"name\":\"urn:comunica:default:extract-links/actors#quad-pattern-query\",\"metadata\":{\"onlyVariables\":true}},\"timestamp\":2553.299712,\"parent\":\"https://www.rubensworks.net/\"},\"query\":\"SELECT DISTINCT ?name ?person WHERE {\\n  \u003chttps://www.rubensworks.net/#me\u003e \u003chttp://xmlns.com/foaf/0.1/knows\u003e ?person.\\n  \u003chttps://ruben.verborgh.org/profile/#me\u003e \u003chttp://xmlns.com/foaf/0.1/knows\u003e ?person.\\n  ?person \u003chttp://xmlns.com/foaf/0.1/name\u003e ?name.\\n}\",\"queue\":{\"size\":1,\"push\":{\"urn:comunica:default:extract-links/actors#quad-pattern-query\":1},\"pop\":{}}},\"msg\":\"Link queue changed\",\"time\":\"2024-08-07T08:21:44.426Z\",\"v\":0}\n```\n\n- Run `comunica-link-queue-parser-rs` (see [usage section](#usage))\n\n```\n./target/release/comunica-link-queue-parser-rs -i path/to/the/comunica/log -o desired/output/path.json\n```\n\nIt should output something similar to the shorten example below.\n\n```json\n{\n    \"SELECT DISTINCT ?name ?person WHERE {   \u003chttps://www.rubensworks.net/#me\u003e \u003chttp://xmlns.com/foaf/0.1/knows\u003e ?person.   \u003chttps://ruben.verborgh.org/profile/#me\u003e \u003chttp://xmlns.com/foaf/0.1/knows\u003e ?person.   ?person \u003chttp://xmlns.com/foaf/0.1/name\u003e ?name. }\": {\n        \"push\": [\n            {\n                \"link\": {\n                    \"parent\": \"https://www.rubensworks.net/\",\n                    \"producedByActor\": {\n                        \"metadata\": {\n                            \"onlyVariables\": true\n                        },\n                        \"name\": \"urn:comunica:default:extract-links/actors#quad-pattern-query\"\n                    },\n                    \"timestamp\": 2553.299712,\n                    \"url\": \"https://data.verborgh.org/people/anastasia_dimou\"\n                },\n                \"queue\": {\n                    \"pop\": {},\n                    \"push\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 1\n                    },\n                    \"size\": 1\n                }\n            },\n            {\n                \"link\": {\n                    \"parent\": \"https://www.rubensworks.net/\",\n                    \"producedByActor\": {\n                        \"metadata\": {\n                            \"onlyVariables\": true\n                        },\n                        \"name\": \"urn:comunica:default:extract-links/actors#quad-pattern-query\"\n                    },\n                    \"timestamp\": 2553.459799,\n                    \"url\": \"https://data.verborgh.org/people/arne_gevaert\"\n                },\n                \"queue\": {\n                    \"pop\": {},\n                    \"push\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 2\n                    },\n                    \"size\": 2\n                }\n            },\n            ...\n            {\n                \"link\": {\n                    \"parent\": \"https://data.verborgh.org/ruben?predicate=http%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2Fname\u0026page=14\",\n                    \"timestamp\": 45449.248452,\n                    \"url\": \"https://data.verborgh.org/ruben?predicate=http%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2Fname\u0026page=15\"\n                },\n                \"queue\": {\n                    \"pop\": {\n                        \"unknown\": 15,\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 407\n                    },\n                    \"push\": {\n                        \"unknown\": 16,\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 407\n                    },\n                    \"size\": 1\n                }\n            }\n        ],\n        \"pop\": [\n            {\n                \"link\": {\n                    \"producedByActor\": {\n                        \"metadata\": {\n                            \"onlyVariables\": true\n                        },\n                        \"name\": \"urn:comunica:default:extract-links/actors#quad-pattern-query\"\n                    },\n                    \"timestamp\": 2558.003431,\n                    \"url\": \"https://data.verborgh.org/people/anastasia_dimou\"\n                },\n                \"queue\": {\n                    \"pop\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 1\n                    },\n                    \"push\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 60\n                    },\n                    \"size\": 59\n                }\n            },\n            {\n                \"link\": {\n                    \"producedByActor\": {\n                        \"metadata\": {\n                            \"onlyVariables\": true\n                        },\n                        \"name\": \"urn:comunica:default:extract-links/actors#quad-pattern-query\"\n                    },\n                    \"timestamp\": 2558.042332,\n                    \"url\": \"https://data.verborgh.org/people/arne_gevaert\"\n                },\n                \"queue\": {\n                    \"pop\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 2\n                    },\n                    \"push\": {\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 60\n                    },\n                    \"size\": 58\n                }\n            },\n            ...\n            {\n                \"link\": {\n                    \"timestamp\": 45458.530643,\n                    \"url\": \"https://data.verborgh.org/ruben?predicate=http%3A%2F%2Fxmlns.com%2Ffoaf%2F0.1%2Fname\u0026page=15\"\n                },\n                \"queue\": {\n                    \"pop\": {\n                        \"unknown\": 16,\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 407\n                    },\n                    \"push\": {\n                        \"unknown\": 16,\n                        \"urn:comunica:default:extract-links/actors#quad-pattern-query\": 407\n                    },\n                    \"size\": 0\n                }\n            }\n        ]\n    }\n}\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconstraintautomaton%2Fcomunica-link-queue-parser-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconstraintautomaton%2Fcomunica-link-queue-parser-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconstraintautomaton%2Fcomunica-link-queue-parser-rs/lists"}