{"id":39164210,"url":"https://github.com/liusemweb/hefquin-dmkg2023-experiments","last_synced_at":"2026-01-17T22:10:32.093Z","repository":{"id":207340197,"uuid":"639548284","full_name":"LiUSemWeb/HeFQUIN-DMKG2023-Experiments","owner":"LiUSemWeb","description":"Artifacts for reproducing the experimental results of our DMKG 2023 paper","archived":false,"fork":false,"pushed_at":"2024-06-24T09:08:32.000Z","size":10681,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-04T21:02:14.450Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LiUSemWeb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-05-11T17:26:26.000Z","updated_at":"2024-06-24T09:08:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"37973d21-4dc4-40aa-92ae-30b8e2a64363","html_url":"https://github.com/LiUSemWeb/HeFQUIN-DMKG2023-Experiments","commit_stats":null,"previous_names":["liusemweb/hefquin-dmkg2023-experiments"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/LiUSemWeb/HeFQUIN-DMKG2023-Experiments","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiUSemWeb%2FHeFQUIN-DMKG2023-Experiments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiUSemWeb%2FHeFQUIN-DMKG2023-Experiments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiUSemWeb%2FHeFQUIN-DMKG2023-Experiments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiUSemWeb%2FHeFQUIN-DMKG2023-Experiments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LiUSemWeb","download_url":"https://codeload.github.com/LiUSemWeb/HeFQUIN-DMKG2023-Experiments/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LiUSemWeb%2FHeFQUIN-DMKG2023-Experiments/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28519687,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T18:55:29.170Z","status":"ssl_error","status_checked_at":"2026-01-17T18:55:03.375Z","response_time":85,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-17T22:10:32.016Z","updated_at":"2026-01-17T22:10:32.085Z","avatar_url":"https://github.com/LiUSemWeb.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Overview\nThis repository contains files for conducting experiments related to the paper [**A Cost Model to Optimize Queries over Heterogeneous Federations of RDF Data Sources**](https://dmkg-workshop.github.io/papers/paper7042.pdf).\n\nWe have implemented the cost model introduced in the paper (as well as the baseline approach) in our query federation engine called [HeFQUIN](https://github.com/LiUSemWeb/HeFQUIN/).\n\nFor running the experiments we utilize [**KOBE**](https://github.com/semagrow/kobe/), a benchmarking system based on Kubernetes infrastructure, to containerize and configure federations of RDF datasets, queries, federation engines, and experiments. The typical workflow for defining a KOBE experiment includes the following steps:\n1. **DatasetTemplate**: Create one for each dataset server used in your benchmark.\n2. **Benchmark**: Define it with a list of datasets and queries.\n3. **FederatorTemplate**: Create one for the federator engine used in your experiment.\n4. **Experiment**: Define it over your previously defined benchmark.\n\n## Installation\n\n### Prerequisites\nBefore installing the KOBE benchmarking engine, ensure the following prerequisites are met:\n\n- **kubectl**: Install using native package management. Detailed instructions for Linux can be found [here](https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-using-native-package-management). The experiments in the paper use version v1.20.7. Verify the installation:\n  ```bash\n  kubectl version --client\n  ```\n- **nfs-common**: Install on the nodes of the cluster. For Debian or Ubuntu:\n  ```bash\n  apt-get install nfs-common\n  ```\n- **minikube**: Detailed installation instructions can be found [here](https://minikube.sigs.k8s.io/docs/start/?arch=%2Fmacos%2Fx86-64%2Fstable%2Fbinary+download).\n\n### Starting the Cluster\nStart your cluster using the following commands:\n```bash\nminikube start\nkubectl get po -A\nminikube dashboard\n```\nOpen the Minikube dashboard locally by accessing the default port in your browser:\n```\nhttp://127.0.0.1:44393/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/\n```\n\nIf the cluster is started on a remote server, such as `ontology.ida.liu.se`, use SSH tunneling to access the dashboard.\n\n### Install KOBE\nA detailed instruction for installing KOBE can be found [here](https://semagrow.github.io/kobe/getting_started/install/). There are some notes for the installation.\n1. When installing the Networking subsystem, you can consult the [official installation guide](https://istio.io/latest/docs/setup/getting-started/) or follow the steps below.\n- To download 'istio' with specific verion, use the follwoing curl command:\n```\n    curl -L https://istio.io/downloadIstio | ISTIO_VERSION=1.11.3 TARGET_ARCH=x86_64 sh -\n```\n- Then, add the istioctl client to your path (Linux or macOS):\n```\n    export PATH=$PWD/istio-1.11.3/bin:$PATH\n```\n- Then, install Istio using the following command:\n```\nistioctl install --set profile=demo -y\n```\n2. Before installing the logging subsystem, 'helm' need to be installed. Note the current commend only works for helm2.\nAn instruction to install and set up helm2 and tiller can be found [here](https://v2.helm.sh/docs/install/)\n\nAfter set up all required components, you can check status of all Pods via browswer, or via command:\n```kubectl get pods```\n\n### Conducting Experiments\nFirst, make sure the cluster is running\n```\nminikube status\n```\n1. Deploy dataset servers: Virtuoso, TPF, and brTPF server\n```bash\nkobectl apply dataset/dataset-virtuoso/virtuosotemplate.yaml\nkobectl apply dataset/dataset-ldfserver-hdt/ldfservertemplate-hdt.yaml\nkobectl apply dataset/dataset-brtpfserver/brtpfservertemplate.yaml\n```\nFor the TPF server and brTPF server, several docker images are used and corresponding source code for these images can be found in the corresponding directory. If any changes applied, these docker images need to be rebuilt using the following command:\n- TPF server:\n```\ncd dataset/dataset-ldfserver-hdt\ncd ldfserver-init-hdt\ndocker build --no-cache -t chengsijin817/ldfserver-init-hdt .\ndocker push chengsijin817/ldfserver-init-hdt\n    \ncd ../ldfserver-main-hdt\ndocker build --no-cache -t chengsijin817/ldfserver-main-hdt .\ndocker push chengsijin817/ldfserver-main-hdt\n```\nNote: '[chengsijin817/ldfserver-init-hdt](https://hub.docker.com/r/chengsijin817/ldfserver-init-hdt)' and '[chengsijin817/ldfserver-main-hdt](https://hub.docker.com/r/chengsijin817/ldfserver-main-hdt)' are image names, which can be renamed but should be the same as specified in ldfservertemplate-hdt.yaml file.\n\n- brTPF server:\n```\ncd dataset/dataset-brtpfserver\ncd brtpfserver-init\ndocker build --no-cache -t chengsijin817/brtpfserver-init .\ndocker push chengsijin817/brtpfserver-init\n    \ncd ../brtpfserver-main\ndocker build --no-cache -t chengsijin817/brtpfserver-main .\ndocker push chengsijin817/brtpfserver-main\n```\nSimilarly, '[chengsijin817/brtpfserver-init](https://hub.docker.com/r/chengsijin817/brtpfserver-init)' and '[chengsijin817/brtpfserver-main](https://hub.docker.com/r/chengsijin817/brtpfserver-main)' are image names, which can be renamed but should be the same as specified in brtpfservertemplate.yaml file.\n\n2. Deploy a benchmark, specifying all federation members and benchmark queries.\n```\nkobectl apply benchmark-fedbench/fedbench-het3-nodelay.yaml\nkobectl show benchmark fedbench-het3-nodelay\n```\nNote: Queries should use correct URIs in SERVICE clause, depending on the type of interface of each federation member.\nTwo example federations are provided under directory 'benchmark-fedbench'.\n\n3. Deploy HeFQUIN engine\n\nUse one of the following commands to apply a implementation of HeFQUIN engine:\n```\nkobectl apply federator-hefquin/hefquin-mincost-greedy.yaml \n```\nOr\n```\nkobectl apply federator-hefquin/hefquin-card-greedy.yaml \n```\nFor example, hefquin-mincost-greedy.yaml invokes a [docker image](https://hub.docker.com/r/chengsijin817/hefquin-mincost-greedy) of HeFQUIN engine, which implements cost-based greedy algorithm applied.\n\nTo integrate the HeFQUIN engine with the KOBE system, use the following Docker images: '[chengsijin817/hefquin-init](https://hub.docker.com/r/chengsijin817/hefquin-init)' and '[chengsijin817/hefquin-init-all](https://hub.docker.com/r/chengsijin817/hefquin-init-all)'. The source code for these images can be found in the repository directory. To rebuild these images after updates:\n```bash\ncd federator-hefquin/hefquin-init\ndocker build --no-cache -t chengsijin817/hefquin-init .\ndocker push chengsijin817/hefquin-init\n\ncd ../hefquin-init-all\ndocker build --no-cache -t chengsijin817/hefquin-init-all .\ndocker push chengsijin817/hefquin-init-all\n```\n4. Deploy experiments based on the provided configuration files, fDepending on the algorithm and federation used, apply the appropriate experiment configuration:\n```bash\nkubectl apply -f experiment/fed3nodelay-mincostgreedy-exp.yaml\n# or\nkubectl apply -f experiment/fed4nodelay-mincostgreedy-exp.yaml\n# or\nkubectl apply -f experiment/fed3nodelay-cardgreedy-exp.yaml\n# or\nkubectl apply -f experiment/fed4nodelay-cardgreedy-exp.yaml\n```\n\nAfter completion, download the log file via the Minikube dashboard.\n\n### Cleaning Up Before Next Experiment\nTo clean up all components:\n```bash\nkubectl delete experiments.kobe.semagrow.org --all\nkubectl delete benchmarks.kobe.semagrow.org --all\nkubectl delete federatortemplates.kobe.semagrow.org --all\nkubectl delete datasettemplates.kobe.semagrow.org --all\nkubectl delete pod kobenfs\n```\nAlternatively, remove specific components using the following command:\n```bash\nkubectl delete experiment het3nodelay-mincostgreedy-exp\nkubectl delete federatortemplate hefquintemplate-mincost-greedy\n```\n\n## Stopping the Cluster\nTo stop the cluster:\n```bash\nminikube status\nminikube stop\nminikube delete\n```\nNote: Previous experiment results will be lost upon restarting Minikube.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliusemweb%2Fhefquin-dmkg2023-experiments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fliusemweb%2Fhefquin-dmkg2023-experiments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fliusemweb%2Fhefquin-dmkg2023-experiments/lists"}