{"id":18376336,"url":"https://github.com/fmind/kubeflow-demo","last_synced_at":"2025-07-17T16:03:18.415Z","repository":{"id":156159080,"uuid":"624549549","full_name":"fmind/kubeflow-demo","owner":"fmind","description":"Kubeflow demo for the MLOps Community Meetup at Luxembourg.","archived":false,"fork":false,"pushed_at":"2023-04-08T13:09:45.000Z","size":327,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-11T05:54:34.384Z","etag":null,"topics":["community","kubeflow","machine-learning","mlops","python"],"latest_commit_sha":null,"homepage":"https://docs.google.com/presentation/d/1WtN5mNAHsz7PiWE5kKZUQfQ9_zO_kfVoSBNTHQM-sMk/edit?usp=sharing","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fmind.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-06T18:08:36.000Z","updated_at":"2025-02-23T09:26:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"75d64751-fabe-486e-b238-38daf831224f","html_url":"https://github.com/fmind/kubeflow-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fmind/kubeflow-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fmind%2Fkubeflow-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fmind%2Fkubeflow-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fmind%2Fkubeflow-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fmind%2Fkubeflow-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fmind","download_url":"https://codeload.github.com/fmind/kubeflow-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fmind%2Fkubeflow-demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265625567,"owners_count":23800623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["community","kubeflow","machine-learning","mlops","python"],"created_at":"2024-11-06T00:22:55.644Z","updated_at":"2025-07-17T16:03:18.365Z","avatar_url":"https://github.com/fmind.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Kubeflow Demo\n\nKubeflow demo for the MLOps Community Meetup at Luxembourg.\n\n# Requirements\n\n- [Python](https://www.python.org/)\n- [Poetry](https://python-poetry.org/)\n- [PyInvoke](https://www.pyinvoke.org/)\n- [Protobuf](https://developers.google.com/protocol-buffers/docs/overview)\n- [Kubeflow Pipelines](https://kubeflow.org/)\n\n# Installation\n\nPlease refer to [this guide on Kubeflow Pipelines v2 to install it on your computer](https://fmind.medium.com/how-to-install-kubeflow-on-apple-silicon-3565db8773f3).\n\nThen, open a terminal in the project directory and run the following commands:\n\n```bash\n# install the project environment\ninv install\n# apply Kubeflow Pipelines manifests\ninv apply\n# setup port forward for the Kubeflow UI\ninv serve\n```\n\nRemember to activate the Virtual Environment before accessing the project:\n\n```bash\n# on Linux and Mac\n./.venv/bin/activate\n```\n\n# Reproduction\n\nTo run the quickstart pipeline from the official documentation (simple example):\n\n```bash\ncd src/kdemo/pipelines/\npython quickstart.py\n```\n\n![Quickstart Pipeline](images/quickstart.png)\n\nRun a more advanced pipeline created to showcase Kubeflow Pipelines:\n\n```bash\ncd src/kdemo/pipelines/\npython wine.py\n```\n\n![Wine Pipeline](images/wine.png)\n\n# Errors\n\n## Can't install the kfp package: protobuf is missing\n\nYou need to install protobuf to communicate with Kubeflow:\n\n```\n# on MacOS\nbrew install protobuf\n```\n\n## Can't execute pipeline function without compilation\n\nThe `kfp.client.Client.create_run_from_pipeline_func` function does not work from the quickstart example:\n\n```python\n# this code does not work! see error below\nrun = client.create_run_from_pipeline_func(\n    my_pipeline,\n    arguments={\n        'a': 1,\n        'b': 2\n    },\n)\n```\n\n\u003e Error: AttributeError: 'GraphComponent' object has no attribute '__name__'\n\nInstead, you should [compile the pipeline](https://www.kubeflow.org/docs/components/pipelines/v2/compile-a-pipeline/) and call the following function to run it:\n\n```python\nrun = client.create_run_from_pipeline_package(\n    KFP_PIPELINE,\n    arguments={\n        \"a\": 1,\n        \"b\": 2,\n    },\n)\n```\n\n## The workflow-controller Pod dies sometime\n\nI noticed that the workflow-controller Pod dies sometimes (e.g., once per week).\n\nThere is nothing I could do besides reinstalling Kubeflow entirely.\n\nI haven't analyzed the root cause that triggers this issue.\n\n## Can't set the parallelism argument in dsl.ParallelFor\n\nSetting the parallelism argument causes the Kubeflow API to return an (obscure) 400 error.\n\nCode:\n```python\n# this example does not work: the parallelism argument creates an error\nwith dsl.ParallelFor(items=n_neighbors_items, parallelism=2) as n_neighbors:\n    create_model_task = create_model(n_neighbors=n_neighbors)\n```\n\nError:\n```yaml\nApiException: (400)\nReason: Bad Request\nHTTP response headers: HTTPHeaderDict({'X-Powered-By': 'Express', 'content-type': 'application/json', 'date': 'Thu, 29 Sep 2022 06:43:50 GMT', 'content-length': '440', 'connection': 'close'})\nHTTP response body: {\"error\":\"Failed to create a new run.: InvalidInputError: unknown template format: pipeline spec is invalid\",\"code\":3,\"message\":\"Failed to create a new run.: InvalidInputError: unknown template format: pipeline spec is invalid\",\"details\":[{\"@type\":\"type.googleapis.com/api.Error\",\"error_message\":\"unknown template format\",\"error_details\":\"Failed to create a new run.: InvalidInputError: unknown template format: pipeline spec is invalid\"}]}\n```\n\n## Can't write for loop in pipeline with more than one task\n\nI was not able to write a for loop to tune several models at once:\n\n```python\nwith dsl.ParallelFor(items=n_neighbors_items, name='tune-model') as n_neighbors:\n    create_model_task = create_model(n_neighbors=n_neighbors)\n    train_model_task = train_model(\n        input_model=create_model_task.outputs[\"output_model\"],\n        input_data_train=split_data_target_task.outputs[\"output_data_train\"],\n        input_target_train=split_data_target_task.outputs[\"output_target_train\"],\n    )\n    evaluate_model_task = evaluate_model(\n        input_model=train_model_task.outputs[\"output_model\"],\n        input_data_test=split_data_target_task.outputs[\"output_data_test\"],\n        input_target_test=split_data_target_task.outputs[\"output_target_test\"],\n    )\n```\n\nThe error I got is: \"component input artifact not implemented yet\".\n\nI don't have the error with only one task, but it happends everytime I include two.\n\n## Unable to retrieve node information in the UI when using ParallelFor\n\nThe UI doesn't report the status when using a `ParallelFor` control flow.\n\nThus, users can't look inside the sub DAG and find their solutions with the UI.\n\nThe only alternative is to use the command line and debug the K8s pods manually.\n\n## Optional arguments are not supported for components/pipelines\n\nWith the new version of Kubeflow Pipelines, you can use optional arguments:\n\n```python\n@kfp.dsl.pipeline(name=\"addition-pipeline\")\ndef my_pipeline(a: int, b: int, c: int = 10):\n    add_task_1 = addition_component(num1=a, num2=b)\n    add_task_2 = addition_component(num1=add_task_1.output, num2=c)\n```\n\nCompiling the pipeline will give you the following error:\n\n```\nFP driver: failed to unmarshal component spec, error: unknown field \"isOptional\" in ml_pipelines.ComponentInputsSpec.ParameterSpec\n```\n\n# Problems\n\n## Pod debugging\n\nMost debugging actions require the use of the command-line and kubectl.\n\nFor instance, you must use the following command to find an error of dependencies.\n\n```\nkubectl logs -n kubeflow wine-pipeline-vw8dp-6510369\n```\n\nAs mentioned, the Kubeflow Pipelines UI doesn't inform you about the problem:\n- The pipeline fails and display an error icon to inform the user\n- The task is still marked as running, but we can't access logs\n- The only logs available require you to use `kubectl` directly\n\n## Can't tune and select the best model\n\nI couldn't find a good solution to run several models in parallel and select the best one.\n\nA possible solution would be to create a full component dedicated to tuning and selecting the best model at the same time.\n\nBut this solution is not a good practice. Ideally, we would like to separate the tuning and the selection in different components.\n\n## Couldn't test the other component definitions\n\nAs described in [the documentation](https://www.kubeflow.org/docs/components/pipelines/v2/author-a-pipeline/components/), there are 3 methods to define a component:\n1. Lightweight Python function-based component\n2. Containerized Python components\n3. Custom container components\n\nThe 2nd and 3rd approach require the installation of a local Docker registry, or the integration with an existing one.\n\nThis step is cumbersome, and I don't see a data scientist using this approach in practice during development.\n\n## Can't use a component without the context of a pipeline\n\nIf you create a task from a component (e.g., t = create_model()), then you can't do anything besides using it inside a pipeline.\n\nMoreover, there are no documented practice to test a component. You can only do an integration test with the pipeline.\n\nThis limits greatly the flexibility of the solution in development workflow and to build robust MLOps pipelines easily.\n\n## You must import/export your data before/after each component\n\nSince components are isolated, data must be transfered between them whenever a new component is used.\n\nThis is verbose and not performant. Moreover, this can incur extra costs for storing all this extra information.\n\nThe alternative is to condense all the components into one, but this removes most of the benefit of using Kubeflow.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffmind%2Fkubeflow-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffmind%2Fkubeflow-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffmind%2Fkubeflow-demo/lists"}