{"id":19329770,"url":"https://github.com/outerbounds/hamilton-metaflow","last_synced_at":"2025-04-22T21:32:02.072Z","repository":{"id":79997021,"uuid":"476583436","full_name":"outerbounds/hamilton-metaflow","owner":"outerbounds","description":"using hamilton + metaflow together","archived":false,"fork":false,"pushed_at":"2022-08-03T20:31:13.000Z","size":524,"stargazers_count":7,"open_issues_count":2,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-02T04:23:30.626Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/outerbounds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-01T05:20:38.000Z","updated_at":"2024-11-22T20:18:12.000Z","dependencies_parsed_at":"2023-04-25T16:31:09.945Z","dependency_job_id":null,"html_url":"https://github.com/outerbounds/hamilton-metaflow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fhamilton-metaflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fhamilton-metaflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fhamilton-metaflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/outerbounds%2Fhamilton-metaflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/outerbounds","download_url":"https://codeload.github.com/outerbounds/hamilton-metaflow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250328596,"owners_count":21412648,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T02:29:52.673Z","updated_at":"2025-04-22T21:32:02.066Z","avatar_url":"https://github.com/outerbounds.png","language":"Python","funding_links":[],"categories":["Third-Party Integrations"],"sub_categories":[],"readme":"# Installation\n```\ngit clone https://github.com/outerbounds/hamilton-metaflow.git\ncd ./absenteeism\n```\n\nThis flow depends on Metaflow's integration with Anaconda. You can [install conda](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html#installation) like: \n```\nbash Miniconda3-latest-MacOSX-x86_64.sh\n```\n\n## Note about visualizing Hamilton flow\nIn the `flow.py` step called `featurize_and_split` the code is configured to visualize the hamilton DAG.\nBy default, we assume you have system graphviz installed. \nIf you do not have graphviz or don't want to use it you can set the `graphviz_flag` parameter defined in `flow.py` to `false`. \nIf you want to visualize the hamilton DAGs in Metaflow Cards here is a link to [install graphviz at system level](https://graphviz.org/download/). \n* on MacOS: `brew install graphviz`\n* on Unix: `sudo apt-get install graphviz`\n* More alternatives are discussed [here](https://stackoverflow.com/questions/35064304/runtimeerror-make-sure-the-graphviz-executables-are-on-your-systems-path-aft).\n\n# Run the flow\nRun the flow:\n```\npython ./flow.py --environment=conda run\n```\n\nNote that this will create a single conda environment to run each step of the flow in. The first time you do this it may take several minutes to fetch all dependencies from conda, but each time after the environment will bootstrap much faster so long as the conda dependencies are unchanged.\n\nAfter configuring your AWS credentials you can run any steps on AWS Batch using [Metaflow decorators](https://docs.metaflow.org/v/r/metaflow/scaling).\n\nThis is one benefit of using Metaflow's `@conda` or `@conda_base decorators`. Metaflow will automatically package the conda environment and start the compute environment on remote enviroments from it. \n\n# Inspecting Results\nThis flow creates several Metaflow [cards](https://docs.metaflow.org/metaflow/visualizing-results/effortless-task-inspection-with-default-cards). Cards are associated with flow steps. These can be viewed locally in the browser running \n```\npython ./flow.py --environment=conda card view \u003cstep name\u003e\n```\n\nFor example, the `start` step displays class label distribution plots:\n```\npython ./flow.py --environment=conda card view start\n```\n\nYou can see Hamilton visualizations by using\n```\npython ./flow.py --environment=conda card view featurize_and_split\npython ./flow.py --environment=conda card view feature_importance_merge\n```\n\nIn a script or notebook you can also look at the scores of each modeling step in a pandas DataFrame like: \n``` python \nfrom metaflow import Flow\nrun = Flow('FeatureSelectionAndClassification').latest_run\nrun.data.results\n```\n\n# Reference\n* Dataset source: [Application of a neuro fuzzy network in prediction of absenteeism at work](https://ieeexplore.ieee.org/document/6263151)\n* Learning task source: [Prediction Of Absenteeism At Work With Multinomial Logistic Regression Model](https://www.researchgate.net/publication/358900589_PREDICTION_OF_ABSENTEEISM_AT_WORK_WITH_MULTINOMIAL_LOGISTIC_REGRESSION_MODEL)\n* [Identification of Important Features and Data Mining Classification Techniques in Predicting Employee  Absenteeism at Work](http://ijece.iaescore.com/index.php/IJECE/article/view/25232)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouterbounds%2Fhamilton-metaflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fouterbounds%2Fhamilton-metaflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fouterbounds%2Fhamilton-metaflow/lists"}