{"id":22753602,"url":"https://github.com/slaclab/lume-model","last_synced_at":"2025-04-14T15:30:58.542Z","repository":{"id":42122200,"uuid":"271648126","full_name":"slaclab/lume-model","owner":"slaclab","description":"Lume-model defines data structures used in the LUME modeling tool set.","archived":false,"fork":false,"pushed_at":"2025-03-24T20:48:12.000Z","size":4633,"stargazers_count":8,"open_issues_count":2,"forks_count":8,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-06T04:59:41.720Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://slaclab.github.io/lume-model/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slaclab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-06-11T21:16:00.000Z","updated_at":"2025-03-24T20:47:19.000Z","dependencies_parsed_at":"2025-03-18T22:22:07.527Z","dependency_job_id":"7396dfda-da39-413a-8f4c-8be7feb15101","html_url":"https://github.com/slaclab/lume-model","commit_stats":{"total_commits":300,"total_committers":3,"mean_commits":100.0,"dds":0.07999999999999996,"last_synced_commit":"fdc988748487b7abaee55af9013224de88d7c887"},"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slaclab%2Flume-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slaclab%2Flume-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slaclab%2Flume-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slaclab%2Flume-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slaclab","download_url":"https://codeload.github.com/slaclab/lume-model/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248906392,"owners_count":21181171,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-11T06:11:52.559Z","updated_at":"2025-04-14T15:30:58.535Z","avatar_url":"https://github.com/slaclab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LUME-model\n\nLUME-model holds data structures used in the LUME modeling toolset. Variables and models built using LUME-model will be compatible with other tools. LUME-model uses [pydantic](https://pydantic-docs.helpmanual.io/) models to enforce typed attributes upon instantiation.\n\n## Requirements\n\n* Python \u003e= 3.10\n* pydantic\n* numpy\n* pyyaml\n\n## Install\n\nLUME-model can be installed with conda using the command:\n\n``` $ conda install lume-model -c conda-forge ```\n\nor through pip (coming soon):\n\n``` $ pip install lume-model ```\n\n## Developer\n\nA development environment may be created using the packaged `dev-environment.yml` file.\n\n```\nconda env create -f dev-environment.yml\n```\n\nInstall as editable:\n\n```\nconda activate lume-model-dev\npip install --no-dependencies -e .\n```\n\nOr by creating a fresh environment and installing the package:\n\n```\npip install -e \".[dev]\"\n```\n\nNote that this repository uses pre-commit hooks. To install these hooks, run:\n\n```\npre-commit install\n```\n\n## Variables\n\nThe lume-model variables are intended to enforce requirements for input and output variables by variable type. For now, only scalar variables (floats) are supported.\n\nMinimal example of scalar input and output variables:\n\n```python\nfrom lume_model.variables import ScalarVariable\n\ninput_variable = ScalarVariable(\n    name=\"example_input\",\n    default_value=0.1,\n    value_range=[0.0, 1.0],\n)\noutput_variable = ScalarVariable(name=\"example_output\")\n```\n\nAll input variables may be made into constants by passing the\n`is_constant=True` keyword argument. These constant variables are always\nset to their default value and any other value assignments on\nthem will raise an error message.\n\n## Models\n\nThe lume-model base class `lume_model.base.LUMEBaseModel` is intended to guide user development while allowing for flexibility and customizability. It is used to enforce LUME tool compatible classes for the execution of trained models.\n\nRequirements for model classes:\n\n* input_variables: A list defining the input variables for the model. Variable names must be unique. Required for use with lume-epics tools.\n* output_variables: A list defining the output variables for the model. Variable names must be unique. Required for use with lume-epics tools.\n* _evaluate: The evaluate method is called by the serving model.\n  Subclasses must implement this method, accepting and returning a dictionary.\n\nExample model implementation and instantiation:\n\n```python\nfrom lume_model.base import LUMEBaseModel\nfrom lume_model.variables import ScalarVariable\n\n\nclass ExampleModel(LUMEBaseModel):\n    def _evaluate(self, input_dict):\n        output_dict = {\n            \"output1\": input_dict[self.input_variables[0].name] ** 2,\n            \"output2\": input_dict[self.input_variables[1].name] ** 2,\n        }\n        return output_dict\n\n\ninput_variables = [\n    ScalarVariable(name=\"input1\", default=0.1, value_range=[0.0, 1.0]),\n    ScalarVariable(name=\"input2\", default=0.2, value_range=[0.0, 1.0]),\n]\noutput_variables = [\n    ScalarVariable(name=\"output1\"),\n    ScalarVariable(name=\"output2\"),\n]\n\nm = ExampleModel(input_variables=input_variables, output_variables=output_variables)\n```\n\n## Configuration files\n\nModels and variables may be constructed using a YAML configuration file. The configuration file consists of three sections:\n\n* model (optional, can alternatively pass a custom model class into the `model_from_yaml` method)\n* input_variables\n* output_variables\n\nThe model section is used for the initialization of model classes. The `model_class` entry is used to specify the model class to initialize. The `model_from_yaml` method will attempt to import the specified class. Additional model-specific requirements may be provided. These requirements will be checked before model construction. Model keyword arguments may be passed via the config file or with the function kwarg `model_kwargs`. All models are assumed to accept `input_variables` and `output_variables` as keyword arguments.\n\nFor example, `m.dump(\"example_model.yml\")` writes the following to file\n\n```yaml\nmodel_class: ExampleModel\ninput_variables:\n  input1:\n    variable_class: ScalarVariable\n    default_value: 0.1\n    is_constant: false\n    value_range: [0.0, 1.0]\n  input2:\n    variable_class: ScalarVariable\n    default_value: 0.2\n    is_constant: false\n    value_range: [0.0, 1.0]\noutput_variables:\n  output1: {variable_class: ScalarVariable}\n  output2: {variable_class: ScalarVariable}\n```\n\nand can be loaded by simply passing the file to the model constructor:\n\n```python\nfrom lume_model.base import LUMEBaseModel\n\n\nclass ExampleModel(LUMEBaseModel):\n    def _evaluate(self, input_dict):\n        output_dict = {\n            \"output1\": input_dict[self.input_variables[0].name] ** 2,\n            \"output2\": input_dict[self.input_variables[1].name] ** 2,\n        }\n        return output_dict\n\n\nm = ExampleModel(\"example_model.yml\")\n```\n\n## PyTorch Toolkit\n\nA TorchModel can also be loaded from a YAML, specifying `TorchModel` in\nthe `model_class` of the configuration file.\n\n```yaml\nmodel_class: TorchModel\nmodel: model.pt\noutput_format: tensor\ndevice: cpu\nfixed_model: true\n```\n\nIn addition to the model_class, we also specify the path to the\nTorchModel's model and transformers (saved using `torch.save()`).\n\nThe `output_format` specification indicates which form the outputs\nof the model's `evaluate()` function should take, which may vary\ndepending on the application. TorchModel instances working with the\n[LUME-EPICS](https://github.com/slaclab/lume-epics) service will\nrequire an `OutputVariable` type, while [Xopt](https://github.\ncom/xopt-org/Xopt) requires either a dictionary of float\nvalues or tensors as output.\n\nThe variables and any transformers can also be added to the YAML\nconfiguration file:\n\n```yaml\nmodel_class: TorchModel\ninput_variables:\n  input1:\n    variable_class: ScalarVariable\n    default_value: 0.1\n    value_range: [0.0, 1.0]\n    is_constant: false\n  input2:\n    variable_class: ScalarVariable\n    default_value: 0.2\n    value_range: [0.0, 1.0]\n    is_constant: false\noutput_variables:\n  output:\n    variable_class: ScalarVariable\n    value_range: [-.inf, .inf]\n    is_constant: false\ninput_validation_config: null\noutput_validation_config: null\nmodel: model.pt\ninput_transformers: [input_transformers_0.pt]\noutput_transformers: [output_transformers_0.pt]\noutput_format: tensor\ndevice: cpu\nfixed_model: true\nprecision: double\n```\n\nThe TorchModel can then be loaded:\n\n```python\nfrom lume_model.torch_model import TorchModel\n\n# Load the model from a YAML file\ntorch_model = TorchModel(\"path/to/model_config.yml\")\n```\n\n\n## TorchModule Usage\n\nThe `TorchModule` wrapper around the `TorchModel` is used to provide\na consistent API with PyTorch, making it easier to integrate with\nother PyTorch-based tools and workflows.\n\n### Initialization\n\nTo initialize a `TorchModule`, you need to provide the TorchModel object\nor a YAML file containing the TorchModule model configuration.\n\n```python\n#  Wrap in TorchModule\ntorch_module = TorchModule(model=torch_model)\n\n# Or load the model configuration from a YAML file\ntorch_module = TorchModule(\"path/to/module_config.yml\")\n```\n\n### Model Configuration\n\nThe YAML configuration file should specify the `TorchModule` class\nas well as the `TorchModel` configuration:\n\n```yaml\nmodel_class: TorchModule\ninput_order: [input1, input2]\noutput_order: [output]\nmodel:\n  model_class: TorchModel\n  input_variables:\n    input1:\n      variable_class: ScalarVariable\n      default_value: 0.1\n      value_range: [0.0, 1.0]\n      is_constant: false\n    input2:\n      variable_class: ScalarVariable\n      default_value: 0.2\n      value_range: [0.0, 1.0]\n      is_constant: false\n  output_variables:\n    output:\n      variable_class: ScalarVariable\n  model: model.pt\n  output_format: tensor\n  device: cpu\n  fixed_model: true\n  precision: double\n```\n\n### Using the Model\n\nOnce the `TorchModule` is initialized, you can use it just like a\nregular PyTorch model. You can pass tensor-type inputs to the model and\nget tensor-type outputs.\n\n```python\nfrom torch import tensor\nfrom lume_model.torch_module import TorchModule\n\n\n# Example input tensor\ninput_data = tensor([[0.1, 0.2]])\n\n# Evaluate the model\noutput = torch_module(input_data)\n\n# Output will be a tensor\nprint(output)\n```\n### Saving using TorchScript\n\nThe `TorchModule` class' dump method has the option to save as a scripted JIT model by passing `save_jit=True` when calling the dump method. This will save the model as a TorchScript model, which can be loaded and used without the need for the original model file.\n\nNote that saving as JIT through scripting has only been evaluated for NN models that don't depend on BoTorch modules.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslaclab%2Flume-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslaclab%2Flume-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslaclab%2Flume-model/lists"}