{"id":48822254,"url":"https://github.com/intersystems-ib/workshop-smart-data-fabric","last_synced_at":"2026-04-14T15:34:47.666Z","repository":{"id":236667805,"uuid":"590502467","full_name":"intersystems-ib/workshop-smart-data-fabric","owner":"intersystems-ib","description":"Learn the main ideas involved in developing a Smart Data Fabric using InterSystems IRIS","archived":false,"fork":false,"pushed_at":"2024-06-06T07:26:29.000Z","size":30209,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-06-06T08:46:48.359Z","etag":null,"topics":["analytics","data","datafabric","interoperability","smart"],"latest_commit_sha":null,"homepage":"","language":"ObjectScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/intersystems-ib.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-18T15:07:34.000Z","updated_at":"2024-06-06T07:26:34.000Z","dependencies_parsed_at":"2024-04-28T06:08:32.467Z","dependency_job_id":"07d25be8-0cc7-41a5-a22b-19723745ac83","html_url":"https://github.com/intersystems-ib/workshop-smart-data-fabric","commit_stats":{"total_commits":43,"total_committers":1,"mean_commits":43.0,"dds":0.0,"last_synced_commit":"8916dcb44b167a773d6cae23cbcf442375c7bc32"},"previous_names":["intersystems-ib/workshop-smart-data-fabric"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/intersystems-ib/workshop-smart-data-fabric","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intersystems-ib%2Fworkshop-smart-data-fabric","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intersystems-ib%2Fworkshop-smart-data-fabric/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intersystems-ib%2Fworkshop-smart-data-fabric/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intersystems-ib%2Fworkshop-smart-data-fabric/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/intersystems-ib","download_url":"https://codeload.github.com/intersystems-ib/workshop-smart-data-fabric/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intersystems-ib%2Fworkshop-smart-data-fabric/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31803628,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-14T11:13:53.975Z","status":"ssl_error","status_checked_at":"2026-04-14T11:13:53.299Z","response_time":153,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["analytics","data","datafabric","interoperability","smart"],"created_at":"2026-04-14T15:34:47.591Z","updated_at":"2026-04-14T15:34:47.656Z","avatar_url":"https://github.com/intersystems-ib.png","language":"ObjectScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"Learn the foundamentals to implement an Smart Data Fabric architect using InterSystems IRIS.\n\n\u003cimg src=\"img/sdf.png\" width=\"1500\"/\u003e\n\n# Learn the basics\n👉 We will combine different features of InterSystems IRIS such as multi-model database, interoperability and analytics.\n\nIf you are not too familiar with InterSystems IRIS technology, you can get a hands-on overview with these resources:\n* Learn about InterSystems IRIS and **Docker containers** - [workshop-containers-intro](https://github.com/intersystems-ib/workshop-containers-intro)\n* Learn how to use InterSystems IRIS and Visual Studio Code - [workshop-vscode-iris](https://github.com/intersystems-ib/workshop-vscode-iris)\n* A very simple introduction to InterSystems IRIS **multi-model database** -  [workshop-multimodel-intro](https://github.com/intersystems-ib/workshop-multimodel-intro)\n* An overview of **interoperability** using IRIS - [workshop-interop-intro](https://github.com/intersystems-ib/workshop-interop-intro)\n* Analytics \u0026 BI in a nutshell with IRIS - [workshop-iris-bi-intro](https://github.com/intersystems-ib/workshop-iris-bi-intro)\n\n# What do you need to install? \n* [Git](https://git-scm.com/downloads) \n* [Docker](https://www.docker.com/products/docker-desktop) (if you are using Windows, make sure you set your Docker installation to use \"Linux containers\").\n* [Docker Compose](https://docs.docker.com/compose/install/)\n* [Visual Studio Code](https://code.visualstudio.com/download) + [InterSystems ObjectScript VSCode Extension](https://marketplace.visualstudio.com/items?itemName=daimor.vscode-objectscript)\n\n# Setup\nClone the GitHub repository to your local computer. This will allow you to acces the code and build the samples:\n```\ngit clone https://github.com/intersystems-ib/workshop-smart-data-fabric\n``` \n\nBuild the image we will use during the workshop:\n```\ndocker-compose build\n```\n\nRun the container:\n```\ndocker-compose up -d\n```\n\nYou should be able to access [InterSystems IRIS Management Portal](http://localhost:52773/csp/sys/UtilHome.csp) and login using `superuser` / `SYS`.\n\n# Environment\nWe are going to use an environment using Docker containers. \n\n\u003cimg src=\"img/docker-environment.png\" width=\"800\" /\u003e\n\n* [docker-compose](docker-compose.yml) - set up the containers (services) we are using.\n* [Dockerfile](Dockerfile) - this file defines how we are building our InterSystems IRIS Container. We will start from an InterSystems IRIS For Health Community version, copy some directories, set up some permissions and finally call `iris.script` to run whatever we need within IRIS.\n* [iris.script](iris.script) - script that runs the setup we need in IRIS, e.g. installing applications, loading source code, etc.\n\nAfter running the environment, you can access to an interactive sesion on IRIS container using:\n```\ndocker-compose exec -it iris bash\n```\n\nYou can also have a look at the container logs using:\n```\ndocker logs iris\n```\n\n# Data model\nHave a look at the main classes of our data model:\n* [Patient](src/sdf/data/Patient.cls) will store patient definitions\n* [Observation](src/sdf/data/Observation.cls) will store different kind of observations for the patients (e.g. diastolic bp, systolic bp, body temperature, etc.)\n\nOur classes are [persistent](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GOBJ_persobj_intro). That means that they can store data, and in InterSystems IRIS we will be able to work with them using objects as well as SQL automatically.\n\nThese classes also use [Relationships](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GOBJ_relationships) and some [Indexes](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GOBJ_relationships). They can also contain methods and logic.\n\nGo to [System Explorer \u003e SQL (SDF)](http://localhost:52773/csp/sys/%25CSP.Portal.Home.zen?$NAMESPACE=SDF\u0026$NAMESPACE=SDF\u0026#), locate the tables corresponding to our persistent classes and display them. They should be empty.\n\n\u003cimg src=\"img/sql-explorer-empty.gif\" width=\"1024\"/\u003e\n\nWe manipulate data using SQL or Objects. Let's create some simple data using objects through the [WebTerminal](http://localhost:52773/terminal/)\n\nFirst, create a patient object:\n\n```objectscript\n    set patientObj = ##class(sdf.data.Patient).%New()\n    set patientObj.Identifier = \"12345\"\n    set patientObj.FirstName = \"John\"\n    set patientObj.LastName = \"Doe\"\n```\n\nThen, create an observation for the patient:\n\n```objectscript\n    set obxObj = ##class(sdf.data.Observation).%New()\n    set obxObj.Code = \"BodyTemp\"\n    set obxObj.ValueNM = \"36\"\n    set obxObj.Units = \"C\"\n```\n\nFinally, insert the observation into the patient record and save it\n\n```objectscript\n    do patientObj.Observations.Insert(obxObj)\n    set sc = patientObj.%Save(1)\n    write !,\"statusCode=\",sc\n```\n\nAfter that, try to run SQL queries again. You can also take advantage of [Implicit Joins (Arrow Syntax)](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GSQL_implicitjoins):\n\n```sql\nSELECT \nPatient-\u003eFirstName, Patient-\u003eLastName,Code, Units, ValueNM\nFROM sdf_data.Observation\n```\n\nAfter your tests, delete all the data you have just created:\n\n```objectscript\n    write ##class(sdf.data.Observation).%KillExtent()\n    write ##class(sdf.data.Patient).%KillExtent()\n``` \n\n# Ingestion\n\n## Data\n* In our example, we are going to use a set of HL7 files that have been generated inspired on [Maternal Health Risk Data](https://www.kaggle.com/datasets/csafrit2/maternal-health-risk-data) dataset on Kaggle. These files will be ingested as they were incoming from an external hospital.\n* Uncompress [data/hl7files.tar.gz](data/hl7files.tar.gz).\n\n```\ncd data\ntar -zxvf hl7files.tar.gz\n```\n\n## DataPipe and DataPipeUI\n* You could implement data ingestion in a lot of different ways. In this example, we will be using a community tool called [DataPipe](https://github.com/intersystems-ib/iris-datapipe) that is already installed in the environment. \n* This will help us on enriching, normalizing and validating the incoming data using InterSystems IRIS interoperability features. \n* Also you can use an user interface [DataPipeUI](https://github.com/intersystems-ib/iris-datapipeUI) to have a look at the incoming data to the system and how it's being handled.\n* In a separate terminal in your system, clone the repo and run the [DataPipeUI](https://github.com/intersystems-ib/iris-datapipeUI) container user interface:\n\n```\ngit clone https://github.com/intersystems-ib/iris-datapipeUI\ncd iris-datapipeUI\ndocker-compose up -d\n```\n\n## Try ingesting some data yourself\n* In the environment, open the [production](http://localhost:52773/csp/sdf/EnsPortal.ProductionConfig.zen?PRODUCTION=sdf.connectors.interop.Production). It is already running.\n* Let's ingest some data.\n* Copy some files from [data/hl7files](data/hl7files) into [data/hl7in](data/hl7in).\n* You can have a look at the [HL7 messages processed](http://localhost:52773/csp/sdf/EnsPortal.MessageViewer.zen?SOURCEORTARGET=HL7%20In) in the production.\n* Access http://localhost:8080 to have a glance at the DataPipeUI\n* After processing data, run some SQL queries again.\n\nYou can also copy all files using:\n```\ncd data\ncp hl7files/*.hl7 hl7in\n```\n\n\u003cimg src=\"img/hl7-ingestion-datapipe.gif\" width=\"1024\" /\u003e\n\n## DataPipe Model\nDataPipe allows you to define an interoperability model with the properties that you need, and then decide how are you going to normalize and validate it. You have to implement a few methods.\n\n\u003cimg src=\"img/datapipe-abstract-model.png\" width=\"200\" /\u003e\n\nIn this case, we are using [R01Model.cls](src/sdf/connectors/interop/datapipe/model/R01Model.cls):\n* It defines the properties we need for processing incoming ORU^R01 HL7 messages with observations.\n* Implements `Serialize` and `Deserialize` methods to serialize and deserialize using JSON format.\n* To `Normalize`, it calls [R01Normalize](http://localhost:52773/csp/sdf/EnsPortal.DTLEditor.zen?DT=sdf.connectors.interop.datapipe.dt.R01Normalize.dtl) data transformation.\n* To `Validate`, implements some checks on the incoming data.\n* Finally, in `RunOperation` implements what are we going to do with the ingested data. In this example it is storing data in `sdf.data.*` classes.\n\n## DataPipe Production\n* The [production](http://localhost:52773/csp/sdf/EnsPortal.ProductionConfig.zen?PRODUCTION=sdf.connectors.interop.Production) that is ingesting data, have some elements you should review:\n* `HL7 In` - built-in HL7 file Business Service that reads HL7 files from a directory.\n* [HL7 Ingestion](http://localhost:52773/csp/sdf/EnsPortal.BPLEditor.zen?BP=sdf.connectors.interop.datapipe.bp.HL7Ingestion.bpl) - Business Process that: \n  * Extract attributes (metadata) from incoming HL7 message using [R01ToInboxAttributes](http://localhost:52773/csp/sdf/EnsPortal.DTLEditor.zen?DT=sdf.connectors.interop.datapipe.dt.R01ToInboxAttributes.dtl) data transform.\n  * Converts incoming HL7 message into [sdf.connectors.interop.datapipe.model.R01Model.cls](src/sdf/connectors/interop/datapipe/model/R01Model.cls) using [R01ToModel](http://localhost:52773/csp/sdf/EnsPortal.DTLEditor.zen?DT=sdf.connectors.interop.datapipe.dt.R01ToModel.dtl) data transform.\n* `HL7 Staging` is a DataPipe business process (`DataPipe.Staging.BP.StagingManager`) that handles the normalization and validation of your DataPipe model.\n* `HL7 Oper` is another DataPipe business process (`DataPipe.Oper.BP.OperManager`) that handles running your DataPipe model operation.\n\n# Services\n\nLet's create a REST service to interact with your `sdf.data.*` classes. But first, we can start by working with JSON.\n\n## JSON\n\n### %JSON.Adaptor\nYour `sdf.data.*` classes already extends from [%JSON.Adaptor](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GJSON_adaptor). It provides some nice features for importing and exporting your objects to and from JSON.\n\nOpen a [WebTerminal](http://localhost:52773/terminal/) session and try the following:\n\nOpen an object and export to JSON:\n\n```objectscript\n    // open an object from a persistent class\n    set patient = ##class(sdf.data.Patient).%OpenId(1)\n    // directly, export to json to current device\n    do patient.%JSONExport()\n```\n\nNow, let's try to format the JSON for our object:\n\n```objectscript\n    // export patient object to a json string\n    do patient.%JSONExportToString(.json)\n    // instantiate a json formatter\n    set formatter = ##class(%JSON.Formatter).%New()\n    do formatter.FormatToString(json, .formattedJson)\n    // print formatted json\n    write formattedJson\n```\n\nIn your [sdf.data.Patient](src/sdf/data/Patient.cls) class, change the `%JSONREFERENCE` attribute from `ID` to `OBJECT` or viceversa and try again the following:\n\n```objectscript\n    // delete previous in-memory object definition\n    kill patient\n    // re-open object (so it can load your change on %JSONREFERENCE)\n    set patient = ##class(sdf.data.Patient).%OpenId(1)\n    // export to a formatted json string\n    do patient.%JSONExportToString(.json)\n    do formatter.FormatToString(json, .formattedJson)\n    // print your json string\n    write formattedJson\n```\n\nCan you tell the difference between using `ID` or `OBJECT`?\n\n⚠️ **Important!** Before going on, be sure your [sdf.data.Patient](src/sdf/data/Patient.cls) class has `(%JSONREFERENCE = \"ID\")` defined.\n\n[%JSON.Adaptor](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GJSON_adaptor) has a lot of nice features that allows you to export and import and customize those behaviours. We'll use them in the REST service we will implement.\n\n`%JSON.Adaptor` is a nice approach if you have already defined classes that you want to serialize or deserialize to JSON format.\n\n### %DynamicObjects\n\n[%DynamicObjects](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=GJSON_create) allows you to work with JSON structures without having a previous definition (dynamically).\n\nIn your [WebTerminal](http://localhost:52773/terminal/) session try the following:\n\n```objectscript\n    set dynamicObject = {\"prop1\":\"a string value\"}\n    write dynamicObject.prop1\n\n    set dynamicArray = [[1,2,3],{\"A\":33,\"a\":\"lower case\"},1.23456789012345678901234,true,false,null,0,1,\"\"]\n    write dynamicArray.%ToJSON()\n```\n\nHave a look at the documentation section [Using JSON in ObjectScript](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=GJSON_intro) to have an overview of all the options you have available with `%JSON.Adaptor` or `%DynamicObjects`.\n\n\n## REST Service\n\nThere are different ways of implement REST services in InterSystems IRIS. We will implement a `%CSP.REST` service. Don't forget to check the documentation section [Introduction to Creating REST Services](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GREST_intro) to have a full view.\n\nOpen [sdf.connectors.api.DataEndpoint](src/sdf/connectors/api/DataEndpoint.cls). This will be our service for accesing some of our `sdf.data.*` classes.\n\nReview the different methods that are implemeted and try to figure out what are they doing.\n\nREST services needs a web application that forwards HTTP requests to them, in this case we have the [/sdf/api](http://localhost:52773/csp/sys/sec/%25CSP.UI.Portal.Applications.Web.zen?PID=%2Fsdf%2Fapi) web application.\n\nAlso, in [iris.script](iris.script) you will find how this web application is imported during the container image build for the environment.\n\nFinally, try your service using **Postman**. Import the [workshop-smart-data-fabric.postman_collection.json](workshop-smart-data-fabric.postman_collection.json) included in the repository and try the different requests:\n\n\u003cimg src=\"img/rest-postman.gif\" witdth=\"1024\"/\u003e\n\n## Embedded Python\nEmbedded Python allows you to use Python to program InterSystems IRIS applications. You can even mix ObjectScript methods and Python methods and refer to objects created in either language! And of course you could use any Python libraries on your implementation. Check the documentation section [Using Embedded Python](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=AEPYTHON) to have a full view on this topic.\n\nWe will use some Python libraries we must install. First, you need to connect to container bash:\n```bash\ndocker compose exec iris bash\n```\n\nThen install the libraries we will use:\n```bash\npip3 install --target /usr/irissys/mgr/python/ pandas openpyxl\n```\n\nWe are going to use Embedded Python to implement in our REST service an operation that will return an Excel file with the observations for an specific patient. We will take advantage of [openpyxl](https://openpyxl.readthedocs.io/en/stable/) Python library to create Excel files:\n* REST service will handle `/patient/:id/observations/xls` requests to return an Excel file.\n* [sdf.connectors.api.DataEndpoint](src/sdf/connectors/api/DataEndpoint.cls):`GetPatientObservationsExcel` method will run a SQL query, create a result set and convert it to Excel.\n* Actual resultset to Excel conversion will be run in [sdf.Utils](src/sdf/Utils.cls):`ResultSetToXls` which is an Embedded Python classmethod that takes advantage of openpyxl](https://openpyxl.readthedocs.io/en/stable/) library.\n\nYou can test it accessing to http://localhost:52773/sdf/api/patient/2/observations/xls in your browser.\n\n# Interoperability\n\nWe are going to create a Telegram bot and use it to send some notifications about our Smart Data Fabric.\n\n## Telegram bot setup\n* Create a Telegram bot using [BotFather](https://t.me/botfather) bot.\n```\n/newbot\n```\nWrite down your Telegram Bot token.\n\n* Open a new Telegram chat with your brand new Telegram bot. Write some dummy messages.\n* Usually, you will process incoming messages to your bot using a WebHook or the getUpdates Telegram API. In this example, we will only focus on sending messages.\n* You will need a **chat id** and your **token** to send messages.\n* Grab your **chat id** by accesing [https://api.telegram.org/bot\u003cyour_token\u003e/getUpdates](https://api.telegram.org/bot\u003cyour_token\u003e/getUpdates). You should get a JSON response from Telegram. Look for something like:\n```\n...\n\"chat\":{\"id\":\u003cyour_chatId\u003e\n...\n```\n\n## Business operation settings\n* Go to [IRIS \u003e Interoperability \u003e Configure \u003e Credentials](http://localhost:52773/csp/sdf/EnsPortal.Credentials.zen?$NAMESPACE=SDF\u0026$NAMESPACE=SDF\u0026) and create a `TelegramBotToken` Credentials with your token as password.\n* Go to our [Interoperability Production](http://localhost:52773/csp/sdf/EnsPortal.ProductionConfig.zen?PRODUCTION=sdf.connectors.interop.Production\u0026$NAMESPACE=SDF), select `TelegramSendMessage` operation and set the following:\n  * TelegramCredentials: `TelegramBotToken`\n  * DefaultTelegramChatId: `your chatId`\n\nYou can now test your Telegram Business Operation:\n\n\u003cimg src=\"img/telegram-bo-test.gif\" width=\"1500\" /\u003e\n\n## Calling your Business operation\n\n### Ingestion\nYou can call your Telegram business operation from the ingestion layer using your Data Pipe model, try to add the following in [R01Model](src/sdf/connectors/interop/datapipe/model/R01Model.cls):\n\n```objectscript\n    $$$AddLog(log, \"Transaction Commited\")\n\n    // you can send messages to other production components (while you are not on an open transaction)\n    if $isobject(bOperation) {\n        set req = ##class(sdf.connectors.interop.msg.TelegramMsgReq).%New()\n        set req.text = \"Patient (\"_..PatientId_\") ingested! 🌡️ \"_$number(..ObxValues.GetAt(\"BodyTemp\"),2)_\" \"_..ObxUnits.GetAt(\"BodyTemp\")\n        $$$ThrowOnError(bOperation.SendRequestAsync(\"TelegramSendMessage\", req))\n    }\n```\n\n### Services\nYou can also call interoperability components from your REST service context.\n\nLet's call the Telegram business operation from the REST service:\n* Service will handle requests to `/summary` to send a summary of the sdf.\n* This will be implemented in [DataEndpoint](sdf.connectors.api.DataEndpoint):`GetSummary`.\n* In the method, we will call interoperability components. For that you need to start your call instatiating a Business Service that will init the interoperability context.\n* We will instantiate [TelegramFromService](sdf.connectors.interop.bs.TelegramFromService) business service. It will simply send a message to our Telegram business operation.\n\n# Enabling FHIR using FHIR Façade Architecture\n\nHL7 FHIR (Fast Healthcare Interoperability Resources) has become the top standard for the exchange of patient data across healthcare systems.\n\nHowever, not all applications can be completely re-written to exchange data using the FHIR standard, and facilities may not be able to deploy a *full FHIR repository*.\n\nYou can use InterSystems IRIS For Health to to create an architecture that acts as a façade for a FHIR repository, allowing you to avoid complete rework while reaping the benefits of using FHIR data in your existing applications.\n\nYou can find more information in [FHIR Façade Architecture Overview](https://learning.intersystems.com/course/view.php?id=2137).\n\nLet's say that you want to implement an architecture that enables FHIR for our classes in `sdf.data.*` package. \n\nYou will now implement a FHIR Façade.\n\n## Create a FHIR Server\nYou will now create a FHIR Server and use your own InteractionsStrategy that implements a FHIR Façade on top of your `sdf.data.*` package:\n\nCreate a FHIR server in *Health \u003e SDF \u003e FHIR Configuration \u003e Server Configuration \u003e Add Endpoint*\n* Core FHIR package: `hl7.fhir.r4.core@4.0.1`\n* URL: `/csp/healthshare/sdf/fhir/r4`\n* Interactions Strategy Class: `sdf.fhirserver.InteractionsStrategy`\n\nNow, edit the FHIR endpoint you have just created:\n* Enable New Service Instance. This is useful in case you want to change InteractionsStrategy class during development and test the new behaviour immediately.\n\n## FHIR Façade implementation\nIn this case these are the main involved classes:\n* [sdf.fhirserver.InteractionsStrategy](src/sdf/fhirserver/InteractionsStrategy.cls), [sdf.fhirserver.Interactions](src/sdf/fhirserver/Interactions.cls) and [sdf.fhirserver.RepoManager](src/sdf/fhirserver/RepoManager.cls) - these are the main classes you should implement when writing you FHIR Server Interactions strategy. You can find more information in [Customizing a FHIR Server](https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI.Page.cls?KEY=HXFHIR_server_customize_arch).\n* [sdf.fhirserver.FHIRFacade](src/sdf/fhirserver/FHIRFacade.cls) - common class we have implemented for this example. It includes some methods that must be implemented in façade classes such as how to export your data as FHIR resource or how to perform searchs.\n* [sdf.data.Patient](src/sdf/data/Patient.cls) and [sdf.data.Observation](src/sdf/data/Observation.cls) - your data classes will now implement *FHIRFaçade* methods.\n\n## Try it out\nUsing the included [Postman collection](workshop-smart-data-fabric.postman_collection.json), try some requests that are already prepared:\n* `metadata`: retrieve your FHIR Façade Capability Statement\n* `Get Patient`: retrieve a particular patient as a FHIR resource\n* `Get Patients. Female. Paginated`: search female patients and retrieve a paginated bundle.\n\n## Adding OAuth to your FHIR Server\n* You will use the webserver container included in the workshop, as it provides you a webserver (Apache) + WebGateway connection to IRIS using HTTPS. This is required for OAuth2.\n* You can test it by accesing the IRIS Management Portal using this URL: https://webserver/iris/csp/sys/UtilHome.csp\n* To use FHIR and OAuth2 you will use SMART On FHIR Scopes, you can find more information [here](https://hl7.org/fhir/smart-app-launch/1.0.0/scopes-and-launch-context/index.html).\n\nYou can also find more information about IRIS and OAuth2 in [workshop-iris-oauth2](https://openexchange.intersystems.com/package/workshop-iris-oauth2).\n\n### Create OAuth2 Server\nIRIS will act as your OAuth Authorization server. You can create a it in *System Administration \u003e Security \u003e OAuth 2.0 \u003e Server*\n\nOr you can simply type this command that will create the OAuth Authorization server for you:\n```\nzn \"SDF\"\ndo ##class(sdf.Utils).CreateOAuth2Server()\n```\n\n### Create OAuth2 Resource Server\nNext, create the OAuth2 Resource server that will be used in your FHIR Server.\n\nGo to *System Administration \u003e Security \u003e OAuth 2.0 \u003e Client \u003e Create Server Description*\n* Issuer endpoint: `https://webserver/iris/oauth2`\n* SSL/TLS configuration: `ssl`\n* Discover and Save\n\nGo to *System Administration \u003e Security \u003e OAuth 2.0 \u003e Client \u003e Client Configurations \u003e Create Client Configuration*:\n* Application name: `fhirserver-resserver`\n* Client name: `fhirserver oauth resource server`\n* Client type: `Resource server`\n* SSL/TLS configuration: `ssl`\n* Dynamic Registration and Save\n\n### Create OAuth2 Client configuration\nNext, add a new client (with a ClientId and a Secret) that will be used while testing your FHIR Server + OAuth from Postman.\n\n* Go to *System Administration \u003e Security \u003e OAuth 2.0 \u003e Server \u003e Client Descriptions*\n* **Important!** write down your ClientId and Secret. You will need them in Postman to try it out.\n\n\u003cimg src=\"img/postman-oauth-client.png\" width=\"700\" /\u003e\n \n\n### Update your FHIR Server to use your OAuth2 configuration\nNow, go back and edit your FHIR Server endpoint in *Health \u003e SDF \u003e FHIR Configuration \u003e Server Configuration \u003e Edit Endpoint*\n\nUpdate the following:\n* OAuth2 Client Name: `fhirserver-resserver`\n\n### Try it using OAuth2!\n* Open the included Postman project\n* Update `oauth-clientid` and `oauth-clientsecret` Postman variables to use your ClientID and Secret.\n* Test the included FHIR OAuth requests by first requesting a token and then sending the request.\n\n\n# Analytics\n\n\u003cimg src=\"img/iris-analytics-platform.png\" width=\"1024\" /\u003e\n\nThere multiple ways in which you can leverage [analytics \u0026 data science](https://docs.intersystems.com/irisforhealthlatest/csp/docbook/DocBook.UI.Page.cls?KEY=PAGE_data_science) using InterSystems IRIS:\n- IRIS Business Intelligence - Allows you to embed business intelligence into your applications. You can have a first look at it in [workshop-iris-bi-intro](https://github.com/intersystems-ib/workshop-iris-bi-intro)\n- Adaptive Analytics - an optional extension that provides a business-oriented, virtual data model layer between InterSystems IRIS and popular Business Intelligence (BI) and Artificial Intelligence (AI) client tools. You can checkout this Spanish Webinar [Self-Service Analytics y Reporting](https://comunidadintersystems.com/webinar-self-service-analytics-y-reporting).\n\nWe will focus on IRIS BI on our first example.\n\n## BI. Defining a Cube\nOpen [Management Portal \u003e Analytics \u003e SDF \u003e Architect \u003e Open Observations Cube](http://localhost:52773/csp/sdf/_DeepSee.UI.Architect.zen?$NAMESPACE=SDF\u0026CUBE=ObxCube.cube) and go through the different dimensions and measures defined for a cube based on the observations persistent class.\n\nThese dimensions and measures define what kind of analysis can be done using this cube.\n\nClick on **Build** to build the cube based on the data you have loaded previously.\n\n\u003cimg src=\"img/bi-architect.png\" width=\"1024\" /\u003e\n\n## BI. Analyzer\nThen, open [Management Portal \u003e Analytics \u003e SDF \u003e Analyzer \u003e Open Observations Cube](http://localhost:52773/csp/sdf/_DeepSee.UI.Analyzer.zen?$NAMESPACE=SDF\u0026$NAMESPACE=SDF\u0026) and try different combinations for rows \u0026 columns on your analysis pivot table.\n\nYou can also open a pre-defined pivot. In your VS Code Import \u0026 Compile [AvgObservationsByAge.pivot.dfi](src/sdf/AvgObservationsByAge.pivot.dfi).\n\u003cimg src=\"img/bi-analyzer.gif\" width=\"1024\" /\u003e\n\n## BI. User portal\nFinally, you can create dashboards and build widgets based on your analysis pivot tables. In your VS code Import \u0026 Compile [AvgObservationsByCode.dashboard.dfi](src/sdf/AvgObservationsByCode.dashboard.dfi).\n\nThen, open [Management Portal \u003e Analytics \u003e SDF \u003e User Portal \u003e Open Avg Values by Sex, Age Dashboard](http://localhost:52773/csp/sdf/_DeepSee.UserPortal.Home.zen)\n\n\u003cimg src=\"img/bi-user-portal.gif\" width=\"1024\" /\u003e\n\n\n## BI. DSW\nIn [Open Exchange](https://openexchange.intersystems.com) you can find awesome applicatiosn like [DSW](https://openexchange.intersystems.com/package/DeepSeeWeb) that enables a whole new great looking UI for your IRIS BI. \n\nYou can checkout in your example accessing http://localhost:52773/dsw/index.html#/SDF \n\n\u003cimg src=\"img/bi-dsw.gif\" width=\"1024\" /\u003e\n\n\n## Machine Learning. IntegratedML\nAgain, there are different ways you can add [Machine Learning features on your InterSystems IRIS Applications](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=GIML_Intro):\n* IntegratedML - is an InterSystems IRIS features that allows you to leverage automated machine learning functions directly from SQL.\n* [PMML](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=APMML) - Predictive Modeling Markup Language is an XML-based standard that expresses analytical models. You can express a model using PMML and deploy it in InterSystems IRIS.\n* Python libraries - and of course, taking advantage of Embedded Python, you can use Python libraries such as `pandas`, `scikit-learn`, `tensorflow`, etc directly with your IRIS data to implement your ML models.\n\nWe will focus on a simple example of [IntegratedML](https://docs.intersystems.com/irisforhealth20222/csp/docbook/Doc.View.cls?KEY=GIML_Intro).\n\n\u003cimg src=\"img/automl.png\" width=\"800\" /\u003e\n\nWe are still using a dataset inspired on [Maternal Health Risk Data](https://www.kaggle.com/datasets/csafrit2/maternal-health-risk-data) dataset from Kaggle. \n\n\"Inspired\" in this particular case means that we won't have all the data available, so the accuracy of our ML model could be better.\n\nIn any case, if you are interested on using the whole dataset check out [workshop-integratedml-intro](https://github.com/intersystems-ib/workshop-integratedml-intro).\n\nNow, in our example check the following:\n* [MaternalRiskTrain.cls](src/sdf/data/MaternalRiskTrain.cls) - a view that we will use as our training data.\n* [MaternalRiskTest.cls](src/sdf/data/MaternalRiskTest.cls) - a view that we will use as our test data for validation.\n\nGo to [Management Portal \u003e Explorer \u003e SQL \u003e SDF](http://localhost:52773/csp/sys/exp/%25CSP.UI.Portal.SQL.Home.zen?$NAMESPACE=SDF) and run the following:\n\nCreate a model for predicting the `RiskLevel` column based on your training data:\n\n```sql\nCREATE MODEL MaternalModel PREDICTING (RiskLevel) FROM sdf_data.MaternalRiskTrain\n```\n\nTrain your model using your training data:\n\n```sql\nTRAIN MODEL MaternalModel\n```\n\nNow, validate your model using your test data:\n\n```sql\nVALIDATE MODEL MaternalModel FROM sdf_data.MaternalRiskTest\n```\n\nYou can check the validation metrics for your model:\n\n```sql\nSELECT * FROM INFORMATION_SCHEMA.ML_VALIDATION_METRICS\n```\n\nAnd finally, you can use your model to get predictions:\n\n```sql\nSELECT *, PREDICT(MaternalModel) AS PredictedRisk FROM sdf_data.MaternalRiskTest\n```\n\n\n## Jupyter Notebooks\nIn [docker-compose.yml](docker-compose.yml) has been added a jupyter notebook service so we can connect to IRIS using [IRIS Native SDK for Python](https://docs.intersystems.com/irisforhealth20222/csp/docbook/DocBook.UI.Page.cls?KEY=BPYNAT).\n\nTry the following:\n* Open your Jupyter Notebook instance in http://localhost:8888\n* Open [IRISPython.ipynb](jupyter/notebooks/IRISPython.ipynb)\n\nTry it! Think about all the available Python ML libraries you could use to analyze your IRIS data from a pure Python context. You can run queries or directly call your IRIS objects methods.\n\n\u003cimg src=\"img/jupyter-iris-native-python.gif\" width=\"1024\" /\u003e\n\n\n# Appendix. Generating data\nDuring the workshop you have been working with already created HL7 files inspired on [Maternal Health Risk Data](https://www.kaggle.com/datasets/csafrit2/maternal-health-risk-data) dataset from Kaggle.\n\nHere is how these HL7 files have been created:\n\nLoad train data into a temporary table in IRIS:\n```objectscript\ndo ##class(community.csvgen).Generate(\"/app/data/maternalRisk/maternal_health_risk.csv\",\",\",\"temp.MaternalHealthRisk\")\n```\n\nThen use a simple tool to generate HL7 files:\n```objectscript\ndo ##class(sdf.tools.HL7Generator).GenerateFilesHL7()\n```\n\nYour files will be generated in `data/maternalRisk/hl7gen`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintersystems-ib%2Fworkshop-smart-data-fabric","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fintersystems-ib%2Fworkshop-smart-data-fabric","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintersystems-ib%2Fworkshop-smart-data-fabric/lists"}