{"id":21526351,"url":"https://github.com/netapp/snapomatic","last_synced_at":"2025-03-17T18:34:19.614Z","repository":{"id":212354393,"uuid":"613476003","full_name":"NetApp/snapomatic","owner":"NetApp","description":"Sample Scripts illustrating ONTAP automation with applications such as Oracle","archived":false,"fork":false,"pushed_at":"2024-09-17T15:15:38.000Z","size":156,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-24T05:41:46.019Z","etag":null,"topics":["netapp-public"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NetApp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-13T16:34:07.000Z","updated_at":"2024-09-17T15:15:42.000Z","dependencies_parsed_at":"2023-12-13T21:18:34.160Z","dependency_job_id":"516bcf88-470b-4a86-a2aa-2943abebf9fa","html_url":"https://github.com/NetApp/snapomatic","commit_stats":null,"previous_names":["netapp/snapomatic"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NetApp%2Fsnapomatic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NetApp%2Fsnapomatic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NetApp%2Fsnapomatic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NetApp%2Fsnapomatic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NetApp","download_url":"https://codeload.github.com/NetApp/snapomatic/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244089379,"owners_count":20396243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["netapp-public"],"created_at":"2024-11-24T01:44:41.147Z","updated_at":"2025-03-17T18:34:19.592Z","avatar_url":"https://github.com/NetApp.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introduction\n\nSnapomatic is a collection of python utilities to perform useful things using ONTAP REST calls on a linux OS. It does not use any components other than Python and standard OS commands. These scripts were written by a longtime sysadmin with no programming education, just an interest in reliably automating as much work as possible.\n\nThe primary scripts are located in the NTAP directory. Many of them are extremely similar. Their function is to perform a RESTful operation and store the data in python lists and dictionaries. NetApp offers a python SDK for REST as well. The difference between that code and snapomatic is snapomatic is intended to be expanded.\n\nYou will also find some implementations of those NTAPlib modules in the root directory. For example, \"snapomatic.snapshot\" shows how to use the NTAPlib modules to list, create, and delete snapshots. The snapomatic.discover utility can perform discover operations on NFS, LVM, and raw devices (it's similar to the NetApp Host Utilities sanlun executable in that respect\"\n\nYou can run one of those \"snapomatic.*\" commands with no arguments and see syntax guidance. Some utilities must be run as root because they perform root-level storage operations. \n\nThe key to these scripts is the debug option\n\n## --debug\n\nThe example scripts accept a --debug argument to print more information about what's happening in the workflow. They also may print the OS commands being executed and display the stdin and stdout. \n\n## --restdebug\n\nThe --restdebug argument is especially useful for anyone looking to automate REST. This flag will display the REST converation, including API calls, JSON, arguments, and responses. It will also show you the polling calls for those APIs that are not synchronous. You can use this information in your own scripts, whether you're using Python, Java, or even basic curl.\n\n# Note\n\nThis is a work in progress. If you have questions, please open an Issue and I'll see what I can do to help. \n\n# Credentials\n\nOne important utility is snapomatic.credential. This is used to populate a file that defaults to /etc/snapomatic to store credentials for ONTAP and other systems. The result is a password that is stored in plaintext. This is somewhat unavoidable. The password could be stored encrypted, but these utilities are python scripts, and could be easily modified to simply print the password once decrypted. If you'd like to see certificates used, please let me know. \n\nYou should also use a username/password that has REST access only, and limit the access only to those RESTful APIs required for your project.\n\nMany functions require root access in order to perform low-level OS operations. If you do not want to store your credentials in /etc or you do not want to use root, you can set an alternate credential file location by exporting SNAPOMATIC_CREDENTIAL_PATH and select a different location.\n\nWhen you first run it, you'll see this:\n\n    [root@jfs0 current]# ./snapomatic.credential\n    ERROR: Unable to access /etc/snapomatic/config.json\n\nYou can initialize the file as follows:\n\n    [root@jfs0 current]$ ./snapomatic.credential INIT\n    Credential cache initialized\n\nYou can then add account information.\n\n## ONTAP Credentials\n\nONTAP credentials are SVM-scoped. This helps avoid user errors. \n\n    [oracle@jfs0 current]$ ./snapomatic.credential\n    Select one of the following\n    (1) Manage ONTAP credentials\n    (2) Manage Oracle databases\n    Selection: (1-2): 1\n    Select one of the following\n    (1) List credentials\n    (2) Add credential\n    (3) Edit credential\n    (4) Delete credential\n    Selection: (1-4): 2\n    Enter SVM name: jfsCloud4\n    Enter new management interface: 10.63.147.209  \u003c- Not real\n    Enter new username: username_with_required_api_access\n    Enter new password:\n    Enter new data interface(s), if any: a200B-nfs1,a200B-nfs2,a200B-iscsi1,a200B-iscsi2\n    /etc/snapomatic/config.json updated\n\nThe result is snapomatic now has credentials for an SVM called `jfsCloud4`, and it knows that this is the SVM providing data services on the four specified data interfaces. \n\n# debugging\n\nMost of the scripts include flags for --debug and --restdebug. The --debug flag will print details of the modules behavior and stdin/stdout. The --restdebug will show the REST conversations between the host and ONTAP system.\n\n# The --target argument\n\nMany scripts require the user to specify an target. The syntax for a target is one of the following, depending on the command:\n\n* svm volume [volume]\n* svm cg-name\n* path to a file\n* path to a LUN\n\n**Not all target types are fully supported. I'm adding them incrementally**\n\n# snapomatic.destroyVolume\n\nThis script is a wrapped that illustrates how to use NTAPlib/destroyVolumes.py \n\nIt does what it implies - it destroys a volume. There are no safeties. It will destroy the volume if the user credentials allow it. The volume will still be available in the recover-queue until expired.\n\n\n    [root@jfs0 current]# ./snapomatic.destroyVolume\n              --target\n              (svm volume)\n    \n              [--debug]\n              (show debug output)\n    \n              [--restdebug]\n              (show REST API calls and responses)\n\nThe script itself is basically a wrapper around the `NTAPlib/destroyVolumes.py` module. \n\n# snapomatic.discover\n\nThis script users several of the NTAPlib modules to perform basic discovery. For example:\n\n    [root@jfs0 current]# ./snapomatic.discover --target /oradata0\n    PATH      MOUNTPOINT FS   VG LV PV SVM      EXPORT         VOLUME        LUN\n    --------- ---------- ---- -- -- -- -------- -------------- ------------- ---\n    /oradata0 /oradata0  nfs4          jfs_svm1 /jfs0_oradata0 jfs0_oradata0\n\nWhat happened here, is the script took the filesystem argument of `/oradata0`, discovered it was an NFS filesystem that originated at a LIF that was registered to the SVM called `jfs_svm1`, and it made a few RESTful calls to get information about this volume. It used `NTAPlib/discoverNFS.py` for most of the work. There's more information about this volume stored in the discoverNFS object too.\n\nHere's a slightly more advanced example:\n\n    [root@jfs0 current]# ./snapomatic.discover --target /myLV\n    PATH  MOUNTPOINT FS  VG   LV   PV                                            SVM      EXPORT VOLUME       LUN\n    ----- ---------- --- ---- ---- --------------------------------------------- -------- ------ ------------ ----\n    /myLV /myLV      xfs myVG myLV /dev/mapper/3600a0980383041327a2b55676c547173 jfs_svm1        jfs0_lvmtest LUN0\n                                   /dev/mapper/3600a0980383041327a2b55676c547174 jfs_svm1        jfs0_lvmtest LUN1\n                                   /dev/mapper/3600a0980383041327a2b55676c547175 jfs_svm1        jfs0_lvmtest LUN2\n\nIn this case, the script discovered that /myLVM was an LVM-based filesystem. It then made a call to `NTAPlib/discoverLVM.py` which mapped this filesystem to its logical volume, and then to the volume group, and then to the underlying physical volumes. It then used NTAPlib/discoverLUN.py to send a specially formatted SCSI command to the LUN device backing the PV and ONTAP responded with identifying information.\n\nFinally, you can run this utility directly against the raw LUNs. This is useful for managing Oracle ASM or newly provisioned LUNs that are not yet part of a filesystem. This script leverages NTAPlib/discoverLUN.py to probe the LUN itself for data. \n\n    [root@jfs0 current]# ./snapomatic.discover --target /dev/mapper/*\n    PATH                                          MOUNTPOINT FS    VG LV PV SVM      EXPORT VOLUME       LUN\n    --------------------------------------------- ---------- ----- -- -- -- -------- ------ ------------ ------\n    /dev/mapper/3600a0980383041327a2b55676c547173            block          jfs_svm1        jfs0_lvmtest LUN0\n    /dev/mapper/3600a0980383041327a2b55676c547174            block          jfs_svm1        jfs0_lvmtest LUN1\n    /dev/mapper/3600a0980383041327a2b55676c547175            block          jfs_svm1        jfs0_lvmtest LUN2\n    /dev/mapper/3600a0980383041334a3f55676c697278            block          jfs_svm1        lvm_convert  bigLUN\n    /dev/mapper/3600a0980383041334a3f55676c697279            block          jfs_svm1        lvm_convert  small0\n    /dev/mapper/3600a0980383041334a3f55676c69727a            block          jfs_svm1        lvm_convert  small1\n    /dev/mapper/3600a0980383041334a3f55676c697330            block          jfs_svm1        lvm_convert  small2\n    /dev/mapper/3600a0980383041334a3f55676c697331            block          jfs_svm1        lvm_convert  small3\n    /dev/mapper/3600a0980383041334a3f55676c697332            block          jfs_svm1        lvm_convert  small4\n    /dev/mapper/3600a0980383041334a3f55676c697333            block          jfs_svm1        lvm_convert  small5\n    /dev/mapper/3600a0980383041334a3f55676c697334            block          jfs_svm1        lvm_convert  small6\n    /dev/mapper/3600a0980383041334a3f55676c697335            block          jfs_svm1        lvm_convert  small7\n    /dev/mapper/bigVG-bigLV                                  block          jfs_svm1        lvm_convert  bigLUN\n    /dev/mapper/myVG-myLV                                    block          jfs_svm1        jfs0_lvmtest LUN0\n    /dev/mapper/rhel-root                         ?          ?     ?  ?  ?  ?        ?      ?            ?\n    /dev/mapper/rhel-swap                         ?          ?     ?  ?  ?  ?        ?      ?            ?\n    /dev/mapper/control                           ?          ?     ?  ?  ?  ?        ?      ?            ?\n    \nMost of these LUNs were identified as being ONTAP-hosted LUNs. Some of them are unknown. In this case, they are VMware virtual LUNs.\n\n# snapomatic.volume\n\nThis script is a wrapper around `NTAPlib/getVolumes.py`, `NTAPlib/cloneVolumes.py`, and `NTAPlib/createSnapshots.py`. As with other scripts, there is a lot more information in the getVolume object. The wrapper shows the basics.\n\n## show\n\n    [root@jfs0 current]# ./snapomatic.volume show --target jfs_svm1 jfs0_oradata0\n    VOLUME        SIZE (GB) AGGREGATE\n    ------------- --------- ------------------\n    jfs0_oradata0 256000.0  rtp_a700s_01_SSD_1\n\nWildcards are also supported. Make sure you enclose the arguments in quotes or the shell will expand them. \n\n    [root@jfs0 current]# ./snapomatic.volume show --target jfs_svm1 'jfs0_*'\n    VOLUME        SIZE (GB) AGGREGATE\n    ------------- --------- ------------------\n    jfs0_arch     102400.0  rtp_a700s_01_SSD_1\n    jfs0_lvmtest  20480.0   rtp_a700s_01_SSD_1\n    jfs0_orabin   73728.0   rtp_a700s_01_SSD_1\n    jfs0_oradata0 256000.0  rtp_a700s_01_SSD_1\n    jfs0_oradata1 256000.0  rtp_a700s_02_SSD_1\n    jfs0_oradata2 256000.0  rtp_a700s_01_SSD_1\n    jfs0_oradata3 256000.0  rtp_a700s_02_SSD_1\n    jfs0_oratmp0  102400.0  rtp_a700s_01_SSD_1\n    jfs0_oratmp1  102400.0  rtp_a700s_02_SSD_1\n    jfs0_redo0    10240.0   rtp_a700s_01_SSD_1\n    jfs0_redo1    10240.0   rtp_a700s_02_SSD_1\n\n## clone\n\nYou can clone a volume too.\n\n    [root@jfs0 current]# ./snapomatic.volume clone --target jfs_svm1 jfs0_oradata0 --name newclone\n    Cloned newclone from jfs0_oradata0\n\nI'll destroy it before I forget it's there.\n\n    [root@jfs0 current]# ./snapomatic.destroyVolume --target jfs_svm1 newclone\n    Destroyed jfs_svm1:newclone\n\n# snapomatic.snapshot\n\nThis script is a wrapper around various `NTAPlib/` modules.\n\n    [root@jfs0 current]# ./snapomatic.snapshot\n    snapshot show\n            [--target]\n            (svm [filesystem, LUN, or volume]\n    \n            (Optional. Name for snapshot. Wildcards accepted.)\n    \n            [--cg]\n            (Restrict search to CG snapshots. Wildcards accepted.)\n    \n            [--debug]\n            (show debug output)\n    \n            [--restdebug]\n            (show REST API debug output)\n    \n    snapshot create\n            [--target]\n            (svm [filesystem, CG, LUN, or volume]\n    \n            [--name||prefix]\n            (Name or prefix for snapshot)\n    \n            [--label|]\n            (Snapmirror label for the snapshot)\n    \n            [--cg]\n            (Create CG snapshots.)\n    \n            [--debug]\n            (show debug output)\n    \n            [--restdebug]\n            (show REST API debug output)\n    \n    snapshot delete\n            [--target]\n            (svm [filesystem, LUN, or volume]\n    \n            [--name]\n            (Optional. Name of snapshot. Wildcards accepted)\n    \n            [--maxcount]\n            (Optional. Maximum number of snapshots to retain)\n    \n            [--maxage]\n            (Optional. Maximum age of snapshots to retain.\n            (Accepts d[ays]/h[ours]/m[inutes]/[s]econds\n    \n            [--force]\n            (Override all safeguards)\n    \n            [--debug]\n            (show debug output)\n    \n            [--restdebug]\n            (show REST API debug output)\n\n## show\n\nFor example, I can view the snapshots like this:\n\n    [root@jfs0 current]# ./snapomatic.snapshot show --target jfs_svm1 jfs0_oradata0\n    VOLUME        SNAPSHOT                           DATE\n    ------------- ---------------------------------- -------------------------\n    jfs0_oradata0 clone_newclone.2024-02-01_204436.0 2024-02-01T20:44:36+00:00\n\n**Note: This only reports regular snapshots, not CG snapshots. \n\nThis could easily be done on the ONTAP CLI too. The point of this script is not to demonstrate how to script such a basic option, the point is to lead you to  understand how the ability to retrieve snapshots via Python can be incorporated into other scripts with more advanced workflows.\n\nIf I wanted to view CG snapshots, I need to use the `--target svm cg-name` syntax.\n    \n    [root@jfs0 current]# ./snapomatic.snapshot show --target jfs_svm1 jfs0 --cg \n    CG   PARENT VOLUME        SNAPSHOT                                                    DATE\n    jfs0        jfs0_arch     every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_arch     every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_orabin   every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_orabin   every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oradata0 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oradata0 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oradata1 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oradata1 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oradata2 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oradata2 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oradata3 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oradata3 every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oratmp0  every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oratmp0  every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_oratmp1  every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_oratmp1  every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_redo0    every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_redo0    every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n    jfs0        jfs0_redo1    every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1921 2024-02-01T19:21:00+00:00\n    jfs0        jfs0_redo1    every5_c1af0dc4-6143-11ee-ae6e-00a098f7d731.2024-02-01_1931 2024-02-01T19:31:00+00:00\n\n## create\n\nCreation of snapshot is similarly intutive. Here's creation of five snapshots, `test0`, `test1`, `test2`, `test3` and `test4`\n\n    [root@jfs0 current]# ./snapomatic.snapshot create --target jfs_svm1 jfs0_oradata0 --name test0\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test0    Success\n    [root@jfs0 current]# ./snapomatic.snapshot create --target jfs_svm1 jfs0_oradata0 --name test1\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test1    Success\n    [root@jfs0 current]# ./snapomatic.snapshot create --target jfs_svm1 jfs0_oradata0 --name test2\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test2    Success\n    [root@jfs0 current]# ./snapomatic.snapshot create --target jfs_svm1 jfs0_oradata0 --name test3\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test3    Success\n    [root@jfs0 current]# ./snapomatic.snapshot create --target jfs_svm1 jfs0_oradata0 --name test4\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test4    Success\n\n## delete\n\nI can also delete snapshots based on age or count. For example, I just created 5 snapshots with a name of `snapshot*`.\n\nI can delete those snapshots (wildcards supported) using a maxcount of 4:\n\n    [root@jfs0 current]# ./snapomatic.snapshot delete --target jfs_svm1 jfs0_oradata0 --name 'test*' --maxcount 4\n    Success\n    VOLUME        SNAPSHOT STATUS\n    ------------- -------- -------\n    jfs0_oradata0 test0    Deleted\n\nThe `test0` snapshot was the oldest snapshot, which is why it was deleted. The youngest 4 snapshots were retained.\n\n\n# snapomatic.splitClone\n\nThis is a wrapper for `NTAPlib/splitClones.py`.\n\n    [root@jfs0 current]# ./snapomatic.splitClones\n              --target\n              (svm volume)\n    \n              --synchronous\n              (wait for split to complete)\n    \n              [--debug]\n              (show debug output)\n    \n              [--restdebug]\n              (show REST API calls and responses)\n\n\nThe example below shows the operation run with `--restdebug`. This demonstrates how multiple NTAPlib modules can be used in concert.\n\n    [root@jfs0 current]# ./snapomatic.splitClones --target jfs_dev1 myclone --restdebug\n    -\u003edoREST:REST:API: GET https://10.192.160.40/api/storage/volumes?fields=uuid,size,svm.name,svm.uuid,nas.path,aggregates,type\u0026name=myclone\u0026svm.name=jfs_dev1\n    -\u003edoREST:REST:JSON: None\n    -\u003edoREST:REST:RESPONSE: {\n    -\u003edoREST:REST:RESPONSE:  \"records\": [\n    -\u003edoREST:REST:RESPONSE:   {\n    -\u003edoREST:REST:RESPONSE:    \"uuid\": \"2288f071-d692-11ee-8ab0-00a098f7d731\",\n    -\u003edoREST:REST:RESPONSE:    \"name\": \"myclone\",\n    -\u003edoREST:REST:RESPONSE:    \"size\": 53687091200,\n    -\u003edoREST:REST:RESPONSE:    \"type\": \"rw\",\n    -\u003edoREST:REST:RESPONSE:    \"aggregates\": [\n    -\u003edoREST:REST:RESPONSE:     {\n    -\u003edoREST:REST:RESPONSE:      \"name\": \"rtp_a700s_01_SSD_1\",\n    -\u003edoREST:REST:RESPONSE:      \"uuid\": \"bb561960-4829-4b0e-bfab-baeb7b4ba3be\"\n    -\u003edoREST:REST:RESPONSE:     }\n    -\u003edoREST:REST:RESPONSE:    ],\n    -\u003edoREST:REST:RESPONSE:    \"svm\": {\n    -\u003edoREST:REST:RESPONSE:     \"name\": \"jfs_dev1\",\n    -\u003edoREST:REST:RESPONSE:     \"uuid\": \"ac509ea6-fa33-11ed-ae6e-00a098f7d731\",\n    -\u003edoREST:REST:RESPONSE:     \"_links\": {\n    -\u003edoREST:REST:RESPONSE:      \"self\": {\n    -\u003edoREST:REST:RESPONSE:       \"href\": \"/api/svm/svms/ac509ea6-fa33-11ed-ae6e-00a098f7d731\"\n    -\u003edoREST:REST:RESPONSE:      }\n    -\u003edoREST:REST:RESPONSE:     }\n    -\u003edoREST:REST:RESPONSE:    },\n    -\u003edoREST:REST:RESPONSE:    \"_links\": {\n    -\u003edoREST:REST:RESPONSE:     \"self\": {\n    -\u003edoREST:REST:RESPONSE:      \"href\": \"/api/storage/volumes/2288f071-d692-11ee-8ab0-00a098f7d731\"\n    -\u003edoREST:REST:RESPONSE:     }\n    -\u003edoREST:REST:RESPONSE:    }\n    -\u003edoREST:REST:RESPONSE:   }\n    -\u003edoREST:REST:RESPONSE:  ],\n    -\u003edoREST:REST:RESPONSE:  \"num_records\": 1,\n    -\u003edoREST:REST:RESPONSE:  \"_links\": {\n    -\u003edoREST:REST:RESPONSE:   \"self\": {\n    -\u003edoREST:REST:RESPONSE:    \"href\": \"/api/storage/volumes?fields=uuid,size,svm.name,svm.uuid,nas.path,aggregates,type\u0026name=myclone\u0026svm.name=jfs_dev1\"\n    -\u003edoREST:REST:RESPONSE:   }\n    -\u003edoREST:REST:RESPONSE:  }\n    -\u003edoREST:REST:RESPONSE: }\n    -\u003edoREST:RESULT: 200\n    -\u003edoREST:REASON: OK\n    -\u003edoREST:REST:API: PATCH https://10.192.160.40/api/storage/volumes/2288f071-d692-11ee-8ab0-00a098f7d731\n    -\u003edoREST:REST:JSON: {'clone.split_initiated': 'true'}\n    -\u003edoREST:REST:RESPONSE: {\n    -\u003edoREST:REST:RESPONSE:  \"job\": {\n    -\u003edoREST:REST:RESPONSE:   \"uuid\": \"0efdf999-d694-11ee-8ab0-00a098f7d731\",\n    -\u003edoREST:REST:RESPONSE:   \"_links\": {\n    -\u003edoREST:REST:RESPONSE:    \"self\": {\n    -\u003edoREST:REST:RESPONSE:     \"href\": \"/api/cluster/jobs/0efdf999-d694-11ee-8ab0-00a098f7d731\"\n    -\u003edoREST:REST:RESPONSE:    }\n    -\u003edoREST:REST:RESPONSE:   }\n    -\u003edoREST:REST:RESPONSE:  }\n    -\u003edoREST:REST:RESPONSE: }\n    -\u003edoREST:RESULT: 202\n    -\u003edoREST:REASON: Accepted\n    -\u003edoREST:REST:API: GET https://10.192.160.40/api/cluster/jobs/0efdf999-d694-11ee-8ab0-00a098f7d731?fields=state,message\n    -\u003edoREST:REST:JSON: None\n    -\u003edoREST:REST:RESPONSE: {\n    -\u003edoREST:REST:RESPONSE:  \"uuid\": \"0efdf999-d694-11ee-8ab0-00a098f7d731\",\n    -\u003edoREST:REST:RESPONSE:  \"state\": \"running\",\n    -\u003edoREST:REST:RESPONSE:  \"message\": \"Clone split initiated.\",\n    -\u003edoREST:REST:RESPONSE:  \"_links\": {\n    -\u003edoREST:REST:RESPONSE:   \"self\": {\n    -\u003edoREST:REST:RESPONSE:    \"href\": \"/api/cluster/jobs/0efdf999-d694-11ee-8ab0-00a098f7d731\"\n    -\u003edoREST:REST:RESPONSE:   }\n    -\u003edoREST:REST:RESPONSE:  }\n    -\u003edoREST:REST:RESPONSE: }\n    -\u003edoREST:RESULT: 200\n    -\u003edoREST:REASON: OK\n    Clone split initiated.\n\nThe first API validated that the target volume myclone exists.\n\n    /api/storage/volumes?fields=uuid,size,svm.name,svm.uuid,nas.path,aggregates,type\u0026name=myclone\u0026svm.name=jfs_dev1\n\nThe second API call started the split operation \n\n    -\u003edoREST:REST:API: PATCH https://10.192.160.40/api/storage/volumes/2288f071-d692-11ee-8ab0-00a098f7d731\n    -\u003edoREST:REST:JSON: {'clone.split_initiated': 'true'}\n\nThis API returned a result code of 202, which means the call was accepted. The final API call used the job UUID to verify that the split operation was running.\n\n    -\u003edoREST:REST:API: GET https://10.192.160.40/api/cluster/jobs/0efdf999-d694-11ee-8ab0-00a098f7d731?fields=state,message\n\nWhen run without --synchronous, this script doesn't wait to check whether the operation completed, it only checked to see if the operation got to the point where it was running, as indicated by the \"state\" field by the response.\n\n# snapomatic.cloneSAN4DR\n\nThis is a more complicated workflow created for a customer POC. It illustrates stringing together multiple modules. The full details will be covered in a future project, but the basic requirement was this:\n\n* The ability to select one or more snapmirrored volumes, and\n* Clone the volumes. Do NOT break the mirrors. This was required so DR can be tested without interrupting replication.\n* Create the clone of the target volumes to a timepoint that is immediately *after* or immediately *before* the desired recovery point. \n* Map all LUNs\n* Bring up databases, but this action is performed by a 2nd script. Keep reading to find that example...\n\nThis script works as follows:\n\n[root@jfs0 current]# ./snapomatic.cloneSAN4DR\nclone4DR --target\n          (name of target svm and volumes)\n          (syntax: svm volume or svm volume/lun, wildcards accepted)\n\n          --snapshot_prefix\n          (optionally restrict search to snapshots with a prefix_ syntax)\n          --igrouptoken\n          (identifies the _ delimited position of the igroup)\n\n          --recoverypoint\n          (recovery timestamp in YYYY-MM-DDTHH:MM:SS+ZZZZ)\n\n          --after|--before\n          (used with --recoverypoint)\n          (look for snapshots before|after specified recoverypoint)\n          (default behavior is AFTER)\n\n          [--igroupname]\n          (optionally specifies the igroup name, %=token\n\n          [--split]\n          (split the cloned volumes)\n\n          [--debug]\n          (show debug output)\n\n          [--restdebug]\n          (show REST API calls and responses)\n\nAs an example, the command could be run like this:\n\n    [root@jfs0 current]# ./snapomatic.cloneSAN4DR --target jfs_dev2 'ora_DRcluster_*' \\ \n                                                  --recoverypoint 2024-02-20T12:30:00+0500 \\\n                                                  --before \\\n                                                  --igrouptoken 2\n\nThe logic would then do this:\n\n1. Identify all snapmirrored volumes on `jfs_dev` matching the pattern `ora_DRcluster_*`\n2. Identify the snapshot immediately *before* the specified timestamp\n3. Create a clone of those volumes. It will prepend the string `failover_` to the clone name\n4. Because we are not breaking the mirrors, that clone will be created using the identified snapshot\n5. Enumerate all of the LUNs in the newly cloned volumes\n6. Extract the 2nd token of the original _ character delimted volume name to be used as the igroup name\n7. In this case, map the LUNs to the igroup \"DRcluster\" because that was the 2nd token in the volume name\n\nWe also had a requirement to selectively clone only certain LUNs. As SnapMirror works on the volume level, we still clone all LUNs but only matching LUNs are mapped. The syntax for this operation is as follows:\n\n\n    [root@jfs0 current]# ./snapomatic.cloneSAN4DR --target jfs_dev2 'ora_DRcluster_*/LUN2' \\ \n                                                  --recoverypoint 2024-02-20T12:30:00+0500 \\\n                                                  --before \\\n                                                  --igrouptoken 2\n\nIn this example, the script is looking for a volume/LUN pattern match of oraDRcluster_*/LUN2.\n\nAs an example of running this at large scale, the following two commands could be run:\n\n\n    [root@jfs0 current]# ./snapomatic.cloneSAN4DR --target jfs_dev2 'ora_DRcluster_.*dbf' \\ \n                                                  --recoverypoint 2024-03-03T12:30:00+0500 \\\n                                                  --before \\\n                                                  --igrouptoken 2\n    \n    [root@jfs0 current]# ./snapomatic.cloneSAN4DR --target jfs_dev2 'ora_DRcluster_.*logs' \\ \n                                                  --recoverypoint 2024-03-03T12:30:00+0500 \\\n                                                  --after \\\n                                                  --igrouptoken 2\n\nThe result of this two commands are the following LUNs, mapped, inside of cloned volumes. The LUNs in the `_dbf` volumes were cloned from a snapshot created *prior* to the desired recoverypoint, and the LUNS in the `_logs` volumes were cloned from a snapshot created immediately *after* the desired recoverypoint. This is the starting point for Oracle and most database recovery operations - you start with a copy of datafiles prior to the point-in-time recovery mark, and a copy of logs from after the PIT mark. You then replay logs to the desired point.\n\n    rtp-a700s-c02::\u003e lun show -vserver jfs_dev2 -fields path /vol/fail*\n    vserver  path\n    -------- ----------------------------------------\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_BII_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_CRM_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_DEV_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_DWH_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_ERP_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_FCST_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_HRS_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_SPC_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_TST_logs/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_dbf/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_dbf/LUN1\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_dbf/LUN2\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_dbf/LUN3\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_logs/LUN0\n    jfs_dev2 /vol/failover_ora_DRcluster_UAT_logs/LUN1\n    60 entries were displayed.\n    \n# snapomatic.RACfailover\n\nThis next script is the next step after using snapmatic.clone4DR. The purpose of clone4DR was to identify and clone the volumes and snapshots that made up a usable DR copy of the database at the desired recoverypoint. The purpose of RACfailover is to discover the contents of the newly cloned LUNs and register and recovery the databases.\n\nSyntax:\n\n    snapmatic.RACfailover --volpattern\n              (target volume pattern)\n\n              --recoverypoint\n              (recovery timestamp in YYYY-MM-DDTHH:MM:SS[TZ])\n\n              --snapshot-optimized\n              (use snapshot-optimized recovery procedure instead of backup mode procedure)\n\n                --debug\n              (print process STDOUT and STDERR)\n\n                --iscsi\n              (Scan for iSCSI LUNs)\n\n                --noscan\n              (Bypass LUN scanning)\n\n                --noafd\n              (Bypass ASM Filter Driver scanning)\n\n                --noasmlib\n              (Bypass ASMlib scanning)\n\n\nMost of those arguments should be self-explanatory, but one requires some explanation. The --snapshot-optimized and --recoverypoint work together to perform recovery in a different way.\n\n1. If --recoverypoint is not specified, then the script will issue a \"recover automatic;\" operation, which will recover all available archive logs and redo logs.\n2. If you specify --recoverypoint and --snapshot-optimized, the script will assume datafiles are not in backup mode and  attempt \"recover database until time YYYY-MM-DD HH:MM:SS snapshot time YYYY-MM-DD HH:MM:SS\"\n3. If you specify --recoverypoint without --snapshot-optimized, the script will assume datafiles are in backup mode and it will attempt a normal \"recover database until time YYYY-MM-DD HH:MM:SS\" operation.\n\nThe following output shows the results of completing the failover of the 10 databases that were cloned in the clone4DR step.\n\nThe command is just this:\n\n    [root@jfs8 current]# ./snapomatic.RACfailover --volpattern 'failover_ora_DRcluster_*' --iscsi --recoverypoint 2024-03-03T12:30:00+0500\n    Updating iSCSI targets\n\nThis is going to trigger the following steps:\n\n1. Rescan all LUNs, including an iSCSI refresh (the test environment uses iSCSI)\n2. Perform discovery on all LUNs that are hosted on volumes matching the pattern \"failover_ora_DRcluster\". The clone4DR script cloned volumes matching ora_DRcluster* and prepended failover_ onto the volumes. The RACfailover script will look for LUNs matching this pattern.\n3. Attempt to discover ASMlib and ASM Filter Driver signatures on the LUNs and build ASMlib/AFD devices if required\n4. Mount all discovered ASM diskgroups on all hosts in the RAC cluster\n5. Read the contents of those ASM diskgroups. The script is looking for directories containing an spfile and a passwd file. If those files are found, that means the containing directory is the name of a database that has been cloned and should be brought online.\n6. Identify all installed versions of Oracle Database. There could be multiple versions. The script will walk the versions and attempt to read the spfile discovered on the ASM diskgroups. Once the script can succesfully read this data, the script can determine the correct version of Oracle for that database.\n7. Register the databases with RAC using srvctl.\n8. Mount the database\n9. Recover the database to the specified recoverypoint\n10. Open the database.\n\nThe output looks like this:\n\n    Sleeping for 10 seconds while multipath maps are built\n\nThis sleep step is required to avoid race conditions between multipathd and optional packages such as the ASM Filter Driver.\n\n    Discovering AFD devices...\n    Refreshing AFD configuration\n\n    Refreshing ASMlib configuration\n    Unable to scan ASMlib disks on host jfs8\n\nSome of the ASM diskgroups may have been under the control of AFD or ASMlib. The required administrative commands to rescan for those devices was invoked. In this test environment, ASMlib was not installed so an error was reported.\n\n    Retrieving ASM diskgroup names\n    \u003e\u003e Identified diskgroup HRLOGS\n    \u003e\u003e Identified diskgroup DWHDATA\n    \u003e\u003e Identified diskgroup SUPPLYLOGS\n    \u003e\u003e Identified diskgroup CRMLOGS\n    \u003e\u003e Identified diskgroup UATDATA\n    \u003e\u003e Identified diskgroup DEVLOGS\n    \u003e\u003e Identified diskgroup TSTLOGS\n    \u003e\u003e Identified diskgroup ERPLOGS\n    \u003e\u003e Identified diskgroup FORECASTDATA\n    \u003e\u003e Identified diskgroup TSTDATA\n    \u003e\u003e Identified diskgroup DEVDATA\n    \u003e\u003e Identified diskgroup BILOGS\n    \u003e\u003e Identified diskgroup CRMDATA\n    \u003e\u003e Identified diskgroup HRDATA\n    \u003e\u003e Identified diskgroup FORECASTLOGS\n    \u003e\u003e Identified diskgroup SUPPLYDATA\n    \u003e\u003e Identified diskgroup ERPDATA\n    \u003e\u003e Identified diskgroup DWHLOGS\n    \u003e\u003e Identified diskgroup BIDATA\n    \u003e\u003e Identified diskgroup UATLOGS\n\nThe names of the ASM diskgroups were identified using kfed, Oracle's low-level disk scanning utility. Once we have the name of the diskgroups, they can be mounted. It's possible this script was run more than once, so before attempting to mount, the script will check on which are already mounted.\n\n    Identifying currently mounted ASM diskgroups\n    \n    Mounting ASM diskgroups...\n    Mounting HRLOGS on host jfs8\n    Mounting DWHDATA on host jfs8\n    Mounting SUPPLYLOGS on host jfs8\n    Mounting CRMLOGS on host jfs8\n    Mounting UATDATA on host jfs8\n    Mounting DEVLOGS on host jfs8\n    Mounting TSTLOGS on host jfs8\n    Mounting ERPLOGS on host jfs8\n    Mounting FORECASTDATA on host jfs8\n    Mounting TSTDATA on host jfs8\n    Mounting DEVDATA on host jfs8\n    Mounting BILOGS on host jfs8\n    Mounting CRMDATA on host jfs8\n    Mounting HRDATA on host jfs8\n    Mounting FORECASTLOGS on host jfs8\n    Mounting SUPPLYDATA on host jfs8\n    Mounting ERPDATA on host jfs8\n    Mounting DWHLOGS on host jfs8\n    Mounting BIDATA on host jfs8\n    Mounting UATLOGS on host jfs8\n\nIn this case, the databases were divided into a datafile and a log ASM diskgroup. This is required for PIT recovery as explained above. The copy of the datafile needs to be from an earlier point in time than the logs. \n    \n    Discovering contents of ASM diskgroup\n    Retrieving contents of diskgroups...\n    \u003e\u003e Running asmcmd...\n    \u003e\u003e Found directory HRU on diskgroup +HRDATA\n    \u003e\u003e Found directory BIU on diskgroup +BILOGS\n    \u003e\u003e Found directory CRMU on diskgroup +CRMLOGS\n    \u003e\u003e Found directory DEVU on diskgroup +DEVLOGS\n    \u003e\u003e Found directory DWHU on diskgroup +DWHLOGS\n    \u003e\u003e Found directory ERPU on diskgroup +ERPLOGS\n    \u003e\u003e Found directory FCSTU on diskgroup +FORECASTLOGS\n    \u003e\u003e Found directory HRU on diskgroup +HRLOGS\n    \u003e\u003e Found directory SUPPLYU on diskgroup +SUPPLYLOGS\n    \u003e\u003e Found directory TSTU on diskgroup +TSTLOGS\n    \u003e\u003e Found directory UATU on diskgroup +UATLOGS\n    \u003e\u003e Found directory BIU on diskgroup +BIDATA\n    \u003e\u003e Found directory CRMU on diskgroup +CRMDATA\n    \u003e\u003e Found directory DEVU on diskgroup +DEVDATA\n    \u003e\u003e Found directory DWHU on diskgroup +DWHDATA\n    \u003e\u003e Found directory ERPU on diskgroup +ERPDATA\n    \u003e\u003e Found directory FCSTU on diskgroup +FORECASTDATA\n    \u003e\u003e Found directory SUPPLYU on diskgroup +SUPPLYDATA\n    \u003e\u003e Found directory TSTU on diskgroup +TSTDATA\n    \u003e\u003e Found directory UATU on diskgroup +UATDATA\n    \u003e\u003e Found directory HRSU on diskgroup +HRDATA\n    \u003e\u003e Identified database HRU\n    \u003e\u003e Identified database BIU\n    \u003e\u003e Identified database CRMU\n    \u003e\u003e Identified database DEVU\n    \u003e\u003e Identified database DWHU\n    \u003e\u003e Identified database ERPU\n    \u003e\u003e Identified database FCSTU\n    \u003e\u003e Identified database SUPPLYU\n    \u003e\u003e Identified database TSTU\n    \u003e\u003e Identified database UATU\n\nThe output above reflects a search for directories containing an spfile and a passwd file. If those two files exist, then the name of the database should be the containing directory. The script has now identified at least part of a database, but there's no guarantee all LUNs were cloned at this point. There's also no way to know which version of Oracle is associated with this database. \n    \n    Oracle version map: {'19.0.0.0.0': '/orabin19', '19.18.0.0.0': '/orabin19'}\n    Extracting spfiles...\n    \u003e\u003e  Attempting to create pfile from spfile +HRDATA/HRU/PARAMETERFILE/spfile.268.1144254031\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database HRU configured for 19.0.0\n    \u003e\u003e Database HRU has db_name of NHRU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for HRU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +BIDATA/BIU/PARAMETERFILE/spfile.267.1132063661\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database BIU configured for 19.0.0\n    \u003e\u003e Database BIU has db_name of NBIU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for BIU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +CRMDATA/CRMU/PARAMETERFILE/spfile.259.1132093037\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database CRMU configured for 19.0.0\n    \u003e\u003e Database CRMU has db_name of NCRMU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for CRMU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +DEVDATA/DEVU/PARAMETERFILE/spfile.267.1132086883\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database DEVU configured for 19.0.0\n    \u003e\u003e Database DEVU has db_name of NDEVU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for DEVU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +DWHDATA/DWHU/PARAMETERFILE/spfile.267.1132093245\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database DWHU configured for 19.0.0\n    \u003e\u003e Database DWHU has db_name of NDWHU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for DWHU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +ERPDATA/ERPU/PARAMETERFILE/spfile.267.1132101237\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database ERPU configured for 19.0.0\n    \u003e\u003e Database ERPU has db_name of NERPU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for ERPU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +FORECASTDATA/FCSTU/PARAMETERFILE/spfile.267.1132102107\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database FCSTU configured for 19.0.0\n    \u003e\u003e Database FCSTU has db_name of NFCSTU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for FCSTU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +SUPPLYDATA/SUPPLYU/PARAMETERFILE/spfile.267.1132102147\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database SUPPLYU configured for 19.0.0\n    \u003e\u003e Database SUPPLYU has db_name of NSUPPLYU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for SUPPLYU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +TSTDATA/TSTU/PARAMETERFILE/spfile.267.1132140189\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database TSTU configured for 19.0.0\n    \u003e\u003e Database TSTU has db_name of NTSTU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for TSTU\n      \u003e\u003e Database registration successful\n    \n    \u003e\u003e  Attempting to create pfile from spfile +UATDATA/UATU/PARAMETERFILE/spfile.267.1132140185\n      \u003e\u003e Using ORACLE_HOME /orabin19\n      \u003e\u003e Parsing pfile\n    \u003e\u003e Database UATU configured for 19.0.0\n    \u003e\u003e Database UATU has db_name of NUATU\n      \u003e\u003e Compatible ORACLE_HOME found at /orabin19\n    \u003e\u003e Creating directory structure\n      \u003e\u003e Running mkdir commands on host jfs8\n    \u003e\u003e Running svrctl add for UATU\n      \u003e\u003e Database registration successful\n    \nThe output above showed the script first identifying all installed versions of Oracle Database on the server, followed by and attempt to create a pfile from the text spfile. The spfile cannot be reliably read directly. It must be converted to a text pfile first. In the unlikely event that a given version of Oracle was unable to read the spfile, another version of Oracle will be used. \n\nEventually, the version of Oracle for each database will be identified. Once this happens, a `srvctl add database` command is performed. You can see the \"Database registration succesful\" message above.\n\nThe next step is mounting the databases. They cannot be directly opened because the state of the datafiles and the state of the log files do not match.\n    \n    Starting databases...\n    \u003e\u003e Mounting database HRU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database BIU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database CRMU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database DEVU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database DWHU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database ERPU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database FCSTU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database SUPPLYU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database TSTU\n      \u003e\u003e Database mounted\n    \u003e\u003e Mounting database UATU\n      \u003e\u003e Database mounted\n    \nThe next step is issuing the recovery command. \n\nIn this case, the command is recover `database until time '2024-03-03 07:30:00';` That timestamp does *not* match the timestamp provided at the CLI exactly. The sqlplus command needs to use the local timezone of the server, which in this case was EST, five hours earlier than the 12:00 time provided at the CLI. \n\n    Recovering databases...\n    \u003e\u003e Recovering database HRU\n    Media recovery complete.\n    \u003e\u003e Database HRU recovery complete\n    \u003e\u003e Recovering database BIU\n    Media recovery complete.\n    \u003e\u003e Database BIU recovery complete\n    \u003e\u003e Recovering database CRMU\n    Media recovery complete.\n    \u003e\u003e Database CRMU recovery complete\n    \u003e\u003e Recovering database DEVU\n    Media recovery complete.\n    \u003e\u003e Database DEVU recovery complete\n    \u003e\u003e Recovering database DWHU\n    Media recovery complete.\n    \u003e\u003e Database DWHU recovery complete\n    \u003e\u003e Recovering database ERPU\n    Media recovery complete.\n    \u003e\u003e Database ERPU recovery complete\n    \u003e\u003e Recovering database FCSTU\n    Media recovery complete.\n    \u003e\u003e Database FCSTU recovery complete\n    \u003e\u003e Recovering database SUPPLYU\n    Media recovery complete.\n    \u003e\u003e Database SUPPLYU recovery complete\n    \u003e\u003e Recovering database TSTU\n    Media recovery complete.\n    \u003e\u003e Database TSTU recovery complete\n    \u003e\u003e Recovering database UATU\n    Media recovery complete.\n    \u003e\u003e Database UATU recovery complete\n\nAs long as recovery completed, the databases are then opened:\n\n    Opening databases...\n    \u003e\u003e Opening database HRU\n    \u003e\u003e Database HRU is open\n    \u003e\u003e Opening database BIU\n    \u003e\u003e Database BIU is open\n    \u003e\u003e Opening database CRMU\n    \u003e\u003e Database CRMU is open\n    \u003e\u003e Opening database DEVU\n    \u003e\u003e Database DEVU is open\n    \u003e\u003e Opening database DWHU\n    \u003e\u003e Database DWHU is open\n    \u003e\u003e Opening database ERPU\n    \u003e\u003e Database ERPU is open\n    \u003e\u003e Opening database FCSTU\n    \u003e\u003e Database FCSTU is open\n    \u003e\u003e Opening database SUPPLYU\n    \u003e\u003e Database SUPPLYU is open\n    \u003e\u003e Opening database TSTU\n    \u003e\u003e Database TSTU is open\n    \u003e\u003e Opening database UATU\n    \u003e\u003e Database UATU is open\n\nA final summary of confirmed failovers is then printed.\n\n    Results:\n    Database HRU failover complete\n    Database BIU failover complete\n    Database CRMU failover complete\n    Database DEVU failover complete\n    Database DWHU failover complete\n    Database ERPU failover complete\n    Database FCSTU failover complete\n    Database SUPPLYU failover complete\n    Database TSTU failover complete\n    Database UATU failover complete\n\nThere are many options for how to complete this procedure. If you can think of something you'd like to see demonstrated, let me know, but the scripts themselves are intended to be modified as required and should be reasonably self-explanatory if you understand the logical workflow being performed.\n    \n# NTAPlib module debug settings\n\nThe following list shows the debug settings when using the NTAPlib modules. \n\n    0000 0001 | Show basic workflow information\n    0000 0010 | Show REST output performed by doREST.py\n    0000 0100 | Show extra workflow steps beyond just the basics\n    0000 1000 | Show exec() calls including stdin/stdout performed by doProcess.py\n    0001 0000 | Show sqlplus related information, mostly within doSqlplus.py and doRMAN.py\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetapp%2Fsnapomatic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetapp%2Fsnapomatic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetapp%2Fsnapomatic/lists"}