{"id":13343268,"url":"https://github.com/antoniogbn/tprog","last_synced_at":"2025-03-12T04:33:02.964Z","repository":{"id":42650969,"uuid":"241209634","full_name":"antoniogbn/tprog","owner":"antoniogbn","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-08T07:04:28.000Z","size":47,"stargazers_count":1,"open_issues_count":6,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-07-30T21:06:47.185Z","etag":null,"topics":["network-automation","network-programming","python","telecommunications","yaml","yang","yang-model"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antoniogbn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-02-17T21:16:25.000Z","updated_at":"2020-04-29T01:51:22.000Z","dependencies_parsed_at":"2022-09-14T16:01:36.870Z","dependency_job_id":null,"html_url":"https://github.com/antoniogbn/tprog","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniogbn%2Ftprog","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniogbn%2Ftprog/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniogbn%2Ftprog/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antoniogbn%2Ftprog/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antoniogbn","download_url":"https://codeload.github.com/antoniogbn/tprog/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221270042,"owners_count":16788788,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["network-automation","network-programming","python","telecommunications","yaml","yang","yang-model"],"created_at":"2024-07-29T19:31:07.691Z","updated_at":"2024-10-24T03:30:34.155Z","avatar_url":"https://github.com/antoniogbn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# TPROG \r\n\r\nTPROG is a Python scripting based solution for automation of telecommunications processes on heterogenous enviroments. \r\nIt uses equipment´s interfaces to interact with them in order to acomplish configuration tasks, the currently version supports the follow integration technologies :\r\n\r\n* [REST](https://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm)\r\n* [SOAP](https://www.w3.org/TR/soap/) \r\n* [SSH](https://tools.ietf.org/html/rfc4253)\r\n\r\n\r\n## 1. Requirements :\r\n\r\n### 1.1 Python :\r\n```\r\nPython 3.6.8 or higher\r\n```\r\n\r\n### 1.2 Libraries :\r\n```\r\nbcrypt==3.1.7\r\nbitarray==1.0.1\r\ncertifi==2019.9.11\r\ncffi==1.13.2\r\nchardet==3.0.4\r\ncryptography==2.8\r\nenum34==1.1.6\r\nfuture==0.18.2\r\nidna==2.8\r\nlxml==4.4.1\r\nnetmiko==2.4.2\r\nparamiko==2.6.0\r\npyang==2.1\r\npyangbind==0.8.1\r\npybind11==2.4.3\r\npycparser==2.19\r\nPyNaCl==1.3.0\r\npyserial==3.4\r\nPyYAML==5.1.2\r\nregex==2019.8.19\r\nrequests==2.22.0\r\nruamel.yaml==0.16.5\r\nruamel.yaml.clib==0.2.0\r\nscp==0.13.2\r\nsiesta==0.5.1\r\nsimplejson==3.16.0\r\nsix==1.12.0\r\ntextfsm==1.1.0\r\nurllib3==1.25.6\r\nxmltodict==0.12.0\r\n```\r\n\r\n## 2. Install\r\n\r\n### 2.1. **Clone the project to a folder on your system :**\r\n```\r\ngit clone https://github.com/antoniogbn/tprog.git\r\n```\r\n\r\n### 2.2. **Install required libraries :**\r\n```\r\npip install -r requirements.txt\r\n```\r\n\r\n## 3. Howto\r\n\r\nThe steps below will guide how to use the TPROG \r\n\r\n### 3.1. **Create tha YANG model file :**\r\n\r\nThe YANG model must be based on equipment API format honouring the data hiehierarchy, once that´s created the [Pyang](https://github.com/mbj4668/pyang) is used to convert the model into python objects that will be later used on TPROG workflow.\r\n \r\nThe data modeled on the YANG file will refer strictly to that  data that will be applied on equipment using the follow YANG structure as reference :\r\n\r\n```\r\nmodule [modulename] {\r\n\r\n    yang-version \"1\";\r\n\r\n    namespace \"https://[yournamespaceurl]/\";\r\n\r\n    prefix \"[]\";\r\n\r\n    typedef data {\r\n        type string;\r\n    }\r\n    \r\n    container [containername]___{\r\n        leaf [fieldname1]___ {\r\n            type data;\r\n        }\r\n        leaf [fieldname2]___ {\r\n            type data;\r\n        }\r\n        .\r\n        .\r\n        .\r\n        leaf [fieldname3]___ {\r\n            type data;\r\n        }\r\n\r\n    }\r\n```\r\n\r\n**Notes** : *- The module information inside of YANG model must be the same as the YANG file name also* \r\n           \r\n\r\n### 3.2. **Generate the python object from YANG model :**\r\n\r\nOnce the YANG file is created,  command line below should be used to generate the python object class that later will ve used on TPROG workflow :\r\n\r\n```\r\npython tprog.cyang.py [yangfilename]\r\n```\r\n\r\n\r\n\r\n### 3.3. **Prepare the YAML file with tasks data :**\r\n\r\nThe YAML file should have the tasks that will be performed when the TPROG is executed, it can contain one more multiple tasks entries that must follow the YAML format rules. The YAML file is read using library [Ruamel](https://pypi.org/project/ruamel.yaml/)\r\n\r\nThe service task file should have the below presented structure :\r\n\r\n```\r\ntask1:\r\n    type : \r\n    action : \r\n    csv : \r\n    uri : \r\n    username : \r\n    password : \r\n    version : \r\n    data :\r\n    .\r\n    .\r\n    .\r\n\r\ntask2:\r\n    type : \r\n    action : \r\n    csv : \r\n    uri : \r\n    username : \r\n    password : \r\n    version : \r\n    data :\r\n    .\r\n    .\r\n    .\r\n```\r\n\r\nIn the table below is listed the tags that are supported on YAML file also their description and mandatory rule accordinly with access technologies. \r\n\r\n\r\n| Tag        | Description                                                                       | REST | SOAP | SSH |\r\n|------------| ----------------------------------------------------------------------------------|------|------|-----|\r\n|task        | Unique task idenfier                                                              |  Y   |  Y   |  Y  | \r\n|type        | Equipment access method type. currently supported  : restjson, soapaxl e ioscli   |  Y   |  Y   |  Y  | \r\n|action      | Action to be called during the process, must match with object class (YANG Model) |  Y   |  Y   |  Y  | \r\n|uri         | Logical address for webservices protocols                                         |  Y   |  Y   |  N  |   \r\n|ip_address  | Logical address for IP protocols                                                  |  N   |  N   |  Y  | \r\n|version     | Equipment version                                                                 |  N   |  Y   |  N  |                                     \r\n|device_type | Equipment type                                                                    |  N   |  N   |  Y  |                                        \r\n|username    | Username access                                                                   |  N   |  Y   |  Y  |                                            \r\n|password    | Password access                                                                   |  N   |  Y   |  Y  |                                               \r\n|csv         | CSV sub-file for batch execution                                                  |  N   |  N   |  N  |          \r\n|data        | Data that will submited in the equipment                                          |  Y   |  Y   |  Y  |                            \r\n|return      | Task return variable                                                              |  N   |  N   |  N  | \r\n\r\n\r\n### 3.4. **Run TPROG in order to get tasks completed :**\r\n\r\n\r\n```\r\npython tprog.py tasks.yaml\r\n```\r\n\r\n## 4. Examples :\r\n\r\n### 4.1. **Using TPROG with REST API :**\r\n\r\nThis example shows how to use TPROG to add customer and accounts information using REST API in a [PortaOne Softswitch](https://www.portaone.com/products/portaswitch) server.\r\n\r\nThe both YANG files were created to serve as model to both API methods that will be called based on service API sctructure :\r\n\r\n- [add_customer](https://github.com/antoniogbn/tprog/blob/master/mdl/add_customer.yang)\r\n- [add_accounts](https://github.com/antoniogbn/tprog/blob/master/mdl/add_account.yang)\r\n\r\nThen the object classes were created from their respectively YANG models using the follow commands:\r\n\r\n```\r\n(tprog) user@localhost:/dev/tprog$ python tprog.cyang.py add_customer\r\npyang -f tree mdl/add_customer.yang\r\nmodule: add_customer\r\n  +--rw auth_info___\r\n  |  +--rw login___?      data\r\n  |  +--rw password___?   data\r\n  +--rw params___\r\n     +--rw customer_info___\r\n        +--rw name___?                data\r\n        +--rw iso_4217___?            data\r\n        +--rw i_traffic_profile___?   data\r\n        +--rw email___?               data\r\npyang --plugindir $PYBINDPLUGIN -f pybind mdl/add_customer.yang \u003e obj/add_customer.py\r\n\r\n(tprog) user@localhost:/dev/tprog$ python tprog.cyang.py add_customer\r\npyang -f tree mdl/add_customer.yang\r\nmodule: add_customer\r\n  +--rw auth_info___\r\n  |  +--rw login___?      data\r\n  |  +--rw password___?   data\r\n  +--rw params___\r\n     +--rw customer_info___\r\n        +--rw name___?                data\r\n        +--rw iso_4217___?            data\r\n        +--rw i_traffic_profile___?   data\r\n        +--rw email___?               data\r\npyang --plugindir $PYBINDPLUGIN -f pybind mdl/add_customer.yang \u003e obj/add_customer.py\r\n\r\n```\r\n\r\n**note :**  *For the TPROG, the YANG files must use the presented format including \"___\" after the container and leaf names.*\r\n\r\nThe object classes files were generated inside **obj** folder, then once that is completed, the next step will be to create the task YAML file, the example below shows the file with 2 tasks :\r\n\r\n```\r\n---\r\ntask1:\r\n    type : restjson\r\n    action : add_customer\r\n    uri : https://apiserver/rest/Customer/add_customer\r\n    data :\r\n        auth_info :\r\n            login : \"UsernameDemo\"\r\n            password : \"PasswordDemo\" \r\n        params :\r\n            customer_info:\r\n                name : \"Telecom International\"\r\n                iso_4217 : \"USD\"\r\n    return : i_customer\r\n  \r\ntask2:\r\n    type : restjson\r\n    action : add_account\r\n    uri : https://apiserver/rest/Account/add_account\r\n    data :\r\n        auth_info :\r\n            login : \"UsernameDemo\"\r\n            password : \"PasswordDemo\" \r\n        params :\r\n            account_info:\r\n                id: 552137525665\r\n                billing_model: 1\r\n                i_product: 1676                 \r\n                i_customer: $i_customer         \r\n                h323_password: 123456           \r\n                batch_name: telecomint_accounts \r\n```\r\n\r\nThe task1 will call the method **add_customer** to add customer information on the server and the task2 will call the method **add_account** to add the accounts posteriorly . The entry **return** present on the task1 refers to the variable returned from method called with customer key that was generated during the process and will avaiable on the task2 be used adding accounts for that recently added customer.\r\n\r\nOnce YAML is complete call tprog main script to execute the tasks file with below line :\r\n\r\n```\r\npython tprog.py demo1.yaml\r\n```\r\n\r\nThe obteined output should be like presented below :\r\n\r\n```\r\nSENT \u003e\u003e\u003e\r\n{'auth_info': '{\"login\": \"UsernameDemo\", \"password\": \"PasswordDemo\"}', 'params': '{\"customer_info\": {\"name\": \"Telecom International\", \"iso_4217\": \"USD\", \"i_traffic_profile\": \"\", \"email\": \"\"}}'}\r\n\r\n\r\nRECEIVED \u003e\u003e\u003e\r\n{'i_customer': 6736}\r\n\r\n\r\nSENT \u003e\u003e\u003e\r\n{'auth_info': '{\"login\": \"UsernameDemo\", \"password\": \"PasswordDemo\"}', 'params': '{\"account_info\": {\"id\": \"552137525665\", \"billing_model\": \"1\", \"i_product\": \"1676\", \"i_customer\": \"6736\", \"h323_password\": \"abc12356\", \"batch_name\": \"telecomint_accounts\"}}'}\r\n\r\n\r\nRECEIVED \u003e\u003e\u003e\r\n{'i_account': 570170}\r\n\r\n--- 3.52347 seconds ---\r\n```\r\n\r\n\r\n### 4.2. **Using TPROG with SOAP API :**\r\n\r\nThis example shows how to use TPROG to add Phones and Numbers  information using SOAP AXL API in a [Cisco CUCM](https://www.cisco.com/c/en/us/products/unified-communications/unified-communications-manager-callmanager/index.html) server.\r\n\r\nFollowing the steps, on the first one the YANG file was created to serve as model to both API methods that will be called based on service API sctructure :\r\n\r\n- [add_customer](https://github.com/antoniogbn/tprog/blob/master/mdl/addPhone.yang)\r\n\r\nThen the object classes were created from their respectively YANG model using the follow commands:\r\n\r\n```\r\n(tprog) user@localhost:/dev/tprog$ python tprog.cyang.py addPhone\r\npyang -f tree mdl/addPhone.yang\r\nmodule: addPhone\r\n  +--rw phone___\r\n     +--rw name___?             data\r\n     +--rw description___?      data\r\n     +--rw product___?          data\r\n     +--rw class___?            data\r\n     +--rw protocol___?         data\r\n     +--rw devicePoolName___?   data\r\n     +--rw lines___\r\n        +--rw line___\r\n           +--rw index___?   data\r\n           +--rw dirn___\r\n              +--rw pattern___?              data\r\n              +--rw routePartitionName___?   data\r\n```\r\n\r\n**note :**  *For the TPROG, the YANG files must use the presented format including \"___\" after the container and leaf names.*\r\n\r\nThe object classes files were generated inside **obj** folder, then once that is completed, the next step will be to create the task YAML file, the example below shows the file with 2 tasks :\r\n\r\n```\r\n---\r\ntask1:\r\n    type : soapaxl\r\n    action : addPhone\r\n    uri : 'https://apiserver:8443/axl/'\r\n    username : 'UsernameDemo'\r\n    password : 'PasswordDemo'\r\n    version : '8.5'\r\n    data :\r\n        phone : \r\n          name : \r\n          description : \"Automation Demo Device\"\r\n          product : \"Cisco 7945\"\r\n          class : \"Phone\"\r\n          protocol : \"SCCP\"\r\n          devicePoolName : \"Default\"\r\n          lines:\r\n            line:\r\n                index: '1'\r\n                dirn: \r\n                    pattern: 552137525665\r\n                    routePartitionName: \"Default_Partition\"            \r\n\r\n```\r\n\r\nThe task1 will call the server SOAP method **addPhone** to add the Phone device and it´s others parameters  like description, phone model, line number, etc.\r\n\r\nOnce YAML is complete call tprog main script to execute the tasks file with below line :\r\n\r\n\r\n\r\n\r\n\r\n### 4.3. **Using TPROG with SSH :**\r\n\r\nThis example shows how to use TPROG to add dial-peer configuration using SSH command line instructions to a [Cisco Voice Gateway](https://www.cisco.com/c/en/us/td/docs/voice_ip_comm/cucme/srnd/design/guide/cmesrnd/gatewy.html) .\r\n\r\nFollowing the steps, on the first one the YANG file was created to serve as model to both API methods that will be called based on service API sctructure :\r\n\r\n- [cisco_vg](https://github.com/antoniogbn/tprog/blob/master/mdl/cisco_vg.yang)\r\n\r\nIn the next step the object classes were created from their respectively YANG models using the follow commands:\r\n\r\n```\r\n(tprog) user@localhost:/dev/tprog$ python tprog.cyang.py cisco_vg\r\npyang -f tree mdl/cisco_vg.yang\r\nmodule: cisco_vg\r\n  +--rw cmd_list___?   data\r\npyang --plugindir $PYBINDPLUGIN -f pybind mdl/cisco_vg.yang \u003e obj/cisco_vg.py\r\n```\r\n**note :**  *For the TPROG, the YANG files must use the presented format including \"___\" after the container and leaf names.*\r\n\r\nThe object classes files was generated inside **obj** folder, the next step will be to create the task YAML file, the example below shows the file with 2 tasks :\r\n\r\n\r\n\r\n\r\n## 5. About :\r\n\r\nThe TPROG tool is the result of a MSc. study currently in course with [Universidade Federal Fluminense](http://mesc.sites.uff.br/)\r\n\r\nAuthors :\r\n\r\nAntonio Garcia Brandão Neto - antoniogbn@id.uff.br (mastering)\r\n\r\nCarlos Bazilio - carlosbazilio@id.uff.br (advisor)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoniogbn%2Ftprog","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantoniogbn%2Ftprog","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantoniogbn%2Ftprog/lists"}