{"id":17676711,"url":"https://github.com/tuladhar/python-for-sysadmin","last_synced_at":"2025-05-16T14:04:43.183Z","repository":{"id":45728209,"uuid":"69093347","full_name":"tuladhar/Python-for-SysAdmin","owner":"tuladhar","description":"Python for Systems Administrator","archived":false,"fork":false,"pushed_at":"2024-11-12T18:18:34.000Z","size":83,"stargazers_count":218,"open_issues_count":2,"forks_count":87,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-04-12T10:58:26.386Z","etag":null,"topics":["hacktoberfest","hacktoberfest-accepted","python","system-administration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"unlicense","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tuladhar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-24T09:36:17.000Z","updated_at":"2025-04-08T21:03:43.000Z","dependencies_parsed_at":"2024-10-24T09:35:00.381Z","dependency_job_id":"e5e28228-b0bd-4edc-85a4-5d117e14a2f2","html_url":"https://github.com/tuladhar/Python-for-SysAdmin","commit_stats":{"total_commits":90,"total_committers":9,"mean_commits":10.0,"dds":0.4,"last_synced_commit":"3edcd9b2054bcba90526716db8e70c6fbf7e34a1"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuladhar%2FPython-for-SysAdmin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuladhar%2FPython-for-SysAdmin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuladhar%2FPython-for-SysAdmin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tuladhar%2FPython-for-SysAdmin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tuladhar","download_url":"https://codeload.github.com/tuladhar/Python-for-SysAdmin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254544146,"owners_count":22088807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","hacktoberfest-accepted","python","system-administration"],"created_at":"2024-10-24T07:26:29.658Z","updated_at":"2025-05-16T14:04:43.158Z","avatar_url":"https://github.com/tuladhar.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Python3 for Systems Administrator\n[![Generic badge](https://img.shields.io/badge/Author-Puru_Tuladhar-\u003cCOLOR\u003e.svg)](https://github.com/tuladhar)\n[![Generic badge](https://img.shields.io/badge/Editor_and_Maintainer-Silent_Mobius-\u003cCOLOR\u003e.svg)](https://github.com/silent-mobius)\n\n## Contents\n- #### Getting Started\n  1. [Standard I/O](#getting-started-standard-input-output-and-error)\n  2. [Working with files](#getting-started-working-with-files)\n  3. [Command-line Arguments](#getting-started-command-line-arguments)\n- #### Exploring Standard Modules\n  1. [Operating System (OS) Module](#exploring-standard-modules-operating-system-os-module)\n  2. [System-specific Module](#exploring-standard-modules-system-specific-module)\n  3. [Shell Modules](#exploring-standard-modules-shell-modules)\n  4. [Date and Time Modules](#exploring-standard-modules-date-and-time-modules)\n  5. [Subprocess Module](#exploring-standard-modules-subprocess-module)\n  6. [Argparse Module](#exploring-argparse-command-line-option-and-argument-parsing)\n  6. [SQLite Module](#exploring-standard-modules-embedded-relational-database-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  7. [XmlTree Module](#exploring-xmltree-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  8. [JSON Module]( #exploring-json-module-exploring-json-module)\n  8. [CSV Module]( #exploring-csv-module-exploring-csv-module)\n  9. [Regular Expressions Module](#exprloting-standard-regular-expression-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  10. [Compression Module](#exploring-compression-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  11. [Platform Module](#exploring-platform-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  12. [Signal Module](#exploring-standard-modules-signal-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  13. [Socket Module](#exploring-standard-modules-socket-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n\n- #### Exploring External Modules\n  -  #### Network Modules\n  1. [Netmiko Module](#exploring-standard-modules-netmiko-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  2. [Twisted Module](#exploring-standard-modules-twisted-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  3. [Ipaddress Module](#exploring-standard-modules-ipaddress-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  4. [Netsnmp Module](#exploring-standard-modules-netsnmp-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n  - #### DB Modules\n  1. [SQLAlchemy Module](#exploring-standard-modules-sqlalchemy-module) ![Generic badge](https://img.shields.io/badge/Coming%20-Soon-blue)\n\n---\n\n## `Getting Started` Standard Input, Output and Error\n\nStandard input, output and error (commonly referred to as `stdin`, `stdout` and `stderr`) are what's called pipes.\n\nThese pipes are normally connected to the terminal window where you are working.\n\nWhen printing something (using `print()`), it goes to the `stdout` pipe by default; when your program needs to print errors (like a traceback in Python), it goes to the `stderr` pipe; and when your program requires input from a user or other programs, it goes to the `stdin` pipe.\n\n```bash\n$ cat /etc/passwd | grep /bin/bash\n#  `\\_ writes to stdout        `\\_ reads from stdin\n\n$ ls no_such_file.txt\n#  `\\_ writes to stderr\n```\n\nPython built-in `sys` module provides `stdin`, `stdout` and `stderr` as a file-objects, exposing `read()`, `readlines()`, `write()`, `writelines()` methods, as shown below:\n\n```python\n#!/usr/bin/env python3\nimport sys\n\n# writing to stdout pipe\nprint ('Hello, world')\nsys.stdout.write('Hello, world\\n')\n\n# writing to stderr pipe\nprint(\"Error occured\")\nsys.stderr.write(\"Error occured\\n\")\n\n# reading single line from stdin pipe\nsys.stdout.write('Username: ')\nusername  = sys.stdin.readline()\nprint (username)\n\n# reading 'n' number of characters from stdin\nprint ('Enter few characters and press Ctrl-D')\ndata = sys.stdin.read(1024)\nprint('len(data):', len(data))\n\n# reading lines (separated by newlines) from stdin\nprint('Enter few lines and press Ctrl-D')\nlines = sys.stdin.readlines()\nprint ('total lines read:', len(lines))\n```\n\n---\n\n## `Getting Started` Working with files\n\nPython built-in `open` function is the standard interface for working with files. `open` returns a file object with familiar methods for performing read/write operations.\n\n```\nopen(file, mode='r')\n```\n\n###### Common arguments:\n\n- __file:__ absolute or relative path of the file. e.g: `/etc/passwd`, `./ip-list.txt`\n- __mode:__ an optional string specifying mode in which the file is opened. Default's to `r` for reading in text mode. Common values are:\n - `w` write only *(truncates the file already exists)*\n - `x` create and write to a new file _(if file already exists then open fails)_\n - `a` append to a file *(insert data to end of the file)*\n - `+` updating a file (reading and writing). e.g: `r+` `w+`\n - `b` binary mode for non-text files. Contents are returned as `byte` objects.\n - `t` text mode (default). Contents are returned as `strings`.\n\n###### File object methods:\n\n- `.close()`\n- `.read(size=-1)` read at most `n` characters. If `n` is negative or ommited, reads the whole file until `EOF`.\n- `.readline()` Read single line until newline or `EOF`.\n- `.readlines()` Read all newlines and returns as a list.\n- `.write(data)` Write data to file. Returns number of characters written.\n- `.writelines(list)` Write all data in the list. Useful for writing multiple lines to a file.\n\n##### Example\n\n```python\n## Reading a file\nr = open('/etc/passwd')\n# same as:\n# r = open('/etc/passwd', 'r')\npasswd = r.readlines()\nfor line in passwd:\n\tline = line.strip()\n\tsize = len(line)\n\tprint(size, line)\nr.close()\n\n## Reading a file using `with` statement, automatically closes the file\nwith open('/etc/passwd') as f:\n\tpass\n\n## Writing to a file\nw = open('test.txt', 'w')\nw.write('127.0.0.1\\n')\nw.flush()\nlines = ['1.1.1.1\\n', '2.2.2.2\\n', '3.3.3.3\\n']\nw.writelines(lines)\nw.close()\n```\n---\n### Handling Errors\n\nAll `open` related errors raises `IOError` exception.\n\n```python\ntry:\n\tf = open('/etc/passwd')\nexcept IOError as e:\n\tprint ('Opps, something went wrong.')\n\tprint (e)\n```\n---\n\n#### Reading `/etc/passwd` and search each line for `/bin/bash` shell; when found print the line number and the line\n\n```python\n#!/usr/bin/env python3\n# working-with-files-01.py\n\ndef main():\n\ttry:\n\t\twith open('/etc/passwd') as f:\n\t\t\tfor no, line in enumerate(f, 1):\n\t\t\t\tif '/bin/bash' in line:\n\t\t\t\t\tline = line.strip()\n\t\t\t\t\tprint(f'{no} {line}')\n\texcept IOError as e:\n\t\tprint(e)\n\nif __name__ == '__main__': main()\n```\n\n---\n\n#### Using `fileinput` High-level module\n\n`fileinput` module allows to quickly write a loop over standard input or a list of files.\n\n##### Example\n\n```python\n#!/usr/bin/env python3\n# linescount-02.py\n\nimport fileinput\n\nlines = 0\nfor line in fileinput.input():\n\tlines += 1\n\nprint('totat lines:', lines)\n```\n\n```bash\n$ cat /etc/passwd | python3 linescount.py\ntotat lines: 86\n$ python3 linescount.py /etc/services\ntotat lines: 13921\n$ python3 linescount.py /etc/services /etc/passwd /etc/hosts\ntotat lines: 14023\n```\n\nBy default, `fileinput.input()` will read all lines from files given as an argument to the script; if no arguments given then defaults to standard input.\n\n---\n\n## `Getting Started` Command-line Arguments\n\n`sys` module provides `argv` variable containing the list of arguments passed to the script when executed as a command-line application.\n\n\u003eNOTE: The first argument `sys.argv[0]` is always the name of the script itself.\n\n```python\n#!/usr/bin/env python3\n# argv_01.py\n\nimport sys\n\nprint ('sys.argv:', sys.argv)\nprint ('length:', len(argv))\n```\n```bash\n$ python3 args_01.py --help\n['argv-01.py', '--help']\n2\n$ python3 args_01.py 1 2 3\n['argv-01.py', '1', '2', '3']\n4\n```\n\n#### Accept specific number of arguments and fail if not satistified.\n\n```python\n#!/usr/bin/env python3\n# args-02.py\nimport sys\n\n# NOTE: [1:] excludes the first item at index 0, i.e script name\nargv_len = len(sys.argv[1:])\n\nif not argv_len == 2:\n\tsys.exit(f'invalid number of arguments (expected 2, given: {argv_len})')\n\nprint('two arguments are:', sys.argv[1:])\n```\n\n```bash\n$ python args_02.py\ninvalid number of arguments (expected 2, given: 0)\n$ python args_02.py 1\ninvalid number of arguments (expected 2, given: 1)\n$ python args_02.py 1 2  \ntwo arguments are: ['1', '2']\n$ python args_02.py 1 2 3\ninvalid number of arguments (expected 2, given: 3)\n```\n\n#### Simple Implementation of `grep' command\n\n```python\n#!/usr/bin/env python3\n# grep.py\n\nimport sys\n\nscript = sys.argv[0]\n\ndef print_usage():\n\tsys.exit(f'Usage: python {script} pattern')\n\ndef main(argv):\n\tif not len(argv) == 1:\n\t\tprint_usage()\n\n\tpattern = argv[0]\n\n\tfor line in sys.stdin:\n\t\tif pattern in line:\n\t\t\tprint(line.strip())\n\nif __name__ == '__main__':\n\tmain(sys.argv[1:])\n```\n\n```bash\n$ cat /etc/services | python3 grep.py 8080\nhttp-alt\t8080/udp     # HTTP Alternate (see port 80)\nhttp-alt\t8080/tcp     # HTTP Alternate (see port 80)\n```\n\n---\n\n#### Improvement Exercises\n- Extend `grep.py` to read from a file instead of standard input, and can be run as:\n\n```bash\n$ python3 grep.py 8080 /etc/services\n```\n\n- Extend `grep.py` to read from a file if given as a second argument or default back to read from standard input similar to what `grep` does.\n\n```bash\n$ cat /etc/services | python3 grep.py 8080\n# and also should works as\n$ python3 grep.py 8080 /etc/services\n```\n\n---\n\n## `Exploring Standard Modules` Operating System (OS) Module\n\nPython built-in `os` module exposes operating system dependent functionality in a portable way, thus works across many different platforms.\n\n```\nimport os\n```\n\n###### Working with environmental variables:\n\n- `os.environ` a dictionary mapping system environment variables. e.g. `os.environ.get('SHELL')`\n- `os.getenv(key)` retrieve a specific variable.\n- `os.putenv(key, value)`. change variable to specific value. e.g. `os.putenv('HOME', '/opt')`\n\n###### Working with files and directories:\n\n- `os.tmpnam` returns a unique file name that can be used to create temporariry file. `mktemp()` from `tempfile` module is more secure.\n- `os.mkdir(path)` `os.rmdir(path)` `os.chdir(path)` `os.listdir(path)`\n- `os.getcwd()` returns current working directory as string. Similar to `pwd` command.\n- `os.stat(path)` similar to `stat` command\n- `os.walk(path)` recursively walk the dictionary tree, similar to `ls -R`. Returns generator yielding 3-tuple `(dirpath, dirnames, files)`\n\n##### Example\n\n```bash\n\u003e\u003e\u003e import os\n\n# environment variables\n\u003e\u003e\u003e os.environ['SHELL'] # or os.environ.get('SHELL') or os.getenv('SHELL')\n'bash'\n\n# Get a temporary filename using `tempfile` module\n\u003e\u003e\u003e from tempfile import mktemp\n\u003e\u003e\u003e temp = mktemp()\n'/var/folders/6s/4xn2fv852_1ghssypng0wwfw0000gn/T/tmp15O9aR'\n\n# Create the temporary file and write 'test'\n\u003e\u003e\u003e with open(temp, 'w') as f:\n... \tf.write('test\\n')\n\n# Get the file stat, such as creation timestamp (ctime)\n\u003e\u003e\u003e st = os.stat(temp)\n\u003e\u003e\u003e st.st_ctime\n1436466688.0\n\n# convert timestamp to human readable format, we'll cover date \u0026 time later.\n\u003e\u003e\u003e import datetime\n\u003e\u003e\u003e d = datetime.datetime.fromtimestamp(st.st_ctime)\n\u003e\u003e\u003e d.year # d.month, d.hour and so on...\n2016\n```\n--\n###### Working with Path names  `os.path`:\n\n- `os.path.abspath(path)` get absolute path.\n- `os.path.basename(path)` get the filename excluding the directory part. Opposite of  `os.path.dirname` which excludes filename part and returns directory path.\n- `os.path.exists(path)` returns `True` if path exists else `False`.\n- `os.path.getsize(path)` return size (in bytes) for a given path.\n- `os.path.isfile(path)`, `os.path.isdir(path)` returns `True` or `False` if path is file or a directory.\n- `os.path.getctime(path)`,  `os.path.getmtime(path)` Similar to `os.stat(path).ctime`, `os.stat(path).mtime()`\n\n#### Learn more at [os.path documentation](https://docs.python.org/3.7/library/os.path.html)\n\n##### Example\n\n```bash\n\u003e\u003e\u003e os.path.isdir('/etc')\nTrue\n\n\u003e\u003e\u003e os.chdir('/etc')\n\n\u003e\u003e\u003e os.getcwd()\n'/etc'\n\n\u003e\u003e\u003e os.path.abspath('hosts')\n'/etc/hosts'\n\n\u003e\u003e\u003e os.path.isfile('/etc/hosts')\nTrue\n\n\u003e\u003e\u003e os.path.getsize('/etc/hosts')\n475\n\n\u003e\u003e\u003e print (os.path.basename('/etc/passwd'))\n'passwd'\n\n\u003e\u003e\u003e print (os.path.dirname('/etc/passwd'))\n'/etc'\n```\n\n\n###### Process related functions:\n\n- `os.getpid()` get id of current process and `os.getppid()`to get parent process id.\n- `os.system(command)` execute a command and returns exit code.\n\n###### Getting System Informations:\n\n- `os.uname()` returns a 5-tuple `(sysname, nodename, release, version, machine)` identifying the current OS. Similar to `uname -a` command.\n- `os.getloadavg()` returns 3-tuple containing 1-min, 5-min and 15-min load average.\n\n--\n\n#### Print size (in bytes) of all files in a given directory\n\n```python\n#!/usr/bin/env python3\n# filesizes.py\n\nimport os\nimport sys\n\nscript = sys.argv[0]\n\ndef print_usage():\n\tprint(f' \u003e\u003e {sys.stderr}, \"Usage: python3 {script} DIR\"')\n\tsys.exit(1)\n\ndef filesizes(path):\n\t''' calculate and print size of each file in a given directory. '''\n\n\tfor dirpath, dirnames, filenames in os.walk(path):\n\t\tfor filename in filenames:\n\t\t\tfilepath = os.path.join(dirpath, filename)\n\t\t\t_bytes = os.path.getsize(filepath)\n\t\t\tprint (f'{_bytes}, {filepath}')\n\ndef main(argv):\n\tif not len(argv) == 1:\n\t\tprint_usage()\n\n\tpath = argv[0]\n\tfilesizes(path)\n\nif __name__ == '__main__':\n\tmain(sys.argv[1:])\n\n```\n\n\u003e```bash\n\t$ python3 filesizes.py .\n\t678 ./filesizes.py\n\t```\n\n#### [Learn more about OS module](https://docs.python.org/3.7/library/os.html)\n\n---\n## `Exploring Standard Modules` System-specific Module\n\nPython built-in `sys` module exposes system-specific parameters and functions. This module provides access to some variables used or maintained by the interpreter and functions that interact strongly with the interpreter. `sys` module is always available.\n\n```\nimport sys\n```\n\n\n\n###### Commonly used variables and functions:\n\n- `sys.argv` contains list of command-line arguments. *Already covered.*\n- `sys.exit([arg])` exit the program. `arg` is optional, can be `0` to indicate success, `\u003e 0` for failure or `string` to display the errors during exit.\n- `sys.version` stores version of Python installed.\n- `sys.stdin` `sys.stdout` `sys.stderr` File objects corresponding to the interpreter's standard input, output and error pipes.\n- `sys.platform` stored the name of the platofrm Python is running. Such as `darwin` for MacOS, `linux` for Linux and others\n- `sys.path` A list of strings that specified the search path while importing modules via `import` statement.\n- `sys.modules` A dictionary mapping the `{ 'module_name': 'module_path', ... }` which have already been loaded.\n\n---\n\n```python\n#!/usr/bin/env python3\n# sys-01.py\n\nimport sys\nfrom sys import argv\n\nprint (f'''\nPython version installed: {sys.version}\n\nPython running on platforn: {sys.platform}\n''')\n\nif len(argv) == 10:\n\tsys.exit('error: too many arguments')\n\nprint(f'''\nargv = {argv}\nlen(argv) = {len(argv)}\n''')\n\nprint ('Printing to stdout')\nprint (f'\u003e\u003e {sys.stderr}, \"Printing to stderr\"')\n\nprint(f'''\ntotal modules search path: {len(sys.path)}\ntotal modules loaded: {len(sys.modules)}\n''')\n\n# success\nsys.exit(0)\n```\n\n###### Output should look like this\n\n```bash\n$ python3 sys-01.py\n\nPython version installed: 3.7.4 (default, Jul  9 2019, 16:48:28)\n[GCC 8.3.1 20190223 (Red Hat 8.3.1-2)]\n\nPython running on platforn: linux\n\n\nargv = ['sys_01.py']\nlen(argv) = 1\n\nPrinting to stdout\n\u003e\u003e \u003c_io.TextIOWrapper name='\u003cstderr\u003e' mode='w' encoding='UTF-8'\u003e, \"Printing to stderr\"\n\ntotal modules search path: 7\ntotal modules loaded: 57\n\n\n$ echo $?\n0\n```\n\n#### [Learn more about sys module](https://docs.python.org/3.7/library/sys.html)\n\n---\n\n## `Exploring Standard Modules` Shell Modules\n\n### shutil module — High-level file operations\n\nThe `shutil` module provides high-level operations on files or collection of files.\n\n###### Copy Operations:\n\n- `shutil.copy(src, dst)` Copy src file to dst file or directory.\n- `shutil.copymode(src, dst)` Copy permission mode of src  file to dst file.\n- `shutil.copystat(src, dst)` Copy permission mode, modification and creation date from src file to dst file.\n- `shutil.copy2(src, dst)` Similar to `cp -p` command or equivalent of `shutil.copy` and `shutil.copystat`.\n\n###### Move Operations:\n\n- `shutil.move(src, dst)` Move src file to dst file or directory.\n\n#### [Learn more about shutil module](https://docs.python.org/3.7/library/shutil.html)\n\n---\n\n### glob module — Unix style pathname pattern expansion\n\nThe `glob` module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell.\n\n- `glob.glob(pattern)` Return a list of path names that match the path pattern. Path can be absolute `/usr/local/bin/*.py` or relative `local/bin/*.py`.\n\n##### Example\n\n```bash\n\u003e\u003e\u003e import glob\n\u003e\u003e\u003e glob.glob('/etc/sysconfig/network-scripts/ifcfg*')\n['/etc/sysconfig/network-scripts/ifcfg-venet0',\n '/etc/sysconfig/network-scripts/ifcfg-lo']\n\n\u003e\u003e\u003e glob.glob('/etc/*.conf')\n['/etc/vnstat.conf',\n '/etc/sudo.conf',\n '/etc/resolv.conf',\n '/etc/host.conf',\n '/etc/logrotate.conf']\n```\n\n#### [Learn more about glob module](https://docs.python.org/3.7/library/glob.html)\n\n---\n\n## `Exploring Standard Modules` Date and Time Modules\n\n### time module — Clock time\n\nThe `time` module exposes the time-related functions from the underlying C library.\n\n###### Commonly used functions:\n\n- `time.time()` returns the number of seconds since the start of the epoch as a floating-point value.\n- `time.ctime()` returns a human-friendly date and time representation.\n- `time.gmtime()` returns an object containing current time in UTC format.\n- `time.localtime()` returns an object containing the current time in current time zone.\n- `time.tzset()` sets the time zone based on `TZ` environment variable: `os.environ.get('TZ')`\n\n##### Example\n\n```bash\n\u003e\u003e\u003e import time\n\u003e\u003e\u003e time.time()\n1459924583.396017\n\n\u003e\u003e\u003e # current time in UTC\n\u003e\u003e\u003e utc = time.gmtime()\n\u003e\u003e\u003e dir(utc)\n['tm_hour',\n 'tm_isdst',\n 'tm_mday',\n 'tm_min',\n 'tm_mon',\n 'tm_sec',\n 'tm_wday',\n 'tm_yday',\n 'tm_year']\n\n\u003e\u003e\u003e # current time in GMT by updating timezone\n\u003e\u003e\u003e import os\n\u003e\u003e\u003e os.putenv('TZ', 'GMT') # or os.environ['TZ'] = 'GMT'\n\u003e\u003e\u003e time.tzset()\n\u003e\u003e\u003e gmt = '{} {}'.format(time.ctime(), time.tzname[0])\n```\n\n#### [Learn more about time module](https://docs.python.org/3.7/library/time.html)\n\n---\n\n## datetime module — Date and Time Value Manipulation\n\nThe `datetime` module includes functions and classes for doing date and time parsing, formatting, and arithmetic.\n\n##### Commonly used functions:\n\n- `datetime.date.today()` returns current date object without the time\n- `datetime.datetime.today()` returns current date and time object\n- `datetime.datetime.fromtimestamp(float)` convert unix timestamp to datetime object\n- `datetime.timedelta` future and past dates can be calculated using basic arithmetic (+, -) on two datetime objects, or by combining a datetime with a timedelta object.\n\n##### Example\n\n```python\n\u003e\u003e\u003e import datetime\n\u003e\u003e\u003e today_date = datetime.date.today()\n\u003e\u003e\u003e ten_days = datetime.timedelta(days=10)\n\u003e\u003e\u003e today_date - ten_days # past\n\u003e\u003e\u003e today_date + ten_days # future\n```\n\n###### Alternate formats can be generated using `strftime()` and `strptime` to convert formatted string to `datetime` object.\n\n```bash\n\u003e\u003e\u003e import datetime\n\n# convert datetime to custom format\n\u003e\u003e\u003e  format = \"%a %b %d %H:%M:%S %Y\"\n\u003e\u003e\u003e today = datetime.datetime.today()\n\u003e\u003e\u003e today.strftime(format)\n'Mon Oct 14 17:56:24 2019'\n\n# convert formatted string to datetime object\n\u003e\u003e\u003e datetime.datetime.strptime('Mon Oct 14 17:56:24 2019', format)\ndatetime.datetime(2019, 10, 14, 17, 56, 24)\n```\n\n#### [Learn more about datetime module](https://docs.python.org/3.7/library/datetime.html)\n\n---\n\n## `Exploring Standard Modules` Subprocess Module\n\n### subprocess module — Subprocess management\n\nThe `subprocess` module allows to run new processes, connect to their input/output/error pipes, and obtain their return codes.\n\nThe module defines a many helper functions and a class called `Popen` which allows to set up the new process so the parent can communicate with it via pipes.\n\n\n### Running external commands\n\nTo run an external command without interacting with, use `subprocess.call(command, shell=True)` function.\n\n```python\n\u003e\u003e\u003e import subprocess\n\u003e\u003e\u003e rc = subprocess.call('ls -l /etc', shell=True)\n0\n\u003e\u003e\u003e rc = subprocess.call('ls -l no_such_file.txt', shell=True)\n1\n```\n\nSetting the **shell** argument to a **True** value causes subprocess to spawn a shell process (normally bash), which then runs the command. Using a shell means that variables, glob patterns, and other special shell features in the command string are processed before the command is run.\n\n```py\n\u003e\u003e\u003e import subprocess\n\u003e\u003e\u003e subprocess.call('echo $PATH' )\n\n# will return error\nsubprocess.call('echo $PATH', shell=True)\n# will print the shell PATH variable value\n```\n\n--\n\n### Error Handling\n\nThe return value from `subprocess.call()` is the exit code of the program and is used to detect errors. The `subprocess.check_call()` function works like `call()`, except that the exit code is checked, and if it returns non-zero, then a `subprocess.CalledProcessError` exception is raised.\n\n```python\n#!/usr/bin/env python3\n# check_call.py\n\nimport sys\nimport subprocess\n\ntry:\n\tcmd = 'false'\n\tprint ('running command:',cmd)\n\tsubprocess.check_call(cmd, shell=True)\nexcept subprocess.CalledProcessError as error:\n\tsys.exit(f'error: {error}')\n```\n\n```bash\n$ python3 check_call.py\nrunning command: false\nerror: Command 'false' returned non-zero exit status 1\n```\n\n---\n\n### Capturing Command Output\n\nTo run an external command and capture it's output, use `check_output(command, shell=True)` to capture the output for later processing. If command returns non-zero, then a `CalledProcessError` exception is raised similar to `check_call`.\n\n\u003eExecute `cat /etc/hosts` and write the output to a file `hosts.txt`\n\n```python\n#!/usr/bin/env python3\n# capture_output.py\n\nimport subprocess\nimport sys\n\ntry:\n\tcmd = 'cat /etc/hosts'\n\tprint('running command:', cmd)\n\toutput = subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)\nexcept subprocess.CalledProcessError as error:\n\tsys.exit('error: {error}')\nelse:\n\tprint('success!')\n\twith open('hosts.txt', 'w') as f:\n\t\tf.write(output)\n\t\t#this should give an error in python3.6+\n```\n\nBy default, `check_output` captures outputs written to `stdout`. Setting the `stderr=subprocess.STDOUT` causes `stderr` outputs to redirected to `stdout`, so errors can be captured as well.\n\n---\n\n### Working directory with Popen\n\nThe `call()`, `check_call()`, and `check_output()` are wrappers around the `Popen` class. Using `Popen` directly gives more control over how the command is run and how its input and output streams are processed.\n\n#### One-Way Communication with a Process\n\nTo run a process and capture it's output, set the `stdout` and `stderr` arguments to `subprocess.PIPE` and call `communicate()`.\n\n`communicate()` returns 2-tuple `(stderr_output, stderr_putput)`\n\n```python\n#!/usr/bin/env python3\n# one-way.py\nimport subprocess\nimport sys\n\nscript = sys.argv[0]\n\ndef main(argv):\n\tif not len(argv) == 1:\n\t\tsys.exit(f'usage: python3 {script} command')\n\tcmd = sys.argv[1]\n\tprint ('running command:', cmd)\n\tproc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)\n\tstdout, stderr = proc.communicate()\n\tprint(f'''\nexit code: {proc.poll()}\n\nstdout:\n{stdout or None}\n\nstderr:\n{stderr or None}\n''')\n\nif __name__ == '__main__':\n\tmain(sys.argv[1:])\n```\n\n```bash\n$ python3 one_way.py ls\nrunning command: ls\n\nexit code: 0\n\nstdout:\nb'capture_output.py\\ncheck_call.py\\nhosts.txt\\none_way.py\\nsys_01.py\\n'\n\nstderr:\nNone\n\n```\n\n---\n\n### Bidirectional Communication with a Process\n\nTo set up the Popen instance for reading and writing at the same time, pass additional argument `stdin=subprocess.PIPE`.\n\n```python\n#!/usr/bin/env python3\n# bidirectional.py\nimport subprocess\nfrom subprocess import PIPE\nimport sys\n\ncmd = 'bc'\nsend = '1 + 1\\n'\n\nprint('running command:', cmd, 'and sending input:', send)\n\nproc = subprocess.Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=PIPE)\nstdout, stderr = proc.communicate(send)\n\nprint(f'''\nexit code: {proc.poll()}\n\nstdin:\n{send}\n\nstdout:\n{stdout or None}\n\nstderr:\n{stderr or None}\n''')\n```\n\n```bash\n$ python3 bidirectional.py\nrunning command: bc and sending input: 1 + 1\n\nexit code: 0\n\nstdin:\n1 + 1\n\nstdout:\n2\n\nstderr:\nNone\n```\n\n#### [Learn more about subprocess module](https://docs.python.org/3.7/library/subprocess.html?highlight=subprocess#module-subprocess)\n\n## `Exploring Argparse` Command-Line Option and Argument Parsing\n\nPython built-in `argparse` is parser for command-line options, arguments and subcommands. The argparse module provides argument management just like `sys.argv`, but with options, e.g it generates help and usage messages and issues errors when users give the program invalid arguments.\nLet’s show the sort of functionality by making use of the ls command:\n\n```bash\n$ ls\nexamples  LICENSE  README.md\n$ ls -l\ntotal 44\ndrwxrwxr-x.  4 que que  4096 Oct 14 18:05 .\ndrwxrwxr-x. 24 que que  4096 Oct 13 15:32 ..\ndrwxrwxr-x.  2 que que  4096 Oct 14 18:48 examples\ndrwxrwxr-x.  8 que que  4096 Oct 15 01:01 .git\n-rw-rw-r--.  1 que que  1210 Oct 13 15:32 LICENSE\n-rw-rw-r--.  1 que que 24357 Oct 15 01:02 README.md\n$ ls --help\nUsage: ls [OPTION]... [FILE]...\nList information about the FILEs (the current directory by default).\nSort entries alphabetically if none of -cftuvSUX nor --sort is specified.\n...\n```\n\nA few concepts we can learn from the four commands:\n\n- When you run the \"ls -l\" command with options, it will default displaying the contents of the current directory\n- The \"-l\" is knowns as an \"optional argument\"\n- If you want to display the help text of the ls command, you would type \"ls --help\"\n\nTo start using the argparse module, we first have to import it.\n\n```py\n\u003e\u003e\u003e import argparse\n```\n\n#### Intro to positional arguments\n\n The following code is a Python program that takes a list of integers and produces either the sum or the max:\n\n##### Example\n\n```py\n#!/usr/bin/env python3\n#app.py\nimport argparse\n\nparser = argparse.ArgumentParser(description='Process some integers.')\nparser.add_argument('integers', metavar='N', type=int, nargs='+',\n                   help='an integer for the accumulator')\nparser.add_argument('--sum', dest='accumulate', action='store_const',\n                   const=sum, default=max,\n                   help='sum the integers (default: find the max)')\n\nargs = parser.parse_args()\nprint(args.accumulate(args.integers))\n```\n\nAssuming the Python code above is saved into a file called `app.py`, it can be run at the command line and provides useful help messages\n\n```bash\n$ app.py -h\nusage: prog.py [-h] [--sum] N [N ...]\n\nProcess some integers.\n\npositional arguments:\n N           an integer for the accumulator\n\noptional arguments:\n -h, --help  show this help message and exit\n --sum       sum the integers (default: find the max)\n\n$ app.py 1 2 3 4\n4\n\n$ app.py 1 2 3 4 --sum\n10\n```\n\n### Creating a parser\n\nThe first step in using the `argparse` is creating an `ArgumentParser` object:\n\n```py\n\u003e\u003e\u003e parser = argparse.ArgumentParser(description='Process some integers.')\n```\n\nThe `ArgumentParser` object will hold all the information necessary to parse the command line into python data types.\n\n### Adding arguments\n\nFilling an`ArgumentParser` with information about program arguments is done by making calls to the `ArgumentParser.add_argument` method. Generally, these calls tell the `ArgumentParser` how to take the strings on the command line and turn them into objects. This information is stored and used when `ArgumentParser.parse_args` is called. For example:\n\n```py\n\u003e\u003e\u003e parser.add_argument('integers', metavar='N', type=int, nargs='+',\n...                     help='an integer for the accumulator')\n\u003e\u003e\u003e parser.add_argument('--sum', dest='accumulate', action='store_const',\n...                     const=sum, default=max,\n...                     help='sum the integers (default: find the max)')\n```\n\nLater, calling `parse_args` will return an object with two attributes, integers and accumulate. The integers attribute will be a list of one or more ints, and the accumulate attribute will be either the `sum` function, if --sum was specified at the command line, or the `max` function if it was not.\n\n### Parsing arguments\n\n`ArgumentParser` parses args through the `ArgumentParser.parse_args` method. This will inspect the command-line, convert each arg to the appropriate type and then invoke the appropriate action. In most cases, this means a simple namespace object will be built up from attributes parsed out of the command-line:\n\n```py\n\u003e\u003e\u003e parser.parse_args(['--sum', '7', '-1', '42'])\nNamespace(accumulate=\u003cbuilt-in function sum\u003e, integers=[7, -1, 42])\n```\n\nIn a script,`ArgumentParser.parse_args` will typically be called with no arguments, and the `ArgumentParser` will automatically determine the command-line args from `sys.argv`.\n\n### ArgumentParser objects\n\nCreate a new `ArgumentParser` object. Each parameter has its own more detailed description below, but in short they are:\n\n- `description` - Text to display before the argument help.\n- `epilog` - Text to display after the argument help.\n- `add_help` - Add a -h/--help option to the parser. (default: True)\n- `argument_default` - Set the global default value for arguments. (default: None)\n- `parents` - A list of `ArgumentParser` objects whose arguments should also be included.\n- `prefix_chars` - The set of characters that prefix optional arguments. (default: '-')\n- `fromfile_prefix_chars` - The set of characters that prefix files from which additional arguments should be read. (default: None)\n- `formatter_class` - A class for customizing the help output.\n- `conflict_handler` - Usually unnecessary, defines strategy for resolving conflicting optionals.\n- `prog` - The name of the program (default:`sys.argv[0]`)\n- `usage` - The string describing the program usage (default: generated)\n\n#### [Learn more about argparse module](https://docs.python.org/3.7/library/argparse.html)\n\n## `Exploring SQLite Module` exploring-standard-modules-embedded-relational-database-module\n\nSQLite is a C-language library that implements a SQL like database engine which is relatively quick, serverless and self-contained, high-reliable. SQLite comes built-in with most of the moden software, hardware devices and browsers, thus Python also has embedded SQLite engine named sqlite3.\n\nTo Start using the sqlite3 library:\n\n```py\n\u003e\u003e\u003e import sqlite3\n```\n\n#### Commonly used functions:\n\n- `sqlite3.connect()` - A connection object is created using the connect() function e.g **connection = sqlite.connect('name_of_file_db.db')**\n- `connection.cursor()` - To execute SQLite statements, cursor object is needed. You can create it using the cursor() method. e.g **cursor_object = connection.cursor()**\n- `connection.execute()` - To create a table in SQLite3, you can use the Create Table, Insert Into Table, Update Table, or Select query with the execute() method of SQLite library. For example **cursor_object.execute(\"CREATE TABLE employees()\"); connection.commit()**\n- `connection.commit()` - The commit() method saves all the changes we make.\n- `cursor_object.fetchall()` -  To fetch the data from a database we will execute the SELECT statement and then will use the fetchall() method of the cursor object to store the values into a variable, e.g  **cursor_object.execute('SELECT * FROM employees') ; rows = cursor_object.fetchall()**\n- `cursor_object.rowcount()` - The SQLite3 rowcount is used to return the number of rows that are affected or selected by the latest executed SQL query\n- `cursor_object.executemany()` - It can use the **executemany** statement to insert multiple rows at once.\n- `connection.close()` - You are done with your database, it is a good practice to close the connection with close() method. e.g. **connection.close()**\n\n#### [Learn More about SQLite3 Module](https://docs.python.org/3.7/library/sqlite3.html)\n\n## `Exploring XmlTree Module` exploring-xmltree-module\n\n\n#### [Learn More about XmlTree Module](https://docs.python.org/3.7/library/xml.etree.elementtree.html#module-xml.etree.ElementTree)\n\n\n\n\n## `Exploring JSON Module` exploring-json-module\n\nJSON is text, written with JavaScript object notation. JSON is a syntax for storing and exchanging data.\nIt is commonly used for transmitting data in web applications (e.g., sending some data from the server to the client, so it can be displayed on a web page\nand vice versa\n\n- `json.loads()`\n- take a file object and returns the json object. A JSON object contains data in the form of key/value pair. The keys are strings and the values are the JSON types\n\n- `json.dumps()`\n-json.dumps() function converts a Python object into a json string\n.It is the exact opposite of json.loads.\n\nTHIS IS THE ENCODING DECODING LIST\n\n JSON -Python \n \n 1)object- DICT   \n 2)array - list  \n 3)string - str   \n 4)int  -  int  \n 5)real  -  float \n 6)true  -  true  \n 7)False - False \n 8)NULL  - NONE  \n\n\n\n\n - `  Encoding is from python to JSON(final type)`\n\u003e\u003e\u003e json.JSONEncoder().encode({\"foo\": [\"bar\", \"baz\"]})\n'{\"foo\": [\"bar\", \"baz\"]}'\n\n\n- `Decoding is from JSON to python(final type)`\n\n## Encoding functions\n\n- `iterencode(o)`\n-Encode the given object, o, and yield each string representation as available. For example:\n\u003e\u003e\u003efor chunk in json.JSONEncoder().iterencode(bigobject):\n    mysocket.write(chunk)]\n    \n` -sort-keys` -Sort the output of dictionaries alphabetically by key.\n\n`-h, --help¶`- help box \n\n`infile`-to check your Json file for syntax\n\n`outfile`-Write the output of the infile to the given outfile\n\n## Note\nIf the optional infile and outfile arguments are not specified, sys.stdin and sys.stdout will be used respectively:\n\n `json.tool `- to validate and pretty-print JSON objects.\n \n `raw_decode`- This can be used to decode a JSON document from a string that may have extraneous data at the end.\n\n#### [Learn More about JSON Module](https://docs.python.org/3.7/library/json.html)\n\n\n## `Exploring CSV Module` exploring-csv-module\nCSV - comma-separated values file is a text file with it's content delimited by comma (most often but any other delimiter is acceptable, yet it is advised to stick to the standards) defined by RFC 4180.\nThese files are really useful when it comes to share, read and save data. Structure of such file is very simple:\n```csv\nheader1,header2,header3\ndatacell11,datacell12,datacell13\ndatacell21,datacell22,datacell23\ndatacell31,datacell32,datacell33\n```\nYou can easily read and write CSV files with Python using `csv` module. To open CSV file we need code like this:\n```python\nimport csv\nwith open('file.csv') as f:\n    f_csv = csv.reader(f, delimiter=',', quotechar='\"')\n    headers = next(f_csv)   # this line uses next() function to save first row of the file and skip it. Rest is our data.\n    # now we are ready to process our rows. Example:\n    for row in f_csv:\n        print(row[0], row[1], row[2])\n```\nScript would just print data like this:\n```shell\n$\u003e datacell11 datacell12 datacell13\n$\u003e datacell21 datacell22 datacell23\n$\u003e datacell31 datacell32 datacell33\n```\nHere's explanation for some parameters of `csv.reader`:\n\n`delimiter` - is as it name shows - character or set of them that determines where are columns of value and how they are separated\n\n`quotechar` - is a char starting and ending a quote in our CSV file. It prevents our script from breaking output when we would have somethin like: `datacell21,\"datacell22\",datacell23`\n\nIf the file is more complicated than this (and it probably will) it is advised to use DictReader. It would look like this:\n```python\nimport csv\nwith open('file.csv') as f:\n    f_csv = csv.DictReader(f, delimiter=',', quotechar='\"')\n    headers = next(f_csv)   # this line uses next() function to save first row of the file and skip it. Rest is our data.\n    # now we are ready to process our rows. Example:\n    for row in f_csv:\n        print(row['header1'], row['header2'], row['header3'])\n```\nInstead of column indexes it is possible to use their names from header.\n\nTo write file we can simply do this:\n```python\nimport csv\ndata = [\n    ['data1', 'data2', 'data3'],\n    ['data4', 'data5', 'data6']\n    # ...\n]\nwith open('newfile.csv', 'w') as nf:\n    headers = ['header_1', 'header_2', 'header_3']\n    csv_writer = csv.writer(nf)\n    csv_writer.writerow(headers)\n    csv_writer.writerows(data)\n```\nWe use `writerow` for writing header for our data and then `writerows` to simply handle a few(hundred) rows of data\n\nAlternatively we DictWriter can be used:\n```python\nimport csv\n\ndata = [\n    {'header_1': 'data1', 'header_2': 'data2', 'header_3': 'data3'},\n    {'header_1': 'data4', 'header_2': 'data5', 'header_3': 'data6'},\n    {'header_1': 'data7', 'header_2': 'data8', 'header_3': 'data9'}\n]\nwith open('countries.csv', 'w', encoding='UTF8', newline='') as f:\n    headers = ['header_1', 'header_2', 'header_3']\n    writer = csv.DictWriter(f, fieldnames=headers)\n    writer.writeheader()\n    writer.writerows(data)\n```\n\n#### [Learn More about CSV Module](https://docs.python.org/3/library/csv.html)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuladhar%2Fpython-for-sysadmin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftuladhar%2Fpython-for-sysadmin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftuladhar%2Fpython-for-sysadmin/lists"}