{"id":16445241,"url":"https://github.com/dgapitts/python-data-analysis","last_synced_at":"2025-02-26T08:25:48.054Z","repository":{"id":145829444,"uuid":"250363873","full_name":"dgapitts/python-data-analysis","owner":"dgapitts","description":"python-data-analysis course exercises","archived":false,"fork":false,"pushed_at":"2020-04-20T21:37:42.000Z","size":292,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-08T22:38:52.642Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dgapitts.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-03-26T20:24:32.000Z","updated_at":"2020-04-20T21:37:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"ae82fc1b-154d-4be4-a6c5-d773637f4778","html_url":"https://github.com/dgapitts/python-data-analysis","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgapitts%2Fpython-data-analysis","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgapitts%2Fpython-data-analysis/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgapitts%2Fpython-data-analysis/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dgapitts%2Fpython-data-analysis/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dgapitts","download_url":"https://codeload.github.com/dgapitts/python-data-analysis/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240816721,"owners_count":19862348,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T09:43:45.323Z","updated_at":"2025-02-26T08:25:48.032Z","avatar_url":"https://github.com/dgapitts.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Overview\n\nSample python scripts from python-data-analysis course on Linkedin Learning  (https://www.linkedin.com/learning/python-data-analysis-2015).\n\nMy objectives are to \n- enhance my python scripting\n- starting moving from python2.7 to python3.x\n- familiarizer myself with python numpy, pandas, matplotlib (I've already done simple line charts with pylab)\n- work with jupyter notebook\n\n## Setup\n\nI've been using both python3 (3.7) (installed via brew)\n\n```\n~/projects/python-data-analysis $ python3\nPython 3.7.7 (default, Mar 10 2020, 15:43:03)\n[Clang 11.0.0 (clang-1100.0.33.17)] on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n\u003e\u003e\u003e import numpy\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\n\u003e\u003e\u003e exit()\n```\n\nand especially regular python (2.7) for quite a while \n```\n~/projects/python-data-analysis $ python\nPython 2.7.10 (default, Feb 22 2019, 21:55:15)\n[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n\u003e\u003e\u003e import numpy\n\u003e\u003e\u003e import panda\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\nImportError: No module named panda\n\u003e\u003e\u003e import pandas\nTraceback (most recent call last):\n  File \"\u003cstdin\u003e\", line 1, in \u003cmodule\u003e\nImportError: No module named pandas\n\u003e\u003e\u003e import matplotlib\n\u003e\u003e\u003e\n```\n\n\n\nFor this course, I want to use `Anaconda`\n\n\u003e Anaconda is a free and open-source distribution of the Python and R programming languages for scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system conda.\n\nso again using brew\n\n```\n~/projects/python-data-analysis $ brew cask install anaconda\nUpdating Homebrew...\n==\u003e Auto-updated Homebrew!\nUpdated 2 taps (homebrew/core and homebrew/cask).\n==\u003e New Formulae\n...\n```\n\n\n\n\nfinally and I think I like this (no affect on my regular python usage) you need to use the z-shell (I'm typically a bash shell person)\n\n```\n~ $ cat ~/.zshrc\nexport PATH=\"/usr/local/anaconda3/bin:$PATH\"\n```\n\nnow I can use Anaconda python \n\n```\nLast login: Thu Mar 26 20:30:22 on ttys014\n~ $ zsh\nDaves-MacBook-Pro% python\nPython 3.7.4 (default, Aug 13 2019, 15:17:50)\n[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n\u003e\u003e\u003e import numpy\n\u003e\u003e\u003e import pandas\ni\u003e\u003e\u003e import matplotlib\n\u003e\u003e\u003e exit ()\n```\nand jupyter notebooks:\n```\nDaves-MacBook-Pro% jupyter notebook\n[I 21:12:21.784 NotebookApp] The port 8888 is already in use, trying another port.\n[I 21:12:21.846 NotebookApp] JupyterLab extension loaded from /usr/local/anaconda3/lib/python3.7/site-packages/jupyterlab\n[I 21:12:21.846 NotebookApp] JupyterLab application directory is /usr/local/anaconda3/share/jupyter/lab\n[I 21:12:21.848 NotebookApp] Serving notebooks from local directory: /Users/dpitts\n[I 21:12:21.849 NotebookApp] The Jupyter Notebook is running at:\n[I 21:12:21.849 NotebookApp] http://localhost:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\n[I 21:12:21.849 NotebookApp]  or http://127.0.0.1:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\n[I 21:12:21.849 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).\n[C 21:12:21.855 NotebookApp]\n\n    To access the notebook, open this file in a browser:\n        file:///Users/dpitts/Library/Jupyter/runtime/nbserver-93038-open.html\n    Or copy and paste one of these URLs:\n        http://localhost:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\n     or http://127.0.0.1:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\n^C[I 21:12:36.561 NotebookApp] interrupted\nServing notebooks from local directory: /Users/dpitts\n0 active kernels\nThe Jupyter Notebook is running at:\nhttp://localhost:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\n or http://127.0.0.1:8889/?token=4bfb635934506cbc764660146e2e6d019f8282e6fa260df1\nShutdown this notebook server (y/[n])? No answer for 5s: resuming operation...\n```\n\n### A little Help with getting started with Jupyter Notebooks\n\nSounds a bit like vi i.e. a good start\n{quote}\nFirst, we need to know that they are 2 modes in the Jupyter Notebook App: command mode and edit mode.\nhttps://towardsdatascience.com/jypyter-notebook-shortcuts-bf0101a98330\n{quote}\n\nThe most important key to know is H for help, which appeaars to OS context sensitive:\n\n ![alt text](Help-shortcuts-Jupyter-Notebooks.png \"Help-shortcuts-Jupyter-Notebooks.png\")\n\n\n\n### ex02-05-namedtuples-python27-script.py\n\nRef: https://pymotw.com/2/collections/namedtuple.html\n```\n~/projects/python-data-analysis $ cat ex02-05-namedtuples-python27-script.py\nimport collections\nPerson = collections.namedtuple('Person', 'name age gender')\n\nprint 'Type of Person:', type(Person)\n\nbob = Person(name='Bob', age=30, gender='male')\nprint '\\nRepresentation:', bob\n\njane = Person(name='Jane', age=29, gender='female')\nprint '\\nField by name:', jane.name\n\nprint '\\nFields by index:'\nfor p in [ bob, jane ]:\n    print '%s is a %d year old %s' % p\n```\noutput\n```\n~/projects/python-data-analysis $ python2.7 ex02-05-namedtuples-python27-script.py\nType of Person: \u003ctype 'type'\u003e\n\nRepresentation: Person(name='Bob', age=30, gender='male')\n\nField by name: Jane\n\nFields by index:\nBob is a 30 year old male\nJane is a 29 year old female\n```\n\n\n\n### ex03-text-manipulation-and-files\n\nOne confusing thing for me with jupyter is how to know and/or set my working directory within the host OS filesysteM? \nSo needed to python ues the os package \n\n```\nimport os\narr = os.listdir()\nprint(arr)\n```\nhttps://stackoverflow.com/questions/3207219/how-do-i-list-all-files-of-a-directory\n\nAlso I was curious to load data from a more interesting dataset i.e. a txt version of Percy Bysshe Shelley's \"Prometheus Unbound\" (http://www.gutenberg.org/cache/epub/4797/pg4797.txt), but then I also need to convert words into lines\n\n```\nYou need str.split() to split each string into words:\nword_list = [word for line in sentence for word in line.split()]\n```\nhttps://stackoverflow.com/questions/8478602/convert-a-list-of-string-sentences-to-words\n\n\n### ex03-reading-a-file-python2.7\n\nRef: https://stackabuse.com/read-a-file-line-by-line-in-python/\n```\n~/projects/python-data-analysis $ cat ex03-filehandling.py\nfilepath = 'Prometheus_Unbound.txt'\nwith open(filepath) as fp:\n   line = fp.readline()\n   cnt = 1\n   while line:\n       print(\"Line {}: {}\".format(cnt, line.strip()))\n       line = fp.readline()\n       cnt += 1\n```\nRunning this:\n```\n~/projects/python-data-analysis $ python2.7 ex03-filehandling.py|head\nLine 1: PROMETHEUS UNBOUND. A LYRICAL DRAMA IN FOUR ACTS.\nLine 2:\nLine 3:\nLine 4: PROMETHEUS UNBOUND. A LYRICAL DRAMA IN FOUR ACTS.\t1\nLine 5: PREFACE.\t1\nLine 6: ACT 1.\t4\nLine 7: ACT 2.\t23\nLine 8: ACT 3.\t38\nLine 9: ACT 4.\t49\nLine 10:\n```\nand...\n```\n~/projects/python-data-analysis $ python2.7 ex03-filehandling.py|tail\nLine 4197: a month or two I shall send it. It is a drama, with characters and\nLine 4198: mechanism of a kind yet unattempted; and I think the execution is\nLine 4199: better than any of my former attempts.'\nLine 4200:\nLine 4201: I may mention, for the information of the more critical reader, that\nLine 4202: the verbal alterations in this edition of \"Prometheus\" are made from a\nLine 4203: list of errata written by Shelley himself.\nLine 4204:\nLine 4205: ***\nLine 4206:\n```\n\n### Developing word split logic\n\n\nCreate file with first 100 words\n```\n~/projects/python-data-analysis $ head -100 Prometheus_Unbound.txt \u003e Prometheus_Unbound_first100_lines.txt\n~/projects/python-data-analysis $ cat Prometheus_Unbound_first100_lines.txt\nPROMETHEUS UNBOUND. A LYRICAL DRAMA IN FOUR ACTS.\n\n\nPROMETHEUS UNBOUND. A LYRICAL DRAMA IN FOUR ACTS.\t1\nPREFACE.\t1\nACT 1.\t4\nACT 2.\t23\nACT 3.\t38\nACT 4.\t49\n\n\nAUDISNE HAEC AMPHIARAE, SUB TERRAM ABDITE?\n\n[Composed at Este, September, October, 1818 (Act 1); at Rome,\nMarch-April 6, 1819 (Acts 2, 3); at Florence, close of 1819 (Act 4).\nPublished by C. and J. Ollier, London, summer of 1820. Sources of the\ntext are (1) edition of 1820; (2) text in \"Poetical Works\", 1839,\nprepared with the aid of a list of errata in (1) written out by\nShelley; (3) a fair draft in Shelley's autograph, now in the Bodleian.\nThis has been carefully collated by Mr. C.D. Locock, who prints the\nresult in his \"Examination of the Shelley Manuscripts in the Bodleian\nLibrary\", Oxford (Clarendon Press), 1903. Our text is that of 1820,\nmodified by edition 1839, and by the Bodleian fair copy. In the\nfollowing notes B = the Bodleian manuscript; 1820 = the editio\nprinceps, printed by Marchant for C. and J. Ollier, London; and 1839 =\nthe text as edited by Mrs. Shelley in the \"Poetical Works\", 1st and\n2nd editions, 1839. The reader should consult the notes on the Play at\nthe end of the volume.]\n\n\nPREFACE.\n\nThe Greek tragic writers, in selecting as their subject any portion of\ntheir national history or mythology, employed in their treatment of it\na certain arbitrary discretion. They by no means conceived themselves\nbound to adhere to the common interpretation or to imitate in story as\nin title their rivals and predecessors. Such a system would have\namounted to a resignation of those claims to preference over their\ncompetitors which incited the composition. The Agamemnonian story was\nexhibited on the Athenian theatre with as many variations as dramas.\n\nI have presumed to employ a similar license. The \"Prometheus Unbound\"\nof Aeschylus supposed the reconciliation of Jupiter with his victim as\nthe price of the disclosure of the danger threatened to his empire by\nthe consummation of his marriage with Thetis. Thetis, according to\nthis view of the subject, was given in marriage to Peleus, and\nPrometheus, by the permission of Jupiter, delivered from his captivity\nby Hercules. Had I framed my story on this model, I should have done\nno more than have attempted to restore the lost drama of Aeschylus; an\nambition which, if my preference to this mode of treating the subject\nhad incited me to cherish, the recollection of the high comparison\nsuch an attempt would challenge might well abate. But, in truth, I was\naverse from a catastrophe so feeble as that of reconciling the\nChampion with the Oppressor of mankind. The moral interest of the\nfable, which is so powerfully sustained by the sufferings and\nendurance of Prometheus, would be annihilated if we could conceive of\nhim as unsaying his high language and quailing before his successful\nand perfidious adversary. The only imaginary being resembling in any\ndegree Prometheus, is Satan; and Prometheus is, in my judgement, a\nmore poetical character than Satan, because, in addition to courage,\nand majesty, and firm and patient opposition to omnipotent force, he\nis susceptible of being described as exempt from the taints of\nambition, envy, revenge, and a desire for personal aggrandisement,\nwhich, in the Hero of \"Paradise Lost\", interfere with the interest.\nThe character of Satan engenders in the mind a pernicious casuistry\nwhich leads us to weigh his faults with his wrongs, and to excuse the\nformer because the latter exceed all measure. In the minds of those\nwho consider that magnificent fiction with a religious feeling it\nengenders something worse. But Prometheus is, as it were, the type of\nthe highest perfection of moral and intellectual nature, impelled by\nthe purest and the truest motives to the best and noblest ends.\n\nThis Poem was chiefly written upon the mountainous ruins of the Baths\nof Caracalla, among the flowery glades, and thickets of odoriferous\nblossoming trees, which are extended in ever winding labyrinths upon\nits immense platforms and dizzy arches suspended in the air. The\nbright blue sky of Rome, and the effect of the vigorous awakening\nspring in that divinest climate, and the new life with which it\ndrenches the spirits even to intoxication, were the inspiration of\nthis drama.\n\nThe imagery which I have employed will be found, in many instances, to\nhave been drawn from the operations of the human mind, or from those\nexternal actions by which they are expressed. This is unusual in\nmodern poetry, although Dante and Shakespeare are full of instances of\nthe same kind: Dante indeed more than any other poet, and with greater\nsuccess. But the Greek poets, as writers to whom no resource of\nawakening the sympathy of their contemporaries was unknown, were in\nthe habitual use of this power; and it is the study of their works\n(since a higher merit would probably be denied me) to which I am\nwilling that my readers should impute this singularity.\n\nOne word is due in candour to the degree in which the study of\ncontemporary writings may have tinged my composition, for such has\nbeen a topic of censure with regard to poems far more popular, and\nindeed more deservedly popular, than mine. It is impossible that any\none who inhabits the same age with such writers as those who stand in\nthe foremost ranks of our own, can conscientiously assure himself that\nhis language and tone of thought may not have been modified by the\nstudy of the productions of those extraordinary intellects. It is\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgapitts%2Fpython-data-analysis","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdgapitts%2Fpython-data-analysis","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdgapitts%2Fpython-data-analysis/lists"}