{"id":16984412,"url":"https://github.com/pedropark99/introd-pyspark","last_synced_at":"2025-06-14T07:12:18.928Z","repository":{"id":39919073,"uuid":"485179725","full_name":"pedropark99/Introd-pyspark","owner":"pedropark99","description":"An open and introductory book for the Python API of Apache Spark (pyspark) 📚📖","archived":false,"fork":false,"pushed_at":"2025-03-20T22:40:19.000Z","size":453715,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-20T23:29:34.510Z","etag":null,"topics":["apache-spark","book","course","pyspark","python","spark","tutorial","tutorial-course"],"latest_commit_sha":null,"homepage":"https://pedropark99.github.io/Introd-pyspark/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc-by-4.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pedropark99.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-25T01:00:23.000Z","updated_at":"2025-03-20T22:40:24.000Z","dependencies_parsed_at":"2023-10-01T23:14:35.225Z","dependency_job_id":"767c9a62-d35c-45ba-b083-57dea82b916c","html_url":"https://github.com/pedropark99/Introd-pyspark","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedropark99%2FIntrod-pyspark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedropark99%2FIntrod-pyspark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedropark99%2FIntrod-pyspark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pedropark99%2FIntrod-pyspark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pedropark99","download_url":"https://codeload.github.com/pedropark99/Introd-pyspark/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248510931,"owners_count":21116310,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-spark","book","course","pyspark","python","spark","tutorial","tutorial-course"],"created_at":"2024-10-14T02:31:44.753Z","updated_at":"2025-04-12T03:23:51.459Z","avatar_url":"https://github.com/pedropark99.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Introd-pyspark\n\n\u003ca href=\"https://pedro-faria.netlify.app/publications/book/introd-pyspark/en/\"\u003e\u003cimg src=\"Cover/cover1.png\" width=\"250\" height=\"366\" class=\"cover\" align=\"right\"/\u003e\u003c/a\u003e An open and introductory book for the Python API of Apache Spark. The book \"Introduction to pyspark\" provides a quick introduction for the `pyspark` Python package, which is the Python API of Apache Spark.\n\nRead the book at: \u003chttps://pedropark99.github.io/Introd-pyspark/\u003e.\n\nYou can buy a copy of the book through Amazon: \u003chttps://www.amazon.com/dp/B0CRYMVWDN\u003e.\n\nPublication page: \u003chttps://pedro-faria.netlify.app/publications/book/introd-pyspark/en/\u003e.\n\nWith `pyspark` you are able to use the Python language to write Spark applications and run them on a Spark cluster in a scalable and elegant way. This book focus on teaching the fundamentals of `pyspark`, and how to use it for big data analysis.\n\nSome of the main subjects discussed in the book are:\n\n- How an Apache Spark application works?\n- What are Spark DataFrames?\n- How to transform and model your Spark DataFrame.\n- How to import data into, and export data out of Apache Spark.\n- How to work with SQL inside pyspark.\n- Tools for manipulating specific data types (e.g. string, dates and datetimes).\n- How to use window functions.\n\n\n## About the author\n\nPedro Duarte Faria have a bachelor degree in Economics from Federal University of Ouro Preto - Brazil. Currently, he is a Data Platform Engineer at Blip, and an Associate Developer for Apache Spark 3.0 certified by Databricks.\n\nThe author have more than 5 years of experience in the data analysis market. He developed data pipelines, reports and analysis for research institutions and some of the largest companies in the brazilian financial sector, such as the BMG Bank, Sodexo and Pan Bank, besides dealing with databases that go beyond the billion rows.\n\nFurthermore, Pedro is specialized on the R programming language, and have given several lectures and courses about it, inside graduate centers (such as PPEA-UFOP), in addition to federal and state organizations (such as FJP-MG). As researcher, he have experience in the field of Science, Technology and Innovation Economics.\n\nPersonal Website: \u003chttps://pedro-faria.netlify.app/\u003e\n\nTwitter: [@PedroPark9](https://twitter.com/PedroPark9)\n\nMastodon: [@pedropark99@fosstodon.org](https://fosstodon.org/@pedropark99)\n\n\n## License\n\nCopyright © 2024 Pedro Duarte Faria. This book is licensed by the CC-BY 4.0 Creative Commons Attribution 4.0 International Public License.\n\n\u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\"\u003e\u003cimg alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https://i.creativecommons.org/l/by/4.0/88x31.png\" /\u003e\u003c/a\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedropark99%2Fintrod-pyspark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpedropark99%2Fintrod-pyspark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpedropark99%2Fintrod-pyspark/lists"}