https://github.com/prachpaveeny/pyspark-project
PySpark-project
https://github.com/prachpaveeny/pyspark-project
azure azure-sql-database google-cloud-platform jupyter-notebook numpy pandas pyspark python
Last synced: 2 months ago
JSON representation
PySpark-project
- Host: GitHub
- URL: https://github.com/prachpaveeny/pyspark-project
- Owner: PrachpaveenY
- Created: 2023-12-03T16:24:39.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-29T13:39:26.000Z (over 2 years ago)
- Last Synced: 2024-05-02T00:59:11.728Z (about 2 years ago)
- Topics: azure, azure-sql-database, google-cloud-platform, jupyter-notebook, numpy, pandas, pyspark, python
- Language: Jupyter Notebook
- Homepage:
- Size: 203 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PySpark-project (Complete)
- Project : PySpark Project
- Tools = Python, PySpark, Pandas, Azure SQL Database
### Detail :
- Create File : CSV, TXT.
- Use PySpark and Pandas : manage data.
- microsoft azure project : use manage data of Azure SQL Database
### Structure :
```
📁 PySpark-project
└── 📁 all project
| └── 📁 apache airflow
| |
| └── 📁 app
| | └── 📁 tables // All CSV Files
| | └── 📄 use.ipynb // Main File of app ( use manage data )
| |
| └── 📁 google cloud platform
| |
| └── 📁 microsoft azure project
| | └── 📄 Azure-sql-database-project.ipynb // Main File of microsoft azure project ( use manage data of Azure SQL Database )
| |
└── 📁 img
└── 📁 tutorial
| └── 📄 HR01.csv // Use CSV of tutorial
| └── 📄 Pandas_Tutorial1.ipynb // Pandas Tutorial
| └── 📄 PySpark_Tutorial1.ipynb // PySpark Tutorial
| └── 📄 Python_Tutorial01.ipynb // Python Tutorial
| └── 📄 python_tutorial01.py // Python Tutorial
|
└── 📄 HR01.csv
└── 📄 hr01.txt
└── 📄 PySpark_createData.ipynb // Create File
└── 📄 README.md
```
## Run project step :
- Run use.ipynb
- Run PySpark_Tutorial1.ipynb
