https://github.com/traunguyentvt/study_big_data_technology

Kafka, Spark Streaming, Spark SQL, Hive, Tableau
https://github.com/traunguyentvt/study_big_data_technology

api hive json kafka spark spark-sql spark-streaming tableau

Last synced: about 1 month ago
JSON representation

Kafka, Spark Streaming, Spark SQL, Hive, Tableau

• Host: GitHub
• URL: https://github.com/traunguyentvt/study_big_data_technology
• Owner: traunguyentvt
• Created: 2023-04-25T18:59:29.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2023-05-03T00:15:28.000Z (almost 2 years ago)
• Last Synced: 2024-11-14T21:28:53.660Z (3 months ago)
• Topics: api, hive, json, kafka, spark, spark-sql, spark-streaming, tableau
• Language: Java
• Homepage:
• Size: 8.03 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# big-data-technology

Course: Big Data Technology (CS-523)

Professor: Mrudula Mukadam

Group Members

	1.	Thanh Do Nguyen

	2.	Van Ngoc Trau Nguyen

	3.	Mihret Ghebremichael Kidane

	4.	Melaku Gossaye Denbel

	

# Download Kafka:

 	- Link: https://archive.apache.org/dist/kafka/2.4.1/kafka_2.12-2.4.1.tgz

 	

 	- Extract the downloaded file, then go to this directory:

 	

     	+ cd /home/cloudera/Downloads/kafka_2.12-2.4.1

# Check service status:

     + service --status-all

 

# Start Zookeeper if it's not running (usually it already started):

     + bin/zookeeper-server-start.sh config/zookeeper.properties

# Start Kafka server:

     + bin/kafka-server-start.sh -daemon config/server.properties

# Build maven project:

     + cd ../project

     + mvn clean package

     

# Run jar file: always run jar file with dependencies

     + java -jar 'consumer/target/consumer-1.0-SNAPSHOT-jar-with-dependencies.jar'

     + java -jar 'producer/target/producer-1.0-SNAPSHOT-jar-with-dependencies.jar'

     

# Run visualizer with jar file: run jar file (without dependencies)

	 + java -jar 'visualizer/visualizer-1.0-SNAPSHOT.jar'

     

#Analyze projects:

     + Please refer the FinalProject_presentation_v2.pptx