https://github.com/apache/tez

Apache Tez
https://github.com/apache/tez

apache big-data hadoop java tez

Last synced: about 2 months ago
JSON representation

Apache Tez

• Host: GitHub
• URL: https://github.com/apache/tez
• Owner: apache
• License: apache-2.0
• Created: 2013-04-08T07:20:23.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2024-08-02T16:18:46.000Z (about 2 months ago)
• Last Synced: 2024-08-02T18:11:22.939Z (about 2 months ago)
• Topics: apache, big-data, hadoop, java, tez
• Language: Java
• Homepage: https://tez.apache.org/
• Size: 28.9 MB
• Stars: 466
• Watchers: 34
• Forks: 415
• Open Issues: 70
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-dataops - Apache Tez - A generic data-processing pipeline engine envisioned as a low-level engine. (Data Processing)

README

        


Apache Tez

==========

Apache Tez is a generic data-processing pipeline engine envisioned as a low-level engine for higher abstractions

such as Apache Hadoop Map-Reduce, Apache Pig, Apache Hive etc.

At its heart, tez is very simple and has just two components:

*   The data-processing pipeline engine where-in one can plug-in input, processing and output implementations to 

    perform arbitrary data-processing. Every 'task' in tez has the following:

   -   Input to consume key/value pairs from.

   -   Processor to process them.

   -   Output to collect the processed key/value pairs.

*  A master for the data-processing application, where-by one can put together arbitrary data-processing 'tasks' 

   described above into a task-DAG to process data as desired. 

   The generic master is implemented as a Apache Hadoop YARN ApplicationMaster.