https://github.com/jmrozanec/hive-unit

Unit testing for Hive
https://github.com/jmrozanec/hive-unit

Last synced: 6 months ago
JSON representation

Unit testing for Hive

• Host: GitHub
• URL: https://github.com/jmrozanec/hive-unit
• Owner: jmrozanec
• License: apache-2.0
• Created: 2014-09-27T22:47:47.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2014-10-08T15:56:57.000Z (over 11 years ago)
• Last Synced: 2025-08-15T22:50:10.046Z (8 months ago)
• Language: Java
• Size: 5.6 MB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
## hive-unit: A simple way to test Hive scripts.

[![Build Status](https://travis-ci.org/jmrozanec/hive-unit.svg)](https://travis-ci.org/jmrozanec/hive-unit)

### Usage

hive-unit provides means to test Hive scripts. This can be done against an Hive instance 

including an embedded Derby database, and a local HiveThriftService; or against an existing Hive cluster. 

This allows to create unit and integration tests very easily.

A DSL is provided following builder pattern, making it easy to declare test conditions.

Hive scripts testing can be done in a similar way to [PigUnit](http://pig.apache.org/docs/r0.8.1/pigunit.html).

For more examples check [this class](https://github.com/jmrozanec/hive_test/blob/master/src/test/java/com/jointhegrid/hive_test/HiveTestTest.java)

        @Test

            public void testScriptListInput() {

                Map> input = Maps.>newHashMap();

                List lines = Lists.newArrayList();

                lines.add("msmith,10");

                lines.add("mjohnson,2");

                lines.add("mwilliamson,7");

                lines.add("mjones,4");

                lines.add("mdavies,5");

                input.put("$INPUT1", lines);

                Response output =

                        HiveBuilder.create()

                                .hiveTestWithEmbeddedHive("src/test/resources/scripts/passing-scores.hql")

                                .outputForInput(input);

                List expected = Lists.newArrayList();

                expected.add("msmith,10");

                expected.add("mwilliamson,7");

                assertEquals(ResponseStatus.SUCCESS, output.getResponseStatus());

                assertEquals(expected, output.getResult());

            }

### Maven

#### Soon will be available at Maven central!

#### Execution profiles

##### Without a local Hadoop install

By default, we're set to download a local copy of Hadoop when you first build Hive Test, or whenever the project is cleaned, just before running our test cases.

You can force a redownload and installation of Hadoop by manually activating the download-hadoop profile

    mvn --activate-profiles download-hadoop test

You can also perform the download and extraction process independent of testing.

Download Hadoop (into the maven target directory)

    mvn --activate-profiles download-hadoop wagon:download-single

Extract Hadoop  (into the maven target directory)

    mvn --activate-profiles download-hadoop exec:exec

##### With a local Hadoop install

We'll skip attempting to download and use a local copy of Hadoop if any of the following are true

* set your HADOOP_HOME environment variable to a hadoop distribution

* hadoop tar extracted to  $home/hadoop/hadoop-0.20.2_local

Hive Test will work so long as you have Hadoop in your path, i.e. /usr/bin/hadoop. In this case, you'll want to deactivate the hadoop download.

    mvn --activate-profiles -hadoop-home-defined test

### Contribute!

Contributions are welcome! You can contribute by

 * starring this repo!

 * adding new features

 * enhancing existing code

 * testing

 * enhancing documentation

 * bringing suggestions and reporting bugs

 * spreading the word / telling us how you use it!