https://github.com/chimpler/hive-solr

Hive Storage Handler for SOLR
https://github.com/chimpler/hive-solr

Last synced: 3 months ago
JSON representation

Hive Storage Handler for SOLR

• Host: GitHub
• URL: https://github.com/chimpler/hive-solr
• Owner: chimpler
• Created: 2013-03-08T01:41:33.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2014-03-17T02:42:16.000Z (over 10 years ago)
• Last Synced: 2024-03-26T23:58:55.826Z (8 months ago)
• Language: Java
• Size: 166 KB
• Stars: 17
• Watchers: 10
• Forks: 26
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-hive - Solr

README

        
Installation

============

To install:

	$ git clone http://github.org/chimpler/hive-solr

	$ cd hive-solr

	$ mvn package

	$ cp target/hive-solr-0.0.1-SNAPSHOT-jar-with-dependencies.jar `$HIVE_HOME/lib`

Usage

=====

If your SOLR schema is something like:

	

	

	  

	   

	   

	   

	   

	   

	  

	 

	   

	   

	   

	   

	   

	   

	   

	   

	   

	   

	   

	 

	 

	 id

	 

	 name

	 

	 

	

You can create an external table as follows:

	hive> create external table solr_items2 (

	    id INT,

	    item_id INT,

	    name STRING,

	    year INT,

	    month INT,

	    shipping_method INT,

	    us_sold INT,

	    ca_sold INT,

	    fr_sold INT,

	    uk_sold INT

	) stored by "com.chimpler.hive.solr.SolrStorageHandler"

	with serdeproperties ("solr.column.mapping"="id,item_id,name,year,month,shipping_method,us_sold,ca_sold,fr_sold,uk_sold")

	tblproperties ("solr.url" = "http://localhost:8983/solr/core0","solr.buffer.input.rows"="10000","solr.buffer.output.rows"="10000");

Note that solr.buffer.input.rows and solr.buffer.output.rows are optional (default is 100000).

Acknowledgements

================

Thank you to yc-huang for his work on hive-mongo (https://github.com/yc-huang/Hive-mongo) that served as a base for hive-solr.