https://github.com/distributedsystemsgroup/bleach
A distributed stream data cleaning system
https://github.com/distributedsystemsgroup/bleach
Last synced: 4 months ago
JSON representation
A distributed stream data cleaning system
- Host: GitHub
- URL: https://github.com/distributedsystemsgroup/bleach
- Owner: DistributedSystemsGroup
- Created: 2017-03-01T17:29:04.000Z (almost 9 years ago)
- Default Branch: master
- Last Pushed: 2017-03-01T21:33:51.000Z (almost 9 years ago)
- Last Synced: 2025-02-17T17:42:15.932Z (11 months ago)
- Language: Java
- Size: 59.6 KB
- Stars: 2
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Bleach
Bleach is distributed stream data cleaning system built on Apache Storm. Unlike other data cleaning systems
which mainly focus on batch data cleaning, Bleach performs data cleaning directly on data streams without
waiting for all the data to be acquired. It aims to achieve efficient and accurate qualitative data cleaning
under real-time constraints. It currently support FD rules and CFD rules. More details can be found in our
[paper](https://arxiv.org/abs/1609.05113#).
## How to run it
First, you need a cluster of machines in which Storm, Kafka and Zookeeper are installed. Next, download Bleach code and compile it by mvn:
$ git clone git://github.com:ychtian/Bleach
$ cd Bleach && mvn assembly:assembly
Then, submit the jar to Storm cluster to start Bleach:
$ storm jar target/bleach-1.0.0-jar-with-dependencies.jar storm.dataclean.TestTopology.TestRepair -config job.conf
All the configuration is included in file job.conf.