https://github.com/bsc-wdc/distro-stream-lib

Distributed Stream Library
https://github.com/bsc-wdc/distro-stream-lib

compss distributed-computing stream-processing

Last synced: about 1 month ago
JSON representation

Distributed Stream Library

• Host: GitHub
• URL: https://github.com/bsc-wdc/distro-stream-lib
• Owner: bsc-wdc
• License: apache-2.0
• Created: 2019-09-17T09:28:30.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2021-12-22T13:59:39.000Z (about 3 years ago)
• Last Synced: 2024-04-14T03:01:10.786Z (9 months ago)
• Topics: compss, distributed-computing, stream-processing
• Language: Java
• Size: 2.33 MB
• Stars: 1
• Watchers: 8
• Forks: 1
• Open Issues: 2
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        


[![Language](https://img.shields.io/badge/language-java-brightgreen.svg)](https://img.shields.io/badge/language-java-brightgreen.svg)

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](http://compss.bsc.es/gitlab/project/distro-stream-lib/blob/master/LICENSE)

# Distributed Stream Library

The implementation includes:

* DistroStream: Stream interface to handle stream accesses homogeneously regardless of the stream engine backing it (Python and Java).

    * ObjectDistroStream: Implementation of the interface for object streams built on top of Kafka (Java).

    * FileDistroStream: Implementation of the interface for file streams that monitors the creation of files inside a directory (Python and Java).

* Client: Acts as a broker on behalf of the application and interacts with the corresponding stream backend (Python and Java).

* Server: Centralized server to manage the streams metadata and coordinate the accesses (Java).

* Test: Basic integration tests.

---

## Table of Contents

* [Contributing](#contributing)

* [Author](#author)

* [Disclaimer](#disclaimer)

* [License](#license)

---

## Contributing

All kinds of contributions are welcome. Please do not hesitate to open a new issue, submit a pull request or contact the author if necessary. 

 

## Author

Cristián Ramón-Cortés Vilarrodona  ([Personal Website][cristian])

This work is supervised by:

- Rosa M. Badia ([BSC][bsc])

- Jorge Ejarque ([BSC][bsc])

- Francesc Lordan ([BSC][bsc])

## Disclaimer

This work is part of a PhD Thesis at the [Workflows and Distributed Computing Team][wdc-bsc] at [BSC][bsc] and is still under development. 

## License

Licensed under the [Apache 2.0 License][apache-2]

[wdc-bsc]: https://www.bsc.es/discover-bsc/organisation/scientific-structure/workflows-and-distributed-computing

[bsc]: https://www.bsc.es/

[cristian]: https://cristianrcv.netlify.com/

[apache-2]: http://www.apache.org/licenses/LICENSE-2.0