Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/apache/pulsar-client-python
Apache Pulsar Python client library
https://github.com/apache/pulsar-client-python
event-streaming messaging pubsub pulsar queuing streaming
Last synced: 3 days ago
JSON representation
Apache Pulsar Python client library
- Host: GitHub
- URL: https://github.com/apache/pulsar-client-python
- Owner: apache
- License: apache-2.0
- Created: 2022-09-29T01:10:23.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-02T17:16:17.000Z (about 2 months ago)
- Last Synced: 2024-12-15T11:06:18.049Z (10 days ago)
- Topics: event-streaming, messaging, pubsub, pulsar, queuing, streaming
- Language: Python
- Homepage: https://pulsar.apache.org/
- Size: 857 KB
- Stars: 53
- Watchers: 34
- Forks: 43
- Open Issues: 75
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
# Pulsar Python client library
Pulsar Python clients support a variety of Pulsar features to enable building applications connecting to your Pulsar cluster. For the supported Pulsar features, see [Client Feature Matrix](https://pulsar.apache.org/client-feature-matrix/).
## Requirements
- Python 3.8, 3.9, 3.10, 3.11, 3.12
- A C++ compiler that supports C++11
- CMake >= 3.18
- [Pulsar C++ client library](https://github.com/apache/pulsar-client-cpp)
- [PyBind11](https://github.com/pybind/pybind11)
PyBind11 is a header-only library and a submodule, so you can simply download the submodule so that CMake can find this dependency.
```bash
git submodule update --init
```
You can also download the pybind11 directly like:
```bash
pip3 install pyyaml
export PYBIND11_VERSION=$(./build-support/dep-version.py pybind11)
curl -L -O https://github.com/pybind/pybind11/archive/refs/tags/v${PYBIND11_VERSION}.tar.gz
tar zxf v${PYBIND11_VERSION}.tar.gz
mv pybind11-${PYBIND11_VERSION} pybind11
```
After that, you only need to install the Pulsar C++ client dependency into the system path. You can [install the pre-built binaries](https://pulsar.apache.org/docs/next/client-libraries-cpp/#installation) or [build from source](https://github.com/apache/pulsar-client-cpp#compilation).
## Install the Python wheel
Make sure the PyBind11 submodule has been downloaded and the Pulsar C++ client has been installed. Then run the following commands:
```bash
cmake -B build
cmake --build build
cmake --install build
python3 ./setup.py bdist_wheel
python3 -m pip install dist/pulsar_client-*.whl --force-reinstall
```
> **NOTE**
>
> 1. The separate `build` directory is created to store all CMake temporary files. However, the `setup.py` requires the `_pulsar.so` to be under the project directory.
> 2. Add the `--force-reinstall` option to overwrite the existing Python wheel in case your system has already installed a wheel before.
> 3. On Windows, the Python command is `py` instead of `python3`.
## Running examples
You can run `python3 -c 'import pulsar'` to see whether the wheel has been installed successfully. If it fails, check whether dependencies (e.g., `libpulsar.so`) are in the system path. If not, make sure the dependencies are in `LD_LIBRARY_PATH` (on Linux) or `DYLD_LIBRARY_PATH` (on macOS).
Then you can run examples as a simple end-to-end test.
```bash
# In terminal 1
python3 ./examples/consumer.py
```
```bash
# In terminal 2
python3 ./examples/producer.py
```
Before executing the commands above, you must ensure the Pulsar service is running. See [here](https://pulsar.apache.org/docs/getting-started-standalone) for quick start.
## Unit tests
Before running the unit tests, you must run a Pulsar service with all things set up:
```bash
./build-support/pulsar-test-service-start.sh
```
The command above runs a Pulsar standalone in a Docker container. You can run `./build-support/pulsar-test-service-stop.sh` to stop it.
Run all unit tests:
```bash
./tests/run-unit-tests.sh
```
Run a single unit test (e.g., `PulsarTest.test_tls_auth`):
```bash
python3 ./tests/pulsar_test.py 'PulsarTest.test_tls_auth'
```
## Generate API docs
Pulsar Python Client uses [pydoctor](https://github.com/twisted/pydoctor) to generate API docs. To generate by yourself, you need to install the Python library first. Then run the following command in the root path of this repository:
```bash
sudo python3 -m pip install pydoctor
cp $(python3 -c 'import _pulsar, os; print(_pulsar.__file__)') ./_pulsar.so
pydoctor --make-html \
--docformat=numpy --theme=readthedocs \
--intersphinx=https://docs.python.org/3/objects.inv \
--html-output= \
--introspect-c-modules \
./_pulsar.so \
pulsar
```
Then the index page will be generated in `/index.html`.
## Contribute
We welcome contributions from the open source community!
If your contribution adds Pulsar features for Python clients, you need to update both the [Pulsar docs](https://pulsar.apache.org/docs/client-libraries/) and the [Client Feature Matrix](https://pulsar.apache.org/client-feature-matrix/). See [Contribution Guide](https://pulsar.apache.org/contribute/site-intro/#pages) for more details.