https://github.com/euiyounghwang/es-job-interface
es-job-interface
https://github.com/euiyounghwang/es-job-interface
data-ingestion elasticsearch fastapi oracle python-elasticsearch sql
Last synced: 18 days ago
JSON representation
es-job-interface
- Host: GitHub
- URL: https://github.com/euiyounghwang/es-job-interface
- Owner: euiyounghwang
- Created: 2024-06-14T21:57:19.000Z (over 1 year ago)
- Default Branch: master
- Last Pushed: 2025-04-03T20:25:19.000Z (7 months ago)
- Last Synced: 2025-04-03T21:28:59.061Z (7 months ago)
- Topics: data-ingestion, elasticsearch, fastapi, oracle, python-elasticsearch, sql
- Language: Python
- Homepage:
- Size: 3.55 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# es-job-interface
es-job-interface
FastAPI is a modern, fast (high-performance), web framework for building APIs with Python 3.8+ based on standard Python.
This is a repository that provides to deliver the records to the Prometheus-Export application.
### Install Poerty
```
https://python-poetry.org/docs/?ref=dylancastillo.co#installing-with-the-official-installer
```
### Using Python Virtual Environment
```bash
python -m venv .venv
source .venv/bin/activate
```
#### Python V3.9 Install
```bash
sudo yum install gcc openssl-devel bzip2-devel libffi-devel zlib-devel git
wget https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tgz
tar –zxvf Python-3.9.0.tgz or tar -xvf Python-3.9.0.tgz
cd Python-3.9.0
./configure --libdir=/usr/lib64
sudo make
sudo make altinstall
# python3 -m venv .venv --without-pip
sudo yum install python3-pip
sudo ln -s /usr/lib64/python3.9/lib-dynload/ /usr/local/lib/python3.9/lib-dynload
python3 -m venv .venv
source .venv/bin/activate
# pip install -r ./dev-requirement.txt
pip install prometheus-client
pip install requests
pip install JPype1
pip install psycopg2-binary
pip install jaydebeapi
pip install pytz
pip install httpx
# when error occur like this
# ImportError: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'OpenSSL 1.0.2k-fips 26 Jan 2017'. See: https://github.com/urllib3/urllib3/issues/2168
pip install urllib3==1.26.18
pip install pytz
```
### Using Poetry: Create the virtual environment in the same directory as the project and install the dependencies:
```bash
python -m venv .venv
source .venv/bin/activate
pip install poetry
# --
poetry config virtualenvs.in-project true
poetry init
poetry add prometheus-client
poetry add psutil
poetry add pytz
poetry add JPype1
poetry add psycopg2-binary
poetry add jaydebeapi
poetry add pytest-cov
```
or you can run this shell script `./create_virtual_env.sh` to make an environment. then go to virtual enviroment using `source .venv/bin/activate`
### Register Service
- sudo service es_job_interface_api status/stop/start/restart
```bash
#-- /etc/systemd/system/es_job_interface_api.service
[Unit]
Description=ES Data Pipeline Interface Service between DB and Elasticsearch Cluster
[Service]
User=devuser
Group=devuser
Type=simple
ExecStart=/bin/bash /home/devuser/rest_api/es_job_interface_api/service-start.sh
ExecStop= /usr/bin/killall /es_job_interface_api
[Install]
WantedBy=default.target
# Service command
sudo systemctl daemon-reload
sudo systemctl enable es_job_interface_api.service
sudo systemctl start es_job_interface_api.service
sudo systemctl status es_job_interface_api.service
sudo systemctl stop es_job_interface_api.service
sudo service es_job_interface_api status/stop/start
```