https://github.com/estebanpdl/osintgpt
An open-source intelligence (OSINT) analysis tool leveraging GPT-powered embeddings and vector search engines for efficient data processing
https://github.com/estebanpdl/osintgpt
embeddings gpt-3-5-turbo gpt-4 osint osint-python osint-tool pinecone python qdrant
Last synced: 22 days ago
JSON representation
An open-source intelligence (OSINT) analysis tool leveraging GPT-powered embeddings and vector search engines for efficient data processing
- Host: GitHub
- URL: https://github.com/estebanpdl/osintgpt
- Owner: estebanpdl
- Created: 2023-05-04T23:41:28.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-12-11T17:06:57.000Z (over 1 year ago)
- Last Synced: 2025-03-28T16:03:52.905Z (29 days ago)
- Topics: embeddings, gpt-3-5-turbo, gpt-4, osint, osint-python, osint-tool, pinecone, python, qdrant
- Language: Python
- Homepage:
- Size: 1.28 MB
- Stars: 409
- Watchers: 6
- Forks: 55
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# **OSINT GPT**
`osintgpt` is a Python package for leveraging OpenAI's GPT models to analyze text data and perform tasks such as calculating text embeddings, searching for similar documents, and more. It is designed for use in open-source intelligence (OSINT) applications and research.
[](https://GitHub.com/estebanpdl/osintgpt/network/)
[](https://github.com/estebanpdl/osintgpt/stargazers)
[](https://github.com/estebanpdl/osintgpt/blob/main/LICENCE)
[](https://twitter.com/estebanpdl)
[](https://www.python.org/)
[](https://twitter.com/estebanpdl)
## **Installation**
You can install the `osintgpt` package using pip:
```bash
pip install osintgpt
```
Given the recent changes and updates, it is highly recommended to upgrade to the latest version of the package by executing the following command
```bash
pip install -U osintgpt
```
This command will update your existing installation of `osintgpt` to the most recent version available, ensuring you have access to the latest features and improvements.
To access the package details and documentation on PyPI, please follow the link [osintgpt on PyPI](https://pypi.org/project/osintgpt/)
## 🚀 **Features**
The `osintgpt` Python package is designed to streamline the process of analyzing text data by leveraging OpenAI's GPT models. Here are some of the key features:
- **Text Analysis**: Utilize OpenAI's GPT models to analyze text data, including calculating text embeddings and searching for similar documents.
- **Interactive Mode**: The package includes an interactive mode that allows users to communicate directly with the GPT model. The user can input a prompt and receive a response from the model, facilitating a more dynamic interaction.
- **Database Management**: The package integrates with SQLite database, enabling easy storage and retrieval of conversation data. The SQLDatabaseManager class creates tables, handles data insertion, and manages transactions.
Please note that the development of `osintgpt` is still in progress, and some features may still be refined or expanded.
## 💾 **Vector store**
Qdrant
The `Qdrant` class is an interface to Qdrant, a high-performance vector similarity search engine. It provides a variety of methods for connecting and interacting with a Qdrant server, such as creating, updating, and deleting collections, and managing vector embeddings along with their associated payloads.
Main Features:
- **Connection Management**: The class allows you to establish and manage connections to a Qdrant server. The server can be accessed remotely or locally.
- **Collection Management**: You can create, update, and delete collections in Qdrant. Each collection can contain multiple vectors.
- **Vector and Payload Management**: The class provides methods to add, update, and search for vector embeddings in collections. Each vector can optionally have an associated payload. The payload represents data associated with the vector, such as metadata or additional features.
- **High Efficiency**: With the ability to efficiently store and search embeddings, Qdrant can support high-dimensional data and large-scale databases.
Setting Up Qdrant:
To use the Qdrant class, you will need access to a Qdrant server, either remotely or locally:
- **Remote Server**: Register for a remote server on [Qdrant Cloud](https://cloud.qdrant.io/).
- **Local Server**: Set up a local server following the instructions on the [Qdrant Quick Start guide](https://qdrant.tech/documentation/quick_start/).
## **Disclaimer**
The `osintgpt` tool is provided for research purposes and intended to assist users in analyzing data from open-source intelligence (OSINT) tools more efficiently. It relies on third-party services, such as the OpenAI API, various database engines, and other resources that may have associated costs. By using this tool, you acknowledge that you are responsible for understanding and managing any costs related to these services. The creators and maintainers of `osintgpt` are not liable for any expenses incurred or any misuse of the tool. Please use this tool responsibly and in compliance with all applicable laws and regulations.