{"id":16328664,"url":"https://github.com/yoheinakajima/mindgraph","last_synced_at":"2025-05-16T15:02:59.710Z","repository":{"id":227992830,"uuid":"772881515","full_name":"yoheinakajima/mindgraph","owner":"yoheinakajima","description":"proof of concept prototype for generating and querying against an ever-expanding knowledge graph with ai","archived":false,"fork":false,"pushed_at":"2024-04-08T09:52:36.000Z","size":197,"stargazers_count":892,"open_issues_count":4,"forks_count":109,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-04-12T12:54:24.313Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yoheinakajima.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-16T06:09:37.000Z","updated_at":"2025-04-12T00:35:51.000Z","dependencies_parsed_at":"2024-11-23T17:02:14.731Z","dependency_job_id":"770f192c-703b-4075-a7cc-605b7f528b7d","html_url":"https://github.com/yoheinakajima/mindgraph","commit_stats":null,"previous_names":["yoheinakajima/mindgraph"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoheinakajima%2Fmindgraph","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoheinakajima%2Fmindgraph/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoheinakajima%2Fmindgraph/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yoheinakajima%2Fmindgraph/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yoheinakajima","download_url":"https://codeload.github.com/yoheinakajima/mindgraph/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254553936,"owners_count":22090415,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T23:14:26.947Z","updated_at":"2025-05-16T15:02:59.647Z","avatar_url":"https://github.com/yoheinakajima.png","language":"Python","readme":"# MindGraph\n\nWelcome to MindGraph, a proof of concept, open-source, API-first graph-based project designed for natural language interactions (input and output). This prototype serves as a template for building and customizing your own CRM solutions with a focus on ease of integration and extendibility. Here is the [announcement on X](https://twitter.com/yoheinakajima/status/1769019899245158648), for some more context.\n\n![flowchart](https://pbs.twimg.com/media/GIzWMHPa4AAakOc?format=jpg\u0026name=large)\n\n## Getting Started\n\n### Prerequisites\nBefore you begin, ensure you have the following installed:\n- Python 3.6 or higher\n- Poetry (dependency management and packaging tool)\n\n### Installation\n\n1. Clone the repository:\n   ```sh\n   git clone https://github.com/yourusername/MindGraph.git\n   ```\n\n2. Navigate to the project directory:\n   ```sh\n   cd MindGraph\n   ```\n\n3. Install the project dependencies using Poetry:\n   ```sh\n   poetry install\n   ```\n   This command will create a virtual environment for the project and install all the required packages specified in the ```pyproject.toml``` file.\n\n- Note: If you get any dependencies error you can run: ```poetry add \u003cname_of_dependency\u003e``` to add dependency to the project.\n\n## Environment Setup\n\n1. Create a ```.env``` file in the project root directory.\n\n2. Open the ```.env``` file and add the following line, replacing ```YOUR_API_KEY``` with your actual OpenAI API key:\n\n   ```sh\n   OPENAI_API_KEY=YOUR_API_KEY\n   ```\n\n## Running the Application\n\n   After installing the dependencies, you can start the Flask server with:\n\n\n   ```sh\n   poetry run python main.py\n   ```\n\n   The server will launch on `http://0.0.0.0:81`.\n\n## Project Structure\n\nMindGraph is organized into several key components:\n\n- `main.py`: The entry point to the application.\n- `app/__init__.py`: Sets up the Flask app and integrates the blueprints.\n- `models.py`: Manages the in-memory graph data structure for entities and relationships.\n- `views.py`: Hosts the API route definitions.\n- `integration_manager.py`: Handles the dynamic registration and management of integration functions.\n- `signals.py`: Sets up signals for creating, updating, and deleting entities.\n\n## Integration System\n\nMindGraph employs a sophisticated integration system designed to extend the application's base functionality dynamically. At the core of this system is `integration_manager.py`, which acts as a registry and executor for various integration functions. This modular architecture allows MindGraph to incorporate AI-powered features seamlessly, such as processing natural language inputs into structured knowledge graphs through integrations like `natural_input.py`. Further integrations, including `add_multiple_conditional`, `conditional_entity_addition`, and `conditional_relationship_addition`, work in tandem to ensure the integrity and enhancement of the application's data model.\n\n## Features\n\n**Entity Management**: Entities are stored in an in-memory graph for quick access and manipulation, allowing CRUD operations on people, organizations, and their interrelations.\n\n**Integration Triggers**: Custom integration functions can be triggered via HTTP requests, enabling the CRM to interact with external systems or run additional processing.\n\n**Search Capabilities**: Entities and their relationships can be easily searched with custom query parameters.\n\n**AI Readiness**: Designed with AI integrations in mind, facilitating the incorporation of intelligent data processing and decision-making.\n\n## API Endpoints\n\nMindGraph provides a series of RESTful endpoints:\n\n- `POST /\u003centity_type\u003e`: Create an entity.\n- `GET /\u003centity_type\u003e/\u003cint:entity_id\u003e`: Retrieve an entity.\n- `GET /\u003centity_type\u003e`: List all entities of a type.\n- `PUT /\u003centity_type\u003e/\u003cint:entity_id\u003e`: Update an entity.\n- `DELETE /\u003centity_type\u003e/\u003cint:entity_id\u003e`: Remove an entity.\n- `POST /relationship`: Establish a new relationship.\n- `GET /search/entities/\u003centity_type\u003e`: Search for entities.\n- `GET /search/relationships`: Find relationships.\n\n### Custom Integration Endpoint\n\n- `POST /trigger-integration/\u003cintegration_name\u003e`: Activates a predefined integration function.\n\n## Frontend Overview\n\nMindGraph's frontend features a lightweight interactive, web-based interface that facilitates dynamic visualization and management of the graph-based data model. While MindGraph is meant to be used as an API, the front-end was helpful for demo purposes. It leverages HTML, CSS, JavaScript, Cytoscape.js for graph visualization, and jQuery for handling AJAX requests.\n\n### Features\n\n- **Graph Visualization**: Uses Cytoscape.js for interactive graph rendering.\n- **Dynamic Data Interaction**: Supports real-time data fetching, addition, and graph updating without page reloads.\n- **Search and Highlight**: Allows users to search for nodes, highlighting and listing matches. Search form is being double used for natural language queries right now, which doesn't really make sense, but was a quick way to showcase functionality. (This is meant to be used as an API, front-end is for demo purpose)\n- **Data Submission Forms**: Includes forms for natural language, URL inputs, and CSV file uploads.\n- **Responsive Design**: Adapts to various devices and screen sizes.\n\n### Workflow\n\n1. **Initialization**: On page load, initializes the graph with styles and layout.\n2. **User Interaction**: Through the interface, users can:\n   - Search for nodes, with results highlighted in the graph and listed in a sidebar.\n   - Add data using a form that supports various input methods.\n   - Refresh the graph to reflect the latest backend data.\n3. **Data Processing**: User inputs are sent to the backend, processed, and integrated, with the frontend graph visualization updated accordingly.\n\n## Schema-driven Knowledge Graph Creation\n\nMindGraph utilizes a `schema.json` file to define the structure and relationships of entities within its knowledge graph. This schema acts as a blueprint for interpreting and structuring natural language inputs into a coherent graph format. It details the types of nodes (e.g., Person, Organization, Concept) and the possible relationships between them, ensuring that the generated knowledge graph adheres to a consistent format. This approach allows for automated, AI-driven processing of natural language inputs to generate structured data that reflects the complex interrelations inherent in the input text.\n\n### Utilizing `schema.json` in AI Integration\n\nWhen the `create_knowledge_graph` function processes an input, it consults `schema.json` to understand how to map the identified entities and their relationships into the graph. This includes:\n\n- Identifying node types and attributes based on the schema definitions.\n- Determining valid relationship types and their characteristics.\n- Structuring the output to match the expected graph format, facilitating seamless integration with the application's data model.\n\nThe schema ensures that the AI-generated knowledge graph is not only consistent with the application's data model but also rich in detail, capturing the nuanced relationships between entities as described in the input.\n\n### Benefits\n\n- **Consistency**: Ensures that all knowledge graphs generated from natural language inputs adhere to the same structural rules, making data integration and interpretation more straightforward.\n- **Flexibility**: Allows for easy updates and expansions of the knowledge graph structure by modifying `schema.json`, without requiring changes to the codebase.\n- **AI Integration**: Facilitates the use of advanced AI models for natural language processing by providing a clear structure for the expected output, enhancing the application's ability to derive meaningful insights from unstructured data.\n\n## Development \u0026 Extension\n\n### Adding New Integrations\n\nTo incorporate a new integration into MindGraph, create a Python module within the `integrations` directory. This module should define the integration's logic and include a `register` function that connects the integration to the `IntegrationManager`. Ensure that your integration interacts properly with the application's components, such as `models.py` for data operations and `views.py` for activation via API endpoints. This approach allows MindGraph to dynamically expand its capabilities through modular and reusable code.\n\n### Utilizing Signals\n\nSignals are emitted for entity lifecycle events, providing hooks for extending functionality or syncing with other systems.\n\n## Database Integration and Usage\n\nMindGraph supports flexible database integration to enhance its data storage and retrieval capabilities. Out of the box, MindGraph includes support for an in-memory database and a more robust, cloud-based option, NexusDB. This flexibility allows for easy adaptation to different deployment environments and use cases.\n\n### Supported Databases\n- InMemoryDatabase: A simple, in-memory graph data structure for quick prototyping and testing. Not recommended for production use due to its non-persistent nature.\n- NexusDB: An all-in-one cloud database designed for storing graphs, tables, documents, files, vectors, and more. Offers a shared knowledge graph for comprehensive data management and analysis.\nConfiguring the Database\n- NebulaGraph: A distributed, scalable, and lightning-fast graph database that supports real-time queries and analytics. Ideal for large-scale graph data storage and processing.\n- FalkorDB: A knowledge DB, providing low latency \u0026 high throughput, with built-in support for vector search and wide coverage of the Cypher query language.\n\nDatabase integration is controlled through the DATABASE_TYPE environment variable. To select a database, set this variable:\n\n-  `memory` for the in-memory database.\n-  `nexusdb` for NexusDB integration.\n\n```sh\nexport DATABASE_TYPE=nexusdb\n```\n\n-  `nebulagraph` for NebulaGraph integration.\n\n\u003e Note: For a running [NebulaGraph](https://github.com/vesoft-inc/nebula), consider using the [Docker Desktop Extension](https://hub.docker.com/extensions/weygu/nebulagraph-dd-ext), [NebulaGraph-Lite](https://github.com/nebula-contrib/nebulagraph-lite) for Colab/Linux with pip install, or explore more options in the [Docs](https://docs.nebula-graph.io/).\n\n```sh\nexport DATABASE_TYPE=nebulagraph\nexport NEBULA_ADDRESS=127.0.0.1:9669\n```\n\n-  `falkordb` for FalkorDB integration.\n\n\u003e Note: For a running [FalkorDB](https://www.falkordb.com), consider using the [Docker Image](https://hub.docker.com/r/falkordb/falkordb).\n\n```sh\nexport DATABASE_TYPE=falkordb\nexport FALKOR_HOST=127.0.0.1\nexport FALKOR_PORT=6379\nexport FALKOR_GRAPH_ID=mindgraph\n```\n\n### Adding New Database Integrations\nTo integrate a new database system into MindGraph:\n\n1) Implement the Database Integration: Create a new Python module under app/integrations/database following the abstract base class DatabaseIntegration defined in base.py. Your implementation should provide concrete methods for all abstract methods in the base class.\n\n2) Register Your Integration: Modify the database type detection logic in app/integrations/database/__init__.py to include your new database type. This involves adding an additional elif statement to check for your database's type and set the CurrentDBIntegration accordingly.\n\n3) Configure Environment Variables: If your integration requires custom environment variables (e.g., for connection strings, authentication), ensure they are documented and set properly in the environment where MindGraph is deployed.\n\n### Schema Management\nFor databases requiring schema definitions (like NexusDB), include a schema management strategy within your integration module. This may involve checking and updating the database schema on startup to ensure compatibility with the current version of MindGraph.\n\n## Example Command\n\nTo create a person via `curl`:\n\n```sh\ncurl -X POST http://0.0.0.0:81/people \\\n-H \"Content-Type: application/json\" \\\n-d '{\"name\":\"Jane Doe\",\"age\":28}'\n```\n\n### Example Use Cases\n\nTo demonstrate the power of MindGraph's integration system, here are some example commands:\n\n#### Triggering Natural Input Integration\n\n```sh\ncurl -X POST http://0.0.0.0:81/trigger-integration/natural_input \\\n-H \"Content-Type: application/json\" \\\n-d '{\"input\":\"Company XYZ organized an event attended by John Doe and Jane Smith.\"}'\n```\n\n## Contributions\n\nLet's be honest... I don't maintain projects. If you want to take over/manage this, let me know (X/Twitter is a good channel). Otherwise, enjoy this proof of concept starter kit as it is :)\n\n## License\n\nMindGraph is distributed under the MIT License. See `LICENSE` for more information.\n\n## Contact\n\nJust tag me on Twitter/X [https://twitter.com/yoheinakajima](https://twitter.com/yoheinakajima)\n\nProject Link: [https://github.com/yoheinakajima/MindGraph](https://github.com/yoheinakajima/MindGraph)\n","funding_links":[],"categories":["📦 Legacy \u0026 Inactive Projects","Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoheinakajima%2Fmindgraph","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyoheinakajima%2Fmindgraph","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyoheinakajima%2Fmindgraph/lists"}