{"id":15626187,"url":"https://github.com/ai-ahmed/ppcax","last_synced_at":"2025-04-13T00:26:52.723Z","repository":{"id":245925518,"uuid":"783923703","full_name":"AI-Ahmed/stochastic_fex","owner":"AI-Ahmed","description":"Probabilistic PCA and PKPCA for Stochastic Feature Extraction and Missing Data Reconstruction","archived":false,"fork":false,"pushed_at":"2024-11-03T15:26:36.000Z","size":110,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-11-03T15:32:29.968Z","etag":null,"topics":["bayesian","chex","distrax","finance","jax","pkpca","ppca","probabilistic","probabilistic-models","python","quantitative-finance","stochastic"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AI-Ahmed.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-04-08T21:00:52.000Z","updated_at":"2024-11-03T15:26:39.000Z","dependencies_parsed_at":"2024-06-24T21:35:31.778Z","dependency_job_id":"4fa403d2-421a-40e4-a614-85f92bfa70b9","html_url":"https://github.com/AI-Ahmed/stochastic_fex","commit_stats":null,"previous_names":["ai-ahmed/ppcax","ai-ahmed/stochastic_fex"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AI-Ahmed%2Fstochastic_fex","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AI-Ahmed%2Fstochastic_fex/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AI-Ahmed%2Fstochastic_fex/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AI-Ahmed%2Fstochastic_fex/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AI-Ahmed","download_url":"https://codeload.github.com/AI-Ahmed/stochastic_fex/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":228993808,"owners_count":18003304,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bayesian","chex","distrax","finance","jax","pkpca","ppca","probabilistic","probabilistic-models","python","quantitative-finance","stochastic"],"created_at":"2024-10-03T10:11:16.903Z","updated_at":"2024-12-10T02:27:47.452Z","avatar_url":"https://github.com/AI-Ahmed.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PPCAx – Probabilistic PCA with JAX\n\n## Overview\n\nProbabilistic Principal Component Analysis (PPCA) model using DeepMind's JAX library. The model is a robust feature extraction and dimensionality reduction technique for high-dimensional, sparse multivariate data.\n\nPPCA is a probabilistic approach to Principal Component Analysis (PCA), which allows for imputing missing values and estimating latent features in the data. By leveraging the power of JAX, this implementation ensures efficient and scalable computation, making it suitable for large-scale financial datasets.\n\nThe methodology used in this project was initially proposed in our research manuscript titled *\"Probabilistic PCA in High Dimensions: Stochastic Dimensionality Reduction on Sparse Multivariate Assets' Bars at High-Risk Regimes\"*. This work presents a novel approach for analyzing portfolio behavior during periods of high market turbulence and risk by:\n\n1. Using information-driven bar techniques to synchronize and sample imbalanced sequence volumes.\n2. Applying a sampling event-based technique, the CUMSUM Filtering method, to create strategic trading plans based on volatility.\n3. Employing an improved version of the Gaussian Linear System called PPCA for feature extraction from the latent space.\n\nOur findings suggest that PPCA is highly effective in estimating sparse data and forecasting the effects of individual assets within a portfolio under varying market conditions. This repository contains the core implementation of the PPCA model, demonstrating its capability to establish significant relationships among correlated assets during high-risk regimes.\n\n## 📁 Directory Structure\n\n```bash\n.\n├── LICENSE\n├── README.md\n├── config\n├── data\n│   ├── bars\n│   ├── metadata\n│   ├── sample\n│   │   ├── r1\n│   │   └── r2\n│   └── tickers\n├── models\n├── notebooks\n├── pyproject.toml\n├── reports\n│   ├── docs\n│   ├── eval\n│   ├── figures\n│   └── train\n├── src\n│   ├── __init__.py\n│   ├── eval\n│   ├── ft_eng\n│   ├── ppcax\n│   │   ├── __init__.py\n│   │   └── _ppcax.py\n│   ├── preprocessing\n│   └── utils\n└── tests\n    ├── __init__.py\n    ├── gen_data.py\n    └── test_ppcax.py\n\n25 directories, 17 files\n```\n\n## 🛠️ Installation and Setup Instructions\n\n### Prerequisites\n\n- **Python**: Ensure you have Python **3.10** or newer installed on your system.\n\n### Installation Steps\n\n1. **Clone the Repository**\n\n   ```shell\n   git clone https://github.com/AI-Ahmed/ppcax.git\n   cd ppcax\n   ```\n\n2. **Install Flit**\n\n   If you don't already have Flit installed, install it using `pip`:\n\n   ```shell\n   pip install flit\n   ```\n\n3. **Install the Package and Dependencies**\n\n   Install the package along with its dependencies using Flit:\n\n   ```shell\n   flit install --deps develop\n   ```\n\n   This command installs the `ppcax` package along with all required dependencies, including development and testing tools like `pytest` and `flake8`.\n\n### Alternative: Install Directly from GitHub\n\nIf you prefer to install the package directly from GitHub without cloning the repository:\n\n```shell\npip install git+https://github.com/AI-Ahmed/ppcax\n```\n\nThis command installs the latest version of `ppcax` from the main branch.\n\n### Importing the Package\n\nAfter installation, you can import the PPCA model in your Python code:\n\n```python\nfrom ppcax import PPCA\n```\n\n## 🧪 Running Tests\n\nTo run the unit tests and ensure everything is working correctly:\n\n1. **Navigate to the Project Directory**\n\n   If you haven't already, navigate to the project's root directory:\n\n   ```shell\n   cd ppcax\n   ```\n\n2. **Run Tests Using pytest**\n\n   ```shell\n   pytest tests/test.py\n   ```\n\n## 📚 Usage Example\n\nHere's a simple example of how to use the `PPCA` class:\n\n```python\nimport numpy as np\nfrom ppcax import PPCA\n\n# Generate some sample data\ndata = np.random.rand(100, 1000)\n\n# Create a PPCA model instance\nppca_model = PPCA(q=150)\n\n# Fit the model to the data\nppca_model.fit(data, use_em=True)\n\n# Transform the data to the lower-dimensional space\ntransformed_data = ppca_model.transform(lower_dim_only=True)\n\nprint(\"Transformed Data Shape:\", transformed_data.shape)\n```\n\n## 📄 License\n\nThis project is licensed under the [Apache License 2.0](LICENSE), which is a permissive open-source license that grants users extensive rights to use, modify, and distribute the software. See the [LICENSE](LICENSE) file for more details.\n\n## 📣 Cite Our Work\n\nIf you find this work useful in your research, please consider citing:\n\n```bibtex\n@article{Atwa2024,\n  author    = {Ahmed Atwa and Ahmed Sedky and Mohamed Kholief},\n  title     = {Probabilistic PCA in High Dimensions: Stochastic Dimensionality Reduction on Sparse Multivariate Assets' Bars at High-Risk Regimes},\n  journal   = {SSRN Electronic Journal},\n  year      = {2024},\n  note      = {Available at SSRN: \\url{https://ssrn.com/abstract=4874874} or \\url{http://dx.doi.org/10.2139/ssrn.4874874}}\n}\n```\n\n---\n\n## 🔧 Development Setup\n\nIf you're planning to contribute to the project or modify the code, follow these steps to set up your development environment:\n\n1. **Clone the Repository**\n\n   ```shell\n   git clone https://github.com/AI-Ahmed/ppcax.git\n   cd ppcax\n   ```\n\n2. **Create a Virtual Environment**\n\n   It's recommended to use a virtual environment to manage dependencies:\n\n   ```shell\n   python -m venv venv\n   source venv/bin/activate  # On Windows use: venv\\Scripts\\activate\n   ```\n\n3. **Install Flit**\n\n   ```shell\n   pip install flit\n   ```\n\n4. **Install the Package in Editable Mode**\n\n   - For development and testing, install the package with the `test` extras:\n\n   ```shell\n   flit install --deps develop --extras test --symlink\n   ```\n\n   The `--symlink` option installs the package in editable mode, so changes to the code are immediately reflected without reinstallation.\n\n5. **Install Pre-commit Hooks (Optional)**\n\n   If you use `pre-commit` for code formatting and linting:\n\n   ```shell\n   pip install pre-commit\n   pre-commit install\n   ```\n\n6. **Run Tests**\n\n   ```shell\n   pytest tests/test.py\n   ```\n\n## 🤝 Contributing\n\nContributions are welcome! Please open an issue or submit a pull request for any improvements or bug fixes.\n\n---\n\n## 📬 Contact\n\nFor any questions or inquiries, please contact [Ahmed Nabil Atwa](mailto:dr.ahmedna.ai@gmail.com).\n\n---\n\n## 📝 Changelog\n\nRefer to the [CHANGELOG](CHANGELOG.md) for details on updates and changes to the project.\n\n---\n\n## 📦 Publishing to PyPI (Maintainers Only)\n\nTo publish a new version of the package to PyPI:\n\n1. **Update the Version Number**\n\n   Increment the version number in `pyproject.toml`.\n\n2. **Build the Package**\n\n   ```shell\n   flit build\n   ```\n\n3. **Publish to PyPI**\n\n   ```shell\n   flit publish\n   ```\n\n---\n\n## 🌐 Links\n\n- **Documentation**: [Github Package documentation](https://github.com/AI-Ahmed/ppcax/README.md)\n- **Issue Tracker**: [GitHub Issues](https://github.com/AI-Ahmed/ppcax/issues)\n- **Source Code**: [GitHub Repository](https://github.com/AI-Ahmed/ppcax)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-ahmed%2Fppcax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fai-ahmed%2Fppcax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fai-ahmed%2Fppcax/lists"}