https://github.com/hermaeus1618/nsdlscrapper
Foreign Institutional Investor NSDL
https://github.com/hermaeus1618/nsdlscrapper
python scrapper selenium stock-market
Last synced: about 1 month ago
JSON representation
Foreign Institutional Investor NSDL
- Host: GitHub
- URL: https://github.com/hermaeus1618/nsdlscrapper
- Owner: Hermaeus1618
- Created: 2024-12-08T14:34:37.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-08T14:36:52.000Z (over 1 year ago)
- Last Synced: 2025-04-07T20:13:04.218Z (about 1 year ago)
- Topics: python, scrapper, selenium, stock-market
- Language: Python
- Homepage:
- Size: 3.91 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# FOREIGN INSTITUTIONAL INVESTOR NSDL
A Selenium approach for scrapping FII trading data from https://www.fpi.nsdl.co.in.
I am only uploading algorithm for data fetching for the time being.
Data is filtered to remove any 'mid-table totaling' so that it can be use for a larger timeframe.
This approach is timeout error proof in most cases.
Speed of this algorithm is quite high because instead of using selenium functions for navigation it executes javascript using `webdriver.Firefox.execute_script()` directly from browsers API.
Data can be combined from ZIP by:
```python
import polars as pl
MAXDATE=3600
DFLIST=[]
with zipfile.ZipFile(NSDLCore.DIRECTORY_NSDL_FII_EQUITY_FILE, "r") as ZFILE:
DATERANGE=[int(F.filename) for F in ZFILE.infolist()][-MAXDATE:]
for i in range(len(DATERANGE)):
with ZFILE.open(f"{DATERANGE[i]}", "r") as WZFILE:
XDF=pl.read_parquet(io.BytesIO(WZFILE.read()))
DFLIST.append(XDF)
DF=pl.concat(DFLIST)
```
In this example i have used polars which is significantly faster than pandas in this algorithm but you can also use pandas simply as:
```python
import pandas as pd
with zipfile.ZipFile(NSDLCore.DIRECTORY_NSDL_FII_EQUITY_FILE, "r") as ZFILE:
DATERANGE=[int(F.filename) for F in ZFILE.infolist()][-MAXDATE:]
for i in range(len(DATERANGE)):
with ZFILE.open(f"{DATERANGE[i]}", "r") as WZFILE:
XDF=pd.read_parquet(io.BytesIO(WZFILE.read()))
DFLIST.append(XDF)
DF=pd.concat(DFLIST)
```
Derivatives section is currently under debugging after major NSDL server update.