https://github.com/publishername/software-company-profiler
SoftwareCompanyProfiler is a tool for extracting and organizing detailed information about software companies. It gathers contact details, career opportunities, LinkedIn links, office locations, and other key insights, streamlining research and decision-making for businesses, recruiters, and professionals.
https://github.com/publishername/software-company-profiler
scraper
Last synced: 8 months ago
JSON representation
SoftwareCompanyProfiler is a tool for extracting and organizing detailed information about software companies. It gathers contact details, career opportunities, LinkedIn links, office locations, and other key insights, streamlining research and decision-making for businesses, recruiters, and professionals.
- Host: GitHub
- URL: https://github.com/publishername/software-company-profiler
- Owner: PublisherName
- Created: 2024-12-27T10:53:27.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-03T11:55:00.000Z (over 1 year ago)
- Last Synced: 2025-01-03T12:39:57.707Z (over 1 year ago)
- Topics: scraper
- Language: Python
- Homepage:
- Size: 15.6 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Software Company Profiler
Software Company Profiler is a comprehensive solution for extracting and organizing detailed information about software companies. It efficiently gathers critical data, including contact details, career opportunities, LinkedIn profiles, office locations, and other key insights, enabling businesses and professionals to streamline research, networking, and decision-making.
## Installation
1. Clone the repository
```bash
git clone git@github.com:PublisherName/Software-Company-Profiler.git
```
2. Change the directory to the project directory
```bash
cd Software-Company-Profiler
```
3. Create a virtual environment using the following command:
```bash
python3 -m venv .venv
```
4. Activate the virtual environment
```bash
source .venv/bin/activate
```
5. Install the required packages using the following command:
```bash
pip install -r requirements.txt
```
6. Run the following command to start the web scraper:
```bash
scrapy crawl -o -a url=""
```
7. To manually populate the data, run the following command:
```bash
python main.py --file
```
options:
- `--update` : Update the data
- `--view` : Search the data
- `--add` : Add the data
## CSV FORMAT
| name | address | city | website | career page | email | linkedIn | phone |
|-------------|-------------|-----------|--------------|--------------|-------------|-------------|-------------|
| | | | | | | | |
## Supported Websites
## 1. [NepalYP](https://www.nepalyp.com/)
### Usage:
```bash
scrapy crawl nepalyp -o nepalyp.csv -a url="https://www.nepalyp.com/category/Software_applications"
```
## 2. [Techbehemoths](https://techbehemoths.com/)
### Usage:
```bash
scrapy crawl techbehemoths -o techbehemoths.csv -a url="https://techbehemoths.com/companies/nepal"
```