Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/CEREBRUS-MAXIMUS/Surfer-Data
Surfer is the world's first personal data scraper.
https://github.com/CEREBRUS-MAXIMUS/Surfer-Data
Last synced: 3 months ago
JSON representation
Surfer is the world's first personal data scraper.
- Host: GitHub
- URL: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data
- Owner: CEREBRUS-MAXIMUS
- License: mit
- Created: 2024-08-09T02:54:17.000Z (3 months ago)
- Default Branch: main
- Last Pushed: 2024-08-19T04:41:26.000Z (3 months ago)
- Last Synced: 2024-08-19T04:55:55.667Z (3 months ago)
- Language: TypeScript
- Size: 82.8 MB
- Stars: 320
- Watchers: 1
- Forks: 21
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Surfer: The World's First Digital Footprint Exporter
[![Contributors][contributors-shield]][contributors-url]
[![Forks][forks-shield]][forks-url]
[![Stargazers][stars-shield]][stars-url]
[![Issues][issues-shield]][issues-url]
[![MIT License][license-shield]][license-url]
[](https://discord.gg/5KQkWApkYC)
Table of Contents
## Demo (click to view)
[](https://www.youtube.com/watch?v=2P25iOd14qw)
Surfer is the world's first digital footprint exporter, designed to centralize all your personal data from various online platforms into a single folder.
Currently, your personal data is scattered across hundreds of platforms and the companies operating these platforms have no incentive to give this data back to you. Surfer solves this problem by navigating to websites and scraping data from these websites.
We believe that personal data centralization is the key to enabling truly useful, universal personal assistants.
## How it works
1. Click on "Export" to initiate the data extraction process.
2. The app waits for the target page to load completely.
3. The system checks if the user is signed in to the platform being scraped.
4. If not signed in, the user is prompted to sign in.
5. If signed in, the process continues.
6. Once signed in, the app interacts with the platform's user interface.
7. The app then scrapes the user's data from the platform.
8. Finally, the extracted data is exported and saved to your local storage.
## Getting Started
To download the app, head over to [https://surfsup.ai](https://surfsup.ai). Or you can go to the releases page.
For instructions on setting up the app locally and contributing to the project, please refer to the [Contributing Guidelines](CONTRIBUTING.md), [Helper Functions Documentation](docs/HELPER_FUNCTIONS.md), and [Guide to Adding New Platforms](docs/ADD_PLATFORMS.md).
See the [open issues](https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/issues) for a full list of proposed features (and known issues).
## Roadmap
### Short-Term
- [ ] Obtain a code signing certificate for Windows
- [x] Replace `setTimeout` with `await` for script execution to ensure elements exist before scraping
- [ ] Implement robust error handling for the scraping process
- [ ] Add support for more online platforms
- [ ] Add verbosity to runs
### Medium to Long-Term
- [ ] Implement concurrent scraping to allow for multiple scraping jobs to run simultaneously
- [ ] Adding sub-tasks within platforms (i.e. Twitter Bookmarks, LinkedIn Connections Data, etc)
- [ ] Integrate with other agentic frameworks like LangChain for advanced personal AI assistants
- [ ] Explore integration with wearable devices for enhanced personal data tracking and acknowledgment
## License
Distributed under the MIT License. See `LICENSE` for more information.
## Built With
* [![Electron][Electron.js]][Electron-url]
* [![React][React.js]][React-url]
* [![Tailwind][Tailwind.css]][Tailwind-url]
* [![Shadcn UI][Shadcn.ui]][Shadcn-url]
## Contact
[Surfer Discord Server](https://discord.gg/Tjg7pjcFNP) - [@SahilLalani0](https://x.com/SahilLalani0) - [@JackBlair87](https://x.com/JackBlair87) - [@T0M_3D](https://x.com/T0M_3D)
Project Link: [https://github.com/CEREBRUS-MAXIMUS/Surfer-Data](https://github.com/CEREBRUS-MAXIMUS/Surfer-Data)
## Star History
[](https://star-history.com/#CEREBRUS-MAXIMUS/Surfer-Data&Date)
## Acknowledgements
- [Electron React Boilerplate](https://github.com/electron-react-boilerplate/electron-react-boilerplate)
[contributors-shield]: https://img.shields.io/github/contributors/CEREBRUS-MAXIMUS/Surfer-Data.svg?style=for-the-badge
[contributors-url]: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/graphs/contributors
[forks-shield]: https://img.shields.io/github/forks/CEREBRUS-MAXIMUS/Surfer-Data.svg?style=for-the-badge
[forks-url]: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/network/members
[stars-shield]: https://img.shields.io/github/stars/CEREBRUS-MAXIMUS/Surfer-Data.svg?style=for-the-badge
[stars-url]: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/stargazers
[issues-shield]: https://img.shields.io/github/issues/CEREBRUS-MAXIMUS/Surfer-Data.svg?style=for-the-badge
[issues-url]: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/issues
[license-shield]: https://img.shields.io/github/license/CEREBRUS-MAXIMUS/Surfer-Data.svg?style=for-the-badge
[license-url]: https://github.com/CEREBRUS-MAXIMUS/Surfer-Data/blob/master/LICENSE
[linkedin-shield]: https://img.shields.io/badge/-LinkedIn-black.svg?style=for-the-badge&logo=linkedin&colorB=555
[linkedin-url]: https://linkedin.com/in/cerebrus-maximus
[React.js]: https://img.shields.io/badge/React-20232A?style=for-the-badge&logo=react&logoColor=61DAFB
[React-url]: https://reactjs.org/
[Tailwind.css]: https://img.shields.io/badge/Tailwind_CSS-38B2AC?style=for-the-badge&logo=tailwind-css&logoColor=white
[Tailwind-url]: https://tailwindcss.com/
[Electron.js]: https://img.shields.io/badge/Electron-2B2E3A?style=for-the-badge&logo=electron&logoColor=9FEAF9
[Electron-url]: https://www.electronjs.org/
[Shadcn.ui]: https://img.shields.io/badge/Shadcn_UI-F05032?style=for-the-badge&logo=shadcn&logoColor=white
[Shadcn-url]: https://ui.shadcn.com/