https://github.com/ganzerli/crowler-for-words

Crowling domain, cut style, script and all HTML tags, gets words and links, following all links and printing all the words found into a file separated by space
https://github.com/ganzerli/crowler-for-words

c crowler curl-library https-client libcurl

Last synced: 5 months ago
JSON representation

Crowling domain, cut style, script and all HTML tags, gets words and links, following all links and printing all the words found into a file separated by space

• Host: GitHub
• URL: https://github.com/ganzerli/crowler-for-words
• Owner: ganzerli
• License: mit
• Created: 2023-01-07T18:01:58.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2024-12-06T17:22:32.000Z (over 1 year ago)
• Last Synced: 2025-06-28T20:11:35.909Z (12 months ago)
• Topics: c, crowler, curl-library, https-client, libcurl
• Language: C
• Homepage:
• Size: 16.6 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# All words from all links found in a domain are saved into a file

Crowler in C, lcurl is needed, you have to install also lcurl.

Crowling domain:

  -cuts all style

  -cuts all script

  -cuts all HTML tags 

  -gets all words

  -gets all links 

  -formats all links

  -calls all links 

  -saves all words found in the html into a file

Compile in linux with:

gcc -o crowler crowler.c -lcurl