https://github.com/zntfdr/Selenops

A Swift Web Crawler 🕷
https://github.com/zntfdr/Selenops

command-line-tool crawler scripting swift web

Last synced: about 1 month ago
JSON representation

A Swift Web Crawler 🕷

• Host: GitHub
• URL: https://github.com/zntfdr/Selenops
• Owner: zntfdr
• License: mit
• Created: 2017-04-04T12:20:43.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2021-01-17T15:15:58.000Z (over 3 years ago)
• Last Synced: 2024-07-30T20:51:13.186Z (about 2 months ago)
• Topics: command-line-tool, crawler, scripting, swift, web
• Language: Swift
• Homepage:
• Size: 520 KB
• Stars: 252
• Watchers: 5
• Forks: 18
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Selenops



    

    


    

    

        

    

     

    

        

    



Welcome to **Selenops**, a Swift Web Crawler.

Given a webpage url and a word to search, Selenops will look for said word in that page, and all pages linked in there, recursively.

## Usage

```swift

import Selenops

// Initialize crawler.

let crawler = Crawler(

  startURL: URL(string: "https://fivestars.blog/")!,

  maximumPagesToVisit: 10,

  wordToSearch: "Swift" // Specify the word to search here.

)

// Set delegate.

crawler.delegate = ... // Needs to conform to CrawlerDelegate

// Start crawling!

crawler.start()

```

The `crawler` delegate will get information such as visited webpages, and in which webpages the specified word has been found.

## Installation

Selenops is distributed via the [Swift Package Manager](https://swift.org/package-manager):  

- to use it into an app, follow [this tutorial](https://developer.apple.com/documentation/swift_packages/adding_package_dependencies_to_your_app) and use this repository URL: `https://github.com/zntfdr/Selenops.git`.

- to use it in a package, add it as a dependency in your `Package.swift`:

```swift

 let package = Package(

     ...

     dependencies: [

         .package(url: "https://github.com/zntfdr/Selenops.git", from: "2.0.0")

     ],

     targets: [

        .target(

            ...

            dependencies: ["Selenops"])

     ],

     ...

 )

```

  ...and then use `import Selenops` whenever necessary.

## Command line tool



    

Selenops also comes with a command line tool that showcases its functionality.

To install it, clone the project and run make:

```shell

$ git clone https://github.com/zntfdr/Selenops.git

$ cd Selenops

$ make

```

## Credits

Selenops was built by [Federico Zanetello](https://twitter.com/zntfdr) as an [example of a Swift script][selenopsArticle].

## Contributions and Support

All users are welcome and encouraged to become active participants in the project continued development — by fixing any bug that they encounter, or by improving the documentation wherever it’s found to be lacking.

If you'd like to make a change, please [open a Pull Request](https://github.com/zntfdr/Selenops/pull/new), even if it just contains a draft of the changes you’re planning, or a test that reproduces an issue.

Thank you and please enjoy using **Selenops**!

[selenopsArticle]: https://www.fivestars.blog/code/build-web-crawler-swift.html