https://github.com/kiliankoe/scpscraper

scrape contents of the SCP wiki
https://github.com/kiliankoe/scpscraper

Last synced: 24 days ago
JSON representation

scrape contents of the SCP wiki

• Host: GitHub
• URL: https://github.com/kiliankoe/scpscraper
• Owner: kiliankoe
• Created: 2014-12-17T00:37:36.000Z (almost 10 years ago)
• Default Branch: master
• Last Pushed: 2014-12-17T11:57:19.000Z (almost 10 years ago)
• Last Synced: 2024-06-11T23:17:11.902Z (5 months ago)
• Language: Python
• Homepage:
• Size: 125 KB
• Stars: 2
• Watchers: 3
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# scpscraper

A small scraper powered by Python and BeautifulSoup to scrape contents of the [SCP Wiki](http://www.scp-wiki.net).

It tries its best to scrape the page for a single SCP to return the following output:

```js

{

    "content": {

        "Description": "...",

        "Item #": "SCP-002",

        "Object Class": "Euclid",

        "Reference": "...",

        "Special Containment Procedures": "..."

    },

    "discussion": "http://www.scp-wiki.net/forum/t-76632/scp-002",

    "id": 2,

    "image": {

        "caption": "SCP-002 in its containment area",

        "src": "http://scp-wiki.wdfiles.com/local--files/scp-002/800px-SCP002.jpg"

    },

    "last_edited": 1409940932,

    "name": "The \"Living\" Room",

    "rating": 327,

    "revision": 45,

    "tags": [

        "alive",

        "euclid",

        "featured",

        "scp",

        "structure",

        "transfiguration"

    ]

}

```

It's quick to fail though, expect lots of errors.

There's also a small Flask server included that throws the above content back as JSON on `127.0.0.1:8080/scp/`.

Why? 'cause.