Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/cyotek/httpcrawlertestsite

A small PHP based website for testing the essentials of a web crawler
https://github.com/cyotek/httpcrawlertestsite

css demo hacktoberfest http php

Last synced: 9 days ago
JSON representation

A small PHP based website for testing the essentials of a web crawler

Awesome Lists containing this project

README

        

# HTTP Crawler Test Site

This website was created to replace the original static website
used for testing [WebCopy][2] features, expanded to cover
additional scenarios that the original didn't handle and to
ensure a lean site that doesn't slow down integration testing
unnecessarily. Of course, writing better tests is just as
important! But every little helps, over time it has been used to
find bugs in WebCopy's crawl engine and to help ensure
regressions don't make it back in.

[Cyotek][1] is a .NET shop and usually all our products are
written in C# using Visual Studio. This time however we decided
to keep it simple and this site has been written with PHP,
originally using WebMatrix and currently Visual Studio Code. As
we aren't PHP coders and don't have much of a clue on best
practices for writing PHP or even most of the language the code
quality is probably less than stellar. Although it was fairly
fun to write as it doesn't need an ever increasing amount of
binary dependencies, no assembly bindings or version woes... it
just works.

You can view a live demo running at
[https://demo.cyotek.com][3]. To run the source you should only
need PHP 5.3 or above and some form of web server. This site has
been tested using IIS Express, IIS 8.5, IIS 10 and Apache
running on some variant of Linux.

## Acknowledgements

CSS has been taken from [Primer][4] by [GitHub][5].

Various bits of PHP were lifted from the [PHP user manual][6].

The fav icon is derived from [Material Core Check][7].

Markdown processing is handled by [PHP Markdown Lib][8].

Some text has been borrowed from [Wikipedia][9] and the [Mozilla
Developer Network][10].

The `srcset` demonstration was derived from a [WebKit.org
demo][11].

The skyline images were generated by the imaginatively titled
[Skyline Generator][12].

The photograph of St Paul's Cathedral in London taken by [Vadim
Sherbakov][13].

The photograph of [Flamborough Head Lighthouse][lighthouse]
copyright (c) 2021 Richard James Moss.

Generic Avatar graphic by [WingTillDie][14]

Formatting of the raw code files is being gradually performed
via [VSCodeBeautify][15]

[1]: https://www.cyotek.com/
[2]: https://www.cyotek.com/cyotek-webcopy
[3]: https://demo.cyotek.com
[4]: http://primercss.io/
[5]: https://github.com/
[6]: http://php.net/manual/en/index.php
[7]: https://www.iconfinder.com/icons/326568/check_circle_outline_icon#size=512
[8]: https://michelf.ca/projects/php-markdown/
[9]: http://en.wikipedia.org/
[10]: https://developer.mozilla.org/
[11]: http://www.webkit.org/demos/srcset/
[12]: https://github.com/cyotek/SkylineGenerator
[13]: https://unsplash.com/photos/xS_RzdD5CFE
[14]: https://pixabay.com/en/avatar-icon-placeholder-1577909/
[15]: https://github.com/HookyQR/VSCodeBeautify
[lighthouse]: https://www.trinityhouse.co.uk/lighthouse-visitor-centres/flamborough-lighthouse-visitor-centre