https://github.com/yokawasa/scrapy-azuresearch-crawler-samples

Scrapy as a Web Crawler for Azure Search Samples
https://github.com/yokawasa/scrapy-azuresearch-crawler-samples

azure azure-search crawler python python3 scrapy search

Last synced: about 2 months ago
JSON representation

Scrapy as a Web Crawler for Azure Search Samples

• Host: GitHub
• URL: https://github.com/yokawasa/scrapy-azuresearch-crawler-samples
• Owner: yokawasa
• License: mit
• Created: 2017-08-14T03:49:42.000Z (almost 8 years ago)
• Default Branch: master
• Last Pushed: 2017-08-16T12:20:15.000Z (almost 8 years ago)
• Last Synced: 2025-03-21T23:34:04.681Z (2 months ago)
• Topics: azure, azure-search, crawler, python, python3, scrapy, search
• Language: Python
• Homepage:
• Size: 387 KB
• Stars: 22
• Watchers: 5
• Forks: 7
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# scrapy-azuresearch-crawler-samples

Scrapy as a Web Crawler for Azure Search Sample

![](https://github.com/yokawasa/scrapy-azuresearch-crawler-samples/raw/master/img/scrapy-azuresearch-arch.png)

## Samples

* [quotes](https://github.com/yokawasa/scrapy-azuresearch-crawler-samples/tree/master/quotes): Web Scraping [Quotes to Scrape](http://quotes.toscrape.com/)'s quote list and indexing them with Azure Search

* [craigslist](https://github.com/yokawasa/scrapy-azuresearch-crawler-samples/tree/master/craigslist): Web Scraping [Craigslist](https://tokyo.craigslist.jp/)'s Jobs in Tokyo and indexing them with Azure Search

* [unofficialism](https://github.com/yokawasa/scrapy-azuresearch-crawler-samples/tree/master/unofficialism): Web Scraping [Unofficialism](http://unofficialism.info/)'s all blog articles and indexing them with Azure Search

## Tips

* [Scrapy.org](https://scrapy.org/)

* [Scrapy Totorial](https://docs.scrapy.org/en/latest/intro/tutorial.html)

* [Welcome to Azure Cosmos DB](https://docs.microsoft.com/en-us/azure/cosmos-db/introduction)

* [What is Azure Search?](https://docs.microsoft.com/en-us/azure/search/search-what-is-azure-search)

* [Azure Search Indexer Overview](https://docs.microsoft.com/en-us/azure/search/search-indexer-overview)

* [Connecting Cosmos DB with Azure Search using indexers](https://docs.microsoft.com/en-us/azure/search/search-howto-index-documentdb)

## Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/yokawasa/scrapy-azuresearch-crawler-samples