{"id":37079357,"url":"https://github.com/micahcochran/scrape-schema-recipe","last_synced_at":"2026-01-14T09:36:22.166Z","repository":{"id":48189970,"uuid":"139584077","full_name":"micahcochran/scrape-schema-recipe","owner":"micahcochran","description":"Scrapes recipes from HTML https://schema.org/Recipe (Microdata/JSON-LD) into Python dictionaries.","archived":false,"fork":false,"pushed_at":"2023-11-19T12:19:33.000Z","size":587,"stargazers_count":57,"open_issues_count":6,"forks_count":10,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-29T13:27:15.599Z","etag":null,"topics":["recipes","schema-org"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/micahcochran.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-07-03T13:01:03.000Z","updated_at":"2025-01-28T18:53:23.000Z","dependencies_parsed_at":"2022-09-26T19:33:38.941Z","dependency_job_id":null,"html_url":"https://github.com/micahcochran/scrape-schema-recipe","commit_stats":{"total_commits":27,"total_committers":4,"mean_commits":6.75,"dds":0.2962962962962963,"last_synced_commit":"73ea461ad0c1ed256e742a0f285eac36f3db15df"},"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/micahcochran/scrape-schema-recipe","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/micahcochran%2Fscrape-schema-recipe","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/micahcochran%2Fscrape-schema-recipe/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/micahcochran%2Fscrape-schema-recipe/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/micahcochran%2Fscrape-schema-recipe/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/micahcochran","download_url":"https://codeload.github.com/micahcochran/scrape-schema-recipe/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/micahcochran%2Fscrape-schema-recipe/sbom","scorecard":{"id":640614,"data":{"date":"2025-08-11","repo":{"name":"github.com/micahcochran/scrape-schema-recipe","commit":"5bf46b406b8816e3015fe63317f2a3d107bc05c9"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":2.8,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Code-Review","score":0,"reason":"Found 2/27 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/python-package.yml:1","Warn: no topLevel permission defined: .github/workflows/python-publish.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":1,"reason":"dependency not pinned by hash detected -- score normalized to 1","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-package.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/micahcochran/scrape-schema-recipe/python-package.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-package.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/micahcochran/scrape-schema-recipe/python-package.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/micahcochran/scrape-schema-recipe/python-publish.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/python-publish.yml:27: update your workflow using https://app.stepsecurity.io/secureworkflow/micahcochran/scrape-schema-recipe/python-publish.yml/master?enable=pin","Warn: pipCommand not pinned by hash: .github/workflows/python-package.yml:25","Warn: pipCommand not pinned by hash: .github/workflows/python-package.yml:26","Warn: pipCommand not pinned by hash: .github/workflows/python-package.yml:27","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:32","Warn: pipCommand not pinned by hash: .github/workflows/python-publish.yml:33","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned","Info:   1 out of   1 third-party GitHubAction dependencies pinned","Info:   0 out of   5 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Vulnerabilities","score":2,"reason":"8 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2014-14 / GHSA-652x-xj99-gmcc","Warn: Project is vulnerable to: GHSA-9hjg-9r4m-mvj7","Warn: Project is vulnerable to: GHSA-9wx4-h78v-vm56","Warn: Project is vulnerable to: PYSEC-2014-13 / GHSA-cfj3-7x9c-4p3h","Warn: Project is vulnerable to: PYSEC-2018-28 / GHSA-x84v-xcm2-53pg","Warn: Project is vulnerable to: PYSEC-2025-49 / GHSA-5rjg-fvgr-3xxf","Warn: Project is vulnerable to: GHSA-cx63-2mw6-8hw5","Warn: Project is vulnerable to: PYSEC-2022-43012 / GHSA-r9hx-vwmv-q579"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 19 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-21T10:36:40.116Z","repository_id":48189970,"created_at":"2025-08-21T10:36:40.116Z","updated_at":"2025-08-21T10:36:40.116Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28416120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T08:38:59.149Z","status":"ssl_error","status_checked_at":"2026-01-14T08:38:43.588Z","response_time":107,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["recipes","schema-org"],"created_at":"2026-01-14T09:36:21.567Z","updated_at":"2026-01-14T09:36:22.159Z","avatar_url":"https://github.com/micahcochran.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# scrape-schema-recipe\n[![PyPI](https://img.shields.io/pypi/v/scrape-schema-recipe)](https://pypi.org/project/scrape-schema-recipe/)\n![Build Status](https://github.com/micahcochran/scrape-schema-recipe/actions/workflows/python-package.yml/badge.svg)\n[![Downloads](https://pepy.tech/badge/scrape-schema-recipe)](https://pepy.tech/project/scrape-schema-recipe)\n\nScrapes recipes from HTML https://schema.org/Recipe (Microdata/JSON-LD) into Python dictionaries.\n\n\n## Install\n\n```\npip install scrape-schema-recipe\n```\n\n## Requirements\n\nPython version 3.6+\n\nThis library relies heavily upon [extruct](https://github.com/scrapinghub/extruct).\n\nOther requirements: \n* isodate (\u003e=0.5.1)\n* requests\n\n## Online Example\n\n```python\n\u003e\u003e\u003e import scrape_schema_recipe\n\n\u003e\u003e\u003e url = 'https://www.foodnetwork.com/recipes/alton-brown/honey-mustard-dressing-recipe-1939031'\n\u003e\u003e\u003e recipe_list = scrape_schema_recipe.scrape_url(url, python_objects=True)\n\u003e\u003e\u003e len(recipe_list)\n1\n\u003e\u003e\u003e recipe = recipe_list[0]\n\n# Name of the recipe\n\u003e\u003e\u003e recipe['name']\n'Honey Mustard Dressing'\n\n# List of the Ingredients\n\u003e\u003e\u003e recipe['recipeIngredient']\n['5 tablespoons medium body honey (sourwood is nice)',\n '3 tablespoons smooth Dijon mustard',\n '2 tablespoons rice wine vinegar']\n\n# List of the Instructions\n\u003e\u003e\u003e recipe['recipeInstructions']\n['Combine all ingredients in a bowl and whisk until smooth. Serve as a dressing or a dip.']\n\n# Author\n\u003e\u003e\u003e recipe['author']\n[{'@type': 'Person',\n  'name': 'Alton Brown',\n  'url': 'https://www.foodnetwork.com/profiles/talent/alton-brown'}]\n```\n'@type': 'Person' is a [https://schema.org/Person](https://schema.org/Person) object\n\n\n```python\n# Preparation Time\n\u003e\u003e\u003e recipe['prepTime']\ndatetime.timedelta(0, 300)\n\n# The library pendulum can give you something a little easier to read.\n\u003e\u003e\u003e import pendulum\n\n# for pendulum version 1.0\n\u003e\u003e\u003e pendulum.Interval.instanceof(recipe['prepTime'])\n\u003cInterval [5 minutes]\u003e\n\n# for version 2.0 of pendulum\n\u003e\u003e\u003e pendulum.Duration(seconds=recipe['prepTime'].total_seconds())\n\u003cDuration [5 minutes]\u003e\n```\n\nIf `python_objects` is set to `False`, this would return the string ISO8611 representation of the duration, `'PT5M'`\n\n[pendulum's library website](https://pendulum.eustace.io/).\n\n\n```python\n# Publication date\n\u003e\u003e\u003e recipe['datePublished']\ndatetime.datetime(2016, 11, 13, 21, 5, 50, 518000, tzinfo=\u003cFixedOffset '-05:00'\u003e)\n\n\u003e\u003e\u003e str(recipe['datePublished'])\n'2016-11-13 21:05:50.518000-05:00'\n\n# Identifying this is http://schema.org/Recipe data (in LD-JSON format)\n \u003e\u003e\u003e recipe['@context'], recipe['@type']\n('http://schema.org', 'Recipe')\n\n# Content's URL\n\u003e\u003e\u003e recipe['url']\n'https://www.foodnetwork.com/recipes/alton-brown/honey-mustard-dressing-recipe-1939031'\n\n# all the keys in this dictionary\n\u003e\u003e\u003e recipe.keys()\ndict_keys(['recipeYield', 'totalTime', 'dateModified', 'url', '@context', 'name', 'publisher', 'prepTime', 'datePublished', 'recipeIngredient', '@type', 'recipeInstructions', 'author', 'mainEntityOfPage', 'aggregateRating', 'recipeCategory', 'image', 'headline', 'review'])\n```\n\n## Example from a File (alternative representations)\n\nAlso works with locally saved [HTML example file](/test_data/google-recipe-example.html).\n```python\n\u003e\u003e\u003e filelocation = 'test_data/google-recipe-example.html'\n\u003e\u003e\u003e recipe_list = scrape_schema_recipe.scrape(filelocation, python_objects=True)\n\u003e\u003e\u003e recipe = recipe_list[0]\n\n\u003e\u003e\u003e recipe['name']\n'Party Coffee Cake'\n\n\u003e\u003e\u003e repcipe['datePublished']\ndatetime.date(2018, 3, 10)\n\n# Recipe Instructions using the HowToStep\n\u003e\u003e\u003e recipe['recipeInstructions']\n[{'@type': 'HowToStep',\n  'text': 'Preheat the oven to 350 degrees F. Grease and flour a 9x9 inch pan.'},\n {'@type': 'HowToStep',\n  'text': 'In a large bowl, combine flour, sugar, baking powder, and salt.'},\n {'@type': 'HowToStep', 'text': 'Mix in the butter, eggs, and milk.'},\n {'@type': 'HowToStep', 'text': 'Spread into the prepared pan.'},\n {'@type': 'HowToStep', 'text': 'Bake for 30 to 35 minutes, or until firm.'},\n {'@type': 'HowToStep', 'text': 'Allow to cool.'}]\n\n```\n\n\n## What Happens When Things Go Wrong\n\nIf there aren't any http://schema.org/Recipe formatted recipes on the site.\n```python\n\u003e\u003e\u003e url = 'https://www.google.com'\n\u003e\u003e\u003e recipe_list = scrape_schema_recipe.scrape(url, python_objects=True)\n\n\u003e\u003e\u003e len(recipe_list)\n0\n```\n\nSome websites will cause an `HTTPError`.\n\nYou may get around a 403 - Forbidden Errror by putting in an alternative user-agent\nvia the variable `user_agent_str`.\n\n\n## Functions\n\n* `load()` - load HTML schema.org/Recipe structured data from a file or file-like object\n* `loads()` - loads HTML schema.org/Recipe structured data from a string\n* `scrape_url()` - scrape a URL for HTML schema.org/Recipe structured data \n* `scrape()` - load HTML schema.org/Recipe structured data from a file, file-like object, string, or URL\n\n```\n    Parameters\n    ----------\n    location : string or file-like object\n        A url, filename, or text_string of HTML, or a file-like object.\n\n    python_objects : bool, list, or tuple  (optional)\n        when True it translates certain data types into python objects\n          dates into datetime.date, datetimes into datetime.datetimes,\n          durations as dateime.timedelta.\n        when set to a list or tuple only converts types specified to\n          python objects:\n            * when set to either [dateime.date] or [datetime.datetimes] either will\n              convert dates.\n            * when set to [datetime.timedelta] durations will be converted\n        when False no conversion is performed\n        (defaults to False)\n\n    nonstandard_attrs : bool, optional\n        when True it adds nonstandard (for schema.org/Recipe) attributes to the\n        resulting dictionaries, that are outside the specification such as:\n            '_format' is either 'json-ld' or 'microdata' (how schema.org/Recipe was encoded into HTML)\n            '_source_url' is the source url, when 'url' has already been defined as another value\n        (defaults to False)\n\n    migrate_old_schema : bool, optional\n        when True it migrates the schema from older version to current version\n        (defaults to True)\n\n    user_agent_str : string, optional  ***only for scrape_url() and scrape()***\n        overide the user_agent_string with this value.\n        (defaults to None)\n\n    Returns\n    -------\n    list\n        a list of dictionaries in the style of schema.org/Recipe JSON-LD\n        no results - an empty list will be returned\n```\n\nThese are also available with `help()` in the python console.\n\n## Example function\nThe `example_output()` function gives quick access to data for prototyping and debugging.\nIt accepts the same parameters as load(), but the first parameter, `name`, is different.\n\n```python\n\u003e\u003e\u003e from scrape_schema_recipe import example_names, example_output\n\n\u003e\u003e\u003e example_names\n('irish-coffee', 'google', 'taco-salad', 'tart', 'tea-cake', 'truffles')\n\n\u003e\u003e\u003e recipes = example_output('truffles')\n\u003e\u003e\u003e recipes[0]['name']\n'Rum \u0026 Tonka Bean Dark Chocolate Truffles'\n```\n\n\n## Files\n\nLicense: Apache 2.0   see [LICENSE](LICENSE)\n\nTest data attribution and licensing: [ATTRIBUTION.md](ATTRIBUTION.md)\n\n## Development\n\nThe unit testing must be run from a copy of the repository folder. Unit testing can be run by:\n```\nschema-recipe-scraper$ python3 test_scrape.py\n```\n\nmypy is used for static type checking\n\nfrom the project directory:\n```\n schema-recipe-scraper$ mypy schema_recipe_scraper/scrape.py\n```\n\nIf you run mypy from another directory the `--ignore-missing-imports` flag will need to be added,\nthus `$ mypy --ignore-missing-imports scrape.py`\n\n`--ignore-missing-imports` flag is used because most libraries don't have static typing information contained\nin their own code or typeshed.\n\n## Reference Documentation\nHere are some references for how schema.org/Recipe *should* be structured:\n\n* [https://schema.org/Recipe](https://schema.org/Recipe) - official specification\n* [Recipe Google Search Guide](https://developers.google.com/search/docs/data-types/recipe) - material teaching developers how to use the schema (with emphasis on how structured data impacts search results)\n\n\n## Other Similar Python Libraries\n\n* [recipe_scrapers](https://github.com/hhursev/recipe-scrapers) - library scrapes\nrecipes by using extruct to scrape the schema.org/Recipe  format or HTML tags with BeautifulSoup.\nThe library has drivers that support many different websites that further parse the information.\nThis is a solid alternative to schema-recipe-scraper that is focused on a different kind of simplicity.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicahcochran%2Fscrape-schema-recipe","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmicahcochran%2Fscrape-schema-recipe","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmicahcochran%2Fscrape-schema-recipe/lists"}