https://github.com/clarius/web
Official docs site for https://github.com/devlooped/Web
https://github.com/clarius/web
css html scraping xml
Last synced: 3 months ago
JSON representation
Official docs site for https://github.com/devlooped/Web
- Host: GitHub
- URL: https://github.com/clarius/web
- Owner: clarius
- License: mit
- Created: 2021-08-26T23:04:18.000Z (almost 5 years ago)
- Default Branch: gh-pages
- Last Pushed: 2026-03-12T21:04:54.000Z (4 months ago)
- Last Synced: 2026-03-13T00:18:04.770Z (4 months ago)
- Topics: css, html, scraping, xml
- Language: HTML
- Homepage:
- Size: 35.2 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
- Changelog: changelog.html
- License: license.txt
Awesome Lists containing this project
README
 HTML => XML + CSS with XLinq 🤘
============
[](osmfeula.txt)
[](https://github.com/devlooped/web/blob/main/license.txt)
[](https://www.nuget.org/packages/Devlooped.Web)
[](https://www.nuget.org/packages/Devlooped.Web)
## Open Source Maintenance Fee
To ensure the long-term sustainability of this project, users of this package who generate
revenue must pay an [Open Source Maintenance Fee](https://opensourcemaintenancefee.org).
While the source code is freely available under the terms of the [License](license.txt),
this package and other aspects of the project require [adherence to the Maintenance Fee](osmfeula.txt).
To pay the Maintenance Fee, [become a Sponsor](https://github.com/sponsors/devlooped) at the proper
OSMF tier. A single fee covers all of [Devlooped packages](https://www.nuget.org/profiles/Devlooped).
Read HTML as XML and query it with CSS over XLinq (or HtmlAgilityPack killer 😉).
Provides `HtmlDocument.Load` and `CssSelectElement(s)` extension methods
for `XDocument`/`XElement`.
No need to learn an entirely new object model for a page 🤘.
This makes it the most productive and lean library for web
scraping using the latest and greatest that .NET can offer.
# Usage
```csharp
using System.Xml.Linq;
using Devlooped.Web;
XDocument page = HtmlDocument.Load("page.html")
IEnumerable elements = page.CssSelectElements("div.menuitem");
XElement title = page.CssSelectElement("html head meta[name=title]");
```
By default, `HtmlDocument.Load` will skip non-content elements `script` and
`style`, turn all element names into lower case, and ignore all XML namespaces
(useful when loading XHTML, for example) for easier querying. These options
as well as granular whitespace handling can be configured using the overloads
receiving an `HtmlReaderSettings`.
The underlying parsing is performed by the amazing [SgmlReader](https://www.nuget.org/packages/Microsoft.Xml.SgmlReader)
library by Microsoft's [Chris Lovett](http://lovettsoftware.com/).
In addition, the following extension methods make it easier to work
with XML documents where you want to query with CSS or XPath without
having to deal with XML namespaces:
```csharp
using System.Xml;
using System.Xml.Linq;
using Devlooped.Web;
var doc = XDocument.Load("doc.xml")
// Will remove all xmlns declarations, and allow querying elements
// as if none had namespaces, returns the root element
XElement nons = doc.RemoveNamespaces();
// Alternatively, you can also ignore at the XmlReader level
using var reader = XmlReader.Create("doc.xml").IgnoreNamespaces();
doc = XDocument.Load(reader);
// Finally, you can also skip elements at the reader level
using var reader = XmlReader.Create("doc.xml").SkipElements("foo", "bar");
doc = XDocument.Load(reader);
```
## CSS
At the moment, supports the following CSS selector features:
- [Type selector](https://www.w3.org/TR/selectors-3/#type-selectors)
- [Universal selector](https://www.w3.org/TR/selectors-3/#universal-selector)
- [Attribute selector](https://www.w3.org/TR/selectors-3/#attribute-selectors)
- [Class selector](https://www.w3.org/TR/selectors-3/#class-html)
- [ID selector](https://www.w3.org/TR/selectors-3/#id-selectors)
- [Pseudo-classes](https://www.w3.org/TR/selectors-3/#pseudo-classes):
* [:checked](https://www.w3.org/TR/selectors-3/#checked)
* [:first-child](https://www.w3.org/TR/selectors-3/#first-child-pseudo)
* [:last-child](https://www.w3.org/TR/selectors-3/#last-child-pseudo)
* [:only-child](https://www.w3.org/TR/selectors-3/#only-child-pseudo)
* [:empty](https://www.w3.org/TR/selectors-3/#empty-pseudo)
* [:first-of-type](https://www.w3.org/TR/selectors-3/#first-of-type-pseudo)
* [:last-of-type](https://www.w3.org/TR/selectors-3/#last-of-type-pseudo)
* [:not(...)](https://www.w3.org/TR/selectors-3/#negation)
* [:nth-of-type(n)](https://www.w3.org/TR/selectors-3/#nth-of-type-pseudo)
* [:nth-child(n)](https://www.w3.org/TR/selectors-3/#nth-child-pseudo)
* [:has(...)](https://www.w3.org/TR/selectors-4/#has-pseudo)
And all [combinators](https://www.w3.org/TR/selectors-3/#combinators)
Non-CSS features:
- `text()` pseudo-attribute selector: selects the node text contents, as specified
in the [XPath](https://www.w3.org/TR/1999/REC-xpath-19991116/) `text()` location
path. Can be used instead of an attribute name selector, such as `div[text()=foo]`.
All [attribute value selectors](https://www.w3.org/TR/selectors-3/#attribute-selectors)
are also supported:
* `[text()=val]`: Represents an element whose text contents is exactly "val".
* `[text()~=val]`: Represents an element whose text contents is a whitespace-separated list of words,
one of which is exactly "val". If "val" contains whitespace, it will never represent anything (since the words
are separated by spaces). Also if "val" is the empty string, it will never represent anything.
* `[text()|=val]`: Represents an element whose text contents either being exactly "val" or
beginning with "val" immediately followed by "-" (U+002D).
* `[text()^=val]`: Represents an element whose text contents begins with the prefix "val".
If "val" is the empty string then the selector does not represent anything.
* `[text()$=val]`: Represents an element whose text contents ends with the suffix "val".
If "val" is the empty string then the selector does not represent anything.
* `[text()*=val]`: Represents an element whose text contents contains at least one instance of the
substring "val". If "val" is the empty string then the selector does not represent anything.
# Dogfooding
[](https://pkg.kzu.app/index.json)
[](https://github.com/devlooped/web/actions)
We also produce CI packages from branches and pull requests so you can dogfood builds as quickly as they are produced.
The CI feed is `https://pkg.kzu.app/index.json`.
The versioning scheme for packages is:
- PR builds: *42.42.42-pr*`[NUMBER]`
- Branch builds: *42.42.42-*`[BRANCH]`.`[COMMITS]`
# Sponsors
[](https://github.com/clarius)
[](https://github.com/MFB-Technologies-Inc)
[](https://github.com/khamza85)
[](https://github.com/sandrock)
[](https://github.com/drivenet)
[](https://github.com/Keflon)
[](https://github.com/tbolon)
[](https://github.com/kfrancis)
[](https://github.com/rbnswartz)
[](https://github.com/jfoshee)
[](https://github.com/Mrxx99)
[](https://github.com/eajhnsn1)
[](https://github.com/Jonathan-Hickey)
[](https://github.com/KenBonny)
[](https://github.com/SimonCropp)
[](https://github.com/agileworks-eu)
[](https://github.com/arsdragonfly)
[](https://github.com/vezel-dev)
[](https://github.com/ChilliCream)
[](https://github.com/4OTC)
[](https://github.com/DominicSchell)
[](https://github.com/adalon)
[](https://github.com/torutek)
[](https://github.com/mccaffers)
[](https://github.com/SeikaLogiciel)
[](https://github.com/wizardness)
[](https://github.com/latonz)
[](https://github.com/prime167)
[](https://github.com/sponsors/devlooped)
[Learn more about GitHub Sponsors](https://github.com/sponsors)