Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/thom4parisot/tld.js

JavaScript API to work easily with complex domain names, subdomains and well-known TLDs.
https://github.com/thom4parisot/tld.js

browser hostname javascript nodejs public-suffix-list subdomain tld tldextract uri validation-library

Last synced: about 2 months ago
JSON representation

JavaScript API to work easily with complex domain names, subdomains and well-known TLDs.

Host: GitHub
URL: https://github.com/thom4parisot/tld.js
Owner: thom4parisot
License: mit
Created: 2012-05-09T14:27:27.000Z (over 12 years ago)
Default Branch: master
Last Pushed: 2019-12-01T09:47:14.000Z (almost 5 years ago)
Last Synced: 2024-04-14T23:58:30.353Z (5 months ago)
Topics: browser, hostname, javascript, nodejs, public-suffix-list, subdomain, tld, tldextract, uri, validation-library
Language: JavaScript
Homepage: https://npmjs.com/tldjs
Size: 745 KB
Stars: 453
Watchers: 12
Forks: 54
Open Issues: 28
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE

Awesome Lists containing this project

README

        # tld.js [![Backers on Open Collective](https://opencollective.com/tldjs/backers/badge.svg)](#backers) [![Sponsors on Open Collective](https://opencollective.com/tldjs/sponsors/badge.svg)](#sponsors) [![Build Status][badge-ci]](http://travis-ci.org/oncletom/tld.js) ![][badge-downloads]

> `tld.js` is a Node.js module written in JavaScript to work against complex domain names, subdomains and well-known TLDs.

It answers with accuracy to questions like _what is `mail.google.com`'s domain?_,  _what is `a.b.ide.kyoto.jp`'s subdomain?_ and _is `https://big.data`'s TLD a well-known one?_.

`tld.js` [runs fast](#performances), is fully tested and is safe to use in the browser (with [browserify][], webpack and others). Because it relies on Mozilla's [public suffix list][], now is a good time to say _thank you_ Mozilla!

# Install

```bash

# Regular install

npm install --save tldjs

# You can update the list of well-known TLD during the install

npm install --save tldjs --tldjs-update-rules

```

The latter is useful if you significantly rely on an up-to-date list of TLDs. You can [list the recent changes][] ([changes Atom Feed][]) to get a better idea of what is going on in the Public Suffix world.

# Using It

```js

const {parse, tldExists} = require('tldjs');

// Checking only if TLD exists in URL or hostname

// First TLD exists; the second does not.

console.log(tldExists('https://www.bbc'));

console.log(tldExists('tld.unknown'));

// Retrieving hostname related informations of a given URL

parse('http://www.writethedocs.org/conf/eu/2017/');

```

👋 [Try it your browser to see how it works][interactive-example].


⬇️ Read the documentation _below_ to find out the available _functions_.

## `tldjs.parse()`

This methods returns handy **properties about a URL or a hostname**.

```js

const tldjs = require('tldjs');

tldjs.parse('https://spark-public.s3.amazonaws.com/dataanalysis/loansData.csv');

// { hostname: 'spark-public.s3.amazonaws.com',

//   isValid: true,

//   isIp: false,

//   tldExists: true,

//   publicSuffix: 's3.amazonaws.com',

//   domain: 'spark-public.s3.amazonaws.com',

//   subdomain: ''

// }

tldjs.parse('gopher://domain.unknown/');

// { hostname: 'domain.unknown',

//   isValid: true,

//   isIp: false,

//   tldExists: false,

//   publicSuffix: 'unknown',

//   domain: 'domain.unknown',

//   subdomain: ''

// }

tldjs.parse('https://192.168.0.0')

// { hostname: '192.168.0.0',

//   isValid: true,

//   isIp: true,

//   tldExists: false,

//   publicSuffix: null,

//   domain: null,

//   subdomain: null

// }

```

| Property Name | Type | |

| ---           | ---       | --- |

| `hostname`    | `String`  |   |

| `isValid`     | `Boolean` | Is the hostname valid according to the RFC?  |

| `tldExists`   | `Boolean` | Is the TLD well-known or not?  |

| `publicSuffix`| `String`  |   |

| `domain`      | `String`  |   |

| `subdomain`   | `String`  |   |

## Single purpose methods

These methods are shorthands if you want to retrieve only a single value.

### tldExists()

Checks if the TLD is _well-known_ for a given hostname — parseable with [`require('url').parse`][].

```javascript

const { tldExists } = tldjs;

tldExists('google.com');      // returns `true`

tldExists('google.local');    // returns `false` (not an explicit registered TLD)

tldExists('com');             // returns `true`

tldExists('uk');              // returns `true`

tldExists('co.uk');           // returns `true` (because `uk` is a valid TLD)

tldExists('amazon.fancy.uk'); // returns `true` (still because `uk` is a valid TLD)

tldExists('amazon.co.uk');    // returns `true` (still because `uk` is a valid TLD)

tldExists('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `true`

```

### getDomain()

Returns the fully qualified domain from a given string — parseable with [`require('url').parse`][].

```javascript

const { getDomain } = tldjs;

getDomain('google.com');        // returns `google.com`

getDomain('fr.google.com');     // returns `google.com`

getDomain('fr.google.google');  // returns `google.google`

getDomain('foo.google.co.uk');  // returns `google.co.uk`

getDomain('t.co');              // returns `t.co`

getDomain('fr.t.co');           // returns `t.co`

getDomain('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `example.co.uk`

```

### getSubdomain()

Returns the complete subdomain for a given string — parseable with [`require('url').parse`][].

```javascript

const { getSubdomain } = tldjs;

getSubdomain('google.com');             // returns ``

getSubdomain('fr.google.com');          // returns `fr`

getSubdomain('google.co.uk');           // returns ``

getSubdomain('foo.google.co.uk');       // returns `foo`

getSubdomain('moar.foo.google.co.uk');  // returns `moar.foo`

getSubdomain('t.co');                   // returns ``

getSubdomain('fr.t.co');                // returns `fr`

getSubdomain('https://user:[email protected]:443/some/path?and&query#hash'); // returns `secure`

```

### getPublicSuffix()

Returns the [public suffix][] for a given string — parseable with [`require('url').parse`][].

```javascript

const { getPublicSuffix } = tldjs;

getPublicSuffix('google.com');       // returns `com`

getPublicSuffix('fr.google.com');    // returns `com`

getPublicSuffix('google.co.uk');     // returns `co.uk`

getPublicSuffix('s3.amazonaws.com'); // returns `s3.amazonaws.com`

getPublicSuffix('tld.is.unknown');   // returns `unknown`

```

### isValidHostname()

Checks if the given string is a valid hostname according to [RFC 1035](https://tools.ietf.org/html/rfc1035).

It does not check if the TLD is _well-known_.

```javascript

const { isValidHostname } = tldjs;

isValidHostname('google.com');      // returns `true`

isValidHostname('.google.com');     // returns `false`

isValidHostname('my.fake.domain');  // returns `true`

isValidHostname('localhost');       // returns `false`

isValidHostname('https://user:[email protected]:8080/some/path?and&query#hash'); // returns `false`

isValidHostname('192.168.0.0')      // returns `true`

```

# Troubleshooting

## Retrieving subdomain of `localhost` and custom hostnames

`tld.js` methods `getDomain` and `getSubdomain` are designed to **work only with *known and valid* TLDs**.

This way, you can trust what a domain is.

`localhost` is a valid hostname but not a TLD. Although you can instanciate your own flavour of `tld.js` with *additional valid hosts*:

```js

const tldjs = require('tldjs');

tldjs.getDomain('localhost');           // returns null

tldjs.getSubdomain('vhost.localhost');  // returns null

const myTldjs = tldjs.fromUserSettings({

  validHosts: ['localhost']

});

myTldjs.getDomain('localhost');           // returns 'localhost'

myTldjs.getSubdomain('vhost.localhost');  // returns 'vhost'

```

## Updating the TLDs List

Many libraries offer a list of TLDs. But, are they up-to-date? And how to update them?

`tld.js` bundles a list of known TLDs but this list can become outdated.

This is especially true if the package have not been updated on npm for a while.

Hopefully for you, even if I'm flying over the world, if I've lost my Internet connection or even if

you do manage your own list, you can update it by yourself, painlessly.

How? By passing the `--tldjs-update-rules` to your `npm install` command:

```bash

# anytime you reinstall your project

npm install --tldjs-update-rules

# or if you add the dependency to your project

npm install --save tldjs --tldjs-update-rules

```

Open an issue to request an update of the bundled TLDs.

# Contributing

Provide a pull request (with tested code) to include your work in this main project.

Issues may be awaiting for help so feel free to give a hand, with code or ideas.

# Performances

`tld.js` is fast, but keep in mind that it might vary depending on your own

use-case. Because the library tried to be smart, the speed can be drastically

different depending on the input (it will be faster if you provide an already

cleaned hostname, compared to a random URL).

On an Intel i7-6600U (2,60-3,40 GHz):

## For already cleaned hostnames

| Methods           | ops/sec      |

| ---               | ---          |

| `isValidHostname` | ~`8,700,000` |

| `extractHostname` | ~`8,100,000` |

| `tldExists`       | ~`2,000,000` |

| `getPublicSuffix` | ~`1,130,000` |

| `getDomain`       | ~`1,000,000` |

| `getSubdomain`    | ~`1,000,000` |

| `parse`           | ~`850,000`   |

## For random URLs

| Methods           | ops/sec       |

| ---               | ---           |

| `isValidHostname` | ~`25,400,000` |

| `extractHostname` | ~`400,000`    |

| `tldExists`       | ~`310,000`    |

| `getPublicSuffix` | ~`240,000`    |

| `getDomain`       | ~`240,000`    |

| `getSubdomain`    | ~`240,000`    |

| `parse`           | ~`230,000`    |

You can measure the performance of `tld.js` on your hardware by running the following command:

```bash

npm run benchmark

```

_Notice_: if this is not fast enough for your use-case, keep in mind that you can

provide your own `extractHostname` function (which is the bottleneck in

this benchmark) to `tld.js`.

## Contributors

This project exists thanks to all the people who contribute. [[Contribute]](CONTRIBUTING.md).



## Backers

Thank you to all our backers! 🙏 [[Become a backer](https://opencollective.com/tldjs#backer)]



## Sponsors

Support this project by becoming a sponsor. Your logo will show up here with a link to your website. [[Become a sponsor](https://opencollective.com/tldjs#sponsor)]





















# License

[MIT License](LICENSE).

[badge-ci]: https://secure.travis-ci.org/oncletom/tld.js.svg?branch=master

[badge-downloads]: https://img.shields.io/npm/dm/tldjs.svg

[public suffix list]: https://publicsuffix.org/list/

[list the recent changes]: https://github.com/publicsuffix/list/commits/master

[changes Atom Feed]: https://github.com/publicsuffix/list/commits/master.atom

[browserify CDN]: https://wzrd.in/

[browserify]: http://browserify.org/

[interactive-example]: https://runkit.com/oncletom/tld.js-runkit-example

[`require('url').parse`]: https://nodejs.org/api/url.html#url_url_parse_urlstring_parsequerystring_slashesdenotehost

[public suffix]: https://publicsuffix.org/learn/