Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ektotv/xmltv
An extremely fast XMLTV parser and generator for Node and the browser.
https://github.com/ektotv/xmltv
epg parser xmltv
Last synced: about 2 months ago
JSON representation
An extremely fast XMLTV parser and generator for Node and the browser.
- Host: GitHub
- URL: https://github.com/ektotv/xmltv
- Owner: ektotv
- License: mit
- Created: 2023-04-05T19:40:08.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2023-11-27T14:55:10.000Z (about 1 year ago)
- Last Synced: 2024-11-06T09:18:57.058Z (2 months ago)
- Topics: epg, parser, xmltv
- Language: TypeScript
- Homepage: https://npmjs.com/package/@iptv/xmltv
- Size: 646 KB
- Stars: 19
- Watchers: 2
- Forks: 3
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE.md
Awesome Lists containing this project
README
# @iptv/xmltv
An extremely fast XMLTV parser and generator for Node and the browser.
Lightweight, dependency-free, and easy to use.
---
[](https://www.npmjs.com/package/@iptv/xmltv)
[](https://github.com/ektotv/xmltv/actions/workflows/ci.yml)
[](https://github.com/ektotv/xmltv/tree/main/tests)
[](LICENSE.md)
---
The average XMLTV file is pretty large, smaller ones around 20MB and larger ones exceeding hundreds of megabytes. The purpose of this library is to facilitate rapid parsing and creation of these files.
---
## âĻ Features
- **Extremely** fast XMLTV parser and generator
- Lightweight (3.74 kB gzipped)
- No dependencies
- ESM and CommonJS support
- Supports Node and the browser
- Types that define an Xmltv interface, which implements the [XMLTV DTD](https://github.com/XMLTV/xmltv/blob/master/xmltv.dtd) as closely as possible
- Supports **all** XMLTV elements and attributes
- Did I mention [it's fast](#-performance)?
---
## ðĨ Installation
To install this library, use the following command:
```bash
# pnpm
pnpm add @iptv/xmltv
# npm
npm install @iptv/xmltv
# yarn
yarn add @iptv/xmltv
```
---
## ð§ Usage
To use this library in your project, first import the functions you need:
```typescript
import { parseXmltv, writeXmltv } from '@iptv/xmltv';
```
Then, you can parse an XMLTV file and receive back an `Xmltv` object:
Example XMLTV File
Examples will be based on this XMLTV file, it can be found in the [tests/fixtures](tests/fixtures) directory.
```xml
Channel One
https://example.com/channel_one
https://example.com/channel_one_alternate
Channel Two: Minimum valid channel
Programme One
Pilot
This programme entry showcases all possible features of the DTD
Samuel Jones
David Thompson
Ryan Lee https://www.example.com/xxx.jpg
https://www.example.com/person/204
Samuel Jones
William Brown
Emily Davis
Max Wright
Nora Peterson
Amanda Johnson
James Wilson
Lucas Martin
Emily Parker
Oliver Nelson
20220401000000 +0000
Crime
Drama
methamphetamine
cancer
English
French
60
https://example.com/programme_one
https://example.com/programme_one_2
US
S01E01
1.1.
yes
no
16:9
HDTV
yes
Dolby Digital
First time on British TV
Last time on this channel
English
Spanish
15
4/5
This is a fantastic show!
https://example.com/programme_one_review
https://tvdb.com/programme_one_poster_1.jpg
https://tvdb.com/programme_one_backdrop_3.jpg
Programme Two: The minimum valid programme
```
```typescript
const xml = `...`; // XMLTV file contents
const xmltv: Xmltv = parseXmltv(xml);
const programmes: XmltvProgramme[] = xmltv.programmes;
const channels: XmltvChannel[] = xmltv.channels;
```
Example output of `parseXmltv()`
```typescript
{
channels: [
{
displayName: [
{
_value: "Channel One",
lang: "en",
},
],
icon: [
{
height: 100,
src: "https://example.com/channel_one_icon.jpg",
width: 100,
},
],
id: "channel_one",
url: [
{
_value: "https://example.com/channel_one",
system: "example",
},
{
_value: "https://example.com/channel_one_alternate",
system: "other_system",
},
],
},
{
displayName: [
{
_value: "Channel Two: Minimum valid channel",
},
],
id: "channel_two",
},
],
date: new Date("2022-04-01T00:00:00.000Z"),
generatorInfoName: "Example Generator",
generatorInfoUrl: "example generator url",
programmes: [
{
audio: {
present: true,
stereo: "Dolby Digital",
},
category: [
{
_value: "Crime",
lang: "en",
},
{
_value: "Drama",
lang: "en",
},
],
channel: "channel_one",
clumpidx: "0/1",
country: {
_value: "US",
},
credits: {
actor: [
{
_value: "David Thompson",
role: "Walter Johnson",
},
{
_value: "Ryan Lee",
guest: true,
image: [
{
_value: "https://www.example.com/xxx.jpg",
type: "person",
},
],
role: "Karl James",
url: [
{
_value: "https://www.example.com/person/204",
system: "moviedb",
},
],
},
],
adapter: [
{
_value: "William Brown",
},
],
commentator: [
{
_value: "James Wilson",
},
],
composer: [
{
_value: "Max Wright",
},
],
director: [
{
_value: "Samuel Jones",
},
],
editor: [
{
_value: "Nora Peterson",
},
],
guest: [
{
_value: "Lucas Martin",
},
{
_value: "Emily Parker",
},
{
_value: "Oliver Nelson",
},
],
presenter: [
{
_value: "Amanda Johnson",
},
],
producer: [
{
_value: "Emily Davis",
},
],
writer: [
{
_value: "Samuel Jones",
},
],
},
date: new Date("2022-04-01T00:00:00.000Z"),
desc: [
{
_value:
"This programme entry showcases all possible features of the DTD",
lang: "en",
},
],
episodeNum: [
{
_value: "S01E01",
system: "onscreen",
},
{
_value: "1.1.",
system: "xmltv_ns",
},
],
icon: [
{
height: 100,
src: "https://example.com/programme_one_icon.jpg",
width: 100,
},
],
image: [
{
_value: "https://tvdb.com/programme_one_poster_1.jpg",
orient: "P",
size: 1,
system: "tvdb",
type: "poster",
},
{
_value: "https://tvdb.com/programme_one_backdrop_3.jpg",
orient: "L",
size: 3,
system: "tvdb",
type: "backdrop",
},
],
keyword: [
{
_value: "methamphetamine",
lang: "en",
},
{
_value: "cancer",
lang: "en",
},
],
language: {
_value: "English",
},
lastChance: {
_value: "Last time on this channel",
lang: "en",
},
length: {
_value: 60,
units: "minutes",
},
new: true,
origLanguage: {
_value: "French",
lang: "en",
},
pdcStart: new Date("2022-03-31T18:00:00.000Z"),
premiere: {
_value: "First time on British TV",
},
previouslyShown: {
channel: "channel_two",
start: new Date("2022-03-31T18:00:00.000Z"),
},
rating: [
{
icon: [
{
src: "15_symbol.png",
},
],
system: "BBFC",
value: "15",
},
],
review: [
{
_value: "This is a fantastic show!",
lang: "en",
reviewer: "Joe Bloggs",
source: "Rotten Tomatoes",
type: "text",
},
{
_value: "https://example.com/programme_one_review",
lang: "en",
reviewer: "Joe Bloggs",
source: "Rotten Tomatoes",
type: "url",
},
],
showview: "12345",
starRating: [
{
icon: [
{
src: "stars.png",
},
],
system: "TV Guide",
value: "4/5",
},
],
start: new Date("2022-03-31T18:00:00.000Z"),
stop: new Date("2022-03-31T19:00:00.000Z"),
subTitle: [
{
_value: "Pilot",
lang: "en",
},
],
subtitles: [
{
language: {
_value: "English",
},
type: "teletext",
},
{
language: {
_value: "Spanish",
lang: "en",
},
type: "onscreen",
},
],
title: [
{
_value: "Programme One",
lang: "en",
},
],
url: [
{
_value: "https://example.com/programme_one",
system: "tvdb",
},
{
_value: "https://example.com/programme_one_2",
},
],
video: {
aspect: "16:9",
colour: false,
present: true,
quality: "HDTV",
},
videoplus: "67890",
vpsStart: new Date("2022-03-31T18:00:00.000Z"),
},
{
channel: "channel_one",
start: new Date("2022-03-31T18:00:00.000Z"),
title: [
{
_value: "Programme Two: The minimum valid programme",
},
],
},
],
sourceDataUrl: "example.com/a",
sourceInfoName: "example",
sourceInfoUrl: "example.com",
};
```
You can also generate an XMLTV file from an `Xmltv` tree:
```typescript
const xml = writeXmltv(xmltvObject);
console.log(xml); // ...
```
Or from a XMLTV DOM tree:
```typescript
const xml = writeXmltv(xmltvDom, { fromDom: true });
console.log(xml); // ...
```
If you want to go even faster you can parse the file into a DOM tree and then traverse it yourself:
```typescript
const xml = '...'; // XMLTV file contents
const parsed = parseXmltv(xml, { asDom: true });
// or import { parser } from '@iptv/xmltv';
// const parsed = parser(xml);
// `parsed` is now a list of XmltvNode objects
// which can be traversed using the `children` property
// and attributes can be accessed using the `attributes` property
// (see the XmltvNode interface for more details)
// this is not the most efficient way to do this, but it's a good example
const programmes = parsed
.find((node) => {
// find the tag
return node.tagName === 'tv';
})
.children.filter((node) => {
// filter it's children for tags
return node.tagName === 'programme';
})
.map((programme) => {
// return a Programme object for each tag
return {
title: programme.children.find((t) => t.tagName === 'title').children[0],
start: new Date(programme.attributes.start),
stop: new Date(programme.attributes.stop),
channel: programme.attributes.channel,
};
});
```
---
## ⥠Performance
This library has been optimized for parsing and generating XMLTV files quickly and efficiently. In my benchmarks, it performs better than other popular XMLTV libraries, including [epg-parser](https://www.npmjs.com/package/epg-parser) and [xmltv](https://www.npmjs.com/package/xmltv). It also beats [fast-xml-parser](https://www.npmjs.com/package/fast-xml-parser).
The speed of this library is down to the implementation of the XML DOM tree parser [tXml](https://www.npmjs.com/package/txml). To learn more about the optimisations made by tXml please read [this blog post](https://tnickel.de/2020/08/30/2020-08-how-the-fastest-xml-parser-is-build/) by [@TobiasNickel](https://github.com/TobiasNickel).
### Benchmarks
#### Parsing XMLTV file (c1-p1.xml)
Library
Ops/sec
@iptv/xmltv
1,094,286
ðĒ
@iptv/xmltv (DOM only)
1,246,439
fast-xml-parser
426,947
ðī
epg-parser
96,468
xmltv
14,750
#### Writing XMLTV file (c1-p1.xml)
Library
Ops/sec
ðĒ
@iptv/xmltv
2,920,103
ðī
fast-xml-parser
1,257,938
#### Time spent parsing different XMLTV files
Channels
1
100
100
100
100
100
1
Programmes
0
100
10,000
100,000
250,000
500,000
1,000,000
ðĒ
@iptv/xmltv
~60 Ξs
~1.24 ms
~39 ms
~342 ms
~980 ms
~2.15 s
~5.37 s
ðī
epg-parser
~362 Ξs
~9.83 ms
~256 ms
~2.25 s
~6.26 s
~13 s
~28 s
I used nanobench to get the above times. Nanobench doesn't support the xmltv's event based architecture, so is not included in file parsing timings.
These benchmarks were run on a 2021 MacBook Pro M1 Max (10 cores) with 64 GB of RAM.
---
## ðŊ Future Goals
### Worker Support
Even though it's fast and it won't block for long, this will block your main thread whilst it runs. I'd like to add support for running the parser in a worker so it doesn't block at all.
### Streaming Support
My initial intent writing this library was to build a streaming parser, however I found the approach of [tXml](https://github.com/TobiasNickel/txml) to be much faster than other methods I tried. There are some issues upstream I'd like to help fix before I can add streaming support but it is something I'd like to do.
### CLI Module
A command line program that can be piped around to convert files from one format to another.
## ðŦ Non-Goals
### Generic XML Parsing
This library is designed to parse and generate XMLTV files only. It is not designed to be a generic XML parser or generator. If you need to parse generic XML files, you should use the excellent [tXml](https://github.com/TobiasNickel/txml) library instead.
### Altering the Shape of the Data
The shape of the `Xmltv` type is designed to match the [XMLTV DTD](https://raw.githubusercontent.com/XMLTV/xmltv/master/xmltv.dtd) as closely as possible. If you need a different object you can traverse the DOM tree yourself and convert it to your desired shape.
---
## ðĪ Contributing
Contributions are welcome! Even better if they align with the [future goals](#-future-goals).
You'll need to be able to run the tests and benchmarks. To do so, you will need to run the `./create-fixtures.sh` script in the `tests/fixtures` directory to generate the necessary fixture files.
To be accepted your PR must pass all tests and not negatively impact the benchmarks. Some commands to help you:
- `pnpm run test` - Run the vitest suite
- `pnpm run benny` - Run additional benchmarks
- `pnpm run benchmark` - Run the benchmarks with vitest
- `pnpm run nanobench` - Run additional benchmarks
This project uses [Changesets](https://github.com/changesets/changesets) to manage releases. For you, this just means your PR must come with an appropriate changeset file. If you're not sure how to do this, just ask and I'll be happy to help, or read the changesets documentation on [adding a changeset](https://github.com/changesets/changesets/blob/main/docs/adding-a-changeset.md).
## ð License & Credit
This library is licensed under the [MIT License](https://github.com/ektotv/xmltv/LICENSE.md) and is free to use in both open source and commercial projects.
The parser is based on the [tXml](https://github.com/TobiasNickel/txml) library by [@TobiasNickel](https://github.com/TobiasNickel).