Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/networkupstools/nut-website

Network UPS Tools website and protocol library
https://github.com/networkupstools/nut-website

Last synced: about 2 months ago
JSON representation

Network UPS Tools website and protocol library

Awesome Lists containing this project

README 

        
Network UPS Tools website

=========================



This repository contains the scripts needed to generate the

link:https://www.networkupstools.org[NUT website].



Since it was originally part of the

link:https://github.com/networkupstools/nut/[NUT source tree],

it shares some history, and you can save some bandwidth if you

already have a copy of the NUT repository downloaded:



----

:; git clone --reference /path/to/nut \

    https://github.com/networkupstools/nut-website.git

----



Once you have cloned the nut-website repository, you can initialize the

submodules, and pull your copy of NUT into the website tree as well:



----

:; git submodule init

Submodule 'ddl' (https://github.com/networkupstools/nut-ddl.git) registered for path 'ddl'

Submodule 'nut' (https://github.com/networkupstools/nut.git) registered for path 'nut'

:; git submodule update --reference /path/to/nut nut

...

----



Required Packages

-----------------



You will need a copy of `AsciiDoc` toolkit, `a2x` (part of AsciiDoc), and

its dependencies for MAN, HTML and PDF document formats generation (including

`dblatex`, `xmllint`, and `xsltproc`). Current version requirements will be

listed in the output of the `./configure` script. To build the Hardware

Compatibility List (HCL) page, you will need either the `simplejson` or `json`

Python module (the json module that comes with Python 2.7 will work) and the

`lxml` module. You will also need `autoconf` and `automake`, and possibly

`libtool` since the NUT module uses it.



[NOTE]

======

With recent Ubuntu/Debian releases, Python2 is deprecated so much that `pip2`

tool seems to be no longer packaged. If you have both `python2` and `python3`

installed, you may have install modules by `pip3` or APT and to either declare

the preference via environment variables for NUT and NUT-Website `configure`

and other scripts, or change system-wide default symlink to `python` once:



----

:; sudo apt-get install python3-pip

:; python3 -m pip install lxml simplejson pycparser pathlib



### Note: Newer Python releases can suggest to use APT packages for modules

### too (note also that "pathlib" may be not available this way, and may be

### in fact part of the base Python distribution); in this case:

:; sudo apt-get install python3-{lxml,simplejson,pycparser}

----



and then either system-wide:

----

:; sudo apt-get install python-is-python3

----



or constrain the preference to nut-website builds:

----

:; PYTHON='/usr/bin/env python3' ./ci_build.sh

----

======



The `source-highlight` package is optional, but if available, will be used by

AsciiDoc for syntax highlighting of examples.



The optional `htmlproofer` tool from https://github.com/gjtorikian/html-proofer

project can be used to sanity-check links and similar aspects of the markup in

generated HTML pages. On Debian/Ubuntu systems you can install it as a package:



----

:; sudo apt-get install ruby-html-proofer

----



[NOTE]

======

If your `htmlproofer` runs complain like this:



----

htmlproofer 3.19.2 | Error:  "\xC3" on US-ASCII

  /usr/lib/.../nokogiri/html5.rb:389:in `encode': "\\xC3" on US-ASCII (Encoding::InvalidByteSequenceError)

----



Try exporting `LANG` and `LC_ALL` environment variables to use UTF-8 capable

locales (already handled in the `Makefile.am` targets by `HTMLPROOFER_ENV`);

this may further require installing some or all locale packages, e.g.:



----

:; sudo apt-get install locales-all

----

======



GNU `make` and GNU `coreutils` are recommended, but if you see any remaining

non-portable constructs in the Makefiles, please let us know.



Building

--------



Editing the Makefile.am source

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



WARNING: Due to lines which `# HIDE FROM AUTOMAKE #` some GNU syntax which

conflicts with automake syntax, you *MUST* use `autogen.sh` to re-generate

the practical `Makefile` during development cycles. If you rely on automatic

typical regeneration of `Makefile.am` -> `Makefile.in` -> `Makefile`, the

resulting file can have crucial parts commented away.



Alternately, `make unhide-from-automake` after edits, e.g. to fiddle with

spell checker Makefile recipes for historic releases:



----

:; make unhide-from-automake ; make spellcheck NUT_SPELL_DICT=nut-website.dict

----



Quick builds for CI and developer iterations

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Main rituals for re-builds should now be handled well by a single script:



----

:; ./ci_build.sh

----



For maintainers (or CI agents) who may push the web-site codebase, further

envvars may be useful to commit changes locally, and to push index upwards:



----

# To publish automatically use:

:;   export CI_AUTOCOMMIT=true

:;   export CI_AUTOPUSH=true



# Optionally (on CI) to avoid rebuilds in cases when Git sources did not

# change after a pull of all nut-website and submodule HEADs (returns exit

# code "42" then, to be handled by the caller):

:;   export CI_AVOID_RESPIN=true



# Optionally - for the rare historic-release sub-sites (by NUT tag), e.g.:

:;   export NUT_HISTORIC_RELEASE=v2.7.4

----



Also, to make sure that syntax of `nut-ddl` data dump files is not ambiguous,

you can tell their parser to abort in case of doubts:



----

:;   export NUT_DDL_PEDANTIC_DECLARATIONS=True



:; ... make ...

Traceback (most recent call last):

  File "/home/jim/nut-website/./tools/nut-ddl.py", line 1532, in 

    commentsMap[pattern](comment)

  File "/home/jim/nut-website/./tools/nut-ddl.py", line 400, in nds_dev_comment_block

    raise RuntimeWarning (msg)

RuntimeWarning: Invalid device block comment: does not end with DEVICE:EOC (blank non-comment lines mid-block?)

make: *** [Makefile:1012: ddl/PowerWalker/PowerWalker__VI_2200_SH__usbhid-ups__2.7.4__01.dev.txt] Error 1

----



Manual building in detail

~~~~~~~~~~~~~~~~~~~~~~~~~



----

:; ./autogen.sh && ./configure && { make -k ; make ; }

----



NOTE: There are currently some issues with parallel builds (e.g. `protocols`

sub-directory should be built before `OUTDIR` but target dependencies do not

say so). Please run `make` sequentially for the time being.



The root of the website will be in the `output/` directory, if all goes well.



Building release-version sites

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



Every once in a while NUT has releases :) and those end up packaged in many

operating system distributions or equivalent bundles of binary code, and are

used by the majority of NUT users.



While the main nut-website now aims to follow the active development based on

current "master" or "main" branches of the components involved, we also publish

reference sub-sites with static data (man pages, configuration, compatibility

information, etc.) for those people to see about setting up their practical

systems properly. Such pages are marked with a note on top that they represent

an old and immutable codebase which may differ from modern project state.



This also impacts some but not all pages of the nut-website as well -- e.g.

the `stable-hcl.txt` file which is a wrapper to include `nut/data/driver.list`

info, but not all of the main website files currently (just because it is too

cumbersome to partially check out arbitrary old codebase to build with current

updated recipes). Likewise, general markup and footers, etc. remain from the

current nut-website codebase at the time of (re-)generation.



In particular, the NUT DDL pertains to all NUT releases (reports the tested

release in the filenames) so is not published separately per historic release.



Normally this should be done once per release, with a call like this:



----

:; export NUT_HISTORIC_RELEASE=v2.7.4

:; ./autogen.sh && \

    ./configure --with-NUT_HISTORIC_RELEASE=${NUT_HISTORIC_RELEASE} && \

    { make -k dist-sig-files || make dist-files; } && \

    { make -k ; make; }

----



Which would populate e.g. `output/historic/v2.7.4` subdirectory that would be

copied and committed into same-named path under `networkupstools.github.io` as

detailed below. Generation of the main nut-website would also populate an index

file of the "historic" subdirectory, based on `historic/index.txt` contents,

to refer to such officially published snapshots. This index is currently

maintained manually, to ensure human decision about publishing (or hiding) an

historic release (especially a release candidate) vs. experimenting with that.



NOTE: `make dist-files` should update the historic release site source tarballs

and related ChangeLog, news and checksum files IFF the release data was not yet

there. You probably need to commit that back to "source" github repository.



NOTE: For hardcore maintainers, there should be a PGP/GPG key to also sign the

release tarball, calling `make dist-sig-files` (would fail without a key).



Sanity-checking the generated HTML files

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



If the `htmlproofer` tool is installed (see above) and detected by the

`configure` script (automatically for presence in `PATH`), you can explicitly

call `make check-htmlproofer` to validate the files present in `OUTDIR_BASE`

and/or `OUTDIR` (if stored separately from the base for custom or historic

website builds).



You can also validate the published website repository (into which you would

upload the generated `OUTDIR` contents) as e.g. prepared by the `ci_build.sh`

using `make check-htmlproofer-OUTDIR_PUBLISHED` (optionally customize the

`OUTDIR_PUBLISHED` environment or `make` variable to point to the checkout

location for that repository, if not using the scripted default).



Note that this check can take about 10 minutes (especially if not disabling

the referenced external site availability), so it is not done by default.

You can pass custom `HTMLPROOFER_OPTIONS` to the `make` operation, if desired;

consider pasting the `HTMLPROOFER_OPTIONS_URLSWAP` in that case.



Publishing

----------



NOTE: These are internal notes for the maintainers.



The build result is published to the

https://github.com/networkupstools/networkupstools-master.github.io[NUT

github.io master site repository]

as well as news maintenance on

https://github.com/networkupstools/networkupstools.github.io[NUT

github.io latest-release site repository]



Hence, the rolling master site publication is as easy as:



----

:; git clone https://github.com/networkupstools/networkupstools-master.github.io

:; rsync -avPHK ./output/* /path/to/networkupstools-master.github.io/

----



Release site publication is much less frequent. It follows the master

site guideline when making a release, but only requires updating the

`index.html` file when `news.txt` is updated, as noted below.



NOTE: Be careful to use `git mergetool -y` to merge the updates from

a newly generated `index.html` into the release site, to keep intact

the title (marked with comments) which specifies the type of site.



----

:; git clone https://github.com/networkupstools/networkupstools.github.io

:; cp -R ./output/index.html /path/to/networkupstools.github.io/

:; (cd /path/to/networkupstools.github.io/ && git difftool -y)

----



NOTE: Maybe also update the `ddl` and `stable-hcl.html` on master site,

as its updates often reflect newly confirmed support of devices by

existing NUT releases.



Updates

-------



If you only have a small patch (fixing a typo or wording), don't feel

obliged to install all of the dependencies listed above just to test it.

Feel free to create a pull request on this repository, or (less preferable

as slower to process) send the patch as an attachment to the

link:https://www.networkupstools.org/support.html#_mailing_lists[nut-upsdev list].



Maintainer note: Publishing became part of NUT CI farm automation in 2022,

so whenever master branch sources of relevant repositories are changed,

the website should not lag behind too long. Needed behavior is defined in

this repository in `Jenkinsfile-infra` file, with job history visible at

https://ci.networkupstools.org/view/InfraTasks/job/nut-website/



* As of this writing, changes of `nut-website` repository should get picked

  up quickly thanks to "web hooks" sent by GitHub to NUT CI farm servers,

  and changes in NUT, NUT-DDL and other repositories involved would be

  evaluated every 3 hours.



* (Re-)builds of historic sub-sites for release candidates etc. are handled

  manually by maintainers, to publish source tarballs as well (in nut-source

  repository, in the web-site, in GitHub releases page), and generally happen

  once per such release with a spell like this:

+

------

:; CI_AUTOCOMMIT=true CI_AUTOPUSH=true NUT_HISTORIC_RELEASE=v2.8.0-rc3 ./ci_build.sh

------



* The `nut-website` specific spell-checking is handled with a dynamic mix of

  original `nut/docs/nut.dict` and custom `nut-website.dict.addon` with key

  words specific to files in the website (including HTML and asciidoc markup).