Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/felixfontein/filetreesubs

Synchronize a file tree with text file substitutions.
https://github.com/felixfontein/filetreesubs

Last synced: 19 days ago
JSON representation

Synchronize a file tree with text file substitutions.

Awesome Lists containing this project

README 

        



File Tree Subs

==============



[![Tests badge](https://github.com/felixfontein/filetreesubs/actions/workflows/python.yml/badge.svg)](https://github.com/felixfontein/filetreesubs/actions/workflows/python.yml)

[![Codecov badge](https://img.shields.io/codecov/c/github/felixfontein/filetreesubs)](https://codecov.io/gh/felixfontein/filetreesubs)



Allows to synchronize a destination file tree from a source file tree while allowing certain substitutions to take place.



File Tree Subs uses [doit](http://pydoit.org/) under the hood to keep track of changes, so that files are only changed if necessary.



See the following three examples for typical use cases of `filetreesubs`. I'm personally using it to preprocess the output of [Nikola](https://getnikola.com/), a static blog/site generator, to insert a sidebar into all generated HTML pages, and a tag cloud into the sidebar and the tag overview page.



To install, use `pip install filetreesubs`.



Example

-------



Assume you have the following file tree:



    input/

        index.html

        team.html

        products.html

        menu.inc

        testimonials.inc



In the `.html` files, you put placeholder strings `INSERT_MENU_HERE` for where the content of `input/menu.inc` should be inserted, and `INSERT_TESTIMONIALS` for where the content of `input/testimonials.inc` should be inserted. Also, you want `COPYRIGHT_YEAR` to be replaced by 2017. The result should be a tree like this, without the `.inc` files:



    output/

        index.html

        team.html

        products.html



with the placeholder string replaced. To do this with `filetreesubs`, create a config file `filetreesubs-config.yaml`:



```yaml

# Source directory

source: input

# Destination directory

destination: output

substitutes:

  # The following is a regular expression to match the filenames:

  '.*\.html':

    # The strings to replace

    'INSERT_MENU_HERE':

      # With what to replace them

      file: menu.inc

    'INSERT_TESTIMONIALS':

      file: testimonials.inc

    'COPYRIGHT_YEAR':

      text: '2017'

```



Then running `filetreesubs` will synchronize `output/` so that it contains the files from `input/`, except `menu.inc`, and makes sure the substitutions take place.



Example: Sidebar in Nikola

--------------------------



You can find an example site for Nikola using the [sidebar plugin](https://plugins.getnikola.com/v8/sidebar/) in [the Github repository felixfontein/filetreesubs-nikola-demo](https://github.com/felixfontein/filetreesubs-nikola-demo/).



A more complex, but less explicit example can be found [in my blog](https://spielwiese.fontein.de/2017/01/06/static-sidebar-and-tag-cloud/), which also includes a tag cloud (rendered by the [static_tag_cloud pugin](https://plugins.getnikola.com/v8/static_tag_cloud/)) into the sidebar.



Example: Substitution chains

----------------------------



Assume that in the above example, you want to use `INSERT_TESTIMONIALS` also in `menu.inc` itself. Running the above example, this substitution will not be done, also if you extend the regular expression matching all HTML files to `.*` to match all files.



To apply substitutions to included files, you need to use substitution chains. Append the following to the configuration above:



```yaml

substitute_chains:

- template: menu.inc

  substitutes:

    'INSERT_TESTIMONIALS':

      file: testimonials.inc

```



This will apply the substitution for `INSERT_TESTIMONIALS` also to `menu.inc`.



Example: Creating index files

-----------------------------



Assume that you have folder structure:



    input/

        index.html

        images/

            logo.jpeg

            2017/

                happynewyear-2017.jpeg



You want to upload the output to a web server so it is available under `http://example.com`, but if someone accesses `http://example.com/images/` or `http://example.com/images/2017/`, you don't want the persons to see a file listing or some error page, but show them a nice message to check out the home page. You can use `filetreesubs` for this. Add the following to the configuration:



```yaml

create_index_filename: index.html

create_index_content: |

  

  

    

      There's nothing to see here.

      

    

    

      There's nothing to see here. Go here instead.

      You will be automatically redirected there in 10 seconds.

    

  

```



Then in every folder not containing a file `index.html`, a file `index.html` will be created with the specified content.



Configuration file format

-------------------------



The configuration file is in [YAML format](https://en.wikipedia.org/wiki/YAML). By default, the configuration is assumed to be in `filetreesubs-config.yaml` in the current directory. If you want to specify a different configuration file name, you can simply specify it on the command line:



    filetreesubs my-config-file.yaml



The following commented YAML file shows all available options:



```yaml

# The source directory. Specify a path here.

source: input



# The destination directory. Specify a path here.

destination: output



# The substitutions to make

substitutes:

  # For every substitution, you need to specify a regex pattern

  # matching the file name. Use '.*' to match everything, and

  # '.*\.html' to match all files ending with '.html'.

  '.*':

    # Now you can specify a number of strings which shall be replaced

    'STRING TO REPLACE':

      # In this case, we want to replace the string by the contents

      # of the file menu.inc. Note that menu.inc won't be copied

      # to the destination directory anymore.

      file: menu.inc

    'ANOTHER_REPLACEMENT_STRING':

      # In this case, we want to replace the string by another string

      # we explicitly specify here.

      text: '(replacement text)'

  # Now we can specify more filename matching patterns ...

  '.*\.html':

    # ... and more replacements

    'YET_ANOTHER_STRING':

      text: '(some more)'



# To do substitutions in files like menu.inc, we need substitution

# chains.

substitute_chains:

# Each substitution chain consists of the name of the file to

# substitute in, like menu.inc:

- template: menu.inc

  # As well as a list of substitutions, using the same syntax as above:

  substitutes:

    # The string to replace:

    'INCLUDE_INCLUDE':

      # What to replace it with

      file: include.inc

    'INCLUDE_STRING':

      text: '...'

# You can have as many substitution chains as you want

- template: include.inc

  substitutes:

    'ONE_MORE':

      text: '(...)'



# To create index files (when not already existing), you must

# specify the name of these files:

create_index_filename: index.html



# This allows to specify the content of index files.

create_index_content: |

  

  

    

      there's nothing to see here.

      

      

    

    

      


        


            there's nothing to see here. go here instead.

          


        


      

    

  



# By default, filetreesubs assumes that all text files it processes

# are UTF-8 encoded. If that's not the case, you can change another

# encoding here.

encoding: utf-8



# In case you need to do so, you can insert configurations for doit

# directly here. See `here `__

# for possible configurations.

doit_config:

  # The following option sets the filename for the dependency database.

  # If you want to execute different filetreesubs commands concurrently

  # from a folder, you need to specify different dependency database

  # names per project config.

  dep_file: '.doit-myproject.db'

```