Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/haxscramper/hmisc

Collection of miscellaneous helper algorithms and types. Helper functions for strings, exceptions etc.
https://github.com/haxscramper/hmisc

Last synced: 1 day ago
JSON representation

Collection of miscellaneous helper algorithms and types. Helper functions for strings, exceptions etc.

Awesome Lists containing this project

README 

        
#+title: readme



I don't want to call this library officially "deprecated", but I haven't

been paying as much attention to the code quality and/or in recent times. I

think everything up until version ~0.14.4~ can be considered "mostly

reliable" - so if you are looking to use this library, I would advise you

to pin dependency to this version.



Maybe in the future I will return to working on this library - for now I

don't think I have the energy required to constantly fight nim

documentation generator, write unit testing framework and test runner and

so on.



------



Collection of miscellaneous helper algorithms and types. Over time this

library accumulated a lot of functionality, not all of this is necessary

and almost everything was made on case-by-case basis instead of being

designed at once.



Most useful parts of the library are probably



1. ~hmisc/other/oswrap~ - wrapper for the ~std/os~ but with support for

   proper ~disctinct~ types

2. ~hmisc/other/hshell~ - shell command execution builder

3. ~hmisc/core/gold~ or ~hmisc/core/all~ - these contain my

   import-everywhere templates, procedures and macros.



In addition to the general helpers, the library also contains forks of some

of the fusion modules - specifically ~hmisc/types/hmap~ is a fork of

~fusion/btreetables~ with saner type naming (no more type clashes all over

the place) and couple helper routines. I also copied ~fusion/matching~

back, since fusion can be considered effective dead and deprecated - not

tagged (at the time of writing ~requires fusion~ still pulls 9-month old

version, that contains *none* of the fixes that was made to pattern

matching after that. For more details see [[https://forum.nim-lang.org/t/8627#56155][forum thread]] - not especially

interesting, but if you want more context ...). Right now it is a simple

copy, in the future I will clean up the implementation since I've got

several ideas on how it can be improved. DSL will be cleaned up as well,

right now it is really overloaded.



** Installation:



#+begin_src sh

nimble install hmisc

#+end_src



** Links



- [[https://nimble.directory/pkg/hmisc][nimble package]]

- [[https://haxscramper.github.io/hmisc/theindex.html][documentation]]

- [[https://github.com/haxscramper/hmisc][github]]



* Macros



** ~hmisc/macros/iflet~



Rust-like iflet. Get value from ~Option[T]~ only if it contains

something, otherwise execute ~else~.



#+begin_src nim :exports both

import options

import hmisc/macros/iflet



iflet (val = some(12)):

  echo typeof val



iflet (val = none(int)):

  echo "???"

else:

  echo "no value"

#+end_src



#+RESULTS:

: int

: no value



* Algorithms

  :PROPERTIES:

  :header-args:nim:+ :import hmisc/algo/halgorithm

  :END:



** ~hmisc/algo/halgorithm~ [[https://haxscramper.github.io/hmisc/src/hmisc/algo/halgorithm.html][documentation]]



- predicates

  - ~allOfIt~

  - ~anyOfIt~

  - ~noneOfIt~

  - ~==~ for Option-Val comparison

- working with sequences

  - ~maxIt~

  - ~disjointIter~

  - ~last~

- working with strings

  - ~joinw~, ~joinq~, ~joinl~, ~joinkv~ - join on whitespaces,

    whitespace + quote each string, newlines or key-value pairs.

    Mostly useful in ~strformat.&~ - can write ~{somevar.join()}~

    instead of ~{somevar.join(\"\\n\")}~

  - ~wrap~ - wrap strings in delimiters. Has convinience overload for

    ~.wrap("()")~ that automatically determines starting/ending

    wrapper strings.

  - Multiple overloads for ~join~ and ~startsWith~

  - ~enclosedIn~ - check if string is wrapped in delimiters

  - ~dedent~ - decease indentation for multiline string

  - ~camelSplit~ - split string as camel case identifier

  - ~abbrevCamel~ - /camelCase/ abbreviation search.

  - Several variations of ~dropPrefix~, ~addPrefix~, ~startsWith~,

    ~addSuffix~ for less common use cases

  - Filtering sequence of strings by prefix

  - Dropping subsequence in strings

  - Finding common prefix in sequence of strings

- other

  - ~ifSomeIt~ - same as ~opt.isSome() and (let it = opt.get();

    predicate)~

  - ~testEq~ - compare two objects. If they are different print first

    mismatching line in their string representation.

  - ~assertEq~ - compare objects using ~testEq~, raise on failed

    comparison.



#+begin_src nim :exports both

  import hmisc/algo/halgorithm, std/strformat

  let v = @["w234", "333"]



  echo ": ", &"{v.joinq()}"



  block:

    echo "-- withIt --"

    let immutable = (a: 12, b: 12)

    echo immutable.withIt do:

      it.a = 909



  block:

    echo "-- withResIt --"

    let immutable = (a: 12, b: "eee")

    echo immutable.withResIt do:

      it.a += 999

      $it.a & it.b



  block:

    echo "-- join* --"

    echo {1 : "22", 3: "333"}.joinkv().join()



  block:

    echo "-- abbrevCamel --"

    echo abbrevCamel("AA", @["ABA", "AZZ", "A)"])

#+end_src



#+RESULTS:

: : "w234" "333"

: -- withIt --

: (a: 909, b: 12)

: -- withResIt --

: 1011eee

: -- join* --

: 0 = (1, "22") 1 = (3, "333")

: -- abbrevCamel --

: @["ABA"]



** ~hmisc/algo/hseqdistance~ [[https://haxscramper.github.io/hmisc/src/hmisc/algo/hseqdistance.html][documentation]]



Fuzzy string matching and generic longest common subsequece

implementation



- ~longestCommonSubsequence~ - generic implementation of LCS algorithm

  for ~seq[T]~

- ~fuzzyMatch~ - weighted sequence fuzzy match. Compare each element

  in the sequence to pattern and assign similarity score. Should

  behave similarly to ~fzf~ or sublime text. Reimplementation of

  [[https://www.forrestthewoods.com/blog/reverse_engineering_sublime_texts_fuzzy_match/]['Reverse engineering subtime text's fuzzy match']]. I haven't used it

  in any interactive applications as of yet, but there are some unit

  tests. It has generic implementation and somewhat annoying to use,

  but provides very flexible interface, allowing to completely

  customize how fuzzy matching is performed.



#+begin_src nim :exports both

  import hmisc/doc_examples



  echo "# ~~~~ leading / ~~~~ #\n|"

  matchTest "//hell.txt", "/nice/we/hell.txt":

    if other[matches[0]] == '/':

      1000 # high cost if have exact match with starting /

    else:

      matches.sum()



  echo "|\n# ~~~~ no leading / ~~~~ #\n|"

  matchTest "nicehell.txt", "/nice/we/hell.txt":

    if other[matches[0]] == '/':

      1000

    else:

      matches.sum()

#+end_src



#+RESULTS:

: # ~~~~ leading / ~~~~ #

: |

: input: /nice/we/hell.txt //hell.txt :1000

: match: /    /   hell.txt

: |

: # ~~~~ no leading / ~~~~ #

: |

: input: /nice/we/hell.txt nicehell.txt :113

: match:  nic   e hell.txt



** ~hmisc/algo/hseq_mapping~



- ~deduplicateIt~

- ~mapPairs~ :: ~mapIt~ for types that implement ~pairs~ iterator, or

  ~items~ that return tuple, or sequence of tuples. Inject index of

  the item, ~lhs~ (first element) and ~rhs~ (second element). Should

  correctly handle ~{.requiresinit.}~ fields.



** ~hmisc/algo/htree_mapping~



- ~mapItBFStoSeq~ :: iterate over tree in BFS order, store mapping

  result in sequence.

- ~iterateItBFS~ :: iterate over tree in BFS order

- ~iterateItDFS~ :: iterate over tree in DFS order. Uses iterative DFS

  instead of recursive call.

- ~mapItDFS~ :: ~mapIt~ for converting trees in DFS order



* Types



** ~hmisc/types/colorstring~



Easier manipulation of colored strings in terminal. Support splitting

regular strin in same-color chunks, finding 'visible' length of the

string (as printed in terminal). Helper functions like ~toYellow()~ or

~toRed()~ to make creation of the colored strings simpler. All

attributes from ~terminal~ module are supported (fg/bg colors and

modifiers).



Provides two types for colored text - ~ColoredString~ (string +

styling) and ~ColoredRune~ (unicode rune + styling).



* Other



~hshell~ and ~oswrap~ modules provide more strictly typed wrappers for

tasks that are usually performed using simple string concatenations.

You get better static safety guarantees (not possible to pass relative

path to function expecting absolute one) and less headaches related to

correct quoting/CLI command syntax at the expense of little more

verbose code.



~oswrap~ is a ~1:1~ mapping of ~std/os~ and is expected to have all

functions reimplemented (wrapped).



~hshell~ also treats non-zero return codes as exceptions, so you can

just execute shell commands without endless checks for ~code != 0:

echo "oh no!"~. This can be turned off, but works by default, so when

writing ~let (output, err, _) = runShell("someCommand")~ you will be

sure that failures won't be silently ignored.



** ~hmisc/other/oswrap~



Wrapper on top of ~os~ and ~nimscsrip~ that allows to use the same

code on ~c~ and ~nimscript~ targets. Some helper templates/functions

are introduced. Provide distinct string for files/directories - e.g.

~RelDIr = distinct string~ as well as overloads for almost all

functions in ~os~ module.



NOTE it is expected to be imported *instead* of ~os~ module -

functions without arguments were update to use ~distinct~ types too,

so if two modules are imported togetether frequent type clashes are

expected.



- ~mkDir~, ~getEnv~, ~delEnv~, ~toExe~, ~listDirs~, ~rmFile~, ~mkDir~,

  ~mvFile~, ~cpFile~, ~cpDir~, ~cd~, ~cwd~ - default file/directory

  manipulation functions

- ~ (prefix tilda) prefix operator to get path relative to home

  directory. Same as ~getHomeDir() / path~

- ~&&~ join shell command strings with correct spacing

- ~withDir~ - temporarily set directory for body

- ~withEnv~ - temporarily set environment variables for body



** ~hmisc/other/hshell~ [[https://haxscramper.github.io/hmisc/src/hmisc/other/hshell.html][documentation]]



Helper functions for running shell commands - reduce need for string

concatenation for shell - ~Cmd~ object supports adding

commands/flags/options/subcommands/arguments while deferring

conversion to string as long as possible and taking care of correct

syntax (correct dashes for ~X11~ CLI tools (always single prefix

dash), key-value separators (nim tooling uses ~:~, GNU is most likely

to expect ~=~ or spaces)).



Possible use case: Imagine you need to write a script that launches new

docker container, mounts some folders, copy files over, and perform some

nontrivial commands inside container (and command is not predetermined - it

is also has to be built in advance).



Regular approach would be to cobble together one giant string that will

then be executed via ~startProcess~. You need to then check for return

code, and hope that you haven't messed up quoting, argument syntax for

particular command and so on (nim tooling uses ~:~, GNU - ~=~ and so on).



~hshell~ hopefully provides solution to most of the usability of command

line programs - you no longer need to worry about correct spacing, quoting

and other stuff like that. Instead, you just build AST for command to be

executed, using set of convenient operators and functions.



#+begin_src nim

  let cmd = shCmd("nimble", "install")

  # Nice side effect - you can now comment on different flags and use

  # checks/loops without worrying about correct

  # spacing/concatnation/prefixes etc.

  let doCleanup = true

  let dockerCmd = shCmd("docker").withIt do:

    it.cmd "run" # Add subcommand

    it - "i"

    it - "t"

    if doCleanup:

      it - "rm" # Remove container after test execution

    it - ("v", "/tmp/tmp-mount:/project") # Key-value pair

    it.arg "nim-base"

    it.arg "sh"

    it - "c"

    it.expr:

      shAnd:

        shCmd(cd, "/project/main")

        cmd # Can easily build complicated commands from variables

#+end_src



NOTE: no special DSL syntax is introduced, just couple of overlads for

common use cases (~-~ proc for flags/options)



- ~runShell~ Raise exception when command has exited with non-zero

  code (because you will be checking return code anyway), get stderr

  and stdout separately. Uses fallback ~exec~ and ~gorgeEx~ on

  nimscript targets and tries to emulate compiled behaviour as close

  as possible (respect execution flags).

- ~iterstdout~ - iterate each line for executed program's stdout

- ~execShell~ - execute shell command, redirect output into parent streams.



NOTE: You might consider this module a 'shell program wrapper'. It was

created to make using external processes from your code easier and safer.

No need to check return codes all the time, think about quoting, correct

arguments and so on. Thus said - it is quite difficult to wrap all

complixity of the command line interfaces, even with quite sophisticated

logic. Several escape hatches are present, to still pass almost arbitrary

strings for shell execution. First: ~ShellExpr~ - thin wrapper, ~distinct

string~. Second is ~raw()~ function for setting command line arguments.



* Contribution & development



Most of the features in this library were implemented on

/do-it-when-I-need-it/ basis. Some of them are tested quite extensively

(sequence and tree mappings, colored strings), but more unit test are

always welcome.



* References



- [[https://research.google/pubs/pub44667/][A new apprach to optimal code formatting]]