software/kisstdlib

A set of modules for the Python programming language designed for system programming with conceptually and algebraically simple primitives.

Distributed via

Recent changes

Files

Raw Source

Table of Contents

(Click me to see it.)

What is kisstdlib?

kisstdlib is a set of modules for the Python programming language, designed mostly for system programming with a touch of everything else, that aims to enhance the standard experience while keeping everything it does conceptually and algebraically simple.

This library borrows heavily from the Haskell programming language, where appropriate. (The Python’s bundled modules do that too in many places, but kisstdlib borrows more.)

In short, kisstdlib mostly implements useful extensions for Python’s bundled modules, keeping the APIs backward compatible and naming things in the same style. The only exception to this is, yet unpublished, io.loop module, which implements Free-Monad-based IO.

At the moment, kisstdlib is an alpha work in progress software with an unstable API.

Parts and pieces

kisstdlib consists of:

Usage

describe-forest

Produce a plain-text recursive deterministic find/ls/stat-like description of given file and/or directory inputs.

The output format is designed to be descriptive and easily diffable while also producing minimally dissimilar outputs for similar inputs, even when those inputs contain lots of symlinks and/or hardlinks. I.e., essentially, this is an alternative to ls -lR and/or find . -exec ls -l {} \; which generates outputs that change very little when files with multiple symlinks and/or hardlinks change.

This is most useful for testing code that produces filesystem trees.

The most verbose output format this program can produce, for a single input file

describe-forest --full path/to/README.md

looks as follows:

. reg mode 644 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

Note how both the path to and the name of the file do not appear in the output. This is what you would want for doing things like

if ! diff -U 0 <(describe-forest --full v1/path/to/README.md) <(describe-forest --full v2/path/to/README.md) ; then
    echo "output changed between versions!" >&2
    exit 1
fi

which this program is designed for.

For a single input directory

describe-forest --full path/to/dir

the output looks similar to this:

. dir mode 700 mtime [2025-01-01 00:00:00]
afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
sub dir mode 700 mtime [2025-01-01 00:03:00]
sub/afile-hardlink.jpg ref ==> afile.jpg
sub/afile-symlink.jpg sym mode 777 mtime [2025-01-01 00:59:59] -> ../afile.jpg
sub/zfile-hardlink.jpg reg mode 600 mtime [2025-01-01 00:02:00] size 256 sha256 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
unix-socket ??? mode 600 mtime [2025-01-01 01:00:00] size 0
zfile.jpg ref ==> sub/zfile-hardlink.jpg

Hardlinks, which are denoted by refs above, are processed as follows:

This way, renaming a file in the input changes at most two lines.

Symlinks are rendered by simply emitting the path they store, unless --follow-symlinks is given, in which case the targets they point to get rendered instead.

Multiple inputs get named by numbering them starting from “0”. Thus, for instance, running this program with the same input file given twice

describe-forest --full path/to/README.md path/to/README.md

produces something like:

0 reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
1 ref ==> 0

And giving the same directory with that file inside twice produces:

0 dir mode 700 mtime [2025-01-01 00:00:00]
0/afile.jpg reg mode 600 mtime [2025-01-01 00:01:00] size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
1 dir mode 700 mtime [2025-01-01 00:00:00]
1/afile.jpg ref ==> 0/afile.jpg

In its default output format, though, the program emits only sizes and sha256s, when appropriate:

. dir
afile.jpg reg size 4096 sha256 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef

which is what you would usually want for writing tests. Though, if you are testing rsync or some such, feel free to use other options described below.

See devscript directory in kisstdlib’s repository for examples of some shell machinery that uses this to implement arbitrary-program fixed-output tests, which is a nice and simple way to test programs by testing their outputs against outputs of different versions of themselves.

Also, internally, this programs is actually a thin wrapper over describe_forest function of kisstdlib.fs Python module, which can be used with pytest or some such.

Development: ./test-example.sh [--help] [--wine]

Check that all kisstdlib/example/*.py are fixed-output.