CLI Abstraction

Calling another program from Python usually starts as a list of strings and ends as a bug: a path that needed quoting, a flag whose spelling differs on Windows, a value concatenated with + in the wrong place. The list is assembled far from where the arguments are decided, and nothing checks it.

pyTooling.CLIAbstraction turns the program into a class and each of its arguments into a typed member. Assembling, escaping and platform-dependent spelling happen once, in the argument class, instead of at every call site.

This tutorial builds that class.

Step 1: `Program` or `Executable`?

Two base-classes, one question: does your class need to run the program, or only to describe the call?

Base-class	Gives you	Choose it when
`Program`	argument members, `ToArgumentList()`, `__str__`, the executable resolved on `PATH` per platform	something else starts the process - a build system, a CI step, `subprocess.Popen` in the caller - or you only want to print the command line
`Executable`	all of that, plus `StartProcess()`, `Send()`, `GetLineReader()`, `Terminate()`, `Wait()` and `ExitCode`	your class starts the process and reads its output

Executable derives from Program, so the choice is not final: describe first, and change the base-class when the class needs to run something.

Hint

Start with Program when in doubt. A class that only assembles a command line is trivially testable - ToArgumentList() returns a list you can assert on - while a class that starts processes needs one to be installed.

Step 2: the class and its first argument

An argument is a nested class derived from one of the predefined argument classes, marked with @CLIArgument(). The nested class contributes nothing but its name and its base - the base decides the pattern, and name= fills it in.

Three things are worth naming:

_executableNames maps platform.system to the file name

The executable is looked up on PATH when the object is constructed, and a missing one raises CLIAbstractionError right there rather than at the first call. Pass executablePath= to bypass the lookup and name a file directly.

The nested class’ own name is how you address the argument

git[git.CommandCommit] - the member name, not the command line spelling. Renaming --version to -V later is a change to name= alone; every call site keeps working.

... is the whole body

The nested class exists to be named and to inherit. Anything else in its body is a sign the wrong base-class was chosen.

Note

@CLIArgument() needs its parentheses even though it takes no arguments, because it is an attribute and an attribute is instantiated before it is applied.

from pyTooling.CLIAbstraction         import CLIArgument, Program
from pyTooling.CLIAbstraction.Command import CommandArgument
from pyTooling.CLIAbstraction.Flag    import ShortFlag, LongFlag

class Git(Program):
  _executableNames = {
    "Darwin":  "git",
    "FreeBSD": "git",
    "Linux":   "git",
    "Windows": "git.exe",
  }

  @CLIArgument()
  class FlagVersion(LongFlag, name="version"): ...

  @CLIArgument()
  class CommandCommit(CommandArgument, name="commit"): ...

  @CLIArgument()
  class FlagAll(ShortFlag, name="a"): ...

Step 3: set the arguments and assemble

Arguments are set through the indexer, with the value the argument carries - or None for one that carries nothing:

Attention

The order of the result is the order the arguments were declared in, not the order they were set. -m precedes -a above because ValueMessage is declared before FlagAll. Declare the nested classes in the order the program expects them - commands before their flags - and the assembled list is right regardless of how the caller fills it in.

Caution

A command and a flag carry no value, and the value assigned is not read. program[program.FlagAll] = True and program[program.FlagAll] = None do the same thing: assigning registers the argument. True is the better spelling because it says what is meant.

``False`` does not turn it off - it registers the flag like any other assignment, and -a is emitted anyway. An argument also can’t be unset afterwards: there is no __delitem__, and assigning a second time raises KeyError. So an argument that depends on a condition is set inside an if, not by assigning a boolean:

if verbose:
  program[program.FlagVerbose] = True

ToArgumentList() returns the list for subprocess.Popen; str(program) returns the same thing quoted, for a log line or an error message.

git = Git()
git[git.CommandCommit] = True             # a command: selects what the program does
git[git.FlagAll]       = True             # a flag: present
git[git.ValueMessage]  = "initial commit"  # a valued flag: name and value

git.ToArgumentList()                       # -> ['git', 'commit', '-m', 'initial commit', '-a']

Step 4: choosing the right argument class

Each predefined class knows one pattern. Pick by what the program’s syntax looks like, then by prefix style - every family has a Short, a Long and a Windows variant, differing only in -, -- and /.

Base-class	Assign	Renders as
`CommandArgument`	`None`	`commit`
`LongFlag`	`True`	`--quiet`
`LongValuedFlag`	`"build/app"`	`--output=build/app`
`LongTupleFlag`	`"/usr/include"`	`--include`, `/usr/include` two entries in the list
`LongValuedFlagList`	`["m", "pthread"]`	`--lib=m`, `--lib=pthread`
`LongOptionalValuedFlag`	`None` `"always"`	`--color` `--color=always`
`LongBooleanFlag`	`True` `False`	`--with-tests` `--without-tests`
`LongKeyValueFlag`	{ "NDEBUG": "1", "LEVEL": "2", }	`--DNDEBUG=1`, `--DLEVEL=2`
`PathArgument`	a `Path`	the path, positionally

Single value vs. tuple vs. list

The three that carry a value differ in how the value reaches argv, and mixing them up is the most common mistake in an abstraction:

LongValuedFlag - one argv entry, name and value joined by =. This is --output=build/app.
LongTupleFlag - two argv entries, the name and then the value. This is -m "initial commit", and it is what a program using getopt() with a separate value expects.
LongValuedFlagList - the flag repeated, once per element of the assigned list. This is --lib=m --lib=pthread, the shape of -I, -D and -l in most compilers.

The test is what the program’s own help text shows. --output=FILE is a valued flag; --output FILE is a tuple flag; --lib LIB (may be given more than once) is a flag list.

Hint

LongOptionalValuedFlag is for the flag that means one thing bare and another with a value - --color versus --color=always. Assigning None gives the bare form.

Step 6: running it

Change the base-class to Executable and the same class can start the process and read its output line by line:

Caution

dryRun=True on the constructor is meant to make the program describe what it would run instead of running it, so a --dry-run mode in your own tool needs one constructor argument rather than a branch around every call. It is not usable as it stands: the dry-run paths call self.LogDryRun(...), which no class in pyTooling defines, so StartProcess() raises AttributeError instead. Provide a LogDryRun method on your class until that is resolved.

GetLineReader() does behave as documented and raises DryRunError when the process was never started.

class Git(Executable, GitArgumentsMixin): ...

git = Git()
git[git.CommandCommit] = None
git[git.ValueMessage]  = "initial commit"

git.StartProcess()
for line in git.GetLineReader():
  print(line)

git.Wait()
if git.ExitCode != 0:
  raise BuildError(f"'{git}' failed with exit code {git.ExitCode}.")