Documentation ¶
Overview ¶
Package execstub provides stubbing for a usage of a command line api which is based on an executable discovered using the environment <PATH> or some sort of <HOME>+<Bin> configuration.
What is a usage of a command line api? Its when you spawn a sub-process using a programming language API. Think go exec.Cmd.Run(..), java Runtime.exec(..), python subprocess.run(..). The sub-process execution will yield an outcome consisting of:
side effects (e.g. a file created when executing touch).
bytes or string in std-out and/or std-err.
an exit code. The executed binary is typically specified using a path like argument. It can be:
a absolute path
a relative path
the name of the binary
Discovering the executable How the binary is discovered depends on the art the specified path like argument. The goal of the discovery process is to transform the path like specification into an absolute path, so that if the specification is done with:
absolute path nothing is left to be done
relative path it will be appended to the current directory to form an absolute path
name of the binary a file with that name will be search in the directories specified by the PATH environment variable. The first found will be used. On some platforms known and specified file extensions (e.g. .exe) will be appended to the name during the search. Even when using absolute paths, you may decide to put a customization strategy using some process environment variable. We call this the home-bin-dir strategy. E.g. Java executed base on JAVA_HOME or JRE_HOME with the java executable located at ${JAVA_HOME}/bin/java
Stubbing mechanism The aim os stubbing is to replace at execution time the actual executable with a fake one which will yield some kind of pre-defined known outcome. This can be done if there is step in the binary discovery which can be tweaked so that the stud-executable is discovered instead of the actual one. It is the case if binary is specified by name and discovered using PATH or when absolute path is used but with a customization layer in place based on defined environment home variable. The mechanism will just have to mutate the PATH or set the home variable accordingly. Stubbing become complicated if the specification is based on absolute (without customization layer) or relative path because it will require e.g.:
a file overley to hide the original executable
a change of the current directory
an in the flight replacement of the original executable and a rollback. These are either not desirable, or too heavy especially in a unit-test-ish context. The execstub module will therefore only provide stubbing for PATH and Home-Bin-Dir based discovery.
Design elements
Modeling the command line usage We have basically 3 aspects to model:
Starting the process It is modeled using StubRequest which hold the the command name and the execution argument so that the fake execution process can be function of them.
Execution The outcome can be a static sum of known in advance stderr, stdout and exit code. This is referred to as static mode. It may also be desirale to have some side effects, e.g.:
related to the execution itself like creating a file
or related to the unit-test as counting the number of calls In other constellations the outocome may requires some computation to determine stderr/stdout/exicode as function of the request. We refer to these cases as dynamic mode. The execution is therefore modelled as function namely StubFunc. Thus, if required the StubFunc corresponding to the current stubbing will be looked-up and run.
Outcome The stderr/stdout/exit-code part of the outcome is modeled as ExecOutcome. The execution the the stubfunc may result however in an error. Such error is not an actual sub-process execution erroneous outcome. It therefore must be modeled separately as opposed to be added to stderr and setting a non-zero exit code. Such and error can be communicated by using the ExecOutcome field InternalErrTxt.
Stub-Executable It is the fake executable used to effectuate the overall outcome. It may release the outcome by itself. It may also cooperate with a test helper to achieve the outcome. The following command line is used: /tmp/go-build720053430/b001/xyz.test -test.run=TestHelperProcess -- arg1 arg2 arg3 It is not possible to use the test helper directly because we will need to inject the args test.run and -- at the actual execution call site. Two kinds of stub-executable are provided:
bash based, which uses a bash script
exec base, which used an go based executable Obviously the go based one is bound to support more platform than the bash based one. Both executable are genric and need context information about the stubbing for which they will be used. This context data is modeled as CmdConfig and saved as file alongside the executable.
Inter process communication (IPC) to support dynamic outcomes The invocation of a StubFunc to produce a dynamic outcome is done in the unit test process. There is no means to access the stub function directly from the fake process execution. IPC is used here to allow the fake process to issue a StubRequest and receive an ExecOutcome. The Serialization/Deserialization mechanism needed for IPC must be understood by both parties (mind bash) and satisfy a domain specific requirement of been able to cope with multiline sterr and stdout. We choose a combination of :
base64 to encode the discrete data (e.g. stderr, stdout)
and comma separated value (easily encoded and decoded in bash) for envelope encoding Named pipes are used for the actual data transport, because they are file based and easily handle in different setups (e.g. in bash).
Stubbing Setup An ExecStubber is provided to manage the stubbing setup. Its key feature is to setup an invokation of a StubFunc to replace the execution of a process, See ExecStubber.WhenExecDoStubFunc(...). It allows settings beyond the basic requirement of specifying the executable to be stubbed and a StubFunc replacement. This is modeled using Settings, which provides the following configuration options:
Selecting and customizing the discovery mode (PATH vs. Home-Bin-Dir)
Selecting the stub executable (Bash vs. Exec(go based))
Selecting the stubbing mode (static vs. dynamic)
Specifying the test process helper method name
specifying a timeout for a stub sub-process execution A stubbing setup is identified by a key. The key can be use to:
unwind the setup(Unregister(..))
access the stubbing data basis of static requests (FindAllPersistedStubRequests, DeleteAllPersistedStubRequests ).
Concurrency and parallelism The mutation of the process environment is the key enabling mechanism of Execstub. The process environment must therefore be guarded against issues of concurrency and parallelism. This also mean that is not possible to have a parallel stubbing setup for the same executable within a test process. The outcome will otherwise be non-deterministic because the setting will likely override each other. ExecStubber provides a locking mechanism to realize the serialization of the mutation of environment. For this to work correctly however there must only be one ExecStubber per unit test process. Note that the code under test can still execute its sub-processes concurrently or in parallel. The correctness of the outcome here dependents on the implementation of the StubFunc function being used. Static outcomes without side effect are of course always deterministic.
Example (DynamicDefaultSettings) ¶
stubber := NewExecStubber() defer stubber.CleanUp() staticOutcome := comproto.ExecOutcome{ Stderr: "err1", Stdout: "sout1", ExitCode: 0, } recStubFunc, reqStore := comproto.RecordingExecutions( comproto.AdaptOutcomeToCmdStub(&staticOutcome)) key, err := stubber.WhenExecDoStubFunc(recStubFunc, rt.EnsureHasExecExt("SuperExe"), comproto.Settings{}) ifNotEqPanic(nil, err, "fail to setup stub") cmd := exec.Command("SuperExe", "arg1", "argb") var bufStderr, bufStdout bytes.Buffer cmd.Stderr = &bufStderr cmd.Stdout = &bufStdout err = cmd.Run() ifNotEqPanic(nil, err, "should have hat successful execution") // accessing and checking stubrequest dynymic mode gotRequests := *reqStore wanRequets := []comproto.StubRequest{ { CmdName: rt.EnsureHasExecExt("SuperExe"), Args: []string{"arg1", "argb"}, Key: key, }, } ifNotEqPanic(wanRequets, gotRequests, "unexpected stub requests") // accessing and checking outcome gotStderr := bufStderr.String() ifNotEqPanic(staticOutcome.Stderr, gotStderr, "unexpected stderr") gotStdout := bufStdout.String() ifNotEqPanic(staticOutcome.Stdout, gotStdout, "unexpected stdout") gotExitCode := cmd.ProcessState.ExitCode() ifNotEqPanic(int(staticOutcome.ExitCode), gotExitCode, "unexpected exec exit code")
Output:
Example (DynamicWithTestHelperProc) ¶
stubber := NewExecStubber() defer stubber.CleanUp() staticOutcome := comproto.ExecOutcome{ Stderr: "", Stdout: `REPOSITORY:TAG golang:1.14 golang:latest golang:1.14-alpine3.12 ubuntu:18.04`, ExitCode: 0, } recStubFunc, reqStore := comproto.RecordingExecutions( comproto.AdaptOutcomeToCmdStub(&staticOutcome)) setting := comproto.Settings{TestHelperProcessMethodName: "TestHelperProcExample_dynamic"} key, err := stubber.WhenExecDoStubFunc(recStubFunc, rt.EnsureHasExecExt("docker"), setting) ifNotEqPanic(nil, err, "fail to setip stub") // args := []string{"image", "ls", "--format", "\"{{.Repository}}:{{.Tag}}\""} args := []string{"image", "ls", "--format", "table '{{.Repository}}:{{.Tag}}'"} cmd := exec.Command("docker", args...) var bufStderr, bufStdout bytes.Buffer cmd.Stderr = &bufStderr cmd.Stdout = &bufStdout err = cmd.Run() ifNotEqPanic(nil, err, "exit code set to 0 ==> execution should succeed") // accessing and checking stubrequest dynymic mode gotRequests := *reqStore wanRequets := []comproto.StubRequest{ { CmdName: rt.EnsureHasExecExt("docker"), Args: args, Key: key, }, } ifNotEqPanic(wanRequets, gotRequests, "unexpected stub requests") // accessing and checking outcome gotStderr := bufStderr.String() ifNotEqPanic(staticOutcome.Stderr, gotStderr, "unexpected stderr") gotStdout := bufStdout.String() ifNotEqPanic(staticOutcome.Stdout, gotStdout, "unexpected stdout") gotExitCode := cmd.ProcessState.ExitCode() ifNotEqPanic(int(staticOutcome.ExitCode), gotExitCode, "unexpected exec exit code")
Output:
Example (HomeBinDir) ¶
stubber := NewExecStubber() defer stubber.CleanUp() staticOutcome := comproto.ExecOutcome{ Stderr: "", Stdout: "%s openjdk version \"11.x.x\" 2020-mm-dd", ExitCode: 0, } recStubFunc, reqStore := comproto.RecordingExecutions( comproto.AdaptOutcomeToCmdStub(&staticOutcome)) settings := comproto.Settings{ DiscoveredBy: comproto.DiscoveredByHomeBinDir, DiscoveredByHomeDirBinData: comproto.DiscoveredByHomeDirBinData{ EnvHomeKey: "JAVA_HOME", BinDirs: []string{"bin"}, }, ExecType: comproto.ExecTypeExe, } key, err := stubber.WhenExecDoStubFunc(recStubFunc, rt.EnsureHasExecExt("java"), settings) ifNotEqPanic(nil, err, "fail to setip stub") javaCmd := os.ExpandEnv("${JAVA_HOME}/bin/java") cmd := exec.Command(javaCmd, "-version") var bufStderr, bufStdout bytes.Buffer cmd.Stderr = &bufStderr cmd.Stdout = &bufStdout err = cmd.Run() ifNotEqPanic(nil, err, "should have hat successful execution") // accessing and checking stubrequest dynymic mode wantRequests := []comproto.StubRequest{ { CmdName: rt.EnsureHasExecExt("java"), Args: []string{"-version"}, Key: key, }, } ifNotEqPanic(wantRequests, *reqStore, "unexpected stub requests") // accessing and checking outcome gotStderr := bufStderr.String() ifNotEqPanic(staticOutcome.Stderr, gotStderr, "unexpected stderr //stdout:"+bufStdout.String()) gotStdout := bufStdout.String() ifNotEqPanic(staticOutcome.Stdout, gotStdout, "unexpected stdout") gotExitCode := cmd.ProcessState.ExitCode() ifNotEqPanic(int(staticOutcome.ExitCode), gotExitCode, "unexpected exec exit code")
Output:
Example (Static) ¶
stubber := NewExecStubber() defer stubber.CleanUp() staticOutcome := comproto.ExecOutcome{ Stderr: "err1", Stdout: "sout1", ExitCode: 0, } recStubFunc, reqStore := comproto.RecordingExecutions( comproto.AdaptOutcomeToCmdStub(&staticOutcome)) settings := comproto.Settings{Mode: comproto.StubbingModeStatic} key, err := stubber.WhenExecDoStubFunc(recStubFunc, rt.EnsureHasExecExt("SuperExe"), settings) ifNotEqPanic(nil, err, "fail to setup stub") ifNotEqPanic( []comproto.StubRequest{{Key: key, CmdName: rt.EnsureHasExecExt("SuperExe"), Args: nil}}, *reqStore, "Static mod evaluate StubFunc at setup with a stubrequest havin nil args") *reqStore = (*reqStore)[:0] cmd := exec.Command("SuperExe", "arg1", "argb") var bufStderr, bufStdout bytes.Buffer cmd.Stderr = &bufStderr cmd.Stdout = &bufStdout err = cmd.Run() ifNotEqPanic(nil, err, "should have hat successful execution") // accessing and checking stubrequest static mode ifNotEqPanic(0, len(*reqStore), "Unexpected StubFunc call in static mode") gotRequests, err := stubber.FindAllPersistedStubRequests(key) ifNotEqPanic(nil, err, "fail to find all persisted stub request") wanRequets := []comproto.StubRequest{ { CmdName: rt.EnsureHasExecExt("SuperExe"), Args: []string{"arg1", "argb"}, Key: key, }, } ifNotEqPanic(wanRequets, *gotRequests, "unexpected stub requests") // accessing and checking outcome gotStderr := bufStderr.String() ifNotEqPanic(staticOutcome.Stderr, gotStderr, "unexpected stderr") gotStdout := bufStdout.String() ifNotEqPanic(staticOutcome.Stdout, gotStdout, "unexpected stdout") gotExitCode := cmd.ProcessState.ExitCode() ifNotEqPanic(int(staticOutcome.ExitCode), gotExitCode, "unexpected exec exit code")
Output:
Example (StaticWithTestHelperProc) ¶
stubber := NewExecStubber() defer stubber.CleanUp() staticOutcome := comproto.ExecOutcome{ Stderr: "err1", Stdout: "sout1", ExitCode: 0, } recStubFunc, reqStore := comproto.RecordingExecutions( comproto.AdaptOutcomeToCmdStub(&staticOutcome)) settings := comproto.Settings{ Mode: comproto.StubbingModeStatic, TestHelperProcessMethodName: "TestHelperProcExample_static", } key, err := stubber.WhenExecDoStubFunc(recStubFunc, rt.EnsureHasExecExt("SuperExe"), settings) ifNotEqPanic(nil, err, "fail to setip stub") ifNotEqPanic( []comproto.StubRequest{{Key: key, CmdName: rt.EnsureHasExecExt("SuperExe"), Args: nil}}, *reqStore, "Static mod evaluate StubFunc at setup with stubrequest having nil args") *reqStore = (*reqStore)[:0] cmd := exec.Command("SuperExe", "arg1", "argb") var bufStderr, bufStdout bytes.Buffer cmd.Stderr = &bufStderr cmd.Stdout = &bufStdout err = cmd.Run() ifNotEqPanic(nil, err, "should have hat successful execution") // accessing and checking stubrequest static mode ifNotEqPanic(0, len(*reqStore), "Unexpected StubFunc call in static mode") gotRequests, err := stubber.FindAllPersistedStubRequests(key) ifNotEqPanic(nil, err, "fail to find all persisted stub request") wanRequets := []comproto.StubRequest{ { CmdName: rt.EnsureHasExecExt("SuperExe"), Args: []string{"arg1", "argb"}, Key: key, }, } ifNotEqPanic(wanRequets, *gotRequests, "unexpected stub requests") // accessing and checking outcome gotStderr := bufStderr.String() ifNotEqPanic(staticOutcome.Stderr, gotStderr, "unexpected stderr") gotStdout := bufStdout.String() ifNotEqPanic("extra_side_effect_"+staticOutcome.Stdout, gotStdout, "unexpected stdout") gotExitCode := cmd.ProcessState.ExitCode() ifNotEqPanic(int(staticOutcome.ExitCode), gotExitCode, "unexpected exec exit code")
Output:
Index ¶
- type ExecStubber
- func (stubber *ExecStubber) CleanUp()
- func (stubber *ExecStubber) DeleteAllPersistedStubRequests(stubKey string) (err error)
- func (stubber *ExecStubber) FindAllPersistedStubRequests(stubKey string) (peristedRequests *[]comproto.StubRequest, err error)
- func (stubber *ExecStubber) Unregister(key string)
- func (stubber *ExecStubber) WhenExecDoStubFunc(cmdStub comproto.StubFunc, cmdToStub string, settings comproto.Settings) (key string, err error)
Examples ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type ExecStubber ¶
type ExecStubber struct {
// contains filtered or unexported fields
}
ExecStubber provides a mechanism to stub command executions.
func NewExecStubber ¶
func NewExecStubber() *ExecStubber
NewExecStubber constructs a new ExecStubber.
func (*ExecStubber) CleanUp ¶
func (stubber *ExecStubber) CleanUp()
CleanUp cleanup resources allocated for stubbing. The stubber become unsable.
func (*ExecStubber) DeleteAllPersistedStubRequests ¶
func (stubber *ExecStubber) DeleteAllPersistedStubRequests( stubKey string, ) (err error)
DeleteAllPersistedStubRequests deletes all stub-requests which have been persisted into the data basis belonging to the stubbing specification identified by the given key.
func (*ExecStubber) FindAllPersistedStubRequests ¶
func (stubber *ExecStubber) FindAllPersistedStubRequests( stubKey string, ) (peristedRequests *[]comproto.StubRequest, err error)
FindAllPersistedStubRequests returns all stub-requests which have been persisted into the data basis belonging to the stubbing specification identified by the given key. Note that stub-requests are only persisted in static mode.
func (*ExecStubber) Unregister ¶
func (stubber *ExecStubber) Unregister(key string)
Unregister remove the stubbing spec associated with the given key.
func (*ExecStubber) WhenExecDoStubFunc ¶
func (stubber *ExecStubber) WhenExecDoStubFunc( cmdStub comproto.StubFunc, cmdToStub string, settings comproto.Settings, ) (key string, err error)
WhenExecDoStubFunc configures stubbing so that the execution of the command is replaced by the call of StubFunc. The following default are used for settings: - Discovery: by PATH - Stubbing mode: dynamic - Stub exececuatble: Exec based