-
Notifications
You must be signed in to change notification settings - Fork 0
Home
I need to write test automation for end-to-end scenarios using the IaaS offerings from a cloud provider. This includes creating compute, networking, and storage primitives and testing the various APIs. A test might be only interested in a simple operation, say adding a data disk, but before you can invoke that API you need a VM, and to get a VM you need some networking and storage primitives. The test quickly becomes complicated and lengthy (both in terms of code and time to execute). The more the test must do to setup before (and cleanup after) its central action, the more chance there is for something to go wrong causing the test to fail, and likely attributing the failure to the wrong thing.
What I would like is a way to write a test that just focuses on its central action, and expresses the requirements for the inputs it consumes as well as the guarantees about the outputs it produces. Other tests can provide outputs that are suitable to pass as inputs to this test. A sufficiently advanced test runner can orchestrate a sequence of tests cases, passing the resources between them, in order to achieve a test pass that efficiently exercises each test at least once.
It seems obvious that the infrastructure for this system would be useful in a number of other contexts, and so I started this project.
In our proposed system here, we envision the test cases as blocks of code that perform actions on a resource and as a consequence transform that resource from one state to another. Once transformed into a new state, that resource may be suitable to provide as input to another block of code that would further transform the resource into a further state. By representing our knowledge of the state of a resource as a set of predicates at each transform, we can determine through static analysis how to construct the graph of transforms that the resource can flow through.
Let's take a moment to define the basic concepts:
A resource is an object that flows through the graph of transforms. The lifetime if this object begins at the entry of the graph and ends at the exit of the graph.
A predicate is a statement about the resource. Predicates are expressed declaratively so that they can be reasoned about via static analysis, but they also are evaluated and validated at runtime to ensure correctness.
A transform receives a resource that matches a set of input predicates and processes the resource in some way that adheres the set of output predicates.