How do you test analogRead and digitalRead?

[jsdevel]

[bxparks]

Please use the GitHub Discussions for general questions as mentioned in the README.md.
Second, you need to provide more context about what you want to do.

[jsdevel]

first of all, thank you for this great lib!

so i'd like to test some code that calls analogRead. i'd like to test what happens when analogRead is a certain value, but i'm having a loss knowing how to go about doing that.

[bxparks]

There are at least 3-4 different ways of doing this, I was hoping to get a sense of the application that you are trying to test, and a sense of your level of expertise of C++.

First, it's important to remember that AUnit and the tests were designed to run on the microcontroller itself. The Arduino API has zero support for testing of any kind, so everything we do is to work around this fundamental limitation of the API. Let's take a look at analogRead() and digitalRead(). If you look at the Arduino.h header file, you will see that these are C functions in the global scope. There is no way to inject a different version of these functions.

In order to test your code that calls these functions, you must manually instrument your code, and manually insert a layer of indirection between your code and the analogRead() and digitalRead() functions. It is outside the scope of AUnit to provide that injection layer, because there are too many different ways of doing it, and the appropriate solution depends on the type of application that you are writing. I'll sketch out a few possibilities below.

Use a custom version of analogRead() and digitalRead()

If your application is written using traditional C procedural style using mostly functions, then instead of using analogRead() and digitalRead() directly, you could insert custom versions of these like this:

extern bool isTestingMode;
extern int analogReadValue;
extern int digitalReadValue;

int injectableAnalogRead(uint8_t pin) {
  if (isTestingMode) {
    return analogReadValue;
  } else {
    return analogRead(pin);
  }
}

int injectableDigitalRead(int8_t pin) {
  if (isTestingMode) {
    return digitalReadValue;
  } else {
    return digitalRead(pin);
  }
}

Then in your app, you call injectableAnalogRead() and injectableDigitalRead(). And you write your AUnit tests using these as well, and in your global setup() of your unit test, you set isTestingMode = true:

test(myTest) {
  analogReadValue = 23;
  // your code
  assertEqual(..., ...);
  ..
}

void setup() {
  isTestingMode = true;
  ...
}

void loop() {
  aunit::TestRunner::run();
}

The disadvantage of this method is that you always pay for the overhead of the isTestingMode flag, even in the production code. It's not as bad as a virtual dispatch (see next), but it's still there.

Use C++ Runtime Polymorphism

If your application is written in OOP style using classes and objects, you could insert a layer of indirection by using a virtual function:

class Reader {
  public:
    virtual int analogRead(uint8 pin) const = 0;
    virtual int digitalRead(uint8_8 pin) const = 0;
    [...]
};

class HardwareReader : public Reader {
  public:
    virtual int analogRead(uint8 pin) const override { return ::analogRead(pin); }
    virtual int digitalRead(uint8_8 pin) const override { return ::digitalRead(pin); }
    [...]
};

class TestableReader : public Reader {
  public:
    int analogRead(uint8 pin) const override { return mAnalogReadValue; }
    int digitalRead(uint8_8 pin) const override { return mDigitalReadValue; }

    void setAnalogReadValue(int value) { mAnalogReadValue = value; }
    void setDigitalReadValue(int value) { mDigitalReadValue = value; }
    [...]

  private:
    int mAnalogReadValue;
    int mDigitalReadValue;
};

Then you write the cod that you want to test using a pointer or reference to the interface Reader:

class MyApp {
  public:
    MyApp(const Reader* reader) : 
      mReader(reader) 
    {}

    void doSomething() {
      int value = mReader->analogRead(pin);
      [...]
    }

    [...]

  private:
    const Reader* mReader;
};

In your real code, you inject the actual HardwareReader:

HardwareReader reader;
MyApp myApp(&reader);

But in your AUnit test code, you inject the TestableReader:

TestableReader reader;
MyApp myApp(&reader);

The disadvantage of this is that dynamic dispatch (i.e. calling functions with the virtual keyword) consumes large amounts of memory on 8-bit processors (30-50 bytes per invocation if I recall), and consume extra CPU time in the actually application. On AVR processors, depending on how many times the virtual methods are called, you may find consuming significantly more flash memory compared to using non-polymorphic functions. (I've seen savings of 500-1500 bytes in a non-trivial app, which is significant if you have only 32kB of flash).

Using C++ Templates for Compile-time Polymorphism

This requires a fairly advanced level of understanding of the C++ language and its template programming techniques. I've been playing around with this the last year or so, and I've had some good success with it, but I have to be honest, this isn't for beginners.

The core idea is similar to the previous runtime polymorphism technique, but you use C++ template to achieve compile-time polymorphism. The advantage is that the compiler optimizes out almost all of the layer of indirection that you add to get testability, and you save in flash memory and performance. The drawback is that the code becomes very complicated.

Here, we have 2 different Readers, but we don't use C++ polymorphic inheritance, there is no Reader pure virtual interface:

class HardwareReader {
  public:
    int analogRead(uint8_t pin) { return ::analogRead(pin); }
    int digitalRead(uint8t_t pin) { return ::digitalRead(pin); }
};

class TestableReader {
  public:
    int analogRead(uint8_t pin) { return mAnalogReadValue; }
    int digitalRead(uint8t_t pin) { return ::digitalRead(pin); }

    int mAnalogReadValue;
    int mDigitalReadValue;
};

Now we write the application, or the code that you want to test as a template class, making the injectable Reader class into a template parameter R:

template <typename R>
class MyApp {
  public:
    MyApp(const R& reader) : // pass by const-reference
       mReader(reader)
    {}
    
    void doSomething() {
      int value = mReader.analogRead(pin);
      [...]
    }

    [...]

  private:
    R mReader;  // hold by-value
};

(Edit: Changed constructor to pass reader by const-reference instead of by-value. C++ has so many gotchas...)

In your real code, you inject the HardwareReader as a compile-time parameter like this:

HardwareReader reader;
MyApp<HardwareReader> myapp(reader);

But in your unit test, you inject the TestableReader like this:

TestableReader reader;
MyApp<TestableReader> myapp(reader);

As I mention, the biggest advantage of this is that you avoid all of the runtime overhead of dynamic polymorphism. The compiler is often able to inline most of the calls through the mReader object, and optimize away almost the entire layer of indirection, so that the generated binary makes just a direct call to analogRead() and digitalRead(). You also save tons of flash memory. The disadvantage is that you now have to write your code, or at least the parts that you want to test, as C++ templates.

I can think of at least one other way of doing things like this, but I hope these 3 techniques points you in the right direction. Remember, the purpose of the AUnit library to give you a framework for making assertions, through the various assertEquals(), assertLess(), etc. macros. It does not care about how you organize your code to provide the ability to unit test your code.

P.S. The code above was written off the top of my head, without any checking of syntax, so there are probably tons of typos and syntax errors. But I hope you get the idea.

[jsdevel]

wow, @bxparks i'm really grateful for you detailed and robust reply! after considering those proposals, i actually decided on something else:

Instead of calling analogRead and digitalRead everywhere in my app, I instead populate a global state object with the values at the beginning of loop(). I then move the rest of my loop() functionality into a doLoop() function in a header file that can then be imported from my test sketch. In this way I can setup the state however I need to in the tests and verify expectations. I also have a seperate state object that represents IO pin output state, and at the end of my loop() function I read each value and digitalWrite accordingly.

[bxparks]

Yup, that works if your state machine is relatively simple, and your event production and event consumption can be decoupled cleanly. But in many cases, it can be difficult to do that decoupling. For example, let's say your digitalRead() has timing requirements and is interspersed with calls to delay() or delayMicroseconds(), and your internal state machine mutates depending on the values that you get from your digitalRead(). (A good example is a button debouncing library, such as my https://github.com/bxparks/AceButton). In such cases, you really want to be able to inject testable versions of various Arduino functions and classes, and the Arduino API makes it difficult. I've had the best success using the C++ Template Compile-time Polymorphism technique.

[jsdevel]

yea. i need to dive into C++. i feel pretty comfortable with C, java, and go, but C++ is completely different...

[bxparks]

Learning some amount of C++ for Arduino stuff will be very useful. Personally though, the cognitive complexity of the language is starting to wear me down, even after (maybe especially after?) using it on and off for 30 years. C does not give me enough abstractions and expressive power, but C++ is out of control. So I'm looking for alternatives.

I looked at MicroPython. But Python is actually a really complex language underneath its friendly veneer. The runtime overhead on a microcontroller seems too much for anything more than prototypes, and MicroPython requires special language extensions to make the resource consumption tolerable on small microcontrollers. The code deployment process also seems inefficient: I don't want to load the actual Python source code to the microcontroller, I want to compile down to bytecode on the host machine, catch all the compile-time errors, then load just the bytecode. This doesn't seem to be supported, at least I couldn't find it. And the ecosystem is fractured into CircuitPython and MicroPython.

TinyGo is next on my list to evaluate, and seems promising.

Maybe I'll look at Nim, which supposed to transpile to C, but I don't know well it supports the Arduino or other microcontroller environments. And maybe Zig, although they say that a stable v1.0 is several years away.

[hsaturn]

Maybe rust will fullfill your needs :-D

I'm working with C++ for 30 years also. I still like this langage, but yes, the complexity of this langage is a brake ! I'll will have a look to Rust this year.

[bxparks]

See bxparks/EpoxyDuino#76 for a similar discussion about mocking/injecting Arduino functions.