Add LMS implementation

[RcColes]

Description

This adds the LMS stateful-hash asymmetric signature scheme, based on RFC8554 (https://datatracker.ietf.org/doc/html/rfc8554). It also adds the required LM-OTS implementation.

It's intended as a cut-down implementation, which is focused primarily on firmware asset validation (Though it is suitable for many generic purposes). The main limitation is that only one parameter-set is supported (SHA256_H10), with leads to each private key being able to be used for up to 1024 signatures. Supporting only one parameter-set leads to saving in complexity in memory allocation, and excludes the larger parameter-sets which require complex optimizations in order to be used in constrained memory implementations. The parameter-set with a merkle tree-height of 5 was discounted since 32 messages is likely to be too constrained where keys are embedded in devices and cannot be easily changed.

Status

READY

Requires Backporting

NO

Migrations

NO (only additions to the API, no changes)

[gilles-peskine-arm]

Thank you for contributing this! As this is a substantial amount cryptography code, we'll need to schedule time to review for it.

In the meantime, please fix the CI issues on Travis. Once Travis passes, if you need help with the Jenkins failures, post a comment and we'll help. The Jenkins logs are unfortunately not public, all you can see is a truncated list of names of failing jobs.

[RcColes]

Right, I think I've fixed most of the CI failures now. There seem to be some random failures occurring in the windows build which I think are CI-related and therefore should be resolved by a re-run.

The main remaining issue is a trickier one, since it's related to one of the algorithmic disadvantages of LMS. Each LMOTS key is ~1KiB, and using H=10 each LMS key contains 1024 LMOTS private keys for ~1MiB. As far as I can tell, this size can't be allocated on the heap by some of the tests which is causing the remaining failures.

There are a few options to fix this:

1. We could change the parameter-set to H=5, which has 32 OTS keys per LMS key and hence uses 2^5 times less memory. The downside here is that only 32 signatures can be done with each LMS key, which makes it far less useful, particularly, in firmware validation (since you would reasonably expect more than 32 upgrades).
2. There are some algorithmic options for generating the OTS keys at each signature. This adds some considerable complexity, which I'm quite concerned about.
3. We could try to get the tests to be able to allocate more memory. This doesn't actually solve the problem that most embedded devices would fail to allocate a megabyte of space however. There is an argument that we can just expect signing to not be done on the device (and instead be done on a PC which can allocate a megabyte easily), but I'm not sure how reasonable that is. This doesn't cause an issues with verify-only implementations, since the context only allocates the space once a private key is generated.

None of the options is clearly favorable however. I'd be interested in your thoughts on this @gilles-peskine-arm

[gilles-peskine-arm]

While we don't have a set maximum amount of memory that tests must fit in, we do expect the tests to run on “reasonable” microcontroller boards. A board with 1MB of RAM would definitely be considered reasonable.

There is a facility in the test suite to skip tests that need a large amount of memory: ASSERT_ALLOC_WEAK. It works well when the test needs to allocate that large amount upfront, but not when the problematic allocation comes from a library function. Can you arrange to use that in signature tests? Or, if it doesn't work as is, something like

ret = mbedtls_lms_alloc();
TEST_ASSUME( ret != ERR_OUT_OF_MEMORY );
TEST_ASSERT( ret == 0 );

rather than

TEST_ASSERT( mbedtls_lms_alloc() == 0 );

[RcColes]

Ah yes, returning an OOM error is a good idea, thanks.

[RcColes]

Right, I think that's the CI sorted. Might want to re-tag this one as a needs: review

[daverodgman]

@RcColes looks like there are some test failures in the LMS tests

[gilles-peskine-arm]

Thanks for all the updates. This looks good to me except for a remaining typo in the code snippets to reproduce test data. I'm approving, but I'd prefer to fix this.

Especially since CI found something I didn't.

[gilles-peskine-arm]

With Raef's agreement I've pushed a fix for my last two comments, including the issue that caused test failures. It should be all good now. At least, it looks good to me.

[gilles-peskine-arm]

I am approving this pull request.

I authored three commits at the end, which someone else needs to review.

[RcColes]

I've reviewed Gilles' last 3 commits, and I approve them

[d3zd3z]

This looks good to me. My concerns have been addressed. It will probably need something to get CI to pass, though.

[gilles-peskine-arm]

The failure of all_u16-test_m32_o2 on pr-merge on the internal CI was an infrastructure glitch, and that job passed on OpenCI and on pr-head. So the CI is a pass.