Structy

Structy is an irresponsibly dumb and simple serialization/deserialization library for C, Python, and vanilla JavaScript. You can think of it like protobuf, thrift, flatbuffers, etc. but imagine that instead of a team of engineers maintaining it, it's instead written by a single moron.

Structy was created to exchange data between C-based firmware on embedded devices and Python- and JavaScript-based programming, test, and calibration scripts running on a big-girl computer. As such, its C implementation is designed specifically for embedded devices: it doesn't do any dynamic allocation and it doesn't have any fancy code for optimizations.

Structy's design goals:

Be small, be simple, be useful

Explicit non-goals:

Be fast, be clever

If you want something far more thought out and comprehensive I'd suggest checking out things like Kaitai Struct.

Using structy

Like protobuf and other complicated data exchange libraries, Structy uses a schema. Structy's schemas use Python's syntax (so it can be lazy and re-use Python's parser):

class UserSettings:
    brightness : uint8 = 127
    dark_mode: bool = False
    user_id : uint32

With this nonsense you can run Structy's generator to generate C, JavaScript, and Python code for this "struct".

$ python3 -m pip install structy
$ structy user_settings.schema --c generated

For C, it just generates a normal struct. You can include it and use it just as you would any other struct:

#include "generated/user_settings.h"

struct UserSettings settings = {
    .brightness = 127,
    .dark_mode = true,
    .user_id = 6,
};

It also creates functions for initialization of default values (UserSettings_init), serialization (UserSettings_pack), and deserialization (UserSettings_unpack).

C implementation

Structy's C implementation is written specifically with microcontrollers in mind. Notably:

• The runtime is tiny, coming in at right at ~250 lines of code and compiles to less than 1kB of thumb code.
• Never uses the heap
• Uses very very little stack space
• Compiles cleanly with -Werror -Wall -Wextra -Wpedantic -std=c17
• It includes optional support for libfixmath's fix16_t.

Including Structy's runtime

You must include the files in runtimes/c in your project to use Structy-generated C code.

Using generated code

Structy generates the struct definition and four functions for use with the struct:

Initialization

void StructName_init(struct StructName* inst);

Initializes an instance of the struct with the default values specified in the struct's schema.

Pack

struct StructyResult StructName_pack(const struct StructName* inst, uint8_t* buf);

Packs an instance of the struct into the given buffer. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes. Example:

uint8_t output_buffer[STRUCTNAME_PACKED_SIZE];
StructName_pack(&instance, output_buffer);

Unpack

struct StructyResult StructName_unpack(struct StructName* inst, const uint8_t* buf);

Unpacks the buffer into the given instance. The buffer must have at least STRUCTNAME_PACKED_SIZE bytes.

Print

void StructName_print(const struct StructName* inst);

Prints out the struct in a nicely formatted way, for example:

struct UserSettings @ 0x0B00B135
- brightness: 127
- dark_mode: 1
- user_id: 6

Since the runtime is designed for embedded systems, Structy doesn't hardcode printf here. You can override the print function used by creating a structy_config.h file and setting the STRUCTY_PRINTF macro:

#include "super_cool_printf.h"

#define STRUCTY_PRINTF(...) super_cool_printf(__VA_ARGS__)

By default, Structy will try to use <stdio.h>'s printf if you're on a big girl computer. If you're on a 32-bit ARM system, Structy will check and see if you have mpland's embedded-friendly printf and use that if you have it. Otherwise, it'll disable printing.

Python implementation

Structy's Python implementation is written to be simple, not fast or "powerful" or whatever. Some notes:

• The runtime depends on Python 3.7+
• The runtime has complete type annotations.
• The runtime supports converting floats between libfixmath's fix16 during packing & unpacking.
• It's built on top of Python's excellent struct module.

Including Structy's runtime

The runtime can be installed by installing the structy package:

$ python3 -m pip install structy

Using generated code

Structy generates the struct as a dataclass in its own module.

Initialization

Since it's a dataclass, an empty constructor gives each field the default value specified in the .structy definition file, but you can pass keyword arguments to override them:

# Create an instance with default values for all fields.
inst = StructName()

# Or, override fields with a custom value while creating an instance.
inst = StructName(field_name=something)

Pack

result: bytes = inst.pack()

Packs an instance and returns a bytes object with the data. len(result) will be StructName.PACKED_SIZE.

Unpack

inst = StructName.unpack(data)

Unpacks the bytes-like buffer into a new instance. The buffer must be at least StructName.PACKED_SIZE long.

Print

Structy does define a nice __str__ method:

UserSettings:
· brightness:   127
· dark_mode:    False
· user_id:      42

JavaScript implementation

Structy's JavaScript implementation, like the others, is intended to be simple. Here's some notes on it:

• It's implemented as a JavaScript module.
• It's meant to be used in a web browser. It also works with Deno, but I only use Deno for testing.
• It's implemented using Javascript classes and uses public static members which means Safari will probably be a dick about it until like Safari 14.1 or 15, who the heck knows?

Including Structy's runtime

The runtime is a single file located at runtimes/js/structy.js. There's no Node/NPM package because the last time I used NPM my nose suddenly starting bleeding and I passed out and woke up with several mysterious lesions, and the last time I used Node.js I managed to somehow unleash a 10,000 year-old demon who's currently causing minor chaos by removing stop signs in low-traffic rural areas.

So just copy it into your project next to wherever you're going to place the generated code, like we did when the web was young. If you're lazy (and you probably are), just run this:

$ wget https://raw.githubusercontent.com/theacodes/structy/runtimes/js/structy.js
$ wget https://raw.githubusercontent.com/theacodes/structy/third_party/struct.js/struct.mjs

Those commands also copy in the one dependency: the excellent and tiny struct.js module- yes, I know it's confusing to have structy.js and struct.js, but I'm sure you'll get used to it. struct.js implements Python struct module in JavaScript which is great because I can write less code.

Using generated code

Structy generates the struct as a nice class in its own module. The module's only export is the class, so you can import it like this:

<script type="module">
import StructName from "./structname.js";
</script>

Initialization

It's a JavaScript class so just use new to make an instance. It uses a named parameter pattern so you can set the value of individual fields during construction without needing to specify all of them:

// Default values.
inst = new StructName();
// Override a default value by passing in an object literal
inst = new StructName({field_name: something});

Pack

result = inst.pack();

Packs an instance and returns a Uint8Array object with the data. result.length() will be StructName.packed_size.

Unpack

inst = StructName.unpack(data);

Unpacks the Uint8Array or ArrayBuffer into a new instance. The buffer must be at least StructName.packed_size long.

FAQ

Does Structy support arrays/lists?

No, Struct structs must be a deterministic length when encoded.

How about bitfields?

No yet, but it could. I'm just too lazy.

Little-endian? Mixed-endianess?

No. Supporting anything other than big-endian would complicate the C runtime and I don't want to do that. Idk, maybe I could be talked into accepting a PR for it.

Custom types?

No, but it's possible in the future. There's a little bit of this thought out because Structy supports Q16.16 fixed-point values.

Nested structs?

No, but it could be added. I just haven't needed it yet.

Why does structy use snake_case for properties and methods even in C and JS where it's common to use lowerCamelCase?

Mostly because I want data access to be identical in all languages, but also because I deeply, deeply, hate lowerCamelCase. I find it hard to read.

Support / issues / etc

Unlike some of my other open-source projects, this project was made with one user in mind: me. I made it open source in case someone else finds it useful but I am not in the business of supporting this project. If you run into issues or need help feel free to submit an issue on GitHub, however, please know that I do not feel any obligation to support this project.

License

The Structy generator and runtime components are all published under the MIT license. See LICENSE for the full text.