Power Overwhelming

This project provides a library for measuring the power consumption of GPUs (and other system components) by various means.

Note If you are here for the instructions for building a bench table for measuring GPU power consumption, look in the docs folder. Over there, you also find some lessons we learned about measuring power with Tinkerforge bricklets.

Note The papers "Power Overwhelming: Quantifying the Energy Cost of Visualisation" and "Power overwhelming: The One With the Oscilloscopes", for which this software was written, can be found on IEEEXplore and on Springer Link respectively.

Building the library

The library is self-contained and most optional external dependencies are in the third_party folder. External dependencies from GitHub are fetched by CMake. Once built, the external dependencies are invisible to the user of the library. However, the required DLLs must be present on the target machine. Configure the project using CMake and build with Visual Studio or alike.

Sensors included in the repository

SDKs included in the repository are the AMD Display Library (ADL), the NVIDIA Management Library (NVML) and support for Tinkerforge bricks and bricklets. On Windows 11, the Energy Meter Interface can be used to query the RAPL (Running Average Power Limit Energy Reporting) registers of the system. This sensor might be available on certain Windows 10 installations, but according to a presentation by the Firefox team, specialised hardware is required for that. The msr_sensor provides access to the RAPL registers on Linux and on Windows systems that run the pwrowgrapdrv driver.

Support for Rohde & Schwarz instruments

The library supports reading Rohde & Schwarz oscilloscopes of the RTB 2000 family and HMC8015 power analysers. In order for this to work, VISA must be installed on the development machine. You can download the drivers from https://www.rohde-schwarz.com/de/driver-pages/fernsteuerung/3-visa-and-tools_231388.html. The VISA installation is automatically detected by CMAKE. If VISA was found POWER_OVERWHELMING_WITH_VISA will be defined. Otherwise, VISA will not be supported and using it will fail at runtime.

Only the power analyser is currently ready to use, support for automating oscilloscopes is work in progress.

Using the library

The podump demo application is a good starting point to familiarise oneself with the library. It contains a sample for each sensor available. Unfortunately, the way sensors are identified and instantiates is dependent on the underlying technology. For instance, ADL allows for creating sensors for the PCI device ID show in Windows task manager whereas NVML uses a custom GUID or the PCI bus ID. Whenever possible, the sensors provide a static factory method for_all(sensor_type *dst, const std::size_t cnt) that creates all available sensors of this type. The usage pattern for this API is:

using namespace visus::power_overwhelming;

std::vector<adl_sensor> sensors;
// Call 'for_all' to determine the required buffer size.
sensors.resize(adl_sensor::for_all(nullptr, 0));
// Call 'for_all' to actually get the sensors.
adl_sensor::for_all(sensors.data(), sensors.size());

Some sensors have a slightly different API. For instance, sensors using Tinkerforge Voltage/Current Bricklets are not directly created, but require enumerating a descriptor object that in turn can be used to make the sensor:

using namespace visus::power_overwhelming;

std::vector<tinkerforge_sensor_definition> descs;
// Call 'get_definitions' to find out how many definitions there are.
descs.resize(tinkerforge_sensor::get_definitions(nullptr, 0));
// Call 'get_definitions' to get the actual descriptors.
auto cnt = tinkerforge_sensor::get_definitions(descs.data(), descs.size());
// As Tinkerforge sensors can be hot-plugged, it might occur that there are now
// less sensors than initially reported. In this case, we need to truncate the
// array.
if (cnt < descs.size()) {
    descs.resize(cnt);
}

// Create a sensor for each of the descriptors.
std::vector<tinkerforge_sensor> sensors;
sensors.reserve(descs.size());

for (auto& d : descs) {
    // Add a description to the sensors in order to identify them later.
    // Typically, you would have map from the unique UID to a description
    // of what is attached to the bricklet.
    d.description(L"One of my great sensors");
    sensors.emplace_back(d);
}

The sensors returned are objects based on the PIMPL pattern. While they cannot be copied, the can be moved around. If the sensor object is destroyed while holding a valid implementation pointer, the sensor itself is freed.

Sensor readings are obtained via the sample method. The synchronous one returns a single reading with a a timestamp:

auto m = sensor.sample();
std::wcout << m.timestamp() << L": S = " << m.power() << " VA << std::endl;

If possible, there is also an asynchronous version that delivers samples to a user-specified callback function. When supported by the API, this method uses the asynchronicity of the API. Otherwise, the library will start a sampler thread that regularly calls the synchronous version. Sensors will be grouped into as few sampler threads as possible:

sensors.sample([](const measurement& m) {
    std::wcout << m.timestamp() << L": S = " << m.power() << " VA << std::endl;
});
// Do something else in this thread; afterwards, stop the asynchronous sampling
// by passing nullptr as callback.
sensor.sample(nullptr);

The collector class is a convenient way of sampling all sensors the library can find on the system it is running:

auto collector = collector::for_all(L"output.csv");
collector.start();
// Do something else in this thread; afterwards, stop the collector again.
collector.stop();

Using the Tinkerforge bricklets for measuring the power lanes of the GPU requires a custom setup. We have compiled some instructions for doing that.

Extending the library

Adding new kinds of sensors requires several steps. First, a new sensor class is required, which needs to satisfy the following requirements:

• All sensors must inherit from visus::power_overwhelming::sensor and implement the interface defined therein.
• The name should end with _sensor.
• All implementation details must be hidden from the public interface using the PIMPL pattern.
• The sensor class must support move semantics (move constructor and move assignment).
• The sensor class must implement a method static std::size_t for_all(emi_sensor *out_sensors, const std::size_t cnt_sensors) that can be used to retrieve all sensors of this kind that are available on the machine. The method shall always return the number of sensors available, even if out_sensors is nullptr or the buffer is too small to hold all sensors. Sensors shall only be written to out_sensors if the buffer is valid and large enough to hold all of them.

Second, in order to be eligible for the automated enumeration by the sensor utility functions,

• a template specialisation of visus::power_overwhelming::detail::sensor_desc must be provided in sensor_desc.h, which provides means to serialise and deserialise sensors,
• the class must be added to the sensor_list template at the bottom of sensor_desc.h.

The specialisation of visus::power_overwhelming::detail::sensor_desc must fulfil the following contract:

• It must have a member static constexpr const char *type_name specifing the unique name of the sensor, which can be declared using the POWER_OVERWHELMING_DECLARE_SENSOR_NAME macro.
• It must have a member static constexpr bool intrinsic_async specifying whether the sensor can run asynchronously without emulating it by starting a sampler thread that regularly polls the sensor. You can use the POWER_OVERWHELMING_DECLARE_INTRINSIC_ASYNC to declare this member.
• It must have a method static inline nlohmann::json serialise(const value_type& value) which serialises the given sensor into a JSON representation. The JSON representation must be an object which contains a string field named "type" (use the visus::power_overwhelming::detail::json_field_type constant in sensor_desc.h for the name) which has the type_name constant as its value. This field is used in conjunction with the aforementioned sensor_list to automatically dispatch deserialisation to your specialisation of the traits class.
• It must have a method static inline value_type deserialise(const nlohmann::json& value) which restores a sensor from a given JSON representation.
• If the sensor can serialise all of its instances more efficiently than creating an instance of it and converting these instances to JSON, it can implement a method static inline nlohmann::json serialise_all(void) which serialises all sensors into a JSON array. The library will prefer this method if it is provided.

Acknowledgments

This work was partially funded by Deutsche Forschungsgemeinschaft (DFG) as part of SFB/Transregio 161 (project ID 251654672).