A way to prevent new sensors from throwing off aggregate values automatically

[bmcclure]

I would love if there was some way that Magic Areas could tell the difference between device temperature sensors and environmental temperature sensors. At the very least, I wish there was a way for device temperature sensors to not automatically get added to my aggregate before I have a chance to remove them.

Currently, excluding all device temp sensors from each MA area seems to be the only way to get an accurate area temperature aggregation. However, this can end up being a significant hassle because whenever I add a new device to my home that has a temperature sensor, then next time HA restarts and MA sees the new entity, suddenly my area temperature automations get thrown out of whack until I reconfigure the area.

Perhaps ultimately this is a Home Assistant issue--the need to separate environmental from device temperatures with some sort of attribute so that other integrations know which is which. But barring a generic HA solution, I wonder if there is a way to improve the disruption this causes in between the time that a new device is added and MA is manually reconfigured to exclude it.

The only somewhat-elegant way I can think of is to allow for aggregate groups to be opt-in. Each type of aggregate sensor could have a field for selecting the entities of that type from the available entities in that area, and if none are selected then it could default to the current behavior of including all entities in the area.

Would something like this be an acceptable solution? I would be happy to work on a PR for this but I'd rather not spend the time if there is a better solution or if there is no interest in this functionality beyond myself.

[cerebrate]

Yeah, this same basic issue is a big part of what I want #178 for. For what it's worth, your proposed PR would also be a workable solution for my issue. 😁

[jseidl]

I really want to treat exclusions at the area level, to not overcomplicate the aggregates, since with the current UI on HA that would make a very big configuration screen (maybe even bigger than light groups). Also I don't know how would I add dynamically fields, say, you selected the temperature device_class, then how to realtime load all entities there and populate fields....

It would require MA to migrate to a UI such as Alarmo does which honestly I have no idea on how to even start doing that.

I think wildcard exclusions mentioned by @cerebrate should do the trick better than having those per-device-class-exclusions.

What do you think @bmcclure ?

[bmcclure]

I think that wildcard exclusions would be a huge help, however I am pretty sure it would only solve part of the issue. The other issue is that you may not know you have to exclude something until it's already been added. One somewhat extreme example scenario:

1. HA automations are configured based on MA's aggregate temperature sensor for a room, such as controlling an HVAC system
2. A new water heater is auto-discovered by an integration previously set up, which has a temperature sensor with a name you didn't know would need to be excluded previously.
3. Everything seems normal, since MA doesn't include the entity in the aggregation until the next restart.
4. Sometime later, HA is restarted, and the aggregate temperature rises greatly. Perhaps the HVAC switches to Cool mode in the middle of the winter and the pipes are at risk of freezing until the issue is discovered.

In a perfect world, wildcard excludes could perhaps help with this, but in reality I think that would often not be the case. There are likely other safeguards that could be put in place, but due to the basic nature of not knowing what you don't know, you often have to react to entities after they have already been added to Home Assistant, rather than knowing ahead of time what entities will be there.

Perhaps it may be best to just not drive critical automations off of MA aggregates in case unexpected entities start getting included. But that limits the usefulness of the aggregates, so it would be nice if there was some safe way to use them in scenarios where you don't necessarily want every new entity to get aggregated.

I don't know of a good solution offhand. It would be neat if there was an optional last step in configuring an area where it listed all the aggregated entities and let you exclude certain ones. But, that seems like it would also require only adding entities to aggregates when (re-)configuring MA, and not when restarting Home Assistant, and I'm not sure how easy that would be.

I may be over-thinking it, and I may be me in the minority of users that are constantly adding and changing entities, but I keep running into issues with my aggregate sensors getting thrown off until I later realize it and go back to fix them.

[cerebrate]

I don't know of a good solution offhand. It would be neat if there was an optional last step in configuring an area where it listed all the aggregated entities and let you exclude certain ones. But, that seems like it would also require only adding entities to aggregates when (re-)configuring MA, and not when restarting Home Assistant, and I'm not sure how easy that would be.

I'd have to say I'm not so fond of this notion because it does somewhat take the magic out of Magic Areas. 🙂 I mean, I started using it originally because I no longer had to micromanage a passel of per-area groups and templates to get my areas to behave the way I wanted.

That said, more transparency would be handy such that we don't need to remember to check if we forgot any entities that would be included, or if entities were included that we didn't believe would be. @jseidl , would it be possible to give MA a memory for its previous [internal] configuration, such that when it rebuilds the set of sensors it's using at startup or configuration-edit, we can access the changes since last time?

I'm imagining something like a "sensor.area_foo_entity_changes" sensor, where the state is a count of changes and a couple of attributes ("added", "removed") have lists of entities being included and/or excluded for that area that weren't previously. That gives us one place to look for config differences, and those of us who change things around a lot can write a simple automation to pop a notification any time the state of that sensor changes and isn't 0.

Concerning exclusions: As an alternative/improvement to wildcards, I'd like to also suggest the possibility of excluding entities at the device level, as well as the entity level. This exclusion setting would be per-area, just like the current per-entity one, and would exclude all entities belonging to a particular device.

This seems like it would be relatively simple to implement (easy enough to enumerate all devices in an area, and entities know which device they belong to), and yet would greatly shorten the exclusion list. It also has the advantage that it doesn't require all entities of a device to share a common feature in their names, or risk accidentally catching similarly-named entities in the exclusion.

[bmcclure]

It wouldn't have to remove any magic because it would be optional, if you don't use the includes then the behavior would be exactly like it is today.

There are two ways I can think of to be able to rely on the aggregated sensor values that MA provides for an important application:

1. Allow specifying includes for certain aggregates
2. Notify you of newly-detected aggregate items and give you a chance to exclude them before their values are actually used.

Without this, I don't think it's possible to avoid the potential issue I mentioned previously, however I understand if that is simply not the intention of these aggregates.

Wildcard exclusions could be useful for this purpose too, since you could add the wildcard exclusion after the new entity is created (so you know what the name is) but before Home Assistant restarts next time.

However that still leaves the issue of not realizing there's a new entity you need to exclude, so having a sensor or attribute for which entities have been added since last time Magic Areas was configured sounds like a cool idea to me too.

[bmcclure]

I've been thinking more about this the past few days, and have come to the conclusion that wildcard excludes would be good enough for the overall goal of preventing new sensors from throwing off my aggregates without me having a chance to exclude them first.

A sensor or attribute of some kind indicating entities that have been newly detected by Magic Areas would still be awesome though, too. It could probably just be what was detected since the last HA restart, but I'm not sure if that information is known to MA currently or not.