Major memory leak in the C target

[cmnrd]

I tried today to compile and execute the Banking C benchmark. Executing the benchmark very quickly eats up all my memory (I have 32 GiB RAM + 32 GiB swap). I am not sure if this is due to a memory leak or due to extensive memory usage in the C target.

[lhstrh]

Some memory leak issues have been solved in the recent past. Did you use the latest version of the runtime and compiler?

[cmnrd]

I used the 0.5.1. release. I don't think we had any fixes for memory usage since then.

[erlingrj]

Looks like something with the initialization of banks and multiports which doesn't scale. We have the following for loops:

for i=0 to 1000:
for j=0 to 1000:
for k=0 to 1e6
end
end
end

That is 1e12 iterations which all uses the mixed_radix stuff.

Then there is another 1e9 iterations in a loop that initializes a trigger_array. This is probably what is crashing the program. Unfortunately I never understood how banks and multiports are implemented in the C target with "src ranges", "dst ranges" and "mixed radix" so I cannot immedietaly look at the generated files and see what is the issue...

[erlingrj]

Maybe @petervdonovan can shed some light on it? We have a bank of 1000 Account Reactors, each has a 1000-wide multiport input, and a 1000-wide multiport output. It seems that we are allocating memory for every possible connection (1e3*1e3*1e3)

[petervdonovan]

I have used the mixed-radix stuff before but mostly I prefer not to touch it unless this issue is a blocker for an actual user or a blocker for someone's research project. Is this an issue that we need to prioritize?

[cmnrd]

I am a bit confused by the mixed radix stuff too. I thought that is done in the LF compiler. Don't we use the instance graph in the C target, precisely so that we can statically code generate all the connections? 1e12 iterations is really a lot

Why 1e3*1e3*1e3 possible connections? There are 1e6 actually used connections (each reactor instance to each reactor instance).

[petervdonovan]

Don't we use the instance graph in the C target, precisely so that we can statically code generate all the connections?

Well, yes, but generating code corresponding to every single individual connection is not a scalable design, so the design using mixed-radix integers was sort of a compromise I guess. I think that the design for the C target is stuck at near a local optimum because we do not want to completely discard the instance-graph-based design, even though the instance-graph-based design does not (to my knowledge) yield as much benefit as people might have expected years ago when we were still learning and experimenting. I think that we have all suspected for a long time that the current design for the C target is not the best and that the design used in the C++ target is preferable (edit: specifically in the way connections are made; I am not trying to argue that the C++ target design is better in every way, that is not the point), but no one has the time to make everything perfect right now.

[cmnrd]

I have used the mixed-radix stuff before but mostly I prefer not to touch it unless this issue is a blocker for an actual user or a blocker for someone's research project. Is this an issue that we need to prioritize?

It means that we cannot run the C banking benchmark and that we have a serious scaling issue in the C target. As far as I know, this is not blocking anyone. From my side, it is Ok to add this to the backlog.

[lhstrh]

@edwardalee and @erlingrj, since you guys are working heavily on the C runtime, you might want to take this in consideration.

[edwardalee]

This is because there are some uses of interleaved connections that I never figured out how to handle with the mixed radix solution.  I suspect there is a way, but I’m focused on federated and enclaves right now, so this will have to wait.