I have used atop succesfully in the past with other setups, but not with linux containers. The case scenario is this:

1. I install atop in the LXD machine
2. I run some hoggy process inside the container
3. Atop shows the running process, so you can actively find out which containers runs it.

What about when you want to find out which container was running a process some hours ago? Is it possible to see in atop under which cgroup was running it?

atop can only check the currently running processes. There is no way "to find out which container was running a process some hours ago" (with atop).
atop itself cannot handle containers.

"atop can only check the currently running processes."

Well, not exactly as long as you can check the history of atop depending on the interval you set in it's config. You can actively see which container is running the process on the go, the question is if you can somehow check it out from the history of it.

Perhaps, atopsar can do this.

"Containers" really have nothing to do with this essential question. What you're looking for is some kind of history of what has happened in the past ... and you also wish to be able to link a particular host process-id to a container.

This necessarily requires that some historical record exists – produced by the container manager –*which will enable you to associate the "pid" of a now-long-gone process with the container-id that was responsible for originally spawning it. I am not presently aware of any tool which does this.

Whenever a process is "running in a container," from the host operating system's perspective it is simply "an ordinary process, running on the host, but wearing rose-colored glasses." Unlike a virtual machine, there is no physical thing which corresponds to "a container." It is simply a set of parameters – system settings.