Read the statement by Michael Teeuw here.
auto recovery from mirror freeze
-
I’ve got the magicmirror software built as a container and deployed to a kubernetes (k3s) cluster. For the front end I’ve got a Raspberry Pi 4 with 4G RAM running Bullseye and Chromium automatically on boot and this same pi is also one of the nodes in the k3s cluster so it’s often the case that the magic mirror server is running on the same machine as the browser showing it.
Occasionally the mirror freezes and stops updating but even while this is the case I can still connect to the mm server from another computer so the server is fine, the issue seems to be client side. Before I switched to the k3s deployment if the mirror froze I’d tell family members it was ok to just unplug/replug the pi but now that other things could be affected I’d rather not do that.
I’m not sure if the Chromium browser is actually crashing or if it is possible that once a periodic update fails for whatever reason that mm doesn’t get updates after that.
So has anyone implemented some kind of watchdog that will restart the browser in such a situation or some kind of plugin for mm that forces a major refresh every once in a while just in case things froze?
-
@sprior there is the MMM-WatchDog module that does that
much of the modules are Javascript code in the front end. if some of it dies, it kills the browser.
-
@sdetweil One thing I’d love to do is to run the browser from a container pinned to that specific node so that if it did die it could be killed/restarted using normal kubernetes container management. But running a GUI program from a container seems to be underexplored territory.
The other approach I can think of would require being able to remotely kill/restart the browser on the console from an ssh connection. Currently I use a ~/.config/wayfire.ini file (created via an Ansible role) to launch chromium on boot, but that doesn’t make it any easier to relaunch after a browser crash.
-
@sprior using the node process manager, aka pm2, you can programmatically restart any managed app. chromium, etc are just apps.
-
@sprior containers by definition don’t have access to the host ui.
-
@sdetweil Not normally no, but for some things you can grant a hole in the container to allow direct access to some things. For example I run Jenkins in a Docker container but grant access to the docker socket so that Jenkins can actually launch build steps in containers. I think that it is possible to run GPU jobs in containers. But I agree with you that none of this is the norm.
I’ll take a look at pm2.
-
maybe the freezing of the client is caused by a restart of the server, e.g. if the server changes the node.
In mm v2.25.0 we implemented an automatic browser reload if the server was restarted, so maybe this helps here (see https://github.com/MagicMirrorOrg/MagicMirror/pull/3188/files)
-
@sprior said in auto recovery from mirror freeze:
Not normally no, but for some things you can grant a hole in the container to allow direct access to some things
that is exactly the same…
the app (jenkins, chrome) running on the docker host can access the service in the container…
it is NOT the other way…
the thing in the container can invoke things on the docker host
anyhow… pm2’s JOB is to launch and keep running apps.
you can use it for anything.
I have a system where I launch the spotify service (raspotify), the camera motion detector (motion) and magicMirror on two different screens…
I can kill any of them and pm2 will relaunch whatever has ended…
SO, the thing being launched cannot ‘end’…
