SOLVED why does my MM on pi crash every night?

sdetweil

I have 4 MM running

pi,
nano
odroid
mac

the pi crashes every night, the other three run great.

I added MMM-Watchdog, and it restarts MM everynight multiple times
i upped the timeout to 60 seconds, with 10 seconds front end update time
still restarts…

IF the UI thread has died, HOW can i figure out what is causing it?
the PM2 log doesn’t give me that info

on the pi, I display date/time, my custom water monitoring module, and background images from my local lan server… (all others are doing mostly the same, clock, pics, and one is showing water monitoring app too)
all are on 2.8.0
all have different versions of node and npm (system level differences)

I haven’t been able to get the inspect mechanism to work
https://electronjs.org/docs/tutorial/debugging-main-process

any guidance welcomed

sdetweil

watchdog has a problem… removing… have not crashed since increase swap space…

will close for now as I work on the watchdog socket problem

dazza120

@sdetweil I believe some one said the swap file was not big enough and the pi was burning through it and when it was full it would crash so I think they upped it to 4Gb

sdetweil

@dazza120 but why does it only crash at night? same system runs great all day long
upped swap to 1g, from 99meg (which was all used)

dazza120

@sdetweil I would imagine it’s full around that time mine use to crash every 12 hours without fail, I put it down to ram on the pi3B+ I got the pi 4 4Gb and it doesn’t crash anymore with the help of Watchdog too. Ram just gets eaten on the pi

sdetweil

@dazza120 thx… i added

free -m

to the mm.sh , so when pm2 restarts I can see memory usage in pm2 logs

dazza120

Link here https://forum.magicmirror.builders/topic/10940/issues-with-mm-freezing-and-rpi3-running-hot/10

sdetweil

@dazza120 thx… had seen that… and it looks like the ui thread had died… (when debugging in the past)
both the other systems have swap at 1024 (odroid has 2meg ram) and 1978 (nano has 4gig ram)

sdetweil

just for info… both the main modules background images and my water monitor app, stop processing while hidden, and my mmm-sleepwake puts the mirror to sleep (by hiding all the modules)

sdetweil

just crashed again, watchdog killed it… not memory…

so I found the only path thru my water monitoring module with code that isn’t mine, and wrapped it in an exception handler to see if it is failing…

we shall see

sdetweil

@sdetweil so far, does not crash.

If I add Watchdog, it gets restarted.
I’ve made the timeouts really long. Still fails overnight