Read the statement by Michael Teeuw here.
Strange issue
-
@evroom Thank you for your reply and i want to try all that you have written as i really feel it may help my case … but i am having a hard time decoding your steps.
Please walk me through it, on a step by step all the way to the end on how to even increase the swap file, and what to put on my scrip and make it executable and what is a cron job?? that will be my first one actually :)
Denis
-
@richland007
Since you managed to setup MM to the point that it is running, but crashing, I take it you know a little bit of Linux.
Otherwise it will be more like a Linux workshop.Do you know how to use nano and/or vi to edit files?
Know what for example cat, tail and head does?
Just some Linux basics.I do not want to sound like a smartarse and knowitall, am a hobbyist just like you.
Just to know the level of describing that is needed. -
@evroom Hahah dont’ worry about coming out the wrong way … i know a little bit and i do know what nano and vi does …i use nano.
I am not familiar with cats and tails etc but i am a fast learner and i have set up quite a bit of MM modules some stubborn ones as well but i am no expert by all means.I couldn’t read your first response at the beginning for some reason it wasn’t structured properly or my browser wasn’t reading it right now its a bit better but i still need some guidance
So i need to set up a script called “top” and make it executable right??
Than i have no idea what to do with cron :) !!!Lets start somewhere :)
Denis
-
Then let’s give it a try :slightly_smiling_face:
First create the script:
pi@MagicPi:~ $ cd pi@MagicPi:~ $ nano top.shCopy-paste this:
!/bin/sh date >> /home/pi/top.txt top -b -o +%MEM | head -n 22 >> /home/pi/top.txt top -b -o +%MEM | head -n 22 >> /home/pi/top.txt echo "=============================================" >> /home/pi/top.txtMake it executable:
pi@MagicPi:~ $ chmod 755 top.shPerhaps a good idea to prepare the output file in order to be sure the cron user can write in it:
pi@MagicPi:~ $ touch /home/pi/top.txt pi@MagicPi:~ $ chmod 666 /home/pi/top.txtNow we add a cronjob:
pi@MagicPi:~ $ crontab -eAdd this line at the end:
00,05,10,15,20,25,30,35,40,45,50,55 * * * * /home/pi/top.sh >/dev/null 2>&1To verify:
pi@MagicPi:~ $ crontab -lTo verify if the cronjob is actually running:
pi@MagicPi:~ $ tail -F /var/log/syslogTo check if something comes in your output file:
pi@MagicPi:~ $ tail -F top.txtThe script will run every 5 minutes, every hour, every day.
It will output the top 15 MEMory usage and the top 15 CPU usage.In your case, it will be interesting to look at the last entries before the crash.
Need to identify first if it is an CPU or MEM issue, or both.
Then we can modify the script.Please do not post lengthy output.
Better we minimise the output by grepping and sorting, etc.Success!
-
@evroom Ahhh you are a great man
I will set it up as soon as i get home
thanks in advance
Denis -
Ha ha, not that great, my MM is as slow as a turtle again.
The used swap on my USB stick in the meanwhile is 1GB.
Cannot use any commands to investigate further.
Not very satisfactory. :frowning_face: -
@evroom Mine is not slow it works “fine” it just keeps restarting
I am really curious to see the output of that file and see whats causing this behavior
D -
If previous suggestions dont help, try removing the MMM-WatchDog module and see if it crashes anymore.
-
@richland007
I re-read your first post and your problem is more restarting then actual crashing.
Of course a restart can be caused by a crash …
MMM-WatchDog tends to restart MM, so I too would suggest to remove this module.
In principle all it does is sending a ping and restarting MM using pm2 when a timeout occurs.
I also would expect to see some lines in the pm2 logs on WatchDog.
Can you do ?:pi@MagicPi:~ $ grep -i watchdog /home/pi/.pm2/logs/mm-out.log pi@MagicPi:~ $ grep -i watchdog /home/pi/.pm2/logs/mm-error.logWhat is alo a good package to install is sysstat:
pi@MagicPi:~ $ sudo apt-get install sysstatThis allows for commands like iostat and sar.
vmstat 10 10 iostat 10 10 sar 10 10 -
@evroom Ok so here is whats writen on that top.txt file
pi@SmartMirror:~ $ tail -F top.txt 834 pi 20 0 155396 17884 8104 S 0.0 1.9 23:20.77 lxpanel 582 root 20 0 211836 13576 2440 S 0.0 1.4 418:48.14 Xorg 457 alexapi 20 0 218108 10768 4772 S 47.4 1.1 878:20.67 python 721 pi 20 0 122032 10600 4792 S 0.0 1.1 8:34.40 PM2 v3.0.3+ 835 pi 20 0 154056 5184 4016 S 0.0 0.5 0:43.57 pcmanfm 18753 pi 20 0 47692 4896 3996 S 0.0 0.5 0:03.94 lxterminal 829 pi 20 0 53536 4432 3556 S 0.0 0.5 0:13.22 openbox 100 root 20 0 35220 3140 2980 S 0.0 0.3 2:16.35 systemd-jo+ 7236 pi 20 0 8112 3140 2712 R 15.8 0.3 0:00.07 top ============================================= Sat Nov 17 11:25:01 CST 2018 top - 11:25:01 up 3 days, 15:28, 1 user, load average: 0.56, 2.88, 4.35 Tasks: 135 total, 1 running, 89 sleeping, 0 stopped, 1 zombie %Cpu(s): 21.4 us, 5.5 sy, 0.0 ni, 69.4 id, 2.3 wa, 0.0 hi, 1.5 si, 0.0 st KiB Mem : 949452 total, 493928 free, 157456 used, 298068 buff/cache KiB Swap: 949440 total, 777124 free, 172316 used. 727276 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 582 root 20 0 218840 24540 12364 S 10.5 2.6 419:19.31 Xorg 502 root 20 0 45188 23400 14452 S 36.8 2.5 27:20.07 vncserver-+ 834 pi 20 0 155396 21680 10616 S 0.0 2.3 23:23.35 lxpanel 7558 pi 20 0 77052 21140 18404 S 0.0 2.2 0:03.37 leafpad 835 pi 20 0 154156 16420 13232 S 0.0 1.7 0:46.57 pcmanfm 721 pi 20 0 122032 14004 7360 S 0.0 1.5 8:34.81 PM2 v3.0.3+ 457 alexapi 20 0 218108 10772 4772 S 36.8 1.1 880:07.93 python 636 root 20 0 15416 9020 8820 S 0.0 1.0 3:55.73 vncagent 18753 pi 20 0 47692 8484 7164 S 0.0 0.9 0:04.22 lxterminal 829 pi 20 0 53724 5888 4220 S 0.0 0.6 0:13.49 openbox 953 pi 20 0 27236 5312 4812 S 0.0 0.6 0:03.05 vncserverui 873 pi 20 0 27816 4328 3676 S 0.0 0.5 0:40.96 vncserverui 804 pi 20 0 39468 3288 3000 S 0.0 0.3 0:02.79 gvfsd 100 root 20 0 35220 3196 3036 S 0.0 0.3 2:16.50 systemd-jo+ 7603 pi 20 0 8112 3180 2752 R 10.5 0.3 0:00.05 top top - 11:25:01 up 3 days, 15:28, 1 user, load average: 0.56, 2.88, 4.35 Tasks: 135 total, 1 running, 89 sleeping, 0 stopped, 1 zombie %Cpu(s): 21.4 us, 5.5 sy, 0.0 ni, 69.4 id, 2.3 wa, 0.0 hi, 1.5 si, 0.0 st KiB Mem : 949452 total, 493776 free, 157600 used, 298076 buff/cache KiB Swap: 949440 total, 777124 free, 172316 used. 727128 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 582 root 20 0 218840 24540 12364 S 11.1 2.6 419:19.33 Xorg 502 root 20 0 45188 23400 14452 S 50.0 2.5 27:20.16 vncserver-+ 834 pi 20 0 155396 21680 10616 S 0.0 2.3 23:23.36 lxpanel 7558 pi 20 0 77052 21140 18404 S 0.0 2.2 0:03.37 leafpad 835 pi 20 0 154156 16420 13232 S 0.0 1.7 0:46.57 pcmanfm 721 pi 20 0 122032 14004 7360 S 0.0 1.5 8:34.81 PM2 v3.0.3+ 457 alexapi 20 0 218108 10772 4772 S 61.1 1.1 880:08.05 python 636 root 20 0 15416 9020 8820 S 0.0 1.0 3:55.73 vncagent 18753 pi 20 0 47692 8484 7164 S 0.0 0.9 0:04.24 lxterminal 829 pi 20 0 53724 5888 4220 S 0.0 0.6 0:13.49 openbox 953 pi 20 0 27236 5312 4812 S 0.0 0.6 0:03.05 vncserverui 873 pi 20 0 27816 4328 3676 S 0.0 0.5 0:40.96 vncserverui 804 pi 20 0 39468 3288 3000 S 0.0 0.3 0:02.79 gvfsd 100 root 20 0 35220 3196 3036 S 0.0 0.3 2:16.50 systemd-jo+ 7608 pi 20 0 8112 3180 2752 R 22.2 0.3 0:00.07 top =============================================I use the command
tail -F top.txtright?? i dont know how to read the above file so let me know if you see anything strange …i have increased the swap file to install opencv a while back for facial recognition to install rapidly but i do not use a usb …i could if i have to.
As far as taking out the watchdog…commenting it out of the config.js should do the trick or do i have to uninstall it form the modules??
Will pm2 still restart the MM if it crashes without the watchdog (it does if you do ctrl+q) ??
D
