Strange issue

evroom · Nov 16, 2018, 8:21 PM

Ha ha, not that great, my MM is as slow as a turtle again.
The used swap on my USB stick in the meanwhile is 1GB.
Cannot use any commands to investigate further.
Not very satisfactory. :frowning_face:

richland007 · Nov 16, 2018, 8:25 PM

@evroom Mine is not slow it works “fine” it just keeps restarting
I am really curious to see the output of that file and see whats causing this behavior
D

vinp · Nov 17, 2018, 3:56 AM

If previous suggestions dont help, try removing the MMM-WatchDog module and see if it crashes anymore.

evroom · Nov 17, 2018, 5:34 PM

@richland007
I re-read your first post and your problem is more restarting then actual crashing.
Of course a restart can be caused by a crash …
MMM-WatchDog tends to restart MM, so I too would suggest to remove this module.
In principle all it does is sending a ping and restarting MM using pm2 when a timeout occurs.
I also would expect to see some lines in the pm2 logs on WatchDog.
Can you do ?:

pi@MagicPi:~ $ grep -i watchdog /home/pi/.pm2/logs/mm-out.log
pi@MagicPi:~ $ grep -i watchdog /home/pi/.pm2/logs/mm-error.log

What is alo a good package to install is sysstat:

pi@MagicPi:~ $ sudo apt-get install sysstat

This allows for commands like iostat and sar.

vmstat 10 10
iostat 10 10
sar 10 10

richland007 · Nov 17, 2018, 5:34 PM

@evroom Ok so here is whats writen on that top.txt file

pi@SmartMirror:~ $ tail -F top.txt
  834 pi        20   0  155396  17884   8104 S   0.0  1.9  23:20.77 lxpanel
  582 root      20   0  211836  13576   2440 S   0.0  1.4 418:48.14 Xorg
  457 alexapi   20   0  218108  10768   4772 S  47.4  1.1 878:20.67 python
  721 pi        20   0  122032  10600   4792 S   0.0  1.1   8:34.40 PM2 v3.0.3+
  835 pi        20   0  154056   5184   4016 S   0.0  0.5   0:43.57 pcmanfm
18753 pi        20   0   47692   4896   3996 S   0.0  0.5   0:03.94 lxterminal
  829 pi        20   0   53536   4432   3556 S   0.0  0.5   0:13.22 openbox
  100 root      20   0   35220   3140   2980 S   0.0  0.3   2:16.35 systemd-jo+
 7236 pi        20   0    8112   3140   2712 R  15.8  0.3   0:00.07 top
=============================================
Sat Nov 17 11:25:01 CST 2018
top - 11:25:01 up 3 days, 15:28,  1 user,  load average: 0.56, 2.88, 4.35
Tasks: 135 total,   1 running,  89 sleeping,   0 stopped,   1 zombie
%Cpu(s): 21.4 us,  5.5 sy,  0.0 ni, 69.4 id,  2.3 wa,  0.0 hi,  1.5 si,  0.0 st
KiB Mem :   949452 total,   493928 free,   157456 used,   298068 buff/cache
KiB Swap:   949440 total,   777124 free,   172316 used.   727276 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  582 root      20   0  218840  24540  12364 S  10.5  2.6 419:19.31 Xorg
  502 root      20   0   45188  23400  14452 S  36.8  2.5  27:20.07 vncserver-+
  834 pi        20   0  155396  21680  10616 S   0.0  2.3  23:23.35 lxpanel
 7558 pi        20   0   77052  21140  18404 S   0.0  2.2   0:03.37 leafpad
  835 pi        20   0  154156  16420  13232 S   0.0  1.7   0:46.57 pcmanfm
  721 pi        20   0  122032  14004   7360 S   0.0  1.5   8:34.81 PM2 v3.0.3+
  457 alexapi   20   0  218108  10772   4772 S  36.8  1.1 880:07.93 python
  636 root      20   0   15416   9020   8820 S   0.0  1.0   3:55.73 vncagent
18753 pi        20   0   47692   8484   7164 S   0.0  0.9   0:04.22 lxterminal
  829 pi        20   0   53724   5888   4220 S   0.0  0.6   0:13.49 openbox
  953 pi        20   0   27236   5312   4812 S   0.0  0.6   0:03.05 vncserverui
  873 pi        20   0   27816   4328   3676 S   0.0  0.5   0:40.96 vncserverui
  804 pi        20   0   39468   3288   3000 S   0.0  0.3   0:02.79 gvfsd
  100 root      20   0   35220   3196   3036 S   0.0  0.3   2:16.50 systemd-jo+
 7603 pi        20   0    8112   3180   2752 R  10.5  0.3   0:00.05 top
top - 11:25:01 up 3 days, 15:28,  1 user,  load average: 0.56, 2.88, 4.35
Tasks: 135 total,   1 running,  89 sleeping,   0 stopped,   1 zombie
%Cpu(s): 21.4 us,  5.5 sy,  0.0 ni, 69.4 id,  2.3 wa,  0.0 hi,  1.5 si,  0.0 st
KiB Mem :   949452 total,   493776 free,   157600 used,   298076 buff/cache
KiB Swap:   949440 total,   777124 free,   172316 used.   727128 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
  582 root      20   0  218840  24540  12364 S  11.1  2.6 419:19.33 Xorg
  502 root      20   0   45188  23400  14452 S  50.0  2.5  27:20.16 vncserver-+
  834 pi        20   0  155396  21680  10616 S   0.0  2.3  23:23.36 lxpanel
 7558 pi        20   0   77052  21140  18404 S   0.0  2.2   0:03.37 leafpad
  835 pi        20   0  154156  16420  13232 S   0.0  1.7   0:46.57 pcmanfm
  721 pi        20   0  122032  14004   7360 S   0.0  1.5   8:34.81 PM2 v3.0.3+
  457 alexapi   20   0  218108  10772   4772 S  61.1  1.1 880:08.05 python
  636 root      20   0   15416   9020   8820 S   0.0  1.0   3:55.73 vncagent
18753 pi        20   0   47692   8484   7164 S   0.0  0.9   0:04.24 lxterminal
  829 pi        20   0   53724   5888   4220 S   0.0  0.6   0:13.49 openbox
  953 pi        20   0   27236   5312   4812 S   0.0  0.6   0:03.05 vncserverui
  873 pi        20   0   27816   4328   3676 S   0.0  0.5   0:40.96 vncserverui
  804 pi        20   0   39468   3288   3000 S   0.0  0.3   0:02.79 gvfsd
  100 root      20   0   35220   3196   3036 S   0.0  0.3   2:16.50 systemd-jo+
 7608 pi        20   0    8112   3180   2752 R  22.2  0.3   0:00.07 top
=============================================

I use the command

tail -F top.txt

right?? i dont know how to read the above file so let me know if you see anything strange …i have increased the swap file to install opencv a while back for facial recognition to install rapidly but i do not use a usb …i could if i have to.

As far as taking out the watchdog…commenting it out of the config.js should do the trick or do i have to uninstall it form the modules??

Will pm2 still restart the MM if it crashes without the watchdog (it does if you do ctrl+q) ??

D

evroom · Nov 17, 2018, 7:38 PM

Okay, this calls for some more basics.

Print the whole file:
$ cat top.txt

Load the file in an editor:
$ nano top.txt
$ vi top.txt

I am old-school, so I use vi, but nano is more Word-like.

To show the last 50 lines:
$ tail -50 top.txt

To show the first 50 lines:
$ head -50 top.txt

Where of course 50 is just an example.

To show text that is appending:
$ tail -f top.txt

To show text that is appending and re-open the file when necessary (useful for rotating log files):
$ tail -F top.txt

On disabling a module see the next reply.

evroom · Nov 17, 2018, 7:45 PM

To disable MMM-WatchDog, edit the config.js file.
Locate:

module: 'MMM-WatchDog',

Edit this like this:

module: 'MMM-WatchDog',
disabled: true,

and restart MM:

$ pm2 restart mm

Use

pm2 list

to check if your application name actually is mm, or something else.

Later you can enable it again by updating config.js like this:

disabled: false,

This works for all modules.
When I test modules I normally leave the config lines and disable a module in this fashion.

evroom · Nov 17, 2018, 8:07 PM

@richland007
On pm2 and restarting I cannot say many.
On using ctrl+q neither, as I only access my MM via ssh.

Can you send the output of the following commands:

$ uname -a
$ swapon -s
$ free -h
$ cat /etc/dphys-swapfile | egrep -v ‘#|^$’
$ sudo service dphys-swapfile status

I just learned that swap is handled as a service.
Being old-school, this is something new to me :-)

evroom · Nov 17, 2018, 8:14 PM

@richland007
Concerning your top.txt, was that the output on the moment that MM was restarted ?
I do not see obvious memory issues.
Only that vncserver and python are CPU hungry, but within limits.
And I do not see any electron processes.

cowboysdude · Nov 18, 2018, 2:18 AM

@oceank The BIGGEST problem that nobody either wants to admit or doesn’t realize is that the pi wasn’t made to be pounded on this hard.

It’s a simple computer…when you start adding constant pulls for data and scrolling text, etc it over heats and overloads the pi.