Read the statement by Michael Teeuw here.
rpi3 freezes every few days .. need help
-
First I thought it was one of the modules that load huge picture or video from google. But it happens as well with a very simple config.
I changed the power supply to 2.5A. But it still happens.
Any help ? What should I look for in the logs ?
Thanks
-
Run pm2 list to get your instance name
pm2 list
Run pm2 logs instanceNameHere to get your error logs.
pm2 logs mm
Let’s see if there are any out of memory errors or a module throwing an error.
-
Here are the output:
pi@MM:~ $ pm2 logs mm
[TAILING] Tailing last 15 lines for [mm] process (change the value with --lines option)
/home/pi/.pm2/logs/mm-error.log last 15 lines:
0|mm | npm WARN npm npm does not support Node.js v10.15.2
0|mm | npm WARN npm You should probably upgrade to a newer version of node as we
0|mm | npm WARN npm can’t make any promises that npm will work with this version.
0|mm | npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
0|mm | npm WARN npm You can find the latest version at https://nodejs.org/
0|mm | npm WARN npm npm does not support Node.js v10.15.2
0|mm | npm WARN npm You should probably upgrade to a newer version of node as we
0|mm | npm WARN npm can’t make any promises that npm will work with this version.
0|mm | npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
0|mm | npm WARN npm You can find the latest version at https://nodejs.org/
0|mm | npm WARN npm npm does not support Node.js v10.15.2
0|mm | npm WARN npm You should probably upgrade to a newer version of node as we
0|mm | npm WARN npm can’t make any promises that npm will work with this version.
0|mm | npm WARN npm Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9.
0|mm | npm WARN npm You can find the latest version at https://nodejs.org//home/pi/.pm2/logs/mm-out.log last 15 lines:
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | There is no scanned photo currently.
0|mm | [GPHOTO] Scan finished : 3972
0|mm | [GPHOTO] Scan finished : 3972
0|mm | [GPHOTO] Scan finished : 3972
0|mm | [GPHOTO] Scan finished : 3972
0|mm | [GPHOTO] Scan finished : 3972^C
pi@MM:~ $Is the npm warning expected ?
-
It frozen again at 10:49:57. I grep that time and found this.
Kernel issue ?root@MM:/var/log# cat error.l Nov 5 10:49:34 MM kernel: [ 6438.728602] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:34 MM kernel: [ 6438.728616] ------------[ cut here ]------------ Nov 5 10:49:34 MM kernel: [ 6438.728649] WARNING: CPU: 3 PID: 0 at mm/slab.h:380 kmem_cache_free+0x1ec/0x238 Nov 5 10:49:34 MM kernel: [ 6438.728657] Modules linked in: fuse 8021q garp stp llc sg uas brcmfmac brcmutil sha256_generic cfg80211 raspberrypi_hwmon hwmon rfkill bcm2835_codec(C) bcm2835_v4l2(C) v4l2_mem2mem snd_bcm2835(C) bcm2835_mmal_vchiq(C) v4l2_common videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops snd_pcm snd_timer snd videobuf2_v4l2 videobuf2_common videodev media vc_sm_cma(C) uio_pdrv_genirq uio fixed i2c_dev ip_tables x_tables Nov 5 10:49:34 MM kernel: [ 6438.728872] CPU: 3 PID: 0 Comm: swapper/3 Tainted: G B C 4.19.75-v7+ #1270 Nov 5 10:49:34 MM kernel: [ 6438.728877] Hardware name: BCM2835 Nov 5 10:49:34 MM kernel: [ 6438.728906] [<80111fcc>] (unwind_backtrace) from [<8010d544>] (show_stack+0x20/0x24) Nov 5 10:49:34 MM kernel: [ 6438.728926] [<8010d544>] (show_stack) from [<80819bc0>] (dump_stack+0xd4/0x118) Nov 5 10:49:34 MM kernel: [ 6438.728945] [<80819bc0>] (dump_stack) from [<80120a88>] (__warn+0x104/0x11c) Nov 5 10:49:34 MM kernel: [ 6438.728964] [<80120a88>] (__warn) from [<80120bd8>] (warn_slowpath_null+0x50/0x58) Nov 5 10:49:34 MM kernel: [ 6438.728980] [<80120bd8>] (warn_slowpath_null) from [<802960d0>] (kmem_cache_free+0x1ec/0x238) Nov 5 10:49:34 MM kernel: [ 6438.729000] [<802960d0>] (kmem_cache_free) from [<802585dc>] (shmem_destroy_callback+0x3c/0x4c) Nov 5 10:49:34 MM kernel: [ 6438.729018] [<802585dc>] (shmem_destroy_callback) from [<8018e288>] (rcu_process_callbacks+0x2cc/0x674) Nov 5 10:49:34 MM kernel: [ 6438.729038] [<8018e288>] (rcu_process_callbacks) from [<80102410>] (__do_softirq+0x190/0x3f0) Nov 5 10:49:34 MM kernel: [ 6438.729057] [<80102410>] (__do_softirq) from [<80126a60>] (irq_exit+0xfc/0x120) Nov 5 10:49:34 MM kernel: [ 6438.729077] [<80126a60>] (irq_exit) from [<8017f22c>] (__handle_domain_irq+0x70/0xc4) Nov 5 10:49:34 MM kernel: [ 6438.729096] [<8017f22c>] (__handle_domain_irq) from [<801021b4>] (bcm2836_arm_irqchip_handle_irq+0x60/0xa4) Nov 5 10:49:34 MM kernel: [ 6438.729112] [<801021b4>] (bcm2836_arm_irqchip_handle_irq) from [<801019bc>] (__irq_svc+0x5c/0x7c) Nov 5 10:49:34 MM kernel: [ 6438.729119] Exception stack(0xb9d51f38 to 0xb9d51f80) Nov 5 10:49:34 MM kernel: [ 6438.729128] 1f20: 80109a84 00000000 Nov 5 10:49:34 MM kernel: [ 6438.729142] 1f40: 40000093 40000093 80d04d70 b9d50000 80d04db8 00000008 80d8ed3e 410fd034 Nov 5 10:49:34 MM kernel: [ 6438.729155] 1f60: 00000000 b9d51f94 80d0517c b9d51f88 00000000 80109a88 40000013 ffffffff Nov 5 10:49:34 MM kernel: [ 6438.729173] [<801019bc>] (__irq_svc) from [<80109a88>] (arch_cpu_idle+0x34/0x4c) Nov 5 10:49:34 MM kernel: [ 6438.729193] [<80109a88>] (arch_cpu_idle) from [<80836c1c>] (default_idle_call+0x34/0x48) Nov 5 10:49:34 MM kernel: [ 6438.729211] [<80836c1c>] (default_idle_call) from [<801526d8>] (do_idle+0xec/0x17c) Nov 5 10:49:34 MM kernel: [ 6438.729232] [<801526d8>] (do_idle) from [<80152a28>] (cpu_startup_entry+0x28/0x2c) Nov 5 10:49:34 MM kernel: [ 6438.729249] [<80152a28>] (cpu_startup_entry) from [<8010fde8>] (secondary_start_kernel+0x134/0x140) Nov 5 10:49:34 MM kernel: [ 6438.729265] [<8010fde8>] (secondary_start_kernel) from [<0010270c>] (0x10270c) Nov 5 10:49:34 MM kernel: [ 6438.729276] ---[ end trace 44e2f56c32b8bf1c ]--- Nov 5 10:49:54 MM kernel: [ 6458.548472] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:54 MM kernel: [ 6458.548493] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:54 MM kernel: [ 6458.548505] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:54 MM kernel: [ 6458.548517] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:54 MM kernel: [ 6458.548529] cache_from_obj: Wrong slab cache. shmem_inode_cache but object is from `����� Nov 5 10:49:57 MM kernel: [ 6460.970198] Unable to handle kernel paging request at virtual address 000064d4 Nov 5 10:49:57 MM kernel: [ 6460.977431] pgd = 31d1d4f4 Nov 5 10:49:57 MM kernel: [ 6460.980130] [000064d4] *pgd=208fc835, *pte=00000000, *ppte=00000000 Nov 5 10:49:57 MM kernel: [ 6460.986403] Internal error: Oops: 17 [#1] SMP ARM Nov 5 10:49:57 MM kernel: [ 6460.991101] Modules linked in: fuse 8021q garp stp llc sg uas brcmfmac brcmutil sha256_generic cfg80211 raspberrypi_hwmon hwmon rfkill bcm2835_codec(C) bcm2835_v4l2(C) v4l2_mem2mem snd_bcm2835(C) bcm2835_mmal_vchiq(C) v4l2_common videobuf2_vmalloc videobuf2_dma_contig videobuf2_memops snd_pcm snd_timer snd videobuf2_v4l2 videobuf2_common videodev media vc_sm_cma(C) uio_pdrv_genirq uio fixed i2c_dev ip_tables x_tables Nov 5 10:49:57 MM kernel: [ 6461.028111] CPU: 0 PID: 1255 Comm: Chrome_IOThread Tainted: G B WC 4.19.75-v7+ #1270 Nov 5 10:49:57 MM kernel: [ 6461.036887] Hardware name: BCM2835 Nov 5 10:49:57 MM kernel: [ 6461.040292] PC is at ___slab_alloc.constprop.11+0x344/0x594 Nov 5 10:49:57 MM kernel: [ 6461.045859] LR is at ___slab_alloc.constprop.11+0x338/0x594 Nov 5 10:49:57 MM kernel: [ 6461.051424] pc : [<80294b14>] lr : [<80294b08>] psr: 60000093 Nov 5 10:49:57 MM kernel: [ 6461.057682] sp : a0a3dc40 ip : a0a3dc40 fp : a0a3dcb4 Nov 5 10:49:57 MM kernel: [ 6461.062898] r10: 00000000 r9 : b69cc000 r8 : b37cf508 Nov 5 10:49:57 MM kernel: [ 6461.068115] r7 : b37cf500 r6 : 00000000 r5 : 000064d0 r4 : ba67fffc Nov 5 10:49:57 MM kernel: [ 6461.074634] r3 : b6a798c0 r2 : 000064d4 r1 : 00000000 r0 : 00000000 Nov 5 10:49:57 MM kernel: [ 6461.081154] Flags: nZCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user Nov 5 10:49:57 MM kernel: [ 6461.088367] Control: 10c5383d Table: 30f0406a DAC: 00000055 Nov 5 10:49:57 MM kernel: [ 6461.094108] Process Chrome_IOThread (pid: 1255, stack limit = 0xdc362e5a) Nov 5 10:49:57 MM kernel: [ 6461.100888] Stack: (0xa0a3dc40 to 0xa0a3e000) Nov 5 10:49:57 MM kernel: [ 6461.105240] dc40: 00000000 80d8ee82 80d04d48 ff778840 80258618 006000c0 a0b364f0 a0a3de3c Nov 5 10:49:57 MM kernel: [ 6461.113412] dc60: 0000001c 00000000 0000001c 80294d94 ff778840 00120012 006000c0 b69cc000 Nov 5 10:49:57 MM kernel: [ 6461.121583] dc80: b9cc8e40 f272aca4 a0a3dcb4 60000013 ff778840 80258618 006000c0 b69cc000 Nov 5 10:49:57 MM kernel: [ 6461.129755] dca0: b9cc8e40 000014ab a0a3dcdc a0a3dcb8 80294da8 802947dc ff778840 00000000 Nov 5 10:49:57 MM kernel: [ 6461.137927] dcc0: 006000c0 b69cc000 00000000 80258618 a0a3dd1c a0a3dce0 80295574 80294d70 Nov 5 10:49:57 MM kernel: [ 6461.146099] dce0: 00000003 80d0d640 a0a3dd14 a0a3dcf8 80201d10 b90bc1e0 b9731800 00000000 Nov 5 10:49:57 MM kernel: [ 6461.154271] dd00: 00000000 b9731800 b90bc1e0 00000000 a0a3dd2c a0a3dd20 80258618 80295374 Nov 5 10:49:57 MM kernel: [ 6461.162442] dd20: a0a3dd44 a0a3dd30 802c7ff4 802585f8 b90bc1e0 00008180 a0a3dd5c a0a3dd48 Nov 5 10:49:57 MM kernel: [ 6461.170613] dd40: 802ca594 802c7fd4 00000000 b90bc1e0 a0a3dd74 a0a3dd60 802ca5f4 802ca588 Nov 5 10:49:57 MM kernel: [ 6461.178786] dd60: 00000000 b90bc1e0 a0a3ddac a0a3dd78 80258c04 802ca5e4 a0a3dd9c a0a3dd88 Nov 5 10:49:57 MM kernel: [ 6461.186957] dd80: 80168384 80168264 b90bc1e0 b8c6b3b8 00000000 98e16000 94838e40 a0a3de90 Nov 5 10:49:57 MM kernel: [ 6461.195129] dda0: a0a3ddec a0a3ddb0 80258f54 80258bd0 00200000 00009a80 a0a3dde4 a0a3ddc8 Nov 5 10:49:57 MM kernel: [ 6461.203300] ddc0: 802c4e70 802c29fc 98e16000 000200c2 b8c6b3b8 00000000 a0a3df50 94838e40 Nov 5 10:49:57 MM kernel: [ 6461.211472] dde0: a0a3ddfc a0a3ddf0 802592b0 80258f2c a0a3de8c a0a3de00 802bb440 8025929c Nov 5 10:49:57 MM kernel: [ 6461.219643] de00: 00000000 f272aca4 a0a3df44 98e16000 00000000 00008180 b90bc1e0 a0a3c000 Nov 5 10:49:57 MM kernel: [ 6461.227815] de20: 98e16000 000200c2 80d04d48 00000041 80d05688 00000040 00000001 00000006 Nov 5 10:49:57 MM kernel: [ 6461.235987] de40: 00001000 80d04d48 a9347c94 00000000 00010001 a0a3de54 a0a3de54 f272aca4 Nov 5 10:49:57 MM kernel: [ 6461.244159] de60: a9347c94 80d04d48 80d04d48 a0a3df50 a0a3de90 00000001 a0a3c000 00000142 Nov 5 10:49:57 MM kernel: [ 6461.252330] de80: a0a3df44 a0a3de90 802bcb80 802ba87c b96ef710 b8c6b3b8 3d1ed1df 0000001d Nov 5 10:49:57 MM kernel: [ 6461.260502] dea0: b9fc6019 006000c0 00000000 b9a3a990 b90bc1e0 00000701 00000002 00000360 Nov 5 10:49:57 MM kernel: [ 6461.268673] dec0: 00000000 00000000 00000000 a0a3ded0 00000001 f272aca4 a0a3df40 0000005e Nov 5 10:49:57 MM kernel: [ 6461.276845] dee0: b6a28e00 b0dda200 00000000 00002000 000200c2 00000100 a0a3df34 a0a3df08 Nov 5 10:49:57 MM kernel: [ 6461.285017] df00: b9fc6000 00000000 000200c2 00000002 ffffff9c ffffff9c b9fc6000 f272aca4 Nov 5 10:49:57 MM kernel: [ 6461.293188] df20: a0a3c000 0000005e 80d04d48 ffffff9c b9fc6000 801011c4 a0a3df94 a0a3df48 Nov 5 10:49:57 MM kernel: [ 6461.301360] df40: 802a8500 802bcb0c 00000000 00000000 000200c2 00008180 00000006 00000700 Nov 5 10:49:57 MM kernel: [ 6461.309531] df60: 00000001 f272aca4 0000020c 8fea2092 76f76968 00000000 00000142 801011c4 Nov 5 10:49:57 MM kernel: [ 6461.317703] df80: a0a3c000 00000142 a0a3dfa4 a0a3df98 802a85fc 802a83cc 00000000 a0a3dfa8 Nov 5 10:49:57 MM kernel: [ 6461.325874] dfa0: 80101000 802a85ec 8fea2092 76f76968 ffffff9c 069dc420 000200c2 00000180 Nov 5 10:49:57 MM kernel: [ 6461.334047] dfc0: 8fea2092 76f76968 00000000 00000142 00000000 0003a2f8 069dc420 00000000 Nov 5 10:49:57 MM kernel: [ 6461.342220] dfe0: 00000002 643b33a0 00000000 7491035c 80000010 ffffff9c 00000000 00000000 Nov 5 10:49:57 MM kernel: [ 6461.350405] [<80294b14>] (___slab_alloc.constprop.11) from [<80294da8>] (__slab_alloc.constprop.10+0x44/0x90) Nov 5 10:49:57 MM kernel: [ 6461.360318] [<80294da8>] (__slab_alloc.constprop.10) from [<80295574>] (kmem_cache_alloc+0x20c/0x23c) Nov 5 10:49:57 MM kernel: [ 6461.369536] [<80295574>] (kmem_cache_alloc) from [<80258618>] (shmem_alloc_inode+0x2c/0x38) Nov 5 10:49:57 MM kernel: [ 6461.377885] [<80258618>] (shmem_alloc_inode) from [<802c7ff4>] (alloc_inode+0x2c/0xb4) Nov 5 10:49:57 MM kernel: [ 6461.385801] [<802c7ff4>] (alloc_inode) from [<802ca594>] (new_inode_pseudo+0x18/0x5c) Nov 5 10:49:57 MM kernel: [ 6461.393627] [<802ca594>] (new_inode_pseudo) from [<802ca5f4>] (new_inode+0x1c/0x30) Nov 5 10:49:57 MM kernel: [ 6461.401278] [<802ca5f4>] (new_inode) from [<80258c04>] (shmem_get_inode+0x40/0x1b8) Nov 5 10:49:57 MM kernel: [ 6461.408929] [<80258c04>] (shmem_get_inode) from [<80258f54>] (shmem_mknod+0x34/0xc8) Nov 5 10:49:57 MM kernel: [ 6461.416667] [<80258f54>] (shmem_mknod) from [<802592b0>] (shmem_create+0x20/0x24) Nov 5 10:49:57 MM kernel: [ 6461.424146] [<802592b0>] (shmem_create) from [<802bb440>] (path_openat+0xbd0/0x1024) Nov 5 10:49:57 MM kernel: [ 6461.431886] [<802bb440>] (path_openat) from [<802bcb80>] (do_filp_open+0x80/0xec) Nov 5 10:49:57 MM kernel: [ 6461.439368] [<802bcb80>] (do_filp_open) from [<802a8500>] (do_sys_open+0x140/0x1f0) Nov 5 10:49:57 MM kernel: [ 6461.447022] [<802a8500>] (do_sys_open) from [<802a85fc>] (sys_openat+0x1c/0x20) Nov 5 10:49:57 MM kernel: [ 6461.454330] [<802a85fc>] (sys_openat) from [<80101000>] (ret_fast_syscall+0x0/0x28) Nov 5 10:49:57 MM kernel: [ 6461.461978] Exception stack(0xa0a3dfa8 to 0xa0a3dff0) Nov 5 10:49:57 MM kernel: [ 6461.467024] dfa0: 8fea2092 76f76968 ffffff9c 069dc420 000200c2 00000180 Nov 5 10:49:57 MM kernel: [ 6461.475196] dfc0: 8fea2092 76f76968 00000000 00000142 00000000 0003a2f8 069dc420 00000000 Nov 5 10:49:57 MM kernel: [ 6461.483368] dfe0: 00000002 643b33a0 00000000 7491035c Nov 5 10:49:57 MM kernel: [ 6461.488416] Code: ebfeb53f e3500000 1affffb3 e2852004 (e5953004) Nov 5 10:49:57 MM kernel: [ 6461.494507] ---[ end trace 44e2f56c32b8bf1d ]--- Nov 5 10:55:01 MM CRON[4379]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1) root@MM:/var/log#
-
Anyone ?
-
@nagaculun sounds like the swap space might need to be enlarged
-
@sdetweil and how do I do that ?
Thanks
-
@nagaculun now now… google is your friend
see https://www.bitpi.co/2015/02/11/how-to-change-raspberry-pis-swapfile-size-on-rasbian/
-
@nagaculun Are you using MMM-GoogleTrafficMap? This module has a severe memory leak. Increasing swap memory will not help, as it just shifts the freeze to a later point in time. Other option is to setup a cron job that reboots regularly the RPi.
-
@Fozi i have a fixed GoogleMapsTraffic module on my github.
I submitted it, but it hasn’t been picked up…