Read the statement by Michael Teeuw here.
Every few hours I get "ERROR:network_service_instance_impl.cc(916)] Network service crashed, restarting service"
-
@tve but see his post
https://forum.magicmirror.builders/topic/16485/temperature-of-a-rpi3
60 is bad if u can do 30!
-
@sdetweil said in Every few hours I get "ERROR:network_service_instance_impl.cc(916)] Network service crashed, restarting service":
@tve but see his post
https://forum.magicmirror.builders/topic/16485/temperature-of-a-rpi3
60 is bad if u can do 30!
Great post(!)
(And here I thought that 60 was the normal/expected working temperature)
I think I’ll add a passive solution (or maybe even an active - but then there is a fan that will die)
(I have some other - rather expensive - equipment that died due to a dead fan, and unfortunately there was no way to measure the temperature - bad design…)I see these two obvious options:
- https://uk.rs-online.com/web/p/raspberry-pi-hats-add-ons/8679039
- https://uk.rs-online.com/web/p/raspberry-pi-hats-add-ons/2020449
The active solution might even add enough cooling even when the fan dies (as it seems that there is a passive “path for the heat” from the CPU
I should try both :-)What is to gain?:
I am aware that the warmer the chip the shorter it’s life
(I have complained to a vendor that one of their products got so warm that I burn myself if I touch the enclosure, but they insist that it’s expected - even though I have had to get at least three of my nine devices replaced!
I think it’s due to a bad design - hence my heat complaint…)The rule of thumb is something like this:
By increasing the device temperature by just 10°C, we have reduced the lifetime by over 2x
BTW: I have not seen any more network-service-instance-crashes, it’s still going strong after 8 days, really weird (but good!)…
-
63.4 C? Could be less!
I have now added one of these [1] to my RPI 3 and returned it to the enclosure (the “starter kit cabinet”).
The temperature is now 48-49 degrees centigrade after 30 min usage of MM.
So it reduces the temperature with ~14K - thats nice (if you don’t mind listening to a spinning fan which is actually fairly loud in at least 2 meters distance)…!I’ll try passive cooling later…
Oh! - and I had a 14 day uptime before powering off (the original reason for this thread :-)
[1] https://uk.rs-online.com/web/p/raspberry-pi-hats-add-ons/2020449
-
@TvE said in Every few hours I get "ERROR:network_service_instance_impl.cc(916)] Network service crashed, restarting service":
Oh! - and I had a 14 day uptime before powering off (the original reason for this thread
nice!!
-
@sdetweil Yeah - that’s perfect (and expected)
I really have (now stopped) wondering what happened initially :-)@thgmirror
I made a quick test where I have removed the power to the fan to see how much that changes the temperature (aka a passive cooling test).- After ca. one hour the temperature rises to a max of 57-58 degrees centigrades
- For the next hour the temperature is not rising any more
- Thus a reduction of ~6K
NB.: I have now removed the lid to see how big (small) a difference that makes
-
So - without the lid I get another 2K as - after 15 min - the temperature stabilized around 54-55 degrees centigrades
* Passive cooling lid on : 63 -> 58 = -5 * Passive cooling lid off : 63 -> 55 = -8 * Active cooling lid on : 63 -> 49 = -14 * Active cooling lid off : 63 -> 42 = -21 -
@TvE I am in the same spot as you.
Raspberry Pi 3b+
Fresh installed Raspbian Bullseye 11
Fresh install of MM with vanilla config and modules - crashes every minute and restarts itself. Are you saying that the config might be the problem?0|MagicMirror | Launching application. 0|MagicMirror | [18725:0322/102613.376542:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization 0|MagicMirror | [18766:0322/102613.700219:ERROR:sandbox_linux.cc(376)] InitializeSandbox() called with multiple threads in process gpu-process. 0|MagicMirror | [22.03.2022 10:26.16.088] [LOG] 0|MagicMirror | Create new calendarfetcher for url: http://www.calendarlabs.com/ical-calendar/ics/76/US_Holidays.ics - Interval: 300000 0|MagicMirror | [22.03.2022 10:26.16.175] [LOG] 0|MagicMirror | Create new newsfetcher for url: https://rss.nytimes.com/services/xml/rss/nyt/HomePage.xml - Interval: 300000 0|MagicMirror | [22.03.2022 10:26.16.248] [INFO] 0|MagicMirror | Checking git for module: default 0|MagicMirror | [22.03.2022 10:26.17.014] [INFO] 0|MagicMirror | Newsfeed-Fetcher: Broadcasting 61 items. 0|MagicMirror | [22.03.2022 10:26.17.522] [INFO] 0|MagicMirror | Calendar-Fetcher: Broadcasting 10 events. 0|MagicMirror | [22.03.2022 10:27.33.823] [LOG] Shutting down server... 0|MagicMirror | [22.03.2022 10:27.33.833] [LOG] Stopping module helper: updatenotification 0|MagicMirror | [22.03.2022 10:27.33.835] [LOG] Stopping module helper: calendar 0|MagicMirror | [22.03.2022 10:27.33.848] [LOG] Stopping module helper: newsfeed 0|MagicMirror | [18692:0322/102733.909781:ERROR:zygote_communication_linux.cc(276)] Failed to send GetTerminationStatus message to zygote 0|MagicMirror | [18692:0322/102733.942607:ERROR:zygote_communication_linux.cc(276)] Failed to send GetTerminationStatus message to zygote 0|MagicMirror | [18692:0322/102733.958362:ERROR:network_service_instance_impl.cc(916)] Network service crashed, restarting service. 0|MagicMirror | [18692:0322/102733.974458:ERROR:gpu_process_host.cc(962)] GPU process launch failed: error_code=1002 0|MagicMirror | [18692:0322/102733.982862:ERROR:gpu_process_host.cc(962)] GPU process launch failed: error_code=1002 0|MagicMirror | [18692:0322/102733.989821:ERROR:gpu_process_host.cc(962)] GPU process launch failed: error_code=1002 0|MagicMirror | [18692:0322/102733.998527:ERROR:gpu_process_host.cc(962)] GPU process launch failed: error_code=1002 0|MagicMirror | > magicmirror@2.18.0 start 0|MagicMirror | > DISPLAY="${DISPLAY:=:0}" ./node_modules/.bin/electron js/electron.js 0|MagicMirror | [22.03.2022 10:27.42.385] [LOG] 0|MagicMirror | Starting MagicMirror: v2.18.0 0|MagicMirror | [22.03.2022 10:27.42.407] [LOG] 0|MagicMirror | Loading config ... 0|MagicMirror | [22.03.2022 10:27.42.421] [LOG] 0|MagicMirror | Loading module helpers ... 0|MagicMirror | [22.03.2022 10:27.42.427] [LOG] 0|MagicMirror | No helper found for module: alert. 0|MagicMirror | [22.03.2022 10:27.42.463] [LOG] -
@TvE That is my experimental setup:
-
@Ivanov_d
Maybe we’re observing the same - currently it’s hard for me to tell.-
You surely have no heat issue (no additional cooling is needed - only to prolong the life of the RPI)
-
OS
As you hopefully have read in this thread I started with the same OS as you, then “downgraded” to the socalled “legacy” version and still saw the issues. then it suddenly stopped.
Thus I need to replace to the latest OS and retest
I have no logical reason to suspect the OS but I need to do the test to be 100% certain…
-
You can follow my test steps (using bisection - remove the handfull of external calls and see which triggers the problem)
-
My guess
So far I can only conclude that something external provoked the network stack to crash.
In the end of my testing () I had returned to my original config and no longer see problems
That something must relate to one of the two services I was using (part of the vanilla config)
UPDATE!!! - I just looked at my test and wow, this morning I had an error again (my test runs with the monitor turned off, thus I did not see this (I’m working on other tasks at the moment…).
Looking in the logs I see calendar related issues a few hours before the service stopped
My error occurred at [21.03.2022 22:39.02.219] in Denmark
Your error occurred at [22.03.2022 10:27.33.823] in US? (guess deduced from your config)
That way too close to just be a coincidence…I don’t know for how long you have been running your system, but I will now start to suspect that the root cause lies in the calendar service and we both are affected by the same root cause
I wonder if we can find another source for those data
(I now regret that I turned of tcpdump for my latest testing…)
stdout
[21.03.2022 22:34.06.084] [INFO] Newsfeed-Fetcher: Broadcasting 20 items. [21.03.2022 22:39.02.219] [LOG] Shutting down server... [21.03.2022 22:39.02.225] [LOG] Stopping module helper: updatenotification [21.03.2022 22:39.02.226] [LOG] Stopping module helper: calendar [21.03.2022 22:39.02.227] [LOG] Stopping module helper: newsfeederr
[21.03.2022 16:43.52.620] [WARN] ^[[33mYou're using a full whitelist configuration to allow for all IPs^[[39m [1002:0321/164356.019064:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization [1040:0321/164356.860589:ERROR:sandbox_linux.cc(376)] InitializeSandbox() called with multiple threads in process gpu-process. [21.03.2022 17:37.30.487] [ERROR] Calendar Error. Could not fetch calendar: http://calendars.icloud.com/holidays/dk_da.ics FetchError: request to https://calendars.icloud.com/holidays/dk_da.ics failed, reason: connect ETIMEDOUT 17.248.150.146:443 at ClientRequest.<anonymous> (/home/pi/MagicMirror/node_modules/node-fetch/lib/index.js:1483:11) at ClientRequest.emit (node:events:394:28) at TLSSocket.socketErrorListener (node:_http_client:447:9) at TLSSocket.emit (node:events:394:28) at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { type: 'system', errno: 'ETIMEDOUT', code: 'ETIMEDOUT' } [21.03.2022 17:44.40.539] [ERROR] Calendar Error. Could not fetch calendar: http://calendars.icloud.com/holidays/dk_da.ics FetchError: request to https://calendars.icloud.com/holidays/dk_da.ics failed, reason: connect ETIMEDOUT 17.248.150.10:443 at ClientRequest.<anonymous> (/home/pi/MagicMirror/node_modules/node-fetch/lib/index.js:1483:11) at ClientRequest.emit (node:events:394:28) at TLSSocket.socketErrorListener (node:_http_client:447:9) at TLSSocket.emit (node:events:394:28) at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { type: 'system', errno: 'ETIMEDOUT', code: 'ETIMEDOUT' } [958:0321/223902.258167:ERROR:zygote_communication_linux.cc(276)] Failed to send GetTerminationStatus message to zygote [958:0321/223902.279012:ERROR:zygote_communication_linux.cc(276)] Failed to send GetTerminationStatus message to zygote -
-
@TvE it’s been running for 25 hours so far:
pm2 status MagicMirror ┌───────────────┬────┬─────────┬──────┬──────┬────────┬─────────┬────────┬─────┬──────────┬──────┬──────────┐ │ App name │ id │ version │ mode │ pid │ status │ restart │ uptime │ cpu │ mem │ user │ watching │ ├───────────────┼────┼─────────┼──────┼──────┼────────┼─────────┼────────┼─────┼──────────┼──────┼──────────┤ │ MagicMirror │ 0 │ 2.18.0 │ fork │ 2911 │ online │ 15 │ 25h │ 0% │ 2.7 MB │ pi │ disabled │ └───────────────┴────┴─────────┴──────┴──────┴────────┴─────────┴────────┴─────┴──────────┴──────┴──────────┘I am still tweaking my config, however, my approach was the following:
- I updated from Rapbian Buster to Bullseye
- That broke my Raspberry Pi 3b+'s WiFi connection (seems like a known bug https://forums.raspberrypi.com/viewtopic.php?t=325484), so I removed connman package and installed network-manager and connected to the WiFi. If you install network-manager, you will also want to disable the MAC randomization:
To disable the WiFi MAC randomization, create the following file using the this command:
sudo nano /etc/NetworkManager/conf.d/100-disable-wifi-mac-randomization.confand paste the following content inside:
[connection] wifi.mac-address-randomization=1 [device] wifi.scan-rand-mac-address=noSave the changes and continue.
- I backed up my MM config and modules folder and started with a fresh install of MM 2.18
- I started MM with the default config - no luck, it was crashing every minute
- I started removing the default modules and installing the ones that I had before
- I was observing the same behavior as you did - the default modules that needed network connection, when fetching data were breaking the network stack somehow … I edited the config to enter my calendar and newsfeed URLs and that seemed to fixed the issue. It might be something in the response of the default calendar and/or newsfeed … but that is just my speculation.
I also have pulled all the latest updates for every module that I had installed previously just to make sure that I am all up to date.
Ever since as you can see, my MM has been running steadily. Definitely not an SoC temp issue, since I never had anything than the stock aluminum radiators on the Pi and it has been running for 3-4 years 24/7 already.
I wouldn’t have decided to upgrade to bullseye if it was not a requirement for the new version of MMM-GoogleAssistant.
Anyways, I hope that this helps.
P.S. I am in Sofia, so the date/time in my log is GMT +02:00
