"Out of memory" issues - where do I begin?

sdetweil

@ember1205 said in "Out of memory" issues - where do I begin?:

Glad I can help you keep improving your code! lol

me too… thought I had handled that… guess not

ember1205

Based on the directions you gave in this thread, the run-start.sh script won’t work. I didn’t take the time to look through the github content to see if this is covered there, but you have to do

chmod +x run-start.sh

Or the script won’t actually execute. Based on old habits, I was executing the chmod statement without even thinking about it. But, after downloading the newest script, I purposefully left it untouched and it wouldn’t run.

sdetweil

@ember1205 hm, thought it was uploaded with exec. will check this evening. have doctor appt shortly

thanks for the info

ember1205

I simply opened the github page, copied the link, and used wget to download it to the mirror. So, it’s entirely possible that the way I downloaded it was partly to blame. Figured I would mention it because I’m likely not the only one that might run into this, and at a minimum it could become a troubleshooting step if others run into issues.

sdetweil

@ember1205 thx. wget & curl won’t copy the execute permissions. when I install this on a pi 0 I chmod it to executable

ember1205

You can’t make this stuff up…

Mirror has been running for a couple of hours now without crashing as it has been doing intermittently in the past. However, there is now a “pop up” on the screen stating that Chromium couldn’t be updated and that I’m missing out on features and fixes.

I found what might be a fix in the way of potentially “configuring” Chromium to ignore updates, but I won’t know if this truly works until probably tomorrow at the earliest.

ember1205

I’m now about 2.5 days into two MM’s running without crashes using Chromium instead of Electron. I’ve also not seen a return of the pop up from Chromium about needing to be updated.

For those that might need to correct the update issue, here’ s what I did on my mirrors:

sudo touch /etc/chromium-browser/customizations/01-disable-update-check;echo CHROMIUM_FLAGS=\"\$\{CHROMIUM_FLAGS\} --check-for-update-interval=31536000\" | sudo tee /etc/chromium-browser/customizations/01-disable-update-check

sdetweil

@ember1205 cool. thx for that info. I will add it to my install script for pi0!!

ember1205

UPDATE: At this point, I’m closing in on a full week of operating two mirrors without issue. I’ve changed to running Chromium instead of Electron and it’s working well all around. Here are some details of the operation in case it’s useful:

Pi 3 Model B (one UK, one China)
MM 2.12.0
Native modules configured: WeatherForecast, CurrentWeather (both weather modules set to different locations as the two Pi’s are in different locations), clock, calendar (2 instances, 2 different Google calendars), alert, updatenotification
Third-party modules: MMM-ImagesPhotos (@sdetweil version)
Customizations:
- Modified the WeatherForecast module according to this thread to get 5 days of forecast data back after recent MM update: https://forum.magicmirror.builders/topic/13187/weatherforecast-showing-only-two-days/31?_=1601303807634
- Converted to using Chromium instead of Electron locally as detailed in this thread
- Chromium update / notice disabled via the step from a couple of posts prior to this one
- Allow connections from local LAN addresses
Pi connected to a smart plug that turns on each day at 7AM, cron job to execute “init 0” each day at 11PM, smart plug turns off at 11:05PM

Prior to successfully switching over to Chromium, the devices would crash at least once every other day (at least one of them would crash during a 48 hour window), although it was much more common to see each one crash multiple times per day. There was no consistency to which one would crash, why, when, etc. The “crash” in question was a black screen on the mirror and no visible info being output although the mouse cursor would sometimes appear. I wrote a cron job that would run every five minutes, look up the location of the pm2 error logs, check to see if there was an “out of memory” error in the log, and do a restart of MM if there was (the restart consisted of shut down, log flush, and start). This job ran every five minutes and would silently exit if there was no error as the mirror was running ok.

Since the switchover, I have seen zero crashes on either device in almost seven days of operation.

Electron has been discussed ad nauseum as having a variety of shortcomings, bugs, issues, etc. - especially in the old version being used with MM. At this point, I have to wonder why it’s still not only the default but the ‘only’ browser that’s really discussed for use with MM. It would seem that it’s time to either forklift an upgrade to it within MM or switch to something else (at a minimum, at least provide a well-documented alternative).

I’m grateful to @sdetweil for his assistance with this, especially since it really took a calendar year almost to get to a point where it’s seemingly working as expected now.

sdetweil

@ember1205 sometimes upgrading one part causes forced upgrades to others and the collection brings more problems…

ember1205

@sdetweil I can appreciate that - I work for a software company, and know about inter-dependencies and the headaches they can cause.

MM is, in a very basic sense, a web server. Electron is a web browser. There shouldn’t be inter-dependencies between a web server and browser, and that seems to be apparent with Chromium not having an issue running against the MM server. So, I would hope that Electron could be brought forward or at least Chromium be documented as an solid alternative.

sdetweil

@ember1205 I got us moved from V3 to V6… we need to add another cycle… it didn’t work with V7 last time.

ember1205

@sdetweil v10 is out and listed as “stable”.

sdetweil

@ember1205 yeh, that and a quarter!!.. V7 was stable when we tried it last year too… running on PI is a whole different thing… they test on x86.

ember1205

Update… 30 full days since my last post about stability and have not experienced a single crash on either mirror. So, Electron is definitely the source of the problem.

Thank you to @sdetweil for the assistance in swapping over to Chromium and getting things to a stable state.

sdetweil

@ember1205 thank you for the feedback… there is a proposal to move to Electron V10

ember1205

Wanted to post what is likely a ‘final update’ to this thread. I was running for literally weeks if not months with zero issues. I even let the system run for at least a month with the “you’re out of date” banner across the top and not one crash.

I updated the mirror and it reverted itself back to using Electron (that in itself is infuriating) and it’s crashing again. So, not only is Electron the problem but it’s STILL a problem.

I’ve reverted my mirror back to the outdated code and will do another update and not allow it to change back to Electron so that it can run without crashing!

sdetweil

@ember1205 what do you mean ‘updated’? if u just did a git pull, then u updated to the same level of electron

new release coming this week (1/1) which moves up to electron 8. 5.3

if u want to test it today, use the develop branch

both still use only electron, you can reinsert run-start.sh to use chromium

ember1205

@sdetweil I used your upgrade script on my mirror. During the upgrade, it reverted the configuration back to using Electron instead of retaining the run-start.sh directive that would launch Chromium.

I wiped out the entire MM directory and copied back the working one that wasn’t upgraded. Unfortunately, it now seems that it is refusing to run the upgrade script and I’m wondering if it’s a certificate issue and I need to update the Pi itself first…

sdetweil

@ember1205 the upgrade doesn’t know about your local change to use run-start… but all u had to do was restore that one line in package.json

it now seems that it is refusing to run the upgrade script

show me the ~/MagicMirror/installers/upgrade.log