@bhepler

That would seem a reasonable approach, but all you can definitively determine is that a specific module is not the sole culprit if the error persists.

• Disable module, error doesn’t return in 24 hours. This does not definitively prove anything as it might be a still-enabled module that causes the fault but only because some other module or setting is creating the condition where the fault occurs.

• Disable module, error returns. This does not definitively prove anything except that the disabled module is not the sole reason for the fault. It could be creating “some” of the faults or be related to the fault occurring.

In other words, this is essentially the situation of it being impossible to prove a negative.

I use ImagesPhotos and have issues with my RPi3 faulting. I also use the default CurrentWeather, WeatherForecast, and GoogleCalendar modules.

@bhepler

Once-a-day reboots don’t generally help much with random lock-ups. I wrote a script that checks the error log for “out of memory”. If the string is found, it shuts down the mirror, flushes the logs, starts the mirror back up, and sends me a push notification so I know of the error. I run the script via cron every five minutes and get anywhere from 2 to 20 events per day with no rhyme or reason to it.

Also, my mirror shuts down each day at 11PM and boots up at 7AM. So, it’s only running for 16 out of 24 hours every day.

I have a few standard modules installed and ONE 3rd party module. Mine locks up -all the time- and I can not figure out why. The 3rd party module is one that the author runs in his own environment with a setup just like mine and never has an issue.

I’ve been chasing the issue with zero progress for weeks and weeks.

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 said in Third-Party Module - how to get help?:

What I saw seemed to be different from a “slow paint” issue - it painted the top half and stopped. It sat there for a few seconds and then the image switched to the next one.

from our text messages, I think this was caused by the bug u found on image update scheduling…
I have fixed that now (thanks by the way, been looking for it for over a year now)

I don’t know that I deserve any credit here as I just passed along an observation after you modified your code based on a question I had about whether it was possible to do something. Glad you found and corrected something you had been chasing, though.

@User_1234 said in Is there any command line that I can send an alert?:

I’m new to programming and I’m trying to create a magic mirror. Do i have to create a .js to execute the alert?

What kind of alert? What is it that you’re trying to accomplish?

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 no… I have seen the slow paint issue myself many times…

electron is having some painting problem… again we are on 3.0.13, no longer supported since way back…

I had tried to get us to 6.0.12 this release, but the test system failed, and they removed this change from the 2.10 release
(but I have seen paint problems on 6.0.12 too)

u can try this manually

cd ~/MagicMirror
npm install electron@6.0.12

What I saw seemed to be different from a “slow paint” issue - it painted the top half and stopped. It sat there for a few seconds and then the image switched to the next one.

While upgrading manually would certainly get me to something newer for Electron, it puts me in the position of not having a “standard install” for future troubleshooting and such. It’s a double-edged sword…

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 you need to do another git pull…

i found a case where it doesn’t update the image anymore…

Will do.

As an aside… I directly witnessed an odd scenario the other day when I had it refreshing every five minutes… one of the images only loaded the upper half on the screen (the bottom half of the screen had no image and was blank). I could not tell what image it was as I have quite a few in rotation and the frame continued on cleanly from there with the next image.

Is that in any way related to what you found?

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 that big pile of warning is just that warning… a couple years ago, the networking community decide that EVEY website should move away from http, to https, to reduce hacking the clear text content as it went by…

so, electron is warning that u didn’t use https

Oh, I’m well aware. This sort of stuff is “what I do” every day. I just find it interesting that I didn’t see it before, see it now, and can’t seem to find the error that is being indicated is present.

Switching to an external browser allows me to find the error - it’s actually related to one of the weather modules. The specific complaint is that I don’t have permission to load the weather map.

Failed to load resource: the server responded with a status of 401 (Unauthorized)
https://api.openweathermap.org/data/2.5/forecast/daily?id=< removed>&units=imperial&lang=en&APPID=< removed>

I’ll have to look and see if I can turn off the attempt to load that map.

I don’t seem to be seeing that error any more, but now it’s crying about not using HTTPS (warning, not error, and specifically for Electron). For whatever reason, that warning wasn’t showing up in the console prior although I’m sure that it was there in some capacity.

I -do- see an indication of an error but can’t seem to find any error in the console.

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 said in Third-Party Module - how to get help?:

:8080/MMM-ImagesPhotos/photos:1

yeh, the code attempts to load the folder… once… I haven’t debugged it cause it doesn’t matter

Thanks.

What about second error I mentioned? Or, is that essentially part of the overall error?

Also - I caught this error with your debug output:

Failed to load resource: the server responded with a status of 404 (Not Found) :8080/MMM-ImagesPhotos/photos:1 

That’s at the beginning of the console output along with another error I have seen later in the console:

MMM-ImagesPhotos Could not load photos
photosrequest.onreadystatechange @ :8080/modules/MMM-ImagesPhotos/MMM-ImagesPhotos.js:65

Any thoughts on those?

I know this is marked ‘solved’, but I wanted to mention that A ) the module does still crash with “out of memory” errors and B ) a little bit of investigative tinkering makes me think that this all ties back to a garbage collection issue with Java.

If I change the timing of how often the images change, I can effectively ‘force’ or almost eliminate a crash. The shorter the duration between photos, the more likely a crash will occur. I changed the timing to 45 seconds and saw the setup run reliably from power on at 7AM until power off at 11PM for multiple days in a row.

Shortening the window down to 5 seconds between images will cause the system to crash typically within about 10-15 minutes at a maximum.

At no time do I actively witness memory levels dropping down to zero, but that could just be “bad luck” in not seeing the issue arise when it errors out. Is there an easy way on the Pi to track a process and watch for a specific, correlating event when something crashes?

There’s a REALLY simple way to do this using a script and a cron job.

From the pi user, execute:

sudo nano /bin/mmreload.sh

Paste the following contents into that file:

#!/bin/sh

if [ /home/pi/MagicMirror/config/config.js -nt /home/pi/.pm2/pids/mm-0.pid ]; then
/usr/bin/pm2 restart mm
fi

Save it (CTRL-X) and then give it execute permissions like this:

sudo chmod +x /bin/mmreload.sh

From the pi user, execute the following:

crontab -e

In the crontab file, add the following line:

0,30 * * * * /bin/mmreload.sh  > /dev/null 2>&1

Save the file (CTRL-X if you’re using nano).

This will check the MM ‘pid’ file against the config.js file to see if the configuration file has been edited since the mirror was last started (it will check at the top and bottom of every hour). If the config file has been edited, it will restart MM and use the new configuration file.

@fbr1969 said in MagicMirror behind a NGinx Reverse Proxy:

@ember1205

Hi,
With my shortcut “mmirror”, I can access directly to the default Magic Mirror page (define in my MM config.js file), at home (LAN) or at work (by Internet).
And effectively, I had needed to handle the /socket.io path independently of the of /mmirror path to correct the error I found I my NGinx error log. And that worked like that.
I’ sorry, but I don’t know how it works exactly…

Thanks for the extra detail - your description increases my suspicion/concern that there is likely a lot of JS content being delivered directly to the browser that is not being otherwise rewritten, hence the need for the additional path for socket.io

On my mirror, with a total of five modules (all default except one), there are twenty-six JS scripts that are delivered to the browser for “local” execution. Fortunately, of them, ONLY the socket.io call has a hard path to the root of the server:

< script type="text/javascript" src="/socket.io/socket.io.js"></script>

Nothing else uses the leading “/” in the path which means they are all relative paths and will properly pass through a reverse proxy for access. Even though the other twenty-five scripts can be downloaded, there’s nothing to say that the content WITHIN them can execute if they make any subsequent calls using hard-coded information that isn’t being rewritten.

Reverse proxy servers are awesome tools, but the configuration has to be ‘perfect’ to ensure everything will work properly now and in the future. And rewriting JS is not always trivial.

@fbr1969

A couple of questions about your configuration…

• It seems that you configured ONLY for access to the mirror with a /mmirror path as well - is this accurate? I can’t tell where the ‘default’ VIP might be pointing to.

• It appears that you have to handle the /socket.io path independently of the of /mmirror path, and that leads me to wonder if there is javascript code being returned to the browser that ALSO needs to be rewritten inline?

When using a proxy server like this, it functions mostly like a content switch. In other words, you can redirect specific paths to different back-end hosts - this is commonly done when you have different servers holding different kinds of content.

When you have a VIP acting as a Content Switch, anything not explicitly defined for a particular path will be handled by the default VIP. In this case, he has not actually defined the /mmirror path, so appending that to his domain will result in the DEFAULT server getting the traffic as opposed to the mirror. That’s why the 404.

I believe that the directive I mentioned earlier is all that needs to be defined in order to get the nginx proxy handling that path accordingly…

@sdetweil said in MagicMirror behind a NGinx Reverse Proxy:

@ember1205 its pretty clear tho

curl -iL http://192.168.10.10/mmirror
HTTP/1.1 301 Moved Permanently
Server: nginx/1.14.2
Date: Mon, 23 Dec 2019 18:14:36 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://[MyDomain]/mmirror

HTTP/1.1 404 Not Found
Server: nginx/1.14.2
Date: Mon, 23 Dec 2019 18:14:36 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 146
Connection: keep-alive
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self'

Error
<pre>Cannot GET /mmirror</pre>

Cannot GET /mmirror

because that is on the client issuing the curl command, url with no ip address is same as localhost

what is this??
Location: https://[MyDomain]/mmirror

again this should be server.mydomain

if u go to client and do an nslookup domain.com
u will get an error
ping domain.com will fail

He stated that he is masking details of what he’s posting, so it’s definitely something to check.

However…

The error is delivered from nginx

Server: nginx/1.14.2

Which means that it isn’t attempting to retrieve the URL from localhost.

The missing directive that I mentioned prior is where I believe the issue is… The client IS connecting and requesting /mmirror, but that is not defined on the nginx host. As a result, it’s being processed by the default host configured there which will be on a different back-end server and that server does not have a /mmirror path available to serve. So, 404.

@fbr1969 said in MagicMirror behind a NGinx Reverse Proxy:

@sdetweil @ember1205 Thank for your help.

I forgot to say :

• My nginx server is fully fonctional with some other locations I’ve masqued in my exemple of configuration files… Like that :

location /mmirror {
    proxy_bind              $server_addr;
    proxy_pass              http://192.168.10.60:8080;
    proxy_set_header        Host            $http_host;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Scheme        $scheme;
    proxy_set_header        X-Script-Name   /;  # IMPORTANT: path has NO trailing slash
}

I think I have some problems with some "proxy_set_header " rules.

It appears that your X-Script-Name IS setting a trailing slash and isn’t inserting / removing mmirror. Shouldn’t that last line be

     proxy_set_header        X-Script-Name   /mmirror;  # IMPORTANT: path has NO trailing slash

??

@JohnDelta said in How to disable auto update from MMM-GooglePhotos:

@sdetweil can you help me out what a cron tool is? Haven’t heard about it yet.

Cron is a scheduling tool that’s available in various linux, BSD, Mac, Unix, etc. operating systems. It allows you to schedule “jobs” (scripts, typically) to run on certain days, at certain times, with certain privileges to get things done.

On my mirror, I have a script that I’ve written to rsync images from a server on my network over to the Pi. Cron allows me to schedule the execution of that script at times that I want it to run (every 30 minutes, in this case) and copy over changes from the server. You could do something similar with a tool that could connect to a Google photo album, Google Drive, or wherever and copy the most current image to a local directory. After that, whatever module you’re using to display that image would be able to update what’s on the screen either by itself, or you could integrate a “notification” within your script to tell the module to reload the image once the copy is done.

Firstly, do you really want to put your mirror behind a piece of software developed by the Russians, released into Open Source, then claimed by another Russian Company as “theirs” after it was purchased out by a commercial entity (F5)? If you need a reverse proxy, you’re likely to have all that you need from a simply Apache setup.

Second - you will need to ensure that all of your rewrite rules take into account the back-end port along with the front-end port information as well as specific host name rules.

For example:

If your MM is 192.168.1.10 and you have a system running reverse proxy with a VIP on 192.168.1.50, you will need to decide if you’re going to continue using port 8080 on the front-end or if you want to fall back to standard port 80 (or even tie it up with a certificate on 443 with HTTPS/SSL).

In other words, you could be doing client to proxy as 192.168.1.100 -> http://192.168.1.50 or 192.168.1.100 -> http://192.168.1.50:8080 or 192.168.1.100 -> https://192.168.1.50 or something else entirely.

After you have decided how you’re handling the front end, you then need to ensure that you’re rewriting ALL of that as it passes through the proxy to/from http://192.168.1.10:8080

You ALSO need to ensure that you have updated the configuration on MM to allow connections from off-box IP’s. Specifically, unless you are forcing the use of the original source IP, connections to the mirror should be coming from the system running your proxy and THAT address needs to have access to the web server on the mirror.