@sdetweil said in Third-Party Module - how to get help?:

@ember1205 that big pile of warning is just that warning… a couple years ago, the networking community decide that EVEY website should move away from http, to https, to reduce hacking the clear text content as it went by…

so, electron is warning that u didn’t use https

Oh, I’m well aware. This sort of stuff is “what I do” every day. I just find it interesting that I didn’t see it before, see it now, and can’t seem to find the error that is being indicated is present.

Switching to an external browser allows me to find the error - it’s actually related to one of the weather modules. The specific complaint is that I don’t have permission to load the weather map.

Failed to load resource: the server responded with a status of 401 (Unauthorized)
https://api.openweathermap.org/data/2.5/forecast/daily?id=< removed>&units=imperial&lang=en&APPID=< removed>

I’ll have to look and see if I can turn off the attempt to load that map.

I don’t seem to be seeing that error any more, but now it’s crying about not using HTTPS (warning, not error, and specifically for Electron). For whatever reason, that warning wasn’t showing up in the console prior although I’m sure that it was there in some capacity.

I -do- see an indication of an error but can’t seem to find any error in the console.

@sdetweil said in Third-Party Module - how to get help?:

@ember1205 said in Third-Party Module - how to get help?:

:8080/MMM-ImagesPhotos/photos:1

yeh, the code attempts to load the folder… once… I haven’t debugged it cause it doesn’t matter

Thanks.

What about second error I mentioned? Or, is that essentially part of the overall error?

Also - I caught this error with your debug output:

Failed to load resource: the server responded with a status of 404 (Not Found) :8080/MMM-ImagesPhotos/photos:1 

That’s at the beginning of the console output along with another error I have seen later in the console:

MMM-ImagesPhotos Could not load photos
photosrequest.onreadystatechange @ :8080/modules/MMM-ImagesPhotos/MMM-ImagesPhotos.js:65

Any thoughts on those?

I know this is marked ‘solved’, but I wanted to mention that A ) the module does still crash with “out of memory” errors and B ) a little bit of investigative tinkering makes me think that this all ties back to a garbage collection issue with Java.

If I change the timing of how often the images change, I can effectively ‘force’ or almost eliminate a crash. The shorter the duration between photos, the more likely a crash will occur. I changed the timing to 45 seconds and saw the setup run reliably from power on at 7AM until power off at 11PM for multiple days in a row.

Shortening the window down to 5 seconds between images will cause the system to crash typically within about 10-15 minutes at a maximum.

At no time do I actively witness memory levels dropping down to zero, but that could just be “bad luck” in not seeing the issue arise when it errors out. Is there an easy way on the Pi to track a process and watch for a specific, correlating event when something crashes?

There’s a REALLY simple way to do this using a script and a cron job.

From the pi user, execute:

sudo nano /bin/mmreload.sh

Paste the following contents into that file:

#!/bin/sh

if [ /home/pi/MagicMirror/config/config.js -nt /home/pi/.pm2/pids/mm-0.pid ]; then
/usr/bin/pm2 restart mm
fi

Save it (CTRL-X) and then give it execute permissions like this:

sudo chmod +x /bin/mmreload.sh

From the pi user, execute the following:

crontab -e

In the crontab file, add the following line:

0,30 * * * * /bin/mmreload.sh  > /dev/null 2>&1

Save the file (CTRL-X if you’re using nano).

This will check the MM ‘pid’ file against the config.js file to see if the configuration file has been edited since the mirror was last started (it will check at the top and bottom of every hour). If the config file has been edited, it will restart MM and use the new configuration file.

@fbr1969 said in MagicMirror behind a NGinx Reverse Proxy:

@ember1205

Hi,
With my shortcut “mmirror”, I can access directly to the default Magic Mirror page (define in my MM config.js file), at home (LAN) or at work (by Internet).
And effectively, I had needed to handle the /socket.io path independently of the of /mmirror path to correct the error I found I my NGinx error log. And that worked like that.
I’ sorry, but I don’t know how it works exactly…

Thanks for the extra detail - your description increases my suspicion/concern that there is likely a lot of JS content being delivered directly to the browser that is not being otherwise rewritten, hence the need for the additional path for socket.io

On my mirror, with a total of five modules (all default except one), there are twenty-six JS scripts that are delivered to the browser for “local” execution. Fortunately, of them, ONLY the socket.io call has a hard path to the root of the server:

< script type="text/javascript" src="/socket.io/socket.io.js"></script>

Nothing else uses the leading “/” in the path which means they are all relative paths and will properly pass through a reverse proxy for access. Even though the other twenty-five scripts can be downloaded, there’s nothing to say that the content WITHIN them can execute if they make any subsequent calls using hard-coded information that isn’t being rewritten.

Reverse proxy servers are awesome tools, but the configuration has to be ‘perfect’ to ensure everything will work properly now and in the future. And rewriting JS is not always trivial.

@fbr1969

A couple of questions about your configuration…

• It seems that you configured ONLY for access to the mirror with a /mmirror path as well - is this accurate? I can’t tell where the ‘default’ VIP might be pointing to.

• It appears that you have to handle the /socket.io path independently of the of /mmirror path, and that leads me to wonder if there is javascript code being returned to the browser that ALSO needs to be rewritten inline?

When using a proxy server like this, it functions mostly like a content switch. In other words, you can redirect specific paths to different back-end hosts - this is commonly done when you have different servers holding different kinds of content.

When you have a VIP acting as a Content Switch, anything not explicitly defined for a particular path will be handled by the default VIP. In this case, he has not actually defined the /mmirror path, so appending that to his domain will result in the DEFAULT server getting the traffic as opposed to the mirror. That’s why the 404.

I believe that the directive I mentioned earlier is all that needs to be defined in order to get the nginx proxy handling that path accordingly…

@sdetweil said in MagicMirror behind a NGinx Reverse Proxy:

@ember1205 its pretty clear tho

curl -iL http://192.168.10.10/mmirror
HTTP/1.1 301 Moved Permanently
Server: nginx/1.14.2
Date: Mon, 23 Dec 2019 18:14:36 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://[MyDomain]/mmirror

HTTP/1.1 404 Not Found
Server: nginx/1.14.2
Date: Mon, 23 Dec 2019 18:14:36 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 146
Connection: keep-alive
X-DNS-Prefetch-Control: off
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Download-Options: noopen
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: default-src 'self'

Error
<pre>Cannot GET /mmirror</pre>

Cannot GET /mmirror

because that is on the client issuing the curl command, url with no ip address is same as localhost

what is this??
Location: https://[MyDomain]/mmirror

again this should be server.mydomain

if u go to client and do an nslookup domain.com
u will get an error
ping domain.com will fail

He stated that he is masking details of what he’s posting, so it’s definitely something to check.

However…

The error is delivered from nginx

Server: nginx/1.14.2

Which means that it isn’t attempting to retrieve the URL from localhost.

The missing directive that I mentioned prior is where I believe the issue is… The client IS connecting and requesting /mmirror, but that is not defined on the nginx host. As a result, it’s being processed by the default host configured there which will be on a different back-end server and that server does not have a /mmirror path available to serve. So, 404.

@fbr1969 said in MagicMirror behind a NGinx Reverse Proxy:

@sdetweil @ember1205 Thank for your help.

I forgot to say :

• My nginx server is fully fonctional with some other locations I’ve masqued in my exemple of configuration files… Like that :

location /mmirror {
    proxy_bind              $server_addr;
    proxy_pass              http://192.168.10.60:8080;
    proxy_set_header        Host            $http_host;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Scheme        $scheme;
    proxy_set_header        X-Script-Name   /;  # IMPORTANT: path has NO trailing slash
}

I think I have some problems with some "proxy_set_header " rules.

It appears that your X-Script-Name IS setting a trailing slash and isn’t inserting / removing mmirror. Shouldn’t that last line be

     proxy_set_header        X-Script-Name   /mmirror;  # IMPORTANT: path has NO trailing slash

??

@JohnDelta said in How to disable auto update from MMM-GooglePhotos:

@sdetweil can you help me out what a cron tool is? Haven’t heard about it yet.

Cron is a scheduling tool that’s available in various linux, BSD, Mac, Unix, etc. operating systems. It allows you to schedule “jobs” (scripts, typically) to run on certain days, at certain times, with certain privileges to get things done.

On my mirror, I have a script that I’ve written to rsync images from a server on my network over to the Pi. Cron allows me to schedule the execution of that script at times that I want it to run (every 30 minutes, in this case) and copy over changes from the server. You could do something similar with a tool that could connect to a Google photo album, Google Drive, or wherever and copy the most current image to a local directory. After that, whatever module you’re using to display that image would be able to update what’s on the screen either by itself, or you could integrate a “notification” within your script to tell the module to reload the image once the copy is done.

Firstly, do you really want to put your mirror behind a piece of software developed by the Russians, released into Open Source, then claimed by another Russian Company as “theirs” after it was purchased out by a commercial entity (F5)? If you need a reverse proxy, you’re likely to have all that you need from a simply Apache setup.

Second - you will need to ensure that all of your rewrite rules take into account the back-end port along with the front-end port information as well as specific host name rules.

For example:

If your MM is 192.168.1.10 and you have a system running reverse proxy with a VIP on 192.168.1.50, you will need to decide if you’re going to continue using port 8080 on the front-end or if you want to fall back to standard port 80 (or even tie it up with a certificate on 443 with HTTPS/SSL).

In other words, you could be doing client to proxy as 192.168.1.100 -> http://192.168.1.50 or 192.168.1.100 -> http://192.168.1.50:8080 or 192.168.1.100 -> https://192.168.1.50 or something else entirely.

After you have decided how you’re handling the front end, you then need to ensure that you’re rewriting ALL of that as it passes through the proxy to/from http://192.168.1.10:8080

You ALSO need to ensure that you have updated the configuration on MM to allow connections from off-box IP’s. Specifically, unless you are forcing the use of the original source IP, connections to the mirror should be coming from the system running your proxy and THAT address needs to have access to the web server on the mirror.

I used an iterative command for this because there are so many photos:

cd ~MagicMirror
find < path to images >/* -exec node_modules/.bin/electron file:///{} \;

For each file in that directory, it would launch the electron browser and open the image. Because electron continues to run until it is terminated, this puts one image at a time on the screen. After each one loaded, I would simply click “X” to close the browser and it would move on to the next image.

While I -did- see one specific image file appear to load but not actually show any content (initially), I believe this may have been an issue with the fact that I walked away from the process for about ten minutes. Expanding the browser window caused it to re-render and it displayed fine. Additionally, this is an image that I have personally seen on the screen many times with no apparent link to any crashes.

It seems that any image-specific issues are possibly cumulative since loading each one in turn starts with a fresh memory allocation as the browser exits after each image.

What would I be looking for?

Turns out I’m wrong on two counts…

1. I -have- been testing the caching disable extensively as it’s currently disabled. :)

2. There is not a slow-paint issue for images with caching disabled.

Unfortunately, I can now say that I’m pretty comfortable with the assessment of this NOT being a caching issue. With caching disabled for electron, the issue has now occurred multiple times in just the last 4-5 hours.

I’m not sure that this is a caching error as it seems related very directly to the displaying of images but “rears up” differently based on the module. In the prior module I was using, displaying ANY graphical content lead to a quick demise of the system (within 15 minutes or so, typically). With the current module, everything was 100% fine until I cropped some of the images. The current module is one you are using with zero images, so there has to be something about the crop method I used that threw things out of whack.

Even if it IS a straight-up caching issue, I can’t disable caching because the image display process looks poor. I’ve tried with caching disabled and it is not at all smooth, and I also haven’t tested that anywhere near enough to actually validate that it corrects anything.

Is there a way for me to be able to at least understand the process in effect when this error gets thrown? The entry in the error log seems super generic with no real information along with it.

Up until yesterday, my MM had been running reliably over a period of days without blanking the screen or locking up.

I have a few default modules (calendar, currentweather, and weatherforecast) enabled and only one third-party (MMM-ImagesPhotos). After correcting some CSS and config.js mistakes over the last couple of days so that it displays what I want, and how I want it, I decided to move on to tinkering with the images themselves.

I have a lot of photos that I am using in the background. Many were shot on my DSLR (24MP) and some came from other DSLR’s (Disney’s Photo Pass) and some smart phones. Using the default image size (3:2 ration) for landscape left gray letterbox bars on the sides of the screen. I began cropping the photos from 3:2 to 16:9 and replacing the original files.

While the photos display mostly “perfect” (for some reason, I am still seeing what looks like a 1-2 pixel “frame” around many images that should be full screen), the RPi (Model 3 v1.2) is back to blanking the screen after some period of run time and the logs show “Out of memory.” I don’t know if it’s related, but I was using a different module to display photos in the background (MMM-BackgroundSlideshow) and was using the exact same set of photos. With that module, I had not cropped ANY of the photos but was getting the same sort of “Out of memory” behavior.

I’ve seen some other possibly anecdotal ‘reports’ of certain image-related details causing this sort of error, so I’m wondering if it’s at all related. I also read through a thread from not-that-long-ago where various bugs in the notification module were being fixed (and the fixes being tested), but there wasn’t any follow-up in that thread as to the status.

I’ve also tried increasing swap space on the MM, but that has not made any difference.

Any tips on how to go about starting to troubleshoot this? The device itself remains 100% responsive and network access is fine. I can send signals to turn the display on and off and that works as well. The only way to “fix” the solid black screen is to restart MM or the entire device.

@sdetweil said in Calendar Doesn't Honor custom.css for Background:

@ember1205 said in Calendar Doesn't Honor custom.css for Background:

the Developer options with a real browser

electron is a real browser, so you can do this ON the mirror,
or from your desktop if the config parms allow remote system access.

I always forget that I can use local browser to view developer window

Now that you mention it, I have used the developer options directly on the appliance and should have remembered that. Thanks for the reminder!

@sdetweil

I found and fixed it (sort of). I had added content to my custom.css file to create a blurred background image as I was experimenting with some display options. I commented that content out (or so I thought) and placed all of these controls “after” that section in the custom.css file.

Through a variety of changes that I was making to try and track this down, I noticed through viewing the Developer options with a real browser and looking at the elements that whatever section came first after my blurred background section wasn’t being picked up. As I rolled the changes down to the bottom, I was seeing different sections stop working.

Seems to be all fixed now, and is clearly not a bug in the product but was a bug in my custom.css content. After deleting the content, all is displaying properly.