Question about module and node_helper communications with large data objects

[skuethe]

Hi everyone,

I started to play around with MM² just some weeks ago, and am currently in the phase of modifying a Module which displays background images from different sources.

One of these sources is a webdav behind basic authentication. Another source would be a “local directory” on the raspberry pi.

My approach is currently the following:

1. Start module, if source is set to “webdav” or “localdirectory” source, issue a sendSocketNotification to node_helper.js
2. node_helper is verifying the sources and creating an array of all images in that source. The array will contain the paths / URLs to each image, so f.e.:

imageList = [
  "https://WEBDAVHOST/path/to/some/image.jpg",
  "https://WEBDAVHOST/path/to/some/otherimage.jpg",
}

3. This array is returned via a new sendSocketNotification to the module
4. The module now has two different approaches:
   4.1 if it is webdav WITHOUT authentication, get one of the image src from the array and use it in getDom()
   4.2 Run a special function if it is webdav WITH authentication OR localdirectory. This special function is basically just base64 encoding one of the images from array and using that for the img srcattribute

4.2 is using base64 encoded images as

• we can’t reference local images from remote systems and
• I don’t want to add basic authentication as “https://USERNAME:PASSWORD@WEBDAVHOST” into the img src

Question / Problem
My problem occurs with 4.2, actually running the function to encode the image. As I only return an array with all the image paths / urls from node_helper and not the acutal images in base64 encoded data (which would probably blow up at some point, f.e. referencing a lot of 4k images), I have to encode them “just in time”. So on each request (images are just shown one at a time as background).

Now to my understanding (and testing) I am unable to add something like const fs = require("fs"); inside the main module, right? How could I actually fetch local files from the main module then?
Or, if this is not possible, how can I send a request from main module to node_helper and wait for its return / callback?

The same with the basic auth webdav option.
I tried using jQuery to inject basic auth header on image load, but that just fails with CORS (because the webdav solution currently has a bug with requiring -X OPTIONS call to be authenticated - which is wrong).
Nevertheless, I could be able to “work around” this bug with using nodes require("https"). But again, I am unable to include / load this into the main module and would need to wait for direct feedback of the node_helper on “just in time encoding”.

Am I going into a completly wrong direction here? Any suggestions how to handle this?

[sdetweil]

@skuethe you can use a web request to get the file. if you look at the MMM-ImagePhotos module, you will see it sets up an express server route/path in the node_helper, which the front end uses in a request, to get the picture data

so this is the front end calling the back end thru request.

there is no sync capability between front and back, only async

[skuethe]

@sdetweil
Thank you for the hint to MMM-ImagesPhotos, I didn’t stumble on that until now.

I will take a shot at it with the express server route. Thanks!

[sdetweil]

@skuethe np… but you could have done the same with socketnotifications… altho more separated

[sdetweil]

@skuethe so the request to get the data is done by the browser, and u don’t have to do anything special in getDom()

specify the url (pointing to the node_helper path and it gets the data and does the encoding and returns it to the browser