setInterval for multiple API requests

[Privacywonk]

Background:

My module pulls from three data sources: NOAA, Wunderground, and Magicseaweed to render the UI. When I initially made the module, I had 3x setInterval functions loaded in node_helper. This did not work well…the setInterval function was firing haphazardly, resulting in 1000s of calls being made to the APIs.

My initial solution for this was to move the update function to the main module (MMM-Surf.js). This produced the desired effect BUT, I later realized, creates an asynchronous update process across multiple clients vs. synchronizing it in node_helper.

Question(s):

• Is it worthwhile to move the update functionality back into node_helper or if it’s working, leave well enough alone?
• if I do move it back, recommendations for how to structure it to avoid the random firing across the three API calls?

Current solution in MMM-Surf:

        scheduleUpdate: function() {
                var nextload = this.config.updateInterval;
                var self = this;
                this.updateTimer = setInterval(function() {
                        if (self.config.debug === 1) { Log.info(moment().format('YYYY-MM-DDTHH:mm:ss.SSSZZ') + "  UPDATE: scheduledUpdate called fetching new data: " + self.config.updateInterval); }
                        self.getNOAA(self);
                        self.getWunder(self);
                        self.getMagicseaweed(self);
                }, nextload);
        },

Legacy node_helper structure (repeated 3x times):

    fetchMagicseaweedData: function() {
        var self = this;
        this.MAGICfetcherRunning = true;

        //block of code removed for brevity

        setInterval(function() {
                    self.fetchMagicseaweedData();
                }, self.config.updateInterval);

            } // end request(function())
        ); //end Magicseaweed request
        this.MAGICfetcherRunning = false;
    }, //end Magicseaweed function

[strawberry 3.141]

@Privacywonk in your old node_helper code you had a recursion. Every time you called the function fetchMagicseaweedData you set a new interval, but the old one(s) was still running. It’s basically like this.

First execution: Interval
Second execution: Interval -- Interval
Third execution: Interval -- Interval -- Interval -- Interval
...

so your intervals are created in O(n²), that explains why you end up with a couple of thousands after a while.

You have multiple options to fix this:

• change setInterval to setTimeout (which gets only executed once)
• move the creation of the interval outside of the function and ensure it will be only called once in the lifetime of your module

[Privacywonk]

Big O strikes again! Thank you for the explanation.

@strawberry-3.141 any guidance on location / structure of that function within node_helper? Create something like I did for MMM-Surf module within node_helper? Tack on a simple set interval @ the end of the NodeHelper.create {} stanza?

Current node_helper code can be found at https://github.com/Privacywonk/MMM-Surf/blob/master/node_helper.js for reference.

[strawberry 3.141]

@Privacywonk i would do it like this, copy your schedule update to node helper

• send a socket notification from client to nodehelper in the start method to initialise the socket connection (e.g. the config)
  this.sendSocketNotification("CONFIG", this.config);
• in the nodehelper when you receive the notification, run schedule update

socketNotificationReceived: function(notification, payload) {
  if (notification === 'CONFIG') {
    this.config = payload;
    if (this.started !== true) {
      this.started = true;
      this.scheduleUpdate();            
    }
  }
}

the started flag makes sure it gets called only once, even if you’re connecting multiple clients.

[Privacywonk]

@strawberry-3.141 - thank you for the approach. This is working out beautifully.