MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    help developing an outofmilk.com module

    Scheduled Pinned Locked Moved Development
    8 Posts 4 Posters 2.1k Views 4 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      mchrdk
      last edited by

      Hi,
      I´m planning on migrating a windows based home-control-screen to magicmirror and so far i have found all the modules i need, except for a module for the website out of milk.

      Since my old solution already has the info on how to get the data from the website, i dont think i would be a big problem.

      The procedure right now is to call the mainsite https://outofmilk.com/ to get a session, and then call the /loginsite with post data containing credentials.
      This request returns a logincookie that you can use to get a shoppinglist from the api.

      im mostly a devops guy using powershell and other basic scripting languages, which makes the whole framework a big bite to take in at once.

      Do you have any good pointers of modules i can look into that uses the same way of getting data?

      i can code i already have in powershell if anyone is interrested.

      1 Reply Last reply Reply Quote 0
      • gauravsaccG Offline
        gauravsacc Module Developer
        last edited by gauravsacc

        Reverse engineer ;)
        Looks like they have data exposed via APIs. See screenshot from chrome->devtools->network log
        You just need to login and set the cookie and make the API calls.
        0_1532615531053_Screen Shot 2018-07-26 at 10.28.39 AM.png

        1 Reply Last reply Reply Quote 0
        • M Offline
          mchrdk
          last edited by

          I have all of that done already, im using the data in another application, done in powershell, i just need a bit of beginners guidance on how to get a magic mirror module to access the data, as i have never done anything in js before.

          1 Reply Last reply Reply Quote 0
          • M Offline
            mchrdk
            last edited by

            #quick mockup in the entirely wrong language :)

            $csrftoken = (($firstrequest.InputFields|where {$_.name -eq “csrfmiddlewaretoken”}).value)[0]
            $username = “insertemailhere”
            $password = “insertpasswordhere”
            $listid = “find listhere”
            $body = “csrfmiddlewaretoken=$csrftoken&login-email=$username&login-password=$password&btnLogin=Submit”
            $headers = @{“Cookie”=“csrftoken=$csrftoken”; “Origin”=“https://www.outofmilk.com”; “Accept”=“application/json, text/javascript, /; q=0.01”; “Referer”=“https://www.outofmilk.com/ShoppingList.aspx?signin=1”; “X-CSRFToken”=“$csrftoken”; “X-Requested-With”=“XMLHttpRequest”; “DNT”=“1”}
            $secondrequest = Invoke-WebRequest -WebSession $Session -Uri “https://www.outofmilk.com/Login” -Method “POST” -Headers $headers -ContentType “application/x-www-form-urlencoded” -Body $Body
            $body = “{"listID”:$listid,"listType":"shopping"}"
            $thirdrequest = Invoke-WebRequest -WebSession $Session -Uri “https://www.outofmilk.com/Services/GenericService.asmx/GetItems” -Method “POST” -Headers $headers -ContentType “application/json; charset=UTF-8” -Body $body

            ? 2 Replies Last reply Reply Quote 0
            • ? Offline
              A Former User @mchrdk
              last edited by

              @mchrdk I didn’t look inside deeply, but how about using ”curl”?

              1 Reply Last reply Reply Quote 0
              • ? Offline
                A Former User @mchrdk
                last edited by A Former User

                @mchrdk or you can handle request and response with ’express’ and ’http’ or ’request’ node modules.

                1 Reply Last reply Reply Quote 0
                • M Offline
                  mchrdk
                  last edited by

                  okay, i see - i have been reading up on the http and request modules now.

                  it should be possible for me to figure out how to get it to output the data directly in the console, and then i can tackle the presentation part of the module later.

                  i have been looking at a lot of sourcecode for other mm modules and i cant seem to find any other projects that handle multiple requests like this directly, am i mistaken?

                  1 Reply Last reply Reply Quote 0
                  • D Offline
                    doubleT Module Developer
                    last edited by

                    Uh, interesting topic. I like API calls. :D Back when I started, my main issue was understanding the node_helper.js – and that it was neccessary.
                    I’m still struggling to make sense of your token complexity. Apart from that, your files would be something like this:

                    MMM-OutOfMilk.js:

                    Module.register("MMM-OutOfMilk",{
                        defaults: {
                            // your defaults
                        },
                    
                        start: function() {
                            var payload = "start"; // example
                            this.sendSocketNotification("GetMyShoppingList", payload);
                        },
                    
                        socketNotificationReceived: function(notification, shoppingList) {
                            if (notification === "ShoppingListRecieved") {
                                this.handeShoppingList(shoppingList);
                            }
                        },
                    
                        handleShoppingList: function(shoppingList) {
                            // do something with your shopping list
                        }
                    }
                    

                    node_helper.js:

                    var NodeHelper = require("node_helper");
                    var request = require("request"); // needed? see below
                    var fs = require("fs"); // needed? see below
                    var timer = 0; // needed? see below "I added a timer"
                    var token;
                    
                    module.exports = NodeHelper.create({
                        start: function() {
                        },
                        socketNotificationReceived: function(notification, payload) {
                            if (notification === "GetMyShoppingList") {
                                // start the logic to recieve the shopping list
                                // get token?
                                // make call
                                this.getShoppingList(payload);
                            }
                        },
                        getToken: function(payload) {
                            // however you get your token
                            getShoppingList(payload);
                        },
                        getShoppingList: function(payload) {
                            var self = this;
                            var source = ; // your source URL
                            if (token) {
                            	request({
                    	            url: source,
                    	            json: true
                    	        }, function (error, response, body) {
                    	            if (!error && response.statusCode === 200) {
                    	                self.sendSocketNotification("ShoppingListRecieved", body);
                    	            }
                    	            else {
                    	            	// check error if it's a token issue
                    	            	// if so, get new token
                    	            	self.getToken(payload);
                    	            }
                    	        })
                            }    
                        }
                    }
                    

                    From another project I know that sometimes a token can be used for a while. In that case I’d directly call the getShoppingList and only on an error that read “wrong token” or something like that, I’d call getToken which saves the token to a global variable and calls back getShoppingList to try again. That way you don’t have to get a new token for every call.

                    Alternatively?

                        getShoppingList: function(token) {
                            var self = this;
                            var source = ; // your source URL
                            var rawdata = fs.readFileSync(source);
                            var history =  JSON.parse(rawdata);
                            self.sendSocketNotification("ShoppingListRecieved", history);
                        }
                    

                    I’d also use a timer to request data on the first call and after that automatically only every full hour:
                    So, in the node_helper.js, in the socketNotificationReceived I don’t forward the payload to this.updateTimer(payload) instead of getToken or getShoppingList to do this:

                        updateTimer: function(payload) {
                            var self = this;
                            var d = new Date();
                            var h = d.getHours();
                            var min = d.getMinutes();
                            var sec = d.getSeconds();
                            if (timer === 0) {
                                timer ++; // prevent unnecessary timer by double calls
                                this.getShoppingList(payload);
                    
                                if((min == '00') && (sec == '00')){
                    //              console.log(h + ":" + min + ":" + sec + " - update and wait 1 hour");
                    //              console.log("restart timer");
                                    setTimeout(() => {
                                        this.clearTimer(payload);
                                    }, 60*1000*60); // 60 sec * 1000 ms = 1 min * 60 = 1 hour
                                } else {
                    //              console.log(h + ":" + min + ":" + sec + " - update waits for " + (59-min) + " min and " + (60-sec) + "sec");
                    //              console.log("restart timer");
                                    setTimeout(() => {
                                        this.clearTimer(payload);
                                    }, ((60-sec)*1000)+(60*1000*(59-min)));
                                }
                            }
                            else {
                    //          console.log("timer already running, data displayed outside of timer run");
                                this.getShoppingList(payload);
                            }
                        },
                    
                        clearTimer: function(payload) {
                            timer --;
                            this.updateTimer(payload);
                        }
                    

                    For one of my modules I couldn’t use fs.readFile... or request because of CORS issues. In that case I had to use a php proxy.
                    I’d strongly suggest trying to solve this with fs.readFile... or request but just for the sake of completeness and maybe to help you understand:

                    In order to use PHP within nodeJS via Child Process Dependency you have to have PHP installed and in your PATH!
                    You can read how I used it, here: https://forum.magicmirror.builders/topic/5830/call-api-no-cors-used-to-do-it-with-php-proxy/9

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    • First post
                      Last post
                    Enjoying MagicMirror? Please consider a donation!
                    MagicMirror created by Michael Teeuw.
                    Forum managed by Sam, technical setup by Karsten.
                    This forum is using NodeBB as its core | Contributors
                    Contact | Privacy Policy