call API (no CORS), used to do it with php proxy

doubleT

Thank you. I got a bit further. It took me a while to figure out that I had to close the session and run npm start dev again for node_helper to work.

I found that strawberry 3.141’s answer in that topic helped me a lot to get node_helper running.

I got to the module to notificate node_helper and I also got node_helper to talk back. What I’m having trouble with is the request():

request({
	url: "modules/station_price/proxytest.txt",
	method: 'GET',
}, function (error, response, body) {
	if (!error && response.statusCode == 200) {
		self.sendSocketNotification("MSG", body);
	}
	else {
		self.sendSocketNotification("MSG", error + " // " + response);
	}
});

Error: Invalid URI "proxytest.txt" // undefined

I tried a few variations for the URI - what’s the path? Or maybe I’m just not seeing some basic mistake here.
this.data.path obviously isn’t working within node_helper.

And is there a documentation for request()? Can/should I use dataType: “json”?

I got this all running on my old mirror and now I just feel like running into a wall over and over again.

yawns

@doubleT
If you want to you can send some more details in the chat or by email (yawns@live.de) and I’ll have a look at it.

strawberry 3.141

@doubleT you can only do requests to servers, so if you want to access your file like this, you have to name the uri similar to localhost:8080/station_price/public/proxytest.txt and piut the file in a public folder, as i remember the public folder in a module directory gets exposed by default to the express server of the magicmirror. But if you want to read a local file, i would rather use the fs module that comes natively with nodejs than request. the filesystem module allows you to read and write files on your harddrive https://nodejs.org/dist/latest-v8.x/docs/api/fs.html. Be aware of that there are synchronous and asynchronous functions to read a file

doubleT

Thank you, that sounds like it might be what I need, I’ll check that out.
Hope this works with a php file that grabs and forwards the content of the API.

strawberry 3.141

@doubleT the fs module will just read the content of the file, if it needs to be executed you need a different solution.

doubleT

That’s what I thought.

I got it done thanks to this blog entry about calling PHP within node.js via Child Process Dependency. Actually, now that I know about this, it’s quite easy, you just have to have PHP installed and in your PATH (that took me a while …).

(relevant part of) node_helper.js

getData: function(payload) {
     var self = this;
     var runner = require("child_process");
     var proxy = "modules/my_module/proxy.php";
     var source = "http://apisource.de/api/getPrices.php?id=12300123";
     runner.exec("php " + proxy + " " + source, function(err, data, stderr) {
          var json = JSON.parse(data);
          self.sendSocketNotification("MSG", json);
     });
}

proxy.php is nearly unchanged from my first post: (php tags not showing up)

$params = explode(",", $argv[1]);
$file = file_get_contents($params[0], true);
echo $file;

A node child process is started, it uses PHP (has to be in the env. PATH) to open the file it is handed (proxy) and attaches the parameter “source” to that call. The proxy.php gets the contents of “source” and echo’s it.
The child process receives the echo’ed data, I parse it to JSON and send that back to the module.

It’s working, but let me know if you see any flaws or anything I could improve. Thanks a lot.

Oh, and no, there’s no way around that “file_get_contents => echo proxy” atm.

Torben

strawberry 3.141

@doubleT personally i wouldn’t take php in the middle just to request the webiste, you could do that in the nodehelper directly

doubleT

The API doesn’t allow CORS, so JS can’t get to it.

strawberry 3.141

@doubleT if you do it in the node_helper with the request module you shouldn’T have an issue with cors. Try something like:

var request = require('request'); // at the top of the file

...

getData: function(payload) {
     request('http://apisource.de/api/getPrices.php?id=12300123', (error, response, body) => {
            if (response.statusCode === 200) {
                const parsedBody = JSON.parse(body);
                this.sendSocketNotification("MSG", parsedBody);
            } else {
                console.log(`Error getting price data ${response.statusCode}`);
            }
        });
}

doubleT

Sorry, but that doesn’t work. (It throws response.statusCode = 403, forbidden.) And it’s not surprising. As I said, the API source server doesn’t allow CORS and is not serving JSONP. So JavaScript calls are blocked.

There are a lot of fine tools for specific jobs, XMLHttpRequest, fetch, request, fs, …, and they work if CORS is set up correctly on the server, allowing you access, or it’s giving you JSONP to handle, but PHP’s file_get_contents is the hammer in your toolbox. If everything else fails, you still can throw this at your problem (provided you have allow_url_fopen).

And I know, it’s not always wise to use (or even throw) a hammer, access might be forbidden (to scripts) for a reason. But if you can read it in your browser, PHP can read, stringify and proxy it to your JS.