Executing python in js - troublshooting/development

[beejay22]

Thanks for the insightful reply! I changed my node_helper to what you posted:

const spawn = require("child_process").spawn
var NodeHelper = require("node_helper")

module.exports = NodeHelper.create({
  socketNotificationReceived: function(notification, payload) {
    switch(notification) {
      case "GIVE_ME_DATA":
        console.log("received notification")
        this.job()
        break
    }
  },
  job: function(){
    var process = spawn("/home/pi/Desktop/realTester.py")
    process.stdout.on("data", (data)=>{
      console.log(data.toString())    
      this.sendSocketNotification("HERE_IS_DATA", data.toString())
    })
  }
})

However, when I changed it I got an error message in the terminal which hasn’t happened before. Error message in terminal:

[2020-08-23 22:36:26.161] [ERROR]  Error: spawn /home/pi/Desktop/testing.py EACCES
    at Process.ChildProcess._handle.onexit (internal/child_process.js:264:19)
    at onErrorNT (internal/child_process.js:456:16)
    at processTicksAndRejections (internal/process/task_queues.js:84:9) {
  errno: 'EACCES',
  code: 'EACCES',
  syscall: 'spawn /home/pi/Desktop/realTester.py',
  path: '/home/pi/Desktop/realTester.py',
  spawnargs: []
}
[2020-08-23 22:36:26.171] [LOG]    MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
[2020-08-23 22:36:26.172] [LOG]    If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
[2020-08-23 22:36:27.069] [LOG]    received notification
[2020-08-23 22:36:27.094] [LOG]    Whoops! There was an uncaught exception...

Still getting the same error log in the magic mirror using dev mode though… lastly this is my pyscript, realTester.py:

from bs4 import BeautifulSoup                                             
import requests                                                           

url=requests.get("https://www.whitehouse.gov/presidential-actions/")      
src=url.content                                                           
soup = BeautifulSoup(src, 'lxml')
for h2_tag in soup.findAll('h2'):
    a_tag = h2_tag.find('a')
    print(a_tag.string)

[sdetweil]

@beejay22 sorry

2more things.

1. u need to add the 1st line to the script to tell the system what processor to use

#!/usr/bin/python

2. u need to make the script executable

chmod +x /home/pi/Desktop/realTester.py

[beejay22]

Okay so now my pycript looks like this:

#!/usr/bin/python
from bs4 import BeautifulSoup                                             
import requests                                                           

url=requests.get("https://www.whitehouse.gov/presidential-actions/")      
src=url.content                                                           
soup = BeautifulSoup(src, 'lxml')
for h2_tag in soup.findAll('h2'):
    a_tag = h2_tag.find('a')
    print(a_tag.string)

If I’m my understanding correctly, adding this shebang line lets shell know that my src code is a script, the type of interpreter we want to use and the path of said interpreter…

Also am I suppose to execute chmod +x /home/pi/Desktop/realTester.py in shell? If so how and when… sorry I’m still trying to understand how exactly to do all of this

[sdetweil]

@beejay22 yes, you need to execute the chmod command once, to mark the python script as executable. do this on a terminal window, or ssh session window

ANY file in Linux can be executable.

if you try to spawn/exec a file which does not have the execute permission set, u get the access error. it is not even opened.

once the bit is set, then it is searched for the shebang, if present the identified file is exec’d and the rest of the parms are passed on.

[beejay22]

Hmmm I did this in the terminal:
$ chmod +x /home/pi/Desktop/realTester.py

Nothing changed… still the same error log message… I think there’s something wrong with my custom’s css file since it says it’s not loading… in my css folder there’s only a main.css file there

[sdetweil]

@beejay22 so, can u run the script from the terminal like before

python /home/pi/Desktop/realTester.py
and then /home/pi/Desktop/realTester.py

both should work, produce the same results
that $ wasn’t part of the chmod

[beejay22]

I didn’t add the $ to the command, I just said that in my post for clarity lol. I typed just this in the command line:
chmod +x /home/pi/Desktop/realTester.py

And yes I can run my script in the terminal with “python3 /home/pi/Desktop/realTester.py” but not with “/home/pi/Desktop/realTester.py” because the module I’m using in my script includes bs4 which is available with python3 I believe so I have to say python3

I also tried ‘touch ~/MagicMirror/css/custom.css’, still showing same error log

[sdetweil]

@beejay22 u can run python3 as the shebang. I was just giving example

your node_helper code was spawning python, not python3. I just copied what u did. u can put python3 in the spawn as well

custom.css, doesn’t matter, it’s empty so no harm

[beejay22]

Tried this (all I changed was “python” to “python3”):

const spawn = require("child_process").spawn
var NodeHelper = require("node_helper")

module.exports = NodeHelper.create({
  socketNotificationReceived: function(notification, payload) {
    switch(notification) {
      case "GIVE_ME_DATA":
        this.job()
        break
    }
  },
  job: function() {
    var process = spawn("python3", ["/home/pi/Desktop/realTester.py"])
    process.stdout.on("data", (data)=>{
      console.log(data)
      var result = String.fromCharCode.apply(null, new Uint16Array((data)))
      this.sendSocketNotification("HERE_IS_DATA", data)
    })
  }
})

And also changed pyscript to this (just added python3 to the shebang):

#!/usr/bin/python3
from bs4 import BeautifulSoup                                             
import requests                                                           

url=requests.get("https://www.whitehouse.gov/presidential-actions/")      
src=url.content                                                           
soup = BeautifulSoup(src, 'lxml')
for h2_tag in soup.findAll('h2'):
    a_tag = h2_tag.find('a')
    print(a_tag.string)

Also tested “python” again (not python3) in my node_helper along with the updated shebang as shown above

If custom.css file doesn’t matter I wonder why it’s giving this error… Have no clue what’s going on now lol

[sdetweil]

@beejay22 u need the fix for the data conversion… as well…

custom.css cause its a file t=hat MM force loads, and while the browser is unhappy (thus the error), the MM runtime doesn’t care WHY there was no data, missing file, empty file, don’t care there were no css updates…

i don’t do python so I can’t help much…

but you gotta put in the fixes I gave you for data…

  process.stdout.on("data", (data)=>{
      console.log(data.toString()) // debugging, log string     
      //var result = String.fromCharCode.apply(null, new Uint16Array((data)))
      this.sendSocketNotification("HERE_IS_DATA", data.toString())   // pass the string, module expects string
    })

[beejay22]

Sorry I copied the wrong node_helper.js file in my last post. This is how it looks like now:

//node_helper.js
const spawn = require("child_process").spawn
var NodeHelper = require("node_helper")

module.exports = NodeHelper.create({
  socketNotificationReceived: function(notification, payload) {
    switch(notification) {
      case "GIVE_ME_DATA":
        console.log("received notification")
        this.job()
        break
    }
  },
  job: function() {
    var process = spawn("python3", ["/home/pi/Desktop/realTester.py"])
    process.stdout.on("data", (data)=>{
      console.log(data.toString())    
      this.sendSocketNotification("HERE_IS_DATA", data.toString()) 
    })
  }
})

I was trying to test if I can at least print something to magic mirror without a pyscript… node_helper looks like this:

//node_helper.js
const spawn = require("child_process").spawn
var NodeHelper = require("node_helper")

module.exports = NodeHelper.create({
  socketNotificationReceived: function(notification, payload) {
    switch(notification) {
      case "GIVE_ME_DATA":
        console.log("received notification")
        this.job()
        break
    }
  },
  job: function() {
    var process = spawn("python3", ["/home/pi/Desktop/test.py"])
    process.stdout.on("data", (data)=>{
      data= "Hello World";
      console.log(data.toString())   
      this.sendSocketNotification("HERE_IS_DATA", data.toString())  
    })
  }
})

Hello World appears on MagicMirror but the test file that’s provided in the path (the test file just has: print(“this works”) does not so I’m not sure why I need to provide var process spawn if Hello World prints by itself… Anyways, I’m aware that there’s other ways to execute the child process however I’m not familiar with it. Please correct me if I’m wrong but can exec work in this situation as well? (pyscipt just prints out the latest titles when magic mirror restarts). I’m trying anything and everything at this point

And I added the default and start function to my core module although I don’t think I have to use it. This is how it looks like now:

/// MMM-Whitehouse.js - Core Module

Module.register("MMM-Whitehouse", {
  defaults: {},
  start: function () {},
  getDom: function() {
    var e = document.createElement("div")
    e.id = "DISPLAY"
    return e
  },
  notificationReceived: function(notification, payload, sender) {
    switch(notification) {
      case "DOM_OBJECTS_CREATED":
        var timer = setInterval(()=>{
          this.sendSocketNotification("GIVE_ME_DATA")
        }, 1000)
        break
    }
  },
  socketNotificationReceived: function(notification, payload) {
    switch(notification) {
      case "HERE_IS_DATA":
        var e = document.getElementById("DISPLAY")
        e.innerHTML = payload
        break
    }
  },
})

[sdetweil]

@beejay22 I don’t understand why u have trouble.

the 3 files I posted work together as u designed.

does your spawn(‘/use/bin/python’, [execname]) work to return the data.
console log to see

spawn and exec have different methods of returning the data. see the doc

Google search nodejs process spawn

[beejay22]

Did you try it with the pyscript? Or are you talking about it the core module, node_helper and config?

And okay

[sdetweil]

@beejay22 I am only working on inside MagicMirror

the spawn on.data() gets called when the pgm launched outputs to stdout. my print hello

of course I made sure the python script worked both ways, before showing that

[beejay22]

Okay. And yes it does return the data as output I believe

[sdetweil]

@beejay22 do you see this output from the node_helper in the terminal window where you do npm start?

console.log(data.toString())

from your python script