MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    Executing python in js - troublshooting/development

    Scheduled Pinned Locked Moved Troubleshooting
    18 Posts 2 Posters 1.4k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • beejay22B Offline
      beejay22
      last edited by

      Hi. I’m very new to coding and I have a pyscript that simply prints out the latest titles from a website using BeautifulSoup. However, when I launch magic mirror it’s just a black screen. I deleted all other modules besides my own in the config file so that it’s just my module that runs. I’ve done ample research and can’t seem to figure out what’s going on (updated mm software, reinstalled dependencies in the css directory just in case it wasn’t installed correctly, checked to see node was up to date - v10.21.0, etc.). I’m starting to think it may be my pyscript but it runs correctly in the console (I can attach my pyscript if needed). I’m not sure what the problem is so I appreciate any and all help!

      Core module: MMM-Whitehouse

      Module.register("MMM-Whitehouse", {
        getDom: function() {
          var e = document.createElement("div")
          e.id = "DISPLAY"
          return e
        },
        notificationReceived: function(notification, payload, sender) {
          switch(notification) {
            case "DOM_OBJECTS_CREATED":
              var timer = setInterval(()=>{
                this.sendSocketNotification("GIVE_ME_DATA")
              }, 1000)
              break
          }
        },
        socketNotificationReceived: function(notification, payload) {
          switch(notification) {
            case "HERE_IS_DATA":
              var e = document.getElementById("DISPLAY")
              e.innerHTML = payload
              break
          }
        },
      })
      

      node_helper.js

      const spawn = require("child_process").spawn
      var NodeHelper = require("node_helper")
      
      module.exports = NodeHelper.create({
        socketNotificationReceived: function(notification, payload) {
          switch(notification) {
            case "GIVE_ME_DATA":
              this.job()
              break
          }
        },
        job: function() {
          var process = spawn("python", ["/home/pi/Desktop/realTester.py"])
          process.stdout.on("data", (data)=>{
            console.log(data)
            var result = String.fromCharCode.apply(null, new Uint16Array((data)))
            this.sendSocketNotification("HERE_IS_DATA", data)
          })
        }
      })
      

      config.js

      var config = {
      	address: "localhost", // Address to listen on, can be:
      	                      // - "localhost", "127.0.0.1", "::1" to listen on loopback interface
      	                      // - another specific IPv4/6 to listen on a specific interface
      	                      // - "0.0.0.0", "::" to listen on any interface
      	                      // Default, when address config is left out or empty, is "localhost"
      	port: 8080,
      	ipWhitelist: ["127.0.0.1", "::ffff:127.0.0.1", "::1"], // Set [] to allow all IP addresses
      	                                                       // or add a specific IPv4 of 192.168.1.5 :
      	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"],
      	                                                       // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format :
      	                                                       // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"],
      
      	useHttps: false, 		// Support HTTPS or not, default "false" will use HTTP
      	httpsPrivateKey: "", 	// HTTPS private key path, only require when useHttps is true
      	httpsCertificate: "", 	// HTTPS Certificate path, only require when useHttps is true
      
      	language: "en",
      	timeFormat: 24,
      	units: "metric",
      	// serverOnly:  true/false/"local" ,
      			     // local for armv6l processors, default
      			     //   starts serveronly and then starts chrome browser
      			     // false, default for all  NON-armv6l devices
      			     // true, force serveronly mode, because you want to.. no UI on this device
      
      	modules: [
      		{
       			module: "MMM-Whitehouse",
       			position: "top_left",
        			config: {}
      		},
      	]
      
      };
      
      /*************** DO NOT EDIT THE LINE BELOW ***************/
      if (typeof module !== "undefined") {module.exports = config;}
      

      Error message (from dev mode):

      Initializing MagicMirror.
      translator.js:202 Loading core translation file: translations/en.json
      translator.js:225 Loading core translation fallback file: translations/en.json
      loader.js:179 Load script: modules/MMM-Whitehouse//MMM-Whitehouse.js
      module.js:479 Module registered: MMM-Whitehouse
      loader.js:150 Bootstrapping module: MMM-Whitehouse
      loader.js:155 Scripts loaded for: MMM-Whitehouse
      loader.js:157 Styles loaded for: MMM-Whitehouse
      loader.js:159 Translations loaded for: MMM-Whitehouse
      loader.js:194 Load stylesheet: css/custom.css
      (index):1 Refused to apply style from 'http://localhost:8080/css/custom.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
      loader.js:203 Error on loading stylesheet: css/custom.css
      module.js:46 Starting module: MMM-Whitehouse
      main.js:474 All modules started!
      /home/pi/MagicMirror…rity-warnings.js:95 Electron Security Warning (Insecure Resources) This renderer process loads resources using insecure
        protocols.This exposes users of this app to unnecessary security risks.
        Consider loading the following resources over HTTPS or FTPS. 
       - http://localhost:8080/css/main.css
      - http://localhost:8080/fonts/roboto.css
      - http://localhost:8080/socket.io/socket.io.js
      - http://localhost:8080/vendor/node_modules/nunjucks/browser/nunjucks.min.js
      - http://localhost:8080/js/defaults.js
      - http://localhost:8080/config/config.js
      - http://localhost:8080/vendor/vendor.js
      - http://localhost:8080/modules/default/defaultmodules.js
      - http://localhost:8080/js/logger.js
      - http://localhost:8080/translations/translations.js
      - http://localhost:8080/js/translator.js
      - http://localhost:8080/js/class.js
      - http://localhost:8080/js/module.js
      - http://localhost:8080/js/loader.js
      - http://localhost:8080/js/socketclient.js
      - http://localhost:8080/js/main.js
      - http://localhost:8080/fonts/node_modules/roboto-fontface/fonts/roboto-condensed/Roboto-Condensed-Regular.woff2
      - http://localhost:8080/translations/en.json
      - http://localhost:8080/translations/en.json
      - http://localhost:8080/modules/MMM-Whitehouse//MMM-Whitehouse.js
        
       
      For more information and help, consult
      https://electronjs.org/docs/tutorial/security.
       This warning will not show up
      once the app is packaged.
      /home/pi/MagicMirror…ity-warnings.js:145 Electron Security Warning (Insecure Content-Security-Policy) This renderer process has either no Content Security
          Policy set or a policy with "unsafe-eval" enabled. This exposes users of
          this app to unnecessary security risks.
       
      For more information and help, consult
      https://electronjs.org/docs/tutorial/security.
       This warning will not show up
      once the app is packaged.
      (index):1 Refused to apply style from 'http://localhost:8080/css/custom.css' because its MIME type ('text/html') is not a supported stylesheet MIME type, and strict MIME checking is enabled.
      
      S 1 Reply Last reply Reply Quote 0
      • S Away
        sdetweil @beejay22
        last edited by sdetweil

        @beejay22 I copy/pasted your code, and only see a black screen cause there is nothing to display…

        there is a bug in the node_helper

        const spawn = require("child_process").spawn
        var NodeHelper = require("node_helper")
        
        module.exports = NodeHelper.create({
          socketNotificationReceived: function(notification, payload) {
            switch(notification) {
              case "GIVE_ME_DATA":
                console.log("received notification")
                this.job()
                break
            }
          },
          job: function() {
        
            // type A, you specify the pgm to execute your script HERE
            var process = spawn("python",["/home/sam/Desktop/realTester.py"])
        
            // type B, you specify the pgm to execute your script IN the script.
            var process = spawn("/home/sam/Desktop/realTester.py")
        
            process.stdout.on("data", (data)=>{
              console.log(data)
              var result = String.fromCharCode.apply(null, new Uint16Array((data)))   // < --- you processed data to result
              this.sendSocketNotification("HERE_IS_DATA", data)    // < ---- then sent data, which is still a 'buffer'
            })
          }
        })
        

        i fixed to this

        onst spawn = require("child_process").spawn
        var NodeHelper = require("node_helper")
        
        module.exports = NodeHelper.create({
          socketNotificationReceived: function(notification, payload) {
            switch(notification) {
              case "GIVE_ME_DATA":
                console.log("received notification")
                this.job()
                break
            }
          },
          job: function() {
        
            // type A, you specify the pgm to execute your script HERE
            var process = spawn("python",["/home/sam/Desktop/realTester.py"])
        
            // type B, you specify the pgm to execute your script IN the script.
           //  I prefer this , before u needed two things, python and the script..
           // but if u changed the script to bash, or java, or whatever, then the app would break..
            var process = spawn("/home/sam/Desktop/realTester.py")
        
            process.stdout.on("data", (data)=>{
              console.log(data.toString()) // debugging, log string     
              //var result = String.fromCharCode.apply(null, new Uint16Array((data)))
              this.sendSocketNotification("HERE_IS_DATA", data.toString())   // pass the string, module expects string
            })
          }
        })
        

        the script for example is

        #!/usr/bin/python3 
        
        print("hello");
        

        Sam

        How to add modules

        learning how to use browser developers window for css changes

        1 Reply Last reply Reply Quote 0
        • beejay22B Offline
          beejay22
          last edited by

          Thanks for the insightful reply! I changed my node_helper to what you posted:

          const spawn = require("child_process").spawn
          var NodeHelper = require("node_helper")
          
          module.exports = NodeHelper.create({
            socketNotificationReceived: function(notification, payload) {
              switch(notification) {
                case "GIVE_ME_DATA":
                  console.log("received notification")
                  this.job()
                  break
              }
            },
            job: function(){
              var process = spawn("/home/pi/Desktop/realTester.py")
              process.stdout.on("data", (data)=>{
                console.log(data.toString())    
                this.sendSocketNotification("HERE_IS_DATA", data.toString())
              })
            }
          })
          

          However, when I changed it I got an error message in the terminal which hasn’t happened before. Error message in terminal:

          [2020-08-23 22:36:26.161] [ERROR]  Error: spawn /home/pi/Desktop/testing.py EACCES
              at Process.ChildProcess._handle.onexit (internal/child_process.js:264:19)
              at onErrorNT (internal/child_process.js:456:16)
              at processTicksAndRejections (internal/process/task_queues.js:84:9) {
            errno: 'EACCES',
            code: 'EACCES',
            syscall: 'spawn /home/pi/Desktop/realTester.py',
            path: '/home/pi/Desktop/realTester.py',
            spawnargs: []
          }
          [2020-08-23 22:36:26.171] [LOG]    MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
          [2020-08-23 22:36:26.172] [LOG]    If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues
          [2020-08-23 22:36:27.069] [LOG]    received notification
          [2020-08-23 22:36:27.094] [LOG]    Whoops! There was an uncaught exception...
          

          Still getting the same error log in the magic mirror using dev mode though… lastly this is my pyscript, realTester.py:

          from bs4 import BeautifulSoup                                             
          import requests                                                           
          
          url=requests.get("https://www.whitehouse.gov/presidential-actions/")      
          src=url.content                                                           
          soup = BeautifulSoup(src, 'lxml')
          for h2_tag in soup.findAll('h2'):
              a_tag = h2_tag.find('a')
              print(a_tag.string)
          
          
          S 1 Reply Last reply Reply Quote 0
          • S Away
            sdetweil @beejay22
            last edited by sdetweil

            @beejay22 sorry

            2more things.

            1. u need to add the 1st line to the script to tell the system what processor to use
            #!/usr/bin/python
            
            1. u need to make the script executable
            chmod +x /home/pi/Desktop/realTester.py
            

            Sam

            How to add modules

            learning how to use browser developers window for css changes

            1 Reply Last reply Reply Quote 0
            • beejay22B Offline
              beejay22
              last edited by

              Okay so now my pycript looks like this:

              #!/usr/bin/python
              from bs4 import BeautifulSoup                                             
              import requests                                                           
              
              url=requests.get("https://www.whitehouse.gov/presidential-actions/")      
              src=url.content                                                           
              soup = BeautifulSoup(src, 'lxml')
              for h2_tag in soup.findAll('h2'):
                  a_tag = h2_tag.find('a')
                  print(a_tag.string)
              
              

              If I’m my understanding correctly, adding this shebang line lets shell know that my src code is a script, the type of interpreter we want to use and the path of said interpreter…

              Also am I suppose to execute chmod +x /home/pi/Desktop/realTester.py in shell? If so how and when… sorry I’m still trying to understand how exactly to do all of this

              S 1 Reply Last reply Reply Quote 0
              • S Away
                sdetweil @beejay22
                last edited by sdetweil

                @beejay22 yes, you need to execute the chmod command once, to mark the python script as executable. do this on a terminal window, or ssh session window

                ANY file in Linux can be executable.

                if you try to spawn/exec a file which does not have the execute permission set, u get the access error. it is not even opened.

                once the bit is set, then it is searched for the shebang, if present the identified file is exec’d and the rest of the parms are passed on.

                Sam

                How to add modules

                learning how to use browser developers window for css changes

                1 Reply Last reply Reply Quote 0
                • beejay22B Offline
                  beejay22
                  last edited by

                  Hmmm I did this in the terminal:
                  $ chmod +x /home/pi/Desktop/realTester.py

                  Nothing changed… still the same error log message… I think there’s something wrong with my custom’s css file since it says it’s not loading… in my css folder there’s only a main.css file there

                  S 1 Reply Last reply Reply Quote 0
                  • S Away
                    sdetweil @beejay22
                    last edited by sdetweil

                    @beejay22 so, can u run the script from the terminal like before

                    python /home/pi/Desktop/realTester.py
                    and then /home/pi/Desktop/realTester.py

                    both should work, produce the same results
                    that $ wasn’t part of the chmod

                    Sam

                    How to add modules

                    learning how to use browser developers window for css changes

                    1 Reply Last reply Reply Quote 0
                    • beejay22B Offline
                      beejay22
                      last edited by

                      I didn’t add the $ to the command, I just said that in my post for clarity lol. I typed just this in the command line:
                      chmod +x /home/pi/Desktop/realTester.py

                      And yes I can run my script in the terminal with “python3 /home/pi/Desktop/realTester.py” but not with “/home/pi/Desktop/realTester.py” because the module I’m using in my script includes bs4 which is available with python3 I believe so I have to say python3

                      I also tried ‘touch ~/MagicMirror/css/custom.css’, still showing same error log

                      S 1 Reply Last reply Reply Quote 0
                      • S Away
                        sdetweil @beejay22
                        last edited by sdetweil

                        @beejay22 u can run python3 as the shebang. I was just giving example

                        your node_helper code was spawning python, not python3. I just copied what u did. u can put python3 in the spawn as well

                        custom.css, doesn’t matter, it’s empty so no harm

                        Sam

                        How to add modules

                        learning how to use browser developers window for css changes

                        1 Reply Last reply Reply Quote 0
                        • 1
                        • 2
                        • 1 / 2
                        • First post
                          Last post
                        Enjoying MagicMirror? Please consider a donation!
                        MagicMirror created by Michael Teeuw.
                        Forum managed by Sam, technical setup by Karsten.
                        This forum is using NodeBB as its core | Contributors
                        Contact | Privacy Policy