• Recent
  • Tags
  • Unsolved
  • Solved
  • MagicMirror² Repository
  • Documentation
  • 3rd-Party-Modules
  • Donate
  • Discord
  • Register
  • Login
MagicMirror Forum
  • Recent
  • Tags
  • Unsolved
  • Solved
  • MagicMirror² Repository
  • Documentation
  • 3rd-Party-Modules
  • Donate
  • Discord
  • Register
  • Login
A New Chapter for MagicMirror: The Community Takes the Lead
Read the statement by Michael Teeuw here.

MMM-voice

Scheduled Pinned Locked Moved Utilities
voice control
329 Posts 55 Posters 847.0k Views 50 Watching
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • D Offline
    dmwilsonkc
    last edited by dmwilsonkc Apr 18, 2018, 6:45 PM Apr 18, 2018, 6:36 PM

    @strawberry-3.141 So I guess I kind of understood what is happening with the MMM-Mobile module of yours. I am very new to all of this and have only been dabbling in writing simple code up to now. It is much easier to build off of existing code that I know already works. If I could figure out a way to continuing to use your MMM-voice module and Mycroft at the same time, I would probably just do that, but as you pointed out both processes can’t use the microphone at the same time. Since Mycroft skills seem pretty straight forward to create, I thought eh… Why not attempt it.

    I can create a Mycoft skill that will send ‘HIDE_CLOCK’ for example to socket.io. What happens when the Websocket-client for Mycroft connects to socket.io and sends ‘HIDE_CLOCK’? How would MagicMirror handle that?
    Keep in mind, I have already added Mykle’s modifications from Hello-Lucy.

    That’s part of what I can’t wrap my head around.

    Thank you for responding and helping me!

    S 1 Reply Last reply Apr 18, 2018, 9:14 PM Reply Quote 0
    • S Offline
      strawberry 3.141 Project Sponsor Module Developer @dmwilsonkc
      last edited by Apr 18, 2018, 9:14 PM

      @dmwilsonkc the module has a socket like https://github.com/MichMich/MagicMirror/blob/master/js/socketclient.js#L11, basically its a channel with the module name, if you then send messages, the module doesn#t differentiate if its from mycroft or node_helper, as it isnt aware of multiple entities.

      Please create a github issue if you need help, so I can keep track

      1 Reply Last reply Reply Quote 0
      • D Offline
        dmwilsonkc
        last edited by Apr 19, 2018, 5:04 AM

        @strawberry-3.141 Oh! Wow! That was awesome! You opened my eyes. I really am starting to understand better thanks to your incite. That makes perfect sense to treat Mycroft more like a remote-control than a voice-control because it handles the STT and can send socket messages to the MagicMirror. The MMM-Mycroft module would need to be very similar in function to your MMM-mobile module.

        For example it could have a number of ways to control the MagicMirror:

                socket.on('SHOW_MODULES', () => {
                    console.log(`${this.name}: Showing modules!`);
                    this.sendSocketNotification('SHOW_MODULES');
                });
        
                socket.on('HIDE_MODULES', () => {
                    console.log(`${this.name}: Hiding modules!`);
                    this.sendSocketNotification('HIDE_MODULES');
                });
        

        Just as an example to handle the messages sent to the socket.

        If I’m reading your code right, you’re using a qr image to handle security. I hadn’t really thought about the need for security with something like this, but I guess that really should be a consideration. In your opinion, would requiring some kind of user password/key set in the config for both MMM-Mycroft module and the Mycroft MM skill be sufficient?

        S 1 Reply Last reply Apr 19, 2018, 6:07 AM Reply Quote 0
        • S Offline
          strawberry 3.141 Project Sponsor Module Developer @dmwilsonkc
          last edited by Apr 19, 2018, 6:07 AM

          @dmwilsonkc I did MMM-Mobile for my bachelor thesis to connect to the MagicMirror via mobile and be able to control and change config of it, the qr code was meant for pairing the devices, so not everyone is able to hook into my socket channel and send commands. But you have to be aware of the differences between socket notifications and normal notifications. socket notifications describe only the communication between your module and his node_helper (or in my case the mobile app, because I extended the socket server), but this gives you no communication with other modules like the clock. Therefore you need normal notifications which describe the communication between modules and the core MM. Be aware of that those notifications can only be sent via the module not the node_helper.

          My workflow was like this. Node_helper extends socket server. Mobile send command to node_helper, then the node_helper sends it to the module and the module broadcasts it to all other modules.

          https://github.com/MichMich/MagicMirror/blob/master/modules/README.md#thissendnotificationnotification-payload
          https://github.com/MichMich/MagicMirror/blob/master/modules/README.md#thissendsocketnotificationnotification-payload

          Please create a github issue if you need help, so I can keep track

          1 Reply Last reply Reply Quote 0
          • D Offline
            dmwilsonkc
            last edited by dmwilsonkc Apr 19, 2018, 4:45 PM Apr 19, 2018, 4:43 PM

            @strawberry-3.141 Thanks for the clarification. The code that I copied came from the node_helper as you are probably aware.
            One more question for you concerning extending the socket with node_helper: The code you have for the MMM-mobile module is written like:

            appSocket() {
            const namespace =
            ${this.name}/app;

                this.io.of(namespace).use((socket, next) => {
                    const hash = crypto.createHash('sha256').update(socket.handshake.query.token).digest('base64');
                    if (this.mobile.user && this.mobile.user === hash) {
                        console.log(`${this.name}: Access granted!`);
                        next();
                    } else {
                        console.log(`${this.name}: Authentication failed!`);
                        next(new Error('Authentication failed!'));
                    }
                });
            
                this.io.of(namespace).on('connection', (socket) => {
                    console.log(`${this.name}: App connected!`); 
            

            Mycroft in my case is on the same RPi so I realize the code will be different and you have created this code in a way to “pair” with the mobile device. Again, I am very new to writing code like this, but can you point in the right direction as far as how to extend the socket with the MMM-mycroft module’s node_helper.js? I do not understand how to extend the socket. Why is the code appSocket() {
            const namespace =
            ${this.name}/app;

            Would the same code work for MMM-mycroft module?

            So my understanding, thanks to your help, to have a flow like this:

            Node_helper extends socket server. Mycroft-core send command to node_helper, then the node_helper sends it to the MMM-mycroft module via this.send socketnotification and the MMM-mycroft module broadcasts it to all other modules via this.sendnotification.

            I am just not sure how to code the Node_helper.js to extend the socket in my case.

            You have been extremely helpful so far and I truly appreciate all of your help.

            Also, since this is all done on the RPi, would the ipwhitelist even come into play here? My guess is no.

            S 1 Reply Last reply Apr 19, 2018, 8:11 PM Reply Quote 0
            • S Offline
              strawberry 3.141 Project Sponsor Module Developer @dmwilsonkc
              last edited by Apr 19, 2018, 8:11 PM

              @dmwilsonkc if you run your mycroft core from the node helper as i do with pocketsphinx in voice then there is no need at all to mess with the socket server

              Please create a github issue if you need help, so I can keep track

              D 1 Reply Last reply Apr 19, 2018, 9:02 PM Reply Quote 0
              • D Offline
                dmwilsonkc @strawberry 3.141
                last edited by Apr 19, 2018, 9:02 PM

                @strawberry-3.141 I didn’t realize that was a possibility. I currently start Mycroft when the pi boots up in a virtualenv as was recommended by the mycroft folks. Both the MM and Mycroft run exactly as they should. The only exception is the microphone issue- not being able to access the microphone from two processes at the same time. I have no idea how I would start Mycroft from the node_helper.js.

                At this point I am very confused on the best way to go. To me, it seems like the socket communication between two concurrently running processes would be easier. But you are the expert.

                Any advice you suggest would be welcomed. Thanks again.

                1 Reply Last reply Reply Quote 0
                • I Offline
                  ime90
                  last edited by Apr 25, 2018, 3:04 PM

                  Hi everyone. I’m having trouble with my voice module.

                  It doesn’t seem to register any of my commands. The microphone icon is blinking but whenever I say “hide modules” or any other command it doesn’t register it.

                  In the terminal it says “Starting pocketsphinx.” Don’t know it that is of any concerns? If anybody has any tips they are more than welcome.

                  S 1 Reply Last reply Apr 25, 2018, 6:58 PM Reply Quote 0
                  • S Offline
                    strawberry 3.141 Project Sponsor Module Developer @ime90
                    last edited by Apr 25, 2018, 6:58 PM

                    @ime90 can you check the .lm and .dic files in your module folder and see if they are empty or include your commands

                    Please create a github issue if you need help, so I can keep track

                    I 1 Reply Last reply Apr 26, 2018, 8:12 AM Reply Quote 0
                    • I Offline
                      ime90 @strawberry 3.141
                      last edited by Apr 26, 2018, 8:12 AM

                      @strawberry-3.141 hi strawberry, thanks for the response.

                      I’ve managed to make it work, but the problem is that the operations were uuuuultra slow.

                      I’ve timed the response time in 10 different occurrences.

                      After saying “MAGIC MIRROR” it takes an average of 7 seconds for the microphone icon to start blinking.

                      After saying “HIDE” it takes an average of 15 second (hope this isn’t normal) to hide the modules.

                      Is this due to my RPI3? Any and all tips and experiences are welcome.

                      1 Reply Last reply Reply Quote 0
                      • 1
                      • 2
                      • 28
                      • 29
                      • 30
                      • 31
                      • 32
                      • 33
                      • 30 / 33
                      30 / 33
                      • First post
                        299/329
                        Last post
                      Enjoying MagicMirror? Please consider a donation!
                      MagicMirror created by Michael Teeuw.
                      Forum managed by Sam, technical setup by Karsten.
                      This forum is using NodeBB as its core | Contributors
                      Contact | Privacy Policy