Voice Controlled Wake/Sleep Function

[jeremytripp]

So after setting up the Snowboy Voice Control module, the camera module, and the motion detector module (all working perfectly), I got a little ambitious. I trained a model for the keywords “Wake Up” and “Go to sleep”, added the keywords to voicecontrol.js, then expanded the node_helper.js file inside the Motion Detector module to act when the WAKE_UP and GO_TO_SLEEP notifications are broadcast, but it’s not working and I’m not sure why. Could use a little help – what have I missed?

My understanding is that voicecontrol.js registers the keywords, Snowboy detects them using the .pmdls in the MM root, then broadcasts the notification to all existing modules, which is why I decided to append to the end of the motion detector module since the action is inherently the same (switch off the monitor, switch on the monitor, given some conditional element) rather than building a separate module.

Only supplying the pertinent code blocks here for brevity as the rest seems to be working, but can supply all if it will help.

voicecontrol.js (defines the keyword, their broadcast notification, and front end elements)

models: [
                    {
                        keyword: "Show Camera",
                        description: "Say 'Show Camera' to display camera",
                        file: "showCamera.pmdl",
                        message: "SHOW_CAMERA"
                    },
                    {
                        keyword: "Hide Camera",
                        description: "Say 'Hide Camera' to hide camera",
                        file: "hideCamera.pmdl",
                        message: "HIDE_CAMERA"
                    },
                    {
                        keyword: "Selfie",
                        description: "Say 'Selfie' when camera is visible",
                        file: "selfie.pmdl",
                        message: "SELFIE"
                    },
                    {
                        keyword: "Wake Up",
                        description: "Say 'Wake Up' to turn on the mirror",
                        file: "wakeUp.pmdl",
                        message: "WAKE_UP"
                    },
                    {
                        keyword: "Go To Sleep",
                        description: "Say 'Go To Sleep' to turn off the mirror",
                        file: "goToSleep.pmdl",
                        message: "GO_TO_SLEEP"
                    },
                ]

motion detector node_helper.js (catches the notification, supposed to execute the monitor on/off action)

// Subclass socketNotificationReceived received.
  socketNotificationReceived: function (notification, payload) {
      const self = this;
      if (notification ===  'MOTION_DETECTED' && this.started == false) {
          const self = this;
          this.started = true;
          this.activateMonitor();
      }
 
      if (notification ===  'WAKE_UP' && this.started == false) {
          const self = this;
          this.started = true;
          this.activateMonitor();
      }
 
      if (notification ===  'DEACTIVATE_MONITOR' && this.started == false) {
          const self = this;
          this.started = true;
          this.deactivateMonitor();
      }
 
      if (notification ===  'GO_TO_SLEEP' && this.started == false) {
          const self = this;
          this.started = true;
          this.deactivateMonitor();
      }
  }

NOTE: Originally, I set it up like below to be more concise, but after running npm start on the mirror, it would load up momentarily, then shut down the monitor almost immediately (without any interaction whatsoever) and no keyboard command or voice command would bring it back up (had to unplug and reboot to bring it back). Therefore, I opted for separate conditional statements as above – which doesn’t deactivate the monitor immediately, but also doesn’t work given the voice prompt, soooooooo…:

// Subclass socketNotificationReceived received.
  socketNotificationReceived: function (notification, payload) {
      const self = this;
      if (notification ===  'MOTION_DETECTED' && this.started == false ||
                            'WAKE_UP && this.started == false) {
          const self = this;
          this.started = true;
          this.activateMonitor();
      }
 
      if (notification ===  'DEACTIVATE_MONITOR' && this.started == false
                            'GO_TO_SLEEP' && this.started == false) {
          const self = this;
          this.started = true;
          this.deactivateMonitor();
      }
   }

Any guidance in my logic would be appreciated! Thanks!

[chengstark]

wow, thanks a lot, I was working on this last night too, i have a similar problem, maybe alexyak can help

[chengstark]

This post is deleted!

[jeremytripp]

@chengstark Did you get any closer than I did? Let’s call on @alexyak and see if he can shed some light.

[chengstark]

@jeremytripp i didnt get any closer, im still waiting for my new mic to be delivered from amazon

[alexyak]

@jeremytripp So, just to confirm you’d like to activate/deactivate the mirror with voice commands without walking into the room, right? I don’t see anything glaringly wrong with the code except for potential conflicts such as the scenario when you’re in the room where the mirror is and you say “Go to sleep”, the monitor will be switched off, but when you start moving it will switch on again via the motion detector.

[jeremytripp]

@alexyak That’a a great point – hadn’t considered that conflict. I wonder if that’s what is preventing the voice commands from working in the first place.

As an aside, I could probably solve that with a quick global variable Boolean to select whether to use voice or motion, essentially only allowing one to be on at a time (with an ‘if’ wrapper to check if true/false). Might be more complicated than it’s worth though.

Fundamentally I’m just wondering where you’re writing your functions for any new voice commands you set up. As a basic example, if I wanted to create a new voice command that just showed some “Hello World” text when I gave it some verbal command, how would you set that up?

1. Train pmdl model, drop in MM root
2. Add keyword info to config defaults in voicecontrol.js
3. Then what? I’ve seen the function itself written in a separate module like camera.js is, in node_helper.js like motion detector is, etc. What’s your suggested best practice?

[alexyak]

@jeremytripp So the idea behind the voice control module is that it’s responsible for recognition of the command and sending a global notification to any other module interested in this.

As for the module that you’re describing like displaying some text based on the command I would recommend creating a separate module in which to listen to this specific message and doing what you need it do.

[SebTota]

Hey guys I’m pretty new to this whole magic mirror thing. I don’t want my mirror on all the time so I wanted to turn the display on and off with my voice. This is the only post I could find trying to accomplish that. So once i get the Voice Control module from alexyak working, how do I get this to work? I’m confused as to where to put the files.