MMM-AssistantMk2

[bachoo786]

@Davido I am using raspberry pi 3b+

[Davido]

@bachoo786 :

i am using a normal rPi 3B without + and the answering time is really fast … maybe some other Module which is interacting?

[bachoo786]

@Davido nope nothing is interacting with my assistantmk2 module.

[bachoo786]

@Davido can you share your config for both mmm-assistant and mmm-hotword please?

[Davido]

@bachoo786

I hope this will help

{
  module: "MMM-AssistantMk2",
  position: "top_right",
  config: {

    // --- ESSENTIALS / modifying for your environment might be needed.


    deviceLocation: {
      coordinates: { // set the latitude and longitude of the device to get localized information like weather or time. (ref. mygeoposition.com)
        latitude: 47.809490, // -90.0 - +90.0
        longitude: 13.055010, // -180.0 - +180.0
      },
    },

    defaultProfile: "default", // If you have several profiles and want to set one of them as default profile, describe here.

    profiles: {
      "default" : { // profile name.
        profileFile: "default.json", // profile file name.
        lang: "de-DE"
        //currently available (estimation, not all tested):
        //  de-DE, en-AU, en-CA, en-GB, en-US, en-IN
        // fr-CA, fr-FR, it-IT, ja-JP, es-ES, es-MX, ko-KR, pt-BR
        // https://developers.google.com/assistant/sdk/reference/rpc/languages
      },
      /* Add your other profiles here, if exists.
      "other_profile" : {
        profileFile: "other.json",
        lang: "de-DE"
      }
      */
    },

    record: { // Full values are in `FOR EXPERTS` section.
      recordProgram: "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
      device: null        // recording device (e.g.: "plughw:1")
    },

    play: { // Full values are in `FOR EXPERTS` section.
      playProgram: "mpg321", // recommended.
    },


    // --- OPTIONAL / not important but customizable for your usage


    responseVoice: true, // If available, Assistant will response with her voice.
    responseScreen: true, // If available, Assistant will response with some rendered HTML
    responseAlert: true, // If available, Assistant will response with Alert module of MM
    // Sometimes, any response might not be returned. responseAlert is useful for displaying error.

    screenZoom: "80%", // Adjust responseScreen to your mirror size.
    screenDuration: 2000, // milliseconds. How long responseScreen will be shown after speech.
    //If you set 0, Screen Output will be closed after Response speech finishes ASAP.

    youtubeAutoplay: true, //If set as true, found Youtube video will be played automatically.
    pauseOnYoutube:true, //If set as true, You cannot activate Assistant during youtube playing. Recommended for the performance (Because permanent hotword detecting might make performance lower)

    useWelcomeMessage: "", //Try "brief today" as this value. You can use this value to check module working when MM is starting.

    onIdle: {
      timer: 1000*60*30, // if you don't want to use this feature, just set timer as `0` or command as ""
      command: "HIDEMODULES"
    },

    onActivate: {
      timer: 0,
      command: "SHOWMODULES"
    },


    // --- FOR EXPERTS / For development, debug or more


    verbose:false, // You can get error or some logs when this value is set as true.

    startChime: "connection.mp3", // you can use `mp3` to play chime when your mic is ready. It should be playable with your `play.playProgram`
    noChimeOnSay: false, // When using the `ASSISTANT_SAY` trigger, you can prevent the chime from being played before your words

    auth: { // I believe you don't need to change this.
      keyFilePath: "./credentials.json"
    },

    record:  { // Full version
      sampleRate    : 16000,      // audio sample rate
      threshold     : 0.5,        // silence threshold (rec only)
      thresholdStart: null,       // silence threshold to start recording, overrides threshold (rec only)
      thresholdEnd  : null,       // silence threshold to end recording, overrides threshold (rec only)
      silence       : 1.0,        // seconds of silence before ending
      verbose       : false,      // log info to the console
      recordProgram : "arecord",  // Defaults to "arecord" - also supports "rec" and "sox"
      device        : null        // recording device (e.g.: "plughw:1")
    },

    play: { // Full version
      encodingOut: "MP3", //'MP3' or 'WAV' is available, but you might not need to modify this.
      sampleRateOut: 24000,
      playProgram: "mpg321", //Your prefer sound play program. By example, if you are running this on OSX, `afplay` could be available.
      playOption: [],
      // If you need additional options to use playProgram, describe here. (except filename)
      // e.g: ["-d", "", "-t", "100"]
    },

    useGactionCLI: false, // If set as true, you can update your gAction when MM is rebooted.
    projectId: "", // Google Assistant ProjectId (Required only when you use gAction.)
    deviceModelId: "", // It should be described in your config.json. In most of case, you don't need to this.
    deviceInstanceId: "", // It should be described in your config.json. In most of case, you don't need to this.

    action:{}, // You can catch your gAction command.

    transcriptionHook: {}, // You can catch transcription hook and be able to make your own `COMMAND` with this.
    //See the `transcriptionHook` section.

    command: {}, // You can make your own MM command for gAction and transcriptionHook
    //See the `command` section.
    record: {
        recordProgram : "arecord",  
        device        : "plughw:1",
      },

      notifications: {
        ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",
        ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",
      },
	
    notifications: { // You can redefine these notifications to communicate with specific modules.
      ASSISTANT_ACTIVATE: "ASSISTANT_ACTIVATE",
      ASSISTANT_DEACTIVATE: "ASSISTANT_CLEAR",
      ASSISTANT_ACTIVATED: "ASSISTANT_ACTIVATED",
      ASSISTANT_DEACTIVATED: "ASSISTANT_DEACTIVATED",
      ASSISTANT_ACTION: "ASSISTANT_ACTION",
      ASSISTANT_ACTIVATED: "HOTWORD_PAUSE",     
      ASSISTANT_DEACTIVATED: "HOTWORD_RESUME",      
      DEFAULT_HOOK_NOTIFICATION: "ASSISTANT_HOOK",
      TEXT_QUERY: "ASSISTANT_QUERY",
      SAY_TEXT: "ASSISTANT_SAY",
    }
  }
},
 {
    module: "MMM-Hotword",
    config: {
      snowboy: [
  {
    hotwords: "smartmirror", //this will be sent to other module for distinguishing which hotword is detected.
    file: "resources/models/smart_mirror.umdl",
    sensitivity: '0.5',
  },
  {
    hotwords: "snowboy",
    file: "resources/models/snowboy.umdl",
    sensitivity: '0.5',
  },
  {
    file: 'resources/models/jarvis.umdl',
    sensitivity: '0.8,0.80',
    hotwords: ['jarvis','jarvis'] //Kitt.ai changed their Jarvis UMDL, it has 2 models in one file. So weird.
    //anyway, you can give different name for each. ['jarvis_1', 'jarvis_2']. Even though I think this is useless.
  }
    ],
    record: {
      sampleRate    : 16000,      // audio sample rate
      threshold     : 0.5,        // silence threshold (rec only)
      thresholdStart: null,       // silence threshold to start recording, overrides threshold (rec only)
      thresholdEnd  : null,       // silence threshold to end recording, overrides threshold (rec only)
      silence       : 1.0,        // seconds of silence before ending
      verbose       : false,      // log info to the console. Use this when you want to check mic working or not.
      recordProgram : 'arecord',  // Defaults to 'arecord' - also supports 'rec' and 'sox'
      device        : null        // recording device (e.g.: 'plughw:1')
    },
    autostart: true,              // if 'false', this module will wait for 'HOTWORD_RESUME' notification to start hotwords detection at the beginning.
    autorestart: false,          // You can set this 'true' when you want this module to go back to listening mode automatically again after hotword is detected. But use this carefully when your other modules are using microphone or speaker.

    // customizable notification trigger
    notifications: {
      PAUSE: "HOTWORD_PAUSE",
      RESUME: "HOTWORD_RESUME",
      LISTENING : "HOTWORD_LISTENING",
      SLEEPING : "HOTWORD_SLEEPING",
      ERROR : "HOTWORD_ERROR",
    },
    onDetected: {
      notification: (payload) => {
        return "HOTWORD_DETECTED"
      },
      payload: (payload) => {
        return payload
      }
    },
	  record: {
        recordProgram : "arecord",  
        device        : "plughw:1",
      },
      autostart:true,
      onDetected: {
        notification: function (payload) {
          return "ASSISTANT_ACTIVATE"
        },
        payload: function (payload){
          return {
            profile: payload.hotword
          }
        }
      },
    },
  },
	

[bachoo786]

@Davido thank you

[A Former User]

UPDATED

[2.1.4] - 2019.03.26

• Added : Now you can use recipes. https://github.com/eouia/MMM-AssistantMk2/wiki/Usage#recipes

  recipe is an external js file containing definitions of command, transcriptionHook and action. Your configuration could be more shorter.

recipes:["hide_and_show_all_modules.js", "reboot.js", "screen_onoff.js", "shutdown.js"],

Feel free to request PR to share your recipes to others.

• Changed : onIdle, onDetected features are disabled by default.

[bryan_1]

@Sean
sorry for my bad English. I use the default recipes and set the language to German for all pattern (“spiegel reboot”, “monitor aus”, etc). Every thing works fine, only the “SHOWMODULES” doesn’t work. Although the hotword is correctly detected, but nothing happens. I set the pattern for SHOWMODULE to pattern: “alle module”,

This is also the same case if I use the OnIdle and OnActivate:
onIdle: {
timer: 10006020,
command: “HIDEMODULES”,
},
onActivate: {
timer: 0,
command: “SHOWMODULES”,
},

I can hide the modules, but after the hotword for SHOWMODULES being recognized the modules aren’t shown again. What do I wrong? Many thanks in advance.

[A Former User]

@bryan_1
Can you show me the recipe(command and transcriptionHook or gAction) about SHOWMODULES?

[bryan_1]

@Sean
thank you sean for your quick response. Actually I use the default hide_and_show_all_modules.js and change the pattern to german:
var recipe = {
transcriptionHook: {
“HIDE_ALL_MODULES”: {
pattern: “alle verstecken”,
command: “HIDEMODULES”
},
“SHOW_ALL_MODULES”: {
pattern: “alle module”,
command: “SHOWMODULES”
},
},
command: {
“HIDEMODULES”: {
moduleExec: {
module:()=>{
return []
},
exec: (module, params, key) => {
module.hide(1000, null, {lockString:“AMK2”})
}
}
},
“SHOWMODULES”: {
moduleExec: {
module:[],
exec: (module, params, key) => {
module.show(1000, null, {lockString:“AMK2”})
}
}
},
},
}

exports.recipe = recipe

[A Former User]

@bryan_1
try this instead module.show(1000, null, {lockString:“AMK2”})

module.show(1000, null, {lockString: "AMK2", force: true})

[bryan_1]

@Sean I’ll try it tonight (I’m still at work now :smiling_face_with_smiling_eyes: ) and let you know. Many thanks…you did really a great job :thumbs_up_light_skin_tone: :folded_hands:

[sdetweil]

related to this topic, I haven’t been able to understand what I need to do.

I am trying to coordinate between the voice module go to sleep/wake up, and my motion detection module MMM-SleepWake.

My displays will not turn off programmatically, so I have to hide/show modules

But you only show what was NOT already hidden at time of sleep.
Voice says sleep, motion says wake… but… I need to know what was already hidden, so I don’t wake them up (restore to same state)…

with MMM-voice we developed a notification process, where the voice module does the sleep, but informs my module who was already hidden… so IF motion wakes up, i don’t accidentally show modules I shouldn’t…

wake up voice command works ok, (changed screenOn to notify my module to wake)
go to sleep is missing…

this is the socketnotification code in MMM-voice.js after voice detects ‘go to sleep’, only if using the hide/show sleep approach

if (notification === 'STAND_BY_ACTION') {
            if (payload.type === 'show') {
                if (payload.hardware === false) {
                    MM.getModules().enumerate((module) => {
                        if (this.previouslyHidden.includes(module.identifier)) {
                            module.show(1000);
                        }
                    });
                    this.previouslyHidden = [];
                }

                this.sendNotification('STAND_BY', { status: false });
            } else if (payload.type === 'hide') {
                if (payload.hardware === false) {
                    MM.getModules().enumerate((module) => {
                        if (module.hidden === true) {
                            this.previouslyHidden.push(module.identifier);
                        } else {
                            module.hide(1000);
                        }
                    });
                }

                this.sendNotification('STAND_BY', { status: true, modules: this.previouslyHidden.slice(0) });

[A Former User]

@sdetweil
First, this module doesn’t manage module’s showing and hiding itself. Of course I provided some recipe about it, but it’s better to be considered as a just example. The main purpose of this AMK2 is to give an ability about voice command and to relay it to other module. Not to control all things by AMK2 itself.
I think real job should be done by proper extern module. So If you have any good module(which is existed already or be planned to be made by yourself) to manage modules’ showing and hiding(There would be several modules already for this job), just transfer proper commands(notifications) to that module to do its job.

[sdetweil]

@Sean ok, thanks… still don’t understand recipe process design…

voice triggers command? command is some routine? run in context of module.js?

[A Former User]

@sdetweil

• To catch the specific voice order : use transcriptionHook or gAction. gAction is more natural way for user without side-effect.(something similar with Alexa Skill). But transcriptionHook is easier to make. Here is a just sample.

transcriptionHook : {
  "MY_TEST_ORDER" : {
    pattern: "execute order 66",
    command: "MY_TEST_COMMAND"
  }
}

This will hook your transcription from your speech when you say “Execute order 66”.
But this is just a kind of trigger. The real job(MY_TEST_COMMAND) needs to be defined.

• To define what to do when this order is triggered. : define command.

command: {
  "MY_TEST_COMMAND": {
    notificationExec: {
      notification: "SHOW_ALERT",
      payload: {
        message: "Yes, my lord."
      }
    }
  }
}

This shows WHAT TO DO - to emit some notification(“SHOW_ALERT”) which alert module can understand and do as it’s job.

This is basic concept.

[sdetweil]

@Sean said in MMM-AssistantMk2:

which alert module can understand and do as it’s job.

well, SOME module MAY or may NOT understand and process

[A Former User]

@sdetweil
Yes. indeed. But it’s that module’s own business. Those kinds of modules are not designed to WORK TOGETHER with other modules. Sorry, but it’s not my fault.
In that case, you can execute module’s internal method by force. (Of course it is also not proper in some modules.)

command: {
  "MY_TEST_COMMAND": {
    moduleExec: {
      module: ["alert"],
      exec: (module, params, key) => {
        module.show_alert({message:"Yes, my lord."}) // this code might make an error on real usage, but you can catch what I mean.
      }
    }
  }
}

Anyway, you can build your own module for your specific job when there is no module to use.

[teitlebot]

@sean
Im planning on using face recognition to show compliments when someone approaches mirror. Or maybe just PIR sensor. Either way, how would I get the compliment text to be said by Google? Would this be difficult?

[A Former User]

@teitlebot You need just TTS(MMM-TTS, MMM-GoogleTTS you can find). Not google Assistant. AI assistant normally be performed by CONVERSATION (text or vocal). face recognition is not CONVERSATION.
If your face-recognition module can emit some notification, MMM-TTS/MMM-GoogleTTS can use that signal as a trigger for speech. (If you need, MMM-NotificationTrigger also could help to relay notification between modules.)