MMM-AssistantMk2

A Former User

To catch the specific voice order : use transcriptionHook or gAction. gAction is more natural way for user without side-effect.(something similar with Alexa Skill). But transcriptionHook is easier to make. Here is a just sample.

transcriptionHook : {
  "MY_TEST_ORDER" : {
    pattern: "execute order 66",
    command: "MY_TEST_COMMAND"
  }
}

This will hook your transcription from your speech when you say “Execute order 66”.
But this is just a kind of trigger. The real job(MY_TEST_COMMAND) needs to be defined.

To define what to do when this order is triggered. : define command.

command: {
  "MY_TEST_COMMAND": {
    notificationExec: {
      notification: "SHOW_ALERT",
      payload: {
        message: "Yes, my lord."
      }
    }
  }
}

This shows WHAT TO DO - to emit some notification(“SHOW_ALERT”) which alert module can understand and do as it’s job.

This is basic concept.

sdetweil

@Sean said in MMM-AssistantMk2:

which alert module can understand and do as it’s job.

well, SOME module MAY or may NOT understand and process

A Former User

@sdetweil
Yes. indeed. But it’s that module’s own business. Those kinds of modules are not designed to WORK TOGETHER with other modules. Sorry, but it’s not my fault.
In that case, you can execute module’s internal method by force. (Of course it is also not proper in some modules.)

command: {
  "MY_TEST_COMMAND": {
    moduleExec: {
      module: ["alert"],
      exec: (module, params, key) => {
        module.show_alert({message:"Yes, my lord."}) // this code might make an error on real usage, but you can catch what I mean.
      }
    }
  }
}

Anyway, you can build your own module for your specific job when there is no module to use.

teitlebot

@sean
Im planning on using face recognition to show compliments when someone approaches mirror. Or maybe just PIR sensor. Either way, how would I get the compliment text to be said by Google? Would this be difficult?

A Former User

@teitlebot You need just TTS(MMM-TTS, MMM-GoogleTTS you can find). Not google Assistant. AI assistant normally be performed by CONVERSATION (text or vocal). face recognition is not CONVERSATION.
If your face-recognition module can emit some notification, MMM-TTS/MMM-GoogleTTS can use that signal as a trigger for speech. (If you need, MMM-NotificationTrigger also could help to relay notification between modules.)

ejay-ibm

@Sean @teitlebot
Might sound stupid and not feasible at first but this is how I would do the trick.
1st I would use a Notification from input ( camera , pir sensor)
for PIR sensor it would be PRESENCE_DETECTED .
Then I would trigger a command and in this command use the repeat after me function .
So you can get voice output directly from the assistant saying “hello” when presence is detected .
Then for facial recognition it’s just a matter of condition and what notification to send. for each notification a different repat after me.

Does it make sense?

Ejay

A Former User

@ejay-ibm @teitlebot
Maybe possible.
Converting PRESENCE_DETECTED to ASSISTANT_SAY with some payload(“Good morning sir”) could be. you can modify PIR module itself or use MMM-NotificationTrigger instead.

bryan_1

@Sean unfortunately it doesn’t help. I wonder, why only this SHOWMODULES doesn’t work but all other recipes do (include HIDEMODULES) :pensive_face:

teitlebot

@Sean @ejay-ibm
so if I can use assistant_say why would I need to install TTS?

A Former User

@teitlebot
Assistant_say is a kind of hack. It cannot be guaranteed to work properly at every case. And not for long text.

A Former User

@bryan_1
Log is needed to figure out what happens. Furthermore for tracing and checking suspicious points, source modifying is needed.
You can contact me on Telegram (ID : @eouia). If you don’t mind, I can help on the telegram when I’m not busy.

Cr4z33

@Sean please check Telegram as I sent you a message with the error I am getting after MM latest update. ;)

A Former User

@Cr4z33
You need electron-rebuild for module. MichMich change electron version in new update. so, new module binary for electron is needed.

Cr4z33

@Sean thank you.

The steps (suggested by @Sean) to fix the black screen issue after latest MagicMirror update are:

cd ~/MagicMirror/modules/MMM-AssistantMk2
npm install --save-dev electron-rebuild
./node_modules/.bin/electron-rebuild

and if you use MMM-Hotword too

cd ~/MagicMirror/modules/MMM-Hotword/node_modules/snowboy
npm install --save-dev electron-rebuild
npm install nan
./node_modules/.bin/electron-rebuild

bryan_1

@Sean after MM latest update found youtube video is not played automatically. It is only shown. I ran npm start dev and get this error: Failed to execute ‘postMessage’ on ‘DOMWindow’: The target origin provided (‘https://www.youtube.com’) does not match the recipient window’s origin (‘http://0.0.0.0:8080’). Do you have any idea? Thanks in advance.

sdetweil

there is a limitation on the version of electron used… they have prevented autoplay of media (video , and audio)
if there is not a user interaction with the window content in advance… (and with a mirror, we don’t click etc)

anyhow… there is a work around in the shorterm

edit MagicMirror/js/electron.js

after line

function createWindow() {

add

    app.commandLine.appendSwitch('autoplay-policy', 'no-user-gesture-required');  //< -------- added

save and restart MM

bryan_1

@sdetweil I’ll try it tonight. Thank you.

bryan_1

@sdetweil it works, thank you.

DopeEthiop

Hey guys, I’m having a problem with my MMM-AssistantMk2 and possibly my MMM-Hotword. I have a Raspberry Pi3 and I’m using a usb mic for recording.

I have both of the modules up and running but when asking Jarvis a question it takes about 7 seconds before I get any response. Then when I do get a response, the transcription is quick and hurried, while the text on the bottom of the screen disappears quickly. I was wondering if there was a way to optimize this through the config file or if this was something different?

A Former User

@DopeEthiop
Your issue always and only happens when you say “jarvis”? or all the same with other hotwords?
Sometimes the time lag could happen, but your case seems something wrong. Leave your issue to git repository issue board to look inside more deeply. here is not so good place to track issue.