MMM-AssistantMk2

komodoteler

Sean, thanks for the help and patience, finally managed to work out the issue.

it was mainly with improper setting on .asoundrc and config.js

This is hands down the best voice recognition with visual there is.

One small thing I noticed is the youtube video opened sometimes throw “invalid parameter”. Could it be default youtube parameter on config.js?

A Former User

@komodoteler Most of those cases are, “Not allowed playing on ‘embeded player(out of youtube)’ by Authorities”.

Sun

Start Create Project, what option to choose, is it for Home control ?
During the process of register a device model, What to choose in Device Type?

A Former User

@sun

You can skip that part or select whatever you want.
That is also whatever you want. Usually, I set as “TV”. (because it definitely has screen to display something)

Sun

Finally, Got the microphone icon in my mirror. Didn’t detect the voice yet I still have to install mmm-hotword.
But I can ask one question by click the microphone icon and see the pop-up.

I use hdmi on my tv as well. Still no sound.

I will install mmm-hotword and test.

Also the mmm-gaction if it make the mirror better.

I will keep you update and might need your help if I got struck in some part.

Thank you very much Mr.Sean.

Sun

Install mmm-hotword everything went fine during installation and load magic mirror normally. But no hotword detected.

also show in command promt.

Refreshed access token because it has expired. Expired at: 18:57:26 now is: 18:57:29
Refreshed access token because it has expired. Expired at: 19:57:29 now is: 18:57:29
Refreshed access token because it has expired. Expired at: 19:57:29 now is: 18:57:29
Refreshed access token because it has expired. Expired at: 19:57:29 now is: 18:57:30

and when click the microphone icon. I can only click once after restart and ask question. Hotword didn’t detect smart mirror, snowboy.

in command promt shows
at Speaker._write (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/speaker/index.js:183:23)
at doWrite (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/readable-stream/lib/_stream_writable.js:428:64)
at writeOrBuffer (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/readable-stream/lib/_stream_writable.js:417:5)
at Speaker.Writable.write (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/readable-stream/lib/_stream_writable.js:334:11)
at Conversation.conversation.on (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_helper.js:75:16)
at emitOne (events.js:116:13)
at Conversation.emit (events.js:211:7)
at ClientDuplexStream.Conversation.conversation.on (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/google-assistant/components/conversation.js:154:12)
at emitOne (events.js:116:13)
Conversation Completed
[HOTWORD] << snowboy >> is detected.
[HOTWORD] stops listening

A Former User

@sun
have you removed or refreshed your credentials.json or default.json in MMM-AssistantMk2?

I suggest you remove or disable all other modules (including MMM-AssistantMk2) except MMM-Hotword in config.js and test it.

Definitely success with MMM-Hotword could make things simple and easy.

After removing all modules except MMM-Hotword in config.js, run MM.(npm start)
You can see;
[HOTWORD] begins listening
say smart mirror
You can see;

[HOTWORD] << smart mirror >> is detected.
[HOTWORD] stop listening

You should confirm this before going forward.

A Former User

@sun
OK. have you installed “MMM-NotificationTrigger” ?
Or give me a chat in this forum, or telegram me (@eouia)

Sun

Not yet install mmm-notificationtrigger.
Do I have to install mmm-notificationtrigger as well?
I manage to get
[HOTWORD] << smart mirror >> is detected.
[HOTWORD] stop listening

but when I say something. it won’t detect what I said. Only detect the hotword.

Sun

Got it done. Very nice. However, the sound is so lag. Might be because I run on rp3 and quite many modules in magic mirror.

A Former User

@sun
Maybe. I always use BT speaker for my RPI and ATB. I cannot bear the crumpled sounds from analog or USB.

jomwells

This is fantastic thank you, I have got everything set up, it hears and responds (on screen) to every request - but unfortunately I get no sound (no speaking feedback), when a youtube video plays, the sound works fine, do I need to change the output device somewhere? I’m struggling to find the config for it.

A Former User

@jomwells is there any suspicious errors?

Sun

@jomwells Are you using rp3 as magic mirror? I did get sound respond now and it really ugly. I do try both sound on tv via hdmi and from usb sound card. Assume that too much modules from magic mirror load.

Sun

@sean I would like to know more about MMM-Gaction regarding the deviceModelId - this should be done since register the device in mmm-assistantmk2. What about deviceInstanceId, where and how to?

A Former User

@sun read this page carefully.
https://developers.google.com/assistant/sdk/reference/device-registration/register-device-manual#get-access-token

A Former User

@sun But In most cases, you might not need instance, because your mirror might be only one instance of your project. Instance is needed when action should select specific device for response. Try without instanceid first.

Sun

@sean Thank you so much.

jomwells

@sean Yes, the pm2 error logs show some alsa related errors, I assume this is why the assistant doesn’t produce audio?

/home/pi/.pm2/logs/MagicMirror-error.log:

[../deps/mpg123/src/output/alsa.c:90] error: initialize_device(): cannot set format S16_LE
MESA-LOADER: failed to retrieve device information
libGL error: MESA-LOADER: failed to retrieve device information
[../deps/mpg123/src/output/alsa.c:94] error: initialize_device(): cannot set 1 channels

I have a pretty fresh install of MagicMirror so shouldn’t be an issue of too many modules, im using an RPI3, and a USB sound card, thanks
on a side note, to get the Hotword and Assistant module to choose the correct microphone, I have had to change the input device to

device: 'plughw:1',

however each time I reboot, the input devices change order and I need to edit the config file again, is there a way to stop them changing device number?

A Former User

@jomwells
Frankly says, I’m not an expert of system administration. Especially Alsa configuration is out of my ability.
Anyway you should success this test.

arecord -d 10 test.wav & aplay test.wav

It should record your voice and play again.

It means, your mic and speaker should be default device.