MMM-AssistantMk2 with Google AIY

[Croeder]

I’ve followed all the instructions on https://github.com/eouia/MMM-AssistantMk2/wiki in order to install the Google Assistant module. I am using the Google AIY kit with their special raspbian image made to work with it.

I’m trying to get the module working with my mirror but I’ve encountered a couple of problems.

First, the module will not appear on my mirror. This is the code I’ve put into my config.js file: https://pastebin.com/UUrtm9tV
It is just the simple configuration found on the module’s github page.

Second, at some point during the installation of the module, it asked me to test my audio environment. The speaker did not play the google beep sound that was supposed to be played, and similarly the mic did not work. This concerns me as even if I am able to get the module to appear on my mirror, I’m sure it won’t function properly if the mic and speaker won’t pass this test. The odd thing is that Google has their own audio test built into their image of raspbian and both the speaker and mic work perfectly fine according to that test.

I’m sorry, I have no programming experience and I’m attempting this project solely off of my intuition. Any help would be appreciated. Thanks!

[nakedrabbit]

@Croeder Howdy!

I’m also working with the Google AIY kit, so I have a little experience to share. First, version 2 of the AssistantMk2 module does not, repeat not, work with the Google Hat. Not at all. A couple of us have found that out, and even the developer suggests that there is something wacky about Google’s use of the ALSA drivers or somesuch.

The reason the moduel may not be showing up could be with having to rebuild electron, which is a documented bug for v2.

I can report that v3 of the module will install on the Google Hat just fine, and will give you all the feedback and test results you want… BUT… right now there seems to be a memory issue with the module and they are still working on it. You can see my part of the discussion here:

https://forum.magicmirror.builders/topic/11991/mmm-assistantmk2-v3/39

I have also tried installing the Google AIY assistant as a service, loading on startup. The advantage there is you get your LED button back. But I had issues with the responses cutting off and being weird, so I deinstalled that.

Just waiting for results on version 3 now…!

[A Former User]

hi,

do you want to try to configure it with Google AIY ?

i think, it should be possible ;)

we can try if you want !

@bugsounet (MMM-AssistantMk2 developer)

[Croeder]

@nakedrabbit

Hello,

I appreciate the reply!

I might be mistaken but I believe that I was running v3 of the module. Odd that it works for you but not for me. May be something with the hardware. I’m sure I’ll figure it out eventually though :thumbs_up:

[Croeder]

@Bugsounet

Hey! Sorry for the late response. I’d love to try to configure it if you are still down!

[A Former User]

so, what is the problem at now ?

memory issue ?
playback issue ?
configuring problem to find mic config ?

for memory issue, i have some solutions …

if it’s for AMK2 memory, i have solution
if memory low is with MMM-Hotword, i have again a solution :)

there many things to test :)

Ready ?

[nakedrabbit]

@Bugsounet Sorry I was away for a bit.

YES! I’m ready to try anything, including burning it all down and starting over. Let me know what I can do!

My issues do not SEEM to be mic or speaker configs. When I install the tests all work just great. I do seem to have memory issues.

Give me the tests! And thank you!

[A Former User]

So go on :)

Generally memory problems are due to the use of sound via HTML5 (we don’t know really why)

Let’s turn off and use the “native” sound of rpi

in your config.js MMM-AssistantMk2

let’s turn off HTML5 sound by using useHTML5: false
if this feature don’t exist in your config

wiki can help you to find needed block to add it

Of course, you will have to restart MagicMiror after this modification

awaiting your first return

[nakedrabbit]

@Bugsounet said in MMM-AssistantMk2 with Google AIY:

useHTML5: false

I’ve had that one in my config all along, so yes, I’ve accomplished this first step a while ago, still have the problem.

[nakedrabbit]

@Bugsounet NO WAIT!

I did not configure properly! I did not realize the HTML5 option needed to be enclosed in the responseConfig section. Thanks for linking the wiki so I could double-check!

No response yet. This is AssistantMk2 with your own snowboy option selected and Hotword NOT enabled. I’m going to try Hotword again now.

[A Former User]

ok , i wait

give me all log plz

and give me your config.js file

[nakedrabbit]

logs from pm2:

0|mm | [AMK2] MMM-AssistantMk2 Version: 3.1.0-3
0|mm | [AMK2] AssistantMk2 is initialized.
0|mm | [HOTWORD] begins.
0|mm | [HOTWORD] Detector starts listening.
0|mm | [HOTWORD] Detected: SMARTMIRROR
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] stops.
0|mm | [HOTWORD] After Recording finised. size: 16050
0|mm | [HOTWORD:B2W] WAV_FILE_CREATED: /home/pi/MagicMirror/modules/MMM-Hotword/temp/afterRecording.wav 16050
0|mm | [HOTWORD] Final Result: { detected: true,
0|mm | hotword: ‘SMARTMIRROR’,
0|mm | file: ‘temp/afterRecording.wav’ }

It is true, Hotword records a 16K file that seems to only include noise. It does recognize and pass me to AssistantMk2, but immediately says “I don’t understand” and then cannot be invoked again.

The file “temp/afterRecording.wav” exists, but it is a 0K file.

Config for MM sections are here:

{
module: “MMM-AssistantMk2”,
position: “top_left”,
config: {
assistantConfig: {
latitude: 51.508530,
longitude: -0.076132,
playProgram: “mpg321”,
},
responseConfig: {
useHTML5: false,
},
micConfig: { // put there configuration generated by auto-installer
recorder: “arecord”,
device: “plughw:0”,
},
}
},
{
module: “MMM-Hotword”,
position: “bottom_left”,
config: {
useDisplay: false,
chimeOnFinish: null,
recipes: [“with-AMk2v3_smart-mirror.js”],
mic: {
recordProgram: “arecord”,
device: “plughw:0”,
},
},
},

Forum does not allow uploading the .js file, if you need the whole thing lemme know and I’ll put it up somewhere else.

[A Former User]

 {
  module: "MMM-Hotword",
  position: "bottom_left",
  config: {
    useDisplay: false,
    chimeOnFinish: null,
    recipes: ["with-AMk2v3-noisy_smart-mirror.js"],
    mic: {
      recordProgram: "arecord",
      device: "plughw:0",
    },
  },
},

can you retry with this MMM-Hotword configuration ?

edit 1: sorry, correct mismake

edit 2:

humm there is some mismake in your AMk2 config
i propose this AMk2 config :

{
  module: "MMM-AssistantMk2",
  position: "top_left",
  config: {
    debug: true,
    ui: "Fullscreen",
    assistantConfig: {
      latitude: 51.508530,
      longitude: -0.076132,
    },
    responseConfig: {
      useHTML5: false,
      playProgram: "mpg321",
      useStaticIcons: false,
    },
    micConfig: {
      recorder: "arecord",
      device: "plughw:0",
    },
    recipes: [ "with-MMM-Hotword.js" ]
    profiles: {
     "default": {
        profileFile: "default.json",
        lang: "en-US"
      }
    },
    addons: false,
  }
}

[sdetweil]

@nakedrabbit please use the code marker for all config entry data u might post…

paste the text,
then mark it, and hit the </> button above the edit box

[nakedrabbit]

@Bugsounet YOU ARE A MAGIC MAN.

And @sdetweil , you are quite right about my forum incompetence. For the record I will try to do better right now.

Bugs, there were some commas missing that threw me off for a while, but this is the config that worked/ I encourage @Croeder to check this out, since I am also using the Google Hat.

{
  			module: "MMM-AssistantMk2",
  			position: "top_left",
  			config: {
    			debug: true,
    			ui: "Fullscreen",
    			assistantConfig: {
      				latitude: 51.508530,
      				longitude: -0.076132,
    				},
    			responseConfig: {
      				useHTML5: false,
      				playProgram: "mpg321",
      				useStaticIcons: false,
    				},
    			micConfig: {
      				recorder: "arecord",
      				device: "plughw:0",
    				},
    			recipes: [ "with-MMM-Hotword.js" ],
    			profiles: {
     				"default": {
        				profileFile: "default.json",
        				lang: "en-US",
      					},
    				},
    			addons: false,
  				}
			},

[A Former User]

I think, it’s works now :)

• Your default location is defined by this block:

    assistantConfig: {
      latitude: 51.508530,
      longitude: -0.076132,
    },

change latitude and longitude value for your actual position
This website can help you to determinate it

• For language change:

change the value lang: "en-US", (actually in english) in

  profiles: {
    "default": {
      profileFile: "default.json",
      lang: "en-US",
    },
  },

Change the lang value by your own.
You can determinate it there

Notes:

• I’m not a magician, just a hobbyist programmer of this module ;)
• Read wiki for advanced config