@Bugsounet YOU ARE A MAGIC MAN.

And @sdetweil , you are quite right about my forum incompetence. For the record I will try to do better right now.

Bugs, there were some commas missing that threw me off for a while, but this is the config that worked/ I encourage @Croeder to check this out, since I am also using the Google Hat.

{
  			module: "MMM-AssistantMk2",
  			position: "top_left",
  			config: {
    			debug: true,
    			ui: "Fullscreen",
    			assistantConfig: {
      				latitude: 51.508530,
      				longitude: -0.076132,
    				},
    			responseConfig: {
      				useHTML5: false,
      				playProgram: "mpg321",
      				useStaticIcons: false,
    				},
    			micConfig: {
      				recorder: "arecord",
      				device: "plughw:0",
    				},
    			recipes: [ "with-MMM-Hotword.js" ],
    			profiles: {
     				"default": {
        				profileFile: "default.json",
        				lang: "en-US",
      					},
    				},
    			addons: false,
  				}
			},

logs from pm2:

0|mm | [AMK2] MMM-AssistantMk2 Version: 3.1.0-3
0|mm | [AMK2] AssistantMk2 is initialized.
0|mm | [HOTWORD] begins.
0|mm | [HOTWORD] Detector starts listening.
0|mm | [HOTWORD] Detected: SMARTMIRROR
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] stops.
0|mm | [HOTWORD] After Recording finised. size: 16050
0|mm | [HOTWORD:B2W] WAV_FILE_CREATED: /home/pi/MagicMirror/modules/MMM-Hotword/temp/afterRecording.wav 16050
0|mm | [HOTWORD] Final Result: { detected: true,
0|mm | hotword: ‘SMARTMIRROR’,
0|mm | file: ‘temp/afterRecording.wav’ }

It is true, Hotword records a 16K file that seems to only include noise. It does recognize and pass me to AssistantMk2, but immediately says “I don’t understand” and then cannot be invoked again.

The file “temp/afterRecording.wav” exists, but it is a 0K file.

Config for MM sections are here:

{
module: “MMM-AssistantMk2”,
position: “top_left”,
config: {
assistantConfig: {
latitude: 51.508530,
longitude: -0.076132,
playProgram: “mpg321”,
},
responseConfig: {
useHTML5: false,
},
micConfig: { // put there configuration generated by auto-installer
recorder: “arecord”,
device: “plughw:0”,
},
}
},
{
module: “MMM-Hotword”,
position: “bottom_left”,
config: {
useDisplay: false,
chimeOnFinish: null,
recipes: [“with-AMk2v3_smart-mirror.js”],
mic: {
recordProgram: “arecord”,
device: “plughw:0”,
},
},
},

Forum does not allow uploading the .js file, if you need the whole thing lemme know and I’ll put it up somewhere else.

@Bugsounet NO WAIT!

I did not configure properly! I did not realize the HTML5 option needed to be enclosed in the responseConfig section. Thanks for linking the wiki so I could double-check!

No response yet. This is AssistantMk2 with your own snowboy option selected and Hotword NOT enabled. I’m going to try Hotword again now.

@Bugsounet said in MMM-AssistantMk2 with Google AIY:

useHTML5: false

I’ve had that one in my config all along, so yes, I’ve accomplished this first step a while ago, still have the problem.

@Bugsounet Sorry I was away for a bit.

YES! I’m ready to try anything, including burning it all down and starting over. Let me know what I can do!

My issues do not SEEM to be mic or speaker configs. When I install the tests all work just great. I do seem to have memory issues.

Give me the tests! And thank you!

Yes, that seems obvious, but I did, in fact, follow the installation instructions, so I’m not sure why I have this error message either… left over from an aborted earlier attempt?

In addition I took out Hotword and used your fine snowboy addon. Exactly the same effect, so I must have something wrong with my general setup… still testing…

Thanks for your patience!

New version installed fine for me, though I did not git pull, just wiped the old one off. Installer worked great, and I no longer have to rebuild electron afterwards, as I did previously. Everything worked great.

HOWEVER

I still can’t get much action out of the plugin. Here’s my latest error dump:

[AMK2] MMM-AssistantMk2 Version: 3.1.0-3
0|mm | [AMK2] AssistantMk2 is initialized.
0|mm | [HOTWORD] begins.
0|mm | [HOTWORD] Detector starts listening.
0|mm | [HOTWORD] Detected: SMARTMIRROR
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] After Recording: 4000
0|mm | [HOTWORD] stops.
0|mm | [HOTWORD] After Recording finised. size: 16050
0|mm | [HOTWORD:B2W] WAV_FILE_CREATED: /home/pi/MagicMirror/modules/MMM-Hotword/temp/afterRecording.wav 16050
0|mm | [HOTWORD] Final Result: { detected: true,
0|mm | hotword: ‘SMARTMIRROR’,
0|mm | file: ‘temp/afterRecording.wav’ }

Seems to indicate Hotword is the problem…?

Here are the last 15 errors:

0|mm | at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/components/assistant.js:235:3)
0|mm | at Module._compile (internal/modules/cjs/loader.js:711:30)
0|mm | at Object.Module._extensions…js (internal/modules/cjs/loader.js:722:10)
0|mm | at Module.load (internal/modules/cjs/loader.js:620:32)
0|mm | { Error: Cannot find module ‘google-assistant’
0|mm | at Module._resolveFilename (internal/modules/cjs/loader.js:602:15)
0|mm | at Function.Module._resolveFilename (/home/pi/MagicMirror/node_modules/electron/dist/resources/electron.asar/common/reset-search-paths.js:35:12)
0|mm | at Function.Module._load (internal/modules/cjs/loader.js:528:25)
0|mm | at Module.require (internal/modules/cjs/loader.js:658:17)
0|mm | at require (internal/modules/cjs/helpers.js:20:18)
0|mm | at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/components/assistant.js:1:180)
0|mm | at Object. (/home/pi/MagicMirror/modules/MMM-AssistantMk2/components/assistant.js:235:3)
0|mm | at Module._compile (internal/modules/cjs/loader.js:711:30)
0|mm | at Object.Module._extensions…js (internal/modules/cjs/loader.js:722:10)
0|mm | at Module.load (internal/modules/cjs/loader.js:620:32) code: ‘MODULE_NOT_FOUND’ }

Triggered the module with MMM-Buttons, but when I do that, AssistantMk2 just waits and doesn’t listen, finally timing out.

@Croeder Howdy!

I’m also working with the Google AIY kit, so I have a little experience to share. First, version 2 of the AssistantMk2 module does not, repeat not, work with the Google Hat. Not at all. A couple of us have found that out, and even the developer suggests that there is something wacky about Google’s use of the ALSA drivers or somesuch.

The reason the moduel may not be showing up could be with having to rebuild electron, which is a documented bug for v2.

I can report that v3 of the module will install on the Google Hat just fine, and will give you all the feedback and test results you want… BUT… right now there seems to be a memory issue with the module and they are still working on it. You can see my part of the discussion here:

https://forum.magicmirror.builders/topic/11991/mmm-assistantmk2-v3/39

I have also tried installing the Google AIY assistant as a service, loading on startup. The advantage there is you get your LED button back. But I had issues with the responses cutting off and being weird, so I deinstalled that.

Just waiting for results on version 3 now…!

Thanks so much for the new version! I know you guys are working hard on this, I love the plugin, and I appreciate everything that went into making it.

Unfortunately I still have some more issues, and they seem memory related this time around as well. To recap: I’m on a Pi 3B with latest MM, but still on Raspbian Stretch.

I cleared out the old AssistantMk2 and pulled the new version from scratch. Though I used the autoinstaller, I still had to manually rebuild electron…? That’s OK, though, because it did work fine.

I used both recommended config options , namely
useHTML5: false,
playProgram: “mpg321”,

Everything seems OK, and my first test yesterday worked great. I asked a simple question, “What time is it?” and got the answer and all worked great. I left it on overnight, but today Hotword does not summon the plugin. I get these messages from PM2 log:

0|mm | [HOTWORD] Detector starts listening.
0|mm | [HOTWORD] Detected: SMARTMIRROR
0|mm | [HOTWORD] stops.
0|mm | [HOTWORD] After Recording is too short
0|mm | [HOTWORD] Final Result: { detected: true, hotword: ‘SMARTMIRROR’, file: null }

And the corresponding error messages:

0|mm | at Module._compile (internal/modules/cjs/loader.js:678:28)
0|mm | at Object.Module._extensions…js (internal/modules/cjs/loader.js:722:10)
0|mm | at Module.load (internal/modules/cjs/loader.js:620:32)
0|mm | at tryModuleLoad (internal/modules/cjs/loader.js:559:12)
0|mm | at Function.Module._load (internal/modules/cjs/loader.js:551:3)
0|mm | at Module.require (internal/modules/cjs/loader.js:658:17)
0|mm | at require (internal/modules/cjs/helpers.js:20:18)
0|mm | mmap() failed: Cannot allocate memory
0|mm | Failed to create permanent mapping for memfd region with ID = 775865347
0|mm | Failed to regester memfd mempool. Reason: could not attach memfd SHM ID to pipe
0|mm | mmap() failed: Cannot allocate memory
0|mm | Failed to create permanent mapping for memfd region with ID = 3943571786
0|mm | mmap() failed: Cannot allocate memory
0|mm | Failed to create permanent mapping for memfd region with ID = 2623177707
0|mm | Ignoring received block reference with non-registered memfd ID = 2623177707

I gave you all 15 lines in case there’s something there.

So I stopped MM entirely, started again. Hotword brought up AssistantMk2, but only once, with identical error messages. Once the plugin told me it did not understand, it was unresponsive to my voice.

@Bugsounet Sure! When I used the auto installer on my RPi3B+, running Raspbian Stretch (I list all these because maybe it’s OK on Buster, I do not know) the npm install created “node-v64-linux-arm-glibc” in the ~/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/node/extension_library/ directory

I know this because I’m running MM with PM2, and the PM2 logs clearly stated that the plugin was looking for “electron-v3.0-linux-arm-glibc” at that location and could not find it.

This was a well-documented bug with the v2 install process, and the commands that fixed it were

npm install --save-dev electron-rebuild
./node_modules/.bin/electron-rebuild

These are run from the MMM-AssistantMk2 directory AND the MMM-Hotword directory. Those were easy!

As to the memory issue, I went back just now to check the pm2 logs for errors, and yes, I see these entries:

0|mm | mmap() failed: Cannot allocate memory
0|mm | Failed to create permanent mapping for memfd region with ID = 3774675295
0|mm | Ignoring received block reference with non-registered memfd ID = 3774675295

But I was not sure if they were related to the npm install issue or not. I thought they might be leftover from when the proper grpc files were not there, because, frankly, I know nothing about this stuff.

And thirdly, yes, I did read about the proper config for the v3 version, and everything seems to be fine there. The plugin loads fine, even listens to me, but simply does not process the commands and then kind of locks up. The rest of the mirror runs fine, but the voice command is no longer responsive.

I set up MMM-Buttons to trigger the Assistant as well, and that also works to set off the listener, with exactly the same result - no parsing of the voice command, the eventual “Can’t understand this” dialogue box, and unresponsive until restart.

Congratulations, @Sean on a new build! The installation method went through like a charm, and I’m even happy to report that my build, a Pi3 with Google Hat, responded perfectly, even though v2 would not recognize it.

But I do have one snag trying to implement it.

Raspbian Stretch, Pi3B+, with Google Voice Hat, latest MM. Clean install of AssistantMk2, although I had an assistant.json and profile saved from a previous iteration. Installation went, as I say, beautifully, with the GCC 7 downgrade, proper assignment of the Google Hat input and output and everything. But MM would not load, and my logs showed the same gRPC module error we used to get on v2 from not having the right npm install.

So I followed the old instructions from that build, namely

npm install --save-dev electron-rebuild
./node_modules/.bin/electron-rebuild

in both the AssistantMk2 and Hotword directories, and that seems to have fixed the issue.

Now I’ve got it working, except it’s not working. The first “smartmirror” command engages the Assistant, the tone sounds, but no matter what I say, the module says it did not understand, goes away, and cannot be invoked again unless I restart MM entirely. Logs end with

[HOTWORD] Final Result: { detected: true,
0|mm | hotword: ‘SMARTMIRROR’,
0|mm | file: ‘temp/afterRecording.wav’ }

…but not a lot more info than that. Could be related to the mmap issue I know you are working on, but what do I know? So I thought I would report it.

@Sean
VERY interesting!

FWIW, there was NO issue with HAT previously! That’s why I thought it would be OK going forward with it. Dammit, I should have left well enough alone and not upgraded! Too late now, everything’s changed!

I don’t know enough about ALSA to fight my way out of a wet paper bag. But it is worth figuring out, because I’m not the only one who picked up a cheap HAT for $5 and figured it would work - the form factor is nice.

So I guess I’ll take this on and report back when I work it out. OR blow it off, connect some externals and get my Mirror done. Either way, thanks so much for the suggestion!

Re: MMM-Hotword
Hello, and hopefully hello to Sean, if you see this.

I have Hotword installed with MMM-AssistantMk2 on the latest build of MM2 (2.90), running on an RPi3. Everything except Hotword is running great! At one point, before upgrading everything, I had all this working fine, but now my pm2 logs show

0|mm  | [HOTWORD] Detector starts listening.
0|mm  | [HOTWORD] Final Result: { detected: false }
0|mm  | [HOTWORD] begins.

over and over in a loop… No attempt on my part gets it to recognize anything. I even removed all modules except Hotword (and the clock so I’d know it was running) and I get the same results.

I have deinstalled and reinstalled Hotword via the automatic script And the manual method. Installation appears to be fine, and components are there. Using this config:

{
			module: "MMM-Hotword",
			config: {
				chimeOnFinish:"resources/dong.wav",
				mic: {
				  recordProgram : "arecord",  
				  device        : "plughw:0",
				},
				models: [
				  {
					hotwords    : "smart_mirror",
					file        : "smart_mirror.umdl",
					sensitivity : "0.5",
				  },
				],
				defaultCommand: {
				  notificationExec: {
					notification: "ASSISTANT_ACTIVATE",
					payload: (detected, afterRecord) => {
					  return {profile:"default"}
					}
				  },
				  afterRecordLimit:0,
				  restart:false,
				},
			  },
		},

The line for “device : “plughw:0”,” is because I’m using the Google HAT and mic/speakers/etc. That’s not the issue, I don’t think, because the HAT has worked fine in the past with Hotword.

Please tell me I’m an idiot and that this is an easy fix I’m just not seeing right now.

Thanks, and thanks to Sean for all the good work on this module!

@sdetweil LIKE A CHARM. All is well with Assistant and Hotword, just as you describe. MMM-Buttons also required and npm install, as well as MMM-PIR-Sensor. So, basically, maintenance on all the plugins.

@sdetweil Thanks for these leads! Now that you mention it, why would I not need to do this… I’ll let you know how it goes…

Mine also not working on RPi3, and removing the exact same plugin does not remedy the situation. Sample config does make it behave, but it seems like there might be another plugin issue?

I’m running (or used to run)

Remote Control
DarkSky Forecast
Network Scanner
Volumio Playing
Roomba
Notification Trigger
Hotword
Assistant Mk2
Buttons
and PIR sensor.

I upgraded because Electron was leaking or whatever it does to slow to a crawl, and slushing back to a previous Electron had not fixed the problem. Too tired to try them all one at a time right now, but I guess I’ll go to it tomorrow.