Read the statement by Michael Teeuw here.
MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working
-
Hi
first: MMM-AssistantMK2 is awesome thanks to the devs ( my wife and kids love it) I just upgraded both moduels to the last version --but I cant get it to work.
Did remove both modules and installed it new, copied credentials and profilesplatform:
raspberrypi 4 , latest buster with all latest updateds** steps to reproduce**
.config.js/* Magic Mirror Config Sample * * By Michael Teeuw http://michaelteeuw.nl * MIT Licensed. * * For more information how you can configurate this file * See https://github.com/MichMich/MagicMirror#configuration * */ var config = { address: "0.0.0.0", // Address to listen on, can be: // - "localhost", "127.0.0.1", "::1" to listen on loopback interface // - another specific IPv4/6 to listen on a specific interface // - "", "0.0.0.0", "::" to listen on any interface // Default, when address config is left out, is "localhost" port: 8080, ipWhitelist: [], // Set [] to allow all IP addresses // or add a specific IPv4 of 192.168.1.5 : // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.1.5"], // or IPv4 range of 192.168.3.0 --> 192.168.3.15 use CIDR format : // ["127.0.0.1", "::ffff:127.0.0.1", "::1", "::ffff:192.168.3.0/28"], language: "de", timeFormat: 24, units: "metric", modules: [ { module: "alert", }, { module: "updatenotification", position: "top_bar" }, { module: "clock", position: "top_bar" }, { module: "compliments", position: "lower_third" }, { module: "MMM-AssistantMk2", position: "top_left", config: { debug:true, ui: "Classic", // or "Classic", "Classic2" assistantConfig: { projectId: "", // Required to use gaction. modelId: "", // (OPTIONAL for gaction) instanceId: "", // (OPTIONAL for gaction) latitude: 51.508530, longitude: -0.076132, }, responseConfig: { useScreenOutput: true, useAudioOutput: true, useChime: true, timer: 5000, myMagicWord: false }, micConfig: { // put there configuration generated by auto-installer recorder: "arecord", device: "plughw:1", }, customActionConfig: { autoMakeAction: false, autoUpdateAction: false, // in RPI, gaction CLI might have some trouble.(current version should be 2.2.4, but for linux-arm, Google haven't updated) so leave this as false in RPI. I don't know it is solved or not. actionLocale: "en-US", // At this moment, multi-languages are not supported, sorry. Someday I'll work. }, recipes: [ "with-MMM-Hotword.js", "with-MMM-TelegramBot.js" ], profiles: { "default": { profileFile: "default.json", lang: "de-DE" } }, }, }, { module: "MMM-Hotword", config: { recipes: ["with-AMk2v3_smart-mirror.js"], chimeOnFinish: null, mic: { recordProgram: "arecord", device: "plughw:1" }, models: [ { hotwords : "computer", file : "computer.umdl", sensitivity : "0.5", }, ], commands: { "computer": { notificationExec: { notification: "ASSISTANT_ACTIVATE", payload: (detected, afterRecord) => { return {profile:"default"}; } }, restart:true, afterRecordLimit:0 } } } }, ] }; /*************** DO NOT EDIT THE LINE BELOW ***************/ if (typeof module !== "undefined") {module.exports = config;}
result:
magic mirror does start.
I can speak “computer” and some words like “wetter in zürich” there is a beep and the mmm-assistantMK2 V3 logo starts to be animated and it looks like it expects some input but nothing happensexpected result:
after speaking "computer " and some words the google assistant starts respondinglog:
[11:13:22.334] [LOG] Server started ... [11:13:22.335] [LOG] Connecting socket for: updatenotification [11:13:22.336] [LOG] Connecting socket for: MMM-AssistantMk2 [11:13:22.337] [LOG] Connecting socket for: MMM-Hotword [11:13:22.338] [LOG] [HOTWORD] MMM-Hotword starts [11:13:22.339] [LOG] Sockets connected & modules started ... [11:13:22.620] [LOG] Launching application. [11:13:25.748] [LOG] [HOTWORD] Recipe is loaded: with-AMk2v3_smart-mirror.js [11:13:25.836] [LOG] [AMK2] MMM-AssistantMk2 Version: 3.0.0 [11:13:25.841] [LOG] [AMK2] RECIPE_LOADED: with-MMM-Hotword.js [11:13:25.845] [LOG] [AMK2] RECIPE_LOADED: with-MMM-TelegramBot.js [11:13:25.846] [LOG] [AMK2] NO_ACTION_TO_MANAGE [11:13:25.871] [LOG] [AMK2] AssistantMk2 v3 is initialized. [11:13:25.907] [LOG] [HOTWORD] begins. [11:13:25.908] [LOG] [HOTWORD] Detector starts listening. [11:13:25.941] [LOG] [AMK2] tmp directory is now cleaned. [11:13:27.739] [LOG] [HOTWORD] Detected: computer [11:13:27.741] [LOG] [HOTWORD] stops. [11:13:27.747] [LOG] [HOTWORD] Final Result: { detected: true, hotword: 'computer', file: null } [11:13:27.831] [LOG] [AMK2] QUERY: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: null, lang: null, useScreenOutput: true, useAudioOutput: true, session: 1579428807774, status: 'standby', secretMode: false, sayMode: false } [11:13:28.816] [LOG] [AMK2:AS] CONVERSATION:RESPONSE [11:13:28.891] [LOG] [HOTWORD] begins. [11:13:28.892] [LOG] [HOTWORD] Detector starts listening. [11:13:58.729] [LOG] [AMK2:AS] CONVERSATION:RESPONSE [11:13:58.737] [LOG] [AMK2:AS] CONVERSATION_ERROR : { Error: 14 UNAVAILABLE: Service unavailable. at Object.exports.createStatusError (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/common.js:91:15) at ClientDuplexStream._emitStatusIfDone (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client.js:233:26) at ClientDuplexStream._receiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client.js:211:8) at Object.onReceiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:1306:15) at InterceptingListener._callNext (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:618:8) at /home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:1123:18 code: 14, metadata: Metadata { _internal_repr: { 'content-disposition': [Array] }, flags: 0 }, details: 'Service unavailable.' } [11:13:58.741] [LOG] [AMK2:AS] >> This error might happen when improper configuration or invalid Mic setup. [11:13:58.743] [LOG] [AMK2] { session: 1579428807774, error: 'NO_RESPONSE', action: null, text: null, screen: null, audio: null, transcription: null, continue: false, lastQuery: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: null, lang: null, useScreenOutput: true, useAudioOutput: true, session: 1579428807774, status: 'standby', secretMode: false, sayMode: false } } [11:18:03.671] [LOG] [HOTWORD] Detected: computer [11:18:03.673] [LOG] [HOTWORD] stops. [11:18:03.675] [LOG] [HOTWORD] Final Result: { detected: true, hotword: 'computer', file: null } [11:18:04.010] [LOG] [AMK2] QUERY: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: null, lang: null, useScreenOutput: true, useAudioOutput: true, session: 1579429083751, status: 'standby', secretMode: false, sayMode: false } [11:18:04.592] [LOG] [AMK2:AS] CONVERSATION:RESPONSE [11:18:04.921] [LOG] [HOTWORD] begins. [11:18:04.922] [LOG] [HOTWORD] Detector starts listening. [11:18:34.579] [LOG] [AMK2:AS] CONVERSATION_ERROR : { Error: 14 UNAVAILABLE: Service unavailable. at Object.exports.createStatusError (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/common.js:91:15) at ClientDuplexStream._emitStatusIfDone (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client.js:233:26) at ClientDuplexStream._receiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client.js:211:8) at Object.onReceiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:1306:15) at InterceptingListener._callNext (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus (/home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:618:8) at /home/pi/MagicMirror/modules/MMM-AssistantMk2/node_modules/grpc/src/client_interceptors.js:1123:18 code: 14, metadata: Metadata { _internal_repr: { 'content-disposition': [Array] }, flags: 0 }, details: 'Service unavailable.' } [11:18:34.584] [LOG] [AMK2:AS] >> This error might happen when improper configuration or invalid Mic setup. [11:18:34.584] [LOG] [AMK2] { session: 1579429083751, error: 'NO_RESPONSE', action: null, text: null, screen: null, audio: null, transcription: null, continue: false, lastQuery: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: null, lang: null, useScreenOutput: true, useAudioOutput: true, session: 1579429083751, status: 'standby', secretMode: false, sayMode: false } }
Any help would be appreciated… AFAIK it must be a hotword config I messed up. after spending now 4 hours tweaking I reach out to the community ( pls make me and my kids happy :-) )
-
ah – progress but not solved (sometimes a hike helps)
changed HOTWORD
in config.js
{ module: "MMM-Hotword", config: { recipes: ["with-AMk2v3_smart-mirror.js"], chimeOnFinish: null, mic: { recordProgram: "arecord", device: "plughw:1" }, } },
and in modules\MMM-Hotword\recipes\with-AMk2v3_smart-mirror.js
var recipe = { models: [ { hotwords : "computer", file : "computer.umdl", sensitivity : "0.5", }, ], commands: { "computer": { notificationExec: { notification: "ASSISTANT_ACTIVATE", payload: (detected, afterRecord) => { var ret = { profile:"default", type: "MIC", } if (afterRecord) { ret.type = "WAVEFILE" ret.key = "modules/MMM-Hotword/" + afterRecord } return ret } }, restart:false, afterRecordLimit: 7 } } } exports.recipe = recipe // Don't remove this line.
now it works with computer
but now
result in log[14:04:37.424] [LOG] [HOTWORD] Detector starts listening. [14:04:38.714] [LOG] [HOTWORD] Final Result: { detected: false } [14:04:40.875] [LOG] [HOTWORD] begins. [14:04:40.971] [LOG] [HOTWORD] Detector starts listening. [14:04:41.478] [LOG] [HOTWORD] Final Result: { detected: false } [14:04:42.023] [LOG] [HOTWORD] begins. [14:04:42.029] [LOG] [HOTWORD] Detector starts listening.
seems to be related to a loop issue https://github.com/eouia/MMM-Hotword/issues/120 and the devs are working on it …
-
@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:
[14:04:37.424] [LOG] [HOTWORD] Detector starts listening.
[14:04:38.714] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:40.875] [LOG] [HOTWORD] begins.
[14:04:40.971] [LOG] [HOTWORD] Detector starts listening.
[14:04:41.478] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:42.023] [LOG] [HOTWORD] begins.
[14:04:42.029] [LOG] [HOTWORD] Detector starts listening.It happens when your mic is not working properly. To diagnose, full log is needed. What kind of mic are you using?
-
Hmm
the detector loop happens in one of ten cases-- expecially when I talk to fast
The mic was working flawlessly with the older version of Hotword and AssistantMk2
Nevertheless
the mic used https://www.aliexpress.com/item/32783207436.html?spm=a2g0s.9042311.0.0.57274c4dlSRFlbarecord --list-devices **** List of CAPTURE Hardware Devices **** card 1: AK5371 [AK5371], device 0: USB Audio [USB Audio] Subdevices: 0/1 Subdevice #0: subdevice #0 lsusb Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub Bus 001 Device 003: ID 0556:0002 Asahi Kasei Microsystems Co., Ltd Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Another side effect after the update of HOTWORD and mmm-AssistantMk2 v3 : MMM-PIR – can not activate the screen anymore …I discovered that
dmesg [ 2718.210637] Out of memory: Kill process 1860 (electron) score 572 or sacrifice child [ 2718.210729] Killed process 1860 (electron) total-vm:604248kB, anon-rss:31300kB, file-rss:31676kB, shmem-rss:198652kB [ 2718.247983] oom_reaper: reaped process 1860 (electron), now anon-rss:0kB, file-rss:0kB, shmem-rss:198564kB
and yes I rebuilt electron
after a rollback to the old version everything works smoothly
-
@Sean
here a larger the snippet of the session09.295] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.297] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.298] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.300] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.302] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.304] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.305] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.308] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.310] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.311] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.313] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.315] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.317] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.319] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.320] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.322] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.324] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.326] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.328] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.332] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.335] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.338] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.340] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.342] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.344] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.351] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.354] [LOG] [AMK2:AS] CONVERSATION:AUDIO 4000 [14:04:09.358] [LOG] [AMK2:AS] CONVERSATION:AUDIO 1198 [14:04:09.361] [LOG] [AMK2:AS] CONVERSATION_ALL_RESPONSES_RECEIVED [14:04:09.363] [LOG] [AMK2:AS] CONVERSATION_END:COMPLETED [14:04:09.363] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_MAKING [14:04:09.370] [LOG] [AMK2:BW] RESPONSE_WAV_FILE_CREATED [14:04:09.371] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav [14:04:10.499] [LOG] [AMK2:BW] RESPONSE_WAV_FILE_CREATED [14:04:10.501] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav [14:04:10.658] [LOG] [AMK2] { session: null, error: null, action: { requestId: '5e23e8fb-0000-246c-a6f8-f4f5e8048f8c' }, text: null, screen: { help: [ 'Und morgen?', 'Und nächste Woche?' ], trysay: 'Sag zum Beispiel …', links: [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ], text: 'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°', photos: [], path: '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html', uri: 'tmp/lastScreenOutput.html' }, audio: { path: '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav', uri: 'tmp/lastResponse.wav' }, transcription: { transcription: 'Wetter vor', done: true }, continue: false, lastQuery: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: 'Wetter vor', lang: null, useScreenOutput: true, useAudioOutput: true, session: null, status: 'continue' } } [14:04:10.738] [LOG] [AMK2] { session: null, error: null, action: { requestId: '5e23e6d7-0000-2ad6-87f4-089e082975cc' }, text: null, screen: { help: [ 'Und morgen?', 'Und nächste Woche?' ], trysay: 'Sag zum Beispiel …', links: [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ], text: 'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°', photos: [], path: '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html', uri: 'tmp/lastScreenOutput.html' }, audio: { path: '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav', uri: 'tmp/lastResponse.wav' }, transcription: { transcription: 'Wetter vor', done: true }, continue: false, lastQuery: { type: 'TEXT', profile: { profileFile: 'default.json', lang: 'de-DE' }, key: 'Wetter vor', lang: null, useScreenOutput: true, useAudioOutput: true, session: null, status: 'think' } } [14:04:11.144] [LOG] [HOTWORD] begins. [14:04:11.146] [LOG] [HOTWORD] Detector starts listening. [14:04:11.496] [LOG] [HOTWORD] Final Result: { detected: false } [14:04:11.718] [LOG] [HOTWORD] begins. [14:04:11.724] [LOG] [HOTWORD] Detector starts listening. [14:04:11.772] [LOG] [HOTWORD] Final Result: { detected: false }
-
@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:
[14:04:09.371] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.499] [LOG] [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:10.501] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wavI don’t know why at this moment, but AMK2 seemed to be activated twice. That makes your issue. Can you send me whole log (from the start of MM)? (eouia0819@gmail.com)
-
This post is deleted! -
@davidoesch
We found there could be potential memory issue, so we are trying to solve this issue by using smaller memory. I hope this could solve your issue also. Wait a while, sorry.