MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working

davidoesch

ah – progress but not solved (sometimes a hike helps)

changed HOTWORD

in config.js

{
  module: "MMM-Hotword",
  config: {
	recipes: ["with-AMk2v3_smart-mirror.js"],
    chimeOnFinish: null,
    mic: {
      recordProgram: "arecord",
      device: "plughw:1"
    },

  }
},

and in modules\MMM-Hotword\recipes\with-AMk2v3_smart-mirror.js

var recipe = {
  models: [
    {
      hotwords    : "computer",
      file        : "computer.umdl",
      sensitivity : "0.5",
    },
  ],
  commands: {
    "computer": {
      notificationExec: {
        notification: "ASSISTANT_ACTIVATE",
        payload: (detected, afterRecord) => {
          var ret = {
            profile:"default",
            type: "MIC",
          }
          if (afterRecord) {
            ret.type = "WAVEFILE"
            ret.key = "modules/MMM-Hotword/" + afterRecord
          }
          return ret
        }
      },
      restart:false,
      afterRecordLimit: 7
    }
  }
}

exports.recipe = recipe // Don't remove this line.

now it works with computer

but now
result in log

[14:04:37.424] [LOG]    [HOTWORD] Detector starts listening.
[14:04:38.714] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:40.875] [LOG]    [HOTWORD] begins.
[14:04:40.971] [LOG]    [HOTWORD] Detector starts listening.
[14:04:41.478] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:42.023] [LOG]    [HOTWORD] begins.
[14:04:42.029] [LOG]    [HOTWORD] Detector starts listening.

seems to be related to a loop issue https://github.com/eouia/MMM-Hotword/issues/120 and the devs are working on it …

A Former User

@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:

[14:04:37.424] [LOG] [HOTWORD] Detector starts listening.
[14:04:38.714] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:40.875] [LOG] [HOTWORD] begins.
[14:04:40.971] [LOG] [HOTWORD] Detector starts listening.
[14:04:41.478] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:42.023] [LOG] [HOTWORD] begins.
[14:04:42.029] [LOG] [HOTWORD] Detector starts listening.

It happens when your mic is not working properly. To diagnose, full log is needed. What kind of mic are you using?

davidoesch

Hmm

the detector loop happens in one of ten cases-- expecially when I talk to fast

The mic was working flawlessly with the older version of Hotword and AssistantMk2

Nevertheless
the mic used https://www.aliexpress.com/item/32783207436.html?spm=a2g0s.9042311.0.0.57274c4dlSRFlb

arecord --list-devices
**** List of CAPTURE Hardware Devices ****
card 1: AK5371 [AK5371], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

 lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0556:0002 Asahi Kasei Microsystems Co., Ltd
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Another side effect after the update of HOTWORD and mmm-AssistantMk2 v3 : MMM-PIR – can not activate the screen anymore …I discovered that

dmesg
[ 2718.210637] Out of memory: Kill process 1860 (electron) score 572 or sacrifice child
[ 2718.210729] Killed process 1860 (electron) total-vm:604248kB, anon-rss:31300kB, file-rss:31676kB, shmem-rss:198652kB
[ 2718.247983] oom_reaper: reaped process 1860 (electron), now anon-rss:0kB, file-rss:0kB, shmem-rss:198564kB

and yes I rebuilt electron

after a rollback to the old version everything works smoothly

davidoesch

@Sean
here a larger the snippet of the session

09.295] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.297] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.298] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.300] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.302] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.304] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.305] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.308] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.310] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.311] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.313] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.315] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.317] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.319] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.320] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.322] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.324] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.326] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.328] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.332] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.335] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.338] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.340] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.342] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.344] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.351] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.354] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.358] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 1198
[14:04:09.361] [LOG]    [AMK2:AS] CONVERSATION_ALL_RESPONSES_RECEIVED
[14:04:09.363] [LOG]    [AMK2:AS] CONVERSATION_END:COMPLETED
[14:04:09.363] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_MAKING
[14:04:09.370] [LOG]    [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:09.371] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.499] [LOG]    [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:10.501] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.658] [LOG]    [AMK2] { session: null,
  error: null,
  action: { requestId: '5e23e8fb-0000-246c-a6f8-f4f5e8048f8c' },
  text: null,
  screen:
   { help: [ 'Und morgen?', 'Und nächste Woche?' ],
     trysay: 'Sag zum Beispiel …',
     links:
      [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ],
     text:
      'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°',
     photos: [],
     path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html',
     uri: 'tmp/lastScreenOutput.html' },
  audio:
   { path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav',
     uri: 'tmp/lastResponse.wav' },
  transcription: { transcription: 'Wetter vor', done: true },
  continue: false,
  lastQuery:
   { type: 'TEXT',
     profile: { profileFile: 'default.json', lang: 'de-DE' },
     key: 'Wetter vor',
     lang: null,
     useScreenOutput: true,
     useAudioOutput: true,
     session: null,
     status: 'continue' } }
[14:04:10.738] [LOG]    [AMK2] { session: null,
  error: null,
  action: { requestId: '5e23e6d7-0000-2ad6-87f4-089e082975cc' },
  text: null,
  screen:
   { help: [ 'Und morgen?', 'Und nächste Woche?' ],
     trysay: 'Sag zum Beispiel …',
     links:
      [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ],
     text:
      'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°',
     photos: [],
     path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html',
     uri: 'tmp/lastScreenOutput.html' },
  audio:
   { path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav',
     uri: 'tmp/lastResponse.wav' },
  transcription: { transcription: 'Wetter vor', done: true },
  continue: false,
  lastQuery:
   { type: 'TEXT',
     profile: { profileFile: 'default.json', lang: 'de-DE' },
     key: 'Wetter vor',
     lang: null,
     useScreenOutput: true,
     useAudioOutput: true,
     session: null,
     status: 'think' } }
[14:04:11.144] [LOG]    [HOTWORD] begins.
[14:04:11.146] [LOG]    [HOTWORD] Detector starts listening.
[14:04:11.496] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:11.718] [LOG]    [HOTWORD] begins.
[14:04:11.724] [LOG]    [HOTWORD] Detector starts listening.
[14:04:11.772] [LOG]    [HOTWORD] Final Result: { detected: false }

A Former User

@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:

[14:04:09.371] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.499] [LOG] [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:10.501] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav

I don’t know why at this moment, but AMK2 seemed to be activated twice. That makes your issue. Can you send me whole log (from the start of MM)? (eouia0819@gmail.com)

Artech

This post is deleted!

A Former User

@davidoesch
We found there could be potential memory issue, so we are trying to solve this issue by using smaller memory. I hope this could solve your issue also. Wait a while, sorry.