MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working

davidoesch · Jan 19, 2020, 1:13 PM

ah – progress but not solved (sometimes a hike helps)

changed HOTWORD

in config.js

{
  module: "MMM-Hotword",
  config: {
	recipes: ["with-AMk2v3_smart-mirror.js"],
    chimeOnFinish: null,
    mic: {
      recordProgram: "arecord",
      device: "plughw:1"
    },

  }
},

and in modules\MMM-Hotword\recipes\with-AMk2v3_smart-mirror.js

var recipe = {
  models: [
    {
      hotwords    : "computer",
      file        : "computer.umdl",
      sensitivity : "0.5",
    },
  ],
  commands: {
    "computer": {
      notificationExec: {
        notification: "ASSISTANT_ACTIVATE",
        payload: (detected, afterRecord) => {
          var ret = {
            profile:"default",
            type: "MIC",
          }
          if (afterRecord) {
            ret.type = "WAVEFILE"
            ret.key = "modules/MMM-Hotword/" + afterRecord
          }
          return ret
        }
      },
      restart:false,
      afterRecordLimit: 7
    }
  }
}

exports.recipe = recipe // Don't remove this line.

now it works with computer

but now
result in log

[14:04:37.424] [LOG]    [HOTWORD] Detector starts listening.
[14:04:38.714] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:40.875] [LOG]    [HOTWORD] begins.
[14:04:40.971] [LOG]    [HOTWORD] Detector starts listening.
[14:04:41.478] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:42.023] [LOG]    [HOTWORD] begins.
[14:04:42.029] [LOG]    [HOTWORD] Detector starts listening.

seems to be related to a loop issue https://github.com/eouia/MMM-Hotword/issues/120 and the devs are working on it …

A Former User · Jan 19, 2020, 3:27 PM

@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:

[14:04:37.424] [LOG] [HOTWORD] Detector starts listening.
[14:04:38.714] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:40.875] [LOG] [HOTWORD] begins.
[14:04:40.971] [LOG] [HOTWORD] Detector starts listening.
[14:04:41.478] [LOG] [HOTWORD] Final Result: { detected: false }
[14:04:42.023] [LOG] [HOTWORD] begins.
[14:04:42.029] [LOG] [HOTWORD] Detector starts listening.

It happens when your mic is not working properly. To diagnose, full log is needed. What kind of mic are you using?

davidoesch · Jan 19, 2020, 5:09 PM

Hmm

the detector loop happens in one of ten cases-- expecially when I talk to fast

The mic was working flawlessly with the older version of Hotword and AssistantMk2

Nevertheless
the mic used https://www.aliexpress.com/item/32783207436.html?spm=a2g0s.9042311.0.0.57274c4dlSRFlb

arecord --list-devices
**** List of CAPTURE Hardware Devices ****
card 1: AK5371 [AK5371], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

 lsusb
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 0556:0002 Asahi Kasei Microsystems Co., Ltd
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Another side effect after the update of HOTWORD and mmm-AssistantMk2 v3 : MMM-PIR – can not activate the screen anymore …I discovered that

dmesg
[ 2718.210637] Out of memory: Kill process 1860 (electron) score 572 or sacrifice child
[ 2718.210729] Killed process 1860 (electron) total-vm:604248kB, anon-rss:31300kB, file-rss:31676kB, shmem-rss:198652kB
[ 2718.247983] oom_reaper: reaped process 1860 (electron), now anon-rss:0kB, file-rss:0kB, shmem-rss:198564kB

and yes I rebuilt electron

after a rollback to the old version everything works smoothly

davidoesch · Jan 19, 2020, 6:00 PM

@Sean
here a larger the snippet of the session

09.295] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.297] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.298] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.300] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.302] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.304] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.305] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.308] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.310] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.311] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.313] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.315] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.317] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.319] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.320] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.322] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.324] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.326] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.328] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.332] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.335] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.338] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.340] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.342] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.344] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.351] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.354] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 4000
[14:04:09.358] [LOG]    [AMK2:AS] CONVERSATION:AUDIO 1198
[14:04:09.361] [LOG]    [AMK2:AS] CONVERSATION_ALL_RESPONSES_RECEIVED
[14:04:09.363] [LOG]    [AMK2:AS] CONVERSATION_END:COMPLETED
[14:04:09.363] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_MAKING
[14:04:09.370] [LOG]    [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:09.371] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.499] [LOG]    [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:10.501] [LOG]    [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.658] [LOG]    [AMK2] { session: null,
  error: null,
  action: { requestId: '5e23e8fb-0000-246c-a6f8-f4f5e8048f8c' },
  text: null,
  screen:
   { help: [ 'Und morgen?', 'Und nächste Woche?' ],
     trysay: 'Sag zum Beispiel …',
     links:
      [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ],
     text:
      'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°',
     photos: [],
     path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html',
     uri: 'tmp/lastScreenOutput.html' },
  audio:
   { path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav',
     uri: 'tmp/lastResponse.wav' },
  transcription: { transcription: 'Wetter vor', done: true },
  continue: false,
  lastQuery:
   { type: 'TEXT',
     profile: { profileFile: 'default.json', lang: 'de-DE' },
     key: 'Wetter vor',
     lang: null,
     useScreenOutput: true,
     useAudioOutput: true,
     session: null,
     status: 'continue' } }
[14:04:10.738] [LOG]    [AMK2] { session: null,
  error: null,
  action: { requestId: '5e23e6d7-0000-2ad6-87f4-089e082975cc' },
  text: null,
  screen:
   { help: [ 'Und morgen?', 'Und nächste Woche?' ],
     trysay: 'Sag zum Beispiel …',
     links:
      [ 'http://www.weather.com/wx/today/?lat=51.51&lon=-0.08&locale=en_US&par=google' ],
     text:
      'Sonntag in London\nSonnig · Ndschlg.: 0 % · Feuchtigkeit: 68 ·\nweather.com\n7\n°C\n↑ 8°\n↓ -2°\nMo\n7° -1°\nDi\n6° 1°\nMi\n8° 4°\nDo\n8° 3°\nFr\n7° 4°',
     photos: [],
     path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastScreenOutput.html',
     uri: 'tmp/lastScreenOutput.html' },
  audio:
   { path:
      '/home/pi/MagicMirror/modules/MMM-AssistantMk2/tmp/lastResponse.wav',
     uri: 'tmp/lastResponse.wav' },
  transcription: { transcription: 'Wetter vor', done: true },
  continue: false,
  lastQuery:
   { type: 'TEXT',
     profile: { profileFile: 'default.json', lang: 'de-DE' },
     key: 'Wetter vor',
     lang: null,
     useScreenOutput: true,
     useAudioOutput: true,
     session: null,
     status: 'think' } }
[14:04:11.144] [LOG]    [HOTWORD] begins.
[14:04:11.146] [LOG]    [HOTWORD] Detector starts listening.
[14:04:11.496] [LOG]    [HOTWORD] Final Result: { detected: false }
[14:04:11.718] [LOG]    [HOTWORD] begins.
[14:04:11.724] [LOG]    [HOTWORD] Detector starts listening.
[14:04:11.772] [LOG]    [HOTWORD] Final Result: { detected: false }

A Former User · Jan 20, 2020, 7:44 PM

@davidoesch said in MMM-AssistantMk2 v3 and MMM-Hotword with custom hotword not working:

[14:04:09.371] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav
[14:04:10.499] [LOG] [AMK2:BW] RESPONSE_WAV_FILE_CREATED
[14:04:10.501] [LOG] [AMK2:AS] CONVERSATION_PP:RESPONSE_AUDIO_CREATED tmp/lastResponse.wav

I don’t know why at this moment, but AMK2 seemed to be activated twice. That makes your issue. Can you send me whole log (from the start of MM)? (eouia0819@gmail.com)

Artech · Jan 21, 2020, 4:43 PM

This post is deleted!

A Former User · Jan 23, 2020, 7:58 AM

@davidoesch
We found there could be potential memory issue, so we are trying to solve this issue by using smaller memory. I hope this could solve your issue also. Wait a while, sorry.