Kalliope assistant + MM

marcnlx · Mar 17, 2018, 1:04 PM

Hi,

I really love the Kalliope Virtual Assistant. I used the Manual Install (Method #2) option, since the one-liner command failed on my RPi.

I managed to install Kalliope (“Kalliope version”: “0.5.1b”), but I have some issues having all the messages show in MMM-Kalliope module (in upper_third) on the mirror.

I have added this to my ~/MagicMirror/config/config.js file:

{
    module: "MMM-kalliope",
    position: "upper_third",
                        config: {
                                max: "5",
                                keep_seconds: "8",
                                title: "Kalliope",
                                notification: "KALLIOPE",
                                payload: "my message"
                                }

}

I have added this to my ~/kalliope_starter_en/brain.yml file:

- name: "mm-say"
  signals: []
  neurons:
    - magic_mirror:
        mm_url: "http://127.0.0.1:8080/kalliope"
        notification: "KALLIOPE"
        payload: "{{ kalliope_memory['kalliope_last_tts_message'] }}"

- name: "mm-alert"
  signals:
    - order: "send an alert to magic mirror"
  neurons:
    - magic_mirror:
        mm_url: "http://127.0.0.1:8080/kalliope"
        notification: "SHOW_ALERT"
        payload:
          title: "my alert title"
          message: "the message"
          timer: 5000

And I have added this to the hooks in my ~/kalliope_starter_en/settings.yml file:

#  on_start_speaking:
  on_start_speaking: "mm-say"

I’m not sure what I am missing here…would anybody be able to help me out?

I also sometimes have an issue after Kalliope gets triggered by the WakeUp word related to the microphone/audio (but this might also be caused by the mic on the usb webcam I am using…):

what can i do for you?
Threshold set to: 4000
Say something!
Google Speech Recognition could not understand audio
I don't recognize that order
Waiting for trigger detection
yes sir?
Threshold set to: 4000
Say something!
Google Speech Recognition could not understand audio
I don't know this order
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/signals/order/order.py", line 84, in run
    self.start_trigger()
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 382, in trigger
    return self.machine._process(func)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 1020, in _process
    self._transition_queue[0]()
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 398, in _trigger
    return self._process(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 408, in _process
    if trans.execute(event_data):
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 262, in execute
    self._change_state(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 273, in _change_state
    event_data.machine.get_state(self.dest).enter(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 119, in enter
    event_data.machine.callback(handle, event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 990, in callback
    func(*event_data.args, **event_data.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/signals/order/order.py", line 92, in start_trigger_process
    self.trigger_instance = TriggerLauncher.get_trigger(settings=self.settings, callback=self.trigger_callback)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/core/TriggerLauncher.py", line 32, in get_trigger
    parameters=trigger.parameters)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/core/Utils/Utils.py", line 140, in get_dynamic_class_instantiation
    return klass(**parameters)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboy.py", line 51, in __init__
    sleep_time=0.03)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboydecoder.py", line 98, in __init__
    stream_callback=audio_callback)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable

When starting kalliope with the –debug option, I see lines like this in the debug log:

2018-03-17 13:55:30 :: kalliope-0.5.1b :: Magic_mirror call Magic Mirror MMM-kalliope-API: http://127.0.0.1:8080/kalliope
2018-03-17 13:55:31 :: kalliope-0.5.1b :: Magic_mirrorcannot get a valid json from returned content

2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from {'notification': 'KALLIOPE', 'mm_url': 'http://127.0.0.1:8080/kalliope', 'is_api_call': True, 'payload': "{{kalliope_memory['kalliope_last_tts_message']}}", 'no_voice': False}, using {}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from KALLIOPE, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from http://127.0.0.1:8080/kalliope, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from True, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from {{kalliope_memory['kalliope_last_tts_message']}}, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from False, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}, 'name': 'sir'}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Run neuron: "{'name': 'magic_mirror', 'parameters': {'notification': 'KALLIOPE', 'mm_url': 'http://127.0.0.1:8080/kalliope', 'is_api_call': True, 'payload': 'what can i do for you?', 'no_voice': False}}"
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [Utils]-> get_dynamic_class_instantiation : package path : kalliope.neurons.magic_mirror.magic_mirror
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [Utils]-> get_dynamic_class_instantiation : loading path : resources/neurons/magic_mirror/magic_mirror.py, as package Magic_mirror
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronModule] TTS args: {'name': 'pico2wave', 'parameters': {'cache': True, 'language': 'en-US'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Magic_mirror call Magic Mirror MMM-kalliope-API: http://127.0.0.1:8080/kalliope
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Magic_mirrorcannot get a valid json from returned content
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronModule] Say() called with message: {'content': '<!DOCTYPE html>\n<html lang="en">\n<head>\n<meta charset="utf-8">\n<title>Error</title>\n</head>\n<body>\n<pre>Cannot POST /kalliope</pre>\n</body>\n</html>\n', 'status_code': 404, 'response_header': {'Content-Length': '148', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'Content-Type': 'text/html; charset=utf-8', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'Connection': 'keep-alive', 'Content-Security-Policy': "default-src 'self'", 'Date': 'Sat, 17 Mar 2018 12:55:09 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'X-DNS-Prefetch-Control': 'off'}}

Accessing the URL http://192.168.1.16:8080/kalliope, gives me this:

Cannot GET /kalliope

I also see the following error using the CURL example (removed the “<” from the beginning on the output of each line):

pi@mirror:~ $ curl -H "Content-Type: application/json" -X POST -d '{"notification":"KALLIOPE", "payload": "my message"}' http://127.0.0.1:8080/kalliope      
!DOCTYPE html>
html lang="en">
head>
meta charset="utf-8">
title>Error</title>
/head>
body>
pre>Cannot POST /kalliope</pre>
/body>
/html>
pi@mirror:~ $

sispheor · Mar 17, 2018, 1:31 PM

@marcnlx why are you trying to reach the URL from your browser? It’s an API. It should be accessed from a script only.

marcnlx · Mar 17, 2018, 1:36 PM

@sispheor said in Kalliope assistant + MM:

@marcnlx why are you trying to reach the URL from your browser? It’s an API. It should be accessed from a script only.

The CURL example gave me this output on the CLI. I was only using my browser to make sure I was able to reach the API.

Any idea what configuration mistake I could have made?

sispheor · Mar 17, 2018, 3:30 PM

@marcnlx @marcnlx you’ve set http://127.0.0.1:8080/kalliope for the neuron but the mirror up is 192.168.1.16. doesn’t it?

Also, notification and payload are not needed in the config of the module.

marcnlx · Mar 17, 2018, 5:34 PM

@sispheor said in Kalliope assistant + MM:

@marcnlx @marcnlx you’ve set http://127.0.0.1:8080/kalliope for the neuron but the mirror up is 192.168.1.16. doesn’t it?

Also, notification and payload are not needed in the config of the module.

@sispheor 127.0.0.1 is the localhost IP, 192.168.1.16 is the external IP…the Magic Mirror is available on every IP in use on the RPi (listening address is set to 0.0.0.0).

I just commented out the notification and payload and thing are now showing on the Mirror! This is really cool! :)

Okay…now I need to figure out how to build the whole virtual assistant…on to “phase #2”.
In your demo video I saw a basically complete brain…is there any place where I can find that one? ;)

sispheor · Mar 17, 2018, 5:50 PM

@marcnlx take a look to starter kits. It contains example. Also there is an example of usage of each neuron on the website.
Have fun !

marcnlx · Mar 17, 2018, 5:52 PM

@sispheor said in Kalliope assistant + MM:

@marcnlx take a look to starter kits. It contains example. Also there is an example of usage of each neuron on the website.
Have fun !

I will! I’m going to check everything out later today, thanks!

marcnlx · Mar 18, 2018, 11:45 AM

I tested Kalliope yesterday night and once you know how it works, it’s pretty sweet! ;)

Unfortunately I noticed some bugs/issues which are discussed multiple times on Github, so for now I think I will wait until a later release since I need to restart Kalliope multiple times after 2-3 phrases because of pulseaudio issues which I can’t seem to fix:

2018-03-18 12:42:27 :: kalliope-0.5.1b :: File found in /home/pi/kalliope_starter_en/trigger/kalliope-EN-12samples.pmdl
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()

  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboydecoder.py", line 98, in __init__
    stream_callback=audio_callback)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable

sispheor · Mar 18, 2018, 12:48 PM

@marcnlx Uninstall pulse audio and work only with alsa.
We’ve remove pulseaudio from our doc.

sispheor · May 13, 2018, 6:03 PM

Kalliope v0.5.1 has been released with all stuff needed to integrate Magic Mirror.