Kalliope assistant + MM

[E3V3A]

@sispheor Cool! Thanks a lot for clarifying.

[marcnlx]

Hi,

I really love the Kalliope Virtual Assistant. I used the Manual Install (Method #2) option, since the one-liner command failed on my RPi.

I managed to install Kalliope (“Kalliope version”: “0.5.1b”), but I have some issues having all the messages show in MMM-Kalliope module (in upper_third) on the mirror.

I have added this to my ~/MagicMirror/config/config.js file:

{
    module: "MMM-kalliope",
    position: "upper_third",
                        config: {
                                max: "5",
                                keep_seconds: "8",
                                title: "Kalliope",
                                notification: "KALLIOPE",
                                payload: "my message"
                                }

}

I have added this to my ~/kalliope_starter_en/brain.yml file:

- name: "mm-say"
  signals: []
  neurons:
    - magic_mirror:
        mm_url: "http://127.0.0.1:8080/kalliope"
        notification: "KALLIOPE"
        payload: "{{ kalliope_memory['kalliope_last_tts_message'] }}"

- name: "mm-alert"
  signals:
    - order: "send an alert to magic mirror"
  neurons:
    - magic_mirror:
        mm_url: "http://127.0.0.1:8080/kalliope"
        notification: "SHOW_ALERT"
        payload:
          title: "my alert title"
          message: "the message"
          timer: 5000

And I have added this to the hooks in my ~/kalliope_starter_en/settings.yml file:

#  on_start_speaking:
  on_start_speaking: "mm-say"

I’m not sure what I am missing here…would anybody be able to help me out?

I also sometimes have an issue after Kalliope gets triggered by the WakeUp word related to the microphone/audio (but this might also be caused by the mic on the usb webcam I am using…):

what can i do for you?
Threshold set to: 4000
Say something!
Google Speech Recognition could not understand audio
I don't recognize that order
Waiting for trigger detection
yes sir?
Threshold set to: 4000
Say something!
Google Speech Recognition could not understand audio
I don't know this order
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/signals/order/order.py", line 84, in run
    self.start_trigger()
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 382, in trigger
    return self.machine._process(func)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 1020, in _process
    self._transition_queue[0]()
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 398, in _trigger
    return self._process(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 408, in _process
    if trans.execute(event_data):
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 262, in execute
    self._change_state(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 273, in _change_state
    event_data.machine.get_state(self.dest).enter(event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 119, in enter
    event_data.machine.callback(handle, event_data)
  File "/usr/local/lib/python2.7/dist-packages/transitions-0.6.4-py2.7.egg/transitions/core.py", line 990, in callback
    func(*event_data.args, **event_data.kwargs)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/signals/order/order.py", line 92, in start_trigger_process
    self.trigger_instance = TriggerLauncher.get_trigger(settings=self.settings, callback=self.trigger_callback)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/core/TriggerLauncher.py", line 32, in get_trigger
    parameters=trigger.parameters)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/core/Utils/Utils.py", line 140, in get_dynamic_class_instantiation
    return klass(**parameters)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboy.py", line 51, in __init__
    sleep_time=0.03)
  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboydecoder.py", line 98, in __init__
    stream_callback=audio_callback)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable

When starting kalliope with the –debug option, I see lines like this in the debug log:

2018-03-17 13:55:30 :: kalliope-0.5.1b :: Magic_mirror call Magic Mirror MMM-kalliope-API: http://127.0.0.1:8080/kalliope
2018-03-17 13:55:31 :: kalliope-0.5.1b :: Magic_mirrorcannot get a valid json from returned content

2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from {'notification': 'KALLIOPE', 'mm_url': 'http://127.0.0.1:8080/kalliope', 'is_api_call': True, 'payload': "{{kalliope_memory['kalliope_last_tts_message']}}", 'no_voice': False}, using {}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from KALLIOPE, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from http://127.0.0.1:8080/kalliope, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from True, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from {{kalliope_memory['kalliope_last_tts_message']}}, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronLauncher] replacing brackets from False, using {'kalliope_memory': {'kalliope_last_tts_message': 'what can i do for you?'}, 'name': 'sir'}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Run neuron: "{'name': 'magic_mirror', 'parameters': {'notification': 'KALLIOPE', 'mm_url': 'http://127.0.0.1:8080/kalliope', 'is_api_call': True, 'payload': 'what can i do for you?', 'no_voice': False}}"
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [Utils]-> get_dynamic_class_instantiation : package path : kalliope.neurons.magic_mirror.magic_mirror
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [Utils]-> get_dynamic_class_instantiation : loading path : resources/neurons/magic_mirror/magic_mirror.py, as package Magic_mirror
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronModule] TTS args: {'name': 'pico2wave', 'parameters': {'cache': True, 'language': 'en-US'}}
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Magic_mirror call Magic Mirror MMM-kalliope-API: http://127.0.0.1:8080/kalliope
2018-03-17 13:55:09 :: kalliope-0.5.1b :: Magic_mirrorcannot get a valid json from returned content
2018-03-17 13:55:09 :: kalliope-0.5.1b :: [NeuronModule] Say() called with message: {'content': '<!DOCTYPE html>\n<html lang="en">\n<head>\n<meta charset="utf-8">\n<title>Error</title>\n</head>\n<body>\n<pre>Cannot POST /kalliope</pre>\n</body>\n</html>\n', 'status_code': 404, 'response_header': {'Content-Length': '148', 'X-XSS-Protection': '1; mode=block', 'X-Download-Options': 'noopen', 'X-Content-Type-Options': 'nosniff', 'Content-Type': 'text/html; charset=utf-8', 'Strict-Transport-Security': 'max-age=15552000; includeSubDomains', 'Connection': 'keep-alive', 'Content-Security-Policy': "default-src 'self'", 'Date': 'Sat, 17 Mar 2018 12:55:09 GMT', 'X-Frame-Options': 'SAMEORIGIN', 'X-DNS-Prefetch-Control': 'off'}}

Accessing the URL http://192.168.1.16:8080/kalliope, gives me this:

Cannot GET /kalliope

I also see the following error using the CURL example (removed the “<” from the beginning on the output of each line):

pi@mirror:~ $ curl -H "Content-Type: application/json" -X POST -d '{"notification":"KALLIOPE", "payload": "my message"}' http://127.0.0.1:8080/kalliope      
!DOCTYPE html>
html lang="en">
head>
meta charset="utf-8">
title>Error</title>
/head>
body>
pre>Cannot POST /kalliope</pre>
/body>
/html>
pi@mirror:~ $

[sispheor]

@marcnlx why are you trying to reach the URL from your browser? It’s an API. It should be accessed from a script only.

[marcnlx]

@sispheor said in Kalliope assistant + MM:

@marcnlx why are you trying to reach the URL from your browser? It’s an API. It should be accessed from a script only.

The CURL example gave me this output on the CLI. I was only using my browser to make sure I was able to reach the API.

Any idea what configuration mistake I could have made?

[sispheor]

@marcnlx @marcnlx you’ve set http://127.0.0.1:8080/kalliope for the neuron but the mirror up is 192.168.1.16. doesn’t it?

Also, notification and payload are not needed in the config of the module.

[marcnlx]

@sispheor said in Kalliope assistant + MM:

@marcnlx @marcnlx you’ve set http://127.0.0.1:8080/kalliope for the neuron but the mirror up is 192.168.1.16. doesn’t it?

Also, notification and payload are not needed in the config of the module.

@sispheor 127.0.0.1 is the localhost IP, 192.168.1.16 is the external IP…the Magic Mirror is available on every IP in use on the RPi (listening address is set to 0.0.0.0).

I just commented out the notification and payload and thing are now showing on the Mirror! This is really cool! :)

Okay…now I need to figure out how to build the whole virtual assistant…on to “phase #2”.
In your demo video I saw a basically complete brain…is there any place where I can find that one? ;)

[sispheor]

@marcnlx take a look to starter kits. It contains example. Also there is an example of usage of each neuron on the website.
Have fun !

[marcnlx]

@sispheor said in Kalliope assistant + MM:

@marcnlx take a look to starter kits. It contains example. Also there is an example of usage of each neuron on the website.
Have fun !

I will! I’m going to check everything out later today, thanks!

[marcnlx]

I tested Kalliope yesterday night and once you know how it works, it’s pretty sweet! ;)

Unfortunately I noticed some bugs/issues which are discussed multiple times on Github, so for now I think I will wait until a later release since I need to restart Kalliope multiple times after 2-3 phrases because of pulseaudio issues which I can’t seem to fix:

2018-03-18 12:42:27 :: kalliope-0.5.1b :: File found in /home/pi/kalliope_starter_en/trigger/kalliope-EN-12samples.pmdl
Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()

  File "/usr/local/lib/python2.7/dist-packages/kalliope-0.5.1b0-py2.7.egg/kalliope/trigger/snowboy/snowboydecoder.py", line 98, in __init__
    stream_callback=audio_callback)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "build/bdist.linux-armv7l/egg/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable

[sispheor]

@marcnlx Uninstall pulse audio and work only with alsa.
We’ve remove pulseaudio from our doc.

[sispheor]

Kalliope v0.5.1 has been released with all stuff needed to integrate Magic Mirror.

[maxbachmann]

@sispheor kalliope uses the Google Speech recognition?

[strawberry 3.141]

@maxbachmann google is the default, but you can select others https://kalliope-project.github.io/kalliope/settings/settings/#speech-to-text-configuration

[sispheor]

@maxbachmann Yes you can choose your STT and TTS engine. Some are cloud based and some can be self hosted.

[dmwilsonkc]

@sispheor I am interested in duplicating your work for a different AI. So I figured the best place to start is to install your module and get it working before I make changes to suit my needs with the other AI. I have installed the MMM-Kalliope module, but the curl commands do not display on my mirror. I am not sure what I am doing wrong. Here’s my entry in config.js

{
    module: "MMM-kalliope",
    position: "upper_third",
                        config: {
                                max: "5",
                                keep_seconds: "8",
                                title: "Kalliope"
                                }

},

I do receive a json response indicating success, however, nothing gets displayed on the mirror.

Here’s the curl command:

curl -H "Content-Type: application/json" -X POST -d '{"notification":"KALLIOPE", "payload": "my message"}' http://localhost:8080/kalliope

Here’s the response:

{"status": "success"}

Yet nothing gets displayed. Do you have any suggestions?

[sispheor]

@dmwilsonkc seems good anyway.
I don’t see any errors. Maybe test another position. This one is not already used by another module?

[dmwilsonkc]

@sispheor I’ve tried multiple positions for the module, still nothing gets displayed. Not sure why though. I’ve tried refresh, reboot etc. still nothing. Maybe I should delete the module and try a fresh install. The curl should work with just the module installed right?

[sdetweil]

@sispheor trying to help, I installed the module on my mirror… but… curl notifications don’t work…

is there an install step missing? or a package.json that defines the install process for the dependencies?

cut paste from the module readme…

~/MagicMirror/modules$ curl -H "Content-Type: application/json" -X POST -d '{"notification":"KALLIOPE", "payload": "my message"}' http://localhost:8080/kalliope
curl: (7) Failed to connect to localhost port 8080: Connection refused

netstat shows nothing listening on 8080

so, my mirror is NOT on 8080, but 8086, due to other components on the normal MM ports…

you should update the instructions to say use the MM port on the curl…

now… I got the notification, and see the debug log (ctrl-shft-i) say

MMM-kalliope received a socket notification: KALLIOPE Payload: my message

and the content shows for the configured amount of time (8 seconds per the above config settings)

[dmwilsonkc]

@sispheor So using the ctrl-shift-i I was able to find the error that it is throwing.

Uncaught TypeError: console.log is not a function

This error is repeated in the log each time it receives the post request.

What are your suggestions?

[sdetweil]

@dmwilsonkc edit the MM/modules/MMM-kalliope/MMM-kalliope.js and change all console. to Log.

console.log is ONLY allowed in the node_helper.js and Log.log is ONLY allowed in the module.js