External access for embedded google assistant with custom actions

[timjong93]

Hi all,

I am aware that there are already a number of google assistant modules.
However these modules lack the ability to actually interact with magic mirror.
I would like to be able to perform actions like:

• Turning display on/off
• Setting brightness
• Show/hide modules

I have some experience with the google assistant sdk and the recent adition of custom device traits should allow us to do this. As an experiment i already have a proof of concept which correctly detects the command show/hide [module_alias].

However this is a python script as it has to interact with the google assistant library. Therefore i have the question what you would think is the best way to communicate with a magic mirror module. I have experimented with websockets, which works. But i am not sure wether this is the right method.

Does anyone have any relevant experience/advice with buidling modules which communicate with third party applications?

With kind regards,
Tim

[sdetweil]

i think you are on the right path…

the module.js can see other modules for show/hide, but the node_helper cannot
the node_helper can import other libraries, where the module.js cannot…

i just made MMM-SleepWake (https://github.com/sdetweil/MMM-SleepWake) which allows some external source (camera via https://motion-project.github.io/) to indicate motion or not, (by writing a file),
and the node helper uses the file system support to detect that file… and then decides what to do…

• if it can directly control the display device (pi, or use DPMS), then the node helper does the work. but my system runs on an ODROID SMC, so the pi tvservice command doesn’t work to turn off the monitor (my TV)…
• the DPMS command (seen used by HelloLucy) turns off the hdmi/dvi source…
  on my TV this shows the big blue ‘NO Input’ screen (counter to what I wanted!)…
• so I have a software driven sleep (that hides all modules, and remembers ones already hidden)… but that has to be implemented in the module.js… so there is communication from the node_helper to the module.

whether its file, or sockets, or some queue… you’ve got the right idea…
I use HelloLucy (on top of MMM-voice) to sleep/wake and hide/show modules via voice.
i don’t see any ability to inject phrases to the processed in that module…

[sdetweil]

and I recently saw MMM-RemoteControl, which seems to support sending command strings …

[timjong93]

Thanks for the info.
I do indeed already use MMM-RemoteControl for manual control and plan on using it for the actual control of the mirror.
Concerning communication between the processes i’m looking at letting the js module spawn the python google assistant as a child process and communicate using process piping.
This would make it way easier to run the module as the user would not need to manage multiple applications.

[sdetweil]

@timjong93 spawn background process, good… that is how voice works and a bunch of other things that have defined processes…

[timjong93]

Just to let you know, i managed to integrate the assistant application as a python child process running in is own virtualenv which is created using npm, limiting the amount of installation steps required by users.

Now the fun part starts, handling commands from the assistant and implementing fancy animations :)

[sdetweil]

@timjong93 nice!

[timjong93]

For a first quick demo:
https://streamable.com/5s6bn