MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    External access for embedded google assistant with custom actions

    Scheduled Pinned Locked Moved Development
    moduledevelopmentvoice controlgoogle assistant
    8 Posts 2 Posters 3.0k Views 3 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T Offline
      timjong93
      last edited by

      Hi all,

      I am aware that there are already a number of google assistant modules.
      However these modules lack the ability to actually interact with magic mirror.
      I would like to be able to perform actions like:

      • Turning display on/off
      • Setting brightness
      • Show/hide modules

      I have some experience with the google assistant sdk and the recent adition of custom device traits should allow us to do this. As an experiment i already have a proof of concept which correctly detects the command show/hide [module_alias].

      However this is a python script as it has to interact with the google assistant library. Therefore i have the question what you would think is the best way to communicate with a magic mirror module. I have experimented with websockets, which works. But i am not sure wether this is the right method.

      Does anyone have any relevant experience/advice with buidling modules which communicate with third party applications?

      With kind regards,
      Tim

      1 Reply Last reply Reply Quote 0
      • S Away
        sdetweil
        last edited by

        i think you are on the right path…

        the module.js can see other modules for show/hide, but the node_helper cannot
        the node_helper can import other libraries, where the module.js cannot…

        i just made MMM-SleepWake (https://github.com/sdetweil/MMM-SleepWake) which allows some external source (camera via https://motion-project.github.io/) to indicate motion or not, (by writing a file),
        and the node helper uses the file system support to detect that file… and then decides what to do…

        • if it can directly control the display device (pi, or use DPMS), then the node helper does the work. but my system runs on an ODROID SMC, so the pi tvservice command doesn’t work to turn off the monitor (my TV)…
        • the DPMS command (seen used by HelloLucy) turns off the hdmi/dvi source…
          on my TV this shows the big blue ‘NO Input’ screen (counter to what I wanted!)…
        • so I have a software driven sleep (that hides all modules, and remembers ones already hidden)… but that has to be implemented in the module.js… so there is communication from the node_helper to the module.

        whether its file, or sockets, or some queue… you’ve got the right idea…
        I use HelloLucy (on top of MMM-voice) to sleep/wake and hide/show modules via voice.
        i don’t see any ability to inject phrases to the processed in that module…

        Sam

        How to add modules

        learning how to use browser developers window for css changes

        1 Reply Last reply Reply Quote 0
        • S Away
          sdetweil
          last edited by

          and I recently saw MMM-RemoteControl, which seems to support sending command strings …

          Sam

          How to add modules

          learning how to use browser developers window for css changes

          1 Reply Last reply Reply Quote 0
          • T Offline
            timjong93
            last edited by

            Thanks for the info.
            I do indeed already use MMM-RemoteControl for manual control and plan on using it for the actual control of the mirror.
            Concerning communication between the processes i’m looking at letting the js module spawn the python google assistant as a child process and communicate using process piping.
            This would make it way easier to run the module as the user would not need to manage multiple applications.

            S 1 Reply Last reply Reply Quote 0
            • S Away
              sdetweil @timjong93
              last edited by

              @timjong93 spawn background process, good… that is how voice works and a bunch of other things that have defined processes…

              Sam

              How to add modules

              learning how to use browser developers window for css changes

              1 Reply Last reply Reply Quote 0
              • T Offline
                timjong93
                last edited by

                Just to let you know, i managed to integrate the assistant application as a python child process running in is own virtualenv which is created using npm, limiting the amount of installation steps required by users.

                Now the fun part starts, handling commands from the assistant and implementing fancy animations :)

                S 1 Reply Last reply Reply Quote 1
                • S Away
                  sdetweil @timjong93
                  last edited by

                  @timjong93 nice!

                  Sam

                  How to add modules

                  learning how to use browser developers window for css changes

                  1 Reply Last reply Reply Quote 0
                  • T Offline
                    timjong93
                    last edited by

                    For a first quick demo:
                    https://streamable.com/5s6bn

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    • First post
                      Last post
                    Enjoying MagicMirror? Please consider a donation!
                    MagicMirror created by Michael Teeuw.
                    Forum managed by Sam, technical setup by Karsten.
                    This forum is using NodeBB as its core | Contributors
                    Contact | Privacy Policy