MagicMirror Forum
    • Recent
    • Tags
    • Unsolved
    • Solved
    • MagicMirror² Repository
    • Documentation
    • 3rd-Party-Modules
    • Donate
    • Discord
    • Register
    • Login
    A New Chapter for MagicMirror: The Community Takes the Lead
    Read the statement by Michael Teeuw here.

    MMM-GoogleAssistant in Server mode

    Scheduled Pinned Locked Moved Troubleshooting
    11 Posts 2 Posters 2.1k Views 2 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • M Offline
      martinhood
      last edited by

      Hello,
      I’m wondering how Google Assistant module (and other voice activated modules also) are supposed to work with a server only setup.
      Is it possible to have audio input and output on the client, and let them communicate with the module in the backend?
      I see there are modules that can stream audio to the client (browser) using HTML5, can this be done for MMM-GoogleAssistant Module?
      My goal is to have a MagicMirror istance running on a remote server in the cloud (no phisical audio), and a browser running on android box at client side (home).

      Thank you in advance

      S 1 Reply Last reply Reply Quote 0
      • S Away
        sdetweil @martinhood
        last edited by sdetweil

        @martinhood i have never seen the audio side remote voice reco work

        snowboy (hotword detector), assumes mic is local, uses node_record_lpcm16 which invokes arecord or rec to collect the audio stream

        on some implementations, once the hotword is detected the audio stream is forwarded thru api to a remote speech reco system (google speech) which streams back the words detected, as a transaction

        word
        word word
        word word word
        done word word word

        then snowboy is restarted, logically, to wait for the hotword and repeat all of that

        the snowboy team has abandonded their project and no one has taken their place.
        https://github.com/Kitt-AI/snowboy

        snowboy hotword detection is pretty much the gold standard

        Sam

        How to add modules

        learning how to use browser developers window for css changes

        1 Reply Last reply Reply Quote 0
        • M Offline
          martinhood
          last edited by

          Thanks for your feedback.
          So, if I understand this correctly, there’s no way to make any snowboy-based module work with remote audio, at the moment. Right?

          S 1 Reply Last reply Reply Quote 0
          • S Away
            sdetweil @martinhood
            last edited by

            @martinhood as we speak, correct.

            i have not seen a local mic, stream audio to a remote word detector and voice reco engine

            Sam

            How to add modules

            learning how to use browser developers window for css changes

            M 1 Reply Last reply Reply Quote 0
            • M Offline
              martinhood @sdetweil
              last edited by

              @sdetweil that’s a pity :) Thank you

              S 1 Reply Last reply Reply Quote 0
              • S Away
                sdetweil @martinhood
                last edited by

                @martinhood ive working on these kinds of systems now for 5 years, never seen anyone attempt it

                Sam

                How to add modules

                learning how to use browser developers window for css changes

                1 Reply Last reply Reply Quote 0
                • M Offline
                  martinhood
                  last edited by

                  I understand the standard deploy for a Magic Mirror is a all-in-one device, but since a “server mode” exists… should be reasonable to think about a 2-tier setup.
                  In my specific case I have an old Android Lollipop box I would like to repurpose as a Magic Mirror frontend (it is small and can run a browser, no way to put another OS on it), and I’ve got plenty of horsepower on a remote hypervisor (on which I deployed a Debian VM with MM on it)

                  S 1 Reply Last reply Reply Quote 0
                  • S Away
                    sdetweil @martinhood
                    last edited by

                    @martinhood i didnt say it wasnt a good dream!

                    Sam

                    How to add modules

                    learning how to use browser developers window for css changes

                    1 Reply Last reply Reply Quote 0
                    • M Offline
                      martinhood
                      last edited by

                      yeah :) I’ll let you know in case the dream would turn true :)

                      1 Reply Last reply Reply Quote 0
                      • M Offline
                        martinhood
                        last edited by martinhood

                        @sdetweil I finally managed to make it work:

                        On client side (old Android Lollipop box) I am using RtpMic app: it is good, simple and effective. It can start at boot and start streaming mic input via RTP.
                        I configured it with remote MagicMirror IP, 47474 port (just picked one) and g722 codec (good balance between quality and bandwidth, it just uses 0,09 Mbps)

                        On server side (MM running on Debian cloud-hosted VM), at first, I would like to only rely on PulseAudio default RTP capabilities.
                        Unfortunately, I still couldn’t get my PA to receive the RTP stream from RtpMic (though I’m quite sure it is possible, but I keep getting “Unsupported SAP” error) so ATM I workaround this using ffplay:

                        sudo apt install ffmpeg
                        ffplay rtp://[local IP]:47474 -acodec g722 -nodisp

                        (note that local MM IP goes here, not client IP)
                        You can check the stream flowing in pavucontrol.

                        Now that RTP stream is correctly flowing, I configure ALSA layer to use Pulseaudio by default (this is needed because GA module relays on ALSA, and Alexa module relays on Sox). This is achieved by creating the file “.asoundrc” under MM user home, and writing the following configuration in it:

                        pcm.!default pulse
                        ctl.!default pulse

                        As a last step, I install and configure GA like that:
                        micConfig: { // put there configuration generated by auto-installer
                        recorder: “arecord”,
                        },
                        When started, GA module (and virtually any other module) is able to receive voice commands from remote client.

                        Please note that this solution only cover one way (MM input), from client (running browser) to server (running MM in serveronly mode), the other way (MM output) is already covered by MM playing its audio output inside the browser by default.
                        This way you can have MM running on any machine in any remote place, and have your mirror displayed by any browser capable client (even an Android one).
                        Also, I had to use RtpMic/ffplay because my client is Android (so no PulseAudio), but I’m quite confident the same schema can be applied natively on any other PulseAudio capable device.

                        S 1 Reply Last reply Reply Quote 1
                        • 1
                        • 2
                        • 1 / 2
                        • First post
                          Last post
                        Enjoying MagicMirror? Please consider a donation!
                        MagicMirror created by Michael Teeuw.
                        Forum managed by Sam, technical setup by Karsten.
                        This forum is using NodeBB as its core | Contributors
                        Contact | Privacy Policy