CALL FOR TESTERS: New install script

[drdeath]

Hey everyone,

There’s a new install script in town.

It’s designed to have as small a footprint in disk usage and system resources as possible and provide a fire-and-forget hands-off install from a fresh system to a running MM. It can easily install on an SD card as small as 8GB and works well on older and smaller Pi’s, so long as they can handle a current OS.

It’s available for your perusal at https://github.com/tfischer4765/mm-install/ .

The script is currently in beta, and I’d be very grateful if everyone who has the time tested it and tried their best to break it.

Thanks for your help!

pi@mmtest:~ $ mpstat
Linux 6.6.31+rpt-rpi-v8 (mmtest)        02/09/24        _aarch64_       (4 CPU)

22:02:38     CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest  %gnice   %idle
22:02:38     all    0.09    0.00    0.07    0.04    0.00    0.01    0.00    0.00    0.00   99.80
pi@mmtest:~ $ free -h
               total        used        free      shared  buff/cache   available
Mem:           907Mi       378Mi       175Mi        32Mi       437Mi       528Mi
Swap:          199Mi       2.5Mi       197Mi
pi@mmtest:~ $ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            318M     0  318M   0% /dev
tmpfs            91M 1016K   90M   2% /run
/dev/mmcblk0p2  6.9G  3.1G  3.4G  48% /
tmpfs           454M   17M  438M   4% /dev/shm
tmpfs           5.0M   12K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   64M  447M  13% /boot/firmware
tmpfs            91M     0   91M   0% /run/user/1000

[KristjanESPERANTO]

@drdeath I’m a bit sceptical about trying it out as it contains a few hundred encoded lines and the comments don’t clearly describe what it does and why it’s encoded.

[BKeyport]

@KristjanESPERANTO Encoded lines? Oh hell no.

Don’t care if it’s fully documented, if I’m not able to at least look at the code, then it’s a hell no for me.

[drdeath]

@KristjanESPERANTO What are you talking about, the listing of those config files is right there in the comments, and the purpose of that binary is quite obvious if you have a basic knowledge of Linux and Xorg. Also the file name the binary gets stored under is a dead giveaway. Don’t expect me to spoon-feed you obvious information.

[drdeath]

@BKeyport @KristjanESPERANTO Turns out I owe you some thanks, if only because you got me thinking and questioning my assumptions.

I had previously assumed the anchor process of an X server needed to be an X application, but that turned out to be not the case, so the whole blackpixel executable could be removed entirely, eliminating a major contributor to the script size and a maintenance headache for me.

It does make the script even more obscure and alien though. That’s easily the strangest X config I’ve ever created.

Code is updated.

[mumblebaj]

@drdeath I think what the guys were saying or are trying to say, is that for the standard MM user, looking at the script and seeing “coded” code they may be hesitant to test the script. Although it can easily be decoded with echo 'base64 string' | base64 -d which would reveal the coded strings, the normal user does not know this.

It does make the script so much smaller but for a newb looking at that would scare them away.

Just my 2pence.

[sdetweil]

@mumblebaj AND there is no practical need for encoding anything

[karsten13]

@sdetweil said in CALL FOR TESTERS: New install script:

@mumblebaj AND there is no practical need for encoding anything

there was a need for the binary which was removed meanwhile, but not for the text files …

so e.g. this

base64 -d > /tmp/mm-install/xinitrc <<< 'IyEvYmluL3NoCgp4c2V0IHMgb2ZmICAgICAgICAgIyBkb24ndCBhY3RpdmF0ZSBzY3JlZW5zYXZl
cgp4c2V0IC1kcG1zICAgICAgICAgIyBkaXNhYmxlIERQTVMgKEVuZXJneSBTdGFyKSBmZWF0dXJl
cy4KeHNldCBzIG5vYmxhbmsgICAgICMgZG9uJ3QgYmxhbmsgdGhlIHZpZGVvIGRldmljZQoKaWYg
WyAtciAiL2V0Yy9tYWdpY21pcnJvci94cmFuZHJfb3B0cyIgXTsgdGhlbgogICAgICAgZWNobyAi
cnVubmluZyB4cmFuZHIgd2l0aCBhcmd1bWVudHMgXCIkKGNhdCAvZXRjL21hZ2ljbWlycm9yL3hy
YW5kcl9vcHRzKVwiIgogICAgICAgRElTUExBWT06MCB4cmFuZHIgJChjYXQgL2V0Yy9tYWdpY21p
cnJvci94cmFuZHJfb3B0cykKZWxzZQoJZWNobyAibm90IHJ1bm5pbmcgeHJhbmRyIgpmaQoKeHNl
dHJvb3QgLXNvbGlkIGJsYWNrCgppZiBbIC1yICIvZXRjL21hZ2ljbWlycm9yL3hfYmFja2dyb3Vu
ZF9pbWFnZSIgXSAmJiBbIC1yICIkKGNhdCAvZXRjL21hZ2ljbWlycm9yL3hfYmFja2dyb3VuZF9p
bWFnZSkiIF07IHRoZW4KCXhsaSAtb25yb290ICQoY2F0IC9ldGMvbWFnaWNtaXJyb3IveF9iYWNr
Z3JvdW5kX2ltYWdlKQpmaQoKd2hpbGUgOjsgZG8gc2xlZXAgMTAwMDA7IGRvbmUKCg=='

could be replaced by

cat > /tmp/mm-install/xinitrc <<"EOF"
#!/bin/sh

xset s off         # don't activate screensaver
xset -dpms         # disable DPMS (Energy Star) features.
xset s noblank     # don't blank the video device

if [ -r "/etc/magicmirror/xrandr_opts" ]; then
       echo "running xrandr with arguments \"$(cat /etc/magicmirror/xrandr_opts)\""
       DISPLAY=:0 xrandr $(cat /etc/magicmirror/xrandr_opts)
else
        echo "not running xrandr"
fi

xsetroot -solid black

if [ -r "/etc/magicmirror/x_background_image" ] && [ -r "$(cat /etc/magicmirror/x_background_image)" ]; then
        xli -onroot $(cat /etc/magicmirror/x_background_image)
fi

while :; do sleep 10000; done

EOF

[drdeath]

@karsten13 I tried here-docs initially, and it caused all kinds of problems, from missing newlines to empty files, although at least some of that probably was due to me writing the string to variables instead of directly to files and causing problems with the shell’s variable space.

Using base64 may not strictly necessary, but it’s robust and portable, and not in the least harmful. It’s also relatively easy to fact-check if you have a basic working knowledge of Linux. If you don’t, well then any software is a total black box to you and you just have to trust the person providing it. See the recent kerfuffle around obscured malicious code in ssh. I wouldn’t have been able to find it.

The way it works is, people who have the knowing fact-check and give the thumbs-up for those who don’t.

I’d also like to point out that

a) the contents of the files are listed as comments
b) if you were looking at a binary instead of a script, you’d be no wiser if you had it in binary rather than base64,
c) if I had made the pivot to makeself instead of encoding the files in b64, you’d see even less of the script before you executed it
d) most of the software you have on your system right now was provided in binary form, and
e) unless you are a closet uber-geek, you probably wouldn’t understand half of it if you had the source code, and
f) even if you are, a single human lifetime wouldn’t be nearly enough to fact-check the software on even a basic Linux install.

I feel you are blowing the fact that I’m using base64 to encode those files WAY out of proportion. As it is, it’s working as intended, and I’m not going running that gauntlet of trying to get here-docs to work again without better reason than people getting queasy over seeing base64 strings in a file.

If you want to modify it, you’re welcome to do so and provide a well-tested pull request. If on the other hand you wish to reject my work on a technicality like that, that’s a shame, but it hurts the community WAY more than it hurts me.

[tdw]

@drdeath sure couldn’t tell from this chain. Feels like a solution in search of a problem.

[drdeath]

Actually, there’s two more things I’d like to point out, now that I think about it.

First, I specifically put in the readme that the script is written for people with a basic understanding of Linux and X. That’s not saying it can’t be used by people who don’t have that, but everything we’ve been discussing so far boils down to “It’s obscure to people who lack a basic understanding of Linux and X”. Well, yes it is, and I won’t apologize for that.

Second, what sparked the whole controversy was originally the presence of the blackpixel executable in the script. Funny thing is, nobody even asked me about the purpose or the source code of the blackpixel executable when it was still in there. I would have gladly shown it off and explained it, but instead of asking, all anyone ever did was basically criticize me for it being there because they didn’t understand it just by looking at it.

Everybody should remember that IT isn’t easy. If it is easy, that’s because people who have the skills and knowledge put in the work to make it easy for those who don’t. I’m very grateful to all the great people who put in the hard work so I can have Linux, X, MagicMirror or a ridiculously cheap versatile computer like the Raspberry Pi. I just wish the world would reciprocate just this once.

[karsten13]

@drdeath

I just presented a way to get by without the base64 that others have criticized.

To address me here now as if I was the critic is far-fetched.

If you call for testing here, you also have to be able to deal with criticism - regardless of whether it is justified or not.

[drdeath]

@karsten13 Sorry, that didn’t come out right. I meant the first paragraph for you and the rest for everyone. Should have made that clearer, my apologies.

[mumblebaj]

@drdeath I applaud you for the effort and time you put into this. I was just explaining what the encoding meant and how the others could go about decoding it. I have a fully setup system and my own dev environment and probably would not be able to test your script. My apologies for that. But again, good work and nice effort.

P/s - do not take offence to the community. We are all dev’s around the table and we try and assist to make things better where we can.

[sdetweil]

@drdeath can you provide user guidance on choosing your solution vs the others available

[drdeath]

@mumblebaj Thank you for your kind words. You can’t imagine how much I needed that right now - for at least one person to actually say “thank you”.

On the subject of the “standard user’s” reaction, I guess this is another occasion to break out my favorite corollary to a corollary to the famous statement of Arthur C. Clarke: “Everything looks like magic to those who don’t understand it”. Whether you condemn it as evil magic or marvel at the wonderful magic however depends entirely on your own personal disposition.

Personally I tend towards the opinion that these days users are totally used to making use of tools they don’t understand but know how to work. If you for instance went into a dark room and asked “How’s the light work?” you would much more likely get pointed to the light switch than an explanation of the function of the power grid. There would certainly be a few die-hard sceptics who refuse to use it on the grounds of “I don’t understand it, hell no” - to then complain about it on the forums they don’t understand either using their smartphone they understand even less, the irony - but most users would probably not look at it at all, or if they did, just shrug and run it anyways.

That however goes for end users, it certainly does not go for reviewers. Someone should definitely understand the code, and I would never hold it against anyone who doesn’t if they decided to wait until those who understand it better give their blessing.

What I will hold against people though is the attitude to condemn a thing merely because they don’t understand it and can’t be bothered to learn. That sort of thing gets right up my nose.

[drdeath]

@sdetweil I most certainly can and will. Where would you like it to live? I’d suggest the README in the repo.

In a nutshell, if you want a reduced footprint in cpu, ram and memory usage and/or want your magic mirror system to be more of an embedded-appliance style rather than a general use computer running an application, then my script is right for you.

By the way, I just thought I should probably add the option to enable automatic system updates via apt as a logical extension of that concept.

What do you think? Should I include it now or when I get around to re-creating automatic updates for MagicMirror and its modules?

[sdetweil]

@drdeath we’ve had lots of successful appliances built so far without your approach.
and the user issues are mostly unrelated to the os or MagicMirror runtime.

look and feel, consistency of error recovery
and module function/integration are the primary issues. power saving , new monitor hardware are the next most.

and user skill is always a critical issue. we have a lot of new users that know nothing without a graphical desktop . cant find help, don’t know file commands, cd? redirects, copy, move, networking
and on and on.

what can the user save by using your approach?

note that the older pis are going by the wayside with the os changes. pi4 and 5 class devices are the norm across sll the vendors

can they do without a fan?
functionally nothing else changes

i am not in favor of automatic updates in this space, the amount of breaking changes are too high. we still have significant disruption of the existing user base on each release

some OS related (tools used for blanking the screen or video feeds dropped out or function limited), some deprecation limited (remove request and other libs no longer supported), new versions of electron taking new code with bugs not yet discovered which cause fatal errors for the user . new electron requires, new node version, which doesn’t run on this os… oops… but discovered too late

some MM related… the deprecated lib thing… we don’t want to carry dead baggage… oops authors didn’t know, we added checking for things to prevent crashes (good thing) which cause accidental side effects
position: “” used to work, now fails.

[drdeath]

@sdetweil Well if you put it this way, the less software you have installed on a system, the less likely it is going to break during an update. That could actually be a major selling point for doing it my way, because almost all of the tools I’m using have been around for at least 30 years and are going nowhere. Desktop environments have big breaking changes all the time (I’ve been there when they swapped out KDE3 for KDE4 in SuSE Linux over night, boy were people enraged) whereas the basic toolset I rely on has stood the test of time and is virtually unchanged save for the occasional bugfix since the mid 90s.

On the subject of how the users interact, I’ve said so here multiple times and I’ve also put it in the README that this script isn’t written with the wet-behind-the-ears noob at the forefront of the brain. That being said, it’s actually really hands-off and easy to use when you think about it. Download it, start it, answer a couple of questions, bam, you’ve got a working mirror. The major concern I have with the current script is actually that may be too easy to use, prompting people to treat their mirror like a maintenance-free washing machine and let the system rot.

[sdetweil]

@drdeath interesting, but there is no less software.

curl, git, node, MagicMirror and all its dependencies .

whatever window manager is noise, there is one

there are a few users like you that want to be near the bare minimum, i don’t think that’s the right place for 99% of our users. they are still exploring the possibilities.