Automatic checking of all MagicMirror² modules

[KristjanESPERANTO]

I started a small project. It’s still in its early stages and I don’t know where it’s going yet, but maybe it’ll be interesting for you.

Since the quality of the modules varies greatly, I thought about how to automatically detect easily recognizable issues (e.g. typos or dependencies on deprecated packages).

Basically it already works.

This is what it does:

1. Clone all modules from the wiki page: https://github.com/MichMich/MagicMirror/wiki/3rd-party-modules
2. Search each module according to certain criteria
3. Write the result to a file

This is the result of the last check: https://github.com/KristjanESPERANTO/MagicMirror-3rd-Party-Modules/blob/main/result.md

Do you have any ideas about useful checks?

[sdetweil]

@KristjanESPERANTO I think your two primary tests are useless

module name pattern is subjective

MagicMirror vs with the superscript 2

those have no operational impact.

many modules will never be updated.

things that would make a difference :

libs referenced that are not shipped in the base

APIs that changed

things that depend of features no longer available in the os
like vcgencmd, and omxplayer

API keys required, API no longer free

[BKeyport]

@KristjanESPERANTO I personally think that it’s pretty useless - for example, it’s being picky about MagicMirror(squared) vs Magic Mirror. We use many different names for the project in shorthand, and it’s generally understood that it all refers to MagicMirror(squared).

As you can tell, I’m using a PC and haven’t bothered to learn how to type a superscript 2, and don’t plan to…

Like Sam said, a lot of the checks are useless - It’s much better to test actual functionality - does it still work? Is it missing declared dependencies, etc.

[Jalibu]

I think that such checks can be valuable. perhaps not the ones which are implemented yet, but this can still be a good starting point for other checks.
but it will be challenging to automate some of the proposed checks.

[MMRIZE]

It is a good try, Although most checklists are meaningless rather than useless. There are also wrong checks. (For example, MMM-ModuleMonkeyPatch doesn’t have node_modules directory. It doesn’t have node_helper.js at all.)

However, I would like to praise it because it is a good-looking list. It’s a good attempt.

In any case, dependency checking is information ordinary users cannot deal with even if they know it. Unless he is a developer, there is nothing he can do with that information.

Rather than performing a dependency check, it would be better to reveal the last update date or the number of unresolved issues to guess the module’s activation level or popularity.

Or, if you are going to parse package.json anyway, I think it could be used to organise installation methods, etc.

[KristjanESPERANTO]

Guys, like I said, the project is in early status! The tests so far should therefore be understood as a proof of concept. Maybe I could have made that a little clearer in my initial post.

In any case, thanks for the feedback! 😃

Categorizing the messages would certainly also make sense. Like: critical, warning and information.

@sdetweil said in Automatic checking of all MagicMirror² modules:

things that depend of features no longer available in the os like vcgencmd

Is vcgencmd really no longer available? I don’t have the current OS version yet, but I can’t find any information about removing it in the documentation:
https://www.raspberrypi.com/documentation/computers/os.html

If so, what would be the alternative?

and omxplayer

Thanks. This is a solid suggestion, just like I was looking for! 😃 I have added this to the check list. There are a handful of modules which contain omxplayer.

I don’t have a concrete approach for your other suggestions.

@MMRIZE said in Automatic checking of all MagicMirror² modules:

There are also wrong checks. (For example, MMM-ModuleMonkeyPatch …

Ah, I used the module to provoke errors manually. I forgot to remove the folder 🤦. The checks themselves are not wrong.

dependency checking is information ordinary users …

Ordinary users are not the target group for this project. The project collects information about modules, the developers can do whatever they want with the information (even ignore it).

For the core developers it is certainly sometimes interesting (e.g. before breaking changes) which modules still depend on a core functionality.

it would be better to reveal the last update date or the number of unresolved issues to guess the module’s activation level or popularity

Those are good points, thank you! I put them on the to-do list.

Or, if you are going to parse package.json anyway, I think it could be used to organise installation methods, etc.

Yes, I had already thought about parsing the package.json. What do you mean by “organise installation methods”?

[karsten13]

@KristjanESPERANTO

from the results:

.gitlab-ci.yml: Change file extention from .yml to .yaml: https://yaml.org/faq.html.

In this case the .gitlab-ci.yml is the default name of the gitlab pipeline definition file (users can change this but 99% will not) so if you really want to check yml vs. yaml you need a whitelist …

[karsten13]

@KristjanESPERANTO

there was a discussion long ago here about 3rd party modules because the current format is very ugly.

So maybe this could be also a starting point to have the module list as e.g. yaml output with all current stuff and your analysis results. With such an input it would be possible to generate a website (as mentioned in the above issue, examples how this could look were https://tiny-helpers.dev/ or https://www.electronjs.org/apps) …

[KristjanESPERANTO]

@karsten13 said in Automatic checking of all MagicMirror² modules:

if you really want to check yml vs. yaml

Yeah, I’ll drop this test for now.

maybe this could be also a starting point to have the module list as e.g. yaml output with all current stuff and your analysis results.

Interesting direction. I’ll take a look at it 🙂

[karsten13]

@KristjanESPERANTO

it would be better to reveal the last update date or the number of unresolved issues

as most modules are hosted on github: You can get such info’s over their api.

As example you can look at https://api.github.com/repos/MichMich/MagicMirror