Automatic checking of all MagicMirror² modules

KristjanESPERANTO

Guys, like I said, the project is in early status! The tests so far should therefore be understood as a proof of concept. Maybe I could have made that a little clearer in my initial post.

In any case, thanks for the feedback! 😃

Categorizing the messages would certainly also make sense. Like: critical, warning and information.

@sdetweil said in Automatic checking of all MagicMirror² modules:

things that depend of features no longer available in the os like vcgencmd

Is vcgencmd really no longer available? I don’t have the current OS version yet, but I can’t find any information about removing it in the documentation:
https://www.raspberrypi.com/documentation/computers/os.html

If so, what would be the alternative?

and omxplayer

Thanks. This is a solid suggestion, just like I was looking for! 😃 I have added this to the check list. There are a handful of modules which contain omxplayer.

I don’t have a concrete approach for your other suggestions.

@MMRIZE said in Automatic checking of all MagicMirror² modules:

There are also wrong checks. (For example, MMM-ModuleMonkeyPatch …

Ah, I used the module to provoke errors manually. I forgot to remove the folder 🤦. The checks themselves are not wrong.

dependency checking is information ordinary users …

Ordinary users are not the target group for this project. The project collects information about modules, the developers can do whatever they want with the information (even ignore it).

For the core developers it is certainly sometimes interesting (e.g. before breaking changes) which modules still depend on a core functionality.

it would be better to reveal the last update date or the number of unresolved issues to guess the module’s activation level or popularity

Those are good points, thank you! I put them on the to-do list.

Or, if you are going to parse package.json anyway, I think it could be used to organise installation methods, etc.

Yes, I had already thought about parsing the package.json. What do you mean by “organise installation methods”?

karsten13

@KristjanESPERANTO

from the results:

.gitlab-ci.yml: Change file extention from .yml to .yaml: https://yaml.org/faq.html.

In this case the .gitlab-ci.yml is the default name of the gitlab pipeline definition file (users can change this but 99% will not) so if you really want to check yml vs. yaml you need a whitelist …

karsten13

@KristjanESPERANTO

there was a discussion long ago here about 3rd party modules because the current format is very ugly.

So maybe this could be also a starting point to have the module list as e.g. yaml output with all current stuff and your analysis results. With such an input it would be possible to generate a website (as mentioned in the above issue, examples how this could look were https://tiny-helpers.dev/ or https://www.electronjs.org/apps) …

KristjanESPERANTO

@karsten13 said in Automatic checking of all MagicMirror² modules:

if you really want to check yml vs. yaml

Yeah, I’ll drop this test for now.

maybe this could be also a starting point to have the module list as e.g. yaml output with all current stuff and your analysis results.

Interesting direction. I’ll take a look at it 🙂

karsten13

@KristjanESPERANTO

it would be better to reveal the last update date or the number of unresolved issues

as most modules are hosted on github: You can get such info’s over their api.

As example you can look at https://api.github.com/repos/MichMich/MagicMirror

KristjanESPERANTO

@karsten13 said in Automatic checking of all MagicMirror² modules:

as most modules are hosted on github: You can get such info’s over their api.

Wow. That’s really nice! There are certainly some things that can be put to good use.

After you showed me the examples, I tinkered a bit with a simple frontend (without dependencies): check it out. I just have to fill it with real data.

karsten13

@KristjanESPERANTO said in Automatic checking of all MagicMirror² modules:

After you showed me the examples, I tinkered a bit with a simple frontend (without dependencies): check it out. I just have to fill it with real data.

cool, exactly what I thougth of …

looking at your example json

  {
    "name": "MMM-PublicTransportHafas",
    "image": "https://raw.githubusercontent.com/KristjanESPERANTO/MMM-PublicTransportHafas/main/img/Goerdelerring_all.png",
    "maintainer": "KristjanESPERANTO",
    "link": "https://github.com/KristjanESPERANTO/MMM-PublicTransportHafas",
    "category": "Traffic",
    "tags": ["Schedule", "Public Transport", "Traffic", "HAFAS"],
    "text": "This module displays real-time departure times for public transportation.",
    "description": "Stay informed about the upcoming departures from your local public transit stations."
  },

there are only a few things which are not available via github api (image, category).

category and tags are similar, do we still need a category?

tags are similar to api topics (may without magicmirror key words)

MMRIZE

One of the most popular modules MMM-Remote-Control uses similar JSON as a module-list.
And MMM-Remote-Control-Repository are serving that JSON file for MMM-Remote-Control.
However, if it is possible to serve a similar JSON URI, it could be a good replacement.

KristjanESPERANTO

@karsten13 said in Automatic checking of all MagicMirror² modules:

category and tags are similar, do we still need a category?

Good point! The mass of tags will certainly become a bit confusing. We could replace the category links in the menu by a few selected tags or we highlight certain tags so that they are more noticeable. I think I’ll try it in this direction.

@MMRIZE said in Automatic checking of all MagicMirror² modules:

MMM-Remote-Control uses similar JSON as a module-list.

You’re right, it would be cool to get compatibility with MMM-Remote-Control. Thanks for the hint!

KristjanESPERANTO

I made some progress: https://kristjanesperanto.github.io/MagicMirror-3rd-Party-Modules/

There’s still a lot on the to-do list and some things aren’t working quite smoothly yet. But the website is now filled with data from all modules from the wiki.

When changes are made in the wiki, it doesn’t take much effort to update the data in this project - I just have to run a few scripts. Furthermore, no manual work is necessary.

KristjanESPERANTO

@karsten13 said in Automatic checking of all MagicMirror² modules:

You can get such info’s over their api.

Unfortunately the API blocks me when I make queries for almost 1000 modules in a short time 🤪

I now take the tags from keywords in the package.json.

karsten13

@KristjanESPERANTO

from your todo list

Is there also a GitLab API?

yes, as example you can look at MMM-RepoStats

Another thing is what the source of the list should be, still the wiki (which is very error prone since anyone can edit things).

Alternatively, you could store a list in your repo and we need something for users to add to their repo (e.g. as a PR in the repo or with website functionality).

But maybe there are even better ideas.

Jalibu

hmm… if this is the cornerstone for a proper MagicMirror Appstore, then I should consider renaming all my modules to start with the letter “A” to attract more attention on the landing page :-D

MMRIZE

@Jalibu Didn’t do that yet? :D

BKeyport

@Jalibu AAA-Module instead of MMM? 🤣🤣🤣🤣

Jalibu

@BKeyport or MMM-a-fancy-weather-module, MMM-a-powerful-stock-ticker, MMM-a-bosch-smart-home-dashboard, etc…

KristjanESPERANTO

@Jalibu said in Automatic checking of all MagicMirror² modules:

hmm… if this is the cornerstone for a proper MagicMirror Appstore, then I should consider renaming all my modules to start with the letter “A” to attract more attention on the landing page :-D

Yeah, I’m already have some ideas about the default sort order (like a combination of stars, issues and activity), this will prevent that :beaming_face_with_smiling_eyes:

KristjanESPERANTO

In the last few days I have worked on some details on the site. So if you’re interested, take a look: https://kristjanesperanto.github.io/MagicMirror-3rd-Party-Modules/.

And there is now a little statistics table on the results page: https://github.com/KristjanESPERANTO/MagicMirror-3rd-Party-Modules/blob/main/result.md#statistics

I have also created a few PRs based on the list for some modules. Most of them have already been accepted :-)

mumblebaj

@KristjanESPERANTO Looks good. Just out of interest, which image does it select as the thumbnail? Just noticed 2 of the modules of mines does not have the correct images. Both Growatt modules are the ones I am referring to.

mumblebaj

@KristjanESPERANTO Stats were great, fixed most of the issues identified in my modules. Majority package.json issues.

Great work @KristjanESPERANTO