Github usage and module continuations

[crisvdn]

Hi everyone,
After using the magic mirror software and 3rd party applications for a couple years now, it has come to my attention there’s a wide variety in how users and module publishers handle their module and Github repository. Also after Magic mirror software being updated, some repositories are obselete, and a new repo is created with the same name, but just a “2” or even “3” added to the module.

Sometimes I see people forking repositories, even adding their own features, but also falling behind the main repository.

I want to create a general discussion on how we, as a community, can improve this. As a software engineer myself, it kind of hurts to see that I have to search for multiple repositories because older ones are just obselete, but are created for the exact same functionality. Obviously sometimes someone wants to rework their code, so one part I understand it, but even then: Github has solutions for it.

For example, you can make releases, mark a specific commit and link it to a MagicMirror version. You can, afterwards, rework your entire code and make another release for, lets say, MagicMirror version 2.26.
Releases actually have automated changelogs that you can create. This will show the PR’s that are pushed between versions.

It is not a rant, as I know a bunch of people just make modules as a hobby project and are not software engineers as a main profession. But I do think some people are amazing at making software and integration of their API. So the knowledge is here, it just needs to be spread and people can learn from it.

How do you guys experience this? Especially in making modules. Do you actively monitor forks and request them making a PR with their added features?
Do you help people with issues, and do you implement new features so debugging or helping them is easier the next time?

Maybe people want to know more of a Github workflow so this can be easier for new 3rd party publishers.

Its all open for discussion here. Furthermore, I am willing to invest some time in a Github workflow or How-to guides if that is missing and needed (obviously I am not all-knowing, but I do know some stuff ;))

[sdetweil]

@crisvdn there are really a list of issues here

most importantly, most modules are created by someone for their own use, and shared
but then they go on to other life things

maybe for a while they hang around and take issues and PRs.

but life goes on.

some modules haven’t been updated in 5-6 years. mm has been active for 10.

I would guess that 99% of the authors are not programmers, and have other day jobs. never experiencing the details , or life cycle.

as this is open source, there is no commitment to maintain a module.

we get waves of new authors maybe 5-6 a year.

without fork, there is no way to ‘fix’ an abandoned module.

now we are at the 'where do I find modules ’ problem. there is one voluntary list. others are trying to improve the usability of the list, but it’s still voluntary.

we keep up the core with issues and PRs and changelog and test cycles. but not any of the 3rd party modules.

I’m a moderator here, never a professional programmer. I do it all for for fun. I keep my stuff up to date. but I don’t list my modules in the 3rd party list.

I don’t use the Developer or Contributor tags on my ID cause I show my support thru my actions. I don’t need an outside label to differentiate myself… I’m no better than anyone else here.

[crisvdn]

@sdetweil
First of all, thank you for responding. And awesome how you guys are maintaining everything.

Its totally understandable that developers abandon their modules. For longevity of this community its not so good. Scrolling through an entire list of 3rd party modules where you can only hope its maintained is a pain.

I think the best case scenario is that modules somehow gets absorbed by the main repository (compareable to ESP-IDF or how HomeAssistant is managing their integrations).
That way you don’t have seperate developers that have their responsibility (and even Github rights) to maintain the code. Single bus factor would be eliminated.

The problem with this is that other people have to take responsibility to maintain 3rd party absorbed modules. Even if that means that they don’t have primary knowledge of that specific module.

[sdetweil]

@crisvdn said in Github usage and module continuations:

I think the best case scenario is that modules somehow gets absorbed by the main repository (compareable to ESP-IDF or how HomeAssistant is managing their integrations).

yeh, I don’t think that will happen… we are a volunteer team, and have plenty of existing core issues to deal with… hundreds of additional modules just won’t be possible.

MAYBE 1 or 2 if they add value to the core.

I do the installer/upgrade/backup/restore scripts because it helps people get up and running… its a daunting learning curve. I’ve added support to try to cover all the abandoned modules during upgrade… but I can’t help on install of them, cause we don’t have a module installer…
there might be a path… but we don’t have a list… again another thing…

so, not specifc module related, but a module installer and a good module list (being the only way the installer would support a module) , and somehow to improve the usability of the config file… (I have have a module that presents a form, dynamically created, but developer styles can cause problems)…

long term those might help go in the direction you want to get to…

[sdetweil]

@crisvdn here is a link to the list work
https://forum.magicmirror.builders/topic/18092/automatic-checking-of-all-magicmirror-modules?_=1704465730999

i’ve just pinned it to the top of the Development section so it won’t get lost
(I couldn’t find it when I mentioned it the first time)

[KristjanESPERANTO]

@crisvdn Thank you for bringing up the subject. I’ve also been thinking about this for a while and have some ideas.

I try to keep the official module list up to date. I do reviews of modules and create PRs. Many are accepted, some are ignored.

I’ve started working on a new list of modules, and doing some rudimentary tests (you can see the result here). A few developers have already started work through the recommendations there. Even though the whole project is still in its infancy, but with that we have a process of assessing the quality of modules. Perhaps the whole thing will develop in the direction of module installer or app store. We will see. I would be delighted if you would take a look at the project and maybe join in.

There are already some great other approaches - for example the package manager mmpm or the “app store” in MMM-GoogleAssistant. Surely such projects could be combined on some level. I’m currently focusing on making my approach more efficient, then it’s time to automate it with actions.

[crisvdn]

@sdetweil said in Github usage and module continuations:

yeh, I don’t think that will happen… we are a volunteer team, and have plenty of existing core issues to deal with… hundreds of additional modules just won’t be possible.

After seeing @KristjanESPERANTO list of modules, seeing that there are over a 1000, I think it is definitely not worth it to push everything to one repository. Especially with duplicates and/or obselete modules.

Although, seeing the list of “default” modules be extended with 3rd party “verified” modules with proper continues integration (tests / and build workflows so it gets automatically build with every software update) would be great.

@KristjanESPERANTO said in Github usage and module continuations:

I’ve started working on a new list of modules, and doing some rudimentary tests (you can see the result here). A few developers have already started work through the recommendations there. Even though the whole project is still in its infancy, but with that we have a process of assessing the quality of modules. Perhaps the whole thing will develop in the direction of module installer or app store. We will see. I would be delighted if you would take a look at the project and maybe join in.

I haven’t seen this yet, this looks pretty good. Awesome to see that there’s a different visual list to find modules and see when its last updated.

As I said in the starting post, it would be nice to have a general sense of direction on how to improve the entire module base and it gets easier and there’s more overview whats available. But also make it easier for people to develop their module and have it gain traction across the community.

[danieeljose]

This sounds amazing as I found it very much useful and informative to be honest. Also, I have gone through this post which definitely helped me out a lot as a new member I am looking forward for more such discussion