Test suite for MagicMirror²

[roramirez]

I fixed before mentioned https://github.com/roramirez/MagicMirror/commit/2a94ee55ccb8fa06f4d07c2a67307b5ed0a659a3

:)

[roramirez]

The last build of today.

[roramirez]

I mention this here if someone can help

If use moment-timezone for a test this fail

• Fail in Travis: https://travis-ci.org/roramirez/MagicMirror/builds/220155902
• Commit example: https://github.com/roramirez/MagicMirror/commit/05d7c9556e773bf2b941035953200ff327efb476

[roramirez]

Hi Guys, are you still alive?

Sometimes the build failed, an example here
https://travis-ci.org/MichMich/MagicMirror/jobs/227577643

Someone have some idea how to fix this?

cc @qistoph @strawberry-3-141 @morozgrafix

[qistoph]

@roramirez sorry, haven’t had much time to spend on MagicMirror lately.

My guess is that some of the tests take too long. Especially the ones actually starting MM and not just load a module.

I imagine Travis runs the tests in some sort of container with multiple containers running on the same system. If there is some heavy load at the same time as the MM test is run this might slow down our test with a certain factor. Seeing the difference in the usual runtime between tests (couple of ms vs multiple seconds) makes me think some of our tests are on the heavier end. Having a multiple second test delayed by a factor, say, 2 makes it easily run 5 seconds. Hitting the 5 sec maximum (see log).

Two possibilities I see to fix this:

1. Reduce the runtime of long running tests
2. Increase the allowed runtime before timeout

Hopefully this helps.

Best regards and thanks for keeping this thread alive!

[roramirez]

@qistoph Well I’ve testing de CircleCI vs Travis.

For same Build
Failed: https://travis-ci.org/roramirez/MagicMirror/builds/234342109
Sucess: https://circleci.com/gh/roramirez/MagicMirror/6

[qistoph]

@roramirez
Except for one of the errors I think all can still be explained by my last post.

[roramirez]

@qistoph said in Test suite for MagicMirror²:

Except for one of the errors I think all can still be explained by my last post.

Yes, you’re rigth. Can you help in this area?

[roramirez]

Yay! After a long fight the test are passing in Travis CI

https://github.com/MichMich/MagicMirror/pull/953

[Cato]

I’m trying to find some information on testing in Magic Mirror for devs/contributors but it’s (too) hard to find. I can’t find any related forum, it’s not mentioned in the “MagicMirror² Module Development Documentation”, and not directly mentioned in any README-ish files on GitHub.

Of course, some of this can be deduced from the source code, but a small “Getting Started” guide would help more people getting … eh started. :-)

I’m living in Visual Studio as my day job, but not so much with writing tests, so my knowledge of other tools are limited. What I need is some information on software requirements and where tests should be and how testing is done.