MMM-CalendarExt3

MMRIZE

@MMRIZE calendar uses built in fetch to get the ics file. we do not have node-ical fetch it for us ( the prior ical lib did not provide fetch)

If so, this is really weird where the undici module is used.

sdetweil

@MMRIZE undici is the implementation code of fetch

princemaxwell

@MMRIZE
Thanks for your answer.

I cleaned up my config.js and only had following modules in it:

alert
updatenotification
clock
calendar
MMM-CalendarExt3

No matter how many calendars I have activated, the default calendar loads all entries quite quickly, although you can notice that the “large” calendars with many entries took a few seconds to load (sometimes up to 20-30 seconds).

Nevertheless, I deleted the fetchInterval (31000) in the default calendar, so the default value of 300000 is now active.

After the default calendar displayed all entries, it took at least another 10 minutes for them to appear in the MMM-CalendarExt3.

Why is this taking so long? They are sent to MMM-CalendarExt3 via “broadcastEvents” or what?

I took a look at the sizes of my calendar:

Calendar 1: 1,068,804 bytes
Calendar 2: 697,796 bytes
Calendar 3: 398,683 bytes
Calendar 4: 150,176 bytes
Calendar 5: 73,128 bytes
Calendar 6: 70,476 bytes
Calendar 7: 23,940 bytes
Calendar 8: 17,221 bytes

Calendars 6-8 are displayed relatively quickly in MMM-CalendarExt3, calendar 5 a little later. The others require the previously mentioned 5-10 minutes.

Could it have something to do with size?

sdetweil

@princemaxwell can you show the console log of mm with the timestamps for the broadcasts from the default calendar?

be careful as these message now expose the calendar url string…

princemaxwell

@sdetweil

No problem, i removed fragments from the share link and named them like my list above, sorted by size, cal-8 is the smallest filesize.

> magicmirror@2.26.0 start
> DISPLAY="${DISPLAY:=:0}" ./node_modules/.bin/electron js/electron.js

[20.03.2024 15:28.52.845] [LOG]   Starting MagicMirror: v2.26.0
[20.03.2024 15:28.52.851] [LOG]   Loading config ...
[20.03.2024 15:28.52.856] [DEBUG] config template file not exists, no envsubst
[20.03.2024 15:28.52.862] [LOG]   Loading module helpers ...
[20.03.2024 15:28.52.864] [LOG]   No helper found for module: alert.
[20.03.2024 15:28.52.898] [LOG]   Initializing new module helper ...
[20.03.2024 15:28.52.898] [LOG]   Module helper loaded: updatenotification
[20.03.2024 15:28.52.900] [LOG]   No helper found for module: clock.
[20.03.2024 15:28.53.231] [LOG]   Initializing new module helper ...
[20.03.2024 15:28.53.232] [LOG]   Module helper loaded: calendar
[20.03.2024 15:28.53.234] [LOG]   No helper found for module: MMM-CalendarExt3.
[20.03.2024 15:28.53.235] [LOG]   All module helpers loaded.
[20.03.2024 15:28.53.244] [LOG]   Starting server on port 8080 ... 
[20.03.2024 15:28.53.275] [LOG]   Server started ...
[20.03.2024 15:28.53.278] [LOG]   Connecting socket for: updatenotification
[20.03.2024 15:28.53.280] [LOG]   Starting module helper: updatenotification
[20.03.2024 15:28.53.281] [LOG]   Connecting socket for: calendar
[20.03.2024 15:28.53.282] [LOG]   Starting node helper for: calendar
[20.03.2024 15:28.53.283] [LOG]   Sockets connected & modules started ...
[20.03.2024 15:28.53.918] [LOG]   Launching application.
[20.03.2024 15:28.57.031] [LOG]   Create new calendarfetcher for url: https://p63-caldav.icloud.com/published/2/MTA-cal-2 - Interval: 3600000
[20.03.2024 15:28.57.194] [LOG]   Create new calendarfetcher for url: https://p63-caldav.icloud.com/published/2/MTA-cal-4 - Interval: 3600000
[20.03.2024 15:28.57.198] [LOG]   Create new calendarfetcher for url: https://p63-caldav.icloud.com/published/2/MTA-cal-3 - Interval: 3600000
[20.03.2024 15:28.57.201] [LOG]   Create new calendarfetcher for url: https://p29-caldav.icloud.com/published/2/ODA-cal-1 - Interval: 3600000
[20.03.2024 15:28.57.205] [LOG]   Create new calendarfetcher for url: https://p63-caldav.icloud.com/published/2/MTA-cal-5 - Interval: 3600000
[20.03.2024 15:28.57.208] [LOG]   Create new calendarfetcher for url: https://i.cal.to/ical/3/borussiadortmund/bundesliga-spielplan/cal-6.ics - Interval: 3600000
[20.03.2024 15:28.57.217] [LOG]   Create new calendarfetcher for url: https://i.cal.to/ical/77/nrw/ferien/cal-8.ics - Interval: 3600000
[20.03.2024 15:28.57.221] [LOG]   Create new calendarfetcher for url: https://i.cal.to/ical/61/nrw/feiertage/cal-7.ics - Interval: 3600000
[20.03.2024 15:28.57.224] [INFO]  updatenotification: Updater Class Loaded!
[20.03.2024 15:28.57.225] [INFO]  updatenotification: Checking PM2 using...
[20.03.2024 15:28.57.227] [INFO]  Checking git for module: MMM-CalendarExt3
[20.03.2024 15:28.57.290] [INFO]  Checking git for module: MagicMirror
[20.03.2024 15:28.57.729] [INFO]  Calendar-Fetcher: Broadcasting 2 events from https://i.cal.to/ical/61/nrw/feiertage/cal-7.ics.
[20.03.2024 15:28.57.825] [INFO]  Calendar-Fetcher: Broadcasting 1 events from https://i.cal.to/ical/77/nrw/ferien/cal-8.ics.
[20.03.2024 15:28.59.069] [INFO]  Calendar-Fetcher: Broadcasting 5 events from https://i.cal.to/ical/3/borussiadortmund/bundesliga-spielplan/cal-6.ics.
[20.03.2024 15:29.00.814] [INFO]  Calendar-Fetcher: Broadcasting 4 events from https://p63-caldav.icloud.com/published/2/MTA-cal-5.
[20.03.2024 15:29.00.869] [INFO]  updatenotification: You are using pm2 with MagicMirror
[20.03.2024 15:29.02.223] [INFO]  Calendar-Fetcher: Broadcasting 1 events from https://p63-caldav.icloud.com/published/2/MTA-cal-4.
[20.03.2024 15:29.04.265] [INFO]  Calendar-Fetcher: Broadcasting 7 events from https://p63-caldav.icloud.com/published/2/MTA-cal-3.
[20.03.2024 15:29.49.438] [INFO]  Calendar-Fetcher: Broadcasting 8 events from https://p63-caldav.icloud.com/published/2/MTA-cal-2.
[20.03.2024 15:30.30.417] [INFO]  Calendar-Fetcher: Broadcasting 12 events from https://p29-caldav.icloud.com/published/2/ODA-cal-1.

MMRIZE

@princemaxwell
Now I know what happens. Your symptom is not related with any errors.
Let me explain the process.

Each calendar parsing/fetching in the calendar module will take some time, a few seconds or some dozen seconds. How long will it take is unpredictable.
The default calendar module will refresh itself and broadcast accumulated events on any calendar fetched. So, in some unlucky cases, the module may flicker due to too-short-term refreshing.
CX3’s view is very complex and heavy for redrawing frequently. To avoid that flickering, CX3 will wait some time after any broadcasted event arrives (waitFetch: 5000ms by default) to check whether the following broadcasting will arrive in the short term. If no other followed broadcasting happens, stop waiting and process collected events at that moment and draw the view.
This means that if you have 8 calendars and 5 calendars complete fetching in 4 seconds and the other 3 calendars may take over 10 seconds, only the events from the earlier 5 calendars will be collected and prepared to show before the first rendering time(waitFetch 5000ms).
After the first rendering, the remaining 3 calendars may complete fetching. CX3 receives those events but does not render them instantly to avoid flickering. The rendering will be done every refreshInterval (by default 10 minutes) with stocked events that are received during that period.
That is the reason why some of your calendars will be shown the first time, but others are applied after 10 minutes.

So, you may have these options.

Do nothing. Just wait 10 minutes; the lazy events will normally be applied to CX3 from the second refreshing cycle. Calendar events are not regarded as needing urgent, instant reflection. Your MM may stay on for a long time. Ten minutes after the first bootup will not be a big issue, and your family will not care.
Or Give enough long waitFetch. Your log showed that almost 2 minute is needed to complete fetching 8 calendars entirely. waitFetch: 1000 * 60 * 2 may work. (The default value is 5000ms, not 300000) (This means you need 2 minutes before drawing the CX3 view at first time.)
Or Give shorter refreshInterval to redraw and reflect the change of events. The default value is 1000 * 60 * 10 (10 minutes) But you can assign shorter. But 99.99% of refreshing will deal unchanged same events. So I think it is a kind of waste. Anyway, if you want instant reflection, give it shorter value.

princemaxwell

@MMRIZE

All right, that makes complete sense.
I understand that now.

CX3 initially only recognizes broadcast events and does not know how many calendars are configured and waits until all of them have been processed.

(The default value is 5000ms, not 300000)

By the way, with the 300000 I meant the time from the default calendar, the “fetchInterval”, whose default value is: 300000 (5 minutes).

I tried the 2nd option, unfortunately it doesn’t work, I set it to 1000 * 60 * 2, but it takes exactly 10 minutes.

When and how often is waitFetch executed, just once? I would have expected CX3 to notice a broadcast and then update according to waitFetch.

MMRIZE

@princemaxwell
waitFetch will be applied only to the very first broadcast notification. If you set waitFetch: 120000, this sequence will happen (I’ll assume the calendar module is described before CX3 in config.js)

the calendar module starts the parsing of calendars.
CX3 module starts. Library is loaded and the DOM is prepared. It is now ready to receive the events.
The first calendars might complete its fetching. then calendar module will broadcast the events at that moment.
CX3 receive the first broadcasted events. Wait 120000ms(2 minute) from that moment to get other lazy broadcasting coming.
Other calendars might be fetched. But some might still need to be completed.
Anyway, after 120000ms from the first broadcast, CX3 will draw the events that have arrived till then. (It means the view of CX3 is shown for the first time after at least 2 minutes from the boot)
Then after 10minutes(refreshInterval), CX3 will refresh again. Maybe until then, all calendars would have been fetched.(I hope.)

Anyway your log was showing the 1st calendar fetching as below;

[20.03.2024 15:28.57.729] [INFO]  Calendar-Fetcher: Broadcasting 2 events from ...

And the 8th fetching ;

[20.03.2024 15:30.30.417] [INFO]  Calendar-Fetcher: Broadcasting 12 events from ...

It takes 93 seconds to complete. So I think waitFetch: 120000 (120 seconds) would be enough; however, if there are still missing events, the possible reasons would be;

It takes more than 120 seconds at that moment randomly to complete fetching of all calendars since first fetching,
some calendars were too fast-fetched before CX3 was ready. so they were missed. (all modules are loaded and starts sequentially, so if you have certain number of modules, CX3 may not be ready until that moment.)

Of course, you can adjust the order of modules in config, assign needed values of waitFetch and refreshInterval to show the entire events properly on the first boot time.

But, my suggestion is,
Just leave them and do nothing.
After 10 minutes from the first bootup, everything will be OK. Your MM might be rebooted at midnight or in the morning on real-life usage. MM would stay turned on all day or for several days. So literally, nobody will care, even realize there had been some missing events on the bootup time. You don’t have to make an effort to solve this unworthy issue.

princemaxwell

@MMRIZE

The standards are absolutely sufficient for everyday use.

But while I position the modules and adapt the CSS to my own wishes and test the functions of the modules, the long time is rather annoying and I wanted to shorten it.

I have now activated the following config and restarted.
I recorded the contents of the screen and looked at exactly when and what happened.

	{
		module: "MMM-CalendarExt3",
		position: "middle_center",
		disabled: false,
		waitFetch: 1000 * 60 * 2,
		refreshInterval: 1000 * 60 * 2,
	},
	{
		module: "MMM-CalendarExt3Agenda",
		position: "top_center",
		disabled: false,
		waitFetch: 1000 * 60 * 2,
		refreshInterval: 1000 * 60 * 2,
		title: "Kalender",
		config: {
			instanceId: "basicCalendar",
			locale: 'de',
			firstDayOfWeek: 1,
			startDayIndex: -1,
			endDayIndex: 10,
			calendarSet: []
	  }
	},

The result is that the config is ignored.

CX3 starts after the calendar module and displayed the first entries of the 3 small calendars as soon as they appeared.
CX3 only refreshes after 10 minutes. You can tell this by the display briefly disappearing and then fading in again. Then all entries are included.
CX3A also starts after the calendar module and also shows the entries of the 3 small calendars directly.
CX3A refreshes after 30 minutes (default?). Even then, all entries are included.

Apparently waitFetch is ignored for both, otherwise, according to your explanation, the first entries would only appear after waitFetch. But these are there immediately as soon as the module has been loaded.

Other configurations are also ignored. For example, I used this for testing:

mode: ‘day’,

But i only get the week view.
When i change it in MMM-CalendarExt3.js, then it works.

Why is the config ignored or what am I missing?

MMRIZE

@princemaxwell
You did a simple mistake. Configuration parameters of the module should be placed inside of the module’s config:{}.