MMM-News and BBC

sdetweil

  source: [Object],

its in there

DarrenHill

@MMRIZE said in MMM-News and BBC:

@DarrenHill
Ok, I found something. I don’t know why, but your result has “articles.source.Name” instead of “articles.source.name” that makes this error.

Anything I can do to overcome that?

No idea why I’m getting different API returns to you guys…?

sdetweil

@DarrenHill @MMRIZE

also doesn’t get ‘content’ or ‘description’, so those will fail too…

DarrenHill

I tried to register for a new api key, but when I tried it told me that they don’t support the UK (where I am) so I wonder if that somehow has something to do with it.

This has obviously changed since I first registered, and given I’m getting some response from the API it is at least partially working, but besides that I have no idea…

mumblebaj

@DarrenHill Hi. Apologies for the late reply. I am on a different timezone than the other guys.

I would suggest using MMM-NewsAPI. The same API key that you are using with MMM-News will work.

DarrenHill

@mumblebaj - Thanks, no need to apologise.

I tried the other module, and it also doesn’t show anything besides it’s “News” header title.

The config I tried was:

        {
            module: "MMM-NewsAPI",
            header: "News",
            position: "bottom_center",
            config: {
                apiKey : "<my API key>",
                domains: "bbc.co.uk",
                language: "en"
                }
        },

And also I tried replacing the domains: with sources: “bbc-news” and that gave the same results.

There’s nothing in the logs, aside from confirming that the module is loading (as can be seen from the displayed title too).

sdetweil

@DarrenHill from the doc

Add DEBUG Handling.
Only add debug: true to the config section if you are experiencing issues and require more information to find issues.

sdetweil

@DarrenHill domins or sources must be inside the much more complex query section of the config

see the doc

                    query: {
                            country: "",
                            category: "",
                            q: "",
                            qInTitle: "",
                            sources: "",
                            domains: "cnn.com,nytimes.com,news24.com",
                            excludeDomains: "",
                            language: ""
                    }

mumblebaj

@DarrenHill MMM-NewsAPI has a whole lot more configs. See below sample.

{
                        module: "MMM-NewsAPI",
                        header: "news",
                       //  disabled: true,
                        position: "bottom_bar",
                        pages: {"sport": "bottom_bar"},
                        config: {
                                apiKey: <api-key>,
                                type: "horizontal",
                                choice: "everything",
                                pageSize: 20,
                                sortBy: "publishedAt",
                                drawInterval: 1000*30,
                                templateFile: "template.html",
                                fetchInterval: 1000*60*60,
                                query: {
                                        country: "",
                                        category: "",
                                        q: "",
                                        qInTitle: "",
                                        sources: "",
                                        domains: "bbc.co.uk",
                                        excludeDomains: "",
                                        language: ""
                                        }
                                }
				},

DarrenHill

@mumblebaj - Thanks for the support, and for both modules.

Unfortunately that gives me a headline in Spanish (I think) about Apple iPhone 14 with “undefined” below it.

I also tried your everything example configs from your GH page and got a similar layout (headline and undefined).

I’m beginning to think NewsAPI itself is screwed for me unfortunately, so we may be onto a loser here regardless of module.

sdetweil

@mumblebaj i commented out your disabled: true,

mumblebaj

@DarrenHill That is very weird as the configs seem to work for me and the other guys who have tried to assist thus far.

DarrenHill

@mumblebaj - Yes, as I said I’m starting to think that your modules are both working fine, but the API feed I’m getting from NewsAPI is somehow corrupted and they are universally having issues with it.

I can’t create a new API key as it won’t let me (it says they don’t support UK now if you try and create a new key from here), and I guess that might be related somehow?

So I think we might be at a dead end here unfortunately with anything that’s using that API as its source.

mumblebaj

@DarrenHill The data coming through is fine as it works for me with the source that you provided. Not sure why you not able to create an API key though. I just tested creating another API Key and that worked as well. Unfortunately I am not able to assist with getting a key created.

sdetweil

@DarrenHill see my chat message

sdetweil

@mumblebaj newsapi.org free is only 100 calls per day

I guess 4/hour is ok

DarrenHill

@sdetweil - thanks - tried your API key and it gives exactly the same results as my original one.

I also put my original API key into the chat, if you have time and are able could you test it for me?

I’m starting to think that for some reason NewsAPI cannot provide a proper service to UK IP addresses or something like that.

And I guess if that’s the case then we’re stuck, as everything provided by MM and you guys is all working fine, it’s just their API data that is for some reason messed up.

By the way, can I take this moment to also thank both you and @mumblebaj for the support to look into this.

DarrenHill

To add - I checked via the web link (in Chrome) and both keys give exactly the same response:

Original:

{"status":"ok","totalResults":10,"articles":[{"title":"Charles formally confirmed as king in ceremony televised for first time","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T09:22:16.0519529Z","url":"http://www.bbc.co.uk/news/uk-62860893"},{"title":"Kharkiv offensive: Russia surprised as Ukraine takes several towns - UK","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T08:37:17.1082493Z","url":"http://www.bbc.co.uk/news/world-europe-62860774"},{"title":"Sir Elton John pays tribute to Queen Elizabeth II","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T01:22:23.3573709Z","url":"http://www.bbc.co.uk/news/entertainment-arts-62860047"},{"title":"In pictures: Queen Elizabeth II in North America","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T21:37:18.1845741Z","url":"http://www.bbc.co.uk/news/world-us-canada-62854864"},{"title":"What do we call the Royal Family now?","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T18:22:20.0913328Z","url":"http://www.bbc.co.uk/news/uk-62854804"},{"title":"In pictures: King Charles' first full day on the throne","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T17:07:22.1996941Z","url":"http://www.bbc.co.uk/news/in-pictures-62797450"},{"title":"Six times the Queen made us laugh","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T16:22:20.8594645Z","url":"http://www.bbc.co.uk/news/uk-62849166"},{"title":"Outpouring of sympathy across the Commonwealth","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T12:52:20.8883515Z","url":"http://www.bbc.co.uk/news/world-62852412"},{"title":"Queen Elizabeth II: Your tributes to UK's longest-reigning monarch","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T09:07:26.059624Z","url":"http://www.bbc.co.uk/news/live/uk-62847455"},{"title":"Royal rebranding: What will happen to stamps, coins, banknotes and passports?","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T07:22:20.9827975Z","url":"http://www.bbc.co.uk/news/uk-59596996"}]}

New:

{"status":"ok","totalResults":10,"articles":[{"title":"Charles formally confirmed as king in ceremony televised for first time","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T09:22:16.0519529Z","url":"http://www.bbc.co.uk/news/uk-62860893"},{"title":"Kharkiv offensive: Russia surprised as Ukraine takes several towns - UK","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T08:37:17.1082493Z","url":"http://www.bbc.co.uk/news/world-europe-62860774"},{"title":"Sir Elton John pays tribute to Queen Elizabeth II","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-10T01:22:23.3573709Z","url":"http://www.bbc.co.uk/news/entertainment-arts-62860047"},{"title":"In pictures: Queen Elizabeth II in North America","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T21:37:18.1845741Z","url":"http://www.bbc.co.uk/news/world-us-canada-62854864"},{"title":"What do we call the Royal Family now?","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T18:22:20.0913328Z","url":"http://www.bbc.co.uk/news/uk-62854804"},{"title":"In pictures: King Charles' first full day on the throne","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T17:07:22.1996941Z","url":"http://www.bbc.co.uk/news/in-pictures-62797450"},{"title":"Six times the Queen made us laugh","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T16:22:20.8594645Z","url":"http://www.bbc.co.uk/news/uk-62849166"},{"title":"Outpouring of sympathy across the Commonwealth","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T12:52:20.8883515Z","url":"http://www.bbc.co.uk/news/world-62852412"},{"title":"Queen Elizabeth II: Your tributes to UK's longest-reigning monarch","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T09:07:26.059624Z","url":"http://www.bbc.co.uk/news/live/uk-62847455"},{"title":"Royal rebranding: What will happen to stamps, coins, banknotes and passports?","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-09-09T07:22:20.9827975Z","url":"http://www.bbc.co.uk/news/uk-59596996"}]}

So the data is there, it’s not the module or the code, so it must be the API response specifically to my IP address or set-up. And if it is, we’re stuck and I guess I’ll just have to remove the module from my set-up.

DarrenHill

Just to come back and close on this - I’ve now tested from Germany with and without a VPN (VPN end point is in the UK), using exactly the same URL and conditions (open in a browser under the VPN, disconnect the VPN and refresh the browser tab).

So I confirm that it is NewsAPI screwing around with the feed to UK IP addresses, so unfortunately nothing that can be done nor anything wrong with MagicMirror or its modules.

Probably best to mark this as solved, albeit basically crippled for UK users.

With VPN (UK IP end point):

{"status":"ok","totalResults":0,"articles":[{"title":"I was dragged into China consulate, protester Bob Chan says","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T17:22:20.9801343Z","url":"http://www.bbc.co.uk/news/world-asia-china-63318285"},{"title":"James Webb telescope spies 'Pillars of Creation'","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T17:22:16.2918895Z","url":"http://www.bbc.co.uk/news/science-environment-63319814"},{"title":"Watch Ukrainian troops under close gunfire","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T16:22:14.7766328Z","url":"http://www.bbc.co.uk/news/world-europe-63318579"},{"title":"Suella Braverman resigns as home secretary","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T15:52:31.1666375Z","url":"http://www.bbc.co.uk/news/live/uk-politics-63309400"},{"title":"Putin decree is doubling down as pressure grows","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T15:07:17.5111432Z","url":"http://www.bbc.co.uk/news/world-europe-63319100"},{"title":"Anna May Wong: Actress becomes first Asian American on US currency","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T14:07:23.619259Z","url":"http://www.bbc.co.uk/news/entertainment-arts-63316940"},{"title":"The 800-day US visa wait causing India despair","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T12:37:20.603458Z","url":"http://www.bbc.co.uk/news/world-asia-india-63301257"},{"title":"Five hours' sleep is tipping point for bad health","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T10:22:24.3232143Z","url":"http://www.bbc.co.uk/news/health-63284305"},{"title":"Liz Truss: What the world is saying about UK turmoil","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-19T10:22:21.1348437Z","url":"http://www.bbc.co.uk/news/world-63293009"},{"title":"Florida flesh-eating illness cases spike after Hurricane Ian","author":"BBC News","source":{"Id":"bbc-news","Name":"BBC News"},"publishedAt":"2022-10-18T22:07:21.4822377Z","url":"http://www.bbc.co.uk/news/world-us-canada-63309246"}]}

Without VPN (German IP end point):

{"status":"ok","totalResults":10,"articles":[{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Joe Biden's unusual fight to bring down gas prices","description":"The US president has gone to unusual lengths to try to rein in the price of fuel.","url":"http://www.bbc.co.uk/news/business-63233077","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/8A9C/production/_127248453_c5781efaa0145b7fe240fb16f434f71e45fab20b.jpg","publishedAt":"2022-10-19T19:07:22.1820718Z","content":"When he campaigned for president two years ago, Joe Biden had lofty economic ambitions - shifting the country away from fossil fuels, shoring up its safety net and advancing fairness and equality.\r\nN… [+7708 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"I was dragged into China consulate, protester Bob Chan says","description":"A Hong Kong pro-democracy protester gives his account of violent scenes at Manchester's Chinese consulate.","url":"http://www.bbc.co.uk/news/world-asia-china-63318285","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/0E96/production/_127243730_ab007ba2-be91-4e18-9589-93e48eab6dea.jpg","publishedAt":"2022-10-19T17:22:20.9801343Z","content":"A protester says he was not trying to enter Manchester's Chinese consulate during a pro-democracy demonstration that saw violent scenes on Sunday.\r\nBob Chan told a news conference he was dragged onto… [+4155 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"James Webb telescope spies 'Pillars of Creation'","description":"One of the most famous scenes in the cosmos is viewed by the new, Nasa-led super space observatory.","url":"http://www.bbc.co.uk/news/science-environment-63319814","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/3666/production/_127262931_weic2216a.jpg","publishedAt":"2022-10-19T17:22:16.2918895Z","content":"It's a classic - one of the most beautiful sights in the cosmos and now the new super space telescope James Webb has visited it anew.\r\nThe so-called \"Pillars of Creation\" are cool, dense clouds of hy… [+1637 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Watch Ukrainian troops under close gunfire","description":"War in eastern Ukraine involves not only artillery strikes, but close combat, as the BBC's Jonathan Beale finds.","url":"http://www.bbc.co.uk/news/world-europe-63318579","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/13E80/production/_127263518_p0d81n2f.jpg","publishedAt":"2022-10-19T16:22:14.7766328Z","content":"Fighting on the front line near the city of Lyman in eastern Ukraine involves not only artillery strikes, but also close quarters combat, as the BBC's Jonathan Beale finds out.\r\nCamera: Lee Durant\r\nP… [+20 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Suella Braverman resigns as home secretary","description":"The home secretary was appointed by Prime Minister Liz Truss in September and her departure will add turmoil around No 10.","url":"http://www.bbc.co.uk/news/live/uk-politics-63309400","urlToImage":"https://m.files.bbci.co.uk/modules/bbc-morph-news-waf-page-meta/5.3.0/bbc_news_logo.png","publishedAt":"2022-10-19T15:52:31.1666375Z","content":"If you're just joining us or are in need of a catch up, here four key things we have learned so far:\r\nPensions triple lock to stay\r\nIn a surprise revelation at Prime Minister's Questions earlier, Liz… [+1452 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Putin decree is doubling down as pressure grows","description":"Vladimir Putin is tightening security across Russia, with the introduction of different security levels.","url":"http://www.bbc.co.uk/news/world-europe-63319100","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/E048/production/_127261475_mediaitem127261471.jpg","publishedAt":"2022-10-19T15:07:17.5111432Z","content":"Vladimir Putin is under growing pressure.\r\nHis \"special military operation\" has not gone according to plan. As a result of the Ukrainian counter-offensive, Russia has been losing territory it had occ… [+3001 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Anna May Wong: Actress becomes first Asian American on US currency","description":"She will appear as part of an effort to feature notable women on American quarters.","url":"http://www.bbc.co.uk/news/entertainment-arts-63316940","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/10686/production/_127260276_gettyimages-154047962.jpg","publishedAt":"2022-10-19T14:07:23.619259Z","content":"Actress Anna May Wong is set to become the first Asian American to be featured on US currency.\r\nShe will appear as part of an effort to feature notable women on American quarters.\r\nWong, who is consi… [+1960 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"The 800-day US visa wait causing India despair","description":"The US is struggling to clear a large backlog of visas, affecting thousands of Indians.","url":"http://www.bbc.co.uk/news/world-asia-india-63301257","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/16020/production/_127244109_p0d7r05y.jpg","publishedAt":"2022-10-19T12:37:20.603458Z","content":"The US is struggling to clear a large backlog of visa applications after it halted almost all processing operations during Covid-19. \r\nThis has severely disrupted Indian businesses and hit hundreds o… [+397 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Five hours' sleep is tipping point for bad health","description":"Getting less than this may increase the risk of a multitude of chronic conditions, say researchers.","url":"http://www.bbc.co.uk/news/health-63284305","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/1F7E/production/_127226080_gettyimages-1333374427.jpg","publishedAt":"2022-10-19T10:22:24.3232143Z","content":"At least five hours sleep a night may cut the over-50s' chances of multiple chronic health problems, researchers say. \r\nIll health can disrupt sleep - but poor sleep may also be a forewarning or a ri… [+2713 chars]"},{"source":{"id":"bbc-news","name":"BBC News"},"author":"BBC News","title":"Liz Truss: What the world is saying about UK turmoil","description":"BBC reporters around the globe explain how the political and economic upheaval is being viewed.","url":"http://www.bbc.co.uk/news/world-63293009","urlToImage":"https://ichef.bbci.co.uk/news/1024/branded_news/8647/production/_127257343_truss_getty976.jpg","publishedAt":"2022-10-19T10:22:21.1348437Z","content":"The world has been watching the political and economic upheaval in the UK over the past few weeks. \r\nThe havoc caused by Prime Minister Liz Truss's tax-cutting plan, followed by its withdrawal this w… [+4815 chars]"}]}

MMRIZE

So, the best solution would be modifying the source to ignore the capital letter of key identifiers on the resulting feed. I think it would be somewhat annoying but not impossible.