Possible cross-origin issue?

[toddk]

I copied the default newsfeed module to a new name and modified it so it will retrieve and display images from the news articles. I’ve run into a problem where I get a broken link icon where the image should be. I don’t see any errors so it seems like it’s a cross-origin problem.

I tested with the CORS proxy on and with it off.

CORS on

• no errors seen but broken link icon displays instead of the actual image
• If I open a browser and enter the CORS URL, e.g. http://localhost:8080/cors?url={image url} also get a broken link but if I enter view-source:http://localhost:8080/cors?url={imageUrl} I get the binary data from the image file returned so it seems the CORS proxy is working. Of course, I’m using the actual image URL in place of the string “{imageUrl}”

CORS off

• when a GET is issues for each image, the error ERR_HTTP2_PROTOCOL_ERROR is displayed. There is no response from the host of the image file

I’ve noticed other news modules support the use of images and I’ve installed one of those and the images display as expected.

What could be stopping images from displaying in my module?

I’m using MM 2.25.0.

[sdetweil]

@toddk and what url are the other new modules using for the url?.

what is the user agent set before the fetch of the news?

[toddk]

@sdetweil, an example of the requested image url when using the CORS proxy is
http://localhost:8080/cors?url=https://i.cbc.ca/1.3555961.1698520686!/fileImage/httpImage/image.jpg_gen/derivatives/16x9_620/the-banff-centre.jpg

The newsfeed is coming from https://www.cbc.ca/webfeed/rss/rss-topstories

I’m using the default user agent:
Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.190 Electron/26.2.4 Safari/537.36

I’m no expert in CORS but some response headers of interest are:

Access-Control-Allow-Origin: * (I assume this is good)
X-Frame-Options: SAMEORIGIN (I assume this isn’t a concern)
X-Permitted-Cross-Domain-Policies: none (I’m concerned this might be a problem)

[toddk]

@sdetweil BTW, I forgot to mention that one other newsfeed module that does display images, MMM-NewsAPI, is using newapi.org URLs, e.g. https://newsapi.org/v2/everything?c=ca&apikey={apikey}

Newsapi.org and cbc.ca handle images differently and have different response headers so that might be the reason one site’s feeds work while another one’s doesn’t.

[toddk]

After more testing, I see the default newsfeed module has the same problem. If I use a CBC news feed such as https://www.cbc.ca/webfeed/rss/rss-topstories and display it in full article mode, I get an HTTP2 protocol error if useCorsProxy is false and no images are returned when the proxy is turned on.

I tried other news feeds with my updated version of the newsfeed module and the images display properly when the CORS proxy is off. Turning the proxy on stops the images from downloading and broken link icons are shown instead.

It looks like the CORS proxy isn’t working in the newsfeed module.

[toddk]

I notice the CORS proxy is doing something odd with the Content-Type and Content-Length. It seems to be setting them wrong.

Most of the news feed images I’ve been accessing are in webp format even though their URLs show they are JPGs.

• When the CORS proxy is off, the Content-Type is correctly set to “image/webp” and the Content-Length is correct.
• When the CORS proxy is enabled, the Content-Type is incorrectly set to “image/jpeg” and the Content-Length is much larger than it should be.

Why would this happen and is there some way to correct it?

I tried adding expectedheaders=Content-Type,Content-Length to the CORS request but that didn’t seem to affect anything.

Here are headers for the same newsfeed image with (fails) and without (works) the CORS proxy.

Without CORS proxy:
Request
Request URL: https://globalnews.ca/wp-content/uploads/2023/11/icbc-pic.jpg?quality=85&strip=all
Request Method: GET
Status Code: 200
Remote Address: 192.0.66.184:443 (???)
Referrer Policy: no-referrer
Request headers
:authority: globalnews.ca
:method: GET
:path: /wp-content/uploads/2023/11/icbc-pic.jpg?quality=85&strip=all
:scheme: https
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB
Sec-Ch-Ua: “Not)A;Brand”;v=“24”, “Chromium”;v=“116”
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: “Linux”
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.190 Electron/26.2.4 Safari/537.36
Response headers
Accept-Ranges: bytes
Cache-Control: max-age=31536000
Content-Length: 84572
Content-Type: image/webp
Date: Mon, 06 Nov 2023 04:46:32 GMT
Etag: “b8bb49c46d684e95”
Last-Modified: Mon, 06 Nov 2023 01:55:46 GMT
Server: nginx
Strict-Transport-Security: max-age=86400
Vary: Accept
X-Cache: HIT
X-Rq: sea4 116 203 443

When the CORS proxy is used, the image isn’t returned.
Request
Global with CORS
Request URL: http://localhost:8080/cors?url=https://globalnews.ca/wp-content/uploads/2023/11/icbc-pic.jpg?quality=85&strip=all
Request Method: GET
Status Code: 200 OK
Remote Address: [::1]:8080
Referrer Policy: no-referrer
Request headers
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/,/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB
Connection: keep-alive
Host: localhost:8080
Sec-Ch-Ua: “Not)A;Brand”;v=“24”, “Chromium”;v=“116”
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: “Linux”
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (X11; Linux armv7l) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.5845.190 Electron/26.2.4 Safari/537.36
Response headers
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 274934
Content-Type: image/jpeg; charset=utf-8
Date: Mon, 06 Nov 2023 04:41:38 GMT
Etag: W/“431f6-FDM8wfRxenBocIzJBv0QxLtyTa0”
Keep-Alive: timeout=5
Referrer-Policy: no-referrer
Strict-Transport-Security: max-age=15552000; includeSubDomains
X-Content-Type-Options: nosniff
X-Dns-Prefetch-Control: off
X-Download-Options: noopen
X-Frame-Options: SAMEORIGIN
X-Permitted-Cross-Domain-Policies: none
X-Xss-Protection: 0

[sdetweil]

@toddk you should open an issue.