Extract parts of web pages

Egnos

Hi there!
I am looking for a way to view and read web pages through the MM.
The biggest problem I notice is that no module has an automatic scrolling that allows the page to be read (I haven’t found any…).
Is there a way to extract even some information from a web page and view it on the MM?
Maybe as an RSS Feed…or some other way…

Thanks.

sdetweil

@Egnos what do you mean extract?

you can get it’s html and then get some of that

see the Cheerios lib

https://cheerio.js.org/

you can display the page using an iframe

karsten13

@Egnos

Maybe as an RSS Feed…or some other way…

the newsfeed module has support for showing the articles in an iframe, but AFAIK you have to send the notitfications yourself (e.g. with another module), see official docs for the module.

The problem (I think of all solutions) is Cors, most browsers including chromium (which is used by electron) become more and more restrictive. There is a pending bugfix to handle this better in the newsfeed module which will released with next mm release.

But some websites set headers to avoid being used in an iframe, at this point we can do nothing at the moment.

Egnos

I tried to use sites that create RSS feeds from a web page but they are very inaccurate.

Nobody knows an iframe module that allows automatic page scrolling?

sdetweil

@Egnos its doable,

iframe auto scrolling

but requires some code… even on the CSS approach cause you would want to wait some time,
then change the css style

I am not aware of a module that does this… ( or how it would decide it NEEDS to on a page by page basis)

Egnos

@sdetweil said in Extract parts of web pages:

( or how it would decide it NEEDS to on a page by page basis)

Scrolling a page would be enough (the web page you set in configuration …), when it arrives at the end it comes back up.
I speak as a non-developer, I don’t know how difficult it is …

sdetweil

@Egnos ok, so you assume that the page always needs to be scrolled

Egnos

@sdetweil Yes, in any case in order to read a web page it must be scrolled. Monitors usually can’t fit the full page …

Egnos

Would it be possible to view the contents of a text file on MM view a file (such as a .js or .json)?
Let me explain: since I don’t have the skills to automatically extract information from a website. I was thinking, for now, of manually updating a file every month.
I would need a form that would print this information contained in a file in local on the MM… (in the future I could automate the writing of this file …)

Do you think it would be possible?
If so, do you know of a module that could do this?

Thanks guys!

sdetweil

@Egnos from the 3rd party module list

https://github.com/SaltyRiver/MMM-SimpleText