Reverse Lookup MMM-FRITZ-Box-Callmonitor - help needed

AxLed

@pinsdorf
I am moving forward and getting closer to my target.
With your tips i got rid of the Express framework (calling http://localhost/scrape to start web scraping). I also found a way two show the results in the console and addition writing a file (for my personal debugging).

Right now i have to files:
package.json

{
  "name"         : "MMM-Alex2",
  "version"      : "0.0.1",
  "description"  : "Reverse Lookup Phonenumber.",
  "main"         : "server.js",
  "author"       : "Alex",
  "repository"   : {
    "type" : "git",
    "url"  : "https://github.com/"
  },
  "dependencies" : {
    "request"    : "latest",
    "cheerio"    : "latest"
  }
}

server.js

var request = require('request');
var cheerio = require('cheerio');
var moment = require('moment');
var fs      = require('fs');

var callnr = '0049xxxx';
var url = 'http://www.dasoertliche.de/?form_name=search_inv&ph=' + callnr;

title = '';
adresse = '';
anrufer = 'kein Treffer auf Örtliche.de für '+ callnr;

request(url, function (error, response, html) {
  if (!error) {
	var $ = cheerio.load(html);
	
	$('.left').filter(function(){
        var data = $(this);
        
		title = data.text().trim();
		//adresse = data.children().children().text().trim();
		adresse = data.children().eq(2).text().trim();
		})
		
	$('.name ').filter(function(){
        var data = $(this);
        anrufer = data.text().trim();
		})

    console.log('Anrufer: ' + anrufer);
	console.log('Adresse: ' + adresse);
    //console.log('Title: ' + title);
	console.log('Gesamt: ' + anrufer + ', ' + adresse + ' - '+ moment().format("DD.MM.YYYY HH:mm"));
    
	//Daten fortschreiben
	fs.appendFile('anrufe.txt', moment().format("DD.MM.YYYY HH:mm") + ' Gesamt: ' + anrufer + ', ' + adresse +'\n', function(err){
      console.log('File successfully written!');
    })
	
  }
  else {
    console.log("We’ve encountered an error: " + error);
  }
});

If i put them in a folder (under MagicMirror/modules) an do a npm install in that folder, it will install the depencies.

If in now run node server.js in MagicMirror/modules/FOLDERNAME i will get the result of a static phonenumber in the console and in a textfile in FOLDERNAME.

My next step will be the implementation in MMM-Fritz-Box-Callmonitor, as you suggested.

AxLED

AxLed

Hi,

i tried to implement Reverse lookup, so i extended the package.json file about the needed depencies and did a npm install afterwards, which worked fine.

Now i am struggeling in implementing aditional code in node_helper.js, as i dont know how to debug it. Some bugs are show in the console, as i start MM, but some not.

Is there a way to test node_helper.js by itself?
Or, how do you test new modules / module modifications?

I am looking forward to get some tipps.

Regards AxLED

pinsdorf

Hi @AxLed, great to see your progress!

There are a few remarks on testing that come to mind:

I personally like to develop the module on my Windows PC using VisualStudio Code and additional tooling from the Internet as I need it. For this you have to install the MagicMirror on your PC. @Mykle1 has a nice walkthrough document for doing so.
MM has a development mode. Start MM (with your to be tested module) with the command npm start dev. This starts a browser UI for MM and in addition shows you the loaded HTML, CSS, source code, etc. In the browser’s source window you can set breakpoints and once they get hit, the program execution pauses and you have time to inspect variables. Please note that you don’t have to stop and start MM all the time with the command above. Just change source code in your programming editor, save it, and reload the MM browser (F5). This works both on Raspberry Pi and Windows PC.
If you want to test part of your code in isolation, then you can simply write a small helper program debug.js which calls your test subject’s methods the way you need it. You find this approach in the debug.js of calendar module to test the calendar fetcher component. In this case you do not start MM, but just execute your helper program with node debug.js.
You can write a real (automated) test suite, but this may go a little beyond your current task. A test suite is helpful when you do test driven development and/or when you want to guarantee that a future change does not break expected behaviour.

pinsdorf

Oh, one idea for your module. The calendar has a dedicated calendar fetcher as mentioned in my previous post. This may be interesting for your reverse lookup as well. Other people may want to use other online phone books, e.g. for different countries, and by configuring the right reverse lookup fetcher they could do so. Maybe have a look at the code of the calendar fetcher and the node helper where it is being called. However, IMHO you should first get your module working as it is. This is a suggestion for a future extension.

AxLed

@pirnsdorf

Thanks for the information, so i started one of my MM-Dev-Environments (Ubuntu 16.04) and run MM with npm start dev, unfortunatelly i can not mark node_helper.js, as i only see the MODULNAME.js and the MODULNAME.css.

The part of code i want to debug is in node_helper.js, how can i do this?
Screenshot:

AxLED

pinsdorf

Hmm, this is weird. The original MMM-FRITZ-Box-Callmonitor comes with a node_helper.js. Are you sure you did not delete it by accident resp. forgot to copy it to your dev environment? Moreover package.json and the README.md are missing.

Are you familiar with git? I’d advice that you fork the MMM-FRITZ-Box-Callmonitor into your own GitHub account and make all your changes there. This would also allow me to read your latest code after each git commit. From this forked repository there is an easy way to get your improvement back into the original repository once you are done.

AxLed

Hi pinsdorf,
all mentioned files are in the folder MMM-FRITZ-Box-Callmonitor but on debugging, i only can choose from the ones above (see my screenshot).

I will follow your suggestion with git, if my code adaptation has passed the beta phase, as i still have some problems.

AxLed

I almost made it, for debugging i went this way (as i dont know any better so far):

Clientside: Browser with debugging feature
Serverside (like node_helper.js): console.log on every important location in the code

My workaround (so far):
I cloned the function getName to getName2 and use Reverselookup of phonenumbers only in getName2.
I changed the call of getName to getName2 in following places:
Line 80-98:

//Incoming call
		monitor.on("inbound", function(call) {
			//If caller is not empty
			if (call.caller != "") {
				self.sendSocketNotification("call", self.getName2(call.caller));
			};
		});

		//Call accepted
		monitor.on("connected", function(call) {
			self.sendSocketNotification("connected", self.getName2(call.caller));
		});

		//Caller disconnected
		monitor.on("disconnected", function(call) {
			//send clear command to interface
			self.sendSocketNotification("disconnected", {"caller": self.getName2(call.caller), "duration": call.duration});
		});
console.log(this.name + " is waiting for incoming calls.");

Reasons why i did this workaround: loadCallList
This function is called initialy, when you start MM and everytime you get a call. I dont understand, why loadCallList goes through all calls from the fritzbox journal in the past (infinitely), no matter what configuration options of the module you choosed. This gives me an "Socket hang up" error on reverse lookup after 100 requests. Now you know why i did this workaround.

Regards

AxLED

barnosch

cool feature, i will definitely upgrade to that version, if it is running well
thumbsup
and now the but…
is this really necessary these days?
I don’t know any of my friends or our generation who really publicate their phonenumber in the public phonebook.
Mobile numbers especially not.
So it is only helpful for bigger companies, right?

yawns

@barnosch
Not necessarily. Even smaller companies (those without one base number and dozens of extensions) do add their phone number to public phonebooks. My phone solution does use online lookup and lots of craftspeople who called during our house building phase where identified on the phone showing “Dachdecker Hildebrandt” (for example).