Get JSON data stuck behind CAPTCHA click???? i.e. US Gas Prices
-
Any suggestions on how to get the up-to-date gas prices in any US locality, such as from this URL:
https://www.geico.com/save/local-gas-prices/
Unfortunately (albeit understandably) there is a CAPTCHA to click before the prices are returned.
Any suggestions on an alternate source? Gas Buddy and the like doesn’t work well in rural US as the last consumer update on prices in my area was 3 years ago.
I contacted Mapquest API…they said that although they provide gas price data to humans on their website, it is not available for scripts running automated requests publicly (they also get data from OPIS).
Contacted OPIS customer support for APIs…no options to access this data for free from the primary source (OPIS).
Whichever source should get their updates from OPIS which charges a hefty price for their API use and is not feasable for end-users like me.
-
@hsukup1 It’s not an easy problem. The crowdsourced solutions are not good, and the private solutions are not cheap. There’s MyGasFeed, but like GasBuddy it relies on crowdsourced data. Do you need station-specific data? AAA has state and municipal data available on their website.
It looks like Geico also offers prices via the mobile app. You could install it, perform the search, and inspect the traffic and query. It could very well be making unobfuscated calls to the API. Just keep in mind that relying on that method, it may eventually break (especially if you attract attention with abuse of the API).
-
@hsukup1
Download their Android app and extract the APK file. then search for their URL (and API key if any).
