New on LowEndTalk? Please Register and read our Community Rules.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
All new Registrations are manually reviewed and approved, so a short delay after registration may occur before your account becomes active.
Scraping website
I have one website where information (source code) is heavily obfuscated. For example in order to download file, link is generated from button like this:
<span class="export-xls dashboard-button-large resize-image-xls" ng-click="ctrl.exportXls()" title="Export data to Excel" ng-show="ctrl.state.filters.displayMode.selected == 'table'">
<img class="svg" src="/Static/burger.svg">
<i class="ie8-fallback fa fa-download"></i>
</span>
How to download such file automatically? I know there is selenium, but is there any other tools to do that?
Comments
That doesn't look obfuscated at all, it looks like normal single page app
I mean that is just for WordPress, but I usually use https://wordpress.org/plugins/wp-scraper/ to import content from another site.
Yes, but with simple cURL it's not possible to download file . So looking for a tool to do that.
Inspect the JS code so you can now what exportXls() does.
Also in Chrome dev tools you can check the Network tab, the XHR should be logged there with the params.
Ok, let's be specific about the task. Actually this is a challenge
Here is URL https://bit.ly/35TXww7
At the top right there is "Export to excel" button. Can anyone automate the download action?
Someone on Discord asked me to post this:
They are sending a request containing the HTML table that gets converted to XLS. The source of data is a simple GET request with JSON output: https://www.nordpoolgroup.com/api/marketdata/page/54?currency=,,,EUR
Thank you! Amazing.