Any ideas how I can download a file from a web site that requires login mimicking clicks

pc3957 · May 26, 2020

Hi everyone,

What I'm trying to do is (I'm hoping) rather straight forward. I want to download a CSV file from a website (the website generates the CSV) and process it with PHP. I know how to do the processing, but I can't figure out how to get it from the website.

In my browser I would do the following:
1) Browse to the login URL
2) Login with my email address and password (via web forms) and click the Login button.
3) I'm redirected to the dashboard where several tables load by AJAX. The first table has a CSV button I click.
4) A CSV is downloaded to my computer. However URL is kind of cryptic (blob:

https://<website>/<random

UID>)

Does anyone know how I can mimic these actions in PHP? Is it possible?

Thanks in advance!

spamjim · May 26, 2020

Is this for some sort of unauthorized scraping or other exploit?

The web site publisher has likely set up this login process to prevent misuse of their content.

These login processes are employed because the site owner may want to trade your data or subscription payment for their data. The login process is most likely employed to prevent system resource abuse.

Instead of PHP, use TXT and send a message to the site's publisher to gain access to their data.

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Any ideas how I can download a file from a web site that requires login mimicking clicks

pc3957

ISP

spamjim

Instructor

Similar threads

Part and Inventory Search

Sponsor