Script for Web Scraping - repost

Hello,

I have a requirement that calls for 2 scripts, both of which will be used for web scraping of datalossdb.org. Ideally the scripts should be writtern in Python, however could be in PHP also.

Script 1 - Will scrape using [login to view URL] through to the most recent incident which currently is [login to view URL]

Script 2 -This script will be used to look at the most recent incident ID contained with the export file created by script 1 and then try searching and scraping (if found) the next 5 incidents i.e. 9811,9812,9813,9813 and 9814. There should also be a timer within the script to set when the script shouls be run i.e. every 30 mins.

Please note -

I need to be able to specify in both of the scripts where the XML(extract) files will be created for windows and Linux

Normally you have to go via a login page, to be able to access the data

I do not want to to have to install any librarys or be tied to a particular web browser

A user should not see a web browser open up when the script is runing

In the attachment I have highlighted the information want scraping with yellow boxes. Both scripts should be for me to configure the location of where the export(outcome) will go.

Please ask if there are any questions

Thanks

PHP Python Web Scraping

Project ID: #4385402

About the project