We require a scrapy web scraper working on python 2.7 that will scrape the following site:
[login to view URL]
and will receive term(s) by user, search for relevant articles, scrape the title, author and article text and then crawl the comments as well, storing it all into a Mongo DB.
Code will require testing (unittest) and only experienced python developers need apply, please include previous (scrapy preferred) scraping projects.
*Immediate start and quick turnaround required*