Development of web scraper/crawler

We are seeking a computer programmer to build a custom web scraper for the HeinOnline law database. The scraper would be using a custom list of international treaties as the inputs and the output would be the number of hits (the number of unique articles that mention the treaty) for each treaty by year from 1945-2014. Ideally, we would want the results in a list format as well as a total for each treaty in each year. We would prefer the programmer to use a publicly available, open source scraper, such as Scrappy, so that the scraper/crawler could be adjusted if there were changes in our needs or to the HeinOnline database. We would provide access to the HeinOnline database. The scraper would require the following features: • The ability to search for a list of user-specified keywords • A GUI for the scraper so that we would be able to run the scraper at the time of our choosing and without reliance on the programmer • The ability to search multiple query names for a single treaty • A sustainable program that could be executed each year • The scraper must be able to search for the treaty names in the order the words appear “Charter of the United Nations” vs. “Charter”…”United Nations” • The output should also include metrics for the scrape itself such as date/time • The scraper must be able to access HeinOnline at varying intervals (1 second, 3 seconds, 7 seconds) so that we do not disturb the HeinOnline server There is the possibility for additional work and the development of additional scraping tools depending on our needs and the performance of the developer. A sample of the query names from a previous scrape of the data is attached. The developer should have proven experience in Python and Webscraping. Data science expertise or previous experience with an open source scrapping program is a plus. If you any questions, please feel free to reach out.