Scraping competitor's website for specific data points
$30-250 USD
Completed
Posted over 8 years ago
$30-250 USD
Paid on delivery
***Please do not reply until you have visited the URL I need you to scrape, which is listed below (removed 'http://www' and using DOT net instead of .net to avoid bots etc., so you will need to adjust the URL to fix it before viewing in your browser)***
powertrainproducts DOT net/Engines-s/[login to view URL]
This is a straightforward scrape job. On this page, scroll down a bit and you will see that there are 100 engine products for sale on each page, and there are 17 pages. There are about 1,700 products total. The "page=" URL parameter controls the pagination.
We need the scraper to request each individual product page, for example this one (URL refromatted): powertrainproducts DOT net/Isuzu-88-91-1-6-4xe1-engine-p/[login to view URL]
There are 8 data points we need the scraper to collect from each individual product page.
You need to deliver this solution as a MySQL dump file (.sql) with 1 main table and 1 subtable. The main table is to store data points 1 through 7. I will explain the subtable in data point #8 below.
The 8 data points are marked in purple and numbered 1 through 8 in the attached screenshot.
Here are the 8 data points shown in the attached screenshot:
1. title
2. subtitle
3. price
4. stock status
5. availability + product code
6. all image URLs (full size)
7. all text from the 'Description' tab
8. all listed applications for each individual product (this should be a sub-table in the MySQL DB and linked by ID columns to the main table).
As you can see this website is organized easy for scraping.
We have the ability and know-how to do this job ourselves, but we do not have the time. So we need to hire you.
Please provide a reasonable quote only after you have FULLY reviewed all URLs, the attachment and you completely understand the requirements. Thank you!
Hi, I have extensive knowledge in web scraping with Ruby. I can get this completed in 4 hours. I recently scraped Amazon,Yell,Zillow,Yellow Pages. And I use these techniques Tor,Proxies, User Agent switching and timeouts. Let me know if you are [login to view URL]
$80 USD in 0 day
5.0 (5 reviews)
3.7
3.7
17 freelancers are bidding on average $117 USD for this job
Hi sir,
I am scraping expert, I have did too many similar projects, please check my feedback then you will know.
Can you tell me more details? then I will provide demo data for you.
Thanks,
Kimi
HI, i have checked your mentioned website, I specialize in web scraping jobs like this one. You can see on my profile that I have completed many similar jobs.
Don't make the mistake of hiring an amateur, who doesn't have the skill set to properly complete this task and will only waste your time.
Hire me, a professional with a rock solid reputation and over 8 years of experience specifically in writing scraping code like this.
I know what I can and cannot do, and I won't waste your time by saying I can do something that I can't. because I exclusively write scraping code
I know what to expect and I will always live up to what I tell you, and will also deliver a higher quality final product than my competitors.
Regards
Hi
How are you?
I have read all your points and also seen attachment.
I have good experties in development also scraping.
I have scraped various sites with millions products!
I can deliver you all products in 2-3 days with all information in database or if you want any other format also.
We can also discuss more if you want.
I am available now at freelancer and also on skype: abu_gosia
I hope you will happy from my work!
Thanks
Hello,
Thank you very much for this Scraping Project. I read through the job details extremely carefully and understand your required, for this I am absolutely sure that I can do the project very well. I can complete this Scraping project on time and within your budget. I have worked on similar Scraping projects, and I am confident I can exceed your expectations.
Regards by
Feroz Ahmed
See My Feedback: www.freelancer.com/u/ferozstk.html
Hello Sir ,
I am 100% Sure That i am Perfect For This Job. I have 5 Year Of Exp. In Same Feild And I am Available For 24*7 Support .
ALso We Offer You a 30 Days Free Maintainance. Please Message Me Once before You Award The Project .
I'll Give You best Deal Possible.
Regards
Webexe Solutions
Hi
I can develop a website scraping script.
The script will get all necessary information and put in a MySql database.
I use Java language and Selenium, Jsoup libraries for developing.