Find Jobs
Hire Freelancers

Web scrape of 5-7 online retailers using Ruby Mechanize/Watir/Nokogiri

$25-50 AUD / hour

Closed
Posted over 6 years ago

$25-50 AUD / hour

I run a data analytics consultancy in Sydney, Australia. I have a requirement to setup weekly scrapes of at least 5 large (50,000+ SKUs) online retail websites. The project will be split to scrape a single website to start with, extending to the others assuming satisfaction with the work completed. Technical requirements: - Scrapes must be written in Ruby using Mechanize/Nokogiri or Watir as appropriate - Where possible, avoiding JS and scrape via direct HTTP requests using Mechanize - i.e requires strong skills in tracing/debugging HTTP requests/responses - Alternatively, load page using Watir webdriver then pass off cookies to Mechanize - Last resort would be brute force Watir webdriver approach (scroll, point, click, etc) - Implement utilities for use of proxy / IP rotation where required (access to pool of anonymous IP addresses will be provided) - Scheduling via cron job to run at least weekly - Must use object oriented approach and design in a DRY manner so that code can be leveraged for future scrapes of additional sites and ensure future code manageability - Must use Activerecord to persist records scraped - Must implement logging (production logging to file + verbose debug option to STDOUT) Personal requirements: - Must be a great communicator both in terms of coding concepts and project management - Good at clearly laying out the workplan, milestones and progress - Must be available for 2x daily stand-up calls with Sydney AUS (timing TBC at mutually agreed time) - Must be able to demonstrate prior experience with ruby web scrape projects similar to above - Code must be well-structured and well-documented / commented Opportunity for ongoing Ruby backend and RoR web app build if this project is a success.
Project ID: 15682987

About the project

13 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
13 freelancers are bidding on average $36 AUD/hour for this job
User Avatar
Hi, aaronjcowper! I’m experienced Python developer, and web-scraping is one of my main fields of knowledge, so I’m 100% confident that I can complete your project using Python as programming language. I completed hundreds of web-scraping projects and there were no situations when I could not retrieve some data.
$45 AUD in 40 days
5.0 (56 reviews)
7.6
7.6
User Avatar
Hi, I am Full stack Ruby on Rails developer. I can provide you best design, highly secure & best performance website regarding your needs. From development to production respecting testing & continuos-zero downtime deployment. I can assure you quality of my work. My Skills: - Ruby - Ruby on Rails - System Admin - Git, Bitbucket - MySQL , PostgreSQL - HTML5 , CSS3, SCSS, HAML, slim - javascript, coffeescript, jQuery - Ajax - Hosting & Maintaining any platform - RSpec (Test Driven Development) - Spree Ecommerce - Web Scraping - Shopify Thank you for your time and consideration. I look forward to speaking with you about this employment opportunity. Sincerely, Mezbah Alam
$27 AUD in 40 days
5.0 (32 reviews)
5.8
5.8
User Avatar
Hi, I'm Prem. I have 5+ years experience with scraping and automating the web with Ruby. I have used Watir, Nokogiri and Mechanize in the past. I have experience scraping fb, twitter, G+, aliexpress, amazon and many more. I can handle proxies and have solved Recaptcha's by using Deathbycaptcha service. Since I'm also a Rails dev I don't have to problem using AR to persist the data. Initiate chat to discuss. Thanks
$44 AUD in 40 days
5.0 (14 reviews)
4.5
4.5
User Avatar
Hi! After reading your job description, I noticed I'm fit freelancer for your project. I agree all of your requirements from you job description. Please send me a message so that we can discuss more. Andrey
$55 AUD in 40 days
5.0 (1 review)
4.3
4.3
User Avatar
We are team of Developers having rich experience with Ruby on Rails & Ruby. We have done extensive research on Heroku/AWS. The Technical team consists of Programmers having experience more than 6 years.
$45 AUD in 58 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of AUSTRALIA
Maroubra, Australia
5.0
3
Payment method verified
Member since Jun 2, 2017

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.