Find Jobs
Hire Freelancers

Scrapping in Python

$30-250 USD

Cancelled
Posted almost 14 years ago

$30-250 USD

Paid on delivery
I have a Python/Django app, and I need to add the following functionality added to it (in Python): - I want a script that will search for a given term (that I provide), and for each of the first 100 results in Google, crawl the website and look for an email address. If one is found, record it in my db. - I have link exchange partners see ([login to view URL]). I need to have a Python script that crawls my partner pages and verifies that they have added a link to my site (and record the page the link is on). - I have a db of a bunch of local businesses (35,000) and I want to add the following functionality: - I am missing the web site for about half the businesses, I need to have a script search Google for the name of the accountant firm and find the URL for their site (if it exists). This should use some simple heuristics for each of the websites on the first page of the Google results, like is the business name in the H1 or title tag of the home page. - I am missing email addresses for most of the businesses. I need a script to crawl the businesses web site and find an email address. It is important that you speak very very good English, as communication is important. If you have any questions, feel free to ask. Keep the following things in mind: - These scripts will be run periodical (maybe daily) as part of a Django app on a Linux/Apache/MySQL machine. - I expect the crawler you use to be multithreaded (or else these scripts will take way too long to run) and be polite to the host domain (no flooding them with requests, respect [login to view URL], etc). - I expect high quality code with tests to verify. I'm a developer myself, and I will be reviewing all the code. Regards, PB.
Project ID: 689376

About the project

Remote project
Active 14 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs

About the client

Flag of IRELAND
Belfast, Ireland
4.9
121
Payment method verified
Member since Jul 29, 2007

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.