Write a scraper in Python/BeautifulSoup

$60 USD

Cancelled

Posted

over 4 years ago

$60 USD

Paid on delivery

The deliverables are a Python/BeautifulSoup scraper, supplied as a text file, that pulls names from a list, searches for each one of them on a website, records the number of hits for each name, and outputs the results to a CSV file. The scraper should be heavily commented and the developer should be available for an hour at the end of the project to answer any questions about how the scraper works. 1. For a list of 10-15 names drawn from a list of 785 that I will supply, go to "[login to view URL]" + name 2. Grab the number of results 3. Output a CSV file with two columns: name and number_of_results 4. I would like to pause by some random number of seconds between each hit on the [login to view URL] server (say, between 15 and 120 seconds). I don't care if it has to run all night - I just don't want to be blocked. I used randint(15,120). If you know of any other ways to avoid detection and keep hitting the server, I'm all ears. At the end of the project, I will will run the scraper myself on the entire list of 785 names. Once I have the resulting file, the project will be complete.

Python

Web Scraping

Project ID: 22801042

About the project

27 proposals

Remote project

Active 4 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

27 freelancers are bidding on average $60 USD for this job

@mantislin

Hi there, This is Lin and I am scraping expert, i have checked all details for your project. can we discuss more info then i can provide example data for you? Please message me then we can discuss more ASAP. Thanks, Lin

$60 USD in 2 days

5.0

(369 reviews)

8.0

@gangabass

I'm one of the best Python and web scraping experts here (check my reviews) that's why I'm sure that you'll be impressed with my work. I can provide you scraping script in Python that will use random proxy servers from a list (I'll provide you 200 free proxy servers) and a delay between requests just like you want. It will perform a search for each name from your list and output into CSV file. Thanks. Roman

$60 USD in 1 day

4.9

(500 reviews)

7.9

@kunitsynartem

Hello. I will make you this task within a couple of hours. And I will answer your questions regarding how the script works. I have 2 years of experience in web scraping using Python.

$60 USD in 1 day

5.0

(40 reviews)

6.1

@fejs

Hi Sir/Madam, I'm expert in Python programming and I've specialized in web scraping, so I can help You with this task. EDIT: I've made a demo showing search results for 5 random names, coming from list. Here's the output: Found 132691 results for Michael Jordan Going to sleep for 3 seconds... Found 158128 results for Al Pacino Going to sleep for 5 seconds... Found 3840 results for Barack Obama Going to sleep for 5 seconds... Found 10611 results for Donald Trump Going to sleep for 4 seconds... Found 0 results for Nbatah Mbawal Going to sleep for 3 seconds... If You're satisfied with this, You can assign me the project, and I'll add reading from file and storing to CSV file. Best regards, Fejs.

$60 USD in 1 day

4.9

(139 reviews)

6.3

@osmanaydin

I can help you to get number of results. Hello sir, Please take a look at my reviews and portfolio. I have completed web/data scraping jobs many times. I am interested in your project as well. I would like to discuss the details via pm. I look forward to hearing from you soon. Best Regards,

$60 USD in 1 day

5.0

(56 reviews)

6.1

@adey665

“Python script” I have worked on similar projects to what you are looking for, and I am confident I can exceed your expectations. I can start your work right away. Please message me. Thanks

$60 USD in 1 day

4.9

(103 reviews)

6.1

@brashel

Dear sir, I read your full description and actually I am a perfect person for this task. Because I've done multiple tasks in the same genre before. If you have time you can take a look at my profile from the link below: https://www.freelancer.com/u/brashel So please give me a chance and give a massage right now. Before starting work I would like to give you a sample as proof of my experience and to better understand your needs. Thanks a lot.

$60 USD in 1 day

5.0

(57 reviews)

5.6

@pandread1

Hi there, I have completed many very similar Scraping tasks. Can deliver within a day or two. Contact me in chat thanks, Pandelis

$60 USD in 2 days

4.9

(17 reviews)

5.3

@unix515

Dear, sir. I am so happy to see your scraping project. I am so familiar with Scrapping and Entry. I can send good result you want. Hoping reply from you. Best Regards.

$60 USD in 2 days

4.4

(15 reviews)

5.0

@fb54edfefc935c9

Hi, I can make that web scrapper for You. I need more information from You about this project, give me an example of output file that You want, for example as I understood You want to have for "Oscar-De-La-Hoya" the row in csv: Oscar-De-La-Hoya, 80019 Or You want more like the title of every article and link for it?

$60 USD in 7 days

5.0

(8 reviews)

4.6

@Fantastic211

Hello, dear Customer. I am interested in your project and feel confident after reading your project description. Please contact me so that we can discuss it in more detail. Looking forward to hearing from you soon. Thank you.

$60 USD in 1 day

5.0

(13 reviews)

4.3

@dhirajbaheti

Hi, My name is Dhiraj and I am a professional web scrapper and data mining expert in python with more than 6 years of experience. I have expertise in scraping complex/dynamic websites using requests, BeautifulSoup, Scrapy, proxy rotation, user agent rotation. I will be able to build python scrapper for your requirement using requests and beautiful Soup which will extract data as per your specifications from any website. Please contact me for further discussion. Major highlights of my work are building scrappers for Amazon, flipkart, justdial, wikepedia, linkedin, Airbnb, Yellopages.

$60 USD in 2 days

5.0

(7 reviews)

3.7

@SamindaPeramuna

Hi I could get this done under your budget. Please drop me a message to discuss the details if you're interested. Cheers, Saminda

$60 USD in 3 days

4.8

(6 reviews)

3.2

@ChrisKostas

Hi , although there are like 30 other bids at the time and the chance that you see my bid is slim , I want to place an offer. Reason is this is the type of work that I can do very well ,and unlike working on such projects . Please consider my offer .

$60 USD in 1 day

5.0

(1 review)

1.0

@topdeveloper000

Hello I am very interested in your post project since i have rich experience on python beautiful soup scrapper. Looking forward to hearing from you. Best regards

$60 USD in 7 days

0.0

(0 reviews)

0.0

@SergeyEybog

Hello, I'm Sergey from Russia. I'm interested in your job description. I wrote several automation and scraping scripts with python, nodejs and php. As I have gained rich experience with web automation, I'm pretty sure I can provide you high quality work within short amount of time. I look forward to your reply. Regards.

$60 USD in 7 days

5.0

(1 review)

0.0

@UlisesAL

Hi, I'm a data scientist at Prexa. We are a team of physicist who turn to data science. We have worked in many data scraping projects. We are available to work on your project right away. We have already checked the website and it can certainly be done. We well be available to explain the code as long as you need.

$60 USD in 2 days

0.0

(0 reviews)

0.0

@sthariddhinp

Hello, I am working as python developer here. I have done a lot of python work. I used python to do data analysis and data processing for machine learning project. i have read the project requirement details and understand that you want to do data scrapping and my previous experience make me that I can complete it. Thank you.

$66 USD in 7 days