Need python script that scrapes articles from industry publications

$250-750 USD

Closed

Posted

over 4 years ago

$250-750 USD

Paid on delivery

I'd like to scrape MotorTrend articles on vehicles over the last five years. Additional requirements are here: [login to view URL] Expected deliverables: 1) A tested and proven python 3 code base, preferably using the Scrapy library, which crawls the website and pulls car industry publication reviews ([login to view URL]). Need car reviews for last 5 years worth of cars. 2) Codebase includes testing, full commenting, and descriptive error messages 3) Scraping is “polite,” meaning it scrapes as a human would browse. 4) Proxies are optional and are able to be used if necessary 5) Project scrapes the key facts, like horsepower, torque, curb weight, and dimensions listed at the bottom of reviews. 6) The output of this scraping would be to a local MySQL database. Fields to populate database are make of car, model of car, year of car, all key facts under respective columns, hyperlink to review, title, and car review wording. Only need the heading, and p tags for the car review. Please include tags as one body of text and exclude actual tags. Dimensions for scraping are in the original document.

Python

Web Scraping

MySQL

Software Architecture

PHP

Project ID: 22129849

About the project

22 proposals

Remote project

Active 4 yrs ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

22 freelancers are bidding on average $467 USD for this job

@seaanddream

Hi, my name is Selim. I am from Solihull, UK. I read your `Need python script that scrapes articles from industry publications` project descriptions carefully before bidding. I checked the attached google doc, and your requirements as well... I got what you need and ready to go ahead as soon as we can clarify further project details. I am 5-star web scraping expert here at freelancer.com. Pls check my profile and feedbacks first to have some idea about the quality of my previous businesses. I had many similar web scraping tasks, including extraction of large data from various websites using python and Scrapy library, all achieved with 5/5 rating. Kindly get back to me if you wish to proceed with a top-rated and reliable partner here...pls note that I only deliver best quality at a reasonable price, and I am open for longterm partnerships, if you wish too... I bid my best for you to start business, and I am ready to prepare a sample task for you to check & validate, once you give me a reply... Kind Regards, Selim

$600 USD in 7 days

5.0

(304 reviews)

8.2

@zekovicm

Hi there,I am Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it within the agreed deadline. Check out my profile and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Best regards, Miljan

$500 USD in 7 days

4.9

(169 reviews)

7.9

@YuriyStepanovich

Hi. I’m senior full-stack developer and expert in Python. I can do your project successfully. My talent and trustworthy will make your project meet success. I hope to contact soon. Thank you. Alex T

$500 USD in 7 days

5.0

(12 reviews)

7.2

@SuiGenSolutions

Hi I have more than 7 years of experience in web scraping domain. I have written more than 300+ crawlers in different domains. I have gone through your requirements and clear regarding instructions except couple of questions. Looking forward to hear from you. Thanks Shiv Kumar

$500 USD in 10 days

4.9

(127 reviews)

7.2

@zeke

I have done many similar projects related to web scraping information from different websites. Very interested to work on this project. I am absolutely confident I can finish this work on time and on budget to highest quality expectations. Looking forward to work on your project. Thank you.

$500 USD in 10 days

4.6

(212 reviews)

7.6

@naravila

Dear sir! ⭐I am very interested in your project and I am exciting. ⭐I read your project details carefully and I though that I am the best fit developer for your project. ⭐I have rich experience about your project, so I have the clear way to complete the project. ⭐Your project is very fit for my skill sets. ⭐If you are willing to award the project to me, I will always be ready to complete it perfectly.

$600 USD in 10 days

5.0

(52 reviews)

6.4

@marcnealer

Hi I'm a python coder with 12+ years experience. Yes, I can scrape with Scrapy, then clean the data before writing to your Mysql DB. I can also make the system multi processing so it pulls several pages at once , thus speeding up processing Thanks Marc Nealer

$750 USD in 7 days

4.9

(13 reviews)

6.0

@FreelancerTechX

Dear Sir/Madam, I specialize in web scraping. Ready to start ***************************************************************

$250 USD in 7 days

5.0

(33 reviews)

5.6

@razajen

I I'm a professional data scientist from Scotland I have got more than enough experience with AI and ml models I am more than happy to discuss your project please drop me a text here and I should reply back.

$833 USD in 1 day

5.0

(7 reviews)

4.1

@safdar968

Hy, I can effectively do web scrapping of car data with python using a variety of tools like scrapy, beautiful soup, selenium, and many others. I am an experienced Python programmer with 4 years of experience. Currently, I am working as a Machine Learning Engineer in a top-notch AI company in my country. I can work with a variety of Python libraries related to _Maths _Stats _Data Cleaning _Image/Video handling _Browser Automation _Human like webBot development _JSON Parsing _Data Sciences My recent web scrapping projects are listed below, which you can also view in my portfolio section. - User Bio Data extraction from a xyz website - Automated form submission with reCaptcha v2 using selenium I will be glad to assist you in your current and all future tasks If you are facing any kind of problem related to Python programming project or troubleshooting or debugging of code as I love to have long future terms with my clients. Feel confident in hiring me and I will deliver you more than just needed with perfection. Waiting for your response. Thanks for your valuable time. Regards

$500 USD in 7 days

4.9

(13 reviews)

4.3

@Nacho1993

Hi there, first of all thanks for commenting so well the requirements since most of the time all the information about site and data is missing. I've been analyzing motortrend and I already found access to all the first-tests of the last 5 years and I'm ready to parse the HTML code and extract all the fields needed, as you require I will provide the python3 code for a scraper built using requests and beautifulsoup. I can leave enought comments to make all the code easy to read and modify in a future. I can define pauses to create a friendly scraper that doesnt collapse the site. I have some code to get free proxies and will add this. I will finaly build a connection to a MySQL database and I can even create it if you need. You can check my profile to see other bots I've made and comments from my clients. For any inquiries contact me via freelancer. Best regards, Ignacio.

$250 USD in 6 days

5.0

(7 reviews)

4.2

@ChaoTian0109

Hello, How are you? I'm very interested in your job post.I believe my skills would be ideal for your project. I am Python Expert and I can help you perfectly. And then I am a good english speaker. So anytime,anywhere you will discuss anything with me freely. My wonderful service will give you pleasure and happiness. I hope you will contact with me and work together. Best Regards.

$500 USD in 7 days

5.0

(8 reviews)

3.8

@rty567

Hi Adrian. We have discussed this project before ======================================================

$250 USD in 5 days

5.0

(4 reviews)

3.5

@Esthermae21

Dear client, nice to meet you. It's Esther - the senior scraping developer from Seattle, WA, United States. As you see, I am new on this platform so I have only one review but my knowledge and experience are enough to complete your project, you will know soon about this. If you want to hire a high-skilled US developer to get a high-quality result, then don't hesitate to contact me. I am always open to you. If you want to see my past work, I will be happy to show them. Good luck.

$500 USD in 7 days

3.5

(4 reviews)

4.1

@wangzy840101

Dear Client. I respect you and will follow all your requirements. I am so happy to see your serious project. Because your project is the perfect match for my skills and experience. I have strong skill and rich experience about web scraping and automation processing, data mining , bypass security verification using python and c#, nodejs, java. Also I have many experience for market place sites such amazon, aliexpress, eaby on etc. So I can write high performance, robust and clean code for your project. I am sure I can deliver a perfect result for you. Please contact me and discuss more. Looking forward to hearing from you. Thanks for taking precious time to look for my cover letter. Best Regards.

$300 USD in 2 days

5.0

(3 reviews)

2.6

@Artemrodin1987

Hello, I have just read your requirement carefully and I am very interesting with your project. Webscraping is my top skill. No problem for everything!!! I can complete your project using spider or selenium etc in python. Please contact me so that we can discuss more in chat. Thanks, best regards.

$400 USD in 2 days

5.0

(4 reviews)

2.7

@kshitijkohli

Hi, I am a professional Python developer working as a Senior Software Engineer at Intuit. I have prior experience in Scrappy and would like to undertake this project. The work will be delivered in 10 days time and meet all the pre-requisites mentioned in the document. Let me know if anh further inputs are required. Thanks

$250 USD in 10 days