Find Jobs
Hire Freelancers

Python Scraper Script

$10-30 USD

Closed
Posted almost 6 years ago

$10-30 USD

Paid on delivery
Given csv file: IP,name,Port,age,year,city,state,zip IP,name,Port,age,year,city,state,zip IP,name,Port,age,year,city,state,zip x100,000 I would need a multi-threaded python script that goes through each csv line, grab the IP and port on each line, and scrape the TITLE of each webpage. (Each IP address with the port links to a website). After it grabs the title, It would need to print the results in a new CSV file like this: IP,name,Port,age,year,city,state,zip,TITLE IP,name,Port,age,year,city,state,zip,TITLE IP,name,Port,age,year,city,state,zip,TITLE There are around 100,000 ips total I would need to get through, hence the multi-threaded code. The next issue is that some of the websites load javascript that will redirect to another directory in the website. In this case you would need to use SELENIUM Headless or something alike to load the website and let it do all it’s redirects and than grab the final page TITLE. Please don't rely on 302 for redirects, some of the websites will load a 200 with a javascript code to redirect which a 302 response code wouldn't catch. If you know how to scrape with selenium than you know what i'm talking about. To prevent the code from running for hours we’ll need to setup a timeout, if a website doesn’t respond in say 12 seconds, print that ip and port to another file. Also, for each IP, I’ll need to check both HTTP and HTTPS results. If HTTP doesn’t load a title or timeouts, check HTTPS. Vice Versa. Please only bid if you are capable of completing the project fully.
Project ID: 17105879

About the project

20 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
20 freelancers are bidding on average $54 USD for this job
User Avatar
Hello, I can help with you in your project Python Scraper Script. I have more than 5 years of experience in Javascript, Python, Software Architecture, Web Scraping. We have worked on several similar projects before! We have worked on 300+ Projects. Please check the profile reviews. I can deliver your job with in your deadline. Please ping me for more discussion. I can assure the 100% job satisfaction. Thanks,
$80 USD in 1 day
5.0 (134 reviews)
7.3
7.3
User Avatar
I have expertise in web-scraping using Python. Client's satisfaction is my first priority and believe in long-term relationship with clients. Thank you..
$70 USD in 1 day
5.0 (41 reviews)
6.3
6.3
User Avatar
Hi Sir, I can complete this project within few hours as I am expert in python scrapping via HTTP and Via headless and head full browsers. Please let me know if you are interested in ..
$100 USD in 1 day
4.8 (50 reviews)
6.1
6.1
User Avatar
I can do the project using Python and headless selenium. Can provide instructions how to install selenium too.
$40 USD in 1 day
4.9 (122 reviews)
6.2
6.2
User Avatar
Hello, Really nice project, i am interested. I suggest to use just simple requests because with a high number of threads selenium will crash your pc probably. Will provide a python script as requests. For more please check my profile and let me know. Thanks!
$70 USD in 1 day
4.9 (118 reviews)
6.2
6.2
User Avatar
Hi employer, I am a professional Python programmer with a lot of experience in turning idea into reality. I write Python program that is original, clean and simple. I will give you a program that will give your expected results. Employer, let's get started and I will give you a high quality job at a less cost and super fast delivery.
$25 USD in 1 day
5.0 (15 reviews)
4.8
4.8
User Avatar
Hello I can achieve this project perfectly using php curl library or visual basic selenium library I can automate the scrapping process then upload the item to your specefic website please contact me for more details about the project best regards
$133 USD in 2 days
4.0 (30 reviews)
5.9
5.9
User Avatar
Ready to start the work to Python Scraper Script, We can discuss more over chat, Thanks Regard Arjun S.
$25 USD in 1 day
4.0 (18 reviews)
5.4
5.4
User Avatar
Hey, I can do this for you with chrome-headless. That will ensure the pages are completely loaded before fetching the title from the actual window, not the "source code" of the initial page. In addition I can also save the screenshots of the pages if you need that. Cheers, Andrew
$108 USD in 3 days
5.0 (7 reviews)
3.7
3.7
User Avatar
I understand the scope of the project. I'm quite good in using Selenium and have completed project with multithreading. I use Python as the language and can handle the redirects as well. Can complete the project in 1-2 days, just kept a day for buffer in case of some unexpected issue. Looks like quite interesting and I would like to work on it. Looking forward to hear from you. Thanks
$50 USD in 3 days
4.9 (14 reviews)
3.4
3.4
User Avatar
I have an experience in scrapping for over 4 years. I have used PHP(curl) for static sites and python(Beautifulsoup and selenium) for scrapping ajax loaded sites. As per the given requirements, I am a potential candidate to complete the specified tasks with my knowledge and skills. Regards,
$49 USD in 3 days
3.6 (2 reviews)
3.2
3.2
User Avatar
I have experience doing exactly this type of work. The biggest challenge of your task will be choosing when to use selenium + chrome. Because if it's used in all queries you will not get the level of parallelism you're looking for because of Chrome's memory and cpu consumption. But I'm sure it's possible to reconcile both. Also I have experience running selenium + chrome inside the docker, so I can deliver you a docker image that will work on any linux distro.
$45 USD in 2 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi there, I do a lot of scraping project for finance data before using python. Would love to help. please drop me a message with details! Thanks, Vincent
$15 USD in 3 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of CANADA
Quebec, Canada
5.0
2
Member since Jun 4, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.