Web Scrape/Crawl from a URL list

Web Scrape or Web Crawling Project:

Because of the volume of responses I delete all offers that do not read the entire task to eliminate those not detail oriented. I am doing market research and I have a list of some ~50,000 URL’s that I need to gather data from. I am seeking someone to do automated web crawling/web scraping across these 50k sites, capture the data from them and store it for analysis. There are 15 potential fields that need to be completed for each URL crawled:

Skills: Web Crawling, Web Scraping, Data Processing

See more: know main web site base url magento, php web scrape script, web forms php province list, add free url list, xml driven url list flash, php crawl url, web scrape asp login, web scrape net, check url list script, web scrape perl javascript post, scrape email addy url list, url list web, web scrape url site java, web channels url list, scrape url list, web adult url list, web scrape list of urls, web scraper url list

About the Employer:
( 0 reviews ) Boulder, United States

Project ID: #25651030

33 freelancers are bidding on average $475 for this job


Hello Sir, I can get you the data in automated way from those 50k urls, if you can please share details I can check more. My name is Jay and I am Excel/Python and Web Scrapping Expert on and have been More

$300 USD in 5 days
(518 Reviews)

Hi sir, Does all 50000 urls from same website? can you give me some example to check? also can you tell me what's details fields you need? Thanks, Lin

$180 USD in 3 days
(334 Reviews)

Hello! Are the links from the same domain? Can you shoe me the list of links? I am available and I can crawl the links. Thank you!

$500 USD in 4 days
(271 Reviews)

Web scraping expert I use python language. My scripts works on windows, mac or linux, but linux is preferably. I can schedule scripts on server if it is required. I have more 100 finished projects (google scraping, fac More

$140 USD in 7 days
(149 Reviews)

Hi! I'm script developer that can retrieve data & do tasks with AUTOMATION.100% ACCURATE! Can you tell me some of the website URL? ## I will write in Python, and can be converted to .exe for you ## Let's discuss more d More

$120 USD in 2 days
(110 Reviews)

Hi I will create a Java application to scrape data from URL's. I will save the data in MySQL database (and also in a spreadsheet if needed). Since the scraped data will be saved in a MySQL database, we can easily ana More

$200 USD in 5 days
(12 Reviews)

Hi, I'm interested to scrap data from the given links. can you share the link? I'm ready to make a sample. If you think the sample is okay for you then we will move on. Waiting for your reply. Thanks!

$300 USD in 4 days
(52 Reviews)

NOTE : I CAN DELIVER THIS IN QUICK TIME. IMPLEMENTED SIMILAR WEB SCRAPING TASK IN THE PAST I have more than 12 years of IT experience. I have successfully completed few projects which involved Programming, scripting & More

$250 USD in 5 days
(39 Reviews)

Hi there, I Have Scraped Amazon, Ali express, Yellow Pages, Yelp, Yellow Pages, Grainger, Boohoo, Etc. I'm interested to hear more about the project. If you award me the project I'd be very happy to discuss this furt More

$200 USD in 5 days
(117 Reviews)

I can scrape 50k sites in 1 or 2 days. 1. My scraping works and experience I have 9 years of experience in web scraping and scraped so many many websites(around 2,000). I've scraped websites like [login to view URL], glas More

$250 USD in 1 day
(41 Reviews)

Hi, are those 50k url's from same websites or different websites? Also what are the fields do you need to extract? please share your requirement in detail with sample urls to start working on sample data. Thanks and More

$200 USD in 7 days
(8 Reviews)

Hi sir, I am a Web & Data Scraping Expert who have career for 6 years over. I am very happy to bid on your job. I have already worked on several similar projects for collect data & contact & business information such a More

$250 USD in 1 day
(4 Reviews)

Greetings, Please provide more details about the URLs and fields of data which needs to be fetched. We have experience in scraping and have scraped many websites. We will work on Python and Scrapy for this task. Look More

$140 USD in 7 days
(9 Reviews)

Hello, I have experience in web scraping with Python. I can use Selenium, Scrapy, BeautifulSoup and Requests to make the best web scrapers! I have created scraping programs to extract data from websites like: Amazon, More

$140 USD in 7 days
(4 Reviews)

If you want someone to do this work MANUALLY, then please ping me. If you want someone to do this work MANUALLY, then please ping me.

$250 USD in 7 days
(4 Reviews)

I can help you in Web Scrape/Crawl from a URL list. I have read and understood all your initial requirements, and I feel,I am producing quality data entry for my clients including; Web Research, Data Mining, Internet R More

$140 USD in 7 days
(8 Reviews)

Hi there. I have read the project description and I understand that you need an automated script to gather the required data from a big list of links. I am Umer, a Python Web Automation and Data Scraping expert. I wi More

$200 USD in 7 days
(2 Reviews)

This will be my first job in the platform, I have extensive experience with beautiful soup and I am eager to apply my python skills to a real project, that is why I am charging so little. Please contact me if you have More

$35 USD in 7 days
(1 Review)

Hi, Manager! How are you today? I have read your requirements carefully. I am scraping expert by using python. And I'm an experienced programmer with over 7 years of experience. I can do that for you right away. I will More

$10000 USD in 30 days
(1 Review)

I have experience programming several web scraper scripts using Python. You can view more info on the most recent projects in my portfolio section. One of the web scrapers I wrote was similar to this, but it was an int More

$140 USD in 7 days
(0 Reviews)