Find Jobs
Hire Freelancers

Python scraping and API expert

$250-750 USD

Closed
Posted almost 6 years ago

$250-750 USD

Paid on delivery
I'd like to deploy a server where an app can send a request to one or more web scrapers and actually choose which scraper returns data based on parameters sent in a JSON string. This is a learning project for me, so I'm looking for someone who is an expert and can help me answer the following design questions: 1) What are the components of this setup that need to be on place on the server? 2) What is the best way to structure communication between the app and the scrapers? 3) What is the best way to store web scraping results? 4) What are the best web technologies to use for the scraper server to ensure continued modularity as the scale of data requested grows? Attached is a simple diagram of how I'd like the system to work. In terms of work style, I prefer to use GitHub for collaboration. Please send a github or bitbucket link of a REST API or web scraper/crawler you have built. Please not the budget is negotiable. Regards.
Project ID: 17144824

About the project

23 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
23 freelancers are bidding on average $558 USD for this job
User Avatar
Hi there,I am Miljan,Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this job ! I can start immediately and finish it within the agreed deadline. Check out my profile, portfolio, and former clients feedback - that'll let you know everything about me. Please feel free to contact me so that we can discuss further details. Thank you for taking the time to read my proposal.I am looking forward to hearing from you. Kind regards
$333 USD in 2 days
4.9 (173 reviews)
7.9
7.9
User Avatar
Hey! I saw your project, and I think that flask-based solution will work best. 1) On a server, you will have running flask app that will accept calls from outside 2) Communication will be done with http requests 3) I propose store results in some web server based DB ( for example postgresDB) Pease contact me to discuss more details. Regards, Vlad
$677 USD in 10 days
4.9 (62 reviews)
8.0
8.0
User Avatar
Hi. I am a prof. scraper and did a lot of technologies related to that. Few questions I have: 1. So, you will have different scrappers that will perform different type of operations? 2. When you call the scrapper via API you want to start specific scraper which will run on the time you call it? This part I don't get, please elaborate. 3. We discuss more about the data you want to extract and based on that we can decide which type of the DB to use. But yes MongoDB is a good solution. Let's get in touch and discuss more, I still don't understand all the components from this big picture. Thx.
$700 USD in 10 days
5.0 (138 reviews)
7.7
7.7
User Avatar
Hi,sir I have read your post and I am very interested in your project. Because I am a expert in web scraping. I have expert and smart technic with web scraping (1000 over projects for scraping), automation and bot(500 over projects for automation and bot), selenium, mobile (android, ios native and others). and then I am a expert in project management using Git. I am free recently days and U could work for u on full times. pls contact me and let's discuss the project in detail. Thanks
$555 USD in 10 days
4.9 (91 reviews)
6.9
6.9
User Avatar
Hello, I can help with you in your project Python scraping and API expert. I have more than 5 years of experience in Git, Linux, Python, RESTful, Web Scraping. We have worked on several similar projects before! We have worked on 300+ Projects. Please check the profile reviews. I can deliver your job with in your deadline. Please ping me for more discussion. I can assure the 100% job satisfaction. Thanks,
$500 USD in 10 days
4.9 (90 reviews)
7.1
7.1
User Avatar
1) I can build scraping tool with node js or python. So we need to place pip python or node and it's required dependency. 2)Api is best way to communicate with scrapper 3) Data can be store to Mongo db or SQL mongo be is best for raw or unstructured data. 4)We can run scrapper with multithread and along with nginx for load balance if data request grow. These are my skills set related to Web Scraping and Web crawling Have done scraping in Nodejs, CasperJS Phantomjs, python Scrapy Framework. Have done testing and automation with selenium also. Know to deal with database like mongo, mysql, Elasticsearch. Also know to handle proxy and captcha while scrapping
$666 USD in 10 days
4.9 (61 reviews)
6.3
6.3
User Avatar
Hello, Greetings! With a proven track record of successful achievements, I am pleased to present my application for your consideration as a Freelancer. Please have a look at my profile and portfolio to get an idea of our capabilities and some previous work on freelancer. This bid is approx. I’ll give you exact budget and price once discussing in details with you. Many Thanks & Regards, Rishi A.
$555 USD in 10 days
5.0 (55 reviews)
5.5
5.5
User Avatar
Hello. I have good skills in Git, Linux, Python, RESTful, Web Scraping, etc. Contact me please. Thanks.........................................................................
$555 USD in 10 days
4.9 (36 reviews)
5.3
5.3
User Avatar
I am a python developer. I have great experience in web scraping and I am an expert in it. I have all necessary skills to scrape almost any website. I have even scraped sites like google, whatsapp web, instagram which you can check in my portfolio and reviews. Ping me to discuss in detail. Budget and time will vary as per your reqirement. Waiting for you response.
$375 USD in 2 days
4.8 (41 reviews)
5.6
5.6
User Avatar
Hey there, Hope you are doing well. We have gone through all the details of your project and after having necessary information to check our caliber, we have come to the result that we can make your project done successfully. You will tell us about your project's Deadline and if we will need any information about your project then we will send you texts right away, but before it, please make sure to send a message to us to discuss your project more. See you..!!! If you want to check our experience related to relevant project then you may visit the link of project which we already did..!!! https://www.freelancer.com/projects/Python/need-help-for-python-computer/ https://www.freelancer.com/projects/java/Extract-Data-from-website/
$400 USD in 5 days
4.9 (14 reviews)
4.5
4.5
User Avatar
hi, I am a professional web scraper. I have scraped over 200 sites in my 3.5 year experience. please ping me to discuss,. I know much about apis, I have good experience in Python. I will help you within your budget.
$466 USD in 5 days
4.9 (15 reviews)
4.3
4.3
User Avatar
Greetings, I develop Python since 2010 so I have a lot of actual epereiince, I would help you to get learned around the subject Sorry but I would share sources once we would make a deal only, no problem would appear here to use Github Please pm once you are interested over the proposal Thank you, Best Regards,
$2,000 USD in 6 days
4.2 (7 reviews)
3.6
3.6
User Avatar
I have 6 year experience Freelancer,up work,Fiverr & 99design market place I have seen your project that i can to do easily because I have many experience to Graphic Design,Webdesign,Web Develop & programming .So I could create it for you as soon as that easy but how this have done ,,,,please discuses before start job,,,,
$388 USD in 10 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
United States
0.0
0
Member since Jun 11, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.