Find Jobs
Hire Freelancers

Custom Web Crawler/Scraper

$250-750 USD

Completed
Posted about 8 years ago

$250-750 USD

Paid on delivery
I am looking for someone to develop code that is capable of scraping 4 sites of their search data. These sites will have publicly accessible US government projects we want stored in the form of JSON files for us to import into our database. The program must be able to scan and store all projects and return to the site each day to look for changes on all projects so our database can stay updated with the latest data and changes. More advanced details on this request: Requirements: * Program Language: C# Preferred, Python ok too. * Scrape / retrieve process complete dataset from websites / other feeds * Save text data in JSON array. * Save and associate accompanying images / documents with their projects. This is just an example of what we expect for output, it is only to be used as a simple guideline, please organize the schema best suited for the data being collected and we will adapt it to fit with our site. JSON Array of Projects accurately defining all project, content, and document attributes. Sample JSON output: ImportedProjects[] Array Project Object ProjectID: "RF211332" Description: "Build a school" Company: "Los Angeles School District" Type: "RFP" DatePosted: "1/1/2016" DateUpdated: "/1/3/2016" DateDownloaded: Today Contacts[] Array Contacts[0] Name: "Person1 Name1" Phone: "555-555-5555" Fax: "555-555-5555" Email: personsEmail Contacts[1] Name: "Person2 Name2" Phone: "555-555-5555" Fax: "555-555-5555" Email: personsEmail File[] Array File[0] Name: "Planning Blueprints" Filename: "[login to view URL]" File[1] Name: "Planning Documents" Filename: "[login to view URL]" Websites to index include: [login to view URL] [login to view URL]
Project ID: 9806637

About the project

5 proposals
Remote project
Active 8 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
Awarded to:
User Avatar
Hello, I have worked on similar scraping scripts before, sounds very familiar to me. C# suggests that you would like a desktop application, right? Or did you mean ASP.NET? If you asked me I would prefer PHP for the implementation as more simple and easy to change/maintain but it's up to you - I can do C#. I can start doing it as soon as you contact me back and will deliver in a few days max. Many Thanks, Sergei
$526 USD in 5 days
5.0 (10 reviews)
5.9
5.9
5 freelancers are bidding on average $400 USD for this job
User Avatar
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
$451 USD in 6 days
5.0 (186 reviews)
7.1
7.1
User Avatar
Hello, Thank you very much for this Web Scraping Project. I read through the job details extremely carefully and understand your required, for this I am absolutely sure that I can do the project very well. I can complete this Web Scraping project on time and within your budget. I have worked on similar Web Scraping projects, and I am confident I can exceed your expectations. Regards by Feroz Ahmed See My Feedback: www.freelancer.com/u/ferozstk.html
$300 USD in 3 days
4.7 (17 reviews)
4.8
4.8
User Avatar
Hello. I have extensive experience in web scraping. I can create the app the following languages: Java, Python, Groovy, Scala (prefer to work in Java, since I senior Java developer). I can create the application with full tracking changes in records. If necessary, make the integration with your system. Regards , Igor Stetsiura.
$444 USD in 7 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
REDONDO BEACH, United States
5.0
3
Payment method verified
Member since Feb 6, 2013

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.