Find Jobs
Hire Freelancers

Build automated data load from websites (automation) & APIs to Postgresql/MS-SQL

₹37500-75000 INR

Closed
Posted over 5 years ago

₹37500-75000 INR

Paid on delivery
We are a startup focussing on blockchain products. One of our project details are as below for which we are looking for a team of freelancers with various skill set who can together build the product. Phase 1 (Duration: 7 days): IT solution to collect, load and process data from various websites. Data will be available : 1. Standard APIs of those websites (json format) 2. Websites crawls (by Automation to collect tabular data) Need to build batch automated processes to collect these data and load in Postgresql/MS-SQL on defined interval of time. This would require setting of PostgresqlMS-SQL server on Ubuntu latest version and further processing to consolidate collected data in desired tables' format. Which requires processing via stored procedures. Parameters, like, frequency of batch run and hyperlink of the API/Website to crawl should be read from a database table. Solution should be generalised, such that, similar pattern websites are read similarly. Ex: 1. Standard APIs (samples): [login to view URL] [login to view URL] [login to view URL] 2. Websites Data Crawl (samples): [login to view URL] [login to view URL] [login to view URL] Once data is collected directly from these websites, it needs to be processed and collated in target summary tables based upon few business logics. Phase 2 (duration: 4 weeks) & 3 (duration: 3 weeks): To be discussed once approaching finish of phase 1. Skills needed for phase 1: JSON processing, API processing, Automation to crawl websites and collect data, Postgresql/MS-SQL scripting, database knowledge etc.
Project ID: 17387188

About the project

11 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
11 freelancers are bidding on average ₹69,924 INR for this job
User Avatar
Dear ukrblockchains, I have experience with these skills. I do have similar experience doing something similar to yours "Build automated data load from websites (automation) & APIs to Postgresql/MS-SQL" therefore I am looking forward to discuss and move ahead. Relevant Skills and Experience I have 10+ years of experience in API, JSON, Microsoft SQL Server, Selenium Webdriver, Website Design I can do this within your budget too ₹37500 - ₹75000 INR please check more from here https://www.freelancer.com/u/mmadi.html?page=portfolio I am ready to work with you,kindly waiting for your response. Thanks & Regards, Mmadi
₹52,500 INR in 24 days
5.0 (16 reviews)
6.3
6.3
User Avatar
Looking for Mobile App Solutions, we've got you covered. Contact us today, our developers are eager to solve your problems.. We specialize in Hybrid and Native Mobile Apps. Our prices are affordable. Contact us today.
₹62,500 INR in 20 days
0.0 (0 reviews)
0.0
0.0
User Avatar
I can do your job.
₹66,666 INR in 20 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello! This appears to be similar to work I did to build data marts hence loading data and writing stored procedures is not an issue
₹62,500 INR in 20 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hi, I have work experience related to Web Crawling, Data Extraction. Currently Working as Data Science. Have knowledge of Postgress, Distributed System.
₹62,500 INR in 24 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of INDIA
India
0.0
0
Member since Jul 18, 2018

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.