Import Rest API Data into SQL server using Azure Data Factory (ADF)

$10-50 AUD

Closed

Posted

6 months ago

$10-50 AUD

Paid on delivery

Hi There, I have REST API and I need to import the data into a Azure SQL Server table. I am using Azure Data Factory Data flow (attached) which was working fine but as the data grew I started getting the below error Azure data factory REST API Job aborted due to stage failure: Serialized task 16:0 was bytes, which exceeds max allowed: [login to view URL] (134217728 bytes). So I had to split the source API data into two parts using $top and $skip Like below, two URL calls. abc?$expand=xx,yy,zz&$top=40000&$skip=0 abc?$expand=xx,yy,zz&$top=40000&$skip=40000 I am looking for efficient way of doing it by using for loop or parametrization or anything which makes it work even if the data further grows. Example API data : [login to view URL]$expand=Trips If you can use above api and import all the data fields into SQL server.

Project ID: 37514388

About the project

12 proposals

Remote project

Active 4 mos ago

Looking to make some money?

Email address

Benefits of bidding on Freelancer

Set your budget and timeframe

Get paid for your work

Outline your proposal

It's free to sign up and bid on jobs

12 freelancers are bidding on average $43 AUD for this job

@vw7590795vw

I read your requirement for Azure Data Factory. I will check your REST API data and help you to resolve your problem. I have good experience in import data at SQL Server. You can view my profile for completed projects. Let me know when we will connect and start on your project. S k p >> j i k e n j d

$29 AUD in 1 day

5.0

(15 reviews)

3.9

@akhileshgandhi

Hello, We would like to assist you with your requirements for importing Rest API data into a SQL server using Azure Data Factory (ADF). We have expertise in ADF, Oracle ADF, Azure, and Microsoft SQL Server. I understand that you need to import the data into an Azure SQL Server table, and I believe that our skillset would be the best fit for this project. Our experience has shown us that we can efficiently use loop or parametrization methods to import large amounts of data into a SQL server, even if it grows further. We would be delighted to help you with your project; please let us know if you have any questions or need more information about our services! Regards, Akhilesh MoreYeahs Inc.

$30 AUD in 7 days

4.8

(7 reviews)

3.2

@Samaji206

Hello I understand your request that you’re demanding Import Rest API Data into SQL server using Azure Data Factory (ADF) As a Professional Geeks, I can avail creative, proactive and high-tech solutions to your development needs. I have excellent experience in Metatrader (4 And 5),Ninjatrader (7 And 8),Indicators & Trading Robots Development,Tradingview (Pinescript & Strategy),Python Trading Api,Machine Learning,,Python, Odoo, HTML, Javascript, PHP,CSS, Laravel,Java,Joomla,Angular.js,MySQL, Node.js, React.js,Wordpress,linux Ruby on Rails, Database, Objective-C and Swift,C, C#, C++, Qt, Linux, Arduino and raspberry. My Engineering skill are unimaginable with lot of success in engineering task My page is clear about my professionalism with awarded task Don't hesitate to contact me to create awesome products! Thanks

$20 AUD in 1 day

0.0

(0 reviews)

0.0

@durgesh8527

Hello there! My name is Pllavi and I am a full-time web and mobile app developer working as a freelancer. I understand that you need to import data from REST API into Azure SQL server using Azure Data Factory (ADF). Specifically, you need to import the data into a Azure SQL table. I have worked with many big companies, creating various High tech mobile and web apps for them. I always do my best to understand the project at hand. I am available in the USA time zone so we can discuss this project further if necessary. I believe I would be the best fit for this project due to my extensive experience in web development, specifically in ReactJs, Nextjs, Strapi, NodeJs, Vue js, AngularJs and others. This knowledge would be invaluable when it comes to producing an efficient solution for importing API data into SQL server.

$30 AUD in 7 days

0.0

(0 reviews)

0.0

@anayapallavi

I can Import Rest API Data into SQL server using Azure Data Factory. Please initiate a message for further discussion.

$50 AUD in 2 days

0.0

(0 reviews)

0.0

@anaya8777

I can help you in Importing Rest API Data into SQL server using Azure Data Factory (ADF). Lets connect and discuss.

$50 AUD in 2 days

0.0

(0 reviews)

0.0

@avetech2013

Hello Employer, How are you? Hope you are in your best spirits. I have 5 years of project management and virtual assistance experience in various industries. In my 10 years of IT programming experience working on technology stack - JAVA, wordpress, figma, etc. I understand that you have REST API and it needs to import the data into a Azure SQL Server table. I can develop that for you as per the above-mentioned requirements. I have updated skills to AI apps like chatGPT -its plugins, Bard - NLP - PowerBuilder - model creation - openAI API - Generative AI - Azure AI I look forward to discussing more about the project. Regards, Ankita

$30 AUD in 7 days

0.0

(0 reviews)

0.0

@fl0colin

For this task, we can use pagination rules to achieve your needs. I have tested with the example API you shared. Dataflow does not support range, but we can use one copy activity follow by your dataflow to flatten the data. If your actual API response body or header contains necessary information, we can also just using one dataflow. We can connect to discuss more. I used to work in the Microsoft ADF team. Master product usage and pipeline performance tuning. Familiarity with Azure platform (network, RBAC) and services (Blob, Sql, Databricks, etc). Master in Python and REST API. Can combine with Azure function or notebooks to implement data processing functions that are not supported by ADF.

$40 AUD in 5 days