Big Data Conversion in R

Task - 1 Billion JSON to CSV file conversion

Structure of file - Deeply nested file (attached data structure)

Package in R --- Sparklyr package reads 1 Billion data file within 30 minutes only, whereas other packages take non-ending time. So I preferred Sparklyr package in R. (If required I can provide you that R-code which I have already tried.)

Important Note -

1) The converted file should get opened in excel properly. Many sub-columns have similar names and that should be renamed uniquely.

2) The reading process 1 billion file should not take non-ending time. Maximum half an hour time for reading will be ok. I have already used Sparklyr package. It takes half an hour for reading this big file. If you have any other package suggestion in R then that would be accepted.

Testing - I will require your R-code so that I can apply to 1 Billion data for the purpose of testing.

Attached Files -

1) Data structure - word file

2) 500 sample records (JSON format)

3) R file (whatever I have tried by using Sparklyr package)

4) One more R code (whatever I have tried by using jsonlite package)

Data size is 53.7 GB, My laptop has 8 GB RAM and 64 bit OS.

JSON file is in streaming format.

Skills: Data Processing, R Programming Language

See more: convert integer to numeric in r, convert multiple columns to numeric r, type.convert r, r convert character to numeric data frame, r convert character to integer, convert vector to dataframe r, convert to numeric r, r convert column to string, foxpro app data conversion, freelance data conversion project, data conversion forum, data conversion jobs, foxpro data conversion, data conversion companies pondicherry, sugarcrm data conversion tools, data conversion quality checking softwares, design logo data conversion company, data conversion quezon city, mysql data conversion spss, design data conversion access retrieval approaches

About the Employer:
( 0 reviews ) Pune, India

Project ID: #18095685

4 freelancers are bidding on average ₹6679 for this job


I have a good hands on working with Advanced R and Python and BI tools and technologies, AI, Big Data. I have quite a good knowledge of DL/ML Algorithm , have also developed Dashboards and Web Applications using flask/ More

₹11666 INR in 1 day
(60 Reviews)

Hello? I am a data scientist with a very strong command in the use and scripting of R programming language. I look forward to chatting on the project details

₹1750 INR in 1 day
(0 Reviews)

Hi there. I have sufficient experience in Big Data and R/Python, so I could help you completely. Please contact me and discuss details. Regards.

₹12000 INR in 1 day
(0 Reviews)

You need a person for data entry in Excel word pdf. I have 1 year of experience in my professional life as a Data Entry clerk. Your Project details that i see was writing in word and excel and table from PDF. i WILL More

₹1300 INR in 1 day
(0 Reviews)