Closed

CSV > MySQL Import Script tweaks and overal DB/speed tune-up.

We currently have a PHP script which downloads and imports about 50 separate CSV files (They all using the exact same column format) into one large database (Roughly 250,000 records):

This script however is running slow 20 records per second. (mysql_query(INSERT INTO))

Initially we'd like someone to look through the script and make any obvious improvements in the code to speed things up (inc suggestions to database structure, use of primary, indexes etc) and also perhaps [url removed, login to view] tweaks.

Longer term we'd like to look into swapping over to LOAD DATA LOCAL INFILE, and also adding a number of rules so we can add product categories based on product name, store name etc.

We see this very much as a 2 stage project and would like to build a long term relationship for further work on the websites which use the product database.

Thanks, Jason.

Skills: MySQL, PHP

See more: import csv into mysql php script, php mysql import script, mysql import database csv, prestashop mysql import script, mysql import script, access mysql import script, php mysql import script json, mysql import export csv magento, mysql import script create database, mysql import script linux, magento import script csv

About the Employer:
( 11 reviews ) Ipswich, United Kingdom

Project ID: #7994387

53 freelancers are bidding on average £26/hour for this job

leadconcept

Greetings Jason, I have read Project Description & understood the initial details but wondering, if you have more detailed requirements, some high level features? I also understand that it is 2 phase project, so I w More

£30 GBP / hour
(72 Reviews)
8.7
meet2amitvw

Let's discuss over freelancer Personal Message Box for the proper estimation of cost and time. I am myself developer so you will directly work with me. No mediators. No managers. No subcontractors. see my recent More

£18 GBP / hour
(110 Reviews)
8.6
kchg

Hello! I am interest your project and I hope to work with you. My Ranking is Top 7th on freelancer. I have enough experience with WEB developing for 5+ years. Client satisfaction is my top priority. I will comple More

£30 GBP / hour
(269 Reviews)
8.7
toinnisfree

I have read your project details, and I am interested in offering my services. I provide 100% satisfaction guaranteed on all my work. Milestone is required to begin; to be released when the job has been satisfactor More

£20 GBP / hour
(685 Reviews)
8.2
seekdeveloper

Hi, I have read your post and understood your requirement. I have great experience in handling /PHP/Wordpress/Magento/Joomla/Drupal/AngularJS/[url removed, login to view] More

£30 GBP / hour
(80 Reviews)
8.0
mmagr99

Hi, We are recommended Freelancer by [url removed, login to view] on basis of our feedback history and [url removed, login to view] once read our [url removed, login to view] are really interested to work on this project :) Thanks Regards DreamSolutions

£36 GBP / hour
(79 Reviews)
7.3
coolguyinus2004

11+ years of experience in programming PHP/MYSQL/API/WEB SCRAPING/DEBUG EXPERT/ANY KIND OF FIX IN [url removed, login to view] PHP CODING You can check my profile for my reputation and ranking . I can fix the PHP code and also can More

£31 GBP / hour
(327 Reviews)
7.6
fattahaabdul

I will look into the script and then can tell you. Please contact me. i have 8+ years of experience. Can we discuss the project. Please initiate a chat with me so that we can discuss the project at a broader level. More

£37 GBP / hour
(28 Reviews)
7.4
pivn

let expert do this.............................................................................................................

£30 GBP / hour
(189 Reviews)
6.6
oscworker

Hello. LOAD DATA LOCAL INFILE do magic. With this method you will be able upload your data in minutes (should be allowed in [url removed, login to view]). I think I can complete this task in 2-3 hours. Thanks, Vitaliy.

£20 GBP / hour
(93 Reviews)
6.9
datadiggers

Hello, I'm Maciej and I'm Web Performance Optimization consultant (I work only on projects related to performance optimization). I optimize both front end and backend (PHP, MySQL, low level Linux configuration). I h More

£53 GBP / hour
(50 Reviews)
6.8
scriptphp87

Hello, my portfolio: [url removed, login to view] I'm a professional programmer for web programming with php language to build the system website, Besides, I'm also expert in MySQL , HTML,HTML5,CSS, JS I'm always to More

£21 GBP / hour
(171 Reviews)
7.5
ssahilsl

Hi, HIRE an expert team of Custom Design, responsive/mobile friendly, HTML, Bootstrap, WordPress, PHP and Joomla who has been working since 7+years on many projects. We will provide you quality and professional web wit More

£18 GBP / hour
(126 Reviews)
7.1
csajit

HELLO THERE, Over 200+ Positive Reviews. Over 90% Completion Rate. 5 Star Rating. 8+ Years of experience in PHP/MYSQL I AM INTERESTED IN WORKING ON THIS PROJECT. PLEASE CHECK MY REVIEWS AND THEN DECIDE ON More

£20 GBP / hour
(212 Reviews)
6.5
Thesynapses

Hello Jason, Greetings From Synapse! Hope this finds you well. I had studied your comment and it seems that at first stage you want some changes so that the system will boost up. I would like to inform you that More

£28 GBP / hour
(12 Reviews)
6.5
eclipsetechno

Hi Jason, I would like to know the script you have written and we would try to optimize the query with the following process: BULK_COLLECT This function is to be used so that max data can be taken from the req More

£28 GBP / hour
(21 Reviews)
6.5
TenStar718

Hello, and thanks for the opportunity to bid on your project. [url removed, login to view] I am an expert in many different area’s of web and mobile applications based on the following languages: P More

£26 GBP / hour
(54 Reviews)
6.1
winmaclin

Hello Jason, In a view of the job posting done by you, we consider ourselves apt fit to increase and optimize the speed of your PHP script to download and import CSV files. As we have developed wide range of PHP scr More

£18 GBP / hour
(13 Reviews)
6.2
prashushinde9

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably More

£25 GBP / hour
(37 Reviews)
6.4
r4rony

Dear Client, We have gone through your job description and understood it well. We are confident to deliver you the desired work. Please let me know what features you need. We are web based Development Company. We ha More

£20 GBP / hour
(31 Reviews)
6.1