We want to build a large web application which will be handling large data processing jobs. The application will be a service which provides marketing for ecommerce stores.
the main fnctions that the application will do will the following.
1- sync product catalog from client's server to our application. can be done via plugins for different shopping platforms and web crawler for custom coded stores.
2- the application will analyse the cleint's data, sort it to categories for further processing and will ask client when certain columns does not match any category, the application needs for further tasks.
3- the applciation will create data feeds (XML files) according to the requirements of the channels (shopping engines and marketplaces)
4- application will sync the created data feed files to the channels via api call or provide client with the URL of the datafeed to upload to the channel.
in next phase we will add the following sections to the application,
dynamic adwords ads
i can provide more details to the right person but basics remains the same as above.
if you do it in php, then i think gearman is the way to go. we need the application to use microservice architecture approach.