Algorithm for data scrapping and sorting

hello, nice project, thank you for the invite, can we discuss it over chat ? the overall scope is cool, scrapping and managing data , what is the algorithm for also i bid for 2 days of work, on your project, however we can discuss exactly the amount after chat.

I am having 1. Expertise in Efficient Information retrieval mechanisms as well as worked on webservices as well 2. Expertise in high-performance, full-featured text search engine Technologies like Lucene, Solr. 3. Expertise in Semantic Web technologies, machine learning. 4. Expertise in Opensource Crawlers like Apache Nutch Research and Development is carried out on the following aspects Social Listening using Facebook, twitter apis to get real time data for analysis. Attribute Analysis to categorise data into predefined categories Data is displayed on google maps for which i can send the screen shot image as well.

4 years of experince in :  Machine learning and data mining : Proficient in implementing various supervised and unsupervised machine learning techniques like, knn-algorithm, regression models , clustering and classification etc.  Text mining and Web scraping and Natural language processing : Proficient in text mining and WORDCLOUD design. Experienced in WEB scraping and parsing XMLs. Recently working with APIs of different social media websites to create customer intelligence.  Statistical analysis of data : Proficient in different statistical tests and statistical models like, Chi-sq test, F-test, T-test, ANOVA, ANCOVA, correlation test , linear and logistic regression models.  Knowledge on Statistical tools: SAS certified Base Programmer for SAS v9. Proficient in R and revolution R along with numerous cran packages.  Programming languages : Proficient in Python scripting for extracting , transforming and performing mathematical and logical operations on unstructured data.  BIG DATA analytics: Intermediate in BIG DATA analytics using MAP REDUCE programming model and different tools like Apache PIG, HIVE. Knowledge on NOSQL database like HBASE.  MAP REDUCE programming : Proficient in implementing MAP REDUCE jobs over HADOOP clusters for data extraction and transformation and performing necessary data merges and joins using Python.  Data Visualization tools : Proficient in reporting in R using Google VIS and shiny. Also proficient in building reports in SAS.

A proposal has not yet been provided

As a final year phd student in machine learning, I have already successfully implemented several web crawlers; My thesis is related to natural language processing from social network data, therefore I am familiar with state-of-the-art models and algorithms that could suit your needs; Proposed solution: 1. spider - custom crawler already implemented in Java 2. classification - topic modeling (LSA or LDA or some extension, depending on data) + Support Vector Machines 3. indexing - Apache Lucene (or Solr if adding additional dependencies is not an issue) 4. query tool - custom code Looking forward to hearing from you. Best regards, CS