Write some Software

Complete custom web scraping.

We need to extract companies and the reviews of them from a number of sites, one site has been done already within a C# solution that will be clear to continue the other sites with, and so the info below is for your estimating.

If you are chosen you will need VS2015 installed (community fine) and you will check-in your work as you go to visual studio online TFS.

The sites below are ordered by priority.

1. [url removed, login to view]

2. [url removed, login to view] (done already)

3. [url removed, login to view] (‘started’ by a friend, but I cannot vouch for progress made or code quality as I haven’t looked into finishing).

4. [url removed, login to view]

5. [url removed, login to view]

6. [url removed, login to view]

7. [url removed, login to view]

8. [url removed, login to view]

9. [url removed, login to view]

10. [url removed, login to view]

For each of the other sites you will take a copy of the ‘TODOExtractor’ file.

I suggest you get ‘plumber’ category working first for each…then the categories for further testing are in the attached [url removed, login to view], these may be named slightly differently depending on the website, e.g. ‘plumbing’ vs ‘plumber’. If any cannot be matched to one or more sites please inform.

You can do each site either by grabbing from any JSON API you discover or by scraping the HTML…here are some links that may be useful for sites that require scraping:

[url removed, login to view]

-Which links to: [url removed, login to view]

[url removed, login to view]

-Which links to: [url removed, login to view]

Once the data is extracted the program will write out to output files. The file writing is done for you in the existing C# project…there is shared code to call that writes out files in the required output format into a ‘TestOutput’ folder within the project. It creates a simple semi-colon separated, ordered list of fields saved as .txt files but none of this will need to change or should be changed.

The program generates an Id for the company during the extraction process. This Id must also be present in the Reviews file as a ‘foreign key’. This is done for you in the existing C# project. This will not change.

For ‘TrueLocal’ we have the ability to run the extraction program on combination of a particular category e.g. ‘plumbing’ and a particular postcode…this ability will depend on the site and postcode can be swapped out ‘Locality’ or ‘RegionCode’ if that makes more sense for the particular site, but otherwise it should run in the same manner.

The data to be extracted is as below.

BusinessLead Review

Id varchar(16) NOT NULL, -- Generated as described later

PhoneNumber varchar(20) NULL,

Email varchar(255) NOT NULL, -- REQUIRED and MUST be UNIQUE

FirstName varchar(50) NULL,

LastName varchar(50) NULL,

[Address] varchar(100) NULL, -- Unit number where applicable

Address2 varchar(100) NULL, -- The Main Street address

Locality varchar(80) NOT NULL, -- REQUIRED! Town/Suburb e.g. RICHMOND

RegionCode varchar(3) NOT NULL, -- state: VIC, NSW, TAS, NT, SA,

PostalCode varchar(4) NULL, -- Australian postcode 2000, 3000

Company varchar(80) NOT NULL, -- Company NAME...REQUIRED and MUST be UNIQUE

BusinessNum varchar(15) NULL, -- This is ABN. ([url removed, login to view])

Description varchar(8000) NULL, -- How the business describes itself

FoundingYear int NULL, -- If available

EmployeeCount int NULL, -- If available

--Following is LIKE an array [], e.g: [Plumbing,Handyman] just extract them with names as they are from the target sitePredictedServiceTypeNames varchar(2000) NOT NULL LeadId varchar(16) NOT NULL, --the ‘foreign key’ ref to BusinessLead

OverallRating int NOT NULL, --out of five.

ReviewText varchar(6000) NOT NULL,

FirstName varchar(50) NOT NULL, --Of Reviewer

LastName varchar(50) NOT NULL, --Of Reviewer

Email varchar(255) NULL, --Of reviewer IF available

ReviewDate datetime NOT NULL --SHOULD be available

Skills: C# Programming, Web Scraping

See more: www yelp com, writing software reviews, writing description of a friend, html, software architecture,, web scraping, wpf, writing companies for custom writing, write your name online, write email to friend, write business reviews, write an email to a friend, work in nsw, working for target reviews, where can i write articles online, website like custom writing, website for software testing, web scraping solution, webscraping software

About the Employer:
( 0 reviews ) Australia

Project ID: #10399388

21 freelancers are bidding on average $1700 for this job


Hi, I’ve had a good look at your project description and I’m very interested in providing a solution. I have 4+ hands on experience on developing quality web solutions as well as desktop standalone solutions. I u More

$526 AUD in 10 days
(60 Reviews)

hi i already coded yelp,[login to view URL],[login to view URL],[login to view URL] .i can scrap your required [login to view URL] send me categories [login to view URL]

$316 AUD in 1 day
(176 Reviews)

Hello, I am VB, VBA, C#, WPF, Java, Delphi, PHP, ASP.NET, CAD and GIS developer, I developed many desktop applications and custom tools, macros and Add-Ins. For reference see my profile. I am ready to discuss and More

$10000 AUD in 90 days
(13 Reviews)

I am an expert in creating bots using C# and can do this scrapping task perfectly............................................ Thanks

$2105 AUD in 10 days
(69 Reviews)

Greeting! My name is Mohammad Taradeh and I am a Microsoft Certified Professional Developer (MCPD). I would love to have the opportunity to discuss your project with you. I read you project details line by line, a More

$1111 AUD in 10 days
(17 Reviews)

I have done many scraping projects including yelp, yellow pages, fanduel, twitter and more. I have reviewed some urls you have provided. and in most of the cases, I have to code. Please contact me, I am ready to More

$2000 AUD in 60 days
(27 Reviews)

Hello Employer, We have analysed the Details all you need from the mentioned website the data that has to be scrapped i need the following points for discussion: 1. Budget [login to view URL] We have an experience of More

$555 AUD in 10 days
(16 Reviews)

hey i can do this job since i am expert in Scrapping using C#, i have more than 2 years of work experience in this field i already have scrapped quite number of websites such as amazon, [login to view URL], ebay, genericparts. More

$600 AUD in 3 days
(21 Reviews)

Hi, My name is Saddam and I am representing MuteX Solutions. We saw your description and we understand the main features you want in the software. You need to give us some more details about the additional feature More

$598 AUD in 10 days
(7 Reviews)

Hi, I have gone through your requirements in brief and understand your idea and preference for job as you are looking for. I can meet up all your requirements. If you give me opportunity to work with you, I'll prove More

$4000 AUD in 30 days
(3 Reviews)

Dear Sir, I have read Project Description & understand the overall requirements but not sure about the budget, as AUD 750 is very low, so would you be flexible in it? My current bid is just a placeholder to submi More

$833 AUD in 30 days
(1 Review)

Hi, I have 10+ years of experience in developing web scraper, websites, windows based applications and mobile applications using ,c#, php, java, j2ee, MS Sql server, My Sql, crystal reports, SSIS, unity 3d, Htm More

$750 AUD in 25 days
(2 Reviews)

Understand exactly what is needed and wanted, do a full business specification with wire-frames if needed. Get approval on the business specifications. Start the development

$556 AUD in 10 days
(1 Review)

I am a .net developer with 4 years of experience in building console and web based application. I have delivered my previous applications in a stipulated time frame and with a good quality code. I am really interest More

$1666 AUD in 25 days
(0 Reviews)
$555 AUD in 10 days
(0 Reviews)

Hello we are 5 years experienced web [login to view URL] work on wprdpress , php , codeignitor, bootstrap,opencart,Angualr js,joomla, Zend ,Android , E-commerce ,Bigcommerce etc. Please take a look at our portfolio to More

$5555 AUD in 45 days
(0 Reviews)

My name is Mike and I’m from UK. I work with individual clients and also provide outsourcing services for a number of UK and USA based agencies. Your project description sounds interesting to me and I do have skills & More

$555 AUD in 10 days
(0 Reviews)

Hi, I have read your post and understood your requirement. I have great experience working on /C# Programming/PHP/MySQL/HTML5/jQuery/Wordpress/Magento/Joomla/Drupal/AngularJS/node.js/CSS3/Java/Python/Django/Javas More

$1546 AUD in 14 days
(0 Reviews)

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably More

$773 AUD in 20 days
(0 Reviews)

I'd like to discuss this with you more in depth first. I have created multiple data scrapers, actually, I have already created one for [login to view URL] Let's chat more and workout the rest. Regards, Alan

$555 AUD in 10 days
(0 Reviews)