Completed

Website crawler

The porpose is to create a website crawler which individues eternal dead links.

It should work on Windows and has not to make use of large ram (urls should be saved on MSsql not in the ram) , external dead links should be checked on whois in order to know if the domain has been registered and when it expires.

Would be great if this app make use of multhithreading, so that we can set how many threads we want to use.

We should also be able to choose the domain (the app should also scan subdomains related to the domain) , threads, frequency (number of page per second) and have, at the end a report of all dead links registered or not.

Skills: ASP.NET, C# Programming, JSON, PHP, Python

See more: online web crawler spider, get list of urls from site, website crawler python, seo crawler tool, how to crawl a website in google, crawl a website for pages, website spider, google site crawler, o create a website with the inventory on http://u, Create our new logo design for our new website! See the attached image for our old boring logo., Create our new logo design for our new website! See the attached image for our old boring logo. The \ G\ was a power \ on/off\ , would like a harverest website (similarly the picture in attachment file, I would like a harverest website (similarly the picture in attachment file), how to create a website in the internet, create website using html code notepad list out the next pages

About the Employer:
( 5 reviews ) trofarello, Italy

Project ID: #15668391

Awarded to:

ispyhumanfly

I have a multi-threaded solution for you. And that can run on Windows, I just need some specifics. Relevant Skills and Experience I am an American software engineer with more than 15 years of professional experience. More

$111 USD in 3 days
(6 Reviews)
3.1
t0x1c3500

Hello, Im an IT Engineer, i master many programming languages but i Love PHP and PYTHON, thats why i have bidded for your project Relevant Skills and Experience About the project : I think i can do this web crawler us More

$88 USD in 5 days
(4 Reviews)
3.6

16 freelancers are bidding on average $153 for this job

mascotindia123

Hello, Myself having has 10 + years experience of web site development. Expert in Custom programming in php / Mysql / Jquery / Wordpress/ Joomla / OSC / Magento /OpenCart / Android / Mobile development / Respons More

$220 USD in 4 days
(233 Reviews)
7.4
tyang0928

[url removed, login to view] I am a PM/Web developer with 7 years experience. I read fully your requirement and I am sure I can finish your project asap. I can work with my team if you want and can start right now. Relevant Skills and Ex More

$155 USD in 2 days
(25 Reviews)
6.0
pkbsdmp

I am a python developer having 6 + experience in IT industry. Relevant Skills and Experience Python, Web Development (Django, Flask),Database Programming, XML & JSON Parsing, Selenium, Web Scrapping, Data Science CSS More

$98 USD in 1 day
(62 Reviews)
5.5
schoudhary1553

Hello Sir, I am the expert freelancer for Website crawler job . i have deliver more than 250 projects with 100% client satisfaction. so please give me chance to work on your job. Relevant Skills and Experience I hav More

$200 USD in 3 days
(35 Reviews)
5.6
sushilpathariya

I have gone through your Job post and I can understand your job requirement thoroughly. Relevant Skills and Experience I have a total of 7 years of experience in ASP.NET, C# Programming, JSON, PHP, Python and had comp More

$222 USD in 3 days
(35 Reviews)
5.2
shinelancer

I am ready to do your project. I have experience writing many web crawler project. You can see my portfolio on software development. I hope to be your good friend through this project. Relevant Skills and Experience w More

$150 USD in 3 days
(17 Reviews)
4.9
pkudriavcevas

My name is Povilas and I would be perfect for your task. I'm specializing in data mining and can provide you top quality web crawlers. I have experence in this field from easy data scraping to complex scraping using More

$170 USD in 5 days
(15 Reviews)
4.7
ahmedh003

Hello, I am an expert Python Developer and I am also experienced with creating windows desktop applications as well as installers. I am also experience in MySQL databases and web scraping. From what I understood I w More

$222 USD in 6 days
(3 Reviews)
5.4
mail2vimalarora

Hi..having gone through your project details on scrapping, i can help you in getting it done quickly and efficiently. i have worked on several projects of scrapping/multi threading and related tasks. Relevant Skills a More

$177 USD in 3 days
(19 Reviews)
4.8
VirtualBrainInc

I have briefly read the description on IOS/Android development, and I can deliver as per the requirements however I need us to discuss for more clarity on the details, deadline and budget as well. Relevant Skills and More

$50 USD in 3 days
(8 Reviews)
3.8
Nada100200

Over 8 +years experience writing almost exclusively web scraping code. I've done it all. I can scrape all LinkedIn profile My languages in order of experience and use is Python, JavaScript, PHP. Python libraries ( sele More

$100 USD in 3 days
(5 Reviews)
4.1
nirmalsarswat

I love web crawling and project is interesting me. About my crawling and scrapping experience, I did crawling on from Twitter to Google. I am bit confused about the project, initiate a chat to discuss more

$111 USD in 5 days
(6 Reviews)
3.0
anstechsolution

I will build a solution in .NET which will be multi-threaded. More details to be discussed Relevant Skills and Experience I have extensive experience in building solutions based on .NET Proposed Milestones $333 USD - More

$333 USD in 10 days
(5 Reviews)
2.4
ITprogrammer1

Hello! I am writing in response to your Project. After carefully reviewing the experience requirements and skills required for the job, I feel that I am a suitable match for the job. I am a data scientist by professi More

$40 USD in 4 days
(0 Reviews)
0.0