creation of a spider and search engine

We are looking for an experienced programmer who can start to create a search engine along the lines of Google. This person will be required to help us specifiy requirement, find existing open source software and help to integrate the elements together as well as any custom programming required. We need: 1. A spider which will go out and trawl the web, with the ability to catalogue entire sites if necessary 2. A database back-end and any necessary algorithms The above is very brief as we (at this stage) do not know fully what is required. We will only look at bids from someone who has already done work along the above lines, so please do not bid if you have not done this before as your bid will be automatically rejected when you post a bid, the money is NOT the most important aspect, but showing existing work along similar lines or a methodology which shows that it is an area you have covered. ## Deliverables 1) Complete and fully-functional working program(s) in executable form as well as complete source code of all work done. 2) Installation package that will install the software (in ready-to-run condition) on the platform(s) specified in this bid request. 3) Exclusive and complete copyrights to all work purchased. (No GPL, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site). ## Platform This MUST run under Linux servers (or a cluster of servers) and be coded in C unless you can convince us that it is more appropriate to code in another language