On bugs.debian.org, we notice that there is a "disallow" directive in your robots.txt that blocks our crawler from accessing pages on your site.
I cannot speak for Debian, but I suspect this is because generating the html version of the bugs site needs more CPU power than they are willing to give search engines for free. Maybe if you were to make some suitably large donation to cover the cost of adding that power, people would reconsider.
[...] Our goal is to provide the most relevant content to web searchersand Debian.org being a major open source/linux web site, we are very interested in having access to the relevant content on your site.
Why do you feel you should index the entire bugs site yourself? If people wish to search for bugs in debian, there are specialised search engines for that. Also, you seem to already index lists.debian.org, which should contain nearly all the useful content on bugs.debian.org AIUI.
By the way, have you read the current site? It says: "Debian is a free operating system (OS)". There is a link there that explains what "free" is, and "open source" is not an exact synonym. Debian is also developing OSes around other kernels than Linux.
Meanwhile, I am very interested in having access to the Ask Jeeves site. However, parts of the site have basic accessibility errors, such as setting text colours and not setting background colours.
One of your legal notices appears unreasonable. It says "You agree not to display or use the Ask Jeeves Marks in any manner without Ask Jeeves' prior written permission." Why do you claim that I have agreed not to use your trademark in any manner, even though it is permitted by the UK Trade Marks Act 1994?
I would greatly appreciate your help in these matters. -- MJR/slef My Opinion Only and possibly not of any group I know. http://www.ttllp.co.uk/ for creative copyleft computing Can you answer emails? http://mjr.towers.org.uk/proj/survey/