2008年11月2日星期日

Google的robots.txt更新

http://www.google.com/robots.txt
偶然发现Google的robots.txt更新和之前相比增加了一些东西
User-agent: zombies
Disallow: /brains

User-agent: *
Allow: /searchhistory/
Disallow: /news?output=xhtml&
Allow: /news?output=xhtml
Disallow: /search
Disallow: /groups
Disallow: /images
Disallow: /catalogs
Disallow: /catalogues
Disallow: /news
Disallow: /nwshp
Allow: /news?btcid=
Disallow: /news?btcid=*&
Allow: /news?btaid=
Disallow: /news?btaid=*&
Disallow: /setnewsprefs?
Disallow: /index.html?
Disallow: /?
Disallow: /addurl/image?
Disallow: /pagead/
Disallow: /relpage/
Disallow: /relcontent
Disallow: /sorry/
Disallow: /imgres
Disallow: /keyword/
Disallow: /u/
Disallow: /univ/
Disallow: /cobrand
Disallow: /custom
Disallow: /advanced_group_search
Disallow: /googlesite
Disallow: /preferences
Disallow: /setprefs
Disallow: /swr
Disallow: /url
Disallow: /default
Disallow: /m?
Disallow: /m/?
Disallow: /m/ig
Disallow: /m/lcb
Disallow: /m/news?
Disallow: /m/setnewsprefs?
Disallow: /m/search?
Disallow: /m/trends
Disallow: /wml?
Disallow: /wml/?
Disallow: /wml/search?
Disallow: /xhtml?
Disallow: /xhtml/?
Disallow: /xhtml/search?
Disallow: /xml?
Disallow: /imode?
Disallow: /imode/?
Disallow: /imode/search?
Disallow: /jsky?
Disallow: /jsky/?
Disallow: /jsky/search?
Disallow: /pda?
Disallow: /pda/?
Disallow: /pda/search?
Disallow: /sprint_xhtml
Disallow: /sprint_wml
Disallow: /pqa
Disallow: /palm
Disallow: /gwt/
Disallow: /purchases
Disallow: /hws
Disallow: /bsd?
Disallow: /linux?
Disallow: /mac?
Disallow: /microsoft?
Disallow: /unclesam?
Disallow: /answers/search?q=
Disallow: /local?
Disallow: /local_url
Disallow: /froogle?
Disallow: /products?
Disallow: /froogle_
Disallow: /product_
Disallow: /products_
Disallow: /print
Disallow: /books
Allow: /booksrightsholders
Disallow: /patents?
Disallow: /scholar?
Disallow: /complete
Disallow: /sponsoredlinks
Disallow: /videosearch?
Disallow: /videopreview?
Disallow: /videoprograminfo?
Disallow: /maps?
Disallow: /mapstt?
Disallow: /mapslt?
Disallow: /maps/stk/
Disallow: /maps/br?
Disallow: /mapabcpoi?
Disallow: /center
Disallow: /ie?
Disallow: /sms/demo?
Disallow: /katrina?
Disallow: /blogsearch?
Disallow: /blogsearch/
Disallow: /blogsearch_feeds
Disallow: /advanced_blog_search
Disallow: /reader/
Disallow: /uds/
Disallow: /chart?
Disallow: /transit?
Disallow: /mbd?
Disallow: /extern_js/
Disallow: /calendar/feeds/
Disallow: /calendar/ical/
Disallow: /cl2/feeds/
Disallow: /cl2/ical/
Disallow: /coop/directory
Disallow: /coop/manage
Disallow: /trends?
Disallow: /trends/music?
Disallow: /notebook/search?
Disallow: /music
Disallow: /musica
Disallow: /musicad
Disallow: /musicas
Disallow: /musicl
Disallow: /musics
Disallow: /musicsearch
Disallow: /musicsp
Disallow: /musiclp
Disallow: /browsersync
Disallow: /call
Disallow: /archivesearch?
Disallow: /archivesearch/url
Disallow: /archivesearch/advanced_search
Disallow: /base/search?
Disallow: /base/reportbadoffer
Disallow: /base/s2
Disallow: /urchin_test/
Disallow: /movies?
Disallow: /codesearch?
Disallow: /codesearch/feeds/search?
Disallow: /wapsearch?
Disallow: /safebrowsing
Disallow: /reviews/search?
Disallow: /orkut/albums
Disallow: /jsapi
Disallow: /views?
Disallow: /c/
Disallow: /cbk
Disallow: /recharge/dashboard/car
Disallow: /recharge/dashboard/static/
Disallow: /translate_c
Disallow: /translate_suggestion
Disallow: /s2/profiles/me
Allow: /s2/profiles
Disallow: /s2
Disallow: /transconsole/portal/
Disallow: /gcc/
Disallow: /aclk
Disallow: /cse?
Disallow: /tbproxy/
Disallow: /MerchantSearchBeta/
Disallow: /ime/
Disallow: /websites?
Disallow: /shenghuo/search?
Disallow: /support/forum/search?
Disallow: /reviews/polls/
Disallow: /hosted/images/
Disallow: /hosted/life/
Disallow: /newspapers?
Disallow: /search2001/search?
Disallow: /ppob/?
Disallow: /ppob?
Sitemap: http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xml

没有评论:

免责声明

1、本人是文盲,以上内容文字均不认识,也看不懂是什么意思(包括但不限于对所以上之内容的识别、阅读、理解、分析、记忆等);

2、本人过去、现在以及将来都不认识本文中提及当事人,且自古以来与该相对人无利益关系;

3、本人昨天、今天以及明天都没有或者不准备去本文所述地点。本文表述之事与本人无关。

4、本人在此发文(包括但不限于汉字、拼音、拉丁字母、斯拉夫字母、日语假名、阿拉伯字母、单词、句子、图片、影像、录音、以及前述之各种任意组合等等)均为随意敲击键盘所出,用于检验本人电脑键盘录入、屏幕显示的机械、光电性能,并不代表本人局部或全部同意、支持或者反对文中观点。如需要详查请直接与键盘发明者及生产厂商法人代表联系;

5、人生有风险,上网需谨慎。本文不暗示、鼓励、支持或映射读者作出生活方式、工作态度、婚姻交友、股票债券买卖、子女教育的积极或消极判断。未成年人请在监护人陪同下阅读本文。无完全民事行为能力者,请立即关闭网页,并用20%高锰酸钾+75%乙醇对键盘、硬盘、电压插座、显示器、鼠标、cpu进行灌溉消毒;

6、如本人留言违反国家有关法律,请网络管理员及时删除本文,本人保留继续发文的权利;

7、因删贴不及时所产生的任何法律(包括宪法、加法、减法、乘法、除法、剑法、拳法、脚法、指法、民法、刑法、书法、公检法、基本法、劳动法、婚姻法、输入法、没办法、国际法、今日说法、吸星大法及文中涉及或可能涉及以及未涉及之法,各地治安管理条例)纠纷或责任本人概不负责;

8、本人谢绝任何跨省(包括但不限于跨国、跨洲、跨星球、跨星系)追捕行为。确因不抓不足以平民愤,或不抓就领不到薪水养家户口的公职人员,建议携带工作证、身份证、结婚证/离婚证、独生子女证、健康证、暂住证、毕业证、边防证、县以上政府机关出具的介绍信温情操作。抓捕按照以下排序倒序:作者、原作者以及网络管理员以及网络运行商、电信运营商、电力供应商、电脑生产销售商;

9、如回复内容导致回复者或第三方、第四方、第N次方怀孕,本人愿配合做亲子鉴定。