如何查看网站被百度抓取的情况？

2011-04-09SEO

3,192 views

百度用于抓取网页的程序叫做Baiduspider – 百度蜘蛛，我们查看网站被百度抓取的情况主要是分析，网站日志里百度蜘蛛Baiduspider的活跃性：抓取频率，返回的HTTP状态码。

查看日志的方式：

通过FTP，在网站根目录找到一个日志文件，文件名一般包含log，下载解压里面的记事本，这即是网站的日志，记录了网站被访问和操作的情况。

因为各个服务器和主机的情况不同，不同的主机日志功能记录的内容不同，有的甚至没有日志功能。

日志内容如下：

61.135.168.22 – - [11/Jan/2009:04:02:45 +0800] “GET /bbs/thread-7303-1-1.html HTTP/1.1″ 200 8450 “-” “Baiduspider+(+http://www.baidu.com/search/spider.htm)”

分析：

GET /bbs/thread-7303-1-1.html 代表，抓取/bbs/thread-7303-1-1.html 这个页面。

200 代表成功抓取。

8450 代表抓取了8450个字节。

如果你的日志里格式不是如此，则代表日志格式设置不同。

很多日志里可以看到 200 0 0和200 0 64 则都代表正常抓取。

抓取频率是通过查看每日的日志里百度蜘蛛抓取次数来获知。抓取频率并没有一个规范的时间表或频率数字，我们一般通过多日的日志对比来判断。当然，我们希望百度蜘蛛每日抓取的次数越多越好。

日志信息 »

该日志于2011-04-09 16:26由 weightlose 发表在SEO分类下，你可以发表评论。除了可以将这个日志以保留源地址及作者的情况下引用到你的网站或博客，还可以通过RSS 2.0订阅这个日志的所有评论。

76条评论▼

www.liligou.com 说：

2012-01-12 于

感谢楼主又学到了一点
Colin Habel 说：

2012-01-15 于

I just want to say I’m beginner to weblog and truly liked this web-site. Most likely I’m planning to bookmark your site . You absolutely have impressive writings. Kudos for sharing your web site.
Magento webshop 说：

2012-01-21 于

I uncovered your web page via search motors even when looking for for the connected topic, your web page demonstrated up up. give many as a consequence of you for the fabulous blog. Amazingg skills! hold on man, you rock!
news 说：

2012-01-22 于

3 cheese
dumpster rental braidwood il 说：

2012-01-23 于

Me too, thanks for sharing this..
find a notary 说：

2012-01-23 于

good article i simply did not desire a initially, however once receiving one for article i am fully converted.
Channel Island Recruitment Agency 说：

2012-01-24 于

Blessed is the man who trusts in the Lord and has made the Lord his hope and confidence. – Jeremiah 17:7
tinnitus cure 说：

2012-01-24 于

Georg Christoph Lichtenberg – is believed to have mentioned Everyone is a genius at least once a year. The real geniuses simply have their bright ideas closer together.
Massage Therapy 说：

2012-01-24 于

Wonderful blog post. Massage along with Acupuncture companies certainly aids in assisting sufferers recover much faster coming from ache.
kathy grider 说：

2012-01-24 于

hi! =) i am at the job currently, so i do not have very much time to write… but! I truly appreciated reading your post. It turned out to be a bunch of excellent stuff. many thanks!
Jobs in the Channel Islands 说：

2012-01-25 于

Being an intellectual creates a lot of questions and no answers. – Janis Joplin
lucrare diploma managment 说：

2012-01-25 于

One more issue is that video games usually are serious in nature with the most important focus on mastering rather than fun. Although, we have an entertainment aspect to keep your kids engaged, just about every game is often designed to improve a specific expertise or programs, such as mathmatical or technology. Thanks for your post.
find cell phone numbers 说：

2012-01-27 于

You are my intake , I own few blogs and often run out from to post : (.
sacoche homme 说：

2012-01-29 于

I’d constantly want to be update on new posts on this website , saved to my bookmarks ! .
help with credit card debt 说：

2012-01-29 于

I think one of your adverts caused my internet browser to resize, you might want to put that on your blacklist.
suspended ceiling 说：

2012-01-29 于

I will immediately grab your rss feed as I can’t in finding your email subscription link or newsletter service. Do you have any? Please permit me recognise in order that I may subscribe. Thanks.
trampoline avec filet 说：

2012-01-30 于

There is perceptibly a bundle to identify about this. I consider you made some nice points in features also.
sorbetiere 说：

2012-01-31 于

You are my aspiration , I possess few blogs and infrequently run out from to post : (.
robot de cuisine 说：

2012-01-31 于

Thank you for sharing excellent informations. Your web site is very cool. I’m impressed by the details that you’ve on this blog. It reveals how nicely you understand this subject. Bookmarked this website page, will come back for more articles. You, my friend, ROCK! I found simply the information I already searched all over the place and simply couldn’t come across. What an ideal website.
online dating websites 说：

2012-01-31 于

But wanna say that this really is invaluable , Thanks for taking your time to write this.
loose tea 说：

2012-01-31 于

As a Newbie, I am constantly exploring online for articles that can help me. Thank you
trampoline pas cher 说：

2012-01-31 于

I believe you have noted some very interesting points , regards for the post.
white ceramic watches for women 说：

2012-01-31 于

I wanted to follow along and let you know how , very much I cherished discovering your blog today. I might consider it a great honor to do things at my place of work and be able to operate on the tips contributed on your site and also engage in visitors’ responses like this. Should a position connected with guest article writer become offered at your end, make sure you let me know.
uk email marketing 说：

2012-02-01 于

Hello would you mind sharing which blog platform you’re using? I’m looking to start my own blog in the near future but I’m having a tough time choosing between BlogEngine/Wordpress/B2evolution and Drupal. The reason I ask is because your design seems different then most blogs and I’m looking for something completely unique. P.S Sorry for getting off-topic but I had to ask!
echelles telescopiques 说：

2012-02-01 于

Well I really liked studying it. This tip procured by you is very useful for accurate planning.
网络电视说：

2012-02-01 于

allintext:SEO博客

SEO博客,在SEO中实战中的阿猎SEO项目管理博客