草根评论

对Majestic-12分布式搜索引擎的看法

zcmhi — Mon, 29 Nov 2010 09:29:34 +0800

最近在WordPress上安装了StatPressCN统计插件，来观察搜索引擎蜘蛛对网站的抓取状况。在统计后台看到除Baiduspider，Googlebot，Slurp(Yahoo)等常见的抓取蜘蛛外很奇怪的MJ12bot。搜索一番发现MJ12bot是Majestic-12分布式搜索引擎的Crawler。通过百度搜索还发现MJ12bot对中文网页的抓取相当普遍。原来Majestic-12是一个位于英国的使用类似于SETI@home和distributed.net项目分布式处理技术的搜索引擎，它利用分布于世界各地网民的计算机闲置的处理能力和带宽来抓取网页。只要上网者在自己的电脑上安装一个叫做MJ12node的客户端软件，就可以帮助Majestic-12抓取网页并上传到主服务器。抓取的网页将会被分析、索引并添加到Majestic-12 搜索引擎。看到这些ZCMHI非常感兴趣，毕竟云计算，分布式处理在现在是非常前沿的技术领域。在Majestic-12 About页表明创立Majestic-12分布式搜索引擎的主要目的，是为了创建大众自己控制的搜索引擎。在互联网有数千万计的网站和数十亿计的网页，目前只有一些大公司有能力创建搜索引擎来提供相关搜索信息给用户。大公司控制着你所查找信息的入口点，广大网民和网站管理员对此毫无发言权。然而点击所谓的大众自己的搜索引擎Majestic-12 search engine时，网页却无法打开。在测试使用了美国、德国、法国等多地多个代理服务器甚至英国的代理时也同样打不开。在Majestic-12网站首页左上方有一搜索框，醒目写着Distributed Search Engine，难道指的就是这个？然而按照提示Enter domain or URL输入了www.zcmhi.com搜索时却跳转到majesticseo.com，显示Majesticseo网站对输入域名反向链接的搜索结果。 Majesticseo网站是一个SEO类网站，提供网站反向链接，反链历史，反链IP等的查询。有价值的相关服务是要付费的，设立铜、银、金级别的付费服务。级别愈高，支付的钱愈多，能够看到的信息就愈多。这就是所谓大众自己的搜索引擎吗？公开表明的搜索引擎入口无法进入，把人误导到Majesticseo SEO网站。只能够搜索查询SEO相关的反向链接信息，而且有价值的还要服务费！这是打着创建大众自己的搜索引擎的幌子，利用分布式计算，把大众计算机的处理能力和带宽转化为白花花的银子流入自己的腰包！本来打算下载安装MJ12node程序加入分布式计算，现在却是越来越失望！相信很多明白的人会有同感。其实在ZCMHI看来分布式搜索引擎是很有前景的。互联网网站网页无数，这对搜索引擎硬件条件、计算能力要求极高。只有大型公司才可能独自建立索引全球互联网的搜索引擎。Google的数据中心规模就相当庞大，服务器遍及全球各地。数据中心以集装箱为单位，每个集装箱里面有1160台服务器，每个数据中心有众多集装箱。查看图说Google数据中心。Google有如此庞大的数据中心，然而面对更庞大的互联网，目前也只能索引40%的网页。如果利用分布式计算就不同了，它利用成千上万的个人电脑来抓取网页处理搜索引擎的任务。许多计算机同时处理一项任务要比一台巨大的服务器来完成这项任务快得多。如果有许多用户加入到分布式搜索引擎，那么网站服务器就会处于搜索引擎的包围之中。这时分布式搜索引擎的处理能力将远大于一般的搜索引擎，将可以抓取和索引更多的网站，将成为全球最大的无可比拟的搜索引擎！创立分布式搜索引擎的首要任务，是想方设法吸引更多的网民加入到分布式计算，而Majestic-12急功近利，只看重眼前利益，将不会形成真正的分布式搜索引擎。 ©版权所有：ZCMHI’S BLOG，转载请注明本文链接:http://www.zcmhi.com/archives/360.html

Dear John

mianbao49 — Fri, 12 Nov 2010 16:02:42 +0800

Mr. Lord is correct, “…conservatism and conservatives will sail on with — or without — McCain in the White House. Conservatives can help Microsoft outlook is convenient! him get there.” His article, as always is insightful and thoughtful, but there are too many unknowns that appear to have indicators as to what will really happen. Could the [...]

Dear John

mianbao49 — Fri, 12 Nov 2010 16:02:28 +0800

Yes, Mr. Lord, in answer to your question, yes McCain could put together such a Cabinet. p>Alternatively — or as preparation for handling that collection — he could take up herding rattlesnakes. br> — A.C. Santore /p> Office 2007 key is available here. A thoughtful article, Mr. Lord. However, assuming that McCain is the last man standing [...]

Beggaring All Description

mianbao49 — Fri, 12 Nov 2010 16:02:14 +0800

That quote illustrates one strength of the book — its extensive use of passages from letters and diaries of those who lived, on and off the Microsoft outlook 2010 is convenient! battlefields, during the Civil War. Susannah Hampton wrote to a Hospital Directory in Philadelphia two months after Gettysburg, br> /p> Many people like Microsoft Office. will [...]

Beggaring All Description

mianbao49 — Fri, 12 Nov 2010 16:02:00 +0800

Even if one knew the names of the slain, many people back home (not far away) often had trouble, Microsoft word is so great! in the terminology of the time, “realizing” Microsoft outlook is many people’s favorite. the deaths of their beloveds, which for them meant “to render it real in their own minds.” Windows 7 make [...]

Beggaring All Description

mianbao49 — Fri, 12 Nov 2010 16:01:47 +0800

Death and the American Civil War br> By Drew Gilpin Faust br> (Knopf, 346 pages, $27.95) /strong> /p> Office 2010 is powerful! “The number of soldiers who died between 1861 and 1865, an estimated 620,000, is approximately equal to the total American fatalities in the Revolution, the War of 1812, the Mexican War, the Spanish-American War, World [...]

What’s to Negotiate?

mianbao49 — Fri, 12 Nov 2010 16:01:33 +0800

Iranian President Mahmoud Ahmadinejad, himself has said he seeks a Muslim brotherhood under the slogan “the family that prays together, stays together.” He indicated that any attempt to create a divide between Shia and Sunni is a Zionist plot Microsoft outlook is convenient! to promote dissension. In negotiation between Egypt and Iran, it was noted that “we are [...]

What’s to Negotiate?

mianbao49 — Fri, 12 Nov 2010 16:01:19 +0800

IF IRAN CEASED its nuclear weapons program in 2003 as the NIE report asserts, it probably did not do so because of negotiation, another NIE assertion, but rather because of the American invasion of Iraq in that year. Guns often speak louder than words.Office 2007 key is available here. But counter force strategy used to deter Iran’s [...]

What’s to Negotiate?

mianbao49 — Fri, 12 Nov 2010 16:00:56 +0800

IS THE U.S. INOCULATED against a grand bargain in which Iran’s regional dominance is acknowledged in return for non-deployment of nuclear Many people like Microsoft Office. weapons? It’s doubtful, and even without the actual bomb, Iran might secure this concession.Windows 7 is convenient and helpful! Cheering on the sidelines would be Russia and China eager to diminish [...]

What’s to Negotiate?

mianbao49 — Fri, 12 Nov 2010 16:00:43 +0800

At the recent Herzliya conference former U.S. Ambassador to the United Nations John Bolton said, “there is almost no chance that President Bush will approve a military strike against Iran before he leaves office.” If there was once a Microsoft word is so great! credible military option, it has been undermined by the National Intelligence Estimate, or I [...]