Archives

Here is the latest issue of the Pandia Post. Go to www.pandia.com/sew/ for more news about web searching, search engines and search engine optimization.

Cuil - new large search engine

A new large search engine called Cuil (pronounced “cool”) was launched this week.

By Pandia Guest Writer Lars Våge, Internetbrus

According to Cuil the search engine is three times as big as Google, Yahoo and the others, with more than 120 billion web pages in its index.

The last time I saw figures this high in relation to other search engines was for Recall, the short lived full text search of the Internet Achive in 2003. (That being said, Google now claims to have found 1 trillion pages on the web).

Anna Patterson from Stanford University was responsible for the Recall Project, and she is actually one of the founders of Cuil.

In this team we also find a certain Mr Louis Monier. Yepp, that was the man that created AltaVista. Both Anna, Louis and some other members in the Cuil team have until recently been working for Google.

Does size matter?

So to what extent does the size of the database mean anything for search engines?

Click here to read the rest of this article!

Will Google’s Knol become the next Wikipedia?

Google has launched its Wikipedia-killer to the world. Will they succeed?

Last week Google made its new site, Knol, open to all the world. Until then only beta-testers had been able to read and contribute.

Knol lets anyone write an article about any topic of interest.

Then readers may make comments and rate the article. They may also may suggest addition and rewrites, but the first author retains control of the text.

The author has the last word

It is the empowerment of the first author that makes Knol different from the Wikipedia. In the Wikipedia anyone may (normally) go in and edit and expand on an article.

In Knol the author may reserve the right to accept or reject such rewrites. This is called “Moderated Collaboration” by Google.

This alternative way of doing it means that some of the articles become less like the authoritative and scholarly encyclopedia articles we find in the Wikipedia, and more like essays.

Click here to read the rest of this article!

How big is the Web?

How many Web pages are there out there, really? More than 1 trillion, Google reports.

The fact is that no one knows the exact number, and the fact that a lot of the content is hidden behind database search forms means that we never will.

Still, the search engine companies make a living out of spidering the Web, trying to find all the good stuff, and they probably have a pretty good idea about what size we are talking about here.

The Google Blog now reports:

“Recently, even our search engineers stopped in awe about just how big the web is these days — when our systems that process links on the web to find new content hit a milestone: 1 trillion (as in 1,000,000,000,000) unique URLs on the web at once!”

The blog truthfully reports that this is not the total number, as Google has not indexed all the pages there is. Moreover, there are problems of defining what a web page is:

Click here to read the rest of this article!

A common standard for the Robots.txt protocol

Microsoft, Yahoo and Google have developed a detailed documentation about how they implement the Robots Exclusion Protocol (REP).

The search engines’ robots go wherever they want as long as they find links to follow. This means that they may index everything they find on your site or server unless you tell them explicitly not to.

There are several reasons for doing so. You may have pages with duplicate content, in which case it makes sense to tell the search engines what pages to index and what pages not to include in the search engine database.

There are two ways of telling the search engines where to (or not to) go:

(1) You can add a meta tag to each and every page explaining what to do, or you may (2) add a text file called robots.txt to the top directory of your site.

Our Search Engine Marketing 101 tutorial explains how to use the meta tags.

See our article on the robots.txt file to learn how to set up such a file.

The following table is adapted from the Live Search Blog, and includes the new standards. The rules described in the guides referred to above still work.
Click here to read the rest of this article!

Scour, social search with a twist

Scour is a brand new meta social search engine that encourages voting and commentary on each of its query results drawn from Google, Yahoo! And Live Search. And they will pay you to use their site.

You score points for searching, voting and commenting and if you invite your friends, Scour will add 20% of your friends’ points to your balance.

From my point of view, the money is the least interesting part. For one thing, in Scour’s own scenario, an average of 4.5 searches per day translates into 25 US dollars a year. So unless you invite lots of friends, you won’t get rich using Scour.

I’m more interested to see if the social search concept of Scour is an improvement on standard web search. If it is, I’ll return even if they don’t pay me. So how does it work and is it any good?

Click here to read the rest of this article!

No means no, Yahoo! says to Microsoft

The never ending Microsoft courtship has led to another loud “No!” from the prospective bride Yahoo!

Microsoft officially gave up its quest for Yahoo! weeks ago. Disgruntled Yahoo! stock owners, seeing their potential wind fall disappear, then made heroic efforts to force CEO Yang & Co to agree to a buyout.

The Icahn rebellion

Microsoft responded by saying to stock owner and rebel Carl Icahn that they might consider a new offer if he could get rid of the present Yahoo! board and get a more sensible one. (We are quoting freely here. They did not actually say “a more sensible one”, although that is clearly what was implied).

Icahn wants to replace Yahoo’s board with nine of his own choosing, himself becoming the director of the board at Yahoo shareholder meeting next month.

The Yahoo! counterattack

Yahoo board chairman Roy Bostock calls this and “an odd and opportunistic alliance of Microsoft and Carl Icahn has anything but the interests of Yahoo’s stockholders in mind,”

“While this type of erratic and unpredictable behavior is consistent with what we have come to expect from Microsoft, we will not be bludgeoned into a transaction that is not in the best interests of our stockholders.”

A Microsoft/Yahoo merger makes sense, doesn’t it?

Some may find Yahoo’s vehement refusal to consider a marriage between Microsoft and Yahoo! odd. After all, it seems like a match made in heaven. Here you have the financial clout of Microsoft coupled with the relatively popular search tool of Yahoo and Yahoo’s content rich portal features. And Yahoo! definitely needs the capital.

The fate of the board

Cynics may say that this is a board and a directorship desperately struggling to keep their positions. Indeed, the board argues that:
Click here to read the rest of this article!

5 basic tactics for improving your search engine rankings

If you have done nothing to improve the search engine rankings of your web pages, here’s some advice that gives results.

The web is full of sound (and unsound) advice on how you can improve your search engine rankings and increase your online visibility. There is a whole industry out there catering to companies that try to get more customers and more sales online.

But what if you can’t afford that? You don’t have the money and you don’t have the time to read books on search engine optimization, attend search engine marketing conferences of hire search marketers.

Actually, unless your site or blog is targeting a very competitive area, you can achieve a lot search engine wise by applying the following basic tactics.

Click here to read the rest of this article!

Clip, save and search text and images with Evernote

Evernote is a cool tool has the potential to become my outsourced brain, a place to store anything I want to remember, tag it, annotate it and make it easy to find when I need it.

I have been testing Evernote for about a month now. This week it is available to anyone in open Beta.

Evernote allows you to capture information in any environment using whatever device you prefer, and makes this information accessible and searchable at any time, from anywhere. Here’s an intro to some of the features and my verdict:

Click here to read the rest of this article!

More social search from Sproose

Sproose has launched apps for Facebook and Twitter that will let you benefit from the collective knowledge of your network of friends. The Sproose home page will be re-designed too.

What is Sproose?

Sproose is a user powered search engine that allows you to contribute to the ranking of web pages by voting for pages you find useful. Sproose also lets you browse pages that have been voted for by other users. This way you can discover new and interesting pages in a social network environment.

Sproose is not an entirely human powered search engine, trying to build an entire index by the contributions of users. Rather, it offers human improved search results. We asked Bob Pack, CEO at Sproose, how this works: “Sproose licenses the search results from Yahoo, MSN and Ask then de-duplicates them to bring about a quality index. Sproose also spiders the web with it’s own crawler technology, using the open source crawler ‘Nutch” adding these results as well.”

Then what is the role of the users? Pack explicate: “We allow the users to vote up their favorites, improving the ranking. For example, on Google, a search for music lists Yahoo as the top search result because it has the most in-bound links. On Sproose, Pandora is number one in the results because our users have voted it number one.”

Click here to read the rest of this article!

Microsoft to put up European search center

Microsoft has announced that it will establish a new European search technology center, but Norway is not listed as one of the options.

Yahoo! is not the only company stumbling around as a confused and dazed kid in the search engine store.

Microsoft is doing its best to catch up with Google in the search engine technology space: The Live Search team has definitely mad progress, and Live Search is more than a decent search tool, but it has still not found its own identity (or a brand name that communicates, for that matter).

Now Microsoft is looking to Europe for help.

In a press release this week Microsoft announced that the company “plans to open a Search Technology Center (STC) in Europe in its fiscal year 2009. The new center will be designed to help accelerate Microsoft’s investments in Live Search and disrupt the search and advertising marketplace.”

The obvious place to put such a center would be Trondheim or Oslo, Norway, the homes of Fast Search and Transfer, the enterprise search company recently bought by Microsoft.

Click here to read the rest of this article!

Microsoft believes Google is a one hit wonder

Steve Ballmer of Mircrosoft talks about the fight with Google and lays out the company’s plans for the future of search.

In an interview with the Financial Times Microsoft CEO Steve Ballmer gives credit to Google for seeing the potential of search at an early stage:

Didn’t see the business model of search

“I do fault us for the speed with which we dove into search, primarily because we didn’t see the business model. And I give Google credit for innovating in the business model around search. They did a nice job on that, and that’s why they won.”

Windows is to blame

Ballmer admits that Google has won this battle, and points out that the focus on Windows is partly to blame:

“I think one of the mistakes we made, and I think we’ve said this before, is having a five-year gap between Windows releases did calcify our ability to react to anything, because there was a five-year window basically where a big part of our R&D resources were fairly locked in. It doesn’t mean everything should be a six-month cycle, I don’t believe in that, but we’ve got to have more flexibility in our R&D commitments than that.”

Three things to do

As for the future, it seems that Microsoft finally has a plan as regards how to catch up and even excel compared to Google:

Click here to read the rest of this article!

Search Twitter in real time

Summize is a search engine that searches Twitter in real time. At first I thought this was a really bad idea. Instant access to a lot of useless information. But Summize is actually a lot of fun and has some real life applications.

If you find Twitter useful, you will enjoy Summize. And I do: Twitter can be a source of data for the anthropology of the present moment. I sometimes lurk on Twitter’s public timeline, amazed to have a glimpse of the lives of people all around the globe and what they are doing right now. But in addition to being fascinating, it’s confusing and very, very fragmented.

With Summize, you can search for a topic like Firefox 3.0 or the UEFA EURO2008 football championship. There are plenty of buzz options here for anyone interested in anything: Politics, music, sports… And when there’s breaking news, this will be a great way to stay on top of it.

You can choose to show results in any language or to extract tweets in Arabic, Dutch, Danish, English, Finnish, French, German, Hungarian, Icelandic, Italian, Japanese, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, and Thai.

All search results are available via standard Atom and JSON feeds, for use in readers and mashups.

Summize Labs are working on prototypes that harness conversations within blogs and reviews.

Be greener with Google Transit

Google Transit is a tool based on Google Maps, which lets you plan trips using public transport. Now it’s even easier to do your part to prevent global warming: Google Transit is part of the latest upgrade of Google Maps for Mobile.

If you already use Google Maps for Mobile, you know that this little app makes it easy to find directions on the go.

If you don’t have Google Maps on your mobile but want to try it out, head over to the Google Maps for Mobile home page. It’s quick and easy to download and install. I was up and searching for directions in a couple of minutes.

Now, when you search for directions with Google Maps for Mobile, you get two tabs, one for driving directions and one labeled Transit. You choose a starting point and a destination, either by typing or by clicking the locations on the map.

Click here to read the rest of this article!

Google News adds related searches

You may have seen this already, but Google News has added a nice little feature that adds alternative search queries to your own.

You will find them near the bottom of the search result page.

A search for Norway, for instance, brings up the following relevant queries:

It is not a big deal, but one of those small details that make search life so much easier. Yahoo, Ask and others have had functionality like this for a long time and we have been missing it in Google.

By the way, Google News has also made a new version of its homepage for the iPhone and iPod Touch. All you have to do is to navigate to the Google News homepage. Google News will identify you as an iPhone user and deliver content with the relevant formatting.

Britannica follows in Wikipedia’s footsteps (sort of…)

Can it really be? Encyclopaedia Britannica will soon be launching a new initiative to promote greater participation by both expert contributors and readers.

Both groups will be invited to play a larger role in expanding, improving, and maintaining the information on the Web under the Encyclopaedia Britannica name. They will also be sharing content they create with other visitors.

This comes as a surprise after years of antagonism between Wikepedia and Britannica where the the main conflict has been on the subject of the value of the contributions of amateurs. But is this really such a quantum leap as Britannica will have us beleive?

Click here to read the rest of this article!

Google Friend Connect turns any website into a social website

Google Friend Connect lets webmasters add gadgets or applications that connect their sites to the social networks more easily.

Our readers might be getting a little tired of all our talk about the social web, but there is a fair argument for saying that the social aspect of the web has become an essential part of Internet searching and navigation during the last couple of years.

Google certainly thinks so, and is now facing the threat from the mighty Facebook with a new toolbox of its own.

Facebook has, as you probably know, added software plug-ins or applications that lets users add social interactivity to their Facebook home pages. These could, for instance, be quizzes, maps over countries you have visited etc.

Facebook is a closed system, though. An application developed for Facebook will not work on other social web sites.

Facebook apps for the rest of us

Google Friend Connect is an tool that lets web masters add functionality that will make their web sites more social, and that help visitors to add info that can be transferred to social web sites like Facebook and Orkut.

It is a preview release, meaning that only selected webmasters are allowed to make use of it at the present time.

Preprepared code can be used to help webmasters add

Click here to read the rest of this article!

Spying on your neighbor

Pandia takes a look at a people search portal that claims they can get you US public records for free.

There was a time when public records were hidden in dusty city hall basements behind cupboards filled with the tax returns from 1889 (or something like that).

If you really wanted to know what your neighbor was up to, you had to fill out a four page form (in triplicate, painstakingly typed out on a manual typewriter) and wait for three years for a bureaucrat to deny you access.

Open online public records

Or maybe it wasn’t all that bad.

Nevertheless, the advent of the Internet coupled with a policy of democratic transparency has made public info very easily available. Did you know that in Norway, you may now actually find out how much anyone in the county earned and paid in taxes last year by using an open website?

Ian Armstrong of SnopStation (beta) tells us that they have developed a portal that helps US Citizens find info from public records:

“SnoopStation is a free, comprehensive public-records ’search wizard’, so to speak, designed particularly for people who aren’t familiar with court records, PACER, etc.” he says.

You have to register to use it, but as soon as you have clicked on the relevant link in the confirmation email, you may start searching.

The site leads you through a step by step query process where you tell SnoopStation what kind of information you are looking for.

The process is a bit complex, that has to be said, and a bit confusing.