The launch of Wolfram Alpha this month has some wondering if we’re moving into a new age of search. We certainly could be, but it’s unlikely it’ll be a Google-free world.
Indeed, this latest development – as cool as it is – is a continuation of a long string of innovation the sector has seen since the birth of the web. Wolfram Alpha is likely to fill one niche area of the market, while other innovations step-up to solve other problems. That may still leave Google holding the bulk of the market, but at the very least it’s encouraging that so many are continuing to develop new ideas despite such a near monopoly.
How search works
Every search engine is a bit different, much to the annoyance of search engine optimisers the world over. That said, search engines basically send out a robot or spider across the web to follow each and every link. The service then indexes what it finds, with different engines storing different bits – for example, Google, stores everything found in the source of the page, while others just look at what’s displayed.
When a user searches all these indexed pages, relevancy comes into play. Google’s PageRank is the most famous algorithm, but every engine has its own way of finding the best results.
But it’s not just the web that’s searched. Wolfram Alpha is promising to return answers, not just documents, from across the web, while many other searches look at more than the web or even limit themselves to specific niche areas of it. Consider the Pirate Bay. The now-notorious site offers a search of BitTorrents, specific file types that are key to its users. Or consider Google Maps – it is searching for anything that can be geographically pin-pointed. Even services like Twitter are changing the way search works, by letting us search what people are tweeting about.
Before the web
Quite simply, the world didn’t need search in the early days. In the beginning, all web servers were listed on a CERN website that was edited by a man then known as Tim Berners-Lee – those were the days before he was knighted.
Once that list became unwieldy, a search tool dubbed Archie – that’s ‘Archive’ without the ‘v’ – searched via a database of web servers. Soon after, Gopher’s rise led to a pair of new search tools, comically dubbed Veronica and Jughead, which searched using file names and menu titles.
It wasn’t until 1993 that the first robot came about. It was called the World Wide Web Wanderer, but it was for measuring the web, not searching it.
Modern search kicked off in December of that year with JumpStation, which used a robot to crawl the web, indexing it to make it searchable – the three key aspects of modern, or at least current, search. JumpStation was limited to titles, but another system called WebCrawler took it a step further the next year, managing to search full text.
Going commercial
Lycos kicked off the money making in 1994. The Carnegie Mellon project not only robotically indexed every word on a page for searching, but it was also used by the public and went commercial.
But it had competition. Among the pack that emerged over the next few years was Excite, Magellan and Infoseek, in addition to Altavista and Yahoo. Perhaps surprisingly now, at the time Yahoo didn’t search via full pages and keywords, but instead used a web directory system.
By 1996, dominant browser Netscape was struggling to keep things fair, so for a fee of $5 million it let search engines buy the search spot on the Netscape page in rotation.
But the search engine market was set for a shakeup. In that same year, Larry Page and Sergey Brin teamed up at Standford University to develop a search engine based on relevancy, initially dubbing it BackRub. Two years later, Google was incorporated as a company with investment of $1 million; a year later, they had $25 million to play with.
Google time
In 2000, the market as we know it now started to take shape. The dot com bust took down some, but Google’s PageRank bumped it into the limelight, and it started offering advertising that year.
Still, competition remained. In 2002, Yahoo upped its game by picking up proper search tech in the form of Linktomi in and later Overture, which owned Altavista. Yahoo even used Google’s search until 2004, when it melded its acquisitions together into a coherent search tool. That year also saw Microsoft enter the ring, creating its own tool in 2004.
But Google’s IPO in 2004 showed who was really in control of the market, putting the net worth of the company at about $23 billion. Now, it gains several hundred million queries a day. That said, Google has been hit by privacy and other controversies, and notably misses searching the “deep web” – all the data that isn’t linked the way Google likes it.
Challenging Google’s dominance
While Google has seen a few challenges to its market dominance, although none have yet to make much of a dent.
Last year, an engine named Cuil launched, to much media coverage. While the company itself didn’t use the phrase “Google Killer,” a lot of journalists and bloggers did. And it sure didn’t look good for Cuil – or any other Google challengers – when the service was derided for poor results and performance.
Last year, Cuil co-founder Tom Costello told IT PRO: “Other people tend to want to actually make it a story about ‘oh, here’s David coming along against Goliath.’ And again, I think that with the story of David and Goliath, David doesn’t always win on the first shot. A lot of time, people want a very quick resolution to these things.”
Even though Cuil continued to struggle to gain market share in the following months, Costello thinks the service has a future. “Even when you see Google – built to the kind of dominant position they have - it took them 10 years to build to that position. Search is not a business where you have overnight success. It’s not a business where you change the behaviour of billions of people worldwide who use search overnight.”
Maybe the challenge won’t come from a new service, but from established players instead. That’s certainly the goal Microsoft had in mind when it made a massive bid for Yahoo last year. While the bid failed, Microsoft has continued to try to partner with Yahoo on search.
At the moment, Microsoft’s third and fourth ranked Live and MSN searches pull in about five per cent of the market, with second-ranked Yahoo adding another 10 per cent, according to Net Applications. Google takes about 81 per cent, though, meaning it has a lot of room for error to stave off any challenges, even from big players.
While a straight-up deal between Yahoo and Microsoft could cause some disruption in the online ad market, it’s going to take something new to topple Google from its throne. Could pairing a big name with a new idea be enough? If Wikia Search is anything to go by, then the answer is no.
That project was backed by Wikipedia founder Jimmy Wales, so it had an attention-pulling big name from the start. It was never intended to overtake Google – Wales said he hoped one day to get five per cent of the market – but to offer an open, transparent search alternative. Wales said at the time: “The idea that Google has some edge because they've got super-duper rocket scientists may be a little antiquated now.”
Instead, Wikia Search looked to crowdsource its relevancy, asking users to rank the results of a search, rather than rely on mysterious algorithms like Google. But the service never took off, and as the recession hit, Wales shut it down, noting it was pulling just 10,000 unique users a month.
What’s next?
While Wales has promised to return to search as soon as finances allow, others are looking for new ways to find information on the web. Sir Tim Berners Lee is still at it, pointing to the semantic web – and in turn search – as the future. This looks to remove ambiguous words from the search process, making results more specific.
Microsoft is also set to be launching its new search tool as early as June. Codenamed “Kumo” at the moment, it looks to be a rebrand of its current Live search, but includes a few changes such as using categories to separate out different types of results.
And then there’s Wolfram Alpha, the engine that promises to return answers, not documents. While it certainly looks impressive, it’s so unlike what people currently use search for that it’s unlikely to be the force that topples Google.
But then, in such a fast moving area of the tech world, who knows when the next Google could be created in a university somewhere around the world…
Source Content : www.itpro.co.uk
