Search engines team up for site crawling

Google, MSN and Yahoo adopt standard to speed up search engine indexing

Written by Shaun Nichols

Yahoo and Microsoft have partnered with Google to support the Sitemaps 0.90 protocol .

Sitemaps allow webmasters to craft an XML file containing information about recently updated pages. When a search engine's indexing program spiders the site, the Sitemap file informs it which pages require indexing.

The protocol was introduced in June 2005 by Google, which has published the technology under a Creative Commons licence .

The indexing system will now be used by Windows Live Search and Yahoo Search in addition to Google Search.

"The launch of Sitemaps is significant because it allows a single, easy way for websites to provide content and metadata to search engines," said Tim Mayer, senior director of product management at Yahoo Search.

Google said that the Sitemap system saves bandwidth for site owners by letting spiders request fewer pages. It also streamlines the indexing process, making updated content available on the search engines sooner.

The three companies have launched a website that explains the Sitemaps protocol and how to generate the Sitemap XML files.

Webmasters can either hand-code the file themselves or use a Sitemap-generator from either Google or a third-party developer .

Tags:

reader comments

related articles

 

New Internet Explorer 8 beta released

Updated test version includes new privacy modes and interactive elements 28 Aug 2008

Web search engines slow on the update

Study finds shortcomings in search engine data management 03 Mar 2008

Google’s Android basks in Mobile World Congress spotlight

Search deals were made, and ended, in Barcelona while vendors showed off new mobile tech 14 Feb 2008

related whitepapers

today's top stories

Technology and privacy

Watch part one of a two-part Computing roundtable debate on the importance of putting data privacy issues at the heart of your IT plans 01 Dec 2008

Got the Knowledge?

Last week the civil service published a new strategy to help government seize the opportunities and meet the challenges of managing knowledge... 01 Dec 2008

Q&A - ntl:Telewest Business managing director Stephen Beynon

The cable provider's chief talks about the future of next-generation broadband access in the UK 28 Nov 2008

Computing podcast - Standard Life's offshoring plans; and the prospects for government IT

The insurance giant outlines its new outsourcing strategy; and we ask if the government's economic bailout will affect its IT plans 28 Nov 2008

India will remain open for business - but that's not the real story

One of the duties I have to fulfil as a director of the National Outsourcing Association is to talk to the media... 28 Nov 2008

Advertisement

Newsletter signup

Sign up for our range of FREE newsletters:

Existing User

Newsletter user login:

Advertisement

Jobs

Related jobs

Job of the week

Job alerts

Sign up here

Find your next job

IT Salary Checker

Check salary here

Advertisement

White papers

Search white papers

Top categories

VPN, Extranet and Intranet Solutions

WAN/ LAN Solutions

Network Security

Interoperability-Connectivity

Grid/ Utility Computing

Latest poll

Will the terrorist attacks in Mumbai affect your offshoring plans?

Will the terrorist attacks in Mumbai affect your offshoring plans?

Is India becoming a risky destination?

Previous poll results

Latest audio and video articles

Padlocked CDVideo

Technology and privacy

Watch part one of a two-part Computing roundtable debate on the importance of putting data privacy issues at the heart of your IT plans 01 Dec 2008

Podcast imageAudio

Computing podcast - Standard Life's offshoring plans; and the prospects for government IT

The insurance giant outlines its new outsourcing strategy; and we ask if the government's economic bailout will affect its IT plans 28 Nov 2008

Latest in-depth articles

ntl:Telewest's Stephen BeynonAnalysis

Q&A - ntl:Telewest Business managing director Stephen Beynon

The cable provider's chief talks about the future of next-generation broadband access in the UK 28 Nov 2008

cowboyFeatures

Guns for hire

David Neal explores the world of interim CIOs and discovers why more firms are turning to them to spur on IT-led change 27 Nov 2008

Advertisement

Primary Navigation