Google Indexing & Mapleleafweb/forums

[August1991]

This question arose on the French forum where I occasionally participate and I thought I'd do the same here.

On Mapleleafweb/forums, we have about 180,000 messages now posted.

A Google search on site:mapleleafweb.com/forums gives about 70,000 hits.

Is there some reason for this discrepancy? IOW, is there something about Mapleleafweb's server arrangement that makes google indexing less effective?

----

Incidentally, the top hits (presumably top in the sense of google's search algorithm) concern election predictions and living conditions in Alberta.

[Riverwind]

gives about 70,000 hits.

A hit is a page. There are more than one message on each page. 70,000 sounds a little high.

[Charles Anthony]

On top of what RWind noted, you can not associate the MLW server configuration to how Google indexes pages.

There are two other things.

1)

The way that a forum software operates is by calling entries (posts) from a database and a recompiling them on the fly for display on your browser. This display is influenced in part by your display preferences set in your user profile. The page you see is never stored on the forum server.

The forum software or server can not absolutely control how Google chooses to index the forum. Thus, when Google indexes a forum, it could conceivably index each thread with 5 posts per page or 25 posts per page or whatever it wants. It could also consider each thread as one continuous page if it wanted -- which it most likely does but nobody knows for sure.

2)

Search engines deliberately do not report what they index. If they did, it would be easier for people to figure out how the results are generated. Discrepancies and inconsistencies between numbers of results reported by Google for a particular search and the amount of results displayed have been noted in the past throughout the internet.

Try this: look up your site search results site:mapleleafweb.com/forums and click on the numbered page links -- below where it says << Goooooooooogle >> at the bottom. Eventually, you will get the following anti-robot page to stop you. Why? Search engines want to make it very difficult for anybody to deduct (or reverse-engineer so to speak) their algorithm. Google takes a risk by indexing any forum or any database-driven site because it is possible to create a robot that may run repeated searches in the attempts to crack the Google code. Google likely changes its algorithm periodically solely for the sake of preventing this. One of the side effects may be what we observe as inconsistencies in search results.

[geoffrey]

I think CA is Google, he knows all about it.

[Greg]

This question arose on the French forum where I occasionally participate and I thought I'd do the same here.

On Mapleleafweb/forums, we have about 180,000 messages now posted.

A Google search on site:mapleleafweb.com/forums gives about 70,000 hits.

Is there some reason for this discrepancy? IOW, is there something about Mapleleafweb's server arrangement that makes google indexing less effective?

----

There are a number of reasons for this discepancy:

1) While the MLW forums may have over 188452 posts, this does not equal 188452 separate pages. Forum postings are arranged into threads, and those threads are broken up into blocks of 10, 15, 100, etc, postings. According to the forum statistics the forum has over 7500 threads. The forum averages around 400 new postings a day, and 10 new forum topics/threads a day.

2) Google does not necessarily provide accurate information about how many pages are in their index. Just because the information in provided, doesn't mean it is accurate. These numbers can fluctuate greatly, particularly if Google makes any changes to the search algorithm,

Incidentally, the top hits (presumably top in the sense of google's search algorithm) concern election predictions and living conditions in Alberta.

Not for me, http://www.google.ca/search?hl=en&client=f...com&btnG=Search

results vary, as you can see,

Cheers,

Greg