| Support the "STOP THE PLEDGE" Campaign and support historical and educational research about the Pledge, its author and his ilk. |
| If you would like to own the historic collectible
out-of-print book "Twenty-Three Words" by Margarette S. Miller, one is
available here for $100. The book is in mint condition, never used.
This may be the only opportunity to obtain this eye-popping book
Just use the donation button below at left to purchase the book and communicate
with this website at lawyer AT RexCurry.net |
Abstract: This report describes the system by which results in the Google search engine are suppressed.
Google is arguably the world's most popular search engine. However, contrary perhaps to a naive impression, in some cases the results of a search are affected by various government-related factors. That is, search results which may otherwise be shown, are deliberately excluded. The suppression may be local to a country, or global to all Google results.
This removal of results was first documented in a report Localized Google search result exclusions by Benjamin Edelman and Jonathan Zittrain , which investigated certain web material banned in various countries. Later, this author Seth Finkelstein discussed a global removal arising from intimidation generated from the United Kingdom town of Chester, in Chester's Guide to Molesting Google .
My discussion here is not meant to criticize Google's behavior in any way. Much of it is in reaction to government law or government-backed pressure, where accommodation is an understandable reaction if nothing else. Rather, documenting and explaining what happens, can inform public understanding, and lead to more informed resistance against the distortion of search results created by censorship campaigns.
A Google search is not simply a raw dump of a database query to the user's screen. The retrieval of the data is just one step. There is much post-processing afterwards, in terms of presentation and customization.
When Google "removes" material, often it is still in the Google index itself. But the post-processing has removed it from any results shown to the user. This system can be applied, for quality reasons, to remove sites which "spam" the search engine. And that is, by volume, certainly the overwhelming application of the mechanism. But it can also be directed against sites which have been prohibited for government-based reasons.
Sometimes the fact that the "removed" material is still in the index can be inferred.
For the case of Chester , which concerned a single "removed" page, the internal indexing of the target page could be established by comparison with a search for the same material on another search engine.
Consider a Google search for the word "lesbian" on the site torkyarkisto.marhost.com . It returns a page titled "The Kurt Cobain Quiz", with a count of
Results 1 - 1 of about 2
The "about" qualifier there represents many factors, but sometimes encompasses blacklisted pages. This can be seen here by comparing to an AltaVista search for the word "lesbian" on the site torkyarkisto.marhost.com
There are two pages visible in that case, the "Quiz" page, and the "Chester" page which caused all the trouble in the first place.
Since we know the "Chester" page was once in the Google index, it must be the other page referred to in "about 2". QED.
In this situation, comparing results from the different Country Google searches, is often revealing. The tests are often best done using the "allinurl:" syntax of Google, which searches for URLs which have the given components (note the separate components can appear anywhere in the URL, so "allinurl:stormfront.org" is "stormfront" and "org" in the URL, not just the string "stormfront.org" as might be naively thought). Stormfront.org is a notorious racist site, often banned in various contexts.
Consider the following US search:
http://www.google.com/search?num=100&hl=en&q=allinurl%3Astormfront.org
This returned: Results 1 - 27 of about 50,700.
Now compare with the German counterpart (Google.DE):
http://www.google.de/search?num=100&hl=en&q=allinurl%3Astormfront.org
This returned: Results 1 - 9 about 50,700.
Immediate observation: The rightmost (total) number is identical. So identical results are in the Google database. It's simply not displaying them. How is it determining which domain results to display?
Note the hosts of which "stormfront.org" URLs are visible on the German page:
irc.stormfront.org:8000/
www4.stormfront.org:81/
lists.stormfront.org:81/
What do these all have in common?
They all have a port number after the host name.
The exclusion pattern obviously isn't matching the ":number" part of the
URL.
It's matching a pattern of "*.stormfront.org/" in the host, as in the
following which are displayed the US search, but not the German search.
www.stormfront.org/
kids.stormfront.org/
women.stormfront.org/
nna.stormfront.org/
www4.stormfront.org/
Even more interesting, the German page has a broken URL listed at the bottom: http/www.stormfront.org/quotes.htm . That's not a valid URL, so it seems to escape the host check.
Thus, the suppression again appears to be implemented as a post-processing step using very simple patterns of prohibited results.
The same behavior is observed in a German "stormfront.org" images
search
This returned: Results 1 - 6 about 1,410.
Versus a US "stormfront.org" images search
This returned: Results 1 - 18 about 1,410.
(note identical right-hand numbers, and hosts matching "*.stormfront.org/"
pattern are suppressed in the German results)
And also in a German "stormfront.org" directory
search
This returned: Results 1 - 8 about 15.
Versus a US "stormfront.org" directory
search
This returned: Results 1 - 10 about 15.
(note again identical right-hand numbers, and hosts matching "*.stormfront.org/"
pattern are suppressed in the German results)
Contrary to earlier utopian theories of the Internet, it takes very little effort for governments to cause certain information simply to vanish for a huge number of people.
Version 1.0 Mar 10 2003
This work was not funded by anyone, and has no connection to any organization. In fact, if anyone is providing financial support for such projects, the author would like to know.
Note: Some of this material appeared earlier in the author's Infothought blog
(if you subscribed a few months ago, please resubscribe due to a crash)
See more of Seth Finkelstein 's Censorware Investigations
| The court case of Frank Herbert Wonschik v.
U.S., argued that the jury selection process was impermissibly tainted
by the trial judge's request that all potential jurors stand and recite
the Pledge of Allegiance prior to jury selection. Furthermore, that bias
also transgressed the Establishment Clause and the Free Exercise Clause
of the First Amendment to the U.S. Constitution. |
|
