This is G o o g l e's cache of http://urbansemiotic.com/2007/08/01/%20urban-semiotic-articles-deleted-from-google/ as retrieved on Feb 12, 2008 19:20:28 GMT.
G o o g l e's cache is the snapshot that we took of the page as we crawled the web.
The page may have changed since that time. Click here for the current page without highlighting.
This cached page may reference images which are no longer available. Click here for the cached text only.
To link to or bookmark this page, use the following url: http://www.google.com/search?q=cache:m1gflUFZ43AJ:urbansemiotic.com/2007/08/01/%2520urban-semiotic-articles-deleted-from-google/+%22Urban+Semiotic+Articles+Deleted+From+Google%3F%22&hl=en&ct=clnk&cd=2&gl=us&client=firefox-a


Google is neither affiliated with the authors of this page nor responsible for its content.
These search terms have been highlighted: urban semiotic articles deleted from google 

Urban Semiotic Articles Deleted From Google?

August 1, 2007 · 34 Comments

Over the past few days I have noticed something curious happening with this Urban Semiotic Blog and Google.

For some strange reason a lot of our articles are no longer being returned in a basic Google web search and that is killing our readership because Google usually sends a lot of traffic our way.

Those articles used to appear in Google — they still appear in Yahoo! and Microsoft Live Search as the top returns — but they have been removed from Google view. We’ll see if this article you’re reading right now ever appears in Google and remains there or if, it too, eventually mysteriously disappears from Google’s search returns.

Yesterday, I sent the following inquiry to WordPress.com support:

I am doing some searches on published articles on my blog and I am not finding them in a Google search. They used to appear there. They still appear in Yahoo! and MSFT searches.

Is WordPress.com in any way editing/blocking/content spidering by disallowing Google access to certain posts with tags/topics/categories that you — or AdSense — find objectionable?

The articles that are missing deal with death, murder, censorship, critical articles, killings and child abuse, etc. You can see some of the articles in the general Google tag indexing but the full articles themselves are missing and I’m wondering why.

I received a crisp reply from WordPress.com that they do not censor content from Google Searches.

I followed up with this reply:

I appreciate your response, however, my experience is proving very interesting in the ongoing discovery of my blog articles that have been deleted from only Google Search returns and it seems they are being removed because of specific title words and WordPress.com tags and categories.

While I realize there is no such thing as a coincidence — does your AdSense deal with Google allow them to disallow placing ads on content they find offensive for their advertising program? If so, is it then equally possible that your deal with Google AdSense and the special server Matt spoke about online, is silently and automatically blocking/blacklisting content for inclusion in Google search returns by first blocking it from having AdSense placed?

Is there any way to opt out of AdSense and then force Google to do a full, unbiased, non-advertising-based, reindexing of my blog?

I am working directly with Google to push back every single article that has been deleted from their system. I just hope it doesn’t all disappear again without notice. Articles critical of Bush and Iraq also seem to be disallowed by default as well as any articles that are critical of WordPress.

I was then asked by WordPress.com support to provide examples of what I was seeing.

I did and I was told by another person in support that what I was seeing was not what they were seeing.

So, for the record, here are the screenshots I took to document the Search Returns issue and I have since deleted Google Desktop Search from my computer in case that was causing strange returns. There was no change in the Google search returns with that local engine removed from my system.  I also did these search tests on other computers with the same results.

You may get a right return now from Google because over the past few days I have been slowly working with them to add back — a URL at a time — the articles that were removed to Google’s search returns.

In the first search example — “Scobleizer Found Dead in His Grave” — Google returned the following. Notice the strange article links to the WordPress.com support forum and tags/category listings on WordPress.com, but there is no longer any direct link to the article proper on the Urban Semiotic URL:

Here’s Yahoo’s return on the same search term and the first return links directly to my article proper:

Here’s the Microsoft return and the first return links directly to my article:

Strange, eh?

Let’s continue now with this search phrase — “Making Published Posts Private” — and again from Google we get links to the WordPress.com support page where the article is linked and the WordPress.com tag system:

Here’s Yahoo:

Here’s Microsoft:

Finally, here’s a search example for an article Winslie Gomez wrote for Urban Semiotic from a Google search on — “If War is Human” — and here’s the return and again the WordPress.com tag system is featured and a direct link to the article does not appear:

Here’s Yahoo:

Finally, Microsoft:

What do you make of these odd Google search returns?

Are you seeing the same thing on your end when try these searches or have these missing files finally been returned to the Google search now after I pressed them back into the Google pool?

This morning we suddenly have a slew of new comments from new readers again — always a sign of good Google indexing — so maybe something broken was fixed?

Perhaps something set askew was indeed set right?

Categories: Writing
Tagged: , , , , , , ,

34 responses so far ↓

  • Winslie Gomez // August 1, 2007 at 8:40 am

    David,
    This article appear as
    Jesus Found Dead in His Grave « David W. Boles’ Urban Semiotic ™My post today, “Jesus Found Dead in His Grave” was made private by someone other than me or my staff. Here is the article: …
    urbansemiotic.com/2007/02/25/jesus-found-dead-in-his-grave/ - 320k - 31 Jul 2007 -

    But you do appear under a wordpress tag
    http://wordpress.com/tag/grave/2/
    Jesus Found Dead in His Grave — 203 comments
    David W. Boles wrote 5 months ago: Big news was broken yesterday: Jesus and his family have been founded dead in their graves in Israel. What does … more »

    Not the original article!
    looks fishy
    winslie

  • David W. Boles // August 1, 2007 at 8:46 am

    Hi Winslie –

    Actually, in your example, that Jesus article is NOT WHAT I AM SEARCHING FOR, the search that is returning that wrong return is actually “Scobleizer Found Dead in His Grave” — the Scobleizer satire was based on, and links to, the Jesus piece but we should not see the Jesus article returned before the Scobleizer version.

    The links to the WordPress.com tag system are curious because the Google traffic is sent there instead of here to the original article – pre-assuming, of course, that the right article is being returned in the first place.

  • Winslie Gomez // August 1, 2007 at 8:56 am

    David,
    My “if war is human…” does not turn up at all even as a wordpress>>tag.

    I am doing my searches in google.co.uk

    Strange world we live in!
    winslie

  • Nicola // August 1, 2007 at 8:57 am

    I just tried the “Making published posts private” search ………. you appear at three after two references from the Wordpress forums …….

    Jesus found dead in his grave is also odd …….. ( Copy and paste of the result)

    Jesus Found Dead in His Grave « David W. Boles’ Urban Semiotic ™ Religion
    My post today, “Jesus Found Dead in His Grave” was made private by someone other than me or my staff. Here is the article: …
    urbansemiotic.com/2007/02/25/jesus-found-dead-in-his-grave/ - 320k - 31 Jul 2007 - Cached - Similar pages - Note this - Filter

    Odd that it quotes what I think is a WP forum post ……. but links to the original article.

    Scobleizer is odd too - here are the text results for that. It is number two AFTER Jesus found Dead in his grave.

    Jesus Found Dead in His Grave « David W. Boles’ Urban Semiotic ™ Religion - 13:47
    Scobleizer Found Dead in His Grave « David W. Boles’ Urban Semiotic ™ (13:45:04) :. […] you can see in the screenshot below my Jesus Found Dead in His Grave …
    urbansemiotic.com/2007/02/25/jesus-found-dead-in-his-grave/ - 321k - Cached - Similar pages - Note this - Filter
    Scobleizer Found Dead in His Grave « David W. Boles’ Urban Semiotic ™
    Scobleizer Found Dead in His Grave. 27 02 2007. Don’t let anyone tell you Jesus isn’t good for business! As you can see in the screenshot below my Jesus …
    urbansemiotic.com/2007/02/27/scobleizer-found-dead-in-his-grave/ - 71k - Cached - Similar pages - Note this - Filter
    [ More results from urbansemiotic.com ]

    “If War is Human” gets pole position on google and direct link.

    It smells fishy - looks fishy ……………………………

  • David W. Boles // August 1, 2007 at 9:02 am

    Winslie!

    Excellent! Thank you for continuing these searches! Keep reporting what you find!

    And the reason “Jesus Found Dead in His Grave” is being returned before “Scobleizer Found Dead in His Grave” is because the Scobleizer article was deleted from Google.

  • David W. Boles // August 1, 2007 at 9:05 am

    Nicola!

    Thank you for your excellent research and you are right — sometimes I post links to my articles here over in the WordPress.com forum — and you are seeing those links to my article, but the traffic is being directed to the WordPress.com support forum instead of here to the original article.

    It certainly is strange and curious and rather new and I went to the WordPress.com support forum today and posted a link to this article so others can test this out on their blogs by doing similar searches.

  • Winslie Gomez // August 1, 2007 at 9:11 am

    David
    I got my article “If war is Human….” by putting my name first:

    Sign in
    Google Web Images News MapsNew! Products Groups Scholar more »

    Web Results 1 - 10 of about 455 for winslie, if war is human. (0.06 seconds)

    Did you mean: winsley, if war is human

    Comments on: If War is Human Why Must We Forgive?http://urbansemiotic.com/2007/05/21/if-war-is-human-why-must-we-forgive/#comment-89761 Winslie Gomez Tue, 22 May 2007 01:41:37 +0000 …
    urbansemiotic.com/2007/05/21/if-war-is-human-why-must-we-forgive/feed/ - 33k - Cached - Similar pages

    winslie

  • David W. Boles // August 1, 2007 at 9:12 am

    It’s interesting that the returns, using the same search phrase on all search engines, returns two identical returns and one not so much.

    Keep testing!

  • Winslie Gomez // August 1, 2007 at 9:16 am

    David
    The Scobleizer search gives 3 returns:

    Sign in
    Google Web Images News MapsNew! Products Groups Scholar more »

    Advanced Search
    Preferences
    Search: the web pages from the UK

    New! View and manage your web history
    Web Results 1 - 3 of 3 for scobilizer found dead in his grave. (0.52 seconds)

    Scobleizer Found Dead in His Grave « David W. Boles’ Urban Semiotic ™As you can see in the screenshot below my Jesus Found Dead in His Grave post is …. upon news of Scobilizer Spotted With Antonella Barba And James Cameron! …
    urbansemiotic.com/2007/02/27/scobleizer-found-dead-in-his-grave/ - 71k - Cached - Similar pages

    Comments on: Scobleizer Found Dead in His GraveJust to let you know - the “Jesus found dead in his grave post” is being …. Crash upon news of Scobilizer Spotted With Antonella Barba And James Cameron! …
    urbansemiotic.com/2007/02/27/scobleizer-found-dead-in-his-grave/feed/ - 29k - Cached - Similar pages

    Announcement, I’m going to Google « ScobleizerMicrosoft Office is dead and I wanted to be at the company that is the future of everything. ….. Pong to Scoblizer, pong to Scoblizer, come in Scobilizer. …
    scobleizer.com/2006/04/01/announcement-im-going-to-google/ - 97k - Cached - Similar pages

    winslie

  • David W. Boles // August 1, 2007 at 9:32 am

    Very interesting returns, Winslie, thanks!

    Some of this might be resolved as my forced Google updates by individual article URL start to propagate across the Google servers again. We shall see if they remain! :grin:

  • Lloyd Budd // August 1, 2007 at 10:02 am

    Because of the severity of the issue, WordPress.com staff Barry and I immediately looked to see if we could discover any issues and searched for any similar reports. My later response focused on recommending seeking assistance directly from Google.

    Including “as well as any articles that are critical of WordPress” seems like stretching your analysis, and being a little sensationalist.

    My questions remain, What do you mean with the help of Google? Do they know what happened? Google most often silently fixes things. I wonder what the pattern really is if anything.

    Fantastic that you so quickly discovered it and get it corrected!

  • that girl again // August 1, 2007 at 10:11 am

    Very weird indeed. Is Scoble aware of the issue? My initial thought was that he or one of his fanboys might have complained about the article to their Google friends and got them to censor it. Which is only a WP issue if the fanboy happens to be with Automattic…

  • David W. Boles // August 1, 2007 at 10:28 am

    Lloyd –

    Yes, it was your non-belief in my complaint yesterday that struck me to write this article and to provide living proof of my argument I tried to quietly and reasonably resolve with Support yesterday.

    The two examples I provide in my article here are WordPress-critical articles.

    As you also may remember, my WordPress Super bug article was also mysteriously made Private by the WordPress.com system as I have mentioned before here and to support.

    I’m not going to reveal my private conversations with Google on this matter, Lloyd.

  • David W. Boles // August 1, 2007 at 10:34 am

    Hi that girl and thanks for thinking about this matter here!

    I have no idea who you are — you could be Matt’s girlfriend for all we know :grin: — but your blog is always a prescient and telling read.

    The missing articles that were deleted from Google were also about unsavory topics as I mentioned in the article: Murder, Child Abuse, Iraq, Death, Killing — and that’s what made me wonder if there was, perhaps, some kind of parsing going on somewhere that was blocking those topics/keywords/tags/categories and AdSense immediately came to mind as the possible content killer.

    The missing articles definitely share a pattern and, what I consider as a writer, a dedicated editorial bent in their removal.

    I know AdSense doesn’t want to place ads that are generally unsavory — and the Google system marks those articles as “not AdSense appropriate” and that’s why I wrote to Support to ask for a clarification on how AdSense is set to run on WordPress.com blogs and if we could opt-out of it.

  • Lloyd Budd // August 1, 2007 at 11:05 am

    Your “WordPress Super bug” was that articles would be marked private because of a bug in Firefox, so there really is no mystery there.

    I really don’t see how we could or can help further, if you don’t share the details of the problem, which as you describe is specific to Google.

    Your conspiracy theory is insulting.

  • David W. Boles // August 1, 2007 at 11:12 am

    Lloyd –

    You seem to enjoy your role as the designated staff goon who goes to other WordPress.com blogs to crush dissent and cast about doubt. You’ve done it here and you’ve done it on Wank’s blog. Where else do you bully around in comments?

    I have no idea if the public/private bug was resolved or not because my trouble ticket is still open.

    You asked for proof. I gave it to you here in the article. I have other readers in the UK who confirm what I have been arguing right here in the comments if you bothered to read them.

    I never claimed any conspiracy. You’re the first and only one in this thread to use that word. What’s your point? Why are you here?

  • David W. Boles // August 1, 2007 at 11:17 am

    UPDATE:

    I’m trying to export my blog to back it up and — not to be a “conspiracy theorist” — but the system keeps giving me a 503 error:

    Goshdarnit!

    Something has gone wrong with our servers. It’s probably Matt’s fault.

    We’ve just been notified of the problem.

    Hopefully this should be fixed ASAP, so kindly reload in a minute and things should be back to normal.

  • David W. Boles // August 1, 2007 at 11:42 am

    Still can’t download my backup file for this blog.

  • Lloyd Budd // August 1, 2007 at 12:10 pm

    We are investigating the backup issue. I will email you a backup directly, it seems that your blog is too large for the memory restrictions on the public facing servers.

    I am very sorry if you feel that way. It is difficult for me not to get upset with what I see as negativism. I still feel that your article suggests that we have some malicious part in the very unfortunate problem you are experiencing. I stand by my recommendation of working directly with Google in this issue, and I’m happy to hear that you are making progress.

    Regarding your “WordPress Super bug”, I see now that all of the loops were not closed. Thank you for pointing this out.

  • David W. Boles // August 1, 2007 at 12:17 pm

    Dear Lloyd –

    Thank you for your reply.

    You don’t have to email me a backup. I am usually able to pull down a backup just fine. I’ll keep working on it.

    I was asking a question and investigating a wondering based on third-party facts and searches. I’m sorry you believe that questioning process was negative. I was very careful in my word choice throughout the article and exchanges with support and commenters here.

    If you closely read what I’ve said, I believe this is more a Google problem than a WP.com problem but, I also suspect, the issue is being caused by a WP.com and Google interaction. Somewhere.

    Thanks for the Super Bug confirmation. The last thing I was told is that you all had no idea why it was happening, it was not reproducible and that it was happening on different blogs other than mine. The recommendation was for me to download my entire blog each week, do a raw HTML search for the special code you provided in the comments on that Super Bug article. That is, and has been, a long and involved process which I just now discovered today is a Firefox problem and not a WordPress problem.

  • Lloyd Budd // August 1, 2007 at 12:43 pm

    Sorry, for my misinterpretation. It will be interesting if more light is ever shined on it, but this sort of knowledge almost always stays locked in the Goog.

    Regarding the backup, it is possibly that you are have some how now hit the ceiling of the public facing servers. Let us know if it doesn’t resolve itself.

    Super Bug confirmation: we really screwed up on not closing the loop with you. We have long worked around the Firefox bug. The core WordPress trac ticket that Mike posted to your article on this issue includes the details of the Firefox issue and how we worked around it.

  • David W. Boles // August 1, 2007 at 12:46 pm

    Thanks for the update, Lloyd!

    Since I had an open ticket in the Support system I didn’t think about looking elsewhere for the answer to the Super Bug.

    I will let you know if I am unable to pull a backup of my blog. That’s a regular routine of mine now.

  • Lloyd Budd // August 1, 2007 at 1:14 pm

    “Since I had an open ticket in the Support system”, nor should you have had to :( Again, I’m very sorry.

    Hope this Google issue is soon a bad memory.

  • David W. Boles // August 1, 2007 at 1:16 pm

    No problem, Lloyd! I understand! We’re all learning as we fight our way into the light through the darkness.

    I, too, hope the Google thing is resolved soon, especially since I’m finishing up writing a book on Google Apps! :grin:

    https://bolesbooks.com/thomson

  • Kathakali Chatterjee // August 1, 2007 at 6:18 pm

    Hi David,

    Does it have anything to do with your “google custom search business edition”?

    I had a feeling that it was not as effective recently as it was for the first few days - but I am not sure.

  • David W. Boles // August 1, 2007 at 6:38 pm

    Katha!

    The Google Custom Search Business Edition:

    http://bolesuniversity.com/search.html

    … is what clued me in to something being wrong! There were articles that used to be there and suddenly they were missing. I then expanded my search for the missing articles to Google proper and found what I found as expressed in today’s article.

  • Kathakali Chatterjee // August 1, 2007 at 6:46 pm

    Yes!!!

    I was about to e mail you!!!

    Last weekend I was working on something and as usual searching for our “old gold” and the return result was extremely disappointing!!!

    If I typed “music” earlier there would have been “n” number of returns including all comments made here - but it was missing!!!

    I should have let you know! Duh!

  • David W. Boles // August 1, 2007 at 6:58 pm

    Katha!

    Yes, let me know those kinds of things. The initial testing was spectacular and then it suddenly stopped providing the right returns. That is incredibly disturbing and I thank you so much for the confirmation that something is up!

  • David W. Boles // August 1, 2007 at 6:59 pm

    ARE YOU LOGGED IN, KATHA?!! :mrgreen:

  • Kathakali Chatterjee // August 1, 2007 at 7:01 pm

    Ugh!!!

    I need a hammer right in my head!!!

  • David W. Boles // August 1, 2007 at 7:03 pm

    Ha! I love it when you’re logged in… :lol:

  • Lloyd Budd // August 2, 2007 at 2:02 pm

    I’m starting to see news reports about Google having changed their algorithms last week.
    Articles like Answers.com Sees 28% Traffic Drop As Google Algortihm Changes; Will Dictionary.com Deal Go Through?

  • David W. Boles // August 2, 2007 at 2:13 pm

    Hi Lloyd –

    An algorithm change wouldn’t result in the deletion of articles from the Google search returns, would it? I can understand some results might move around, but a top search return doesn’t just get deleted, does it?

    Usually when Google makes changes like that this blog does really well because we have long articles, lots of comments and we don’t have any Spam or advertising on the site and we’ve been around a long while.

    The PageRank of this blog has been a steady 5 for a time now and as far as I can see that number is still active.

  • David W. Boles // August 4, 2007 at 10:38 am

    As of yesterday, this problem seems to have cleared up as mysteriously as it occurred and all the missing articles have been restored.

    Somewhere, a switch was flipped.

    Somewhere, the other shoe dropped.

    As the genius songwriter Neil Young said about the creative process and working with others, “Sometimes things fall apart as fast as they came together.”

    I’m pleased to learn sometimes the opposite is true.

You must log in to post a comment.