Back in late March you may have noticed that this blog was shut down. In fact the entire Softduit site was down for several hours.
We never entirely isolated the issue, but basically there were 2 things going on that could be identified:

1. Wordpress 2.1, the software that runs this Maven Mapper’s Information, was leaking bandwidth severely - leaving a call open when something created ‘persistent connections’. Persistent Connections are not always bad, but they are dangerous when not used correctly.
2. I had a meta tag plugin (Autometa) that was creating an error that was not visible on the blog but creating significant havoc with my host.

When this happened, I was a bit frantic. I didn’t know a persistent connectin from a hole in my head. My host shut my site down, much to my dismay. Thinking that there might be something like a denial of service attack going on, but there was not.

Right away, I got rid of the Autometa (bye bye). It may or may not have been a big portion of the problem, but for troubleshooting reasons it was in the way and had to go.

The thing is that persistent connections are not always bad, but it was a bit beyond me to identify the good from the bad in this case.

Here is some of the information they gave me:

System administration has noted that your account appears to be using persistent connections in some of the software used on your site. Persistent connections are by and large not necessary for most software to function, and they can cause issues with your account.

To explain a bit further, persistent connections are one method PHP scripts may use to open a connection to a MySql database. Using persistent connecting is only useful in an environment with a high overhead in connecting to the MySql database itself - in your case the connection (and ‘cost’ in resources) is negligible compared to using persistent connections as all of your queries are executed immediately.

The persistent connections in this case are simply sitting idle consuming memory and may bring your site close to the predefined limits set for accounts in the shared hosting environment. If you have a database intensive site, this could make the site appear sluggish or appear to be down while the initial queries time out (this can take up to 300 seconds, depending on several factors).

You will need to review your code and see where these persistent connections are coming from, as spikes in traffic could cause your site to appear to be unavailable to your visitors.

Plus

I had to suspend your site due to the very high load that it was taking on. There were multiple IP addresses connecting to it over and over again. Some of the IP’s had over 80 connections each.

Here is what I learned.

1. After you install a plugin, check your error logs. Just because it seems to be working doesn’t mean that it really is.
2. The jury is still out on some things about Wordpress 2.1 that haven’t been documented well by the community yet, so pay attention to your bandwidth and be careful.

Note for all people new to hosting their blog on their own domain. - Be Careful!

It is likely that you have a hosting plan that allows a certain amount of bandwidth per month. If you go over that bandwidth a couple things are likely to happen:

1. Your host might shut your site down.
2. You might get an overage charge (like going over your minutes on your cell phone bill - very very scary if it happens and you are not prepared)

We are all used to getting typically hundreds to thousands of hits per day, but if your site goes into the millions of hits per day or hour you better make sure you have a good solid way of monetizing that traffic or else you are going to be in trouble.

Adsense May Not solve the Problem

You might think that Adsense will earn a tremendous amount if you see a spike in traffic (ergo you write the golden article that captures the attention of the world and that CPM number makes you rich!)

Wrong!

Google will sometimesshut down your Adsense account because they think there might be fraud going on (Google is like that ~unpredictable) . They like to see nice constant (expected/predicted) growth in your traffic. If you jump from 100 hits per day to several million hits in an hour (and your server doesn’t crash), Google may nix your account and not pay you your earnings blaming it on fraud.

Now you have a bandwidth issue, commonly referred to as a Big Fat Bill and you have no Adsense revenue and no Adsense Account. Its kind of like getting your right hand chopped off, having salt rubbed in the wound while someone slaps you in the face repeatedly with your own hand.

Page Popularity for Site: 27% [?]

For about $6 million Earthlink will deploy a wireless network in San Francisco after coming to terms with the City of San Francisco.  Google is also involved.  Google will provide free access to the network at low transmission speeds.  Earthlink and others will charge a monthly fee of $21.95 and $12.95 for about 3,200 low-income residents, which begs a tangential question, “Are there only 3,200 low income residents left in San Francisco?”

The network will be privately owned by Earthlink, which was a major source of contention.  Many were concerned that the city should own the network and had other fears of privacy protection from Earthlink and Google, who will now provide additional privacy protection disclosing their policies fully.

For skeptics of the free service, users will not have to look at advertising any more than is normal on the internet already.  At just $6 million for deployment this might be the cheapest project launched by the city providing the biggest return.

Page Popularity for Site: 8% [?]

Google’s blog search functionality may be starting to eke past Technorati. Google now offers users the option to get to its Blog search functionality from the main page on Google. Users can search blogs, by clicking the more button above the search box and then selecting Blogs, which is the top option in the drop down box.

Google launched the blog search option in September of 2005, but apparently didn’t realize that linking to it from their own home page would be the thing necessary to make it successful. This is a rather startling mistake in that Google is lauded as being a very smart company, but took over a year to figure this out.

Now that they are past that lesson straight out of the obvious book, they are making up time and picking up searches from key age demographics among searchers in the age group of 18-24 year olds, while Technorati’s shows a strength among searchers that are over 45 years in age.

Page Popularity for Site: 15% [?]

Google Search technology took the world by storm.  The tempest was originally hidden within Yahoo! Search and later broke away to become its own company. 

As it did this, Google assumed the throne as the best way to find information on the internet quickly and efficiently.  That was years ago and many people have since learned better how to manipulate this super highway of information, and more importantly internet readers and coral the to a site to earn advertising money from them, often with Google’s help.

One of Google’s primary advantages was its advanced and continually advancing algorithms that enables the company to continually and endlessly index the internet and all of the billions of web pages that get added, removed and updated through out the months and years.  This was a major advance over Yahoo!’s original preference for editorialized directories that served to categorize information and provide a contextual description.

Jimmy Wales the founder of Wikia the company that created Wikipedia is hoping to use a type of intelligence even greater than a Google created algorithm.  Wales would like to use NAI or Non-Artificial Intelligence.  If you are reading through the pun, you have deciphered it correctly, he wants to use people. 

This almost seems like a throw back to editorialized categories, but instead of providing descriptions, this Wiki version of internet search would provide user generated context.  Instead of a single editor or small group of editors that would be responsible as experts for a topic, the model would be more Wiki like, enabling people to contribute and refine the context from all around the world in many different languages.

Wikipedia provides an online user contributed and edited encyclopedia of everything that is entered, modified and updated in real time.

Creating search tools that are modified and refined by actual people as opposed to algorithms might just push ahead of Google Search.  Google does incorporate surfing metrics from people as well as it attempts to gauge how successfully a person has been guided to a destination website.  Wikipedia Search could take a different form where a user searching for information reads not a description but actual secondary or tertiary source background on the topic with hyperlinks off to the primary and secondary sources of that information, thus educating visitors and directing them to a site in context as they read.

Think of the tool like a three year old asking questions:

Why do fish swim?

They swim because they live in the water.

Why do they live in the water?

Because they cannot breathe air directly but use gills to extract air from the water.

What are gills?

At each branch the child or searcher or questioning group, can opt to pursue what might seem like a tangent, but in actuality they are refining their understanding so that they can gain better understanding and to possibly ask better questions.

A person sometimes has some information and might ask a question that takes them into the middle of a topic.  If they do not find the context to show them that they are learning about the process half way through, they could miss a crucial step.  Like a skydiver that misses the step 1. Pull rip chord and jumps directly to the instructions on navigating an inflated parachute and how to land.

A Wiki Search service might guide us searchers safely to Earth with a better comprehension of the topic that we are researching, and prevent the befuddlement that comes from a search result that tells us that the answer to everything in the Universe is 42.

Page Popularity for Site: 10% [?]

Its a Saturday and low on my priorities today is serious writing.  I need a quick tech fill however and hopped over to Digg where I came across a great story from a writer that works for Google debunking the myth that Google works for or in collaboration with the CIA.

Now the article doesn’t really debunk anything with proof, the debunkation buck stops short of the mark and sidetracks on a very funny dialogue between a Google Public Relations Person and a Google Founder.

From a personal perspective, I don’t care one way or the other.  Both groups will do what they have to do in order to get ahead.

Could Google have been cooperating with the US Government when they moved their Chinese user database out of China and out of the hands of the Chinese government and Red Army?

Sure such a move was good for corporate relations, but it would have been good for the US government too.

Of Course Google had previously agreed to censor searches in China.

So on an evil scale of 1 to 10

Google might see giving search records to the Chinese government as an 8, but censoring information from Chinese citizens is only evil at about a 5 or 6 and advertising for russian dating women at a 0.5 to a -2, but the Google culture might even see importing Russian brides as a public service for the brides and the nerds that bring them to the states so maybe its a -10 on the evil scale.   Either way they see it that helping the Chinese Government directly is more evil than withholding the raw information from their Chinese customers.  Its a corporate dilemma worthy of Solomon and worked out by mere mortal Googlions.

Playing into the hands and designs of the CIA and or some other group yet to be named by conspiracy theorists is off the scales (you be the judge of which direction it is going off the scales).  Then again is Google being devilishly smart, kind of like The Firm Character, Mitch (aka Google) has secreted away its database some where in a safe harbor where no government can touch it, not the Chinese and not the US.  Its good for the CIA the Chinese can’t touch it but may not be helping the US stay the course in the War on Terror.

But then again the NSA has been working on systems and codes for a lot longer than Google has been putting together algorithms.   Google may know how to get the right search result most of the time off the internet but I’ll bet you dollars to donuts the NSA knows how to get information out of a Google DB when they need it, safe harbor or not.

[insert maniacal laugh here]

Page Popularity for Site: 8% [?]