Twitter - The Price of Success?

Been doing a little fiddling to ijy.cc, and noticed that twitter.com was down...

I've redone ijy.cc so that rather than having to deal with backend security myself, now I use a number of high profile online services, and instead pull in the content from them into a single "digital footprint". The end result is that my page is currently dependant on;

Blogger - [link]
Picasa - [link]
YouTube - [link]
Twitter - [link]

I then grab the RSS feed for each of these sites, do some fairly simple data manipulation, and throw it into a fairly simple XHTML/CSS template. Its very easy for me to maintain, and also to chuck the stuff out to other places (for example, my Blogger blog also posts to MySite here on lack-of...

Twitter being down has the effect of killing ijy.cc, as I haven't bothered with any kind of cacheing, and PHP4 (which 34sp is running on) does not support timeouts on the PHP function running in the background capturing the RSS feeds ( file_get-contents, for those who care... ). So I thought I'd do a bit of checking to see if the Twitter downtime was a common occurance, or just bad luck...

Turns out that they have been struggling for about a year, ever since they "went big" in March '07. So much so that a monitoring site made their downtimes public and gave free access to their downtime reports. I know within my company that our targetted uptime for customer facing systems is in the region of 99.8% over the space of a year (and if we get anywhere near that everyone starts running round like headless chickens), so 98% average, and dips as low as 92% is fairly shocking. Of course, I haven't seen the other feeds go down as yet, however they are all backed up by Googles frankly imposing infrastructure, which suffered a whole 7 minutes of downtime in 2007...

...that said, how can Twitter actually make money to fund their horrendous bandwidth and data centre costs? While their page rank is not massive, behind all that you have loads of user-developed apps throwing data too and from them...it must all add up to one fat pipe somewhere. Becase of the nature of the service (small messages, accessible freely from many 3rd party apps) selling advertising is going to be very hard, unless you plan of irritating your entire userbase by doing mass broadcasting. I really see how they are going to struggle to make money, as quite simply people are not visiting their online real-estate, so they aren't getting the eyeballs. It seems that general internet opinion is that like many other big names out there now (google, facebook, skype etc etc) Twitter can plan a business model once they have the userbase...I suppose we'll have to see, but right now I'm a little skeptical.

Of course, all this doesn't help me much... I like the concept of Twitter, and it fits in with the theory of how I've set ijy.cc up. The box will stay for now, and I'll have to live with Twitter's unreliable uptime, however if it stoops too much worse I may have to look at either a technical solution (cronjob-ing and cacheing the feed, which is what happens on lack-of), or looking for a more reliable alternative to fill my top right-hand corner...

Comments

Twitter is pretty unreliable, but I don't think your experience has been that representative; it's been particularly bad over the past week or so. They got a lot of press over effectively breaking the Chinese earthquake, and I guess that's driven a lot of traffic their way.

In terms of accessing their service, you really should cache the data anyway; it's considered very bad ettiqutte for an aggregator to make requests to their providers for every page-view. I don't know what (if any) policy twitter have on the subject, but I know some sites consider it abuse, and will terminate your accout for it. Having said that, unless you're getting a lot of pageviews I doubt they'd notice. In fact, in a fit of rampant hypocrisy, I'm pretty sure my site just forwards requests to twitter for the sidebar widget. I'll have to do something about that.

AggroBoy's picture

I won't tell if you don't...

Having thought about it, setting up cacheing wouldn't be the hardest thing in the world...it would just be a matter of writing a script to copy the feeds to a local folder, linking that script to a cron job (which I'd need to read up on, but I'm fairly sure 34sp supports) and then re-pointing the current scripts to the local copies...

That said, having the feeds come in directly makes debugging one metric shit-tonne easier, though for the life of me I cannot find where this errant "8" character is appearing from...

babychaos's picture

How about putting it through a Yahoo Pipe? The Pipe can then cache it and you can just worry about getting the Pipe through. That's why Lack-Of's twitter feed is never down.

Loving the fact you've gone lite! Will post more when work is not mental.

brainwipe's picture

Also, cron is really easy to use through the 34sp admin system.

brainwipe's picture

Not a bad idea. I'll have a look at Pipes tonight (of course that moves the reliance from Twitter onto Yahoo, however from the stuff I posted above Yahoo managed no downtime at all last year, which probably chips in as a little more reliable...)

babychaos's picture

As an interesting addition to this thread, I went onto Twitter last night to do an update, and saw a "we will be doing some scheduled maintainence in the wee ours of the morning", and today its down.

I'm guessing someone dropped a spanner!

babychaos's picture

Well, in their defence, they're Californian, so it's still the "wee hours of the morning" there...

AggroBoy's picture

It seems to be working now.

Althought my "twit" doesn't seem to appear on the lack-of feed.

baron's picture

Now it's broken again

baron's picture

It takes a while for Lack-Of to update from Yahoo Pipes. It checks every 15 minutes. Yahoo's checks are a bit more random, I think.

brainwipe's picture

pattiminne

Who is pattiminne?

Some random that is following 259 people and doesn't post.

One of those 259 people is me.

Am I famous or subject to some kind of surveilance?

baron's picture

Its an advertiser I think...I have one as well called PrivateMessages.

Baron, you're banned from doing long word in twit posts, they break the page format!

babychaos's picture

Yeah I noticed that too. Sorry, but when you have to laugh manically, you have to laugh manically, concerns over page format take second place when you have played with lightning to bring the beast alive.

I'll try to keep the manic under control.

MWAHaha.... Sorry.

:)

baron's picture

Yeah - it's twitter spam. Either they're randomly following people to try to get reciprocal follows before they start pumping out adverts (which is unlikely if they're following less than 1,000,) or it's harvesting your (and a bunch of other people's) tweets to aggregate into some other bot account, to make it look more legitimate, and try to hide it's spammy nature.

AggroBoy's picture

You can block them...I just did with my 2 :-D

Most satisfying...

babychaos's picture

I blocked your privatemessages one too. Just in case it got any ideas.

A pre-emtive "Fuck off" if you will

baron's picture

Being an insane egomaniac, I welcome people to follow me. Even if it means very little, in my own mind it makes me feel like I've got a lot of people following me. Oh yes. Followers. Dedicated ones.

brainwipe's picture

This is the real world Rob. Not some Terry Pratchett novel. You won't become a god because you have followers.

Speaking of TP, I heard a good interview of him on Radio 4 this morning. Lasted about a journey's worth for me (30mins?) I think it's being repeated, but for those that like the guy you might find it easier in some listen again form on the BBC site. Think it started at 9am.

Yeah I left for work a bit late this morning....

Mwahaha... ahem...

baron's picture