Twitter and ITDI - Part 2

In my "Twitter and ITDI" article I showed how we can use ITDI to send a tweet and I stated that getting ITDI to read tweets might be a good next step in the process of fully Twitter-enabling our ITDI Assembly Lines. This article will hopefully shed some light on how we could go about doing just that.

The first thing we need to do is create ourselves an ITDI Assembly Line and drop in a FOR-EACH Connector Loop of type HTTP Client. I configured my HTTP Client connector to perform a GET on the following URL:

http://api.twitter.com/1/statuses/user_timeline.xml?screen_name=stephenjswann&count=5

I slected the Simple XML Parser as a parser for this connector and assigned the Root Tag and Entry Tag as "statuses" and "status" respectively, as such:

I performed a connection on my Input Map tab, clicked on next and dragged the created_at and text schema attributes to my map.

Fantastic! I now have a connector which will loop around the tweets it receives as a result of performing a GET request on the user_timeline.xml resource on Twitter. So what next? Well, within the loop, I would probably want to drop in an IF branch with the following script in place:

a = work.getString("text");
if (a.contains("#tditest")) {
    return true;
} else {
    return false;
}

This script will query the contents of each tweet looking for the hashtag #tditest. Any tweet containing this hashtag will invoke the contents of our IF branch. Of course, you could put whatever you want into the IF branch but for the purposes of my test, I dropped in the sendTweet function used in yesterday's article to post a response back to Twitter stating that a tweet with the #tditest hashtag had been found. I'm quite sure there are many more practical actions which could be performed at this stage!

Now the intelligencia reading this will be wondering what use this loop connector is in its present form. After all, once we have read through the tweets that we have requested, the Assembly Line will stop. And if we run it again, it will just process the same tweets again, right?

Right. Unless we expand our solution with a couple of minor tweaks. We can enable the delta processing in our HTTP Client Connector which ensures that we will only ever process NEW tweets. Any unique attribute of the tweet should suffice (and there are a few) but I settled on created_at, for no particular reason:

Now I can run my Assembly Line over and over again safe in the knowledge that it won't trigger my IF branch at all unless there is a new Tweet that it hasn't ever seen before. The final step for me was to wrap this Assembly Line in a timer controlled Assembly Line. I created a readTweetsController Assembly Line with a timer function as an iterator and all time attributes set to * (which means the AL will fire every minute). In my data flow, I placed an Assembly Line Function component to call the AL I built above.

The result? A mechanism which polls Twitter every minute looking for new tweets with a particular hashtag and "does stuff" if a matching tweet is found.

So what are the real world applications for such a mechanism? Theoretically, I could:

• post a tweet "TDI: getStatistics" and my TDI AL could return "TDI RESPONSE: 20,000 records processed"
  post a tweet "TDI: startSync" and my TDI AL would kick off a synchronisation process and return "TDI RESPONSE: synchronisation started" followed some time later with "TDI RESPONSE: synchronisation complete"
• post a tweet "TDI: switchOnLights" and my TDI AL could switch on the lights in my house
  post a tweet "TDI: exec "su - shutdown now" and my TDI AL would ignore me as a fool

Of course, there may be better ways of doing any or all of these tasks but using Twitter for non-critical communcations with your ITDI Assembly Lines should be relatively straightforward. After all, we aren't always within easy reach of our laptops, but we rarely go anywhere without our phones and remote management via phone and Twitter could be useful.

Hopefully this article will give you some food for thought.

Twitter and ITDI

It is becoming clear that Twitter isn't just a platform for telling the world what you are having for your dinner. Twitter is used as a marketing tool, a news information service and a tool for comedians to test new jokes. It's robustness and message persistence make it an excellent candidate for creating a Message Queue service for those who haven't the inclination for deploying a "proper" MQ service and it can also be used as an alerting mechanism.

For example, I would be quite happy to receive a tweet stating that something had happened to my enterprise application during the night with relevant information like an error code. After all, it is a free way to send the equivalent of a SMS.

Twitter offers a great API and a simple single command can update your Twitter status as such:

curl -u user:password -d "status=This is my first tweet via the Command Line"

But what if curl isn't available on your system? What if you have a suite of ITDI Assembly Lines which you would like to interface with Twitter? Well, nothing could be simpler (until Twitter disable their Basic Authentication mechanism).

An HTTP Client Connector should be created in LookUp mode within your Assembly Line and the connection details updated as such:

The Link Criteria should be updated to include a status parameter:

And the returning http.body attribute can be converted into a string for further analysis such as checking that the tweet was sent successfully:

Our sendTweet connector would be a great addition to our arsenal of alerting mechanisms which should already include sendMail, sendSNMPAlert and updateLog.

Next Steps

Building a Twitter Assembly Line which can perform the OAuth authentication would be a natural next step as would a connector which can read a "twitter stream" (as the equivalent of an MQ Consumer).

NOTE: The above screenshots were taken from a TDI v7.0 instance running on Windows 2003 but the concepts can be used in older versions of TDI with little (or no) modification.

The Power Of Twitter & God's IT Usage

When I posted my musings on "Identity & Access Management In The Cloud" the other day, I did something I don't normally do. I advertised the fact that I had posted something via Twitter.

Now, my blog is mainly a way of recording my own thoughts as I travel through space and time and I treat it like an online diary that I can look back on with fondness. I don't really expect anyone to read the stuff. I certainly don't expect anyone to agree with my thoughts. And the notion that people would even take the time to comment on the ramblings never entered my head. But then there was Twitter!

My "tweet" mentioned the words identity, access, management and cloud and seems to have been picked up by quite a large number of people - comparatively speaking! I had 3x more visitors in one day than I normally do in a month!

If anything, this turn of events impresses upon me the following:

• People are interested in the Cloud
• People are interested in security when it comes to the Cloud
• If people are interested in what I have to say, I need to be very careful what I say!

That last one might seem strange, but I've always been careful with my online persona - I think. I don't use bad language whether it be within my blog entries, on Twitter, on Facebook or wherever. There's no need for it and we should remember that it's permanent! I'm also a little nervy about writing anything that is controversial. (I guess I just wanna be loved and can't bear the thought of upsetting anyone?)  In other words, my reputation is obviously very important to me.

Facebook & Twitter
There has been a lot of online discussions surrounding the management of identity with regards to online services such as Facebook & Twitter. While enterprises won't be too impressed with this notion, it is quite understandable that the likes of Facebook & Twitter could emerge as identity provider kings! I can't afford to have my Facebook account suspended and I certainly don't want my Twitter feed to suffer any kind of service interruption. As such, behaving appropriately when using these services is important to me. And, of course, because I'm a well behaved boy on these services, there's a good chance that they could be used to assert my identity quite faithfully.

Think about it. Would I be keen to authenticate myself to a dubious website using my reputable Facebook credentials? Reputation management, for me, is just as important as identity management (if not more so).

God
DISCLAIMER: If Pope Benedict and Richard Dawkins were lined up in the school playground pulling together their "gangs", I'd line up behind Dawkins. Sorry Benny.

Someone told me today that they doubted whether they would make it to heaven because they reckoned that God's choice of IT components would be akin to how government's go about their purchasing of IT components. It got me thinking...

• Would God choose Oracle, DB2, MS SQL Server or MySQL? Nobody ever got fired by buying IBM, but who could fire God?
• Would God choose Windows, AIX, Solaris or Linux for his servers?
• Would God go Mac?
• Would God deploy IIS or WebSphere?
• Would God embrace open-source?

And what about Dawkins? Presumably he would prefer to select IT services based on the survival of the fittest model?

I'm having a laugh, of course. But the selection of any IT component can't possibly be determined to be right or wrong based on the component itself. It can be determined to be right or wrong based on how it interacts with the user and other IT components but I can't tell you that Macs are better than PCs. I can't tell you that Apache HTTP Server is better than Sun's offering. I can't tell you that PHP is better than Python which is better than COBOL which is better than C#, etc.

And the point? Well, I was asked yesterday whether I could help a customer select a database vendor and the options were Oracle and IBM. My answer? Technically, I come from the "a DBMS is a DBMS". The real questions are:

• Do you have in-house skills in one of the technologies
• Do you have existing relationships with either vendor
• What is the cost to you - TCO-wise

Technically? Maybe I'm past caring. The "religious" questions are so much more important!

NOTE: The answer is DB2. No. Oracle. No. MySQL. Yeah. That's the one. Oh. Maybe not :-)