The IBM Tivoli Identity Manager API - Jythonised

I wanted to build a new ITIM environment recently and figured it was about time I started to put together a proper set of Jython scripts to help me automate the process. Below, I've detailed my thinking behind a Jython script which will be capable of taking a delimited file defining an Organisational Structure, and load it into ITIM using ITIM APIs and the Jython scripting framework.

My virtual environment for this purpose was:

• Windows 2003 R2 Standard Edition Service Pack 2
• IBM WebSphere Application Server v7.0.0.9
• IBM Tivoli Identity Manager v5.1.0.0
• APISCRIPT v5.0.0.4 (available from OPAL)

The Setup
The apiscript tool (v5.0.0.4) should be downloaded from the OPAL site and deployed as per the instructions. For example,  the following files need to be configured to match the ITIM environment being managed:

• etc/host/{hostname}.properties
• bin/env_master.bat

But because I'm using an ITIM v5.1 system, I need to also make a "tweak" to the apiscript.bat (or apiscript.ksh) file as the structure of the extensions directory has been updated in this release. I need to include a 5.1 directory between extensions and examples as such:

set APISCRIPT_LOGIN_CONFIG=%APISCRIPT_ITIM_HOME%\extensions\5.1\examples\apps\bin\jaas_login_was.conf

Input File
The following input file is to be used to define the organisational structure:

ParentOrgUnit|OrgUnit|

|internal|

|external|

internal|unit1|

internal|unit2|

external|unit3|

unit2|unit4|

unit2|unit5|

unit10|unit7|

unit4|unit6|

external|unit4|

external|unit8|

external|unit8|

unit8|unit9|

unit9|unit8|

Each line was terminated with a | character because early in the testing process I noticed that carriage return/line feed characters were making their way into the ITIM environment.

The Code

The code required to process such an input file should be broken down into a number of sections:


Section 1: Process The Command Line Arguments
In order to ensure that the script can process a variety of input files, the file to be processed should be passed as a command line argument. An additional argument is being processed here to enable verbose logging to take place:

try:
   opts, args = getopt.getopt(sys.argv[1:], "qi:", ["inputfile="])
except getopt.GetoptError, err:
   print str(err)
   usage()
   sys.exit(2)

inputfile = ""
quietmode = "false"

for opt, arg in opts:
   if opt == "-q":
      print "Quiet mode enabled"
      quietmode = "true"
   elif opt == "-i":
      inputfile = arg

Section 2: Read the Input File
The processing of the input file should be wrapped in a while loop and each line processed should be split into its constituent parts using the split method on the line object:

infile = open(inputfile,"r")
while infile:
   line = infile.readline()
   if not line: break
   items=line.split("|")
   parentorg=items[0]
   org=items[1]

At this point, we now have an Organisational Unit Name and a Parent Organisational Unit Name ready for placement in the ITIM data model.

Section 3: Process Each Entry
For each record retrieved above, we need to determine that the Parent Organisational Unit Name actually exists in the data model. If it does not, we cannot process the entry. If it does exist, we need to check that the child OU doesn't already exist. If it also exists, then there is no point in continuing processing of this entry, otherwise we need to get an OrganizationalUnitMO object for the Parent OU to enable us to create an OU using the child Organisational Unit Name.

if parentorg == 'ParentOrgUnit':
   # Do Nothing - it's the header
   logit('Processing Header')
else:
   logit('**********************************************')
   logit('Processing ' + org)

   if org_exists(parentorg) != 'False':
      if org_exists(org) != 'False':
         logit('Child OU already exists:' + org)
      else:
         # Create OU
         logit('Child OU does not exist:' + org)
         parent_org_mo = get_org_mo(parentorg)
         orgchart.do_create_ou_from_args(org, parent_org_mo)
      # end if loop
   else:
      logit('Parent Org Unit does not exist:' + parentorg)
   # end if loop
# end if

Section 4: Check That An OU Exists
To check that an OU exists, we need to perform a search using a SearchMO object for an ORGUNIT object using a filter based on the OU name provided in the data file. If an object is found, we should return TRUE, else return FALSE:

def org_exists(orgunit):
   foundit = 'False'
   if orgunit == "":
      foundit = 'true'
   else:
      def_org_cont_mo = orgchart.get_default_org_mo()
      myPlatform = apiscript.util.get_default_platform_ctx_and_subject()
      search_mo = SearchMO(myPlatform)
      search_mo.setCategory(ObjectProfileCategoryConstant.ORGUNIT)
      myFilter = "(ou=" + orgunit + ")"
      search_mo.setFilter(myFilter)
      search_mo.setScope(SearchParameters.SUBTREE_SCOPE)
      search_mo.setContext(def_org_cont_mo)
      results_mo = search_mo.execute()
      for result in results_mo.getResults():
         if orgunit == result.name:
            foundit = 'True'
   return foundit
# end org_exists

Section 5: Get the MO Of An OU
An OrganizationalUnitMO object can be generated after searching for an OU by performing two additional functions:

• getDistinguishedName on the search result
• create_org_container_mo using the DN returned from the above method

def get_org_mo(orgunit):
   if orgunit == "":
      org_mo = orgchart.get_default_org_mo()
   else:
      def_org_cont_mo = orgchart.get_default_org_mo()
      search_mo = SearchMO( *apiscript.util.get_default_platform_ctx_and_subject())
      search_mo.setCategory(ObjectProfileCategoryConstant.ORGUNIT)
      myFilter = "(ou=" + orgunit + ")"
      search_mo.setFilter(myFilter)
      search_mo.setScope(SearchParameters.SUBTREE_SCOPE)
      search_mo.setContext(def_org_cont_mo)
      results_mo = search_mo.execute()
      for result in results_mo.getResults():
         if orgunit == result.name:
            mydn = result.getDistinguishedName()
            org_mo = orgchart.create_org_container_mo(mydn)
   return org_mo
# End get_org_mo

The Result
The output from the script is:

C:\work\apiscript-5.0.0.4\apiscript>c:\work\apiscript-5.0.0.4\apiscript\bin\apiscript.bat -f c:\work\apiscript-5.0.0.4\apiscript\py\orgs.py -z -i c:\\work\\apiscript-5.0.0.4\\apiscript\\data\\orgs.dat
Using master environment: "C:\work\apiscript-5.0.0.4\apiscript\bin\env_master.bat"
Using custom BIN_HOSTNAME: "stephen-w0ckd5b"
Using custom ETC_HOSTNAME: "stephen-w0ckd5b"
Using host properties: "C:\work\apiscript-5.0.0.4\apiscript\etc\host\stephen-w0ckd5b.properties"
Using APISCRIPT_WAS_HOME: "C:\Program Files\IBM\WebSphere\AppServer"
Using APISCRIPT_ITIM_HOME: "C:\Program Files\IBM\itim"
WASX7357I: By request, this scripting client is not connected to any server process. Certain configuration and application operations will be available in local mode.
Welcome to IBM Tivoli Identity Manager API Scripting Tool (apiscript) version: 5.0.0.4
Setting system property: java.security.auth.login.config
Setting com.ibm.CORBA properties: loginSource, loginUserid, loginPassword
WASX7303I: The following options are passed to the scripting environment and are available as arguments that are stored in the argv variable: "[-z, -i, c:\\work\\apiscript-5.0.0.4\\apiscript\\data\\orgs.dat]"
Logging configuration file is not found. All the logging information will be sent to the console.
**********************************************
Input file selected is c:\work\apiscript-5.0.0.4\apiscript\data\orgs.dat
Processing Header
**********************************************
Processing internal
Child OU does not exist:internal
**********************************************
Processing external
Child OU does not exist:external
**********************************************
Processing unit1
Child OU does not exist:unit1
**********************************************
Processing unit2
Child OU does not exist:unit2
**********************************************
Processing unit3
Child OU does not exist:unit3
**********************************************
Processing unit4
Child OU does not exist:unit4
**********************************************
Processing unit5
Child OU does not exist:unit5
**********************************************
Processing unit7
Parent Org Unit does not exist:unit10
**********************************************
Processing unit6
Child OU does not exist:unit6
**********************************************
Processing unit4
Child OU already exists:unit4
**********************************************
Processing unit8
Child OU does not exist:unit8
**********************************************
Processing unit8
Child OU already exists:unit8
**********************************************
Processing unit9
Child OU does not exist:unit9
**********************************************
Processing unit8
Child OU already exists:unit8

C:\work\apiscript-5.0.0.4\apiscript>

Visually, this gets represented in ITIM as:

In conclusion, I'm not a Jython/Python sripting guru. Indeed, this was my first foray into Python. It is, however, a relatively straightforward language to learn and can be a very powerful tool in your ITIM Administrative toolset.

The full script can be downloaded at downloads/loadous.zip.

The Computer That Never Dies

I've been reminded of the halcyon days a number of times in recent weeks. Meeting old colleagues and working on integration middleware components with legacy systems has taken me back to my days as a developer on mainframes.

For me, my mainframe of choice was a Bull DPS8000/DPS9000 which ran the GCOS8 operating system. Time-Sharing, JCL and EBCDIC were all terms I grew up with and I really enjoyed my time working on the platform. The code I wrote 20 years ago still exists which is reassuring and precisely the point of this article. The code was designed and written in a manner which meant it would stand the test of time.

Indeed, the code for that particular environment was so good that it required no modification in the run-up to the Year 2000 anti-climax.

In my early days, the mainframe took up almost an entire floor of an office block in the centre of Belfast. To be honest, this was at a time when my PC housed an enormous 8088 processor, with 12" long expansion cards and was painful to move about because of its sheer bulk. After a while, the 8088 was replaced with the 80286, then the 80386, then the 80486, then Pentium processors and so on. The mainframe adopted a similar evolution by getting quicker and smaller to the point where it no longer looked like a mainframe and took up little more space than other servers littered around the data centre.

Of course, during that period, most commentators expected the mainframe to finally die and be replaced by the modern array of UNIX based servers from HP, IBM and Sun. Why spend all that money on a mainframe when a couple of RS6000s could do the job just as well? Except they couldn't. Middleware did middleware very well. Mainframe did mainframe very well.

But the mainframe hasn't died. It's still alive and well in most of the big data crunching data centres and some would say that it has a new lease of life. Why? Because mainframe can now do middleware, that's why. The mainframe is no longer limited to long-running COBOL based batch processes invoked via JCL put together by Sys-Progs who look like dinosaurs. Mainframes can happily host our web servers, application servers and message buses as well our databases and batch routines.

But there is a problem! Mainframes aren't sexy and aren't attracting young talent into the arena. Sys-Progs are a dying breed. Guys who know their mainframes are in short supply. But at least we have tools to help the mainframe newbies administer their system.

Which brings me on to an exciting new initiative I've been privileged to be involved with. In conjunction with System Z and RACF guru, Alan Harrison (of Practically Secure fame), I've been working with building an extension to the IBM Tivoli Identity Manager interface to provide slick integration with zSecure Audit to provide a complete RACF management web based solution that integrates with an enterprise wide provisioning system. The result is now available from Pirean (where both Alan and I currently ply our trade).

Mainframes are suddenly sexy again and it almost feels like a home-coming for me.

Apply That Patch....

What should you do when system performance starts to deteriorate?

More specifically, if password changes are taking upwards of 12 minutes to complete, what should you do?

Even more specifically, what if password changes invoked from IBM Tivoli Identity Manager through the TAM Combo Adapter to Tivoli Access Manager are taking upwards of 12 minutes to complete yet manually changing the passwords via TAM's pdadmin command line tool completes sub-second?

This was the dilemma I faced yesterday and it seemed to happen "all of a sudden".

The password change via pdadmin confirmed that we weren't talking about a DB2 or LDAP issue. They had recently been tuned and everything was performing as expected. So I did what any decent IT professional would do - the on/off approach.

I stopped and restarted the TAM Policy Server - for no real reason, to be honest. I then stopped and restarted the TDI RMI Dispatcher that was "hosting" the TAM Combo Adapter.

The next password change that came through the system took approximately 6 minutes to complete. A 50% improvement but nowhere near good enough in an environment hosting 300,000 users eager to change their passwords (with the result being a bottle-necked system).

The TAM Combo Adapter was v5.0.5 and the Assembly Line for the password change seemed to be as straight-forward as an Assembly Line can be. At no point could I find:

if (systemAge > 12Months) {
   sleep(600);
}

Google was no help either. Nobody on the planet had experienced this sudden slowness and documented it!

Running out of ideas, I decided to upgrade the adapter to v5.0.9. A quick download, install, copy of the TAMComboUtils.jar file to the TDI directory structure and a restart of the TDI RMI Dispatcher was all it took.

Moments later, a password change came into the system. How long would it take? I guessed it would be around 6 minutes again but I was wrong. A handful of milli-seconds!

I quickly looked through the supporting documentation for the adapter to see what fixes were incorporated. No mention of password change "slowness". And no mention in the v5.0.6, v5.0.7 and v5.0.8 release notes!

So what's the moral of the story?

Developers don't change fix bugs when a new release of code is coming out. They frequently "tidy" things in a non-functional way which may have positive impacts!

For those of you still running your ITIM v5.0 environments with an old TAM Combo Adapter - upgrade now. You won't regret it. Maybe. Unless one of those "tidy" code changes has a negative impact on your environment. In that case - forget you ever read this post!

Of course, the approach adopted for this particular scenario can also be applied to all software environments. Keeping up-to-date with the latest patches is a good thing to do even though it can be time-consuming. But how do you do that in a manner which meets all your Change Management processes? Can you really patch WebSphere without having to perform a full regression test of all your J2EE applications? Can you upgrade your LDAP without a full regression test of all applications that make use of its services?

The answer is you can but it takes you to be convincing when it comes to getting the authorities to take that particular leap of faith. And therein lies the problem. Far too often, sensible environment management is put into the "too hard bucket" not for technical reasons, but for political reasons.

When will we learn.

NOTE: As my experience yesterday can attest, sometimes the best way to get patches applied is in a Sev 1, emergency scenario. Don't go creating those scenarios though!

Infosec 2010 Review

I managed to spend a little bit of time this week taking in the spectacle that is Infosec at Earls Court, London.

The first thing that struck me about the event was the vastness of it. The number of exhibitors was really quite staggering and the quality of some of the stands was very impressive indeed.

If I had a particular interest in anti-virus, one-time password generation via SMS and hardened USB storage devices, I would've been in heaven as these particular products were over-represented at the event. But how do anti-virus vendors differentiate themselves at an event like Infosec? Well, by giving away an Apple iPad each day! That did the trick for Symantec.

It was interesting to see the various approaches that vendors took to attracting visitors to their stand. The guys at Qualys found a great way of attracting large numbers of visitors by giving away free beer from mid-afternoon onwards. Others tackled the marketing problem by using scantily-clad girls. I'm not sure what the link between scantily-clad girls and security software is but then again, Grolsch and Qualys don't seem to have a natural partnership either.

Wandering around the event entices the sales men and women to accost you. Free pens, stress balls and T-Shirts will be thrust into your possession with white papers and brochures, of course. These freebies have already been passed to my daughters and the white papers and brochures still haven't been read four days later!

And here's the crux of Infosec. For many people, I'm guessing the event has got very little to do with sales leads and more to do with CISSP CPEs and networking.

Of course, Infosec isn't just about vendors trying to showcase their wares. There were plenty of seminars, speeches, workshops and other types of get-togethers. But again, there seemed to be little by way of new or innovative ideas being discussed. A discussion on "Mash-Ups" reminded me of a similar discussion five years ago on "Process Orchestration"! A discussion on "The Cloud" reminded me of a similar discussion five years ago on "Application Service Providers". In other words, the terms may have changed but the concepts have not.

Of course, this doesn't mean the experience wasn't fruitful. It is still a great event well worth attending and it's a great opportunity to meet other vendors/suppliers and catch up with what they are getting up to. I, for one, am already looking forward to Infosec 2011.

Essential Freeware Applications For Your Desktop

Within the last couple of weeks, I've bought a new PC and re-imaged a laptop - both with variants of the Windows 7 operating system. There is something comforting about a clean system - it's a chance to install only the applications that you absolutely need and also a chance to ensure that you have the most up-to-date versions of those applications.

So here is my list of essential applications (for Windows) that everyone should install - at least everyone who performs the kind of tasks that I do!

Communications

Firefox is an absolute must have but there's no point in having Firefox without the add-ons. For me, that means Live HTTP Headers, Favicon Picker and Web Developer add-ons get installed straight-away.

Thunderbird is my eMail client of choice but again this is due to the add-ons. Lightning, Provider for Google Calendar, Google Contacts and an ability to manually sort folders are all necessary additions to the base product!

Filezilla is my FTP client.

PuTTY is my preferred terminal client. So lightweight. So robust.

Skype is used for phone calls - I typically use Messenger for instant messaging rather than Skype but when it comes to telephone calls, Skype can't be beaten.

Security
Password Safe because I'm a responsible citizen who uses different passwords for different applications/websites and I can't possibly remember them all.

AVG Free still makes it on to my machines. I've tried others but found AVG Free to be the least invasive.

Development
I still like to handcraft my HTML - I don't know why. I can't help it and because I code in PHP, JavaScript, HTML and various other scripting technologies, I find it easiest to use Notepad++. Macro recording and code highlighting without the bloat? Install it!

MySQL Workbench is an application that I use not just to manage MySQL databases but to design databases in general (even if the target platform is DB2 or Oracle).

Toad for DB2 (and other variants). MySQL Workbench creates very pretty diagrams, but Toad is just plain superb for remote management of a database.

TortoiseSVN helps me keep my source code under control!

I have a lot of demonstration environments and have spent many years using VMWare's Workstation. However, I've recently been converted to Sun's xVM Virtualbox product. I haven't performed any serious benchmarking but I've found it just a little more slick when using virtual machines on my laptop. Serious serving of virtual machines is something I may still turn to VMWare to perform, though.

Miscellaneous
The following get installed just because:

• Google Earth
• Tweetdeck
• Other browsers (to verify cross-browser compatibility): Google Chrome; Safari; Opera
• Google Picasa
• Adobe Acrobat Reader (of course)
• Spotify because I sometimes like to work while listening to music
• CutePDF for PDF generation from any of my apps
• Microsoft Media Encoder for video clip production
• JZip for archiving
• GNU Tools for Windows which gives me tar, sed, awk and grep capabilities
• OpenProj for project management

All of the above applications are freely available so there is no financial excuse for not installing them. I do have some commercial applications installed as well though. Despite the great strides that OpenOffice and Star Office have made in recent years, I still can't seem to do without Microsoft Office. And Visio is still tops for me!

Messaging Systems

Inter-application communication has evolved over the years from proprietary protocols and fixed message formats to message queues and web services based systems using XML.

But it isn't unusual to see an application communicate via mechanisms more usually associated with real-world messaging between real people. eMail, for example, can be used to convey messages from one system to another although eMail isn't necessarily a great way of guaranteeing delivery of any message in my experience!

Bringing us right up-to-date is Twitter. Although Twitter is a way of communicating using short-based messages only (as it is limited to a mere 140 characters), it does have a good record of being almost always available. Sending a short-based message from one system to another could utilise the power of Twitter quite easily, surely?

Consider the following usage pattern...

An offsite managed service is being provided by a third party with support hours of 08:00 - 18:00. Overnight batch processes are to be checked each morning by the service provider. The service provider, however, introduces monitoring of systems which trigger exception alerts via Twitter for off-site pick-up via any amount of Twitter clients on any number of devices. Think about the following message:

2010-04-07 22:50:05 FATAL: Batch Feeder [BACS: Invalid Trailer Record]

Instead of waiting for support staff to analyse logs the following morning, they have a head start telling them exactly where to look.

Of course, we could just have the monitoring service raise a ticket or send an email or send SMS alerts. The point is not to show that Twitter is better than any of these delivery mechanisms but to show that Twitter is an alternative. (After all, it could be that we have a Twitter listener which queries these messages and raises the necessary problem records in the service desk.)

The point is the ultimate delivery mechanism of the message to the client isn't the responsibility of the message generating application. The application doesn't care if the message is to be viewed by a custom-built message reader on an iPhone, Tweetdeck on a PC or a shell script sending curl requests at twitter.com!

I'm sure my avid readers can come up with many more sophisticated use cases for the integration of Twitter into the enterprise.

NOTE: Any such inter-application communication using Twitter should ensure that the tweets are protected. There's no point in having "randoms" looking at your tweets, after all.

ITIM Messaging & Synchronisation

WebSphere gurus all the world over will probably understand what a messaging cluster is; what tranlogs are; and how all this "stuff" works.

For systems integrators who don't necessarily specialise in any one technology, these things may seem more like a dark art and certainly the information made available in the myriad of documents produced by the various authors of WebSphere technical books doesn't seem to shed enough light on what is going on.

Yesterday, I had a misbehaving IBM Tivoli Identity Manager (ITIM) instance which saw all transactions sitting in a pending state with not a single transaction being flushed through to completion.

The setup was: ITIM v5.0 running in a WebSphere v6.1.0.9 clustered environment with two physical servers. Everything about the deployment was fairly vanilla (though the amount of data going through the system is quite large).

The problem started on Sunday night during the automatic regeneration of LTPA keys - it seems. The Deployment Manager lost the ability to control the Node Agents/Clusters which forced the following sequence of events:

• Forced shutdown of the Application Servers and Node Agents
• Removal of Global Security
• Manual synchronisation of the nodes on the physical servers
• Startup of the Node Agents
• Reconfiguration of Global Security
• Resynchronisation of the nodes (via the Deployment Manager)
• Startup of the Messaging Cluster
• Startup of the Application Cluster

Everything looked OK, until transactions started to appear in ITIM and they still sat pending!

The logs showed that the Messaging Cluster would start, then stop, then start, then stop, then start, then stop.....

Communications channels between the local queues and the shared queues weren't as they ought to have been and the root cause seems to be an inconsistency between transactions that ITIM expected to be pending; transactions the Messaging Cluster thought it had; transactions stored in the "physical" storage of DB2 for the messaging cluster and the tranlog.

NOTE: The recovery procedure for this is probably not something anyone should undertake but it was a final straw after 12 hours of trying various tactics which could have saved my pending transactions (which proved futile).

• Stop the clusters
• Stop the Deployment Manager
• Kill any rogue WebSphere processes (of which there were a few)
• Stop the database manager supporting ITIMDB
• Delete the various tranlog files for the two application server instances
• Reboot the two servers (which probably wasn't required, but these were Windows 2003 server instances which seemed to be hanging on to various ports needlessly and I wanted a clean environment to start everything up again)
• DANGER: delete from itiml000.sib000 in the ITIM Database (as well as itiml001 and itims000 and for sib001 as well)
• Start up ITIM

At this stage, all was well. Apart from my physical well being.

CAUTION: The above approach was a very brutal way of clearing out everything to a point where the system was operational once again. I never want to have to repeat this process and having a clean ITIM instance which is well tuned and well looked after is very much more preferable than having to perform this kind of recovery.