There has been an issue discovered when installing on the AIX machine. This is a known issue with respect to running an Installshield Multi-Platform installer on an AIX 5.3 machine.
If the process creates a core dump when executing the installation of the agent on your AIX 5.3 machine. Here is the important information from the core dump of the java process:
SIGSEGV received at 0x362220e4 in /opt/JDEdwards/Install/ismp001/libaixppk.so. Processing terminated.
If you see the above in the dump file then use this link to solve the problem. Follow the instructions closely and this should allow you to install the agent. It is a fix to the libaixppk.so for ISMP. Here is the link:
http://knowledge.macrovision.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=Q111262&sliceId=