|
(Formerly known as Application Integration Framework or AIF)
Release Date: 08 June 2010
Last Modified: 08 June 2010
Support Contact: pbssupport@altair.com
Contents
Upgrade and Compatibility Notes
| Important | To use the new features or enhancements for the PBS Professional Application Services (formerly known as Application Integration Framework or AIF), the Enterprise PBS Professional cluster must be upgraded to PBS Professional 10.4.
|
Migrate your existing application definitions from prior 10.x releases to 10.4
To migrate your existing 10.x application definitions to 10.4:
1) If using PBS Catalyst (web portal) export the configurations prior to upgrade shutdown the PBS Professional Application Services service.
2) Shut down existing PBS Professional Application Services service.
3) If using other than PBS Catalyst (web portal) copy the following to a temporary location:
a) application definitions from the old <AIF_HOME>/applications directory
b) site-config.xml from the old <AIF_HOME>/conf directory
4) Install new version of PBS Professional Application Services (do not start)
5) If using other than PBS Catalyst (web portal) copy the following back from the temporary location:
a) The application definitions into your new <AIF_HOME>\applications directory
b) The site-config.xml into the new <AIF_HOME>\conf directory
6) Start the new PBS Professional Application Services daemon/service.
7) If using PBS Catalyst (web portal) import the previously exported configurations.
Application Administrators Web Services port is deprecated
The Application Administrators Web Services port and methods have been depreciated as of release 10.2 and have been replaced by the new port called AdministrationServices.
NOTE: Schema of deprecated port is http://schemas.altair.com/pbs/2008/01/admin
NOTE: Schema of the new, currently supported port is http://schemas.altair.com/pbs/2009/08/administration
The independent installation of Python on each PBS Professional execution node is no longer required
New Installations of PBS Professional Application Services (PPAS) no longer requires Python to be installed by cluster administrators on each PBS Professional execution node. Python is included in the PBS Professional release 10.4 execution (pbs_mom) package. Additionally, the PBS Professional Application Services installer will recognize if Python is not present on a node where PBS Professional Application Services is being installed (for example a headnode without pbs_mom) and install Python as needed.
Features
Support for PBS Professional Application Services file operations when a job is submitted from PBS Professional CLI
The PBS Professional Application Services (PPAS) file operation feature available via PBS Portal and Catalyst enables users to be able to perform tasks such as creating/deleting files or directories, listing files, reading files, etc. This feature has a pre-requisite that several file operation transfer scripts be included with each job. If a job is submitted through PBS Portal or Catalyst, then PBS Professional Application Services takes care of making sure these scripts are included with the job.
The py_spawn process
There are two scripts that PBS Professional Application Services needs to be able to execute the file operation commands on the execution hosts. Upon installation of PBS Professional Application Services, these scripts are included in the following location in the PBS Professional Application Services directory structure:
<AIF_EXEC>\pbs_spawn\SpawnWrapper.py
<AIF_EXEC>\pbs_spawn\TransferFile.py
The file SpawnWrapper.py is responsible for staging data that is to become the stdin to the target script to the subdirectory where the job is executing, attaches stdin/out of target script to those files now co-located in same subdirectory, and then stages the jobs stdout data back to PBS Professional Application Services server. The target script is provided by PBS Professional Application Services and depends upon the action being requested.
The file TransferFile.py is responsible for staging in/out job data files to/from the local directories where the job is executing from/to remote hosts.
To enable the PBS Professional Application Services file operations capability for PBS Professional CLI job submission PBS Professional Application Services installs a PBS Professional hook that adds the stage-in of these files to the job script. The hook is automatically enabled when PBS Professional Application Services is installed and requires no additional configuration.
Resolved Issues
PBS Professional Application Services missing CPUs Requested and Used job attributes
PBS Professional Application Services (PPAS) now exposes these job attributes to consumers of the GetJobs method in the PBS Professional Application Services API (e.g. PBS Portal and Catalyst).
Input strings containing “+” not parsed correctly
Jobs submitted with select specifications including a “+” (e.g. -lselect=1:ncpus=1:mem=10+1:ncpus=1:mem=20) are now parsed correctly.
StageIn not correctly transferring files with spaces in their filenames
Files specified on both Windows and Linux with filenames that include spaces are successfully uploaded to job execution location.
File staging failing when spaces are used in directory path names
The job file stage area or STAGE_ROOT allows directory names with spaces in them.
Known Issues
PBS Professional Application Services server returns 404 error following Linux re-installation/upgrade
This issue has been observed following the re-installation of the PBS Professional Application Services into a different location (that is, different than the previously installed location) on a Linux platform. Browsing to the http://<hostname>:8084/manager/html URL may result in the following error message:
Requested resource is not available (404 error)
Workaround:
Do not start the PBS Professional Application Services during the installation. Start the service manually once installation is complete, for example:
/etc/init.d/aif start
If you do start the service during the installation and see the error message above, stop and restart the service:
/etc/init.d/aif stop
/etc/init.d/aif start
TransferFile.py fails with very short hostnames
In order to determine whether it should do a local file (i.e. cp/copy) or a remote copy (rcp/scp), the TransferFile.py script (see The py_spawn Process described above) looks at the "hostname" portion of a file URL. One of the tests it applies is a check as to whether the hostname is longer than 2 characters. This check returns an erroneous result if the PBS Professional complex in question has very short hostnames (e.g. n1, n2, etc.)
Workaround:
Hardcode a check for site's hostname into TransferFile.py, for example (see text in bold):
def separate_file_name(name): #{{{
'''
Break up a file URL in a form of "user@host:/filepath", "host:/filepath"
or "filepath" into core components. URL is separated into a hash with
keys "user", "hostanem", "filename"
param: URL name
return: hash object.
'''
# Hardcoded array of very short hostnames
very_short_hosts = ["n0", "n1", "n2", "n3", "n4", "n5", "n6", "n7"]
tail = name
# extract username
tokens = tail.split("@",1)
if (len(tokens) > 1):
user = tokens[0]
tail = tokens[1]
else:
user = None
# extract hostname
tokens = tail.split(":",1)
if (len(tokens) > 1):
# check if we are not dealing with windows letter names, like C:, D:, etc.
# legitimate host names are three characters or longer
if (len(tokens[0]) > 2):
hostname = tokens[0]
tail = tokens[1]
# Add additional check for short host names
elif (len(tokens[0]) == 2) and (tokens[0] in very_short_hosts):
hostname = tokens[0]
tail = tokens[1]
else:
hostname = None
else:
hostname = None
# the rest is filename
filename = tail
Python DLL files sometimes not copied to correct location on x86_64 platforms
As part of the PBS Professional Application Services (PPAS) installation process on x86_64 platforms several Python DLL files (python25.dll, pythoncom25.dll and pywintypes25.dll) are copied from the PBS Professional directory <PBS_EXEC>\python\bin to the system directory C:\windows\sysWOW64. Occasionally the installer misinterprets the platform type and ends up copying these DLL’s to C:\windows\system32 instead. If the installation requirements were correctly adhered to, and there was no existing Python installation on the system, everything will still work as expected. But if there was an existing Python installation in the C:\windows\sysWOW64 directory and it is a different version than what PBS Professional Application Services expects, PBS Professional Application Services will not function as expected.
Workaround:
After installation on an x86_64 platform check the log files for the messages:
Execute Script/Batch file: Check if the OS is a 32 bit or 64 bit
Status: SUCCESSFUL
Execute Script/Batch file: COPY TO System32
Status: SUCCESSFUL
This will indicate that the Python DLL’s were copied to the C:\windows\system32 directory. If you see this message manually copy the DLL’s from C:\windows\system32 directory to the proper C:\windows\sysWOW64 directory:
copy C:\windows\system32\python25.dll C:\windows\sysWOW64\python25.dll
copy C:\windows\system32\pythoncom25.dll C:\windows\sysWOW64\pythoncom25.dll
copy C:\windows\system32\pywintypes25.dll C:\windows\sysWOW64\pywintypes25.dll
Copyright
Copyright © 2003-2010 Altair Engineering, Inc. All rights reserved.
PBS™, PBS Works™, PBS GridWorks®, PBS Professional®, PBS Analytics™, PBS Catalyst™, e-Compute™, and e-Render™ are trademarks of Altair Engineering, Inc. and are protected under U.S. and international laws and treaties. All other marks are the property of their respective owners.
ALTAIR ENGINEERING INC. Proprietary and Confidential. Contains Trade Secret Information. Not for use or disclosure outside ALTAIR and its licensed clients. Information contained herein shall not be decompiled, disassembled, duplicated or disclosed in whole or in part for any purpose. Usage of the software is only as explicitly permitted in the end user software license agreement.
Copyright notice does not imply publication.
|