Chapter 8 Writing Log Operations

Sun Java^TM System Access Manager7 2005Q4 provides a Logging Service for recording information such as user activity, traffic patterns, and authorization violations. The Access Manager Logging APIs enable external applications to take advantage of the Logging Service.

For information about how the Logging Service works and what it logs, see Chapter 6, Logging, in Sun Java System Access Manager 7 2005Q4 Technical Overview. This chapter describes how to use the Logging APIs to write log operations and customize logging plug-ins. Topics in this chapter include:

• About the Logging Samples

• Writing LogRecords To A Log File or Table

• Reading LogRecords From A Log File or Table

• Compiling Logging Programs

• Implementing a Remote Logging Application in a Container

• Logging to a Second Access Manager Server

• Using the Logging Sample Files

• Using the Logging SPIs

About the Logging Samples

Access Manager provides two comprehensive Logging example programs in the AccessManager-base /SUNWam/samples/logging directory. LogSample.java is a log-writing program, and LogReaderSample.java is a log-reading program. The logging directory also includes the Makefile for compiling and scripts to facilitate running the programs.

Writing LogRecords To A Log File or Table

LogSample.java takes several command-line arguments, authenticates with the Access Manager server, creates a LogRecord, then logs the log record to the specified log file or table. The Access Manager Logging Service determines whether the log records go to a flat file or to a relational database management system (RDBMS), according to the service configuration. The following example command line uses the LogSample script:

./RunSample -o dc=iplanet,dc=com -u amadmin -p mypassword -n mylog \
            -m "my message to log in mylog" -l user1 -w user1password

In LogSample.java, the command-line arguments are read. The following arguments are used to acquire the SSOToken that is specified in invoking the LogRecord(loglevel, message, token) method:

-o

organization name

-u

userID

-p

userID password

The Logging Service extracts other pieces of information from this userID SSOToken when processing the LogRecord request. Ideally, the userID specified is the user who is the subject of the record being logged. The -m (message) argument is also used in the LogRecord call.

userToken =getSessionToken(orgname, args[userSID], args[userPWD]); 
logRecord = new LogRecord(java.util.logging.Level.INFO, args[message], userToken); 
logRecord.addLogInfo("ModuleName", "MyModule");

MyModule is added as the ModuleName property is added to the LogRecord using the addLogInfo() call. The -n (log name) argument is used in the Logger.getLogger(logname) call. The -l (logged by userID) and -w (logged by userID's password) are used to get the SSOToken specified in the logger.log(logRecord, loggedByToken) call. Where the userID associated with the LogRecord SSOToken is usually the subject of the log record, the userID associated with the log() SSOToken is the user doing the logging. In the actual log file, the values for the log record fields come from the following parameters:

time

added by the Logging Service, and is taken from the Access Manager system clock when the LogRecord is instantiated.

Data

The message as specified in the LogRecord() call. In LogSample.java, the value after the -m option: my message to log in mylog.

ModuleName

The value specified for the ModuleName property (or LogConstants.MODULE_NAME property in the addLogInfo() call. If no value is specified, this field will read:Not Available .

MessageID

The value specified for theMessageID property (or LogConstants.MESSAGE_ID property in an addLogInfo() call. If no value is specified, this field will read: Not Available. LogSample.java does not add a value for this property.

Domain

The value for this field is extracted from the SSOToken specified in the LogRecord() call. This corresponds to the subject, userID's domain, or organization.

ContextID

The value for this field is extracted from the SSOToken specified in the LogRecord() call.

LogLevel

The value specified in the LogRecord() call. In LogSample.java, the value is java.util.logging.Level.INFO (INFO in the log file).

LoginID

The value for this field is extracted from the SSOToken specified in the LogRecord()call. For example, the value can be the DN for the userID specified in the -u command-line option.

IPAddr

The value for this field is extracted from the SSOToken specified in the LogRecord() call.

LoggedBy

The value for this field is extracted from the SSOToken specified in the logger.log() call. For example, the value can be the DN for the userID specified in the -l command-line option.

HostName

The value for this field is extracted from the SSOToken specified in the LogRecord() call. The value is the host name that corresponds to the address in the IPAddr field, if it can be resolved.

Reading LogRecords From A Log File or Table

The log writing sample program LogSample.java is fairly straightforward in the way the program writes a single record to a file or table as determined by the Logging Service's configuration. In contrast, the log reading sample program is more complex because you can specify that queries are applied to multiple files or tables.

Log files and tables in particular can become very large. If you specify multiple logs in a single query, create queries that are very specific, or limited in the number of records to return, or both specific and limited. If a large number of records are returned, the Access Manager resource limits (including those of the hosting system) may be exceeded.

LogReaderSample.java requires three command-line arguments which are used to authenticate with the Access Manager server. If you specify a log name, then the sample becomes a single-log reading application. If you don't specify a log name, reading from multiple logs is allowed. Reading from multiple logs does not preclude reading from a single log. Reading from multiple logs is useful when the exact log names available are unknown. The log reading sample is also very interactive. The following command-line example uses the LogReaderSample script:

./RunLogReader -o dc=iplanet,dc=com -u amadmin -p mypassword

In LogReaderSample.java, the command-line arguments are read. The following arguments are used to obtain the SSOToken that is specified in invoking the various LogReader.read() methods:

-o

organization name

-u

userID

-p

userID password

The LDAP login utility ldapLogin() is provided in a separate file, LogSampleUtils.java.

Next, the Logging Service configuration is read to determine, for example, whether file or database logging is specified and which log fields are logged.

manager.readConfiguration();     
String logStorageType = manager.getProperty(LogConstants.BACKEND);

Depending on whether Access Manager Logging Service is logging to a file or to a database, when the LogReader.getSize() method is invoked on a particular log name, LogReader.getSizeUnits() will return either LogConstants.NUM_BYTES or LogConstants.NUM_RECORDS. For example:

 i3 = LogReader.getSizeUnits(); 

The LogConstants.LOG_FIELDS property specifies which log fields have been specified for inclusion in the log record. For example:

String selFldsStr = manager.getProperty(LogConstants.LOG_FIELDS);

The time and Data fields are mandatory, thus they do not appear in the Logging Service list. They must be explicitly added to the Set of Fields to Retrieve.

	StringTokenizer stoken = new StringTokenizer(selFldsStr, ", ");
					String [] sFields = new String[stoken.countTokens() + 3];     
					Set allFields = new HashSet();

					allFields.add("time");
					allFields.add("data");

To get the Set of Log Names Available to read and their sizes:

	Set filesThereAre = LogReader.getLogNames();
for (Iterator it=filesThereAre.iterator(); it.hasNext(); ) {
	String fileName = (String)it.next();
	long li = 0;
	try {
		li = LogReader.getSize(fileName);
	} catch (Exception ex) {             
		System.out.println("got exception on file " +
				fileName + ". " + ex.getMessage());
	}         
	System.out.println (fileOrTable + " " + (i2++) +
		" = " + fileName + " contains " + li + " " +
		sizeUnit + ".");     
}

LogReaderSample.java allows you to select reads on a single or multiple logs. If a log name was specified on the command line with the -n option, then you can select from among the following types of reads:

	
			1. read all records
			2. specify logType
			3. specify logType and timeStamp
			4. specify logType and logQuery
			5. specify logType, timeStamp, and logQuery
			6. specify logQuery

If no log name was specified on the command line, and you select single log to read, you may select from only a list of pre—configured reports:

	Single (s) or multiple (m) file/table read: [s]
			What type of audit report to generate:
			 1. all records from file/table
			 2. authentication successes
			 3. authentication failures
			 4. login/logout activity
			 5. policy allows
			 6. policy denies
			 7. amAdmin CLI activity
			 8. amAdmin console activity
			 9. Federation access
			10. Federation errors
			11. Liberty access
			12. Liberty errors
			13. SAML access
			14. SAML error          
					enter type [1..14]: 

If you want to read from a selected single log, but specify the logQuery settings, do not use the -n command-line option. Select multiple log read, and then select the single log from which to read:

	Available files:
			file 0 = amAuthentication.access contains 1595 bytes.
			file 1 = amPolicy.access contains 2515 bytes.
			...
			file 13 = amAuthentication.error contains 795 bytes.

			Single (s) or multiple (m) file/table read: [s] m

			Available files:
			0: amAuthentication.access			
			1: amPolicy.access
			...
			12: amConsole.access-1					
			13: amAuthentication.error

			Enter selections (space-separated): 0
			What type of read to use: 
				1. read all records
				2. specify logQuery          
					 enter type [1 or 2]:  

The following table provides brief descriptions of the LogReader.read() methods.

read(String fileName,
         Object userCrdential)

read(String logName,
         String logType,
         Object userCrdential)

read(String logName,
	String logType,
	String timeStamp,
	Object userCrdential)

read(String logName,
	String logType,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	String logType,
	String timeStamp,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	LogQuery logQuery,
	Object userCrdential)

read(String logName,
	Set fileNames,
	LogQuery logQuery,
	Object userCrdential)

The LogQuery, along with the QueryElements that may be specified, are constructed in the getLogQuery() routine in LogReaderSample.java.

The following are brief descriptions of the LogQuery constructors.

LogQuery()

Creates a new LogQuery object with the following default values:

maxRecord = 
	LogQuery.MOST_RECENT_MAX_RECORDS 	 
	globalOperand = 
	LogQuery.MATCH_ANY_CONDITION 	    
queries = null (QueryElement) 	    
columns = null (columns to return)
sortBy = null (field to sort on)

LogQuery(int max_record)

Creates a new LogQuery object with the following values:

maxRecord = max_record
globalOperand = 	LogQuery.MATCH_ANY_CONDITION
queries = null (QueryElement)
columns = null (columns to return)
sortBy = null (field to sort on)

LogQuery(int max_Record, int matchCriteria, java.lang.String sortingBy)

Creates a new LogQuery object with the following values:

maxRecord = max_Record 	    
globalOperand = matchCriteria 	    
queries = null (QueryElement) 	    
columns = null (columns to return) 	    
sortBy = sortingBy (field to sort on)

The LogQuery object created with the constructors may be subsequently modified with the following set* methods:

• setColumns(java.util.ArrayList columns)

• setGlobalOperand(int no)

• setMaxRecord(int value)

• setSortingField(java.lang.String fieldName)

A LogQuery may specify a List of QueryElements, each containing a value for a field (column) and a relationship. The following sample code queries for all successful authentications in domain dc=iplanet,dc=com, and returns the time, Data, MessageID, ContextID, LoginID, and Domain fields, sorted on the LoginID field:

ArrayList al = new ArrayList();
al.add (LogConstants.TIME);
al.add (LogConstants.Data);
al.add (LogConstants.MESSAGE_ID);
al.add (LogConstants.CONTEXT_ID);
al.add (LogConstants.LOGIN_ID);
al.add (LogConstants.DOMAIN);
LogQuery lq = new LogQuery(LogQuery.ALL_RECORDS,
		LogQuery.MATCH_ALL_CONDITIONS,
		LogConstants.LOGIN_ID);

QueryElement qe1 = new QueryElement(LogConstants.MESSAGE_ID,
		"AUTHENTICATION-105",
		QueryElement.EQ);
lq.addQuery(qe1);

QueryElement qe2 = new QueryElement(LogConstants.DOMAIN,
		"dc=iplanet,dc=com",
		QueryElement.EQ);
lq.addQuery(qe2);

QueryElement supports the following relationships:

QueryElement.GT

Greater than

QueryElement.LT

Less than

QueryElement.EQ

Equal to

QueryElement.NE

Not equal to

QueryElement.GE

Greater than or equal to

QueryElement.LE

Less than or equal to

QueryElement.CN

Contains

QueryElement.SW

Starts with

QueryElement.EW

Ends with

In the example, assuming that dc=iplanet,dc=com is the root domain, changing the qe2relationship field to QueryElement.EW (Ends with) or QueryElement.CN (Contains) changes the query to include all successful authentications in all domains. To read the example query from the amAuthentication.access log, assuming the SSOToken is in ssoToken:

 String[][] result = new String[1][1];
    result = read("amAuthentication.access", lq, ssoToken);

The first record (row 0) contains the field and column names. See the printResults() method in LogReaderSample.java for a sample display routine.

Compiling Logging Programs

Included with the sample log programs is a gmake Makefile which compiles both LogSample.java and LogReaderSample.java, as well as the utilities module LogSampleUtils.java. The item of most interest is the CLASSPATH setting.

Executing Logging Programs

The sample standalone log programs include ksh scripts. There are considerations for running on Solaris or Linux handled by the scripts, but a few less obvious settings concern whether there is local or remote logging, if database logging is configured, and if Access Manager is configured for SSL. The LOCAL_LOGGING shell variable is set to true by default. If the logging program is executing on a remote system, using the Access Manager client APIs, then the LOCAL_LOGGING shell variable this must be set to false. The LOCAL_LOGGING setting later determines the setting of the CONFIGOPTION variable. When the logging program is running on the same system as the Access Manager server, and logging to a database is configured, then the database JDBC driver must also be included in the CLASSPATH. If the Access Manager server is configured for SSL, and the logging program is executing on a remote system using the Access Manager client APIs, be sure that the following parameter is set in the script:

-D"java.protocol.handler.pkgs=com.iplanet.services.comm"

The certificate database conforming to the Access Manager server container must be provided, and the com.iplanet.am.admin.cli.certdb.dir property in the AMConfig.properties file must point to the Access Manager server container. For example, for non-production testing to an Access Manager server running in a Application Server 8.1 container, you can copy (assuming default installation of AS 8.1) /var/opt/SUNWappserver/domains/domain1/config to the remote system, and set com.iplanet.am.admin.cli.certdb.dir to that location. You must also set the following:

com.iplanet.am.admin.cli.certdb.prefix=
    com.iplanet.am.admin.cli.certdb.passfile=/etc/opt/SUNWam/config/.wtpass

The .wtpass file needs to be created. More detailed information about certificates, see the file AccessManager-base/SUNWam/samples/authentication/api/Readme_setup.html .

Implementing a Remote Logging Application in a Container

If your remote logging application is running in a container such as Sun Java System Application Server or Web Server, at the command line, set the following properties:

-Ds1is.java.util.logging.config.class=
			com.sun.identity.log.s1is.LogConfigReader

-DLOG_COMPATMODE=Off

-Djava.util.logging.manager=
			com.iplanet.ias.server.logging.ServerLogManager

The -Djava.util.logging.manager property occurs in the Java System Web Server server.xml file. JVM options are typically added to the server.xml file in Java System Web Server, or to the domain.xml file in Java System Application Server.

Setting Environment Variables

You must set the following shared library environment variables in the executable for an application that is using the Logging Service. You can determine how to set the variables depending upon three things:

• Whether the application can execute in the local Access Manager server, or executes only a in remote server

• Whether or not you want the Access Manager LogManager class to override the native LogManager class

• Whether or not SSL is enabled in your deployment

If Client Can Execute in the Local Access Manager Server

When the client application can execute in either the local Access Manager server JVM or in a remote server JVM, choose one of the following two configurations:

• If it is acceptable for the native LogManager class to be overridden by the Access Manager LogManager class in the JDK1.4 environment, then set the following variables:

  -D"java.util.logging.manager=com.sun.identity.log.LogManager"
  -D"java.util.logging.config.class=com.sun.identity.log.
  			s1is.LogConfigReader"
  

• If it is not acceptable for the native LogManager class to be overridden by the Access Manager LogManager class in the JDK1.4 environment, then set the following variables:

  -DLOG_COMPATMODE=Off
  -Ds1is.java.util.logging.config.class=com.sun.identity.log.
  			s1is.LogConfigReader
  

If Client Executes Only in a Remote Server

When the client application can execute only in a remote server JVM, choose one of the following two configurations:

• If it is acceptable for the native LogManager class to be overridden by the Access Manager LogManager class in the JDK1.4 environment, then follow these steps:

  1. Set the following variables:

     -Djava.util.logging.manager=com.sun.identity.log.LogManager

     -Djava.util.logging.config.file=/AccessManager_base/SUNwam/
     		lib/LogConfig.properties
     

  2. In LogConfig.properties, or in the logging.properties file supplied by JDK, set the following properties:

     iplanet-am-logging-remote-handler=com.sun.identity.log.handlers. 
     		RemoteHandler
     

     iplanet-am-logging-remote-formatter=com.sun.identity.log. 
     		handlers.RemoteFormatter
     

     iplanet-am-logging-remote-buffer-size=1

     iplanet-am-logging-buffer-time-in-seconds=3600

     iplanet-am-logging-time-buffering-status=OFF

• If it is not acceptable for the native LogManager class to be overridden by the Access Manager LogManager class in the JDK1.4 environment, then follow these steps:

  1. Set the following variables:

     -DLOG_COMPATMODE=Off

     -Ds1is.java.util.logging.config.file=/AccessManager-base/SUNwam/lib/LogConfig.properties

  2. In LogConfig.properties, or in the logging.properties file supplied by JDK, set the following properties:

     iplanet-am-logging-remote-handler=com.sun.identity.log.
     			handlers.RemoteHandler
     iplanet-am-logging-remote-formatter=com.sun.identity.log.
     			handlers.RemoteFormatter
     

     iplanet-am-logging-remote-buffer-size=1

     iplanet-am-logging-buffer-time-in-seconds=3600

     iplanet-am-logging-time-buffering-status=OFF

The Client APIs use this logging configuration by default. In this case, the Logging API will configure a remote handler for all logs. Access to the Directory Server is not required in this case.

If SSL is Enabled

If SSL is enable and uses JSS for Access Manager, set the following parameter:

-D"java.protocol.handler.pkgs=com.iplanet.services.comm"

Logging to a Second Access Manager Server

For a remote Access Manager server to use another Access Manager server's logging service, set the Logging Service URL in the remote Access Manager server Naming Service to specify the Access Manager server that will be performing the actual logging. User the following form:

http://host:port/amserver/loggingservice

Using the Logging Sample Files

The sample files demonstrate how you can use the Access Manager Logging APIs for to log operations. You can execute the samples through the command line. You must have super user privileges to run the RunSample and RunLogReader programs and to access AMConfig.properties.

To Run the Sample Programs on Solaris

1. In the Makefile, RunSample, and RunLogReader files, set the following variables. The variables may already have been set during installation.

   AM_HOME

   Set this to refer to the where Access Manager server is installed.

   JAVA_HOME

   Set this variable to your installation of the JDK. The JDK version should be greater than or equal to 1.3.1_06.

   JDK14

   Set this variable to true if your JAVA_HOME points to JDK 1.4 or newer version else set it to false

   LOCAL_LOGGING

   Set this variable to true if you are executing this sample at complete Access Manager installation which will perform local logging. If you are executing this sample from a SUNWamsdk only install then set it to false which will perform remote logging (logging at server side).

2. Set the LD_LIBRARY_PATH as is appropriate for your installation.

3. Run the gmake command to compile the sample program.

4. Run the following chmod command:

   chmod +x RunSample RunLogReader

5. Run the following command to run the logging sample program:

   ./RunSample [ -o organizationName] [ -u userName -p userPassword ] -n logName -m message -l loggedByUser -w loggedByUserPassword

   orgName

   Name of the organization. This is an optional parameter. If a value is not provided, Access Manager assumes the value to be the root organization.

   userName

   Name of the user on whose behalf the logging is performed. This is an optional parameter.

   userPassword

   Password for authenticating the user. This value must be provided if userName is provided.

   logName

   Name of the log file.

   message

   Message to be logged to the log file.

   loggedByUser

   Name of the administrator user who is logging the message.

   loggedByUserPassword

   Password to authenticate the administrator user.

   Example:

   $ ./RunSample -u amadmin -p 11111111 -n testLog.access -m "trying test logging" -l amadmin -w 11111111

6. Run the log reader program by running the following command:

   ./RunLogReader -o organizationName -u userName 
   			-p  userPassword 	[-n logName]
   

   organizationName

   Name of the organization. This is a required parameter.

   username

   Name of the user who is accessing the log file or table. This is a required parameter.

   userpassword

   Password to authenticate the user. This is a required parameter.

   logName

   Name of the log file or table. This parameter is optional. You can select the log file or table when running the program.

   Example :

   $ ./RunLogReader -u amadmin -p 11111111 -o dc=example,dc=com
   			 -n testLog.access
   

To Run the Sample Programs on Windows 2000

1. In the make.bat file, set the following variables:

   BASE

   Set this to refer to the where Access Manager server is installed.

   JAVA_HOME

   Set this variable to your installation of the JDK. The JDK version should be greater than or equal to 1.3.1_06.

   JDK14

   Set this variable to true if your JAVA_HOME points to JDK 1.4 or newer version. Otherwise, set it to false.

   LOCAL_LOGGING

   Set this variable to true if you are executing this sample at complete Access Manager installation which will perform local logging. If you are executing this sample from an SUNWamsdk only install then set it to false which will perform remote logging (logging at server side).

2. Set the LD_LIBRARY_PATH as is appropriate for your installation.

3. Compile the program by running the make command.

4. Run the sample program by running the make run command:

   make run [-o organizationName]
   			[-u userName -p  userPassword]	-n logName  
   						-m message -l  loggedByUser 
   										-wloggedByUserPassword
   

   orgName

   Name of the organization. This is an optional parameter. If a value is not provided, Access Manager assumes the value to be the root organization.

   userName

   Name of the user on whose behalf the logging is performed. This is an optional parameter.

   userPassword

   Password for authenticating the user. This value must be provided if userName is provided.

   logName

   Name of the log file.

   message

   Message to be logged to the log file.

   loggedByUser

   Name of the administrator user who is logging the message.

   loggedByUserPassword

   Password to authenticate the administrator user.

   Example:

   c> make run -u amadmin -p 11111111 -n testLog.access 
   			-m "trying test logging" -l amadmin -w 11111111
   

Using the Logging SPIs

The Logging SPI are Java packages that can be used to develop plug-ins for customized features. The SPI are organized in the com.sun.identity.log.spi package. For more information, see the Sun Java System Access Manager 7 2005Q4 Java API Reference.

Log Verifier Plug-In

If secure logging is enabled, the log files are verified periodically to detect any attempt of tampering. If tampering is detected, the action taken can be customized by following the steps.

To Customize Actions to be Taken in Secure Logging

1. Implement the com.sun.identity.log.spi.IVerifierOutput interface with the desired functionality.

2. Add the implementing class in the classpath of Access Manager.

3. Modify the property iplanet-am-logging-verifier-action-class in the /etc/opt/SUNWam/config/xml/amLogging.xml file with the name of the new class.

Log Authorization Plug-In

The Logging Service enables you to plug in a class that will determine whether a LogRecord is logged or discarded. The determination is based on the authorization of the owner of the session token performing the event.

The IAuthorizer interface accepts an SSOToken and the log record being written.

There are several ways to accomplish this. The following procedure is one example.

To Implement a Log Authorization Plug-In

1. Get the applicable role or DN of the user from the SSOToken and check it against a pre-configured (or hardcoded) list of roles or users that are allowed access.

   The administrator must configure a role and assign all policy agents and entities such as applications that can possibly log into Access Manager and into this role.

2. Instantiate a PolicyEvaluator and call PolicyEvaluator.isAllowed(ssotoken, logname);.

To Instantiate a PolicyEvaluator

This entails defining a policy XML to model log access and registering it with Access Manager.

1. Implement the com.sun.identity.log.spi.IAuthorizer interface with the desired functionality.

2. Add the implementing class in the classpath of Access Manager.

3. Modify the property iplanet-am-logging-authz-class in the /etc/opt/SUNWam/config/xml/amLogging.xml file with the name of the new class.