Chapter 10 System Accounting
(Reference)
This chapter provides reference information about system accounting.
This is a list of the reference information in this chapter.
For more information about system accounting tasks, see Chapter 9, Managing System Accounting (Tasks).
runacct Script
The
main daily accounting script, runacct, is normally invoked
by the cron command outside of normal business hours. The runacct script processes connect, fee, disk, and process accounting
files. This script also prepares daily and cumulative summary files for use
by the prdaily and monacct scripts for
billing purposes.
The runacct script takes care not
to damage files if errors occur.
A series of protection mechanisms that are used to perform the following
tasks:
This script records its progress by writing descriptive messages to
the active file. Files used by the runacct script
are assumed to be in the /var/adm/acct/nite directory,
unless otherwise noted. All diagnostic output during the execution of the runacct script is written to the fd2log file.
When the runacct script is invoked, it creates
the lock and lock1 files. These
files are used to prevent simultaneous execution of the runacct script.
The runacct program prints an error message if these files
exist when it is invoked. The lastdate file contains
the month and day the runacct script was last invoked,
and is used to prevent more than one execution per day.
If the runacct script detects an error, the following
occurs:
For instructions on how to restart the runacct script,
see How to Restart the runacct Script.
To allow the runacct script to be restarted, processing is broken down into separate
re-entrant states. The statefile file is used to track
the last state completed. When each state is completed, the statefile file
is updated to reflect the next state. After processing for the state is complete,
the statefile file is read and the next state is processed.
When the runacct script reaches the CLEANUP state,
it removes the locks and ends. States are executed as shown in the following
table.
Table 10–1 States of the
runacct Script
|
State
|
Description
|
|
SETUP
|
The turnacct switch command is
executed to create a new pacct file. The /var/adm/pacctn process accounting files (except for
the pacct file) are moved to the /var/adm/Spacctn.MMDD files. The /var/adm/wtmpx file is
moved to the /var/adm/acct/nite/wtmp.MMDD file
(with the current time record added on the end) and a new /var/adm/wtmp file is created. The closewtmp and utmp2wtmp programs add records to the wtmp.MMDD file and the new wtmpx file to account
for users who are currently logged in.
|
|
WTMPFIX
|
The wtmpfix program checks the wtmp.MMDD file in the nite directory for accuracy.
Because some date changes cause the acctcon program to
fail, the wtmpfix program attempts to adjust the time stamps
in the wtmpx file if a record of a date change appears.
This program also deletes any corrupted entries from the wtmpx file.
The fixed version of the wtmp.MMDD file
is written to the tmpwtmp file.
|
|
CONNECT
|
The acctcon program is used to record connect accounting
records in the file ctacct.MMDD.
These records are in tacct.h format. In addition, the acctcon program creates the lineuse and reboots files. The reboots file records all
the boot records found in the wtmpx file.
|
|
PROCESS
|
The acctprc program is used to convert the /var/adm/Spacctn.MMDD process accounting files into
complete accounting records in the ptacctn.MMDD files. The Spacct and ptacct files
are correlated by number so that if the runacct script
fails, the Spacct files are not processed.
|
|
MERGE
|
The acctmerg program merges the process accounting
records with the connect accounting records to form the daytacct file.
|
|
FEES
|
The acctmerg program merges ASCII tacct records
from the fee file into the daytacct file.
|
|
DISK
|
The dodisk script produces the disktacct file.
If the dodisk script has been run, which produces the disktacct file, the DISK program merges the
file into the daytacct file and moves the disktacct file to the /tmp/disktacct.MMDD file.
|
|
MERGETACCT
|
The acctmerg program merges the daytacct file
with the sum/tacct file, the cumulative total accounting
file. Each day, the daytacct file is saved in the sum/tacct.MMDD file so that the sum/tacct file can be re-created if it is corrupted or lost.
|
|
CMS
|
The acctcms program is run several times. This program
is first run to generate the command summary by using the Spacctn files and write the data to the sum/daycms file.
The acctcms program is then run to merge the sum/daycms file with the sum/cms cumulative command
summary file. Finally, the acctcms program is run to produce nite/daycms and nite/cms, the ASCII command
summary files from the sum/daycms and sum/cms files,
respectively. The lastlogin program is used to create the /var/adm/acct/sum/loginlog log file. This file reports when each
user last logged in. If the runacct script is run after
midnight, the dates showing the time last logged in by some users will be
incorrect by one day.
|
|
USEREXIT
|
Any installation-dependent (local) accounting program can be run at
this point. The runacct script expects this program to
be called the /usr/lib/acct/runacct.local program.
|
|
CLEANUP
|
This state cleans up temporary files, runs the prdaily script
and saves its output in the sum/rpt.MMDD file,
removes the locks, and then exits.
|

Caution –
When restarting the runacct script in the CLEANUP state, remove the
last ptacct file because this file will not be complete.
Daily Accounting Reports
The runacct shell script generates five basic
reports upon each invocation. The following table describes these reports.
Table 10–2 Daily Accounting Reports
|
Report Type
|
Description
|
|
Daily Report
|
Shows terminal line utilization by tty number.
|
|
Daily Usage Report
|
Indicates usage of system resources by users (listed in order of user
ID).
|
|
Daily Command Summary
|
Indicates usage of system resources by commands, listed in descending
order of memory use. In other words, the command that used the most memory
is listed first. This same information is reported for the month in the monthly
command summary.
|
|
Monthly Command Summary
|
A cumulative summary that reflects the data accumulated since the last
invocation of the monacct program.
|
|
Last Login Report
|
Shows the last time each user logged in (listed in chronological order).
|
Daily Report
This report gives information
about each terminal line used. The following is a sample Daily Report.
Jan 16 02:30 2004 DAILY REPORT FOR venus Page 1
from Mon Jan 15 02:30:02 2004
to Tue Oan 16 02:30:01 2004
1 runacct
1 acctcon
TOTAL DURATION IS 1440 MINUTES
LINE MINUTES PERCENT # SESS # ON # OFF
console 868 60 1 1 2
TOTALS 868 -- 1 1 2
|
The from and to lines
specify the time period reflected in the report. This time period covers the
time the last Daily Report was generated to the time the current Daily Report
was generated. Then, the report presents a log of system reboots, shutdowns,
power failure recoveries, and any other record written to the /var/adm/wtmpx file by the acctwtmp program. For more information,
see the acct(1M) man
page.
The second part of the report is a breakdown of terminal line utilization.
The TOTAL DURATION tells how long the
system was in multiuser mode (accessible through the terminal lines). The
following table describes the data provided by the Daily Report.
Table 10–3 Daily Report Data
|
Column
|
Description
|
|
LINE
|
The terminal line or access port.
|
|
MINUTES
|
The number of minutes that the line was in use during the accounting
period.
|
|
PERCENT
|
The TOTAL DURATION divided by
the number of MINUTES.
|
|
# SESS
|
The number of times this line or port was accessed for a login session.
|
|
# ON
|
Same as SESS. (This column no longer has meaning.
Previously, this column listed the number of times that a line or port was
used to log in a user.)
|
|
# OFF
T
|
The
number of times a user logs out and any interrupts that occur on that line.
Generally, interrupts occur on a port when ttymon is first
invoked after the system is brought to multiuser mode. If the # OFF exceeds
the # SESS by a large factor, the multiplexer, modem,
or cable is probably going bad. Or, a bad connection exists somewhere. The
most common cause is an unconnected cable dangling from the multiplexer.
|
During real time, you should monitor the /var/adm/wtmpx file
because it is the file from which the connect accounting is derived. If the wtmpx file grows rapidly, execute the following command to see
which tty line is the noisiest.
# /usr/lib/acct/acctcon -l file < /var/adm/wtmpx
|
If interruption is occurring frequently, general system performance
will be affected. Additionally, the wtmp file might become
corrupted. To correct this problem, see How to Fix a Corrupted wtmpx File.
Daily Usage Report
The Daily Usage Report breaks down system resource utilization
by user. A sample of this report follows.
Jan 16 02:30 2004 DAILY USAGE REPORT FOR skisun Page 1
LOGIN CPU (MINS) KCORE- MINS CONNECT (MINS) DISK # OF # OF # DISK FEE
UID NAME PRIME NPRIME PRIME NPRIME PRIME NPRIME BLOCKS PROCS SESS SAMPLES
0 TOTAL 72 148 11006173 51168 26230634 57792 539 330 0 2150 1
0 root 32 76 11006164 33664 26230616 22784 0 0 0 127 0
4 adm 0 0 22 51 0 0 0 420 0 0 0
101 rimmer 39 72 894385 1766020 539 330 0 1603 1 0 0
|
The following table describes the data provided by the Daily Usage Report.
Table 10–4 Daily Usage Report Data
|
Column
|
Description
|
|
UID
|
User ID number.
|
|
LOGIN NAME
|
Login (or user) name of the user. Identifies a user who has multiple
login names.
|
|
CPU (MINS)
|
Amount of time, in minutes, that the user's process used the central
processing unit. Divided into PRIME and NPRIME (nonprime)
utilization. The accounting system's version of this data is located in the /etc/acct/holidays file.
|
|
KCORE-MINS
|
A cumulative measure of the amount of memory in Kbyte segments per minute
that a process uses while running. Divided into PRIME and NPRIME utilization.
|
|
CONNECT (MINS)
|
Amount of time, in minutes, that the a user was logged in to the system,
or “real time.” Divided into PRIME and NPRIME utilization. If these numbers are high while the # OF
PROCS is low, you can conclude that the user logs in first thing
in the morning and hardly touches the terminal the rest of the day.
|
|
DISK BLOCKS
|
Output from the acctdusg program,
which runs the disk accounting programs and merges the accounting records
(daytacct). For accounting purposes, a block is 512 bytes.
|
|
# OF PROCS
|
Number of processes invoked by the user. If large numbers appear, a
user might have a shell procedure that has run out of control.
|
|
#
OF SESS
|
Number of times that a user logged in to the system.
|
|
# DISK SAMPLES
|
Number of times that disk accounting was run to obtain the average number
of DISK BLOCKS.
|
|
FEE
|
Often unused field that represents the total accumulation of units
charged against the user by the chargefee script.
|
Daily Command Summary
The Daily Command
Summary report shows the system resource utilization by command. With this
report, you can identify the most heavily used commands. Based on how those
commands use system resources, you can then gain insight on how best to tune
the system.
These reports are sorted by TOTAL KCOREMIN,
which is an arbitrary gauge but often useful for calculating drain on a system.
A sample Daily Command Summary follows.
TOTAL COMMAND SUMMARY
COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS
NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ
TOTALS 2150 1334999.75 219.59 724258.50 6079.48 0.10 0.00 397338982 419448
netscape 43 2456898.50 92.03 54503.12 26695.51 2.14 0.00 947774912 225568
adeptedi 7 88328.22 4.03 404.12 21914.95 0.58 0.01 93155160 8774
dtmail 1 54919.17 5.33 17716.57 10308.94 5.33 0.00 213843968 40192
acroread 8 31218.02 2.67 17744.57 11682.66 0.33 0.00 331454464 11260
dtwm 1 16252.93 2.53 17716.57 6416.05 2.53 0.00 158662656 12848
dtterm 5 4762.71 1.30 76300.29 3658.93 0.26 0.00 33828352 11604
dtaction 23 1389.72 0.33 0.60 4196.43 0.01 0.55 18653184 539
dtsessio 1 1174.87 0.24 17716.57 4932.97 0.24 0.00 23535616 5421
dtcm 1 866.30 0.18 17716.57 4826.21 0.18 0.00 3012096 6490
|
The following table describes the data provided by the Daily Command
Summary.
Table 10–5 Daily Command Summary
Data
|
Column
|
Description
|
|
COMMAND NAME
|
Name of the command. All shell procedures are lumped together under
the name sh because only object modules are reported by
the process accounting system. You should monitor the frequency of programs
called a.out or core, or any other
unexpected name. You can use the acctcom program to determine
who executed an oddly named command and if superuser privileges were used.
|
|
NUMBER CMDS
|
Total number of times this command was run.
|
|
TOTAL KCOREMIN
|
Total cumulative measurement of the Kbyte segments of memory used by
a process per minute of run time.
|
|
TOTAL CPU-MIN
|
Total processing time this program accumulated.
|
|
TOTAL REAL-MIN
|
Total real-time (wall-clock) minutes this program accumulated.
|
|
MEAN SIZE-K
|
Mean (average) of the TOTAL KCOREMIN over
the number of invocations reflected by the NUMBER CMDS.
|
|
MEAN CPU-MIN
|
Mean (average) derived from the NUMBER CMDS and
the TOTAL CPU-MIN.
|
|
HOG FACTOR
|
Total CPU time divided by elapsed time. Shows the ratio of system availability
to system utilization, providing a relative measure of total available CPU
time consumed by the process during its execution.
|
|
CHARS TRNSFD
|
Total number of characters transferred by the read and write system
calls. Might be negative due to overflow.
|
|
BLOCKS READ
|
Total number of the physical block reads and writes that a process performed.
|
Monthly Command Summary
The format of the Daily Command Summary and the Monthly Command Summary
reports are virtually the same. However, the daily summary reports only on
the current accounting period while the monthly summary reports on the start
of the fiscal period to the current date. In other words, the monthly report
is a cumulative summary that reflects the data accumulated since the last
invocation of the monacct program.
A
sample Monthly Command Summary follows.
Jan 16 02:30 2004 MONTHLY TOTAL COMMAND SUMMARY Page 1
TOTAL COMMAND SUMMARY
COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS
NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ
TOTALS 42718 4398793.50 361.92 956039.00 12154.09 0.01 0.00 16100942848 825171
netscape 789 3110437.25 121.03 79101.12 25699.58 0.15 0.00 3930527232 302486
adeptedi 84 1214419.00 50.20 4174.65 24193.62 0.60 0.01 890216640 107237
acroread 145 165297.78 7.01 18180.74 23566.84 0.05 0.00 1900504064 26053
dtmail 2 64208.90 6.35 20557.14 10112.43 3.17 0.00 250445824 43280
dtaction 800 47602.28 11.26 15.37 4226.93 0.01 0.73 640057536 8095
soffice. 13 35506.79 0.97 9.23 36510.84 0.07 0.11 134754320 5712
dtwm 2 20350.98 3.17 20557.14 6419.87 1.59 0.00 190636032 14049
|
For a description of the data provided by the Monthly Command Summary,
see Daily Command Summary.
Last Login Report
This report gives the date when a particular
login was last used. You can use this information to find unused logins and
login directories that can be archived and deleted. A Last Login Report follows.
Jan 16 02:30 2004 LAST LOGIN Page 1
01-06-12 kryten 01-09-08 protoA 01-10-14 ripley
01-07-14 lister 01-09-08 protoB 01-10-15 scutter1
01-08-16 pmorph 01-10-12 rimmer 01-10-16 scutter2
|
Examining the pacct File
With acctcom
At any time, you can examine the contents of the /var/adm/pacctn files, or any file with records in
the acct.h format, by using the acctcom program.
If you do not specify any files and do not provide any standard input when
you run this command, the acctcom command reads the pacct file. Each record read by the acctcom command
represents information about a terminated process. Active processes can be
examined by running the ps command.
The default output of the acctcom command provides
the following information:
# acctcom
COMMAND START END REAL CPU MEAN
NAME USER TTYNAME TIME TIME (SECS) (SECS) SIZE(K)
#accton root ? 02:30:01 02:30:01 0.03 0.01 304.00
turnacct adm ? 02:30:01 02:30:01 0.42 0.01 320.00
mv adm ? 02:30:01 02:30:01 0.07 0.01 504.00
utmp_upd adm ? 02:30:01 02:30:01 0.03 0.01 712.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 824.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 912.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 920.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 1136.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 576.00
closewtm adm ? 02:30:01 02:30:01 0.10 0.01 664.00
|
|
Field
|
Explanation
|
|
COMMAND NAME
|
Command name (pound (#) sign if the command was executed with superuser
privileges)
|
|
USER
|
User name
|
|
TTYNAME
|
tty name (listed as ? if unknown)
|
|
START TIME
|
Command execution starting time
|
|
END TIME
|
Command execution ending time
|
|
REAL (SECS)
|
Real time (in seconds)
|
|
CPU (SECS)
|
CPU time (in seconds)
|
|
MEAN SIZE (K)
|
Mean size (in Kbytes)
|
You can obtain the following information by using acctcom command
options.
The following table describes the acctcom command
options.
Table 10–6 Options for the
acctcom Command
|
Option
|
Description
|
|
-a
|
Shows average statistics about the processes selected. The statistics
are printed after the output is recorded.
|
|
-b
|
Reads the files backward, showing latest commands first. This option
has no effect if reading standard input.
|
|
-f
|
Prints the fork/exec flag and system exit status
columns. The output is an octal number.
|
|
-h
|
Instead of mean memory size, shows the hog factor, which is the fraction
of total available CPU time consumed by the process during its execution.
Hog factor = total-CPU-time/elapsed-time.
|
|
-i
|
Prints columns that contains the I/O counts in the output.
|
|
-k
|
Shows total kcore minutes instead
of memory size.
|
|
-m
|
Shows mean core size. This size is the default.
|
|
-q
|
Prints average statistics, not output records.
|
|
-r
|
Shows CPU factor: user-time/(system-time
+ user-time).
|
|
-t
|
Shows separate system and user CPU times.
|
|
-v
|
Excludes column headings from the output.
|
|
-C sec
|
Shows only processes with total CPU time (system plus user) that exceeds sec seconds.
|
|
-e time
|
Shows processes existing at or before time,
given in the format hr[:min[:sec]]
|
|
-E time
|
Shows processes starting at or before time,
given in the format hr[:min[:sec]]. Using the same time
for both -S and -E, shows processes that existed
at the time.
|
|
-g group
|
Shows only processes that belong to group.
|
|
-H factor
|
Shows only processes that exceed factor,
where factor is the “hog factor” (see
the -h option).
|
|
-I chars
|
Shows only processes that transferred more characters than the cutoff
number specified by chars.
|
|
-l line
|
Show only processes that belong to the terminal /dev/line.
|
|
-n pattern
|
Shows only commands that match pattern (a
regular expression except that “+” means one
or more occurrences).
|
|
-o ofile
|
Instead of printing the records, copies them in acct.h format
to ofile.
|
|
-O sec
|
Shows only processes with CPU system time that exceeds sec seconds.
|
|
-s time
|
Show processes existing at or after time,
given in the format hr[:min[:sec]].
|
|
-S time
|
Show processes starting at or after time,
given in the format hr[:min[:sec]].
|
|
-u user
|
Shows only processes that belong to user.
|
System Accounting Files
The /var/adm directory contains
the active data collection files. The following table describes the accounting
files in this directory.
Table 10–7 Files in the
/var/adm Directory
|
File
|
Description
|
|
dtmp
|
Output from the acctdusg program
|
|
fee
|
Output from the chargefee program, which are the
ASCII tacct records
|
|
pacct
|
Active process accounting file
|
|
pacctn
|
Process accounting files that are switched by running the turnacct script
|
|
Spacctn.MMDD
|
Process accounting files for MMDD during
execution of the runacct script
|
The /var/adm/acct directory contains the nite, sum,
and fiscal directories. These directories contain the
actual data collection files. For example, the nite directory
contains files that are reused daily by the runacct script.
A brief summary of the files in the /var/adm/acct/nite directory
follows.
Table 10–8 Files in the
/var/adm/acct/nite Directory
|
File
|
Description
|
|
active
|
Used by the runacct script to record progress and
print warning and error messages
|
|
active.MMDD
|
Same as the active file after the runacct script
detects an error
|
|
cms
|
ASCII total command summary used by the prdaily script
|
|
ctacct.MMDD
|
Connect accounting records in tacct.h format
|
|
ctmp
|
Output of acctcon1 program, which consists of connect
session records in ctmp.h format (acctcon1 and acctcon2 are provided for compatibility purposes)
|
|
daycms
|
ASCII daily command summary used by the prdaily script
|
|
daytacct
|
Total accounting records for one day in tacct.h format
|
|
disktacct
|
Disk accounting records in tacct.h format, created
by the dodisk script
|
|
fd2log
|
Diagnostic output during execution of the runacct script
|
|
lastdate
|
Last day the runacct script executed (in date
+%m%d format)
|
|
lineuse
|
tty line usage report used by the prdaily script
|
|
lock
|
Used to control serial use of the runacct script
|
|
log
|
Diagnostic output from the acctcon program
|
|
log.MMDD
|
Same as the log file after the runacct script
detects an error
|
|
owtmpx
|
Previous day's wtmpx file
|
|
reboots
|
Beginning and ending dates from the wtmpx file,
and a listing of reboots
|
|
statefile
|
Used to record current state during execution of the runacct script
|
|
tmpwtmp
|
wtmpx file corrected by the wtmpfix program
|
|
wtmperror
|
Contains wtmpfix error messages
|
|
wtmperror MMDD
|
Same as the wtmperror file after the runacct script
detects an error
|
|
wtmpMMDD
|
The runacct script's copy of the wtmpx file
|
The sum directory
contains the cumulative summary files updated by the runacct script
and used by the monacct script. The following table summarizes
the files in the /var/adm/acct/sum directory.
Table 10–9 Files in the
/var/adm/acct/sum Directory
|
File
|
Description
|
|
cms
|
Total command summary file for current fiscal period in binary format
|
|
cmsprev
|
Command summary file without latest update
|
|
daycms
|
Command summary file for the day's usage in internal summary format
|
|
loginlog
|
Record of last date each user logged in; created by the lastlogin script
and used in the prdaily script
|
|
rprt.MMDD
|
Saved output of prdaily script
|
|
tacct
|
Cumulative total accounting file for current fiscal period
|
|
tacctprev
|
Same as the tacct file without latest update
|
|
tacct.MMDD
|
Total accounting file for MMDD
|
The fiscal directory contains periodic summary files
that are created by the monacct script. The following table
summarizes the files in the /var/adm/acct/fiscal directory.
Table 10–10 Files in the
/var/adm/acct/fiscal Directory
|
File
|
Description
|
|
cmsn
|
Total command summary file for fiscal period n in
internal summary format
|
|
fiscrptn
|
Report similar to rprtn for
fiscal period n
|
|
tacctn
|
Total accounting file for fiscal period n
|
Files Produced by the runacct Script
The following table summarizes the most useful files
produced by the runacct script. These files are found in
the /var/adm/acct directory.
Table 10–11 Files Created by the
runacct Script
|
File
|
Description
|
|
nite/daytacct
|
The total accounting file for the day in tacct.h format.
|
|
nite/lineuse
|
The runacct script calls the acctcon program
to gather data on terminal line usage from the /var/adm/acct/nite/tmpwtmp file and writes the data to the /var/adm/acct/nite/lineuse file.
The prdaily script uses this data to report line usage.
This report is especially useful for detecting bad lines. If the ratio between
the number of logouts to logins is greater than three to one, the line is
very likely failing.
|
|
sum/cms
|
This file is the accumulation of each day's command summaries. The accumulation
restarts when the monacct script is executed. The ASCII
version is the nite/cms file.
|
|
sum/daycms
|
The runacct script calls the acctcms program
to process the commands used during the day to create the Daily Command Summary
report and stores the data in the /var/adm/acct/sum/daycms file.
The ASCII version is the /var/adm/acct/nite/daycms file.
|
|
sum/loginlog
|
The runacct script calls the lastlogin script
to update the last date logged in for the logins in the /var/adm/acct/sum/loginlog file. The lastlogin command also removes from
this file any logins that are no longer valid.
|
|
sum/rprt.MMDD
|
Each execution of the runacct script saves a copy
of the daily report that was printed by the prdaily script.
|
|
sum/tacct
|
Contains the accumulation of each day's nite/daytacct data
and is used for billing purposes. The monacct script restarts
accumulating this data each month or fiscal period.
|