Chapter 20 System Accounting (Reference)
This chapter provides reference information about system accounting.
This is a list of reference information in this chapter.
The runacct Script
The main daily accounting script, runacct, is normally
invoked by the cron command outside of prime business hours.
The runacct script processes connect, fee, disk, and process
accounting files. This script also prepares daily and cumulative summary files
for use by the prdaily and monacct scripts
for billing purposes.
The runacct script takes care not to damage files
if errors occur. A series of protection mechanisms are used to recognize an
error, provide intelligent diagnostics, and complete processing in such a
way that the runacct script can be restarted with minimal
intervention. It records its progress by writing descriptive messages into
the active file. Files used by the runacct
script are assumed to be in the /var/adm/acct/nite directory,
unless otherwise noted. All diagnostic output during the execution of the runacct script is written into the fd2log file.
When the runacct script is invoked, it creates the lock and lock1 files. These files are used
to prevent simultaneous execution of runacct. The runacct program prints an error message if these files exist when
it is invoked. The lastdate file contains the month and
day the runacct script was last invoked, and is used to
prevent more than one execution per day. If the runacct
script detects an error, a message is written to the console, mail is sent
to root and adm, locks might be removed,
diagnostic files are saved, and execution is ended. For instructions on how
to start the runacct script again, see How to Restart the runacct Script.
To allow the runacct script to be restarted, processing
is broken down into separate re-entrant states. The statefile
file is used to keep track of the last state completed. When each state is
completed, the statefile file is updated to reflect the
next state. After processing for the state is complete, the statefile file is read and the next state is processed. When the runacct script reaches the CLEANUP state, it
removes the locks and ends. States are executed as shown in the following
table.
Table 20–1
runacct States
|
State
|
Description
|
|
SETUP
|
The turnacct switch command is executed to create a new pacct
file. The /var/adm/pacctn
process accounting files (except for the pacct file)
are moved to the /var/adm/Spacctn.MMDD
files. The /var/adm/wtmpx file is moved to the /var/adm/acct/nite/wtmp.MMDD file (with
the current time record added on the end) and a new /var/adm/wtmp file is created. The closewtmp and utmp2wtmp programs add records to the wtmp.MMDD file and the new wtmpx file to account
for users currently logged in.
|
|
WTMPFIX
|
The wtmpfix program
checks the wtmp.MMDD file
in the nite directory for accuracy. Because some date
changes cause the acctcon program to fail, the wtmpfix program attempts to adjust the time stamps in the wtmpx file if a record of a date change appears. This program also
deletes any corrupted entries from the wtmpx file. The
fixed version of the wtmp.MMDD
file is written to the tmpwtmp file.
|
|
CONNECT
|
The acctcon program
is used to record connect accounting records in the file ctacct.MMDD. These records are in tacct.h format. In addition, acctcon creates
the lineuse and reboots files. The reboots file records all the boot records found in the wtmpx file.
|
|
PROCESS
|
The acctprc program
is used to convert the /var/adm/Spacctn.MMDD process accounting files into total accounting records in the ptacctn.MMDD files. The Spacct and ptacct files are correlated by number
so that if the runacct script fails, the Spacct files are not processed.
|
|
MERGE
|
The acctmerg program
merges the process accounting records with the connect accounting records
to form the daytacct file.
|
|
FEES
|
The acctmerg program
merges ASCII tacct records from the fee
file into the daytacct file.
|
|
DISK
|
If the dodisk script
procedure has been run, which produces the disktacct
file, the DISK program merges the file into the daytacct file and moves the disktacct file
to the /tmp/disktacct.MMDD
file.
|
|
MERGETACCT
|
The acctmerg program
merges the daytacct file with the sum/tacct file, the cumulative total accounting file. Each day, the daytacct file is saved in the sum/tacct.MMDD, file so that the sum/tacct file can
be re-created if it is corrupted or lost.
|
|
CMS
|
The acctcms program
is run several times. This program is first run to generate the command summary
by using the Spacctn files
and write the data to the sum/daycms file. The acctcms program is then run to merge the sum/daycms
file with the sum/cms cumulative command summary file.
Finally, the acctcms program is run to produce nite/daycms and nite/cms, the ASCII command
summary files from the sum/daycms and sum/cms files, respectively. The lastlogin program
is used to create the /var/adm/acct/sum/loginlog log
file, the report of when each user last logged in. If the runacct script is run after midnight, the dates showing the time last logged
in by some users will be incorrect by one day.
|
|
USEREXIT
|
Any installation-dependent (local)
accounting program can be included at this point. The runacct
script expects it to be called the /usr/lib/acct/runacct.local
program.
|
|
CLEANUP
|
Cleans up temporary files, runs the prdaily script and saves its output in the sum/rpt.MMDD file, removes the locks, and then exits.
|

Caution –
When restarting the runacct script in the CLEANUP state, remove the last ptacct file
because it will not be complete.
Daily Accounting Reports
The runacct shell script generates five basic reports
upon each invocation. The following table describes the five basic reports
generated.
Table 20–2 Daily Accounting Reports
|
Report Type
|
Description
|
|
Daily Report
|
Shows terminal line utilization by tty number.
|
|
Daily Usage Report
|
Indicates usage of system resources
by users (listed in order of user ID).
|
|
Daily Command Summary
|
Indicates usage of system resources
by commands, listed in descending order of memory use. In other words, the
command that used the most memory is listed first. This same information is
reported for the month with the monthly command summary.
|
|
Monthly Command Summary
|
A cumulative summary that reflects the data
accumulated since the last invocation of the monacct program.
|
|
Last Login Report
|
Shows the last time each user logged
in (arranged in chronological order).
|
Daily Report
This report gives information about each terminal line used. The following
is a sample Daily Report.
Oct 16 02:30 2002 DAILY REPORT FOR venus Page 1
from Mon Oct 15 02:30:02 2002
to Tue Oct 16 02:30:01 2002
1 runacct
1 acctcon
TOTAL DURATION IS 1440 MINUTES
LINE MINUTES PERCENT # SESS # ON # OFF
console 868 60 1 1 2
TOTALS 868 -- 1 1 2
|
The from and to lines specify
the time period reflected in the report. This time period covers the time
the last Daily Report was generated to the time the current Daily Report was
generated. Then, comes a log of system reboots, shutdowns, power failure recoveries,
and any other record dumped into the /var/adm/wtmpx file
by the acctwtmp program. For more information, see acct(1M).
The second part of the report is a breakdown of terminal line utilization.
The TOTAL DURATION tells how long the
system was in multiuser mode (accessible through the terminal lines). The
following table describes the data provided in the Daily Report.
Table 20–3 Daily Report Data
|
Column
|
Description
|
|
LINE
|
The terminal line or access port.
|
|
MINUTES
|
The total number of minutes that the
line was in use during the accounting period.
|
|
PERCENT
|
The TOTAL DURATION divided by the total number of MINUTES.
|
|
# SESS
|
The number of times this line or port
was accessed for a login session.
|
|
# ON
|
Same as SESS. (This
column no longer has meaning. Previously, it listed the number of times that
a line or port was used to log in a user.)
|
|
# OFF
|
This column reflects the number of
times a user logs out and any interrupts that occur on that line. Generally,
interrupts occur on a port when ttymon is first invoked
after the system is brought to multiuser mode. If the # OFF
exceeds the # SESS by a large factor, the multiplexer,
modem, or cable is probably going bad, or there is a bad connection somewhere.
The most common cause is an unconnected cable dangling from the multiplexer.
|
During real time, you should monitor the /var/adm/wtmpx
file because it is the file from which the connect accounting is derived.
If the wtmpx file grows rapidly, execute the following
command to see which tty line is the noisiest.
# /usr/lib/acct/acctcon -l file < /var/adm/wtmpx
|
If interruption is occurring frequently, general system performance
will be affected. Additionally, the wtmp file might become
corrupted. To correct this problem, see How to Fix a Corrupted wtmpx File.
Daily Usage Report
The Daily Usage Report gives a breakdown of system resource utilization
by user. A sample of this report follows.
Oct 16 02:30 2002 DAILY USAGE REPORT FOR skisun Page 1
LOGIN CPU (MINS) KCORE- MINS CONNECT (MINS) DISK # OF # OF # DISK FEE
UID NAME PRIME NPRIME PRIME NPRIME PRIME NPRIME BLOCKS PROCS SESS SAMPLES
0 TOTAL 72 148 11006173 51168 26230634 57792 539 330 0 2150 1
0 root 32 76 11006164 33664 26230616 22784 0 0 0 127 0
4 adm 0 0 22 51 0 0 0 420 0 0 0
101 rimmer 39 72 894385 1766020 539 330 0 1603 1 0 0
|
The following table describes the data provided in the Daily Usage Report.
Table 20–4 Daily Usage Report Data
|
Column
|
Description
|
|
UID
|
User ID number.
|
|
LOGIN NAME
|
Login (or user) name
of the user. Identifies a user who has multiple login names.
|
|
CPU (MINS)
|
Amount of time, in minutes, that the user's
process used the central processing unit. Divided into PRIME
and NPRIME (non-prime) utilization. The accounting system's
version of this data is located in the /etc/acct/holidays
file.
|
|
KCORE-MINS
|
A cumulative measure
of the amount of memory in Kbyte segments per minute that a process uses while
running. Divided into PRIME and NPRIME
utilization.
|
|
CONNECT (MINS)
|
Amount of time in minutes, that the
a user was logged into the system, or “real time.” Divided into PRIME and NPRIME use. If these numbers are high
while the # OF PROCS is low, you can conclude that the
user logs in first thing in the morning and hardly touches the terminal the
rest of the day.
|
|
DISK BLOCKS
|
Output from the acctdusg program, which runs the disk accounting programs and merges
the accounting records (daytacct). For accounting purposes,
a block is 512 bytes.
|
|
# OF PROCS
|
Number of processes
invoked by the user. If large numbers appear, a user might have a shell procedure
that has run out of control.
|
|
# OF SESS
|
Number of times a user logged on to the system.
|
|
# DISK SAMPLES
|
Number of times disk
accounting was run to obtain the average number of DISK BLOCKS.
|
|
FEE
|
Often unused field that represents the total accumulation of units charged
against the user by the chargefee script.
|
Daily Command Summary
The Daily Command Summary report shows the system resource use by command.
With this report, you can identify the most heavily used commands and, based
on how those commands use system resources, gain insight on how best to tune
the system.
These reports are sorted by TOTAL KCOREMIN, which is an arbitrary gauge but often a good one for calculating
drain on a system.
A sample daily command summary follows.
TOTAL COMMAND SUMMARY
COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS
NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ
TOTALS 2150 1334999.75 219.59 724258.50 6079.48 0.10 0.00 397338982 419448
netscape 43 2456898.50 92.03 54503.12 26695.51 2.14 0.00 947774912 225568
adeptedi 7 88328.22 4.03 404.12 21914.95 0.58 0.01 93155160 8774
dtmail 1 54919.17 5.33 17716.57 10308.94 5.33 0.00 213843968 40192
acroread 8 31218.02 2.67 17744.57 11682.66 0.33 0.00 331454464 11260
dtwm 1 16252.93 2.53 17716.57 6416.05 2.53 0.00 158662656 12848
dtterm 5 4762.71 1.30 76300.29 3658.93 0.26 0.00 33828352 11604
dtaction 23 1389.72 0.33 0.60 4196.43 0.01 0.55 18653184 539
dtsessio 1 1174.87 0.24 17716.57 4932.97 0.24 0.00 23535616 5421
dtcm 1 866.30 0.18 17716.57 4826.21 0.18 0.00 3012096 6490
|
The following table describes the data provided in the Daily Command
Summary.
Table 20–5 Daily Command Summary
|
Column
|
Description
|
|
COMMAND NAME
|
Name of the command.
Unfortunately, all shell procedures are lumped together under the name sh because only object modules are reported by the process accounting
system. You should monitor the frequency of programs called a.out or core or any other unexpected name. You
can use the acctcom program to determine who executed an
oddly named command and if superuser privileges were used.
|
|
NUMBER CMDS
|
Total number of times
this command was run during prime time.
|
|
TOTAL KCOREMIN
|
Total cumulative measurement
of the Kbyte segments of memory used by a process per minute of run time.
|
|
TOTAL CPU-MIN
|
Total processing time
this program accumulated during prime time.
|
|
TOTAL REAL-MIN
|
Total real-time (wall-clock)
minutes this program accumulated.
|
|
MEAN SIZE-K
|
Mean of the TOTAL KCOREMIN over the number of invocations
reflected by NUMBER CMDS.
|
|
MEAN CPU-MIN
|
Mean derived between the NUMBER CMDS and TOTAL CPU-MIN.
|
|
HOG FACTOR
|
Total CPU time divided
by elapsed time. Shows the ratio of system availability to system use, providing
a relative measure of total available CPU time consumed by the process during
its execution.
|
|
CHARS TRNSFD
|
Total number of characters
pushed around by the read and write system calls. Might be negative due to
overflow.
|
|
BLOCKS READ
|
Total number of the physical block
reads and writes that a process performed.
|
Monthly Command Summary
The format of the Daily Command Summary and the Monthly Command Summary
reports are virtually the same. However, the daily summary reports only on
the current accounting period while the monthly summary reports on the start
of the fiscal period to the current date. In other words, the monthly report
is a cumulative summary that reflects the data accumulated since the last
invocation of the monacct program.
A sample report follows.
Oct 16 02:30 2002 MONTHLY TOTAL COMMAND SUMMARY Page 1
TOTAL COMMAND SUMMARY
COMMAND NUMBER TOTAL TOTAL TOTAL MEAN MEAN HOG CHARS BLOCKS
NAME CMDS KCOREMIN CPU-MIN REAL-MIN SIZE-K CPU-MIN FACTOR TRNSFD READ
TOTALS 42718 4398793.50 361.92 956039.00 12154.09 0.01 0.00 16100942848 825171
netscape 789 3110437.25 121.03 79101.12 25699.58 0.15 0.00 3930527232 302486
adeptedi 84 1214419.00 50.20 4174.65 24193.62 0.60 0.01 890216640 107237
acroread 145 165297.78 7.01 18180.74 23566.84 0.05 0.00 1900504064 26053
dtmail 2 64208.90 6.35 20557.14 10112.43 3.17 0.00 250445824 43280
dtaction 800 47602.28 11.26 15.37 4226.93 0.01 0.73 640057536 8095
soffice. 13 35506.79 0.97 9.23 36510.84 0.07 0.11 134754320 5712
dtwm 2 20350.98 3.17 20557.14 6419.87 1.59 0.00 190636032 14049
|
For a description of the data provided in the Monthly Command Summary,
see Daily Command Summary.
Last Login Report
This report gives the date when a particular login was last used. You
can use this information to find unused logins and login directories that
can be archived and deleted. A sample report appears follows.
Oct 16 02:30 2002 LAST LOGIN Page 1
01-06-12 kryten 01-09-08 protoA 01-10-14 ripley
01-07-14 lister 01-09-08 protoB 01-10-15 scutter1
01-08-16 pmorph 01-10-12 rimmer 01-10-16 scutter2
|
Looking at the pacct File With acctcom
At any time, you can examine the contents of the /var/adm/pacctn files, or any file with records in
the acct.h format, by using the acctcom
program. If you do not specify any files and do not provide any standard input
when you run this command, the acctcom command reads the pacct file. Each record read by the acctcom
command represents information about a terminated process. Active processes
can be examined by running the ps command. The default
output of the acctcom command provides the following information:
Sample acctcom output follows:
# acctcom
COMMAND START END REAL CPU MEAN
NAME USER TTYNAME TIME TIME (SECS) (SECS) SIZE(K)
#accton root ? 02:30:01 02:30:01 0.03 0.01 304.00
turnacct adm ? 02:30:01 02:30:01 0.42 0.01 320.00
mv adm ? 02:30:01 02:30:01 0.07 0.01 504.00
utmp_upd adm ? 02:30:01 02:30:01 0.03 0.01 712.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 824.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 912.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 920.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 1136.00
utmp_upd adm ? 02:30:01 02:30:01 0.01 0.01 576.00
closewtm adm ? 02:30:01 02:30:01 0.10 0.01 664.00
|
You can obtain the following information by using acctcom
options:
Table 20–6
acctcom Options
|
Option
|
Description
|
|
-a
|
Shows average statistics about the
processes selected. The statistics are printed after the output is recorded.
|
|
-b
|
Reads the files backward, showing latest
commands first. This option has no effect if reading standard input.
|
|
-f
|
Prints the fork/exec
flag and system exit status columns. The output is an octal number.
|
|
-h
|
Instead of mean memory size, shows
the hog factor, which is the fraction of total available CPU time consumed
by the process during its execution. Hog factor = total_CPU_time/elapsed_time.
|
|
-i
|
Prints columns containing the I/O counts
in the output.
|
|
-k
|
Shows total kcore minutes instead of memory size.
|
|
-m
|
Shows mean core size. This is the default.
|
|
-q
|
Prints average statistics, not output
records.
|
|
-r
|
Shows CPU factor: user_time/(system_time + user_time).
|
|
-t
|
Shows separate system and user CPU
times.
|
|
-v
|
Excludes column headings from the output.
|
|
-C sec
|
Shows
only processes with total CPU time (system plus user) exceeding sec seconds.
|
|
-e time
|
Shows
processes existing at or before time, given in
the format hr[:min[:sec]].
|
|
-E time
|
Shows
processes starting at or before time, given in
the format hr[:min[:sec]]. Using the same time for both -S and -E, shows processes that existed at the time.
|
|
-g group
|
Shows only processes that belong to group.
|
|
-H factor
|
Shows only processes that exceed factor, where factor is the “hog factor” (see the -h
option).
|
|
-I chars
|
Shows only processes that transferred more characters than the cutoff number
specified by chars.
|
|
-l line
|
Show
only processes belonging to the terminal /dev/line.
|
|
-n pattern
|
Shows only commands matching pattern (a regular
expression except that “+” means one or more
occurrences).
|
|
-o ofile
|
Instead of printing the records, copies them in acct.h
format to ofile.
|
|
-O sec
|
Shows
only processes with CPU system time exceeding sec
seconds.
|
|
-s time
|
Show
processes existing at or after time, given in the
format hr[:min[:sec]].
|
|
-S time
|
Show
processes starting at or after time, given in the
format hr[:min[:sec]].
|
|
-u user
|
Shows
only processes that belong to user.
|
System Accounting Files
The /var/adm directory contains the active data
collection files. The following table describes the accounting files in this
directory.
Table 20–7 Files in the
/var/adm Directory
|
File
|
Description
|
|
dtmp
|
Output from the acctdusg
program
|
|
fee
|
Output from the chargefee program, which are the ASCII tacct records
|
|
pacct
|
Active process accounting file
|
|
pacctn
|
Process
accounting files switched by running the turnacct script
|
|
Spacctn.MMDD
|
Process accounting files for MMDD
during execution of the runacct script
|
The /var/adm/acct directory contains the nite, sum, and fiscal
directories, which contain the actual data collection files. For example,
the nite directory contains files that are reused daily
by the runacct script. A brief summary of the files in
the /var/adm/acct/nite directory follows.
Table 20–8 Files in the
/var/adm/acct/nite Directory
|
File
|
Description
|
|
active
|
Used by the runacct
script to record progress and print warning and error messages
|
|
active.MMDD
|
Same as the active file after the runacct
script detects an error
|
|
cms
|
ASCII total command summary used by prdaily
|
|
ctacct.MMDD
|
Connect accounting records in tacct.h format
|
|
ctmp
|
Output of acctcon1
program, connect session records in ctmp.h format (acctcon1 and acctcon2 are provided for compatibility
purposes)
|
|
daycms
|
ASCII daily command summary used by
the prdaily script.
|
|
daytacct
|
Total accounting records
for one day in tacct.h format
|
|
disktacct
|
Disk accounting records
in tacct.h format, created by the dodisk
script
|
|
fd2log
|
Diagnostic output during execution
of the runacct script
|
|
lastdate
|
Last day the runacct script executed (in date +%m%d format)
|
|
lock
|
Used to control serial use of the runacct script
|
|
lineuse
|
tty line usage report
used by the prdaily script
|
|
log
|
Diagnostic output from the acctcon program
|
|
log.MMDD
|
Same as the log file after the runacct
script detects an error
|
|
owtmpx
|
Previous day's wtmpx
file
|
|
reboots
|
Beginning and ending
dates from the wtmpx file and a listing of reboots
|
|
statefile
|
Used to record current
state during execution of the runacct script
|
|
tmpwtmp
|
wtmpx
file corrected by the wtmpfix program
|
|
wtmperror
|
Contains wtmpfix error messages
|
|
wtmperror.MMDD
|
Same as the wtmperror file after the runacct script detects an error
|
|
wtmp.MMDD
|
The runacct script's copy of the wtmpx file
|
The sum directory contains the cumulative summary
files updated by the runacct script and used by the monacct script. The following table summarizes the files in the /var/adm/acct/sum directory.
Table 20–9 Files in the
/var/adm/acct/sum Directory
|
File
|
Description
|
|
cms
|
Total command summary file for current
fiscal period in binary format
|
|
cmsprev
|
Command summary file
without latest update
|
|
daycms
|
Command summary file for the day's
usage in internal summary format
|
|
loginlog
|
Record of last date
each user logged on; created by the lastlogin script and
used in the prdaily script
|
|
rprt.MMDD
|
Saved output of prdaily script
|
|
tacct
|
Cumulative total accounting file for
current fiscal period
|
|
tacctprev
|
Same as tacct
without latest update
|
|
tacct.MMDD
|
Total accounting file for MMDD
|
The fiscal directory contains periodic summary files created by the monacct script. The following table summarizes the files in the /var/adm/acct/fiscal directory.
Table 20–10 Files in the
/var/adm/acct/fiscal Directory
|
File
|
Description
|
|
cmsn
|
Total
command summary file for fiscal period n in internal
summary format
|
|
fiscrptn
|
Report
similar to rprtn for fiscal
period n
|
|
tacctn
|
Total
accounting file for fiscal period n
|
Files Produced by the runacct Script
The following table summarizes most useful files produced by the runacct script found in the /var/adm/acct directory.
Table 20–11 Files Produced by
runacct
|
File
|
Description
|
|
nite/lineuse
|
The runacct script calls the acctcon program to gather data on terminal line usage from the /var/adm/acct/nite/tmpwtmp file and writes the data to the /var/adm/acct/nite/lineuse file. The prdaily
script uses this data to report line usage. This report is especially useful
for detecting bad lines. If the ratio between the number of logouts to logins
is greater than about three to one, the line is very likely failing.
|
|
nite/daytacct
|
The total accounting file for the day in tacct.h format.
|
|
sum/tacct
|
Contains the accumulation
of each day's nite/daytacct data and is used for billing
purposes. The monacct script restarts accumulating this
data each month or fiscal period.
|
|
sum/daycms
|
The runacct script calls the acctcms program to process
the commands used during the day to create the Daily Command Summary report
and stores the data in the /var/adm/acct/sum/daycms file.
The ASCII version is the /var/adm/acct/nite/daycms file.
|
|
sum/cms
|
This file is the accumulation
of each day's command summaries. It is restarted by the execution of the monacct script. The ASCII version is the nite/cms
file.
|
|
sum/loginlog
|
The runacct script calls lastlogin script to update the
last date logged in for the logins in the /var/adm/acct/sum/loginlog file. The lastlogin command also removes from
this file any logins that are no longer valid.
|
|
sum/rprt.MMDD
|
Each execution of the runacct script saves a copy of the
daily report that was printed by the prdaily script.
|