Chapter 1 Overview of Solaris System Tuning

This section provides overview information about the format of the tuning information in this manual. This section also describes the different ways to tune a Solaris system.

• What's New in Solaris System Tuning?

• What's New in Solaris System Tuning in the Solaris 10 Release?

• Tuning a Solaris System

• Tuning Format of Tunable Parameters Descriptions

• Tuning the Solaris Kernel

• Special Solaris tune and var Structures

• Viewing Solaris System Configuration Information

• kstat Utility

What's New in Solaris System Tuning?

This section describes new or changed parameters in this Solaris release.

• Solaris 10 10/09: This release includes the zfs_arc_min and zfs_arc_max parameter descriptions. For more information, see zfs_arc_min and zfs_arc_max.

• Solaris 10 10/09: This release includes the ddi_msix_alloc_limit parameter that can be used to increase the number of MSI-X interrupts that a device instance can allocate. For more information, see ddi_msix_alloc_limit.

• Solaris 10 5/09: A previous version of this manual incorrectly identified the range of the tcp_local_dack_interval parameter as 1 millisecond to 1 minute. The correct range is 10 milliseconds to 1 minute. For more information, see tcp_local_dack_interval.

• Solaris 10 10/08: The Solaris 10 version of this manual inadvertently included the nfs4_shrinkreaddir parameter information. This parameter is not available.

• Solaris 10 10/08: For information about tuning ZFS file systems, see the following site:

  http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide

• Solaris 10 5/08: Memory locality group parameters will be provided in a Solaris 10 5/08 kernel patch. For more information about these parameters, see Locality Group Parameters.

• Solaris 10 5/08: The Solaris 10 version of this manual inadvertently included the nfs4_dynamic parameter information. This parameter is not available.

• Solaris 10 5/08: The translation storage buffers parameters in the sun4u or sun4v Specific Parameters section are being revised to provide better information. In this release, the following parameters have changed:

  • default_tsb_size

  • enable_tsb_rss_sizing

  • tsb_rss_factor

• Solaris 10 5/08: The Solaris 10 version of this manual inadvertently included the tcp_keepalive_abort_interval parameter information. This parameter is only available in the Solaris Express release. In addition, the tcp_keepalive_interval parameter information has moved to the IP Tunable Parameters With Additional Cautions section.

• Solaris 10 8/07: Parameter information was updated to include sun4v systems. For more information, see the following references:

  • maxphys

  • tmpfs:tmpfs_maxkmem

  • sun4u or sun4v Specific Parameters

• Solaris 10 8/07: The range value for the maxpgio parameter information that was previously published in this book was incorrect. For more information, see maxpgio.

• Solaris 10 8/07: The IP instances project enables you to configure a zone as an exclusive-IP zone and assign exclusive access of some LANs or VLANs to that zone.

  The previous behavior of shared-IP zones remains the default behavior. The exclusive-IP zone means that all aspects of the TCP/IP state and policy are per exclusive-IP zone, including TCP/IP tunable parameters.

  The introduction of the IP instances feature means that the following TCP parameters can only be set in the global zone because they require the PRIV_SYS_NET_CONFIG privilege:

  • ip_squeue_fanout

  • ip_squeue_worker_wait

  • ip_soft_rings_cnt

  The other TCP, IP, UDP, and SCTP parameters and route metrics only require the PRIV_SYS_IP_CONFIG privilege. Each exclusive-IP zone controls its own set of these parameters. For shared-IP zones, TCP, IP, UDP, SCTP, and route parameters are controlled by the global zone since the settings of these parameters are shared between the global zone and all shared IP zones.

  For more information about using IP instances in Solaris zones, see System Administration Guide: Solaris Containers-Resource Management and Solaris Zones.

• Solaris 10 8/07: The ip_squeue_write parameter information that was previously published in this book was incorrect and has been removed.

• Solaris 10 11/06: The default value of the ncsize parameter was incorrectly documented in the Solaris 10 release. For more information, see ncsize.

• Solaris 10 11/06: The default value of the nfs:nfs3_nra parameter was incorrectly documented in the Solaris 10 release. The default value is 4. For more information, see nfs:nfs3_nra.

• Solaris 10 6/06: The ip_squeue_fanout parameter has been modified. For more information, see ip_squeue_fanoutand ip_soft_rings_cnt.

• Solaris 10 6/06: The ip_multidata_outbound parameter has been enhanced. For more information, see ip_multidata_outbound.

• Solaris 10 6/06: The ip_forward_src_routed and ip6_forward_src_routed parameters have been corrected. The default value of this parameter since the Solaris 9 release is disabled, not enabled. For more information, see ip_forward_src_routed and ip6_forward_src_routed.

• Solaris 10 6/06: The UDP parameters have been corrected. The default values of these parameters changed in the Solaris 10 release and the new default values were previously undocumented. For more information, see UDP Tunable Parameters.

• Solaris 10 6/06: The default value of the sq_max_size parameter was incorrectly documented in the Solaris 10 release. For more information, see sq_max_size.

What's New in Solaris System Tuning in the Solaris 10 Release?

This section describes new or changed parameters in the Solaris 10 release.

• Default Stack Size

• System V IPC Configuration

• NFSv4 Parameters

• New and Changed TCP/IP Parameters

• SPARC: Translation Storage Buffer (TSB) Parameters

• SCTP Tunable Parameters

Default Stack Size

A new parameter, default_stksize, specifies the default stack size of all threads, kernel or user. The lwp_default_stksize parameter is still available, but it does not affect all kernel stacks. If default_stksize is set, it overrides lwp_default_stksize. For more information, see default_stksize.

System V IPC Configuration

In this Solaris release, all System V IPC facilities are either automatically configured or can be controlled by resource controls. Facilities that can be shared are memory, message queues, and semaphores.

Resource controls allow IPC settings to be made on a per-project or per-process basis on the local system or in a name service environment.

In previous Solaris releases, IPC facilities were controlled by kernel tunables. You had to modify the /etc/system file and reboot the system to change the default values for these facilities.

Because the IPC facilities are now controlled by resource controls, their configuration can be modified while the system is running.

Many applications that previously required system tuning to function might now run without tuning because of increased defaults and the automatic allocation of resources.

The following table identifies the now obsolete IPC tunables and the possible resource controls that could be used as replacements. An important distinction between the obsolete IPC tunables and resource controls is that the IPC tunables were set on a system-wide basis and the resource controls are set on a per-project or per-process basis.

* Note that the project.max-shm-memory resource control limits the total amount of shared memory of one project, whereas previously, the shmsys:shminfo_shmmax parameter limited the size of a single shared memory segment.

For more detailed descriptions of the resource controls, see Available Resource Controls in System Administration Guide: Solaris Containers-Resource Management and Solaris Zones.

Obsolete parameters can still be included in the /etc/system file on a Solaris system. If so, the parameters are used to initialize the default resource control values as in previous Solaris releases. For more information, see Parameters That Are Obsolete or Have Been Removed. However, using the obsolete parameters is not recommended.

The following related parameters have been removed. If these parameters are included in the /etc/system file on a Solaris system, the parameters are commented out.

For the current list of available resource controls, see rctladm(1M). For information about configuring resource controls, see project(4), and Chapter 6, Resource Controls (Overview), in System Administration Guide: Solaris Containers-Resource Management and Solaris Zones.

NFSv4 Parameters

The following parameters for the NFSv4 protocol are included in this release:

• nfs:nfs4_pathconf_disable_cache

• nfs:nfs4_cots_timeo

• nfs:nfs4_do_symlink_cache

• nfs:nfs4_lookup_neg_cache

• nfs:nfs4_max_threads

• nfs:nfs4_nra

• nfs:nfs4_bsize

• nfs:nfs4_async_clusters

• nfs:nfs4_max_transfer_size

For information about NFSv4 parameters, see NFS Module Parameters.

New and Changed TCP/IP Parameters

The following IP parameters have been added in this Solaris release:

• ip_squeue_worker_wait

• ip_squeue_fanout

• ipcl_conn_hash_size

The following TCP parameters are new in this Solaris release:

• tcp_rst_sent_rate_enabled

• tcp_rst_sent_rate

• tcp_mdt_max_pbufs

The following TCP/IP parameters are obsolete in this Solaris release.

• ipc_tcp_conn_hash_size

• tcp_compression_enabled

• tcp_conn_hash_size

• ip_forwarding

• ip6_forwarding

• xxx_forwarding

IP Forwarding Changes

In this Solaris release, IP forwarding is enabled or disabled by using the routeadm command or the ifconfig commands instead of setting the following tunable parameters with the ndd command:

• ip_forwarding

• ip6_forwarding

• xxx_forwarding

Using the routeadm command and the ifconfig command instead of the ndd command to set IP forwarding provides the following advantages:

• All settings are persistent across reboots

• The new ifconfig router and -router commands can be placed in the /etc/hostname.interface files, along with other ifconfig commands that are run when the interface is initially configured.

To enable IPv4 or IPv6 packet forwarding on all interfaces of a system, you would use the following commands:

# routeadm -e ipv4-forwarding

# routeadm -e ipv6-forwarding

To disable IPv4 or IPv6 packet forwarding on all interfaces of a system, you would use the following commands:

# routeadm -d ipv4-forwarding

# routeadm -d ipv6-forwarding

In previous Solaris releases, you would enable IPv4 or IPv6 packet forwarding on all interfaces of a system as follows:

# ndd -set /dev/ip ip_forwarding 1

# ndd -set /dev/ip ip6_forwarding 1

In previous Solaris releases, you would disable IPv4 or IPv6 packet forwarding on all interfaces of a system as follows:

# ndd -set /dev/ip ip_forwarding 0

# ndd -set /dev/ip ip6_forwarding 0

If you want to enable IP forwarding on a specific IPv4 interface or IPv6 interface, you would use syntax similar to the following for your interface. The bge0 interface is used an as example.

# ifconfig bge0 router

# ifconfig bge0 inet6 router

If you want to disable IP forwarding on a specific IPv4 interface or IPv6 interface, you would use syntax similar to the following for your interface. The bge0 interface is used an as example.

# ifconfig bge0 -router

# ifconfig bge0 inet6 -router

Previously, IP forwarding was enabled on a specific interface as follows:

# ndd -set /dev/ip bge0:ip_forwarding 1

# ndd -set /dev/ip bge0:ip_forwarding 1

Previously, IP forwarding on a specific interface was disabled as follows:

# ndd -set /dev/ip ip_forwarding 0

# ndd -set /dev/ip ip6_forwarding 0

If you want any of the preceding routeadm settings to take effect on the running system, use the following command:

# routeadm -u

For more information, see routeadm(1M) and ifconfig(1M).

SPARC: Translation Storage Buffer (TSB) Parameters

New parameters for tuning Translation Storage Buffer (TSB) are included in this release. For information about TSB parameters, see sun4u or sun4v Specific Parameters.

SCTP Tunable Parameters

Stream Control Transmission Protocol (SCTP), a reliable transport protocol that provides services similar to the services provided by TCP, is provided in this Solaris release. For more information about SCTP tunable parameters, see SCTP Tunable Parameters.

Tuning a Solaris System

The Solaris OS is a multi-threaded, scalable UNIX® operating system that runs on SPARC and x86 processors. It is self-adjusting to system load and demands minimal tuning. In some cases, however, tuning is necessary. This book provides details about the officially supported kernel tuning options available for the Solaris OS.

The Solaris kernel is composed of a core portion, which is always loaded, and a number of loadable modules that are loaded as references are made to them. Many variables referred to in the kernel portion of this guide are in the core portion. However, a few variables are located in loadable modules.

A key consideration in system tuning is that setting system parameters (or system variables) is often the least effective action that can be done to improve performance. Changing the behavior of the application is generally the most effective tuning aid available. Adding more physical memory and balancing disk I/O patterns are also useful. In a few rare cases, changing one of the variables described in this guide will have a substantial effect on system performance.

Remember that one system's /etc/system settings might not be applicable, either wholly or in part, to another system's environment. Carefully consider the values in the file with respect to the environment in which they will be applied. Make sure that you understand the behavior of a system before attempting to apply changes to the system variables that are described here.

We recommend that you start with an empty /etc/system file when moving to a new Solaris release. As a first step, add only those tunables that are required by in-house or third-party applications. Any tunables that involve System V IPC (semaphores, shared memory, and message queues) have been modified in the Solaris 10 release and should be changed in your environment. For more information, see System V IPC Configuration. After baseline testing has been established, evaluate system performance to determine if additional tunable settings are required.

The tunable parameters described in this book can and do change from Solaris release to Solaris release. Publication of these tunable parameters does not preclude changes to the tunable parameters and their descriptions without notice.

Tuning Format of Tunable Parameters Descriptions

The format for the description of each tunable parameter is as follows:

• Parameter Name

• Description

• Data Type

• Default

• Range

• Units

• Dynamic?

• Validation

• Implicit

• When to Change

• Zone Configuration

• Commitment Level

• Change History

Parameter Name

Is the exact name that is typed in the /etc/system file, or found in the /etc/default/facility file.

Most parameters names are of the form parameter where the parameter name does not contain a colon (:). These names refer to variables in the core portion of the kernel. If the name does contain a colon, the characters to the left of the colon reference the name of a loadable module. The name of the parameter within the module consists of the characters to the right of the colon. For example:

module_name:variable

Description

Briefly describes what the parameter does or controls.

Data Type

Indicates the signed or unsigned short integer or long integer with the following distinctions:

• On a system that runs a 32-bit kernel, a long integer is the same size as an integer.

• On a system that runs a 64-bit kernel, a long integer is twice the width in bits as an integer. For example, an unsigned integer = 32 bits, an unsigned long integer = 64 bits.

Units

(Optional) Describes the unit type.

Default

What the system uses as the default value.

Range

Specifies the possible range allowed by system validation or the bounds of the data type.

• MAXINT – A shorthand description for the maximum value of a signed integer (2,147,483,647)

• MAXUINT – A shorthand description for the maximum value of an unsigned integer (4,294,967,295)

Dynamic?

Yes, if the parameter can be changed on a running system with the mdb or kmdb debugger. No, if the parameter is a boot time initialization only.

Validation

Checks that the system applies to the value of the variable either as specified in the /etc/system file or the default value, as well as when the validation is applied.

Implicit

(Optional) Provides unstated constraints that might exist on the parameter, especially in relation to other parameters.

When to Change

Explains why someone might want to change this value. Includes error messages or return codes.

Zone Configuration

Identifies whether the parameter can be set in a exclusive-IP zone or must be set in the global zone. None of the parameters can be set in shared-IP zones.

Commitment Level

Identifies the stability of the interface. Many of the parameters in this manual are still evolving and are classified as unstable. For more information, see attributes(5).

Change History

(Optional) Contains a link to the Change History appendix, if applicable.

Tuning the Solaris Kernel

The following table describes the different ways tunable parameters can be applied.

/etc/system File

The /etc/system file provides a static mechanism for adjusting the values of kernel parameters. Values specified in this file are read at boot time and are applied. Any changes that are made to the file are not applied to the operating system until the system is rebooted.

Prior to the Solaris 8 release, /etc/system entries that set the values of parameters were applied in two phases:

• The first phase obtains various bootstrap parameters (for example, maxusers) to initialize key system parameters.

• The second phase calculates the base configuration by using the bootstrap parameters, and all values specified in the /etc/system file are applied. In the case of the bootstrap parameters, reapplied values replace the values that are calculated or reset in the initialization phase.

The second phase sometimes caused confusion to users and administrators by setting parameters to values that seem to be impermissible or by assigning values to parameters (for example, max_nprocs) that have a value overridden during the initial configuration.

Starting in the Solaris 8 release, one pass is made to set all the values before the configuration parameters are calculated.

Example—Setting a Parameter in /etc/system

The following /etc/system entry sets the number of read-ahead blocks that are read for file systems mounted using NFS version 2 software.

set nfs:nfs_nra=4

Recovering From an Incorrect Value

Make a copy of the /etc/system file before modifying it so that you can easily recover from incorrect value. For example:

# cp /etc/system /etc/system.good

If a value specified in the /etc/system file causes the system to become unbootable, you can recover with the following command:

ok boot -a

This command causes the system to ask for the name of various files used in the boot process. Press the Return key to accept the default values until the name of the /etc/system file is requested. When the Name of system file [/etc/system]: prompt is displayed, type the name of the good /etc/system file or /dev/null:

Name of system file [/etc/system]: /etc/system.good

If /dev/null is specified, this path causes the system to attempt to read from /dev/null for its configuration information. Because this file is empty, the system uses the default values. After the system is booted, the /etc/system file can be corrected.

For more information on system recovery, see System Administration Guide: Basic Administration.

kmdb Command

kmdb is a interactive kernel debugger with the same general syntax as mdb. An advantage of interactive kernel debugger is that you can set breakpoints. When a breakpoint is reached, you can examine data or step through the execution of kernel code.

kmdb can be loaded and unloaded on demand. You do not have to reboot the system to perform interactive kernel debugging, as was the case with kadb.

For more information, see kmdb(1).

mdb Command

Starting with the Solaris 8 release is the modular debugger, mdb, is unique among Solaris debuggers because it is easily extensible. A programming API is available that allows compilation of modules to perform desired tasks within the context of the debugger.

mdb also includes a number of desirable usability features, including command-line editing, command history, built-in output pager, syntax checking, and command pipelining. mdb is the recommended post-mortem debugger for the kernel.

For more information, see mdb(1).

Example—Using mdb to Change a Value

To change the value of the integer parameter maxusers from 495 to 512, do the following:

# mdb -kw
Loading modules: [ unix krtld genunix ip logindmux ptm nfs ipc lofs ]
> maxusers/D
maxusers:
maxusers:       495
> maxusers/W 200
maxusers:       0x1ef           =       0x200
> $q

Replace maxusers with the actual address of the item to be changed, as well as the value the parameter is to be set to.

For more information on using the modular debugger, see the Solaris Modular Debugger Guide.

When using either kmdb or mdb debugger, the module name prefix is not required. After a module is loaded, its symbols form a common name space with the core kernel symbols and any other previously loaded module symbols.

For example, ufs:ufs_WRITES would be accessed as ufs_WRITES in each debugger (assuming the UFS module is loaded). The ufs: prefix is required when set in the /etc/system file.

Special Solaris tune and var Structures

Solaris tunable parameters come in a variety of forms. The tune structure defined in the/usr/include/sys/tuneable.h file is the runtime representation of tune_t_fsflushr, tune_t_minarmem, and tune_t_flkrec. After the kernel is initialized, all references to these variables are found in the appropriate field of the tune structure.

Various documents (for example, previous versions of Solaris System Administration Guide, Volume 2) have stated that the proper way to set parameters in the tune structure is to use the syntax, tune:field-name where field-name is replaced by the actual parameter name listed above. This process silently fails. The proper way to set parameters for this structure at boot time is to initialize the special parameter that corresponds to the desired field name. The system initialization process then loads these values into the tune structure.

A second structure into which various tunable parameters are placed is the var structure named v. You can find the definition of a var structure in the /usr/include/sys/var.h file. The runtime representation of variables such as autoup and bufhwm is stored here.

Do not change either the tune or v structure on a running system. Changing any field in these structures on a running system might cause the system to panic.

Viewing Solaris System Configuration Information

Several tools are available to examine system configuration information. Some tools require superuser privilege. Other tools can be run by a non-privileged user. Every structure and data item can be examined with the kernel debugger by using mdb on a running system or by booting under kmdb.

For more information, see mdb(1) or kadb(1M).

sysdef Command

The sysdef command provides the values of System V IPC settings, STREAMS tunables, process resource limits, and portions of the tune and v structures. For example, the sysdef “Tunable Parameters” section from on a 512-Mbyte Sun^TM Ultra^TM 80 system is as follows:

10387456        maximum memory allowed in buffer cache (bufhwm)
    7930        maximum number of processes (v.v_proc)
      99        maximum global priority in sys class (MAXCLSYSPRI)
    7925        maximum processes per user id (v.v_maxup)
      30        auto update time limit in seconds (NAUTOUP)
      25        page stealing low water mark (GPGSLO)
       5        fsflush run rate (FSFLUSHR)
      25        minimum resident memory for avoiding deadlock (MINARMEM)
      25        minimum swapable memory for avoiding deadlock (MINASMEM)

For more information, see sysdef(1M).

kstat Utility

kstats are data structures maintained by various kernel subsystems and drivers. They provide a mechanism for exporting data from the kernel to user programs without requiring that the program read kernel memory or have superuser privilege. For more information, see kstat(1M) or kstat(3KSTAT).

kstat data structures with system_pages name in the unix module do not report statistics for cachefree. cachefree is not supported, starting in the Solaris 9 release.