performancewiki.com - Tuning cpu, tcp, disk io, memory on aix for better performance

performance / tuning tips. to the point.

del.icio.us performancewiki.com Latest Items

AIX (v5.3+) Tuning Parameters

Child processes cannot start under heavy workload?
Database server reports memory errors while there is plenty free?
Not taking advantage of external storage cache?

CPU, Memory & Kernel Network Disk I/O Disable Daemons

CPU, Memory & Kernel

ulimit

'ulimit' defines file descriptors limits. It specifies the number of open files permitted. If this value is set too low, a memory allocation error will occur on AIX and a too many files open will be logged to the to the stderr log file. Set this value higher than the default system value. For large multi-processor machines, set to unlimited.

# ulimit -a // to query
# ulimit -n <new_value> // to set to a new value
# ulimit -n -1 // to set it to unlimited

Maximum Number of Processes Per User

'maxuproc' defines the max number of processes each AIX user can 'fork'. If it is set too low, you may not be able to create high number of database agents necessary to handle connections from your application. So it is recommended to set this to at least 4096.

To query current setting:
# lsattr -D -l sys0 -a maxuproc

To set to a different value:
# chdev -l sys0 -a maxuproc='4096'

Shared Memory Segments

The 'EXTSHM' environment variable defines the max number of memory segments shared by all user-mode applications. Some database servers, such as IBM DB2, relies on this environment variable to support large workloads. To set it, add the following lines to a user's profile:

TEXTSHM=ON
export EXTHSM

Network

In high volume environment, TCP layer on AIX should be tuned to the similar values listed below:

Tune TCP

To display a list of current tunable values, use:
/usr/sbin/no -a

To make TCP changes permanent, add the following lines into the /etc/rc.net file:

/usr/sbin/no -o sb_max=6192000
/usr/sbin/no -o tcp_sendspace=4096000
/usr/sbin/no -o tcp_recvspace=4096000
/usr/sbin/no -o udp_sendspace=65536
/usr/sbin/no -o udp_recvspace=655360
/usr/sbin/no -o rfc1323=1
/usr/sbin/no -o ipqmaxlen=150
/usr/sbin/no -o clean_partial_conns=true

Ethernet Device Driver Properties

To display a list of current ethernet adapter device driver settings such as MTU, IPs, etc., use:
lsattr -E -l en2

where en2 is the adapter name - chances are you have several, en0, en1, etc. A typical output will have the following:

root@pw4:~# lsattr -E -l en2
alias4                      IPv4 Alias including Subnet Mask           True
alias6                      IPv6 Alias including Prefix Length         True
arp           on            Address Resolution Protocol (ARP)          True
authority                   Authorized Users                           True
broadcast                   Broadcast Address                          True
mtu           1500          Maximum IP Packet Size for This Device     True
netaddr       10.1.3.4      Internet Address                           True
netaddr6                    IPv6 Internet Address                      True
netmask       255.255.255.0 Subnet Mask                                True
prefixlen                   Prefix Length for IPv6 Internet Address    True
remmtu        576           Maximum IP Packet Size for REMOTE Networks True
rfc1323                     Enable/Disable TCP RFC 1323 Window Scaling True
security      none          Security Level                             True
state         up            Current Interface Status                   True
tcp_mssdflt                 Set TCP Maximum Segment Size               True
tcp_nodelay                 Enable/Disable TCP_NODELAY Option          True
tcp_recvspace 1024000       Set Socket Buffer Space for Receiving      True
tcp_sendspace 1024000       Set Socket Buffer Space for Sending        True

To change the adapter network settings, one way is to run the command line tool chdev:

# ifconfig en0 detach
# chdev -l ent0 -a tx_que_size=128
# ifconfig en0 hostname up

where tx_que_size is an example parameter.  The second way to change them is through the SMIT, for example:

   1. Detach the interface by running the following command:

      # ifconfig en0 detach

      where en0 represents the adapter name.
   2. Use SMIT to display the adapter settings. Select Devices -> Communications -> adapter type -> Change/Show...
   3. Move the cursor to the field you want to change, and press F4 to see the minimum and 
      maximum ranges for the field (or the specific set of sizes that are supported).
   4. Select the appropriate size, and press Enter to update the ODM database.
   5. Reattach the adapter by running the following command:

      # ifconfig en0 hosthame up

Example SMIT screen:

Change/Show Characteristics of an Ethernet Adapter

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

                                                        [Entry Fields]
  Ethernet Adapter                                    ent2
  Description                                         10/100/1000 Base-TX PCI-X Adapter (14106902)
  Status                                              Available
  Location                                            1V-08
  Receive descriptor queue size                      [1024]             +#
  Transmit descriptor queue size                     [512]              +#
  Software transmit queue size                       [8192]             +#
  Transmit jumbo frames                               no                +
  Enable hardware transmit TCP resegmentation         yes               +
  Enable hardware transmit and receive checksum       yes               +
  Media Speed                                         Auto_Negotiation  +
  Enable ALTERNATE ETHERNET address                   no                +
  ALTERNATE ETHERNET address                         [0x000000000000]   +
  Apply change to DATABASE only                       no                +

F1=Help                       F2=Refresh                    F3=Cancel                     F4=List
Esc+5=Reset                   Esc+6=Command                 Esc+7=Edit                    Esc+8=Image
Esc+9=Shell                   Esc+0=Exit                    Enter=Do

Disk I/O

Tune FS Performance

The following file system mount option settings allow you to take advantage of caching, throughput optimization, and performance of UNIX attached external storage device, like a SAN.

Update the /etc/filesystems, and modify the wanted file system section(s) to use 'dio':

/mySQLdb:
    dev = /dev/fwvol
    vfs = jfs2
    log = INLINE
    mount = true
    options = dio,rw
    account = false

Enabling file system direct I/O, on AIX, allows the file system buffer cache to be bypassed which eliminates the double buffering of data which can adversely affect file system I/O performance. For changes to take effect, you may run 'mount -a', if the file system you are modifying is not mounted. If the file system is mounted it will need to be unmounted and then remounted or a system reboot will need to be executed.

Disable Unused Daemons

To reduce the workload on system make sure you are only running the daemons you need, disabling the unnecessary daemons can significantly improve the load on system. Know what your AIX server is used for, and disable the following recommended daemons:

accounting : The command accton enables system-wide accounting services, if you are not using accounting on your system then disabling accton command will increase productivity of your system.
biod: Daemon allows the system to access filesystems via NFS.
comsat: program that prints "you have new mail" on your screen.
lpd or lpsched: printer daemons.
mounted: this daemon listens for remote mount requests.
nfsd: This Daemon services NFS requests from remote systems.
nntpd : This Daemon supports USENET network news services.
quotas: /etc/rc quotaon enables disk quota checking.
rlogind: services rlogin and rsh commands.
routed: This daemon routes network packets destined for other networks. If your local network has only one gateway to the outside world you can disable routed. Then make sure that /etc/rc.local has a line such as route add default gateway 1
rwhod: This daemon provides information about users on other systems, rwho and ruptime commands use this daemon.
sendmail: This provides e-mail services both internally and externally (between other systems). Sendmail uses lot of memory.
talkd: This Daemon supports the talk command.
timed: This daemon attempts to synchronize system clocks across a network. If you are working across different systems then this is necessary.
ypbind: This daemon lets the system look up information in NIS database. Atleast one system must be running ypserv before you can run ypbind.
ypserv: This daemon makes a system act as an NIS server, the system that can send information about the NIS database to other systems on network. It must not be running if system isn't a NIS server.