What are CPU as a compute resource and the CPU domains in ONTAP 9?
Applies to
- ONTAP 9
- Data ONTAP 8
- Data ONTAP 8 7-Mode
Answer
- Data ONTAP uses a Coarse Symmetric Multiprocessing (CSMP) design which partitions system functions into logical processing domains.
- Each logical processing domain has a set of rules that govern how and when the logical CSMP domain can be scheduled across physical CPU cores.
- These rules are designed to ensure that all processing occurs in a safe and efficient manner.
- The following table describes some of the common logical processing domains, their typical tasks
- It also describes if the logical domain can run on one or more CPU cores concurrently, along with any specific scheduling rules:
|
Domain name |
Typical Tasks in Domain |
CPU Concurrency |
Notes |
|---|---|---|---|
|
nwk_exclusive |
IP processing, NFS protocol processing |
1 |
Networking code which can only run on a single CPU concurrently |
|
nwk_exempt |
IP processing, NFS protocol processing (7-mode and cDOT), SMB processing (cDOT) |
1+ |
Maximum number of CPUs is dependent on controller model and Data ONTAP release |
|
nwk_legacy |
IP processing, NFS protocol processing |
1 |
Networking code which can only run on a single CPU concurrently |
|
storage |
SCSI communication |
1+ |
Concurrency of 1 prior to Data ONTAP 8.2.1 or if less than 6 CPUs |
|
raid |
RAID subsystem |
1 |
|
|
raid_exempt |
RAID subsystem |
1+ |
Introduced in Data ONTAP 8.2 |
|
XOR_Ex |
RAID subsystem XOR parity processing |
1+ |
Introduced in ONTAP 9.0 |
|
target |
SCSI (FCP/iSCSI) processing |
1 |
7-mode only |
|
ssan_exempt |
SCSI (FCP/iSCSI) processing |
1+ |
Introduced in clustered Data ONTAP 8.2 |
|
Kahuna |
Serialized WAFL and anything not in another domain |
1 |
Exclusive with WAFL_Ex (i.e. either Kahuna can be active on 1 CPU or WAFL_Ex can be active on 1+ CPUs, but both cannot be active at the same time) |
|
WAFL_Ex |
Parallelized WAFL |
1+ |
Exclusive with Kahuna (i.e. either Kahuna can be active on 1 CPU or WAFL_Ex can be active on 1+ CPUs, but both cannot be active at the same time) |
|
WAFL_MPClean |
WAFL |
1+ |
|
|
SM_Exempt |
SnapMirror |
1+ |
|
|
cifs |
SMB protocol processing (7-mode only) |
1 |
Initial decoding only; majority of SMB processing occurs in WAFL |
|
exempt |
General parallelized work |
1+ |
|
|
hostOS |
Tasks owned by the BSD layer including NTP, environmental sensor monitoring, ZAPI handling, autosupport |
1+ |
|
- Logical CSMP domains are scheduled to run on physical CPU cores by the Data ONTAP kernel.
- The scheduling logic is unique to a given Data ONTAP release and hardware platform and is tuned to maximize the overall system performance.
- The level of parallelism seen for a given logical domain varies based on a number of factors, including:
- Incoming workload rate
- Type of work being requested
- Data ONTAP OS version and more.
Additional Information
Scheduling behaviors:
- Pinning of heavily used logical domains to physical CPU core to maximize cache efficiency.
- Shutdown of physical CPU cores at various load points to optimize processing of the run queue
- These optimizations might appear to result in an uneven balancing of workload across physical CPU cores.
- This behavior is by design and is optimized for each specific Data ONTAP release and platform combination.