What are the NFSv4 Storepools, why do they exist?
Applies to
- ONTAP 9
- NFSv4
Answer
- ONTAP stores information about the NFSv4+ states for client connections in memory allocations called storepools
- Each storepool is a collection of memory allocations (pools) that are used to facilitate NFSv4/4.1 state work for client connections.
- There are 12 different storepool pairs that encompass a variety of areas:
| StorePool Name | Description |
|---|---|
| storePool_ByteLockAlloc | Byte-range locks |
| storePool_ClientAlloc | ClientIDs |
| storePool_CopyStateAlloc | Used for copy offload related work |
| storePool_DelegAlloc | Delegations for reads and writes (client caching) |
| storePool_DelegStateAlloc | Delegation StateID |
| storePool_LayoutAlloc | Layouts (used for pNFS referrals) |
| storePool_LayoutStateAlloc | Layout StateID |
| storePool_LockStateAlloc | Lock StateID |
| storePool_OpenAlloc | Opens |
| storePool_OpenStateAlloc | Open StateID |
| storePool_OwnerAlloc | Owners |
| storePool_StringAlloc | Stores opaque data sent for ClientID and Owner |
- Where are storePool resources located?
- Each node has its own unique storePool resources.
- These storePool resources are stored in the Network blade (nblade) of each node.
- They are utilized by clients that are mounting LIFs local to the node.
- How do storePool resources differ from lock manager objects?
- The storePool resources are specifically for doing NFSv4/4.1 state work, they do not keep track of open/locked files at the file system layer (dblade).
- Lock manager objects are responsible for the file system open/lock tracking, including resolution of open/lock conflicts.
- How does storePool exhaustion occur differ from lock manager object exhaustion?
- Exhaustion of the storePool occurs at the Network layer (nblade) of the node where the LIF is mounted.
- This is an exhaustion of the NFSv4/4.1 objects for tracking state.
- Lock manager exhaustion is exhaustion of the objects responsible for opening/locking files at the file system layer (Data blade or dblade).
- Lock manager exhaustion would affect the volumes located on the node where exhaustion occurred.
- Monitoring storePool consumption:
- storePool objects can be monitored with
statisticscounters and alert warnings when a specific "pool" becomes full. - The pool allocation numbers are dynamic and should be changing constantly as files are opened and closed by clients.
- NetApp has the capability to manually retrieve the counters via command which could be leveraged via scripting for alerting.
- In ONTAP 9.2 and later, an EMS event exists to alert when a storePool resource reaches 80% of its maximum threshold.
- storePool objects can be monitored with
- How can specific clients potentially cause problems?
- In certain circumstances, clients do not close their OPENs in a way that the ONTAP node is expecting.
- When this occurs, the client is unaware that it still has that OPEN allocated.
- In this case, the server will not remove the OpenState object and the resource is never returned to the pool.
- If this behavior continues, storePool exhaustion can occur as the client behavior orphans resources on the server.
- Dumping nfsv4 locks can show which client is taking up all of the allocations in the storePool.
- Once this client is restarted, ONTAP frees associated storePool resources associated with that client.
- A client mounts a LIF on node A and a LIF on node B via NFSv4. Both of these mounts access a volume on node A. Exhaustion of storePool resource occurs on Node A. Which mount will be affected?
- Only connections on Node A will be affected.
- These allocations are on the "networking" layer (nblade) of ONTAP which is the node that is being used for connections.
- Where the volumes are located is not relevant for the issue.