Clock skew with NTP server after the ONTAP upgrade
Applies to
- ONTAP 9
- NTP Server
Issue
- From the secd logs, "Clock skew too great" will be reported.
Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.017.571] info : [krb5 context 14EF7600] Got cred; -1765328347/Clock skew too greatTue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.017.927] ERR : RESULT_ERROR_KERBEROS_SKEW:7537 in getUserCredViaS4U2Self() at src/utils/secd_krb_utils.cpp:1101Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.018.145] ERR : getUserCredViaS4U2Self: GSSAPI Error: (d0000), Kerberos Error: (Clock skew too great)Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.031.534] ALERT: sending EMS. Logging the RPC to secd.log { in shouldLogInEms() at src/utils/secd_ems_utils.cpp:263 }Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.031.553] debug: Logged secd.krb.s4u2self.failure to EMS. { in logEmsEventForS4U2SelfError() at src/utils/secd_ems_utils.cpp:1797 }Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.032.064] ERR : RESULT_ERROR_KERBEROS_SKEW:7537 in tryKerberosAuthentication() at src/authentication/secd_rpc_auth.cpp:2052Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.032.076] info : Kerberos authentication failed. Trying NTLM { in secd_rpc_ontap_admin_cifs_auth_basic_1_svc_secd() at src/authentication/secd_rpc_auth.cpp:2195 }Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.039.287] info : Login attempt by domain user 'cii_encrypt/RUFagDi2+1pWycdom0MwLGIMmNqARYLv2uPQ3S+VRWk=/cii_encrypt\pii_encrypt/RUFagDi2+1pWycdom0MwLPaCxKELKJID7mGVzCNBpCA=/pii_encrypt' using NTLMv2 style securityTue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.039.313] debug: Looking for NetLogon cache (key: "ad.keysight.com") in vserver 7 { in getConnectionCache() at src/connection_manager/secd_connection_cache.cpp:702 }Tue Sep 30 2025 00:33:00 -07:00 [kern_secd:info:14012] | [000.039.373] debug: Looking for a connection to NetLogon for AD.KEYSIGHT.COM { in getConnection() at src/connection_manager/secd_connection_manager.cpp:647 }
- From the event logs, the error "Clock skew too great" will be reported.
Tue Sep 30 00:33:00 -0700 [srsntap1e: secd: secd.krb.s4u2self.failure:error]: A Kerberos S4U2self failure occurred for SVM "srsnas3" and user avscan@AD.KEYSIGHT.COM. Error: Clock skew too great.
- From the "NTPDC-PEER" logs in the autosupport, that the NTP server is trying to establish connectivity via Intercluster-LIFs (not recommeneded).
remote local st poll reach delay offset disp
========================================================================169.254.40.228 169.254.39.208 13 64 377 0.00008 +0.000451 0.05080 <<<< Partner node's LIF
=10.24.31.10 10.176.216.37 16 1024 0 0.00000 +0.000000 3.99217 <<<< NTP server not reachable over Intercluster LIF
=10.127.32.11 10.176.216.37 16 1024 0 0.00000 +0.000000 3.99217 <<<< NTP server not reachable over Intercluster LIF
- From the message.log, the NTP server trying to communiate with mgmt LIFs(recommended) and also with IC lif (not-recommended), where the communication via IC Lif caused Clock Unsynchronized.
Tue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035215-07:00 srsntap1e ntpd 61182 - - ntpd 4.2.8p18@1.4062 (no 3877) Sun Aug 4 06:09:19 UTC 2024 (1): StartingTue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035512-07:00 srsntap1e ntpd 61182 - - Command line: /usr/sbin/ntpd -n -c /var/etc/ntp.conf -g -p /var/run/ntpd.pidTue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035525-07:00 srsntap1e ntpd 61182 - - ----------------------------------------------------Tue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035534-07:00 srsntap1e ntpd 61182 - - ntp-4 is maintained by Network Time Foundation,Tue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035541-07:00 srsntap1e ntpd 61182 - - Inc. (NTF), a non-profit 501(c)(3) public-benefitTue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035547-07:00 srsntap1e ntpd 61182 - - corporation. Support and training for ntp-4 areTue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035556-07:00 srsntap1e ntpd 61182 - - available at https://www.nwtime.org/supportTue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035563-07:00 srsntap1e ntpd 61182 - - ----------------------------------------------------Tue Sep 30 2025 00:33:00 -07:00 [ntp:notice] 1 2025-09-30T00:33:00.035570-07:00 srsntap1e ntpd 61182 - - DEBUG behavior is enabled - a violation of any diagnostic assertion will cause ntpd to abortTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.035712-07:00 srsntap1e ntpd 61182 - - proto: precision = 0.095 usec (-23)Tue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037286-07:00 srsntap1e ntpd 61182 - - basedate set to 1970-01-02Tue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037303-07:00 srsntap1e ntpd 61182 - - gps base set to 1980-01-06 (week 0)Tue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037319-07:00 srsntap1e ntpd 61182 - - clamp-systime: clock (2025-09-30/07:33:00.037314) in allowed rangeTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037453-07:00 srsntap1e ntpd 61182 - - initial drift restored to 28.414000Tue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037858-07:00 srsntap1e ntpd 61182 - - Listen normally on 0 e0M 10.176.192.41 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037882-07:00 srsntap1e ntpd 61182 - - Listen normally on 1 e0M 10.176.192.14 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037920-07:00 srsntap1e ntpd 61182 - - Listen normally on 2 lo0 ::1 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037937-07:00 srsntap1e ntpd 61182 - - Listen normally on 3 lo0 127.0.10.1 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037951-07:00 srsntap1e ntpd 61182 - - Listen normally on 4 lo0 127.0.20.1 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037966-07:00 srsntap1e ntpd 61182 - - Listen normally on 5 lo0 127.0.0.1 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.037979-07:00 srsntap1e ntpd 61182 - - Listen normally on 6 a0a-1748 10.176.216.37 ipspace DEFAULTTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.038008-07:00 srsntap1e ntpd 61182 - - Listen normally on 7 lo0 127.0.0.1 ipspace CLUSTERTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.038022-07:00 srsntap1e ntpd 61182 - - Listen normally on 8 lo0 127.0.10.1 ipspace CLUSTERTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.038037-07:00 srsntap1e ntpd 61182 - - Listen normally on 9 lo0 127.0.20.1 ipspace CLUSTERTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.038050-07:00 srsntap1e ntpd 61182 - - Listen normally on 10 e0a 169.254.39.208 ipspace CLUSTERTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.038064-07:00 srsntap1e ntpd 61182 - - Listen normally on 11 e0b 169.254.251.93 ipspace CLUSTERTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.045884-07:00 srsntap1e ntpd 61182 - - kernel reports TIME_ERROR: 0x4041: Clock UnsynchronizedTue Sep 30 2025 00:33:00 -07:00 [ntp:info] 1 2025-09-30T00:33:00.045910-07:00 srsntap1e ntpd 61182 - - kernel reports TIME_ERROR: 0x4041: Clock UnsynchronizedTue Sep 30 2025 00:33:07 -07:00 [kern_ntpd:info:61182] step_systime: step 25200.654000 residual 0.000000
- The Intercluster LIF is having "management-ntp-client" in the service-list, which is not recommended.
