CONTAP-120359: System Manager becomes unresponsive as MGWD ran out of memory
Issue
- The MGWD process ran out of memory which caused System Manager to become unresponsive
- Errors were observed in the
/etc/log/mlog/mgwd.log
[kern_mgwd:info:3153] A [src/rdb/sql_local_unit.cc 806 (0x823d03e00)]: sqlite_logger: SQLite Error: (1) statement aborts at 4: [ATTACH DATABASE '/mroot/etc/cluster_config/rdb/VLDB/_sql/rdb.db' AS vldb;] database vldb is already in use
[kern_mgwd:info:3153] 0x823d03e00: 8803eb00002a504a: ERR: SQL_CONTEXT: _execute_setup_stmts:src/sql_context.cc:1056 _execute_setup_stmts: failed to exec statement 'ATTACH DATABASE '/mroot/etc/cluster_config/rdb/VLDB/_sql/rdb.db' AS vldb;' for on demand connection 0x834cabd08: SQL logic error database vldb is already in use
[kern_mgwd:info:3153] sqlite exec message: database vldb is already in use
- REST GET requests using the SQL pass-through also fail
- Adding cluster to AIQUM fails by
Unexpected HTTP response code received for cluster-identity-get from <CLUSTER_IP>: 502
HTTP 502 Proxy Error
is reported in/etc/log/mlog/audit.log
[kern_audit:info:11111] 1234e8000001f000 :: Cluster1:ontapi :: <AIQUM_IP>:11111 :: <NODE>:admin :: POST /servlets/netapp.servlets.admin.XMLrequest_filer HTTP/1.1 :: Error: 502 Proxy Error
/etc/log/mlog/apache_logs/apache_error.log
indicates errorreading status line from remote server
[proxy:error] [pid 95609:tid 34402477568] [client <AIQUM_IP>:54858] AH00898: Error reading from remote server returned by //servlets/netapp.servlets.admin.XMLrequest_filer
[proxy_ http:error] [pid 95609:tid 34402477568] (20014)Internal error (specific information not available): [client <AIQUM_IP>:54858] AH01102: error reading status line from remote server 127.0.0.1:936