Skip to main content
Effective December 3 - NetApp adopts Microsoft’s Business-to-Customer (B2C) identity management to simplify and provide secure access to NetApp resources. For accounts that did not pre-register (prior to Dec 3) access to your NetApp data may take up to 1 hour as your legacy NSS ID is synchronized to the new B2C identity. To learn more, Read the FAQ and Watch the video.
NetApp Knowledge Base

OnCommand Insight Web GUI is unresponsive or not fully loaded

Views:
231
Visibility:
Public
Votes:
0
Category:
oncommand-insight
Specialty:
oci
Last Updated:

Applies to

  • OnCommand Insight 7.3
  • Red Hat Enterprise Linux / CentOS

Issue

OnCommand Insight Web GUI is unresponsive and/or not fully loaded.
oci-cluster.log of elasticsearch includes 'Too many open files' error like the following example.
      [DEBUG][o.e.a.a.i.s.TransportIndicesStatsA ction] [xxx.xxx.local] [indices:monitor/stats] failed to execute operation for shard [[oci-inventory-qtree-2018-04-02-12-48][1], node[gtIfs6jjQdmsxx5Ji4Uo7g], [P], s[STARTED], a[id=2YH5gjjRQk-ooHjtZr3uhg]]
      org.elasticsearch.ElasticsearchException: failed to refresh store stats
       at org.elasticsearch.index.store.Store$StoreStatsCache.refresh(Store.java:1393) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.index.store.Store$StoreStatsCache.refresh(Store.java:1378) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.common.util.SingleObjectCache.getOrRefresh(SingleObjectCache.java:54) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.index.store.Store.stats(Store.java:332) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.index.shard.IndexShard.storeStats(IndexShard.java:703) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:177) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:163) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.admin.indices.stats.TransportIndicesStatsAction.shardOperation(TransportIndicesStatsAction.java:47) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.onShardOperati on(TransportBroadcastByNodeAction.java:433) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceive d(TransportBroadcastByNodeAction.java:412) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.action.support.broadcast.node.TransportBroadcastByNodeAction$BroadcastByNodeTransportRequestHandler.messageReceive d(TransportBroadcastByNodeAction.java:399) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.transport.TransportRequestHandler.messageReceived(TransportRequestHandler.java:33) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.6.3.jar:5.6.3]
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_162]
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_162]
       at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
      Caused by: java.nio.file.FileSystemException: /var/lib/netapp/oci/elasticsearch/nodes/0/indices/tKE39cPAR5CG88PIDruJRg/1/index: Too many open files
       at sun.nio.fs.UnixException.translateToIOException(UnixException.java:91) ~[?:?]
       at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?]
       at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?]
       at sun.nio.fs.UnixFileSystemProvider.newDirectoryStream(UnixFileSystemProvider.java:427) ~[?:?]
       at java.nio.file.Files.newDirectoryStream(Files.java:457) ~[?:1.8.0_162]
       at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:215) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
       at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:234) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
       at org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:57) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
       at org.apache.lucene.store.FilterDirectory.listAll(FilterDirectory.java:57) ~[lucene-core-6.6.1.jar:6.6.1 9aa465a89b64ff2dabe7b4d50c472de32c298683 - varunthacker - 2017-08-29 21:54:39]
       at org.elasticsearch.index.store.Store$StoreStatsCache.estimateSize(Store.java:1399) ~[elasticsearch-5.6.3.jar:5.6.3]
       at org.elasticsearch.index.store.Store$StoreStatsCache.refresh(Store.java:1391) ~[elasticsearch-5.6.3.jar:5.6.3]

 

 

 

CUSTOMER EXCLUSIVE CONTENT

Registered NetApp customers get unlimited access to our dynamic Knowledge Base.

New authoritative content is published and updated each day by our team of experts.

Current Customer or Partner?

Sign In for unlimited access

New to NetApp?

Learn more about our award-winning Support