Unable to start up OnCommand Insight service with 404 error in web browser

Last updated
Save as PDF
Share
1. Share
2. Tweet
3. Share

Views:: 402

Visibility:: Public

Votes:: 0

Category:: oncommand-insight

Specialty:: oci

Last Updated:

Applies to

OnCommand Insight (OCI) 7.3.2
Compatible Windows platform

Issue

One or more of the following symptoms may be seen when all of the OCI services are up:

1. When attempting to access OCI, 404 error appears in web browser.

2. ElasticSearch (ES) just keeps timing out and no node is available for use. To check, when entering https://locahost/cat/health and https://localhost/cat/indices (within the web browser) in the OCI server, the web page will either:

Try to load up, then eventually time out
Show yellow or red at 0%.

3. ElasticSearch will fail to respond to any request, as similarly seen from the following logs:

oci-cluster.log (located in <installed_path>\SANscreen\program_data\elasticsearch\logs) shows all of the requests timing out with failures to delete indices.

[2020-04-28T07:25:48,321][WARN ][o.e.t.TransportService ] [VWTSOCI001] Received response for a request that has timed out, sent [30485ms] ago, timed out [15469ms] ago, action [cluster:monitor/nodes/stats[n]], node [{VWTSOCI001}{dx4j5WoDQia-22-BA7uHxA}{o4Th7W8_TuOL8lrYj8eGyQ}{127.0.0.1}{127.0.0.1:9310}], id [13438]

[2020-04-23T01:13:30,112][DEBUG][o.e.a.a.i.d.TransportDeleteIndexAction] [VWTSOCI001] failed to delete indices [] org.elasticsearch.cluster.metadata.ProcessClusterEventTimeoutException: failed to process cluster vent (delete-index) within 30s

server-indexer.log (located in <installed_path>\SANscreen\wildfly\standalone\log) will show failures for the incremental indexer due to version conflicts during updates. This implies that multiple updates are occurring between scheduled scans. There are also errors present that denote failures to update aliases.

2020-04-23 03:04:14,350 ERROR [EE-ManagedThreadFactory-default-Thread-11] IncrementalIndexUpdater (IncrementalIndexUpdater.java:107) - ElasticsearchException while updating application fields for type HV_VirtualMachine com.netapp.oci.es.ElasticsearchException: Error bulk indexing data. failure in bulk execution: [0]: index [oci-inventory-hv_virtualmachine-2020-04-07-23-03-00], type [hv_virtualmachine], id [3838685751485770960], message [[oci-inventory-hv_virtualmachine-2020-04-07-23-03-00/7KzZYy9tTaK4uokvMyg8Tg][[oci-inventory-hv_virtualmachine-2020-04-07-23-03-00][1]] VersionConflictEngineException[[hv_virtualmachine][3838685751485770960]: version conflict, current version [10] is different than the one provided [9]]]

2020-04-23 00:32:52,003 ERROR [EJB default - 7] QueryIndexBuilder (QueryIndexBuilder.java:202) - Error indexing Disk com.netapp.oci.es.ElasticsearchException: Failed to switch alias: oci-inventory-disk

Event queue is almost near capacity, i.e. more than 1950 out of 2000, as seen in events.log (located in <installed_path>\SANsceen\wildfly\standalone\log). A lot of annotation and application update events can also be observed at the same time.

2020-04-23 03:28:42,052 INFO [default task-52] AsyncGuavaEventBus (EventBusLogHelper.java:28) - Posting event: AnnotationAssignmentEvent 2020-04-23 03:28:42,052 INFO [default task-52] AsyncGuavaEventBus (EventBusLogHelper.java:28) - Posting event: EventBusStatusEvent 2020-04-23 03:28:42,052 WARN [default task-52] AsyncGuavaEventBus (AsyncGuavaEventBus.java:126) - Event queue size: 1961 out of 2000