Skip to main content

NetApp_Insight_2020.png 

NetApp Knowledgebase

setup-mnode script on mNode failing with HTTP 500: Error adding storage asset

Views:
151
Visibility:
Public
Votes:
0
Category:
element-software
Specialty:
solidfire
Last Updated:

Applies to

Element OS 11.3+

Issue

setup-mnode script on mNode 11.3+ fails with HTTP 500: Error adding storage asset

mNodeCLI # /sf/packages/mnode/setup-mnode --mnode_admin_user XXX--storage_mvip XXX--storage_username XXX --telemetry_active true

[2019-10-24T11:02:38.088538Z]:[mnodecfg:69] ERROR:Error: HTTP 500: Error adding storage asset. - Internal Server Error
Traceback (most recent call last):
  File "setup_scripts/mnodecfg.py", line 67, in config_pre_networked_mnode
  File "setup_scripts/mnodecfg.py", line 172, in _config_mnode
  File "setup_scripts/config_util.py", line 501, in launch_mnode_assets
  File "setup_scripts/config_util.py", line 609, in _ensure_storage_asset
  File "setup_scripts/config_util.py", line 115, in retry_operation
  File "setup_scripts/config_util.py", line 106, in retry_operation
  File "setup_scripts/mnode_api_client.py", line 81, in add_storage_asset
  File "setup_scripts/mnode_api_client.py", line 334, in handle_response
sfprime.MnodeCfgError: Error: HTTP 500: Error adding storage asset. - Internal Server Error
[2019-10-24T11:02:38.088952Z]:[mnodecfg:74] ERROR:mNode configuration has failed in a unrecoverable state. Attempting to cleanup docker environment.
[2019-10-24T11:02:38.089303Z]:[config_util:1107] WARNING:Starting docker environment cleanup.
[2019-10-24T11:02:38.113904Z]:[config_util:1074] INFO:Finding networks to delete.
[2019-10-24T11:02:38.114269Z]:[config_util:1078] INFO:Found following networks to attempt to delete: docker_gwbridge, internal
[2019-10-24T11:02:38.114588Z]:[config_util:1082] INFO:Finding volumes to delete.
[2019-10-24T11:02:38.115059Z]:[config_util:1085] INFO:Found following volumes to attempt to delete: NetApp-HCI-nks-service, NetApp-HCI-mnode-api, NetApp-HCI-credential-svc, NetApp-HCI-cloud-service, NetApp-HCI-hci-monitor, NetApp-Simple-data
[2019-10-24T11:02:38.115968Z]:[config_util:1112] INFO:Leaving swarm.
[2019-10-24T11:02:51.363140Z]:[config_util:1114] INFO:Pruning any stopped containers.
[2019-10-24T11:02:51.365058Z]:[config_util:1091] INFO:Deleting networks.
[2019-10-24T11:02:51.511237Z]:[config_util:1097] INFO:Deleting volumes.
[2019-10-24T11:02:51.549372Z]:[config_util:1103] INFO:Docker environment cleanup complete.
[2019-10-24T11:02:51.564742Z]:[config_util:1132] INFO:Cleaned up vcp database folder.


The script will perform automatic clean up, so after a failure there will be no running containers.

mNodeCLI $ sudo su
mNodeCLI # docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES


To get mNode logs, interrupt the script before it cleans up and run following commands:

mNodeCLI# docker ps
mNodeCLI# docker service logs mnode_mnode.1.XXXXXXXXXXXXXX --follow > /tmp/mnode.logs

mnode_mnode.1.oviesreopg7z@SF-323B  | 2019-10-24T13:20:12.592312Z |    10 |  197568 |    MainProcess-uWSGIWorker3Core0  | bearer_scheme:51 | WARNING | Unable to authenticate request: (JWTClaimsError('The token is not yet valid (nbf)',), 'The token is not yet valid, please ensure client system and auth-server clocks match.')
mnode_mnode.1.oviesreopg7z@SF-323B  | [pid: 10|app: 0|req: 21/47] 10.0.1.29 () {30 vars in 1104 bytes} [Thu Oct 24 13:20:12 2019] GET /1/assets => generated 0 bytes in 1 msecs (HTTP/1.1 401) 5 headers in 192 bytes (0 switches on core 0)


After checking mNode container logs you noticed: "The token is not yet valid, please ensure client system and auth-server clocks match" message

 

CUSTOMER EXCLUSIVE CONTENT

Registered NetApp customers get unlimited access to our dynamic Knowledge Base.

New authoritative content is published and updated each day by our team of experts.

Current Customer or Partner?

Sign In for unlimited access

New to NetApp?

Learn more about our award-winning Support