Thanks @Dino
The steps for tearing everything down worked! I was able to install the datastore on the first 3 nodes and the management server on node 1. (I’m doing a 5 node install btw)
However, now the Router/MP installation on node 2 is failing with the below message:
apigee-configutil: edge-router: # OK
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
Server is already stopped.
apigee-service: edge-router: Not running (DEAD)
apigee-configutil: edge-router: # OK
apigee-service: edge-router: Not running (NO_LOCKFILE)
apigee-service: edge-router: status=2, continuing
apigee-service: edge-router: OK
apigee-service: edge-router: edge-router is running
Checking for router on 127.0.0.1 port 8081 … OK
Checking for router uuid …
Unable to get uuid for router.
Error: setup.sh: /opt/apigee/apigee-service/bin/apigee-service exited with unexpected status 1
I was able to find the following errors in the following egde router logs:
/opt/apigee/var/log/edge-router/edge-router.log:
java.io.IOException: Cannot run program “/opt/nginx/scripts/apigee-nginx”: error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
/opt/apigee/var/log/edge-router/logs/system.log:
2017-01-05 13:11:09,012 main-EventThread ERROR o.a.c.ConnectionState - ConnectionState.checkState() : Authentication failed
…
…
2017-01-05 12:37:12,907 main INFO NGIX-ADAPTOR - NginxLoadBalancingAdaptor.clearExistingConfigs() : Attempting to clear existing conf files at /opt/nginx/conf.d
2017-01-05 12:37:12,911 main WARN LB-SYNC - FileWriter$1.postVisitDirectory() : dir found under conf.d
2017-01-05 12:37:12,912 main WARN NGIX-ADAPTOR - NginxUtil.checkConfig() : config testing with all files
2017-01-05 12:37:12,912 main INFO NGIX-ADAPTOR - NginxUtil.checkConfig() : Testing config of Nginx on this machine using command /opt/nginx/scripts/apigee-nginx configtest and file
2017-01-05 12:37:12,915 main INFO NGIX-ADAPTOR - NginxLoadBalancingAdaptor.initialize() : Nginx config test failed
2017-01-05 12:37:12,915 main ERROR LOAD-BALANCER - LoadBalancingManagementServiceImpl.start() : Could not Initialize adaptor nginx
2017-01-05 12:37:12,915 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : LoadBalancingManagementService
2017-01-05 12:37:12,918 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/dc-1/pods/gateway/bindings/servers/{uuid}/organizations/{org}/environments/{env}, async is true
2017-01-05 12:37:12,921 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/dc-1/pods/gateway/bindings/orgs/{org}, async is true
2017-01-05 12:37:12,924 main INFO ZOOKEEPER - ZooKeeperServiceImpl.registerPathWatcher() : attaching watcher on path /regions/dc-1/pods/gateway/bindings/orgenvs/{org}/environments/{env}, async is true
2017-01-05 12:37:12,928 main INFO KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Successfully started the service : MessageProcessorManagementService
2017-01-05 12:37:12,928 main ERROR Router-Service - RouterServiceImpl.start() : Router not started because, Load Balancing could not be initialized
2017-01-05 12:37:12,936 main ERROR KERNEL.DEPLOYMENT - ServiceDeployer.startService() : ServiceDeployer.deploy() : Got a life cycle exception while starting service [RouterService, Nginx Initialization failed] : {}
com.apigee.errors.http.server.ServiceFailureException: Nginx Initialization failed
at com.apigee.proxy.service.RouterServiceImpl.start(RouterServiceImpl.java:65) ~[message-router-proxy-1.0.0.jar:na]
It appears that the issue points to a root cause which is related to nginx setup. Not sure why nginx setup would fail by itself.
Is there somewhere else I can look for clues? I have the process very well understood to this point, so tearing everything down again (and again) is not a problem for me.
Thanks again for all your help so far, Dino!
Best,
Chris