When we are trying to deploy one of our API Proxies, we are getting the following error:
Error in deployment for environment dev.The revision is deployed, but traffic cannot flow. com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}
2017-12-22 00:13:49,057 Apigee-Main-87446 INFO MESSAGING.CONFIGURATION - MessageProcessorServiceImpl.configure() : configuring [DeployEvent{organization='myorg', application='myapi', applicationRevision='42', deploymentSpec=basepath=/;env=dev;, deploymentID=null}]
2017-12-22 00:13:49,263 Apigee-Main-87446 INFO MESSAGING.RUNTIME - AbstractConfigurator.loadXMLConfigurations() : loading entities [] for class class com.apigee.messaging.config.beans.TargetServerBean
2017-12-22 00:13:49,318 Apigee-Main-87446 ERROR c.a.p.h.d.DNSCachedAddress - DNSCachedAddress.refresh() : Unable to resolve host : input-prd-sps.cloud.splunk.com: Name or service not known
2017-12-22 00:13:49,323 Apigee-Main-87446 ERROR MESSAGING.RUNTIME - AbstractConfigurator.handleUpdate() : Fatal error deploying proxy: {}
com.apigee.rest.framework.ValidationException: Invalid syslog config: Invalid HostName 'input-prd-sps.cloud.splunk.com' for Syslog handler
at com.apigee.messaging.runtime.destinations.SyslogDestination.<init>(SyslogDestination.java:44) ~[message-processor-1.0.0.jar:na]
at com.apigee.messaging.runtime.destinations.SysLoggerFactory.getInstance(SysLoggerFactory.java:39) ~[message-processor-1.0.0.jar:na]
at com.apigee.messaging.runtime.destinations.DestinationRegistry.newDestination(DestinationRegistry.java:44) ~[message-processor-1.0.0.jar:na]
at com.apigee.steps.messagelogging.MessageLoggingStepDefinition.populateDestinations(MessageLoggingStepDefinition.java:118) ~[message-logging-1.0.0.jar:na]
at com.apigee.steps.messagelogging.MessageLoggingStepDefinition.handleAdd(MessageLoggingStepDefinition.java:99) ~[message-logging-1.0.0.jar:na]
...<snipped>
[adevegowda@rmp1 ~]$ telnet input-prd-sps.cloud.splunk.com
telnet: input-prd-sps.cloud.splunk.com: Name or service not known
input-prd-sps.cloud.splunk.com: Unknown host
[adevegowda@rmp1 ~]$
Checked the API Proxy and found that the host name “input-prd-sps.cloud.splunk.com” was used in one of the MessageLogging policy.
Replaced the Host name “input-prd-sps.cloud.splunk.com” with the proper host name. Saved the changes, undeployed and deployed the API Proxy successfully without any errors.
We need to ensure that we are using the proper/valid host name and port # in MessageLogging, ServiceCallout or any other policies to avoid this type of errors.
Three of our running cluster started showing the same error.
Following is the screenshot of error:
We are using Edge 4.17.09 on private cloud, with CentOS 7.2
Status of services on all RMP:
Interestingly I don’t see system.log for message processor in all three clusters.
[root@fqdn]# ls -ltr
total 47799084
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 kvm-service.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 kvm-client.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 quota-service.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 quota-client.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 expiringcounter-service.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 expiringcounter-client.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 startupruntimeerrors.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 access.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 audit.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 envvalidator.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 advisory.log
-rw-r--r--. 1 apigee apigee 0 Feb 21 07:06 cps_delta_migration.log
-rw-r--r--. 1 apigee apigee 22345 Feb 21 08:04 transactions.log437034443676139.tmp
-rw-r--r--. 1 apigee apigee 67154 Feb 22 00:09 system-2018-02-21.0.log.gz
-rw-r--r--. 1 apigee apigee 12871 Feb 23 00:09 system-2018-02-22.0.log.gz
-rw-r--r--. 1 apigee apigee 12778 Feb 24 00:09 system-2018-02-23.0.log.gz
-rw-r--r--. 1 apigee apigee 12920 Feb 25 00:09 system-2018-02-24.0.log.gz
-rw-r--r--. 1 apigee apigee 52439473 Feb 25 08:19 system.log352377893494692.tmp
-rw-r--r--. 1 apigee apigee 48851705856 Feb 25 13:07 system.log352517493464058.tmp
-rw-r--r--. 1 apigee apigee 5316608 Feb 26 00:19 system.log409405389236704.tmp
-rw-r--r--. 1 apigee apigee 36647530 Mar 4 00:09 system.log927593931590177.tmp
<br>
I tried restarting the services on one RMP using “apigee-all restart” and the services don’t come up.
Still NO system.log file created:
What are the possible causes of this error? We need to debug, as all 3 of our clusters are practically unusable.
Status of all services, initially (these are all dead now, when I attempted to restart these)
<strong>[root@fqdn]# apigee-all status
+ apigee-service edge-message-processor status
apigee-service: edge-message-processor: OK
+ apigee-service edge-router status
apigee-service: edge-router: OK<br></strong>
System.log NOT available:
[root@fqdn]# tail -100f /opt/apigee/var/log/edge-message-processor/logs/system.log
tail: cannot open ‘/opt/apigee/var/log/edge-message-processor/logs/system.log’ for reading: No such file or directory
tail: no files remaining
[root@fqdn]#<br><strong></strong>
Checked status of one of the API-Proxy, using following GET:
http://<FQDN_OF_MS_NODE>:8080/v1/organizations//apis//deployments
Reponse body:
{
"environment": [
{
"name": "test",
"revision": [
{
"configuration": {
"basePath": "/",
"steps": []
},
"name": "1",
"server": [
{
"error": "com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}",
"status": "error",
"type": [
"message-processor"
],
"uUID": "a0a990a3-9fa9-433f-b5e1-ef7b1f39ea08"
},
{
"error": "com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}",
"status": "error",
"type": [
"message-processor"
],
"uUID": "a0f0ed01-23a2-4086-a666-b5e257a05ac6"
},
{
"error": "com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}",
"status": "error",
"type": [
"message-processor"
],
"uUID": "40404ec0-d6db-49ee-978f-11db1127c1c1"
},
{
"error": "com.apigee.kernel.exceptions.spi.UncheckedException{ code = application.bootstrap.FailedToConfigure, message = Configuration failed, associated contexts = []}",
"status": "error",
"type": [
"message-processor"
],
"uUID": "b5e87c96-b9de-4322-aa15-f565324bfc14"
},
{
"status": "deployed",
"type": [
"router"
],
"uUID": "2b24dc59-1694-4c5f-8b9f-9b4400bdb102"
},
{
"status": "deployed",
"type": [
"router"
],
"uUID": "17f14f58-960a-45ed-9112-91d9d9fdc38c"
},
{
"status": "deployed",
"type": [
"router"
],
"uUID": "aa12b78c-fff6-4e8a-90c9-e26a9464ffaf"
},
{
"status": "deployed",
"type": [
"router"
],
"uUID": "1baf718b-eff1-47e8-b73d-85daa82d83aa"
}
],
"state": "error"
}
]
}
],
"name": "legacy_api-v1-CurrentUser",
"organization": "org-name"
}<br>