Unable to Add APIs and/or seeing DocumentNotFound or FileNotFound errors logged in API Developer Portal v3.1 and newer, causing unexpected behavior.

Products

• CA API Developer Portal
• CA API Gateway

Issue/Introduction

Symptom:

Unexpected behaviour may occur in the environment, such as certain pages may return 404 errors, APIs might not show in the webpage, and navigation menus may not display correctly even if communication with the API Gateway is working. The DocumentNotFound or FileNotFound errors may be logged in the catalina.out log file on the API Developer Portal.

One of the usual symptoms of this problem is the fact that API's cannot be added to the portal. When the "Add API" button is pressed the behaviour exhibited in the following behaviour is exhibited:

<Please see attached file for image>

The primary indication for this concern is the log entries that are generated when a page is requested. For example:

11/05 17:17:15.340 ERROR (http-37080-10 - [general] -- FileSystemXMLStore:
 Unable to retrieveDocument -repository/META/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
11/05 17:17:15.341 ERROR (http-37080-10 - [general] -- FileSystemXMLStore:
 Unable to retrieveDocument -repository/META/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
11/05 17:17:15.341 ERROR (http-37080-10 - [general] -- com.thelevel.cms.xmlsources.exceptions.XMLSourceException: com.thelevel.repository.DocumentNotFound:
 DOCUMENT_NOT_FOUND:repository/META/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
at com.thelevel.cms.xmlsources.forms.includes.Validation.getImportRules(Validation.java:159)
at com.thelevel.cms.xmlsources.forms.includes.Validation.getValidationURIXML(Validation.java:143)
[...]
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.thelevel.repository.DocumentNotFound: DOCUMENT_NOT_FOUND:repository/META/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
at com.thelevel.repository.filesystem.FileSystemRepository.riGetDocumentMetaInfo(FileSystemRepository.java:366)
at com.thelevel.repository.helper.AbstractRepository.getDocumentByPath(AbstractRepository.java:177)
[...]
at com.thelevel.cms.admin.ImportAdmin.getImportRules(ImportAdmin.java:621)
at com.thelevel.cms.xmlsources.forms.includes.Validation.getImportRules(Validation.java:152)
... 40 more

Despite the error suggesting that a file may not exist, it does indeed exist under this particular scenario. To prove this is the scenario you are affected by, you may run a find command against the API Developer Portal file system for the file reported as missing in the logs, and the file will appear in the output. For example:

find / -wholename */API/xml_template/validation.xml -exec md5sum {} +
8141bd0dcc20b2ab75f59a6b9750e4db /opt/Deployments/lrs/repository/META/HEAD/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
62370cb035f9fedd35d605b4ba073349 /opt/Deployments/lrs/repository/META/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
81d78bdf5ea617c75b380529212a46c0 /opt/Deployments/lrs/repository/DATA/HEAD/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
81d78bdf5ea617c75b380529212a46c0 /opt/Deployments/lrs/repository/DATA/PUBLISHED/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml
md5sum: /opt/Deployments/lrs/repository/VERSIONS/SYSTEM/conf/sitebuilder/packages/layer7/content_types/API/xml_template/validation.xml: Is a directory

If listing the files in the affected directories where the files are located, a user will find that the owner is no longer the required l7portal user but is a different user instead.

Environment:

This typically occurs in v3.1 of API Developer Portal, however it may occur in newer versions as well.

Cause:

In API Developer Portal v3.1, the catalina service was configured as a daemonized process and is referred as the apiportal service. A user of v3.1 and newer must use the new method for starting and stopping the API Developer Portal service. The root cause of this unexpected behaviour in this scenario is a user running the catalina.sh script to start/stop the service in version 3.1 of API Developer Portal rather than using supported method of starting and stopping the API Developer Portal service. Running the catalina.sh script in version 3.1 of API Developer Portal will inadvertently change the ownership on files which then cause the unexpected behaviour to occur.

Resolution:

To resolve this issue, the following commands should be run in a privileged command prompt of the API Developer Portal to change the incorrect owner of the file back to the required owner (requires a restart):
# chown -R l7portal:portalusers /opt/Deployments/lrs/repository
# service apiportal restart

To prevent this behavior from occurring again in a v3.1 and newer API Developer Portal:

• the catalina.sh script must never be run as root
• a user must always use the officially supported method for starting and stopping the API Developer Portal service in v3.1 and newer

Environment

Release:
Component: APIPRD

Resolution

Please Update This Required Field