1.) Information required while reporting Teamforge UI defects:[#]#

                          i). Is the issue specific to a single user/ repository/ project or if it is happening for all users and across all projects in the site.

                         ii). When was the last time/ date the application worked fine without showing the current issue?

                         iii). Are there any recent application/ Network/ Hardware configuration changes by other teams.

                         iv). Is the issue existing in Prod/ UAT/ Stage? Or all the instances?

                         v). Debug logs will only be helpful if they are captured from the time when the issue/ error is recreated.

                         vi). A detailed description of all the steps that lead to reproducing the issue.

Steps to enable DEBUG: Steps to Change LOG LEVEL: In Teamforge > Click 'Admin' > System Tools > Configure Logging > Log Level > Debug > Save.

Note: Make sure to enable INFO level logging once the required logs are captured. Having the DEBUG log level for a long time will cause a high server load and may result in application performance issues.


2.) Information required while reporting Teamforge Site Outages or Slow performance issues:[#]#

                           1. top

                           2. df -h

            /opt/collabnet/teamforge/log/apps/

           /opt/collabnet/teamforge/log/runtime/


3.) Information required while reporting Teamforge mail service not functional:[#]#

                   teamforge status -s mail

                   teamforge status -s phoenix

                   /opt/collabnet/teamforge/log/phoenix/

                   /opt/collabnet/teamforge/log/apps/

                  select distinct(queue), count(*) from stored_message group by queue;


4.) Information required while reporting Teamforge search service not functional:[#]#

                     teamforge status -s codesearch-elasticsearch

                   /opt/collabnet/teamforge/log/phoenix/

                  /opt/collabnet/teamforge/log/apps/

                  select distinct(queue), count(*) from stored_message group by queue;


5.) Information required while reporting LDAP related issues:[#]#

telnet  <LDAP server hostname>  <port>
ldapsearch -D '<bind DN>' -W -p <port> -h '<LDAP hostname>' -b <base DN> -s sub "cn=<canonical name>"

Note: If 'telnet' and 'ldapsearch' packages not installed on the server, above commands would give 'command not found' error. Install telnet using 'yum install telnet' and ldapsearch using 'yum install openldap-clients'.


6.) Information required on artifact display/ Markdown editor related issues:[#]#


...

7.) BCP process in NSE:[#]#

 i). Verify the hostnames of BCP servers(DB, App, SVN and eventq) & their respective IP address in /etc/hosts file of all the server during the IP swap.
ii). Make sure all the host names of BCP server is properly configured in the site-options.conf during the hostname swap.
iii). Provision servers in this order: DB > App > eventq & SCM
iv). Finally make sure all the services are up and running on all the servers. Then try to access the PROD Teamforge URL on the browser and verify the login.

Refer:

Issues encountered in Step iii Provisioning

Check List before proceeding with BCP

...


...

8.) Teamforge Stop and Start Process:[#]#

While using multi-host setup of Teamforge application, the stop and start process has to be followed in a specified order. There is a possibility that services may go doomed if the order if not followed properly.
---
Stopping a running Teamforge Server:
SCM server > EventQ server > Application Server > DB server
....
Starting a Teamforge Server:
DB server > Application Server > EventQ server > SCM server

...


...

9.) SSL certificate renewal:[#]#

1. Open /opt/collabnet/teamforge/etc/site-options.conf and verify current certificate locations by below tokens and take backup of the same:
SSL_CERT_FILE=
SSL_KEY_FILE=
SSL_CHAIN_FILE=

           (Reviewing site-options.conf of NSE, the certs are in /opt/collabnet/cert/ location)

2. Generate new certificate with Certificate in .crt format, Key in .key format and Chain certificate in .crt format.
3. Replace the certificate, key and chain certificate in /opt/collabnet/cert/ with new one. Perform this action in App server and SCM server. Assuming NSE is generating wildcard certificate with CN = *.nse.co.in, you can use same certificate for App and SCM servers, if not separate certificates are required with appropriate domain names.
4. Reload apache in App server and SCM server: systemctl reload httpd
5. Verify if the new certificate is reflecting in UI.

...


10.) Teamforge Logs for appropriate information:[#]#

/opt/collabnet/teamforge/log/apps/vamessages.log --> Application level log entries

/opt/collabnet/teamforge/log/apps/server.log --> Server level log entries (JBOSS)

/opt/collabnet/teamforge/log/apps/vasession.log --> User login/logout

/opt/collabnet/teamforge/log/apps/vadaemon.log --> Event logs and search logs

/opt/collabnet/teamforge/log/apps/jboss_console.log --> JBOSS dumps

/opt/collabnet/teamforge/log/apps/service.log --> JBOSS service log

/opt/collabnet/teamforge/log/httpd/access --> soap level logs

/opt/collabnet/teamforge/log/httpd/error --> apache errors

/opt/collabnet/teamforge/log/phoenix/mailet.log --> outgoing emails

/opt/collabnet/teamforge/log/phoenix/smtpserver.log --> incoming emails

/opt/collabnet/teamforge/log/phoenix/spoolmanager.log --> mail spooling

/opt/collabnet/teamforge/log/phoenix/default-serach.log --> search index

/opt/collabnet/teamforge/log/integration --> logs on tomcat integrations

/opt/collabnet/teamforge/log/iaf --> logs on binary

/opt/collabnet/teamforge/log/etl --> logs on ETL

/opt/collabnet/teamforge/log/runtime --> logs on runtime recreation, post-install and templates

/opt/collabnet/teamforge/log/add-ons --> logs on add-ons installed in the serve