Issue while starting application services after new cloned instance in 12.2.5

Issue

The Node Manager is already up.
ERROR: Unable to start up the managed server forms_server1

Server specific logs are located at /ebiz/test8040/testapps/apps/fs1/FMW_Home/user_projects/domains/EBS_domain_TESTDB/servers/forms_server1/logs

01/10/19-17:00:46 :: admanagedsrvctl.sh: exiting with status 1

=================================================

Solution


[root@ebizstb ContentsXML]# chmod -R 777 /tmp/
[root@ebizstb ContentsXML]# chmod -R 777 /ebiz/test8040/appltmp/

How to compile apps schema?

How to compile apps schema?

SQL> select count(*) from dba_objects where status='INVALID';

  COUNT(*)
----------
         3

SQL> select name from v$database;

NAME
---------
VISDB

[applvis@apps ~]$ adadmin

                     Copyright (c) 2002 Oracle Corporation
                        Redwood Shores, California, USA

                     Oracle Applications AD Administration

                                 Version 12.0.0

NOTE: You may not use this utility for custom development
      unless you have written permission from Oracle Corporation.

Your default directory is '/apps/vision/apps/apps_st/appl'.
Is this the correct APPL_TOP [Yes] ?

AD Administration records your AD Administration session in a text file
you specify.  Enter your AD Administration log file name or press [Return]
to accept the default file name shown in brackets.

Filename [adadmin.log] : practice.log

************* Start of AD Administration session *************
AD Administration version: 12.0.0
AD Administration started at: Mon Sep 16 2013 00:59:44

APPL_TOP is set to /apps/vision/apps/apps_st/appl

You can be notified by email if a failure occurs.
Do you wish to activate this feature [No] ?

Please enter the batchsize [1000] :


Please enter the name of the Oracle Applications System that this
APPL_TOP belongs to.

The Applications System name must be unique across all Oracle
Applications Systems at your site, must be from 1 to 30 characters
long, may only contain alphanumeric and underscore characters,
and must start with a letter.

Sample Applications System names are: "prod", "test", "demo" and
"Development_2".

Applications System Name [vision] : vision *


NOTE: If you do not currently have certain types of files installed
in this APPL_TOP, you may not be able to perform certain tasks.

Example 1: If you don't have files used for installing or upgrading
the database installed in this area, you cannot install or upgrade
the database from this APPL_TOP.

Example 2: If you don't have forms files installed in this area, you cannot
generate them or run them from this APPL_TOP.

Example 3: If you don't have concurrent program files installed in this area,
you cannot relink concurrent programs or generate reports from this APPL_TOP.


Do you currently have files used for installing or upgrading the database
installed in this APPL_TOP [YES] ? YES *


Do you currently have Java and HTML files for HTML-based functionality
installed in this APPL_TOP [YES] ? YES *


Do you currently have Oracle Applications forms files installed
in this APPL_TOP [YES] ? YES *


Do you currently have concurrent program files installed
in this APPL_TOP [YES] ? YES *


Please enter the name Oracle Applications will use to identify this APPL_TOP.

The APPL_TOP name you select must be unique within an Oracle Applications
System, must be from 1 to 30 characters long, may only contain
alphanumeric and underscore characters, and must start with a letter.

Sample APPL_TOP Names are: "prod_all", "demo3_forms2", and "forms1".

APPL_TOP Name [apps] : apps *



You are about to use or modify Oracle Applications product tables
in your ORACLE database 'vision'
using ORACLE executables in '/apps/vision/apps/tech_st/10.1.2'.

Is this the correct database [Yes] ?

AD Administration needs the password for your 'SYSTEM' ORACLE schema
in order to determine your installation configuration.

Enter the password for your 'SYSTEM' ORACLE schema:


The ORACLE username specified below for Application Object Library
uniquely identifies your existing product group: APPLSYS

Enter the ORACLE password of Application Object Library [APPS] :

AD Administration is verifying your username/password.
The status of various features in this run of AD Administration is:

                                           <-Feature version in->
Feature                          Active?   APPLTOP    Data model    Flags
------------------------------   -------   --------   -----------   -----------
CHECKFILE                        Yes       1          1             Y N N Y N Y
PREREQ                           Yes       6          6             Y N N Y N Y
CONCURRENT_SESSIONS              No        2          2             Y Y N Y Y N
PATCH_TIMING                     Yes       2          2             Y N N Y N Y
PATCH_HIST_IN_DB                 Yes       6          6             Y N N Y N Y
SCHEMA_SWAP                      Yes       1          1             Y N N Y Y Y
JAVA_WORKER                      Yes       1          1             Y N N Y N Y
CODELEVEL                        Yes       1          1             Y N N Y N Y



Identifier for the current session is 16762

Reading product information from file...

Reading language and territory information from file...

Reading language information from applUS.txt ...

AD Administration warning:
 Product Data File


Reading database to see what industry is currently installed.

Reading FND_LANGUAGES to see what is currently installed.
Currently, the following language is installed:

Code   Language                                Status
----   --------------------------------------- ---------
US     American English                        Base

Your base language will be AMERICAN.

Setting up module information.
Reading database for information about the modules.
Saving module information.
Reading database for information about the products.
Reading database for information about how products depend on each other.
Reading topfile.txt ...

Saving product information.

AD code level : [B.1]

            AD Administration Main Menu
   --------------------------------------------------

   1.    Generate Applications Files menu

   2.    Maintain Applications Files menu

   3.    Compile/Reload Applications Database Entities menu

   4.    Maintain Applications Database Entities menu

   5.    Change Maintenance Mode

   6.    Exit AD Administration

Enter your choice [6] : 3

        Compile/Reload Applications Database Entities
   ---------------------------------------------------

   1.    Compile APPS schema

   2.    Compile menu information

   3.    Compile flexfields

   4.    Reload JAR files to database

   5.    Return to Main Menu

Enter your choice [5] : 1

sqlplus -s APPS/***** @/apps/vision/apps/apps_st/appl/ad/12.0.0/sql/adutlrcmp.sql APPLSYS ***** APPS ***** ***** 6  0 NONE FALSE

Review the messages above, then press [Return] to continue.


after compiling apps schema invalid objects are 0.
SQL>  select count(*) from dba_objects where status='INVALID';

  COUNT(*)
----------
         0

SQL>

    Disabling Network Manager

1. Verify Network Manager is currently enabled using the chkconfig command.

   # chkconfig --list NetworkManager

   The output displayed by the chkconfig command inicates whether or not the Network Manager service is enabled.
   • The system displays an error if the Network Manager service is not currently installed:

     error reading information on service NetworkManager: No such file or directory

     If this error is displayed then no further action is required to disable the Network Manager service.
   • The system displays a list of numerical run levels along with a value of on or off indicating whether the Network Manager service is enabled when the system is operating in the given run level.

     NetworkManager  0:off 1:off 2:off 3:off 4:off 5:off 6:off

     If the value displayed for all run levels is off then the Network Manager service is disabled and no further action is required. If the value displayed for any of the run levels is on then the Network Manager service is enabled and further action is required.
2. Ensure that the Network Manager service is stopped using the service command.

   # service NetworkManager stop

3. Ensure that the Network Manager service is disabled using the chkconfig command.

   # chkconfig NetworkManager off

4. Open each interface configuration file on the system in a text editor. Interface configuration files are found in the /etc/sysconfig/network-scripts/ directory and have names of the form ifcfg-X where X is replaced by the name of the interface. Valid interface names include eth0, p1p5, and em1.
   In each file ensure that the NM_CONTROLLED configuration key is set to no and the ON_BOOT configuration key is set to yes.

   NM_CONTROLLED=no
   ONBOOT=yes

   This action ensures that the standard network service will take control of the interfaces and automatically activate them on boot.
5. Ensure that the network service is started using the service command.

   # service network start

6. Ensure that the network service is enabled using the chkconfig command.

   # chkconfig network on

The Network Manager service has been disabled. The standard network service has been enabled and configured to control the required network interfaces.

Sql query to find all scheduled concurrent request

SELECT DISTINCT frl.responsibility_name,
fu.user_name,
fcr.request_id,
(SELECT meaning
FROM apps.fnd_lookups
WHERE lookup_type='CP_PHASE_CODE'
AND lookup_code =fcr.phase_code
) Phase,
(SELECT meaning
FROM apps.fnd_lookups
WHERE lookup_type='CP_STATUS_CODE'
AND lookup_code =fcr.status_code) Status,
fcs.program,
to_date(fcr.requested_start_date,'DD-MM-RRRR hh24:mi:ss') Start_Date,
fcr.resubmit_interval || ' ' ||fcr.resubmit_interval_unit_code Resubmit_Interval,
NVL2(fcr.resubmit_interval,'PERIODICALLY',NVL2(fcr.release_class_id, 'ON SPECIFIC DAYS','ONCE')) schedule_type,
fcs.argument_text
FROM apps.fnd_concurrent_requests fcr,
apps.fnd_concurrent_programs_tl fcp,
apps.fnd_responsibility_tl frl,
apps.fnd_user fu,
apps.fnd_conc_req_summary_v fcs
WHERE fcr.phase_code ='P'
AND fcr.request_id = fcs.request_id
AND frl.language ='US'
AND fcr.requested_by =fu.user_id
AND fcr.responsibility_id =frl.responsibility_id
AND fcr.status_code IN ('P','Q','I')
AND fcp.language ='US'
AND fcp.source_lang ='US'
AND (NVL(fcr.request_type, 'X') != 'S')
AND fcr.concurrent_program_id =fcp.concurrent_program_id
AND fcr.requested_start_date >= SYSDATE
--AND to_date(fcr.requested_start_date,'DD-MM-RRRR hh24:mi:ss')
--BETWEEN NVL(to_date(:p_from_date,'DD-MM-RRRR hh24:mi:ss'),fcr.requested_start_date) AND NVL(to_date(:p_to_date,'DD-MM-RRRR hh24:mi:ss'),fcr.requested_start_date)
ORDER BY program DESC

Get a list of all functions and procedures in an Oracle database

SELECT * FROM ALL_OBJECTS WHERE OBJECT_TYPE IN ('FUNCTION','PROCEDURE','PACKAGE')

The column STATUS tells you whether the object is VALID or INVALID. If it is invalid, you have to try a recompile, ORACLE can't tell you if it will work before.

OS Watcher Install & Run

Installing OSWbb

To install OSWbb:

1. Log on to My Oracle Support (MOS) at https://support.oracle.com.
2. Download the file oswbb601.tar, which is available at https://support.oracle.com/epmos/main/downloadattachmentprocessor?attachid=301137.1:OSW_file.
3. Copy the file to the directory where you want to install OSWbb, and run the following command:
   

   # tar xvf oswbb601.tar

   
   Extracting the tar file creates a directory named oswbb, which contains all the directories and files that are associated with OSWbb, including the startOSWbb.sh script.
4. If the ksh package is not already installed on your system, use yum to install it.
   

   # yum install ksh

5. Create a symbolic link from /usr/bin/ksh to /bin/ksh.
   

   # ln -s /bin/ksh /usr/bin/ksh

   
   This link is required because the OSWbb scripts expect to find ksh in /usr/bin.
6. To enable the collection of iostat information for NFS volumes, edit the OSWatcher.sh script in the oswbb directory, and set the value of nfs_collect to 1:
   

   nfs_collect=1

Running OSWbb

To start OSWbb, run the startOSWbb.sh script from the oswbb directory.

# ./startOSWbb.sh [frequency duration]

The optional frequency and duration arguments specifying how often in seconds OSWbb should collect data and the number of hours for which OSWbb should run. The default values are 30 seconds and 48 hours. The following example starts OSWbb recording data at intervals of 60 seconds, and has it record data for 12 hours:

# ./startOSWbb.sh 60 12
Testing for discovery of OS Utilities
.
.
.
VMSTAT found on your system.
IOSTAT found on your system.
MPSTAT found on your system.
NETSTAT found on your system.
TOP found on your system.
Testing for discovery of OS CPU COUNT
.
.
.
Starting Data Collection...
oswbb heartbeat: date/time
oswbb heartbeat: date/time + 60 seconds
.
.
.

To stop OSWbb prematurely, run the stopOSWbb.sh script from the oswbb directory.

# ./stopOSWbb.sh

OSWbb collects data in the following directories under the oswbb/archive directory:

Directory	Description
oswiostat	Contains output from the iostat utility.
oswmeminfo	Contains a listing of the contents of /proc/meminfo.
oswmpstat	Contains output from the mpstat utility.
oswnetstat	Contains output from the netstat utility.
oswprvtnet	If you have enable private network tracing for RAC, contains information about the status of the private networks.
oswps	Contains output from the ps utility.
oswslabinfo	Contains a listing of the contents of /proc/slabinfo.
oswtop	Contains output from the top utility.
oswvmstat	Contains output from the vmstat utility.

OSWbb stores data in hourly archive files named system_name_utility_name_timestamp.dat, and each entry in a file is preceded by the characters *** and a timestamp.

The Node Manager is already up.ERROR:

Unable to start up the managed server forms_server1

Oracle Apps R12 (12.2.2.5) Application startup after the clone How to solve java.lang.OutOfMemoryError: unable to create new native thread Solution Check number of processes per user $ ps -elf | wc -l 220 ------------------------------------------------------------------------------------- If you try to run ps with a -T you will see all of the threads as well: $ ps -elfT | wc -l 385 ------------------------------------------------------------------------------- you need to increase the number of processes for the user. This can be done with the following command: $ ulimit -u 4096 ---------------------------------------------------------------------------- Check your threads PID limit Once that you have counted the number of threads, then you should verify that you are not hitting system limits, specified by the kernel.pid_max limit parameter. You can check this value by executing: $ sysctl -a | grep kernel.pid_max kernel.pid_max = 32768 -----------------------------------------------------------------