Upgrading a kernel was an easy move. But the growing complexity of SAP-systems, makes an
upgrade of the kernel more difficult than it was in the past.
I was motivated to change the kernel by the PAM on the SAP service marketplace. I checked the "lifetime" of the kernel by:
PAM-page
searching my sap product
Hmm bad news
A System without supported kernel? No way !
I needed a new kernel and the newest was 7.21 EXT I looked at the PAM and found:
In order to be able to do the kernel upgrade in a proper way, I searched for the necessary
notes. I found 3 essential notes for the kernel upgrade.
Note 1716826 - Usage of the downward compatible kernel 721 (EXT)
Note 1728283 - SAP Kernel 721: General Information
Note 1713986 - Installation of kernel 721 (EXT)
Now I had all the information I needed. The next task was to download the new kernel.
http://service.sap.com/patches -> Support Packages and Patches -> A - Z Index -> ....
And now a little happy question time:
Question: When I search the kernel for a SAP PI 7.1 EHP 1 system, where do I have to search?
1. S for SAP?
2. P for PI?
3. Z for Zorro?
Answer: No, it's N for Netweaver -> 1 hour lost for the answer of this question
http://service.sap.com/patches -> Support Packages and Patches -> A - Z Index -> SAP NETWEAVER PI 7.1X -> SAP EHP1 FOR SAP NW PI 7.1 -> Entry by Component -> Application Server ABAP
I downloaded the following archives:
SAPCAR_315-20010445.EXE SAPCAR
DBATL720O10_31-20006704.SAR DBATOOLS Package for Oracle 10g and 11g
SAPEXE_100-10011322.SAR Kernel Part I (database independent)
SAPEXEDB_100-10011323.SAR Kernel Part II (database specific)
igsexe_5-20007786.sar Internet Graphic Server (IGS)
igshelper_3-10010245.sar IGS Helper
sapwebdisp_421-20008606.sar SAP Webdispatcher
SAPCRYPTOLIB_34-10010842.SAR SAP Kryptolib
SAPHOSTAGENT147_147-20005726.SAR SAPhostagent 7.20
After analyzing the 3 notes, I identified the next tasks:
1. ) check Note 1610716 - Correcting runtime objects with incorrect alignment
2.) update SAPJVM 5.1.047 to 5.1.084
3.) implement the latest SBP (SAP bundle patch)
4.) Upgrade the kernel
ok 4 tasks to do. Let's do it:
Task 1: check Note 1610716 - Correcting runtime objects with incorrect alignment
I had to implement Note 1610716 with the transaction SNOTE and I got the report RUT_NOTE_1610716. I started the report first with option check and after that with option repair. It ran several minutes. Task was done.
Task 2: update SAPJVM 5.1.047 to 5.1.084
Ok, now I need to search all relevant notes for patching a SAPJVM. I found:
1683392 - SAP JVM 5.1 Patch-Collection 58 (Build 5.1.074)
1434916 - How to find out the SAP JVM build version
1367498 - Installationsvoraussetzungen für SAP JVM
1025085 - How to manually patch the SAPJVM
1133020 - Importieren eines SAP-JVM-Patchs in Application Server Java
With note 1434916 I was able to identify the SAPJVM-Version. I was not able to identify the version of sapjvm on the system information page:( In my opinion the system information page of NW 7.0 is much better than on NW 7.1! The system information page of NW 7.0 is more simple, but the information is well-arranged.
Note 1025085 states:
"Patching the SAP JVM on an SAP NetWeaver system is only supported using the Software Update Manager (SUM) or the Java Support Package Manager (JSPM)."
I chose the JSPM.
After patching sapjvm I started the system and... it started without any error and... no java page could be displayed
I posted my problem and a guru (big thanx to Reagan Benjamin) found the solution see:
http://scn.sap.com/thread/3388835 (No http page available after upgrade of SAPJVM)
After implementing Note 1625051 - "Wily Introscope agent: IllegalAccessError" the system runs fine again
ok next step:
Task 3) implement the latest SBP (SAP bundle patch)
Where can I find the SBP?
http://service.sap.com/patches -> Database and Database Patches (from other vendors) -> Oracle -> Oracle Patches 11.2.0.2.0
a) Checks
orasid> cd /oracle/SID/11203; bdf .
orasid> cd /oracle/stage/11203/database/SAP; bdf .
both filesystems should be 1GB free
b) Patche Opatch and MOPatch
orasid> setenv IHRDBMS /oracle/SID/11203; setenv OHRDBMS /oracle/SID/112_64; setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> cp -p /inst_cd_sap/IA64/Oracle110203/updates/SAP_112035_201303/SAP_112035_201303_HPUXIA64.zip .
orasid> /inst_cd_sap/IA64/SOLMAN_71/unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip 'SBP_112035_201303/OPatch/*'
orasid> mv $IHRDBMS/OPatch $IHRDBMS/OPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/OPatch $IHRDBMS/OPatch
orasid> unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip 'SBP_112035_201303/MOPatch/*'
orasid> test -d $IHRDBMS/MOPatch && mv $IHRDBMS/MOPatch $IHRDBMS/MOPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/MOPatch $IHRDBMS/MOPatch
orasid> $ORACLE_HOME/OPatch/opatch version
orasid> $ORACLE_HOME/MOPatch/mopatch.sh -h
The following versinos should be displayed:
OPatch version 11.2.0.3.1
MOPatch version 2.1.13
c) stop the system
sidadm> stopsap
orasid> lsnrctl stop
orasid> ps -efax | grep ora
no ora process should run!
d) install SBP
orasid> setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> /bin/sh $ORACLE_HOME/MOPatch/mopatch.sh -v -s SAP_112035_201303_HPUXIA64.zip
orasid> lsnrctl start
<open a new session>
oraSID> sqlplus "/as sysdba"
SQL> startup
<old session>
oraSID> cd $OHRDBMS/rdbms/admin
oraSID> env ORACLE_HOME=$OHRDBMS $OHRDBMS/bin/sqlplus "/as sysdba"
SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql
Hmmmm, INCOMPLETE. This is a word sap administrators don't like! After reading Note 1509324 I did as it was recommended in the note.
SQL> select action_time from registry$history group by action_time having count(*) > 1;
If rows appear then read OSS-Note: 1508602.
no rows, great.
SQL> @?/rdbms/admin/utlrp.sql;
SQL> shutdown
SQL> startup
SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql
No errors and two COMPLETEs. That was good.
I had to set two parameters ->
SQL> ALTER SYSTEM SET "_FIX_CONTROL"=
'5099019:ON','5705630:ON','6055658:OFF','6399597:ON','6430500:ON',
'6440977:ON','6626018:ON','6972291:ON','7168184:OFF','8937971:ON',
'9196440:ON','9495669:ON','13077335:ON','13627489:ON','14255600:OFF',
'14595273:ON' COMMENT='SAP_112035_201302 RECOMMENDED SETTINGS' SCOPE=SPFILE;
SQL> ALTER SYSTEM SET EVENT=
'10027',
'10028',
'10142',
'10183',
'10191',
'10995 level 2',
'31991',
'38068 level 100',
'38085',
'38087',
'44951 level 1024'
COMMENT='SAP_112035_201302 RECOMMENDED SETTINGS' SCOPE=SPFILE;
SQL> shutdown immediate
SQL> startup
SQL> shutdown immediate
SBP is installed, Task done!
Task 4: Upgrade the kernel
This is not as easy as it was in the good old days! Now you have to do some more steps. In the good old days (when SAP ERP was R/3) you patched the system between 12:00 and 12:30 when users were at lunch! (Is it lunch or dinner? Hmm I don't know. Please excuse my bad English!) But today you have to do it on weekend.
So let's start.
Step 1: saphostagent upgrade
root> cd /tmp
root> mkdir saphostagent; cd saphostagent
root> /sapmnt/SID/SAPCAR -xvf /inst_cd_sap/IA64/kernel/PI_721_EXT_100/SAPHOSTAGENT147_147-20005726.SAR
root> ./saphostexec -upgrade
Step 2: stop the system
Stopping the saphostagent:
root> /usr/sap/hostctrl/exe/saphostexec -stop
root> /usr/sap/hostctrl/exe/saposcol -k
Stopping the sapwebdisp:
<user of sapwebdisp> stopsap all W90
Stopping sapstartsrv:
sidadm> sapcontrol -nr <Systemnummer Instanz z.B. 00> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer ASCS> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer SCS> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer ERS> -prot NI_HTTP -function StopService;
Deregister and stop CCMS-Agenten:
sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> ./sapccm4x -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host>
sidadm> ./ccmsping -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host> -push -n<Sy.Nr>
stop Diagnosticagent
<D-Agentuser> stopsap SMDA90
Step 3: Clear shared memory
sidadm> /usr/sap/SID/SYS/exe/run/showipc all
Shows all shared memorys of all systems.
sidadm> /usr/sap/SID/SYS/exe/run/cleanipc <SystemNrIPC> remove
Now check for open shared memory segments. There should be no segment
sidadm> /usr/sap/SID/SYS/exe/run/showipc all
Step 4: get SAPCAR
sidadm> mkdir /tmp/sapcar
sidadm> cd /tmp/sapcar
sidadm> SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> cp SAPCAR /sapmnt/SID/
Step 5: removing the old kernel
root> cd /usr/sap/SID/SYS/exe/run/
root> rm -rf *
Step 6: implement the new kernel
sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXEDB_100-10011323.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/DBATL720O10_31-20006704.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igsexe_5-20007786.sar
Step 7: implement sapcryprtolib
sidadm> mkdir /tmp/sapcrypto
sidadm> cd /tmp/sapcrypto
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPCRYPTOLIB_34-10010842.SAR
sidadm> cp -p hpia64-11.31-64/* /usr/sap/SID/SYS/exe/run/
sidadm> cp -p hpia64-11.31-64/ticket /usr/sap/SID/DVEBMGS<Sy.Nr>/sec
Step 8: start saproot
root> cd /usr/sap/SID/SYS/exe/run/
root> ./saproot.sh SID
Step 9: install igshelper
sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr.>; mv igs igs_old;
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igshelper_3-10010245.sar
Step 10: new kernel for sapwebdisp
<user of sapwebdisp> cd /usr/sap/SWP_SID/SYS/exe/run
<user of sapwebdisp> rm -rf *
<user of sapwebdisp> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/sapwebdisp_421-20008606.sar
Step 11: delete the exe-directories of all instances
root> cd /usr/sap/SID/DVEBMGS<Sy.Nr. of instance>/exe
root> rm -rf *
root> cd /usr/sap/SID/ASCS<Sy.Nr. of ASCS>/exe
root> rm -rf *
root> cd /usr/sap/SID/SCS<Sy.Nr. of SCS/exe
root> rm -rf *
root> cd /usr/sap/SID/ERS<Sy.Nr.of ERS>/exe
root> rm -rf *
Step 12: execute SAPCPE
It is recommended by sap to start sapcpe for every instance. Be aware to use the scs.lst file for the central services and the ERS!
For the central instance:
sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr. instance>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host>
For the central services:
sidadm> cd /usr/sap/SID/SCS<Sy.Nr. SCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_SCS<Sy.Nr. SCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
sidadm> cd /usr/sap/SID/ASCS<Sy.Nr. ASCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ASCS<Sy.Nr. ASCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
And for the Enhanced Replication Services (I only show it for one ERS because I'm too lazy;))
sidadm> cd /usr/sap/SID/ERS<Sy.Nr. ERS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ERS<Sy.Nr. ERS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
Last but not least the jvm needs it's binaries too:
sidadm> sapcpe /usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host> source:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084 list:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084/sapjvm_5.lst
Step 12: post activities
Enter line
rslg/new_layout = 9
in file /usr/sap/SIS/SYS/profile/DEFAULT.PFL
Then you have to delete the files of the syslog.
Final Step (13): startsap
Puhhh 13 steps…I entered
sidam> startsap
Sometimes startsap is a layer, so verified it:
sidadm> ps -efax | grep sap
ok. ABAP was running, but J2EE is a diva. I took a look at the dev_server0 and…
Thanx god java (the beast) is up and running
That was my adventure kernel patching a PI system. Finally I leaved the office and tried to relax at the rest of Sunday 
Best regards
Willi Eimler