== Comment: #0 - Application Cdeadmin <email address hidden> - 2018-04-20 05:56:03 ==


== Comment: #1 - Application Cdeadmin <email address hidden> - 2018-04-20 05:56:05 ==
==== State: Open by: stccdenv on 18 April 2018 06:27:28 ====

This defect was opened automatically using defect_template with only logs. Please refer to 2nd seq for the problem description.

==========================Automatic entries==========================
Full Log: http://w3.austin.ibm.com/afs/awd.austin.ibm.com/u/stccdenv/logs/devops4fp1_defect_template_20180418060138

Contact: Thirukumaran V T (<email address hidden>), Tommy Adams(<email address hidden>)
Backup: Atit Patel (<email address hidden>)

System Name: devops4fp1
FSP IP: 9.3.136.192 (devops4fp1.aus.stglabs.ibm.com)
System Firmware Level:
 Current Side Driver:.....fips861/b0413a_1816.861
 Non-Current Side Driver:.fips861/b0410a_1816.861

Lpar Access: Please refer https://pcajet.aus.stglabs.ibm.com/ for the lab password. (The Lab Test Passwords are now accessible only through the auto or manual install web apps. For example, from the manual install web app, enter your email address, check the Lab Passwords checkbox and then click on Submit.)
(To access the Lpars in 10.33.x.x network, login to any LCB or HMC then ssh/telnet to 10.33.x.x lpars)

-------------------------------------------------------
HMC IP: 9.3.118.110 (vhmccloudtst100.aus.stglabs.ibm.com)
HMC Version:
"version= Version: 9
 Release: 1
 Service Pack: 910
HMC Build level 1803052221
","base_version=V9R1
"

-------------------------------------------------------

=================================Logs================================
SNAP : 
devops4p02: /tmp/ibmsupt/snap.pax.Z.devops4p02.180418060149 and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/snap.pax.Z.devops4p02.180418060149  /tmp/IBM.DRM.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060149.tar.gz.devops4p02  /tmp/htx.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060149.tar.gz.devops4p02  
devops4p03: /tmp/ibmsupt/snap.pax.Z.devops4p03.180418060149 and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/snap.pax.Z.devops4p03.180418060149  /tmp/IBM.DRM.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060149.tar.gz.devops4p03  /tmp/htx.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060149.tar.gz.devops4p03  
devops4p04: /tmp/ibmsupt/snap.pax.Z.devops4p04.180418060148 and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/snap.pax.Z.devops4p04.180418060148  /tmp/IBM.DRM.180418060148.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060148.tar.gz.devops4p04  /tmp/htx.180418060148.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060148.tar.gz.devops4p04  
devops4p05: /tmp/ibmsupt/snap.pax.Z.devops4p05.180418060148 and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/snap.pax.Z.devops4p05.180418060148  /tmp/IBM.DRM.180418060148.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060148.tar.gz.devops4p05  /tmp/htx.180418060148.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060148.tar.gz.devops4p05  
devops4p06: /tmp/ctsupt/ctsnap.linux-hjmh.04180701.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/ctsnap.linux-hjmh.04180701.tar.gz  /tmp/var_log.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/var_log.180418060149.tar.gz  /var/log/nts_linux-hjmh_180418_0703.tbz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/nts_linux-hjmh_180418_0703.tbz  /tmp/IBM.DRM.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060149.tar.gz.devops4p06  /tmp/htx.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060149.tar.gz.devops4p06  
devops4p07: /tmp/ctsupt/ctsnap.devops4p07.04180601.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/ctsnap.devops4p07.04180601.tar.gz  /tmp/var_log.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/var_log.180418060149.tar.gz  /var/tmp/sosreport-devops4p07.aus.stglabs.ibm.com-20180418060350.tar.xz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/sosreport-devops4p07.aus.stglabs.ibm.com-20180418060350.tar.xz  /tmp/IBM.DRM.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060149.tar.gz.devops4p07  /tmp/htx.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060149.tar.gz.devops4p07  
devops4p08: /tmp/ctsupt/ctsnap.devops4p08.04180601.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/ctsnap.devops4p08.04180601.tar.gz  /tmp/var_log.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/var_log.180418060149.tar.gz  /var/tmp/sosreport-devops4p08-20180418060355.tar.xz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/sosreport-devops4p08-20180418060355.tar.xz  /tmp/IBM.DRM.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060149.tar.gz.devops4p08  /tmp/htx.180418060149.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060149.tar.gz.devops4p08  
devops4p09: /tmp/ctsupt/ctsnap.devops4p09.04180602.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/ctsnap.devops4p09.04180602.tar.gz  /tmp/var_log.180418060152.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/var_log.180418060152.tar.gz  /tmp/sosreport-devops4p09-20180418060347.tar.xz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/sosreport-devops4p09-20180418060347.tar.xz  /tmp/IBM.DRM.180418060152.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060152.tar.gz.devops4p09  /tmp/htx.180418060152.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060152.tar.gz.devops4p09  
devops4p10: /tmp/ctsupt/ctsnap.devops4p10.0418060209.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/ctsnap.devops4p10.0418060209.tar.gz  Warning: Permanently added 'devops4p10,9.3.13.168' (ECDSA) to the list of known hosts.
 tar: Removing leading `/' from member names /var/log/ /var/log/drmgr /var/log/epow_status /var/log/htx_install_path /var/log/dmesg /var/log/alternatives.log /var/log/kern.log /var/log/auth.log /var/log/fontconfig.log /var/log/ppc64-diag/ /var/log/ppc64-diag/diag_disk/ /var/log/apt/ /var/log/apt/history.log /var/log/apt/term.log /var/log/apt/eipp.log.xz /var/log/platform /var/log/unattended-upgrades/ /var/log/unattended-upgrades/unattended-upgrades-shutdown.log /var/log/dump/ /var/log/installer/ /var/log/installer/initial-status.gz /var/log/installer/lsb-release /var/log/installer/syslog /var/log/installer/hardware-summary /var/log/installer/status /var/log/installer/cdebconf/ /var/log/installer/cdebconf/templates.dat /var/log/installer/cdebconf/questions.dat /var/log/installer/partman /var/log/syslog /var/log/btmp /var/log/sysstat/ /var/log/lastlog /var/log/fsck/ /var/log/fsck/checkfs /var/log/fsck/checkroot /var/log/wtmp /var/log/faillog /var/log/rtas_errd.log /var/log/indicators /var/log/dpkg.log /var/log/dist-upgrade/ /var/log/dist-upgrade/apt-clone_system_state.tar.gz /var/log/dist-upgrade/20180418-0044/ /var/log/dist-upgrade/20180418-0044/xorg_fixup.log /var/log/dist-upgrade/20180418-0044/apt-term.log /var/log/dist-upgrade/20180418-0044/history.log /var/log/dist-upgrade/20180418-0044/apt.log /var/log/dist-upgrade/20180418-0044/main.log /var/log/dist-upgrade/apt-term.log /var/log/dist-upgrade/history.log /var/log/dist-upgrade/apt.log /var/log/dist-upgrade/main.log /var/log/dist-upgrade/eipp.log.xz /var/log/dist-upgrade/lspci.txt /var/log/journal/ /var/log/journal/ef15610498e144f3ad490f1510be51fb/ /var/log/journal/ef15610498e144f3ad490f1510be51fb/system.journal /var/log/journal/ef15610498e144f3ad490f1510be51fb/system@89fa121dce60471fb8f8db895a3ee928-0000000000000001-00056a16c388300c.journal tar: /var/log/journal/ef15610498e144f3ad490f1510be51fb/system.journal: file changed as we read it /tmp/sosreport-devops4p10-20180418060610.tar.xz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/sosreport-devops4p10-20180418060610.tar.xz  /tmp/IBM.DRM.180418060156.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060156.tar.gz.devops4p10  /tmp/htx.180418060156.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/htx.180418060156.tar.gz.devops4p10  
devops4p01: /home/padmin/snap.pax.Z.180418060145 and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/snap.pax.Z.devops4p01.180418060145  /home/padmin/IBM.DRM.180418060145.tar.gz and also @ (NFS)9.41.164.242:/fspmount/hmc_dumps/IBM.DRM.180418060145.tar.gz.devops4p01  

Jenkins Log:
http://gfwa123.aus.stglabs.ibm.com:8082/job/STC%20Automation%20Test/401/
===================================END===============================
==== State: Open by: stccdenv on 20 April 2018 05:54:35 ====

Hi Ubuntu Screen Team,

We encountered the following message while polling for dmesg on the LINUX Ubuntu LPAR using the command, dmesg -xT -l emerg,alert,crit,err

(0) root @ devops4p10: /tmp
# dmesg -xT -l emerg,alert,crit,err
kern  :err   : [Wed Apr 18 05:23:53 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Wed Apr 18 06:07:02 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Fri Apr 20 05:28:44 2018] print_req_error: critical target error, dev sr0, sector 1457408

Attached with this CQ is the drmgr, dmesg and sosreport logs.

Thank you.

stccdenv (<email address hidden>) added native attachment /opt/IBM/WebSphere/AppServer/profiles/cqweb/temp/cqweb06Node01/server1/TeamEAR/cqweb.war/rkJwXSo9czeR_XEsVOVNgOr_1524221306674/dmesg on 2018-04-20 05:53:32
stccdenv (<email address hidden>) added native attachment /opt/IBM/WebSphere/AppServer/profiles/cqweb/temp/cqweb06Node01/server1/TeamEAR/cqweb.war/rkJwXSo9czeR_XEsVOVNgOr_1524221322776/drmgr on 2018-04-20 05:53:32
stccdenv (<email address hidden>) added native attachment /opt/IBM/WebSphere/AppServer/profiles/cqweb/temp/cqweb06Node01/server1/TeamEAR/cqweb.war/rkJwXSo9czeR_XEsVOVNgOr_1524221405139/sosreport-devops4p10-20180420052752.tar.xz on 2018-04-20 05:53:32

stccdenv (<email address hidden>) added native attachment /opt/IBM/WebSphere/AppServer/profiles/cqweb/temp/cqweb06Node01/server1/TeamEAR/cqweb.war/rkJwXSo9czeR_XEsVOVNgOr_1524221306674/dmesg on 2018-04-20 05:54:35
stccdenv (<email address hidden>) added native attachment /opt/IBM/WebSphere/AppServer/profiles/cqweb/temp/cqweb06Node01/server1/TeamEAR/cqweb.war/rkJwXSo9czeR_XEsVOVNgOr_1524221322776/drmgr on 2018-04-20 05:54:35

== Comment: #2 - Application Cdeadmin <email address hidden> - 2018-04-20 05:56:09 ==


== Comment: #3 - Application Cdeadmin <email address hidden> - 2018-04-20 05:56:10 ==


== Comment: #4 - STC-Continous Delivery-Environment <email address hidden> - 2018-04-20 06:00:27 ==
sosreport

== Comment: #5 - Application Cdeadmin <email address hidden> - 2018-04-20 06:04:08 ==
==== State: Open by: wongklb on 20 April 2018 06:02:53 ====

sosreport link location => https://bugzilla.linux.ibm.com/attachment.cgi?id=126432

== Comment: #7 - Application Cdeadmin <email address hidden> - 2018-04-23 02:24:36 ==
==== State: MoreInfo by: tthananj on 23 April 2018 02:14:00 ====

Yes, these errors were reported with quiet splash.

(0) root @ devops4p10: /root
# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinux-4.15.0-15-generic root=UUID=ffd7bed7-8088-4c3d-9e31-51ce962cb044 ro quiet splash

(0) root @ devops4p10: /root
# dmesg -xT -l emerg,alert,crit,err
kern  :err   : [Wed Apr 18 05:23:55 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Wed Apr 18 06:07:04 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Fri Apr 20 05:28:46 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Fri Apr 20 05:55:33 2018] print_req_error: critical target error, dev sr0, sector 1457408
kern  :err   : [Fri Apr 20 05:59:08 2018] print_req_error: critical target error, dev sr0, sector 1457408