With kernel.core_pattern=|, format_corename() still interprets this as the pipe path with:
191 static int format_corename(struct core_name *cn, struct coredump_params *cprm,
192 size_t **argv, int *argc)
193 {
...
196 int ispipe = (*pat_ptr == '|');
...
335 return ispipe;
336 }
Which checks the first byte of the core_pattern string to see if it is a "|".
This means we really want the call to call_usermodehelper_exec() in do_coredump() to fail, so we take the if(retval) step and output "Core dump to |%s pipe failed\n".
Note that cn.corename is NULL when kernel.core_pattern=|.
For some reason with 5.3 and later, this call to call_usermodehelper_exec() is not failing, and execution continues on and breaks at file_start_write(cprm.file); at the end of do_coredump().
call_usermodehelper_exec() calls umh_pipe_setup() and creates the pipe used. Something has likely changed in the pipe subsystem.
Yes, you are absolutely correct. I can reproduce this now, when kernel.core_pattern is set to "|".
I can also confirm that the first kernel that this is broken in is 5.3, as it works fine in 5.2 and below.
I will look into this and hopefully get this fixed for you. Thanks for reporting!
Engineering Notes:
RIP: 0010:do_ coredump+ 0x536/0xb30
$ eu-addr2line do_coredump+0x536 -e vmlinux- 5.3.0-29- generic linux-OZAq_ R/linux- 5.3.0/include/ linux/fs. h:2841: 7
/build/
This is the function file_start_write():
2839 static inline void file_start_ write(struct file *file) file_inode( file)-> i_mode) ) write(file_ inode(file) ->i_sb, SB_FREEZE_WRITE, true);
2840 {
2841 if (!S_ISREG(
2842 return;
2843 __sb_start_
2844 }
This is called from do_coredump():
565 void do_coredump(const kernel_siginfo_t *siginfo) interrupted( )) { write(cprm. file); >core_dump( &cprm); write(cprm. file);
566 {
...
788 if (!dump_
789 file_start_
790 core_dumped = binfmt-
791 file_end_
792 }
...
810 }
On kernels 5.2 and before, kernel. core_pattern= | normally outputs:
[ 39.328638] Core dump to | pipe failed
This is output is from the pipe section of do_coredump():
565 void do_coredump(const kernel_siginfo_t *siginfo) corename( &cn, &cprm, &argv, &argc); lper_setup( helper_ argv[0] , lper_exec( sub_info,
566 {
...
623 ispipe = format_
624
625 if (ispipe) {
...
681 sub_info = call_usermodehe
682 helper_argv, NULL, GFP_KERNEL,
683 umh_pipe_setup, NULL, &cprm);
684 if (sub_info)
685 retval = call_usermodehe
686 UMH_WAIT_EXEC);
687
688 kfree(helper_argv);
689 if (retval) {
690 printk(KERN_INFO "Core dump to |%s pipe failed\n",
691 cn.corename);
692 goto close_fail;
693 }
...
810 }
With kernel. core_pattern= |, format_corename() still interprets this as the pipe path with:
191 static int format_ corename( struct core_name *cn, struct coredump_params *cprm,
192 size_t **argv, int *argc)
193 {
...
196 int ispipe = (*pat_ptr == '|');
...
335 return ispipe;
336 }
Which checks the first byte of the core_pattern string to see if it is a "|".
This means we really want the call to call_usermodehe lper_exec( ) in do_coredump() to fail, so we take the if(retval) step and output "Core dump to |%s pipe failed\n".
Note that cn.corename is NULL when kernel. core_pattern= |.
For some reason with 5.3 and later, this call to call_usermodehe lper_exec( ) is not failing, and execution continues on and breaks at file_start_ write(cprm. file); at the end of do_coredump().
call_usermodehe lper_exec( ) calls umh_pipe_setup() and creates the pipe used. Something has likely changed in the pipe subsystem.
I will update when I figure out what.