[Ubuntu 17.04] 'make test' failures seen when e2fsprogs instrumented with 'undefined behavior sanitizer'

Bug #1702240 reported by bugproxy on 2017-07-04
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
The Ubuntu-power-systems project
Medium
Canonical Foundations Team
e2fsprogs (Ubuntu)
Medium
Ubuntu on IBM Power Systems Bug Triage

Bug Description

---Problem Description---
'make test' shows '262 tests failed' when e2fsprogs instrumented with 'Undefined Behavior Sanitizer'

Contact Information = <email address hidden>

---uname output---
Linux powerkvm1-lp1 4.10.0-9-generic #11-Ubuntu SMP Mon Feb 20 13:45:11 UTC 2017 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = 8284-22A

---Debugger---
A debugger is not configured

---Steps to Reproduce---
 1. Prepare the build environment and download e2fsprogs
apt-get build-dep -y e2fsprogs
mkdir pkg
cd pkg
apt-get source e2fsprogs
cd e2fsprogs-1.43.4/

2. Now instrument with 'undefined behavior sanitizer' and run 'make test'
dh_testdir
dh_auto_configure -- CFLAGS="-fsanitize=undefined -fsanitize=address" LDFLAGS="-fsanitize=undefined -fsanitize=address" CXXFLAGS="-fsanitize=undefined -fsanitize=address"
dh_auto_build
dh_auto_test

...
65 tests succeeded 262 tests failed
Tests failed: d_dumpe2fs_group_only d_fallocate_bigalloc d_fallocate_blkmap d_fallocate d_loaddump d_punch_bigalloc d_punch d_special_files d_xattr_edits d_xattr_sorting f_16384_block f_8192_block f_bad_bbitmap f_badbblocks f_bad_bmap_csum f_badcluster f_baddir2 f_baddir f_bad_disconnected_inode f_baddotdir f_bad_gdt_csum f_bad_ibitmap f_bad_inode_csum f_badinode f_badjourblks f_badjour_indblks f_bad_local_jnl f_badorphan f_badprimary f_badroot f_badsymlinks f_badtable f_bbfile f_bb_in_bb f_bbinode f_big_sparse f_bitmaps f_boundscheck f_clear_xattr f_cloneblock_alloc_error f_collapse_extent_tree f_compress_extent_tree_level f_convert_bmap_and_extent f_convert_bmap f_corrupt_dirent_tail f_create_symlinks f_deleted_inode_bad_csum f_desc_size_128 f_desc_size_bad f_dir_bad_csum f_dir_bad_mode f_dirlink f_dup2 f_dup3 f_dup4 f_dup_ba f_dup_de2 f_dup_de f_dupdot f_dup f_dupfsblks f_dup_resize f_dupsuper f_ea_bad_csum f_ea_checks f_ea_value_crash f_emptydir f_encrypted_lpf f_end-bitmap f_eofblocks f_expand f_expandroot_create_lnf f_extent_bad_node f_extent_htree f_extent_int_bad_csum f_extent_int_bad_extent f_extent_int_bad_magic f_extent_interior_start_lblk f_extent_leaf_bad_csum f_extent_leaf_bad_extent f_extent_leaf_bad_magic f_extent_oobounds f_extents2 f_extents f_extent_too_deep f_ext_journal f_extra_journal f_ext_zero_len f_fast_symlink_extents f_file_acl_high f_filetype f_first_meta_bg_too_big f_full_bg f_h_badnode f_h_badroot f_h_normal f_holedir2 f_holedir3 f_holedir4 f_holedir f_h_reindex f_htree_bad_csum f_htree_leaf_csum f_hugedir_blocks f_h_unsigned f_hurd f_idata_and_extents f_illbbitmap f_illibitmap f_illitable f_illitable_flexbg f_imagic f_imagic_fs f_ind_inode_collision f_inlinedata_dirblocks f_inlinedata_repair f_inlinedir_detector f_inode_ea_collision f_invalid_bad_inode f_invalid_extent_symlink f_itable_collision f_jnl_32bit f_jnl_64bit f_jnl_errno f_jnl_etb_alloc_fail f_journal f_lotsbad f_lpf2 f_lpf f_lpffile f_messy_inode f_miss_blk_bmap f_miss_journal f_misstable f_mke2fs2b f_mmp_garbage f_no_cache_corrupt_inode f_no f_noroot f_nospc_create_lnf f_okgroup f_opt_extent_ext3 f_opt_extent f_orphan_dotdot_ft f_orphan_extents_inode f_orphan f_orphan_indirect_inode f_overfsblks f_preen f_quota f_readonly_fsck f_rebuild_csum_rootdir f_recnect_bad f_reconnect f_rehash_dir f_resize_inode f_salvage_dir f_sb_extra_isize f_selinux f_short_encrypted_dirent f_special_ea f_summary_counts f_super_bad_csum f_toobig_extent_dir f_trunc_dirent_header f_uninit_cat f_uninit_dir f_uninit_ext_past_eof2 f_uninit_ext_past_eof f_uninit_last_uninit f_unsorted_EAs f_unused_itable f_valid_ea_in_inode f_write_ea_no_extra_isize f_write_ea_toobig_extra_isize f_write_ea_toosmall_extra_isize f_yesall f_yes f_yesthenall f_yesthenno f_zeroed_ext_header f_zero_group f_zero_inode_size f_zero_super f_zero_xattr i_bad_csum j_corrupt_commit_csum j_corrupt_commit_tid j_corrupt_descr_csum j_corrupt_descr_tid j_corrupt_ext_jnl_sb_csum j_corrupt_journal_block j_corrupt_revoke_block j_corrupt_revoke_csum j_corrupt_revoke_rcount j_corrupt_sb_csum j_corrupt_sb_magic j_ext_long_revoke_trans j_ext_long_trans j_long_revoke_trans j_long_revoke_trans_mcsum_32bit j_long_revoke_trans_mcsum_64bit j_long_trans j_long_trans_mcsum_32bit j_long_trans_mcsum_64bit j_recover_csum2_32bit j_recover_csum2_64bit j_short_revoke_trans j_short_revoke_trans_mcsum_64bit j_short_trans_64bit j_short_trans j_short_trans_mcsum_64bit j_short_trans_old_csum j_short_trans_open_recover j_short_trans_recover j_short_trans_recover_mcsum_64bit j_short_uncommitted_trans j_short_uncommitted_trans_mcsum_64bit m_64bit_flexbg m_bigjournal m_dasd_bs m_desc_size_128 m_extent_journal m_hugefile m_hugefile_slack m_large_file m_meta_bg m_minrootdir m_mmp_bad_csum m_mmp_bad_magic m_mmp m_no_opt m_quota m_raid_opt m_rootdir m_root_owner m_std m_uninit r_32to64bit_expand_full r_32to64bit r_32to64bit_meta r_32to64bit_move_itable r_64to32bit r_64to32bit_meta r_expand_full r_fixup_lastbg_big r_fixup_lastbg r_inline_xattr r_min_itable r_move_itable r_resize_inode t_dangerous t_disable_mcsum t_disable_mcsum_noinitbg t_disable_mcsum_yesinitbg t_enable_mcsum_ext3 t_enable_mcsum t_enable_mcsum_initbg t_iexpand_full t_iexpand_mcsum t_replay_and_set t_uninit_bg_rm
Makefile:367: recipe for target 'test_post' failed
make[1]: *** [test_post] Error 1
make[1]: Leaving directory '/root/pkg/e2fsprogs-1.43.4/tests'
Makefile:394: recipe for target 'check-recursive' failed
make: *** [check-recursive] Error 1
dh_auto_test: make -j1 check VERBOSE=1 returned exit code 2

== Comment: #1 - NAGESWARA R. SASTRY <email address hidden> - 2017-03-10 09:45:50 ==
When instrumented only with 'Address Sanitizer' not seeing any failures

dh_testdir
dh_auto_configure ?- CFLAGS="-fsanitize=address" LDFLAGS="-fsanitize=address" CXXFLAGS="-fsanitize=address"
dh_auto_build
dh_auto_test

...
327 tests succeeded 0 tests failed
Creating test_script...
make[1]: Leaving directory '/root/pkg/e2fsprogs-1.43.4/tests'

Default Comment by Bridge

tags: added: architecture-ppc64le bugnameltc-152460 severity-medium targetmilestone-inin16043
bugproxy (bugproxy) wrote : sosreport

Default Comment by Bridge

Changed in ubuntu:
assignee: nobody → Ubuntu on IBM Power Systems Bug Triage (ubuntu-power-triage)
affects: ubuntu → e2fsprogs (Ubuntu)
Changed in ubuntu-power-systems:
assignee: nobody → Canonical Server Team (canonical-server)

Hi,
that is a very interesting report, but I looked at the source and there is no Ubuntu and not even a lot of Debian delta over upstream e2fsprogs.

I'm almost convinced that this should happen when running on latest upstream as-is as well.
That said if you could confirm that on [1] it triggers as well, then IMHO the right way would be to address that with a bug to upstream and pull in from there once fixed.

Please report back here if that triggers on upstream as well and if you happen to open a bug there please report it back here so it can be tracked and tested once a solution is available.

[1]: https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git

------- Comment From <email address hidden> 2017-07-04 11:00 EDT-------
(In reply to comment #8)
> Hi,
> that is a very interesting report, but I looked at the source and there is
> no Ubuntu and not even a lot of Debian delta over upstream e2fsprogs.
>
> I'm almost convinced that this should happen when running on latest upstream
> as-is as well.
> That said if you could confirm that on [1] it triggers as well, then IMHO
> the right way would be to address that with a bug to upstream and pull in
> from there once fixed.
>
> Please report back here if that triggers on upstream as well and if you
> happen to open a bug there please report it back here so it can be tracked
> and tested once a solution is available.
>
> [1]: https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git

Sure, will check with upstream. If occurs there will raise a bug and update this bugzilla with the findings and bug number (if there is any). Thanks!!

Theodore Ts'o (tytso) wrote :

You don't need to check with upstream e2fsprogs; I do keep an eye on Launchpad bugs.

None of UBSAN warnings are actually dangerous assuming sane[1] compilers and reasonable architecture. (e.g., a compiler that can correctly compile the Linux kernel and an architecture that run Linux should have no problems with e2fsprogs).

[1] "Any sufficiently advanced compiler is indistinguishable from a malicious adversary."

Still, it's better to fix these sorts of issues than not, so please see the attached patch. It will be included in the next release of e2fsprogs.

The attachment "0001-Fix-warnings-found-using-UBSAN.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Manoj Iyer (manjo) on 2017-07-19
Changed in e2fsprogs (Ubuntu):
importance: Undecided → Medium
Changed in ubuntu-power-systems:
importance: Undecided → Medium
Manoj Iyer (manjo) on 2017-07-24
tags: added: triage-g
Changed in ubuntu-power-systems:
assignee: Canonical Server Team (canonical-server) → Canonical Foundations Team (canonical-foundations)
Manoj Iyer (manjo) wrote :

Once this is released upstream we can consider backporting the patch to Ubuntu.

Theodore Ts'o (tytso) wrote :

The commit is upstream in git now: https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/commit/?h=maint&id=c8ca23979fa75df15208922422e81c83cf112320

I will be releasing e2fsprogs 1.43.5 soonish and I plan to do a release to Debian Unstable/Testing at the same time. So you might just want to wait for the Debian package upload, though. As a side note, I'll double check to see if Ubuntu has any distribution local patches, but if there's anything you want to make sure I look at for inclusion in 1.43.5, now would be a good time...

bugproxy (bugproxy) wrote : sosreport

Default Comment by Bridge

Matthias Klose (doko) wrote :

Theodore, looking at https://tracker.debian.org/pkg/e2fsprogs, you see that Ubuntu currently has no delta.

Steve Langasek (vorlon) wrote :

The upstream commit message notes: "None of the warnings should cause e2fsprogs malfunction given a sane compiler running on architectures that Linux can support. Still, it's better to clean up to code than not."

We would therefore not cherry-pick this change from upstream into an SRU to 16.04 by itself.

For 17.10 and above, this will be fixed by way of the upstream release, synced from Debian.

Changed in e2fsprogs (Ubuntu):
status: New → Won't Fix
Changed in ubuntu-power-systems:
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers