Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2 or Xen) with default configuration

Bug #1313450 reported by Erik Andersson
140
This bug affects 31 people
Affects Status Importance Assigned to Milestone
linux (Ubuntu)
Fix Released
High
Unassigned
Trusty
Fix Released
High
Unassigned
vsftpd (Ubuntu)
Invalid
Undecided
Unassigned
Trusty
Invalid
Undecided
Unassigned

Bug Description

The configuration submitted was modified during testing, but this also failed with the default configuration. The only solution I have found so far was to delete the configuration, this would allow the service to start, but obviously not work as intended.

If I try to start the service manually using vsftpd or sudo vsftpd I receive the following error message:
500 OOPS: munmap

I also tried the default commands
/etc/init.d/vsftpd or service vsftpd start

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: vsftpd 3.0.2-1ubuntu2
ProcVersionSignature: User Name 3.13.0-24.46-generic 3.13.9
Uname: Linux 3.13.0-24-generic x86_64
ApportVersion: 2.14.1-0ubuntu3
Architecture: amd64
Date: Sun Apr 27 21:23:18 2014
Ec2AMI: ami-896c96fe
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1b
Ec2InstanceType: t1.micro
Ec2Kernel: aki-52a34525
Ec2Ramdisk: unavailable
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: vsftpd
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.vsftpd.conf: [modified]
mtime.conffile..etc.vsftpd.conf: 2014-04-27T21:16:28.222467
---
ApportVersion: 2.14.1-0ubuntu3
Architecture: amd64
DistroRelease: Ubuntu 14.04
Ec2AMI: ami-896c96fe
Ec2AMIManifest: (unknown)
Ec2AvailabilityZone: eu-west-1b
Ec2InstanceType: t1.micro
Ec2Kernel: aki-52a34525
Ec2Ramdisk: unavailable
Package: vsftpd 3.0.2-1ubuntu2
PackageArchitecture: amd64
ProcEnviron:
 TERM=xterm
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: User Name 3.13.0-24.46-generic 3.13.9
Tags: trusty ec2-images
Uname: Linux 3.13.0-24-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm audio cdrom dialout dip floppy netdev plugdev sudo video
_MarkForUpload: True
modified.conffile..etc.vsftpd.conf: [modified]
mtime.conffile..etc.vsftpd.conf: 2014-04-29T19:03:28.834467

Revision history for this message
Erik Andersson (eandersson1) wrote :
Revision history for this message
Robie Basak (racb) wrote :

Thank you for taking the time to report this bug and helping to make Ubuntu better.

Is this the same as bug 1219857? Does adding seccomp_sandbox=NO to /etc/vsftpd.conf work around the issue for you?

Changed in vsftpd (Ubuntu):
status: New → Incomplete
Revision history for this message
Erik Andersson (eandersson1) wrote :

That made no difference.

My configuration:
seccomp_sandbox=NO
Listen=YES
#Listen_IPV6
Anonymous=YES
#Local_enabled=YES
#Write_enabled=YES

This is taken from the kernel log (also visible in the syslogs).

Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160354] BUG: Bad page map in process vsftpd pte:8000000000000965 pmd:23042067
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160372] page:ffffea0000000000 count:-259 mapcount:-259 mapping: (null) index:0x0
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160379] page flags: 0x10(dirty)
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160387] addr:00007fd251b67000 vm_flags:08100071 anon_vma:ffff88002381d700 mapping: (null) index:7fd251b67
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160397] CPU: 0 PID: 14426 Comm: vsftpd Tainted: G B 3.13.0-24-generic #46-Ubuntu
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160399] ffff880023ca6c00 ffff880016565c70 ffffffff81715a64 00007fd251b67000
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160404] ffff880016565cb8 ffffffff81174183 8000000ffa106965 00000007fd251b67
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160408] ffff880023042b38 ffffea0000000000 00007fd251b67000 00007fd251b68000
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160411] Call Trace:
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160421] [<ffffffff81715a64>] dump_stack+0x45/0x56
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160427] [<ffffffff81174183>] print_bad_pte+0x1a3/0x250
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160430] [<ffffffff81175b67>] unmap_page_range+0x717/0x7f0
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160433] [<ffffffff81175cc1>] unmap_single_vma+0x81/0xf0
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160436] [<ffffffff81176d39>] unmap_vmas+0x49/0x90
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160441] [<ffffffff8117feec>] exit_mmap+0x9c/0x170
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160445] [<ffffffff8106482c>] mmput+0x5c/0x120
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160449] [<ffffffff81069bbc>] do_exit+0x26c/0xa50
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160453] [<ffffffff8109dd84>] ? vtime_account_user+0x54/0x60
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160459] [<ffffffff8114d52f>] ? context_tracking_user_exit+0x4f/0xc0
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160462] [<ffffffff8106a41f>] do_group_exit+0x3f/0xa0
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160465] [<ffffffff8106a494>] SyS_exit_group+0x14/0x20
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160469] [<ffffffff8172663f>] tracesys+0xe1/0xe6
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160552] BUG: Bad rss-counter state mm:ffff8800030af500 idx:0 val:-1
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.160559] BUG: Bad rss-counter state mm:ffff8800030af500 idx:1 val:1
Apr 29 19:03:44 ip-10-34-255-23 kernel: [21095107.195341] BUG: Bad page map in process vsftpd pte:8000000000000965 pmd:23d36067

Revision history for this message
Erik Andersson (eandersson1) wrote : Dependencies.txt

apport information

tags: added: apport-collected
description: updated
Revision history for this message
Erik Andersson (eandersson1) wrote : Re: Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2) with default configuration
Revision history for this message
Erik Andersson (eandersson1) wrote :
Revision history for this message
AC (acrunner8) wrote :

I'm also having the exact same issue running vsftpd on Ubuntu 14.04 in an Amazon/EC2 instance. Adding seccomp_sandbox=NO does not fix the problem for me either.

Upgraded to vsftpd 3.0.2-1ubuntu2.14.04.1 that was just released on 2014-05-05 but this does not fix the issue.

here the message in dmesg after to 500 OOPS: munmap:

[25101608.375027] BUG: Bad page map in process vsftpd pte:8000000000000965 pmd:4fda0067
[25101608.375047] page:ffffea0000000000 count:-24 mapcount:-24 mapping: (null) index:0x0
[25101608.375055] page flags: 0x14(referenced|dirty)
[25101608.375065] addr:00007f868625c000 vm_flags:08100071 anon_vma:ffff880028e087c0 mapping: (null) index:7f868625c
[25101608.375076] CPU: 0 PID: 4509 Comm: vsftpd Tainted: GF B 3.13.0-24-generic #46-Ubuntu
[25101608.375079] ffff880003674540 ffff88002a1e3c70 ffffffff81715a64 00007f868625c000
[25101608.375083] ffff88002a1e3cb8 ffffffff81174183 800000079e60d965 00000007f868625c
[25101608.375086] ffff88004fda02e0 ffffea0000000000 00007f868625c000 00007f868625d000
[25101608.375089] Call Trace:
[25101608.375100] [<ffffffff81715a64>] dump_stack+0x45/0x56
[25101608.375107] [<ffffffff81174183>] print_bad_pte+0x1a3/0x250
[25101608.375111] [<ffffffff81175b67>] unmap_page_range+0x717/0x7f0
[25101608.375114] [<ffffffff81175cc1>] unmap_single_vma+0x81/0xf0
[25101608.375117] [<ffffffff81176d39>] unmap_vmas+0x49/0x90
[25101608.375123] [<ffffffff8117feec>] exit_mmap+0x9c/0x170
[25101608.375128] [<ffffffff8106482c>] mmput+0x5c/0x120
[25101608.375133] [<ffffffff81069bbc>] do_exit+0x26c/0xa50
[25101608.375138] [<ffffffff8109dd84>] ? vtime_account_user+0x54/0x60
[25101608.375144] [<ffffffff8114d52f>] ? context_tracking_user_exit+0x4f/0xc0
[25101608.375148] [<ffffffff8106a41f>] do_group_exit+0x3f/0xa0
[25101608.375152] [<ffffffff8106a494>] SyS_exit_group+0x14/0x20
[25101608.375158] [<ffffffff8172663f>] tracesys+0xe1/0xe6
[25101608.375264] BUG: Bad rss-counter state mm:ffff88000352d180 idx:0 val:-1
[25101608.375274] BUG: Bad rss-counter state mm:ffff88000352d180 idx:1 val:1

Revision history for this message
AC (acrunner8) wrote :

After doing a little more investigation, it looks like there are other distributions that are having problems with vsftpd running on AWS EC2 instances.

It sounds like it may be related to a kernel bug that affect Xen paravirtualized hosts.

http://www.gossamer-threads.com/lists/linux/kernel/1854156?search_string=%5BBISECTED%5D%20Linux%203.12.7%20introduces%20page%20map%20handling%20regression;#1854156

AC (acrunner8)
tags: added: paravirtualization xen
Revision history for this message
triden (triden) wrote :

It is related to a kernel bug.

Don't know if you have the ability to upgrade kernels on EC2, but this bug can be worked around by installing one of the latest Ubuntu Mainline kernels.

You need at least: linux-image-3.15.0-031500rc2

Reference: http://www.deinon.com/wiki/doku.php?id=linux:ubuntu

Changed in vsftpd (Ubuntu):
status: Incomplete → Confirmed
triden (triden)
summary: - Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2) with default
+ Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2 or Xen) with default
configuration
Stefan Bader (smb)
Changed in linux (Ubuntu):
status: New → Confirmed
importance: Undecided → High
Changed in vsftpd (Ubuntu):
status: Confirmed → Invalid
Revision history for this message
Stefan Bader (smb) wrote :

As far as I can see from the upstream discussion this is indeed a kernel issue. Initially it was intended to be fixed by

Author: Mel Gorman <email address hidden>
    xen: properly account for _PAGE_NUMA during xen pte translations
...
Steven Noonan forwarded a users report where they had a problem starting vsftpd on a Xen paravirtualized guest, with this in dmesg: BUG: Bad page map in process vsftpd pte:8000000493b88165 pmd:e9cc01067
...

However the patch got reverted later (which also made it back into 14.04 via stable because it caused regressions on migration. With v3.15-rc2 there was a new change to address this:

commit 29c7787075c92ca8af353acd5301481e6f37082f
Author: Mel Gorman <email address hidden>
    mm: use paravirt friendly ops for NUMA hinting ptes

It looks that change (while tagged for stable) has not yet been picked up for 3.13.

Stefan Bader (smb)
Changed in linux (Ubuntu Trusty):
importance: Undecided → High
status: New → Confirmed
Changed in vsftpd (Ubuntu Trusty):
status: New → Invalid
Revision history for this message
Ryan Finnie (fo0bar) wrote :

I can confirm that 29c7787075c92ca8af353acd5301481e6f37082f applies cleanly again the latest trusty tree (3.13.0-27.50), and does indeed fix the vsftpd problem without apparent side effects.

Revision history for this message
Joel (joel-8) wrote :

Not here, still seeing the same problem, and not just with vsftpd:

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trusty

Linux ip-10-0-0-46 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:06:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615216] BUG: Bad page map in process vsftpd pte:8000000000000965 pmd:031b1067
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615228] page:ffffea0000000000 count:-34 mapcount:-34 mapping: (null) index:0x0
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615232] page flags: 0x14(referenced|dirty)
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615237] addr:00007fd10f153000 vm_flags:08100071 anon_vma:ffff88000314d8c0 mapping: (null) index:7fd10f153
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615242] CPU: 0 PID: 1927 Comm: vsftpd Tainted: G B 3.13.0-27-generic #50-Ubuntu
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615244] ffff8800e86b8540 ffff880003387c70 ffffffff817199c4 00007fd10f153000
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615247] ffff880003387cb8 ffffffff81174503 8000001c689fe965 00000007fd10f153
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615248] ffff8800031b1a98 ffffea0000000000 00007fd10f153000 00007fd10f154000
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615250] Call Trace:
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615256] [<ffffffff817199c4>] dump_stack+0x45/0x56
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615260] [<ffffffff81174503>] print_bad_pte+0x1a3/0x250
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615262] [<ffffffff81175ee7>] unmap_page_range+0x717/0x7f0
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615264] [<ffffffff81176041>] unmap_single_vma+0x81/0xf0
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615266] [<ffffffff811770b9>] unmap_vmas+0x49/0x90
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615268] [<ffffffff8118026c>] exit_mmap+0x9c/0x170
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615270] [<ffffffff8106482c>] mmput+0x5c/0x120
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615273] [<ffffffff81069bbc>] do_exit+0x26c/0xa50
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615275] [<ffffffff8109dd84>] ? vtime_account_user+0x54/0x60
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615279] [<ffffffff8114d51f>] ? context_tracking_user_exit+0x4f/0xc0
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615280] [<ffffffff8106a41f>] do_group_exit+0x3f/0xa0
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615282] [<ffffffff8106a494>] SyS_exit_group+0x14/0x20
Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615285] [<ffffffff8172a5bf>] tracesys+0xe1/0xe6

Revision history for this message
Joel (joel-8) wrote :

This works:

Linux ip-10-0-0-46 3.15.0-031500rc2-generic #201404201435 SMP Sun Apr 20 18:36:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

From:

http://www.deinon.com/wiki/doku.php?id=linux:ubuntu

How stable is this and can it be trusted for production use?

Revision history for this message
Oleg "Nightwing" Lomakin (nightwing666) wrote : Re: [Bug 1313450] Re: Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2 or Xen) with default configuration
Download full text (3.2 KiB)

Which software also has this problem?
04 июня 2014 г. 11:51 пользователь "Joel" <email address hidden> написал:

> Not here, still seeing the same problem, and not just with vsftpd:
>
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 14.04 LTS
> Release: 14.04
> Codename: trusty
>
> Linux ip-10-0-0-46 3.13.0-27-generic #50-Ubuntu SMP Thu May 15 18:06:16
> UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>
>
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615216] BUG: Bad page map in
> process vsftpd pte:8000000000000965 pmd:031b1067
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615228] page:ffffea0000000000
> count:-34 mapcount:-34 mapping: (null) index:0x0
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615232] page flags:
> 0x14(referenced|dirty)
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615237] addr:00007fd10f153000
> vm_flags:08100071 anon_vma:ffff88000314d8c0 mapping: (null)
> index:7fd10f153
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615242] CPU: 0 PID: 1927 Comm:
> vsftpd Tainted: G B 3.13.0-27-generic #50-Ubuntu
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615244] ffff8800e86b8540
> ffff880003387c70 ffffffff817199c4 00007fd10f153000
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615247] ffff880003387cb8
> ffffffff81174503 8000001c689fe965 00000007fd10f153
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615248] ffff8800031b1a98
> ffffea0000000000 00007fd10f153000 00007fd10f154000
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615250] Call Trace:
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615256] [<ffffffff817199c4>]
> dump_stack+0x45/0x56
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615260] [<ffffffff81174503>]
> print_bad_pte+0x1a3/0x250
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615262] [<ffffffff81175ee7>]
> unmap_page_range+0x717/0x7f0
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615264] [<ffffffff81176041>]
> unmap_single_vma+0x81/0xf0
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615266] [<ffffffff811770b9>]
> unmap_vmas+0x49/0x90
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615268] [<ffffffff8118026c>]
> exit_mmap+0x9c/0x170
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615270] [<ffffffff8106482c>]
> mmput+0x5c/0x120
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615273] [<ffffffff81069bbc>]
> do_exit+0x26c/0xa50
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615275] [<ffffffff8109dd84>]
> ? vtime_account_user+0x54/0x60
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615279] [<ffffffff8114d51f>]
> ? context_tracking_user_exit+0x4f/0xc0
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615280] [<ffffffff8106a41f>]
> do_group_exit+0x3f/0xa0
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615282] [<ffffffff8106a494>]
> SyS_exit_group+0x14/0x20
> Jun 4 07:35:05 ip-10-0-0-46 kernel: [ 217.615285] [<ffffffff8172a5bf>]
> tracesys+0xe1/0xe6
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1313450
>
> Title:
> Unable to start vsftpd on Ubuntu 14.04 (Amazon/EC2 or Xen) with
> default configuration
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/...

Read more...

Revision history for this message
triden (triden) wrote :

The problem can occur with any application as stated by Boris Ostrovsky

------------
"This can be easily triggered by Linux' page-types
(tools/vm/page-types.c) after save/restore.

All it does is it walks the page tables (in fs/proc/task_mmu.c) and
eventually trips on bad page. For example:

# /page-types -p 2273 -L> /tmp/new
[ 2634.501440] pfn 0x159f75 highest_memmap_pfn=0x3ffff
[ 2634.502345] BUG: Bad page map in process page-types pte:159f75420
pmd:3178a067
"
Which means that any application should be able to trigger this depending
on whether 'mprotect' was doing its stuff and migration happend.

Please note that these guests have no NUMA information exposed at all.
------[Ref: http://www.gossamer-threads.com/lists/xen/devel/322227?page=last ]------

Revision history for this message
triden (triden) wrote :

I've been using 3.15.0-031500rc2-generic #201404201435 SMP Sun Apr 20 18:36:18 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux from the Ubuntu Mainline kernel PPA on all my Xen guests for weeks without issue.

Looks like the patch is scheduled for the 3.13.11.3 kernel update.

Ref: https://lists.ubuntu.com/archives/kernel-team/2014-May/042946.html

I'll be moving my production boxes to that version once released. It is not clear (from that thread) when that version will be released.

Revision history for this message
Oleg "Nightwing" Lomakin (nightwing666) wrote :

3.13.0-29 kernel doesn't fix this bug.

*WBR, Lomakin Oleg*

Revision history for this message
triden (triden) wrote :

A simple test case was created by Steven Noonan to trigger the issue:

Copy the attached code into a file named bugtest.c
Compile with "gcc -o bugtest bugtest.c"
Run ./bugtest (does not have to be run as root)
If you see no output, bug does not exist; if it does you will immediately see something similar to:

triden@tweety:~$ ./bugtest
[ 837.276658] BUG: Bad page map in process bugtest pte:8000000000000965 pmd:3c076067
[ 837.276666] page:ffffea0000000000 count:-31 mapcount:-31 mapping: (null) index:0x0
[ 837.276669] page flags: 0x10(dirty)
[ 837.276672] page dumped because: bad pte
[ 837.276674] addr:00007f0639a5e000 vm_flags:08100071 anon_vma:ffff88003b1fdbc0 mapping: (null) index:7f0639a5e
[ 837.276821] BUG: Bad rss-counter state mm:ffff88003d2e8700 idx:0 val:-1
[ 837.276825] BUG: Bad rss-counter state mm:ffff88003d2e8700 idx:1 val:1

Reference: https://lkml.org/lkml/2014/1/21/544

Revision history for this message
Brad Figg (brad-figg) wrote :

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-trusty' to 'verification-done-trusty'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

tags: added: verification-needed-trusty
Revision history for this message
triden (triden) wrote :

Confirmed kernel from -proposed fixes the issue. Test case program and vsftpd both run without triggering the kernel messages.

[ 0.000000] Linux version 3.13.0-30-generic (buildd@allspice) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #54-Ubuntu SMP Mon Jun 9 22:45:01 UTC 2014 (Ubuntu 3.13.0-30.54-generic 3.13.11.2)
[ 0.000000] Booting paravirtualized kernel on Xen
[ 0.000000] Xen version: 4.4.0 (preserve-AD)

tags: added: verification-done-trusty
removed: verification-needed-trusty
Revision history for this message
adam@adamhooper.com (adam-adamhooper) wrote :

I'm confused.

As I understand it, a simple solution exists upstream, it's been understood for over a month, and it's easy to apply. I also understand that this bug affects everybody who uses Ubuntu 14.04 on AWS, and it prevents anybody from using Ubuntu 14.04 in a production environment on AWS. A sketchy workaround -- replacing the kernel -- might suffice for test or development environments. (I'm rehashing what I've learned from https://groups.google.com/forum/#!topic/logstash-users/sFNrqRbI6EM and https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1308796 and the LKML....)

So why hasn't Ubuntu updated its AWS cloud images? Why didn't Ubuntu fix its kernel before re-releasing its AWS cloud images on 2014-06-07, at http://cloud-images.ubuntu.com/releases/14.04/release/? Was anybody aware, when releasing those images, that they are unreliable? If so, was there a release note that I missed?

I don't understand the logistical issues that are keeping this bug unresolved. What would it take to escalate this bug to Critical?

Either I'm affected by this kernel bug far more than everybody else is, I have unrealistic expectations about Ubuntu 14.04 LTS, or Ubuntu is being too opaque.

What's happening?

Revision history for this message
zespri (andrewsav) wrote :

It looks like it was not fixed in ubuntu on time to make it to 2014-06-07 release. It also looks like now it made its way to trusty-proposed so things are definitely look moving.

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (19.3 KiB)

This bug was fixed in the package linux - 3.13.0-30.54

---------------
linux (3.13.0-30.54) trusty; urgency=low

  [ Adam Conrad ]

  * [Config] Enable building the sata-modules udeb on ppc64el.
    - LP: #1323980

  [ Anton Blanchard ]

  * SAUCE: (no-up) powerpc: 64bit sendfile is capped at 2GB
    - LP: #1328230

  [ Ben Widawsky ]

  * SAUCE: i915_bdw: drm/i915: Fix PSR programming
    - LP: #1321729
  * SAUCE: i915_bdw: drm/i915: Correct PPGTT total size
    - LP: #1321729

  [ Chris Wilson ]

  * SAUCE: i915_bdw: drm/i915: Broadwell expands ACTHD to 64bit
    - LP: #1321729

  [ Damien Lespiau ]

  * SAUCE: i915_bdw: drm/i915/bdw: Implement
    Wa4x4STCOptimizationDisable:bdw
    - LP: #1321729

  [ Dave Chiluk ]

  * [Config] Enable CONFIG_IP_VS_IPV6=y
    - LP: #1300739

  [ Kamal Mostafa ]

  * [Config] add debian/gbp.conf
  * Release Tracking Bug
    - LP: #1328286

  [ Mika Kuoppala ]

  * SAUCE: i915_bdw: drm/i915: add render state initialization
    - LP: #1321729

  [ Paulo Zanoni ]

  * SAUCE: i915_bdw: drm/i915: fix assert_cursor on BDW
    - LP: #1321729

  [ Steven Rostedt ]

  * SAUCE: i915_bdw: drm/i915: Do not dereference pointers from ring buffer
    in evict event
    - LP: #1321729

  [ Tim Gardner ]

  * [Config] CONFIG_POWERNV_CPUFREQ=y for ppc64el
    - LP: #1324571
  * [Debian] Treat vdso install as an environment variable
  * [Config] Treat vdso install as an environment variable
  * [Config] CONFIG_MLX4_DEBUG=y
    - LP: #1328256
  * [Config] CONFIG_I40EVF=m, CONFIG_I40E_DCB=y, CONFIG_I40E_VXLAN=y
    - LP: #1328037

  [ Timo Aaltonen ]

  * SAUCE: i915_bdw: Rebase to drm-intel-next-2014-03-07 + fixes
    - LP: #1321729
  * SAUCE: i915_bdw: Add BDW specific power well calls
    - LP: #1317865

  [ Upstream Kernel Changes ]

  * drm: expose subpixel order name routine v3
  * drm: dp helper: Add DP test sink CRC definition.
  * drm: export cmdline and preferred mode functions from fb helper
  * hugetlb: ensure hugepage access is denied if hugepages are not
    supported
    - LP: #1328251
  * powerpc/powernv: Move SG list structure to header file
    - LP: #1326015
  * powerpc/powernv: Read OPAL error log and export it through sysfs
    - LP: #1326015
  * powerpc/powernv Platform dump interface
    - LP: #1326015
  * pci_regs.h: Add PCI bus link speed and width defines
    - LP: #1328037
  * net_tstamp: Add SIOCGHWTSTAMP ioctl to match SIOCSHWTSTAMP
    - LP: #1328037
  * PCI/MSI: Add pci_enable_msi_range() and pci_enable_msix_range()
    - LP: #1328037
  * net: Change skb_get_rxhash to skb_get_hash
    - LP: #1328037
  * net: Add utility functions to clear rxhash
    - LP: #1328037
  * net: Add function to set the rxhash
    - LP: #1328037
  * i40e: set pf_id based on device and function numbers
    - LP: #1328037
  * i40e: register file updates
    - LP: #1328037
  * i40e: clear AQ head and tail registers
    - LP: #1328037
  * i40e: simplify aq head-tail-len setups
    - LP: #1328037
  * i40e: firmware version fields offsets update
    - LP: #1328037
  * i40e: allow one more vector for VFs
    - LP: #1328037
  * i40e: select reset counters correctly
    - LP: #1328037
  * i40e: retry call on timeout
    - LP: #...

Changed in linux (Ubuntu Trusty):
status: Confirmed → Fix Released
Revision history for this message
Stefan Bader (smb) wrote :

The changes are in 3.15, so Utopic/development branch should be fixed already.

Changed in linux (Ubuntu):
status: Confirmed → Fix Released
Revision history for this message
Robie Basak (racb) wrote :

Looks like this was fixed in the 20140724 image (now released).

Revision history for this message
Ciro Vargas (cirobsb-br) wrote :

Hi, i have the same problem on the 14.10 ubuntu server:

http://serverfault.com/questions/675936/vsftpd-dont-start-listen-on-ubuntu-14-10

Revision history for this message
Stefan Bader (smb) wrote :

Please open a new bug report using "ubuntu-bug linux" if the problem involves the same stacktrace in dmesg. If not it is even more likely a different problem and you should start with "ubuntu-bug vsftp" (in any case the command should be run on the affected server). Also note, that this bug here was restricted to server installs running as paravirtualized guests on Xen.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.