Render process deadlock at shutdown

Bug #1442413 reported by Chris Coulson on 2015-04-09
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Oxide
High
Chris Coulson
1.7
High
Chris Coulson

Bug Description

Running the tests occasionally leaves deadlocked render processes around. Attaching GDB shows the main thread is waiting here:

Thread 1 (Thread 0x7f8ff2588940 (LWP 18290)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f8fee70144b in base::WaitableEvent::TimedWait (this=this@entry=0x7fff1638c2c0, max_time=...) at ../../../../third_party/chromium/src/base/synchronization/waitable_event_posix.cc:214
#2 0x00007f8fee701504 in base::WaitableEvent::Wait (this=this@entry=0x7fff1638c2c0) at ../../../../third_party/chromium/src/base/synchronization/waitable_event_posix.cc:154
#3 0x00007f8fef00cab7 in Wait (this=0x7fff1638c2c0) at ../../../../third_party/chromium/src/cc/base/completion_event.h:41
#4 cc::ThreadProxy::Stop (this=0x2ad0d7361a80) at ../../../../third_party/chromium/src/cc/trees/thread_proxy.cc:621
#5 0x00007f8feefdb764 in cc::LayerTreeHost::~LayerTreeHost (this=this@entry=0x2ad0d7351400, __in_chrg=<optimised out>) at ../../../../third_party/chromium/src/cc/trees/layer_tree_host.cc:186
#6 0x00007f8feefdbc49 in cc::LayerTreeHost::~LayerTreeHost (this=0x2ad0d7351400, __in_chrg=<optimised out>) at ../../../../third_party/chromium/src/cc/trees/layer_tree_host.cc:198
#7 0x00007f8ff108e2a9 in content::RenderWidgetCompositor::~RenderWidgetCompositor (this=0x2ad0d7399140, __in_chrg=<optimised out>) at ../../../../third_party/chromium/src/content/renderer/gpu/render_widget_compositor.cc:478
#8 0x00007f8ff10fddd3 in operator() (this=0x2ad0d73a4c30, ptr=<optimised out>) at ../../../../third_party/chromium/src/base/memory/scoped_ptr.h:128
#9 reset (p=0x0, this=0x2ad0d73a4c30) at ../../../../third_party/chromium/src/base/memory/scoped_ptr.h:248
#10 reset (p=0x0, this=0x2ad0d73a4c30) at ../../../../third_party/chromium/src/base/memory/scoped_ptr.h:377
#11 content::RenderWidget::Close (this=this@entry=0x2ad0d73a4c00) at ../../../../third_party/chromium/src/content/renderer/render_widget.cc:1575
#12 0x00007f8ff10fa91a in content::RenderViewImpl::Close (this=0x2ad0d73a4c00) at ../../../../third_party/chromium/src/content/renderer/render_view_impl.cc:3064
#13 0x00007f8fee6bfaa9 in Run (this=0x7fff1638c5a8) at ../../../../third_party/chromium/src/base/callback.h:396
#14 base::debug::TaskAnnotator::RunTask (this=this@entry=0x2ad0d735588c, queue_function=queue_function@entry=0x7f8ff14c4f70 "TaskQueueManager::PostTask", run_function=run_function@entry=0x7f8ff14c4feb "TaskQueueManager::RunTask", pending_task=...) at ../../../../third_party/chromium/src/base/debug/task_annotator.cc:63
#15 0x00007f8ff100da5e in content::TaskQueueManager::ProcessTaskFromWorkQueue (this=this@entry=0x2ad0d7355870, queue_index=<optimised out>, has_previous_task=<optimised out>, previous_task=previous_task@entry=0x7fff1638c650) at ../../../../third_party/chromium/src/content/child/scheduler/task_queue_manager.cc:642
#16 0x00007f8ff100dc97 in content::TaskQueueManager::DoWork (this=0x2ad0d7355870, posted_from_main_thread=<optimised out>) at ../../../../third_party/chromium/src/content/child/scheduler/task_queue_manager.cc:599
#17 0x00007f8fee6bfaa9 in Run (this=0x7fff1638c898) at ../../../../third_party/chromium/src/base/callback.h:396
#18 base::debug::TaskAnnotator::RunTask (this=this@entry=0x2ad0d7388d50, queue_function=queue_function@entry=0x7f8ff14405a1 "MessageLoop::PostTask", run_function=run_function@entry=0x7f8ff14405c0 "MessageLoop::RunTask", pending_task=...) at ../../../../third_party/chromium/src/base/debug/task_annotator.cc:63
#19 0x00007f8fee6e15a4 in base::MessageLoop::RunTask (this=this@entry=0x2ad0d7388c00, pending_task=...) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:445
#20 0x00007f8fee6e1881 in base::MessageLoop::DeferOrRunPendingTask (this=this@entry=0x2ad0d7388c00, pending_task=...) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:454
#21 0x00007f8fee6e1d7b in base::MessageLoop::DoWork (this=0x2ad0d7388c00) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:566
#22 0x00007f8fee6e23d9 in base::MessagePumpDefault::Run (this=0x2ad0d735aa20, delegate=0x2ad0d7388c00) at ../../../../third_party/chromium/src/base/message_loop/message_pump_default.cc:32
#23 0x00007f8fee6f47f8 in base::RunLoop::Run (this=this@entry=0x7fff1638ca40) at ../../../../third_party/chromium/src/base/run_loop.cc:55
#24 0x00007f8fee6de5b5 in base::MessageLoop::Run (this=<optimised out>) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:303
#25 0x00007f8ff110de5a in content::RendererMain (parameters=...) at ../../../../third_party/chromium/src/content/renderer/renderer_main.cc:220
#26 0x00007f8ff0fb3fa0 in content::RunZygote (main_function_params=..., delegate=0x7fff1638ce70) at ../../../../third_party/chromium/src/content/app/content_main_runner.cc:307
#27 0x00007f8ff0fb433a in content::ContentMainRunnerImpl::Run (this=0x2ad0d73562d0) at ../../../../third_party/chromium/src/content/app/content_main_runner.cc:775
#28 0x00007f8ff0fb2db1 in content::ContentMain (params=...) at ../../../../third_party/chromium/src/content/app/content_main.cc:19
#29 0x00007f8fee66f6dc in oxide::OxideMain (params=...) at ../../../../shared/app/oxide_main.cc:33
#30 0x00007f8fee650567 in oxide::qt::OxideMain (argc=5, argv=0x7fff1638cff8, release_free_memory_function=<optimised out>) at ../../../../qt/core/app/oxide_qt_main.cc:40
#31 0x00007f8fed4a0ec5 in __libc_start_main (main=0x4022f0 <main(int, char const**)>, argc=5, argv=0x7fff1638cff8, init=<optimised out>, fini=<optimised out>, rtld_fini=<optimised out>, stack_end=0x7fff1638cfe8) at libc-start.c:287
#32 0x0000000000402561 in _start ()

The main thread is blocked on the compositor impl thread, which appears to be stuck in a sync call to the now non-existant GPU service:

Thread 3 (Thread 0x7f8fdc1ff700 (LWP 18305)):
#0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f8fee7021a8 in base::WaitableEvent::WaitMany (raw_waitables=raw_waitables@entry=0x7f8fdc1fe630, count=count@entry=2) at ../../../../third_party/chromium/src/base/synchronization/waitable_event_posix.cc:276
#2 0x00007f8fef31d125 in IPC::SyncMessageFilter::Send (this=0x2ad0d7355b40, message=0x2ad0d76369b0) at ../../../../third_party/chromium/src/ipc/ipc_sync_message_filter.cc:59
#3 0x00007f8ff0fc2294 in content::ChildThreadImpl::AllocateSharedMemory (buf_size=<optimised out>, sender=0x2ad0d735a720) at ../../../../third_party/chromium/src/content/child/child_thread_impl.cc:573
#4 0x00007f8ff10df300 in HostAllocateSharedMemoryBuffer (size=<optimised out>, this=<optimised out>) at ../../../../third_party/chromium/src/content/renderer/render_thread_impl.cc:1147
#5 content::RenderThreadImpl::AllocateSharedMemory (this=<optimised out>, size=<optimised out>) at ../../../../third_party/chromium/src/content/renderer/render_thread_impl.cc:1465
#6 0x00007f8feeb061a9 in content::CommandBufferProxyImpl::CreateTransferBuffer (this=0x2ad0d73906c0, size=1048576, id=0x7f8fdc1fe78c) at ../../../../third_party/chromium/src/content/common/gpu/client/command_buffer_proxy_impl.cc:309
#7 0x00007f8fef010e1b in gpu::CommandBufferHelper::AllocateRingBuffer (this=0x2ad0d73e6600) at ../../../../third_party/chromium/src/gpu/command_buffer/client/cmd_buffer_helper.cc:99
#8 0x00007f8fef0113e8 in gpu::CommandBufferHelper::WaitForAvailableEntries (this=this@entry=0x2ad0d73e6600, count=count@entry=1) at ../../../../third_party/chromium/src/gpu/command_buffer/client/cmd_buffer_helper.cc:254
#9 0x00007f8fef60ceea in gpu::CommandBufferHelper::GetSpace (this=0x2ad0d73e6600, entries=1) at ../../../../third_party/chromium/src/gpu/command_buffer/client/cmd_buffer_helper.h:127
#10 0x00007f8fef616d93 in GetCmdSpace<gpu::gles2::cmds::Finish> (this=<optimised out>) at ../../../../third_party/chromium/src/gpu/command_buffer/client/cmd_buffer_helper.h:163
#11 Finish (this=<optimised out>) at ../../../../third_party/chromium/src/gpu/command_buffer/client/gles2_cmd_helper_autogen.h:576
#12 gpu::gles2::GLES2Implementation::FinishHelper (this=0x2ad0d73cdc00) at ../../../../third_party/chromium/src/gpu/command_buffer/client/gles2_implementation.cc:906
#13 0x00007f8fef00a226 in cc::ThreadProxy::FinishGLOnImplThread (this=0x2ad0d7361a80, completion=0x7fff1638c2c0) at ../../../../third_party/chromium/src/cc/trees/thread_proxy.cc:1250
#14 0x00007f8fee6bfaa9 in Run (this=0x7f8fdc1fead8) at ../../../../third_party/chromium/src/base/callback.h:396
#15 base::debug::TaskAnnotator::RunTask (this=this@entry=0x2ad0d750ced0, queue_function=queue_function@entry=0x7f8ff14405a1 "MessageLoop::PostTask", run_function=run_function@entry=0x7f8ff14405c0 "MessageLoop::RunTask", pending_task=...) at ../../../../third_party/chromium/src/base/debug/task_annotator.cc:63
#16 0x00007f8fee6e15a4 in base::MessageLoop::RunTask (this=this@entry=0x2ad0d750cd80, pending_task=...) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:445
#17 0x00007f8fee6e1881 in base::MessageLoop::DeferOrRunPendingTask (this=this@entry=0x2ad0d750cd80, pending_task=...) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:454
#18 0x00007f8fee6e1d7b in base::MessageLoop::DoWork (this=0x2ad0d750cd80) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:566
#19 0x00007f8fee6e23d9 in base::MessagePumpDefault::Run (this=0x2ad0d73ed6a0, delegate=0x2ad0d750cd80) at ../../../../third_party/chromium/src/base/message_loop/message_pump_default.cc:32
#20 0x00007f8fee6f47f8 in base::RunLoop::Run (this=this@entry=0x7f8fdc1fec80) at ../../../../third_party/chromium/src/base/run_loop.cc:55
#21 0x00007f8fee6de5b5 in base::MessageLoop::Run (this=<optimised out>) at ../../../../third_party/chromium/src/base/message_loop/message_loop.cc:303
#22 0x00007f8fee70e505 in Run (message_loop=<optimised out>, this=0x2ad0d74dcb80) at ../../../../third_party/chromium/src/base/threading/thread.cc:185
#23 base::Thread::ThreadMain (this=0x2ad0d74dcb80) at ../../../../third_party/chromium/src/base/threading/thread.cc:239
#24 0x00007f8fee70ad1f in base::(anonymous namespace)::ThreadFunc (params=<optimised out>) at ../../../../third_party/chromium/src/base/threading/platform_thread_posix.cc:77
#25 0x00007f8fed84b0a5 in start_thread (arg=0x7f8fdc1ff700) at pthread_create.c:309
#26 0x00007f8fed578cfd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

Related branches

Changed in oxide:
importance: Undecided → High
status: New → Triaged
Chris Coulson (chrisccoulson) wrote :

So, more info:

- The compositor thread isn't doing a sync call that relies on the GPU service - it's handled on the browser side by the IO and FILE_USER_BLOCKING threads.

- The browser process has definitely gone away.

- Despite this, IPC::SyncMessageFilter::OnChannelError() hasn't been called (IPC::SyncMessageFilter::sender_ hasn't been cleared), which would unblock the waiting thread. This should be called on the IO thread from the corresponding IPC::ChannelProxy when the underlying IPC::ChannelMojo is closed.

I suspect this is caused by https://code.google.com/p/chromium/issues/detail?id=377980 (it's enabled by default in 1.7, but not in 1.6). I'm going to disable this on the 1.7 branch just to see.

Changed in oxide:
milestone: none → branch-1.8
assignee: nobody → Chris Coulson (chrisccoulson)
Chris Coulson (chrisccoulson) wrote :

I've not noticed this for a while now. I'm going to assume that this was fixed in Chromium

Changed in oxide:
status: Triaged → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.