Hi! Odd are... bug in drizzleslap. I've not looked at it in sometime. It is still using the my_getopt junk so it links to the mysys mess... so it probably has stuff dinking with its threads. Cheers, -Brian On Nov 5, 2009, at 6:33 PM, Eric Day wrote: > Something seems to be broken once we exceed the back_log count. Once > getting drizzleslap to hang, I dumped the active sockets for each > process: > > eday@orisndriz01:~/bzr/drizzle/eday-dev/tests$ lsof -p 32531 -i4 -a > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > drizzlesl 32531 eday 3u IPv4 10320113 TCP localhost:54858- > >localhost:9306 (CLOSE_WAIT) > drizzlesl 32531 eday 185u IPv4 10324113 TCP localhost:56240- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 186u IPv4 10324115 TCP localhost:56241- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 187u IPv4 10324117 TCP localhost:56242- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 188u IPv4 10324119 TCP localhost:56243- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 189u IPv4 10324121 TCP localhost:56244- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 190u IPv4 10324123 TCP localhost:56245- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 191u IPv4 10324125 TCP localhost:56246- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 192u IPv4 10324127 TCP localhost:56247- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 193u IPv4 10324129 TCP localhost:56248- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 194u IPv4 10324131 TCP localhost:56249- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 195u IPv4 10324133 TCP localhost:56250- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 196u IPv4 10324135 TCP localhost:56251- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 197u IPv4 10324137 TCP localhost:56252- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 198u IPv4 10324139 TCP localhost:56253- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 199u IPv4 10324141 TCP localhost:56254- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 200u IPv4 10324143 TCP localhost:56255- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 201u IPv4 10324145 TCP localhost:56256- > >localhost:9306 (ESTABLISHED) > drizzlesl 32531 eday 202u IPv4 10324147 TCP localhost:56257- > >localhost:9306 (ESTABLISHED) > eday@orisndriz01:~/bzr/drizzle/eday-dev/tests$ lsof -p 20886 -i4 -a > COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME > drizzled 20886 eday 10u IPv4 10302592 TCP *:9306 (LISTEN) > eday@orisndriz01:~/bzr/drizzle/eday-dev/tests$ netstat -n|grep 9306 > tcp 0 0 127.0.0.1:56243 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56255 127.0.0.1:9306 > ESTABLISHED > tcp 1 0 127.0.0.1:54858 127.0.0.1:9306 > CLOSE_WAIT > tcp 0 0 127.0.0.1:56247 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56252 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56249 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56256 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56251 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56240 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56257 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56254 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56245 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56241 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56246 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56242 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56248 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56250 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56244 127.0.0.1:9306 > ESTABLISHED > tcp 0 0 127.0.0.1:56253 127.0.0.1:9306 > ESTABLISHED > > > As you can see, drizzleslap thinks it has valid connections and are > all blocking in poll(), but the server has no knowledge of the > active connections. The OS should be causing poll() to return > POLLERR or POLLHUP inside of drizzleslap, rather than have them > block indefinitely for POLLIN. > > -- > Drizzleslap hangs with over 175 connections > https://bugs.launchpad.net/bugs/455137 > You received this bug notification because you are a member of > Drizzle- > developers, which is subscribed to Drizzle. > > Status in A Lightweight SQL Database for Cloud and Web: Confirmed > > Bug description: > Running drizzleslap with any number connections great than 175 > causes it to hang: > > Example: > drizzleslap --port=9306 --label=write --concurrency=200 -- > iterations=10 --engine=innodb --number-int > -cols=2 --number-char-cols=3 --auto-generate-sql --auto-generate-sql- > add-autoincrement --auto-genera > te-sql-load-type=write --number-of-queries=100 > > attaching to the process through gdb gives: > (gdb) bt > #0 0x00007f3c0d98756d in pthread_cond_timedwait@@GLIBC_2.3.2 () > from /lib/libpthread.so.0 > #1 0x00000000004051c2 in run_scheduler (sptr=0xa462e0, > stmts=0xa44350, concur=200, limit=0) at client/drizzleslap.cc:2137 > #2 0x0000000000408e12 in concurrency_loop (con=0x7fff16fdbe80, > current=200, eptr=0xa45970) at client/drizzleslap.cc:495 > #3 0x0000000000409101 in main (argc=0, argv=0xa44178) at client/ > drizzleslap.cc:377 > > adding -v -v -v for lots of verbose output shows that it hangs while > doing a bunch of inserts: > ........ > INSERT INTO t1 VALUES (NULL, > 1576223060,942232734 > ,'nHujWOi8H3SwwB0cewKe8d5AyZL6EZ01qmkDAILKLh89t8lXwx9xQ67OXKWltWek9EpBYAXljfvcfY0cM1Zo8YDXsbHlynL7uAIQCi4wDrOtFOTdQuQQQXpI6y3wmH > ','3gREyYw2S9ls219NeRHwlbRatQZpMJ6IrPYppKkHTT2OWBf1WpxFOHpAbMzpZx7OEX64rqDDBFdaIGbo5O3MhlYG7XyZK5xQTvWKlcZxj2X2JyiGMee4NACMZ2vl8l > ','4dgQpQ2FFmHegccMlq0FmNIrmjthL1IPWzrCHuRYgk2KDWaqKb57qNyczrAKlSl7JcJcMe8uN26SYhItGF0yXybWhdqQxRQHvltIGuCfwJzTs9M0OEyv4PHl274sQW > '); > INSERT INTO t1 VALUES (NULL, > 1322166549,823399162 > ,'7ugNokudvtyn2CnJQmEMBO76N3yqZzCZtkwaSrBGc93XDjqfLwtOK0WkwKaLJeu5MrfiaiyAjuZwdNRyKKMYClG85iMHKGMaztHPR7SA11yWPPKddj9pKQqH8CQLKg > ','vKZ4d0C52570cOHw1uXAkjSQ1TeJqy0lAPqoQ2lKzsLR8WG9PnC1NYRG1xbrvbdy1ReJLNfC8sgGFwIgbMhyu0EwxinsHGiAbKLoct1klhs0d2Y4O53J5972run13L > ','3eiPRAISuv1eoWhKaxQTGNfFIMA5F5QJj0k2A2WwxNRlt8yS5GpEfSTxjXuycD9CDtEGnd4d0XEufCEdS3j0O4p7ckx4x7G2sNAg0FH0mXmIc0nwwWoKaeJ2E873FF > '); > INSERT INTO t1 VALUES (NULL, > 116314197,805287600 > ,'e8YWNpN1C9JgA6T6SHI3vdx1lw4tC2liAH5PySQ2dejfcCd5Lv0HOxA2uFnyHYYJhdcf52ZbYGE9KkEifErw31qx8dnH1KSax4DvzmWXvcY7KmhssOowIWlQzZc0l4 > ','Qi9fT8HrX4S4CoilYbACNSi0mhPMZARzlcWEcg6zkQ3o6eafDcrS4QSq7u5Z4YYh0vwSbuRBKN0QtQ562pQyFmgEYlnT9CA176Ta0plC4dfxvkvpZDngzKvY5it7m3 > ','8t2WJuJx7FamCMMzBoCrSBBFb8ySFkYGWQmhKxeRC55ORSNWGbLceYsEhiaoIy4gOiMlfs4rxA7GeMnmp0ESZ6a6g8mz7jXNPt85mDwbN4ktIXf8XlcwkAuOAZF9Gn > '); > INSERT INTO t1 VALUES (NULL, > 1407841593,32683953 > ,'YZeAmRm1NWuhtPpG3P5nRrBWOJXYy46WWkhG3JAQhw8QEpaH7f4QN7wfISCGOInuuK3qX5G41GM76wG5SCXrKKMWgan4kQrXmmL9rTDkmbskN8prLCaZYx3CNiH880 > ','XumsvTL8Dzbxbz6GiJTHk16n6TMFdWF8HtsCM6DRxoPZfNhEiAXQ4T5AyRPSyYS7PLJfJO6G5XpkuXycZLuTqsfhBw91Sd8lqSO9sXQx27apWOHweCPKw517t29EgH > ','zMdGYMnyJhxlWS2gOgsp0gmunYwwmSEdrrtFDg5PE22cM4Gek0Kdg6XSWJGGoWuXEneJKjcQlf20jBeKBypkxK4RXKkBrWax9CGTvlvZOqZz13BC20orCsIcDunwGM > '); > INSERT INTO t1 VALUES (NULL, > 1471079933,266531426 > ,'a1vMmjn2ZKuaFxe9yR02BJeO5IKMg5o67bKlI7gjJQHbNokD7lGIwmjt4vFc1vb0Xvdh3HPEbipzMQCT3Koz7Ns3I0gJv7JQ2x8x6P4i7tZMBlrEyXDxuysDqOOlXc > ','4QZ4NXTJ5SDeqOGaXE5k30av5OKT2r56HW2ZuXszrxeiEm8lsDLv5oiA42g6md4v86K2T5tLuNv893l18zoEfNOjPwq3amratBSE8Db3dM3eHhYQgKgJjwu1lkwJXn > ','lGr5WzIy2vC6aWn6oJIKWuGOnt1jCOXpYMu2LDsG85EaRtYFCrbadjyOdr7hFWXfkHiyWQEXNKxqDW5HDYrq8h6dP5AYcZCKrmADCGkccaQp4O69opzoN5PoASCe1O > '); > > For now modifying all tests at http://hudson.drizzle.org/view/Drizzle-staging/job/Drizzleslap-staging/ > to only use a max of 175 connections