V3 mysqld hangs on select from OQGRAPH engine table after deleting records from underlying table provided at least one select made and on record exists before the delete

[Andrew McDonnell]

#--reproduce bug where select * from graph after delete from graph_base hangs the server
--disable_warnings
DROP TABLE IF EXISTS graph_base;
DROP TABLE IF EXISTS graph;
--enable_warnings

# Create the backing store
CREATE TABLE graph_base (
    from_id INT UNSIGNED NOT NULL,
    to_id INT UNSIGNED NOT NULL,
    PRIMARY KEY (from_id,to_id),
    INDEX (to_id)
  ) ENGINE=MyISAM;

CREATE TABLE graph (
    latch VARCHAR(32) NULL,
    origid BIGINT UNSIGNED NULL,
    destid BIGINT UNSIGNED NULL,
    weight DOUBLE NULL,
    seq BIGINT UNSIGNED NULL,
    linkid BIGINT UNSIGNED NULL,
    KEY (latch, origid, destid) USING HASH,
    KEY (latch, destid, origid) USING HASH
  ) ENGINE=OQGRAPH DATA_TABLE='graph_base' ORIGID='from_id', DESTID='to_id';

INSERT INTO graph_base(from_id, to_id) VALUES (1,2), (2,1);

--echo One select of any clauses at all on graph here will cause a hang on the select after the DELETE FROM
#-- even this if it is the only one - but it doesnt hang here ... SELECT * FROM graph;
SELECT * FROM graph WHERE destid=2 and origid=1;

DELETE FROM graph_base;
#-- hangs on the next line--------------------------------------------------------------
SELECT * from graph;
FLUSH TABLES;
TRUNCATE TABLE graph_base;
DROP TABLE graph_base;
--error S42S02
SELECT * FROM graph WHERE latch='dijkstras' AND origid=1 AND destid=6;
DROP TABLE graph;

[Andrew McDonnell (andymc73)]

Accoring to HTOP thread 2 is the culprit, the other threads are 0% CPU.
(The others are block on pthread/sigwait anyway)

It seems to be stuck somewhere in our code, so I guess there is a bug where
something has been retained in memory or we are otherwise not properly
detecting that the backing store has no data.

(gdb) inf thr
  Id Target Id Frame
  4 Thread 0x7f6a03815700 (LWP 22491) "mysqld"
pthread_cond_timedwait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:211
  3 Thread 0x7f69f6a1d700 (LWP 22494) "mysqld" do_sigwait
(set=0x7f69f6a1cdb0, sig=0x7f69f6a1ce68) at
../nptl/sysdeps/unix/sysv/linux/../../../../../sysdeps/unix/sysv/linux/sigwait.c:65
  2 Thread 0x7f69f69d4700 (LWP 22502) "mysqld" _db_return_ (_line_=1972,
_stack_frame_=0x7f69f69d21f0) at
/home/andrew/develop/maria/repo/oqgraph-varchar/dbug/dbug.c:1200
* 1 Thread 0x7f6a04844720 (LWP 22489) "mysqld" 0x00007f6a038da9d3 in
*__GI___poll (fds=<optimized out>, nfds=<optimized out>, timeout=-1) at
../sysdeps/unix/sysv/linux/poll.c:87

(gdb) thr 2
[Switching to thread 2 (Thread 0x7f69f69d4700 (LWP 22502))]
#0 _db_return_ (_line_=1972, _stack_frame_=0x7f69f69d21f0) at
/home/andrew/develop/maria/repo/oqgraph-varchar/dbug/dbug.c:1200
1200 cs->framep= cs->framep->prev;
(gdb) bt
#0 _db_return_ (_line_=1972, _stack_frame_=0x7f69f69d21f0) at
/home/andrew/develop/maria/repo/oqgraph-varchar/dbug/dbug.c:1200
#1 0x00007f69f6cfbfb1 in handler::ha_rnd_init (this=0x2279388, scan=true) at
/home/andrew/develop/maria/repo/oqgraph-varchar/sql/handler.h:1972
#2 0x00007f69f6cfb87b in oqgraph3::cursor::seek_to (this=0x2326ea0,
origid=..., destid=...) at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/oqgraph_thunk.cc:512
#3 0x00007f69f6cf8a04 in oqgraph3::edge_iterator::seek (this=0x7f69f69d2470)
at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/oqgraph_shim.cc:35
#4 0x00007f69f6cf8b1a in oqgraph3::edge_iterator::operator*
(this=0x7f69f69d2470) at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/oqgraph_shim.cc:52
#5 0x00007f69f6ce10a3 in open_query::edges_cursor::fetch_row (this=0x2072a90,
row_info=..., result=...) at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/graphcore.cc:1074
#6 0x00007f69f6ce02fb in open_query::oqgraph::fetch_row (this=0x208ce90,
result=...) at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/graphcore.cc:937
#7 0x00007f69f6cdbe9e in ha_oqgraph::rnd_next (this=0x2275628, buf=0x2276898
 <incomplete sequence \361>) at
/home/andrew/develop/maria/repo/oqgraph-varchar/storage/oqgraph/ha_oqgraph.cc:991
#8 0x00000000007e53a1 in handler::ha_rnd_next (this=0x2275628, buf=0x2276898
 <incomplete sequence \361>) at
/home/andrew/develop/maria/repo/oqgraph-varchar/sql/handler.cc:2411
#9 0x000000000091d706 in rr_sequential (info=0x227d2d8) at
/home/andrew/develop/maria/repo/oqgraph-varchar/sql/records.cc:458
#10 0x000000000066e829 in sub_select (join=0x23806e8, join_tab=0x227d228,
end_of_records=false) at
/home/andrew/develop/maria/repo/oqgraph-varchar/sql/sql_select.cc:16278
#11 0x000000000066e011 in do_select (join=0x23806e8, fields=0x232f280,
...

[Andrew McDonnell (andymc73)]

Looks like it happens because we have our view of the backing table, ha_oqgraph::edges and we use edges->file->stats_records. But this doesnt get updated after delete * , and we dont have close()/open() being called in the interim.

We do call edges->prepare_for_position() on enrty to ha_oqgraph::rnd_init() but this is obviously not enough

[Andrew McDonnell (andymc73)]

Need to call table->file->info() to ensure count is update when starting random scan. Fixed...