To be clear, the git submodules we use are only small parts of the overall source tree. TokuDB and its massive mtr suite comes along with the tree regardless of submodule, same will be for MyRocks. Just because _right_now_ we only use submodules for these external libraries that are pert of components that may be conditionally compiled in or out does not mean that submodule use will not be introduced or expanded elsewhere in the future. Specifically we have issues with overlapping imports of compression libraries. One of the possible (and perhaps likely) solutions is to import them as submodules for static compilation into a Percona third party repo which would pulled into Percona Server via a submodule. These compression libraries could/would also be utilized by other areas of the server (InnoDB column compression already had a request to try zstd) and thus would require the same process.
So yes, in order to have a truly complete working tree, git submodule init && update is required. Our jenkins templates all do it and they are our golden standard. This is why this issue was not caught by jenkins testing as one would have expected.
The use of submodules is actually intended to _improve_ the self compilation case. Rather than forcing a user to go out and grab code from several different locations and perfectly overlay them into the proper places in the source tree, we use the submodules instead. This is the way TokuDB used to be built. You had to manually copy code from 5 different external repos (IIRC they were tokutek/tokudb-engine, tokutek/percona-server, tokutek/ft-index, tokutek/enterprise-backup, and tokutek/backup-plugin) into several locations in the Percona Server source tree (6th repo) before you built. Now one single command 'git clone --recursive xxx' properly builds out everything needed to compile.
To be clear, the git submodules we use are only small parts of the overall source tree. TokuDB and its massive mtr suite comes along with the tree regardless of submodule, same will be for MyRocks. Just because _right_now_ we only use submodules for these external libraries that are pert of components that may be conditionally compiled in or out does not mean that submodule use will not be introduced or expanded elsewhere in the future. Specifically we have issues with overlapping imports of compression libraries. One of the possible (and perhaps likely) solutions is to import them as submodules for static compilation into a Percona third party repo which would pulled into Percona Server via a submodule. These compression libraries could/would also be utilized by other areas of the server (InnoDB column compression already had a request to try zstd) and thus would require the same process.
So yes, in order to have a truly complete working tree, git submodule init && update is required. Our jenkins templates all do it and they are our golden standard. This is why this issue was not caught by jenkins testing as one would have expected.
The use of submodules is actually intended to _improve_ the self compilation case. Rather than forcing a user to go out and grab code from several different locations and perfectly overlay them into the proper places in the source tree, we use the submodules instead. This is the way TokuDB used to be built. You had to manually copy code from 5 different external repos (IIRC they were tokutek/ tokudb- engine, tokutek/ percona- server, tokutek/ft-index, tokutek/ enterprise- backup, and tokutek/ backup- plugin) into several locations in the Percona Server source tree (6th repo) before you built. Now one single command 'git clone --recursive xxx' properly builds out everything needed to compile.