3 controller ( >1 number of computes. )
We notice the following error ( which is not 100% repeatable ) often on our deployment
queens with containers.
"+ echo 'Running command: '\\''/usr/bin/gnocchi-upgrade --sacks-number=128'\\'''",
"+ exec /usr/bin/gnocchi-upgrade --sacks-number=128",
"2018-03-13 09:24:34,859 [7] WARNING oslo_config.cfg: Option \"coordination_url\" from group \"storage\" is deprecated. Use option \"coordination_url\" from group \"DEFAULT\".",
"2018-03-13 09:24:35,164 [7] INFO gnocchi.cli.manage: Upgrading indexer SQLAlchemyIndexer: mysql+pymysql://gnocchi:YqvWJ7bex7KZJG9WzCe2kHHFp@172.17.1.16/gnocchi?read_default_
tc/my.cnf.d/tripleo.cnf",
..
....
"2018-03-13 09:24:38,914 [7] CRITICAL root: Traceback (most recent call last):",
" File \"/usr/bin/gnocchi-upgrade\", line 10, in <module>",
" sys.exit(upgrade())",
" File \"/usr/lib/python2.7/site-packages/gnocchi/cli/manage.py\", line 81, in upgrade",
" index.create_archive_policy(ap)",
" File \"/usr/lib/python2.7/site-packages/gnocchi/indexer/sqlalchemy.py\", line 616, in create_archive_policy",
" raise indexer.ArchivePolicyAlreadyExists(archive_policy.name)",
"ArchivePolicyAlreadyExists: Archive policy high already exists",
We also saw this error on different deployment-
"+ exec /usr/bin/gnocchi-upgrade --sacks-number=128\"...
...
...
\" File \\"/usr/lib/python2.7/site-packages/pymysql/err.py\\", line 107, in raise_mysql_exception\",
\" raise errorclass(errno, errval)\",
\"InternalError: (1050, u\\"Table 'archive_policy_rule' already exists\\")\",
\"2018-03-13 00:11:59,916 [7] CRITICAL root: Traceback (most recent call last):\",
\" File \\"/usr/bin/gnocchi-upgrade\\", line 10, in <module>\",
\" sys.exit(upgrade())\",
What we suspect is some sort of race condition where one out of the 3 controller creates the table and others throws the exception.
This is the cod, my question in comment - # Yossi
def create_ archive_ policy( self, archive_policy):
name= archive_ policy. name,
back_ window= archive_ policy. back_window,
definition =archive_ policy. definition,
aggregatio n_methods= list(archive_ policy. aggregation_ methods) , writer( ) as session:
session. add(ap) DBDuplicateEntr y:
ap = ArchivePolicy(
)
try:
with self.facade.
except exception.
# Yossi comment - can the below change to 'Warning' or something? if the table exists that's great, we can skip. or am i missing something ?
raise indexer. ArchivePolicyAl readyExists( archive_ policy. name)
return ap