maas deployer stuck at "MAAS controller is now configured

[Narinder Gupta]

while deploying MAAS i am facing the following issue during maas installation. Complete logs can be find at
/var/log/cloud-init-output.log
http://paste.ubuntu.com/16628285/

pastebinit /var/log/cloud-init.log
http://paste.ubuntu.com/16628292/

pastebinit /var/log/syslog
http://paste.ubuntu.com/16628295/

pastebinit < maas_deployer.log
http://paste.ubuntu.com/16628312/

Issue is intermittent 1 out of 3-5 install passes.

 > maasserver:0186_space_fabric_names_optional_and_not_unique
 > maasserver:0187_link_missing_subnets
 > maasserver:0188_dli_power_driver_system_id_to_outlet_id
 > maasserver:0189_remove_block_poweroff
 > maasserver:0190_add_mtu_to_vlan_model
 > maasserver:0191_remove_comma_from_dns_servers
 - Loading initial data for maasserver.
Traceback (most recent call last):
  File "/usr/sbin/maas-region-admin", line 68, in <module>
    main()
  File "/usr/sbin/maas-region-admin", line 64, in main
    run()
  File "/usr/sbin/maas-region-admin", line 57, in run
    management.execute_from_command_line()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/dist-packages/south/management/commands/migrate.py", line 107, in handle
    ignore_ghosts = ignore_ghosts,
  File "/usr/lib/python2.7/dist-packages/south/migration/__init__.py", line 219, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/usr/lib/python2.7/dist-packages/south/migration/migrators.py", line 237, in migrate_many
    self.load_initial_data(target, db=database)
  File "/usr/lib/python2.7/dist-packages/south/migration/migrators.py", line 228, in load_initial_data
    call_command('loaddata', 'initial_data', verbosity=self.verbosity, database=db)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 55, in handle
    self.loaddata(fixture_labels)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 84, in loaddata
    self.load_label(fixture_label)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 140, in load_label
    obj.save(using=self.using)
  File "/usr/lib/python2.7/dist-packages/django/core/serializers/base.py", line 164, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 654, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 687, in _do_insert
    using=using, raw=raw)
  File "/usr/lib/python2.7/dist-packages/django/db/models/manager.py", line 232, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 1511, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 898, in execute_sql
    cursor.execute(sql, params) > maasserver:0186_space_fabric_names_optional_and_not_unique
 > maasserver:0187_link_missing_subnets
 > maasserver:0188_dli_power_driver_system_id_to_outlet_id
 > maasserver:0189_remove_block_poweroff
 > maasserver:0190_add_mtu_to_vlan_model
 > maasserver:0191_remove_comma_from_dns_servers
 - Loading initial data for maasserver.
Traceback (most recent call last):
  File "/usr/sbin/maas-region-admin", line 68, in <module>
    main()
  File "/usr/sbin/maas-region-admin", line 64, in main
    run()
  File "/usr/sbin/maas-region-admin", line 57, in run
    management.execute_from_command_line()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 242, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/dist-packages/south/management/commands/migrate.py", line 107, in handle
    ignore_ghosts = ignore_ghosts,
  File "/usr/lib/python2.7/dist-packages/south/migration/__init__.py", line 219, in migrate_app
    success = migrator.migrate_many(target, workplan, database)
  File "/usr/lib/python2.7/dist-packages/south/migration/migrators.py", line 237, in migrate_many
    self.load_initial_data(target, db=database)
  File "/usr/lib/python2.7/dist-packages/south/migration/migrators.py", line 228, in load_initial_data
    call_command('loaddata', 'initial_data', verbosity=self.verbosity, database=db)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 159, in call_command
    return klass.execute(*args, **defaults)
  File "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line 285, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 55, in handle
    self.loaddata(fixture_labels)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 84, in loaddata
    self.load_label(fixture_label)
  File "/usr/lib/python2.7/dist-packages/django/core/management/commands/loaddata.py", line 140, in load_label
    obj.save(using=self.using)
  File "/usr/lib/python2.7/dist-packages/django/core/serializers/base.py", line 164, in save
    models.Model.save_base(self.object, using=using, raw=True)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 654, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 687, in _do_insert
    using=using, raw=raw)
  File "/usr/lib/python2.7/dist-packages/django/db/models/manager.py", line 232, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 1511, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 898, in execute_sql
    cursor.execute(sql, params)
  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python2.7/dist-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: Problem installing fixture '/usr/lib/python2.7/dist-packages/metadataserver/fixtures/initial_data.yaml': Could not load auth.User(pk=1): deadlock detected
DETAIL: Process 21073 waits for ShareLock on transaction 1160; blocked by process 22689.
Process 22689 waits for AccessExclusiveLock on relation 16458 of database 16385; blocked by process 21073.
HINT: See server log for query details.

  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute

  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/usr/lib/python2.7/dist-packages/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
django.db.utils.OperationalError: Problem installing fixture '/usr/lib/python2.7/dist-packages/metadataserver/fixtures/initial_data.yaml': Could not load auth.User(pk=1): deadlock detected
DETAIL: Process 21073 waits for ShareLock on transaction 1160; blocked by process 22689.
Process 22689 waits for AccessExclusiveLock on relation 16458 of database 16385; blocked by process 21073.
HINT: See server log for query details.

  File "/usr/lib/python2.7/dist-packages/django/db/backends/util.py", line 53, in execute

[fox (shimingtao)]

how do you install maas-deployer?

[Edward Hope-Morley (hopem)]

Hi Narinder, can you tell me what version of MAAS this is and what version of Ubuntu you are installing on and from? Also since this error occurred a little while back now could you possibly confirm whether you are still seeing this issue?

For the record, we do not yet provide support for deploying MAAS 2.0 or deploying to Xenial. This is principally due to changes in cloud-init and the MAAS cli that we have yet to implement support for.

@shimingtao for Trusty/1.x installs you can follow the normal install procedure at https://launchpad.net/~maas-deployers.

[Narinder Gupta (narindergupta)]

Hi Edward and Fox I am using maas-deployer stable PPA and I am still seeing
this issue on all labs but it is 1 out of 2 installations still gets stuck
at same point.

I am using MAAS 1.9.3 with Trusty here for deployment. This issue is during
MAAS deployment and configuration using the maas-deployer.

Thanks and Regards,
Narinder Gupta (PMP) <email address hidden>
Canonical, Ltd. narindergupta [irc.freenode.net]
+1.281.736.5150 narindergupta2007[skype]

Ubuntu- Linux for human beings | www.ubuntu.com | www.canonical.com

On Mon, Jul 18, 2016 at 4:41 AM, Edward Hope-Morley <
<email address hidden>> wrote:

> Hi Narinder, can you tell me what version of MAAS this is and what
> version of Ubuntu you are installing on and from? Also since this error
> occurred a little while back now could you possibly confirm whether you
> are still seeing this issue?
>
> For the record, we do not yet provide support for deploying MAAS 2.0 or
> deploying to Xenial. This is principally due to changes in cloud-init
> and the MAAS cli that we have yet to implement support for.
>
> @shimingtao for Trusty/1.x installs you can follow the normal install
> procedure at https://launchpad.net/~maas-deployers.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1584584
>
> Title:
> maas deployer stuck at "MAAS controller is now configured
>
> Status in maas-deployer:
> New
>
> Bug description:
> while deploying MAAS i am facing the following issue during maas
> installation. Complete logs can be find at
> /var/log/cloud-init-output.log
> http://paste.ubuntu.com/16628285/
>
> pastebinit /var/log/cloud-init.log
> http://paste.ubuntu.com/16628292/
>
> pastebinit /var/log/syslog
> http://paste.ubuntu.com/16628295/
>
> pastebinit < maas_deployer.log
> http://paste.ubuntu.com/16628312/
>
> Issue is intermittent 1 out of 3-5 install passes.
>
> > maasserver:0186_space_fabric_names_optional_and_not_unique
> > maasserver:0187_link_missing_subnets
> > maasserver:0188_dli_power_driver_system_id_to_outlet_id
> > maasserver:0189_remove_block_poweroff
> > maasserver:0190_add_mtu_to_vlan_model
> > maasserver:0191_remove_comma_from_dns_servers
> - Loading initial data for maasserver.
> Traceback (most recent call last):
> File "/usr/sbin/maas-region-admin", line 68, in <module>
> main()
> File "/usr/sbin/maas-region-admin", line 64, in main
> run()
> File "/usr/sbin/maas-region-admin", line 57, in run
> management.execute_from_command_line()
> File
> "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line
> 399, in execute_from_command_line
> utility.execute()
> File
> "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line
> 392, in execute
> self.fetch_command(subcommand).run_from_argv(self.argv)
> File
> "/usr/lib/python2.7/dist-packages/django/core/management/base.py", line
> 242, in run_from_argv
> self.execute(*args, **options.__dict__)
> File
> "/usr/lib/python2.7/dist-packages/dja...