SYSTEM mode, instances do not run

Bug #430957 reported by Neil Soman on 2009-09-16
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Eucalyptus
Fix Released
Critical
Unassigned
1.6
Fix Released
Critical
Unassigned
eucalyptus (Ubuntu)
Medium
Dustin Kirkland 

Bug Description

Multicluster (CLC/CC/SC Walrus/NC).

Resources are allocated, run instances not sent to the CC.

12:16:08 WARN ClusterNodeState | BEFORE ALLOCATE ============================
12:16:08 WARN ClusterNodeState | [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 WARN ClusterNodeState | DURING ALLOCATE ============================
12:16:08 WARN ClusterNodeState | TAILSET: [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=2}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=1}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 WARN ClusterNodeState | HEADSET: []
12:16:08 WARN ClusterNodeState | AFTER ALLOCATE ============================
12:16:08 WARN ClusterNodeState | [VmTypeAvailability{type=VmType{name='m1.small', cpu=1, disk=2, mem=128}, max=2, available=1}
                                    | , VmTypeAvailability{type=VmType{name='c1.medium', cpu=1, disk=5, mem=256}, max=2, available=1}
                                    | , VmTypeAvailability{type=VmType{name='m1.large', cpu=2, disk=10, mem=512}, max=1, available=0}
                                    | , VmTypeAvailability{type=VmType{name='m1.xlarge', cpu=2, disk=20, mem=1024}, max=1, available=0}
                                    | , VmTypeAvailability{type=VmType{name='ZERO', cpu=-1, disk=-1, mem=-1}, max=0, available=0}
                                    | , VmTypeAvailability{type=VmType{name='c1.xlarge', cpu=4, disk=20, mem=2048}, max=0, available=0}
                                    | ]
12:16:08 INFO SLAs | [ preallocate] NetworkToken{networkName='default', cluster='mycluster', vlan=10, userName='admin', name='admin-default'}
12:16:08 INFO SLAs | [ accepted] NetworkToken{networkName='default', cluster='mycluster', vlan=10, userName='admin', name='admin-default'}
12:16:08 INFO VmAdmissionControl | Taking address index: 2

Neil Soman (neilsoman) on 2009-09-16
Changed in eucalyptus:
importance: Undecided → Critical
Dustin Kirkland  (kirkland) wrote :

I haven't confirmed, but I'm matching upstream's priority of Critical. This seems like core, broken functionality that simply must work for Eucalyptus to be useful.

:-Dustin

Changed in eucalyptus (Ubuntu):
importance: Undecided → Critical
Neil Soman (neilsoman) wrote :

This was a problem with SYSTEM mode (UEC default is supposed to be MANAGED_NOVLAN if I'm not mistaken so this shouldn't affect that). I believe this was fixed but I need to confirm that against the upstream (I will update this bug when I do).

Matt Zimmerman (mdz) on 2009-09-25
Changed in eucalyptus (Ubuntu):
status: New → Triaged
Changed in eucalyptus (Ubuntu):
importance: Critical → High
Neil Soman (neilsoman) wrote :
Download full text (5.9 KiB)

MANAGED and MANAGED_NOVLAN work fine with revno 894.

SYSTEM mode shows the following behavior now.

The NC keeps getting run instance over and over.

cc.log

[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] RunInstances(): called
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] refresh_resources(): called
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] node=eucabuild29 mem=3728/3728 disk=117249/117249 cores=2/2
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] refresh_resources(): done
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] calling http://eucabuild29:8775/axis2/services/EucalyptusNC
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] running instance i-35E9072D with emiId emi-E750108E...
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] assigning MAC/IP: d0:0d:35:E9:07:2D/0.0.0.0/0.0.0.0/-1
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] time left for next op: 60
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] node=eucabuild29 mem=3728/3728 disk=117249/117249 cores=2/2
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] refresh_resources(): done
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] running instance i-35E9072D with emiId emi-E750108E...
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] scheduler using ROUNDROBIN policy to find next resource
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] scheduler state starting at resource 0
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] scheduler state finishing at resource 0
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] scheduler decided to run instance 'i-35E9072D' on resource 'http://eucabuild29:8775/axis2/services/EucalyptusNC'
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] assigning MAC/IP: d0:0d:35:E9:07:2D/0.0.0.0/0.0.0.0/-1
[Fri Sep 25 20:18:30 2009][019952][EUCAINFO ] client (http://eucabuild29:8775/axis2/services/EucalyptusNC) running instance: i-35E9072D emi-E750108E d0:0d:35:E9:07:2D d0:0d:35:E9:07:2D 10 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTMSCSb4VovoHjYjlm5QpmSHZi0WPe4HD5PGoN+jqPoLESJBNIM85g2DPQnEtxy3JIQk5iLpM6yqM+qsZKjnTRg9OIid2Ile/dI5nVjt6UgCcnz9KJSlzdKPhsGxsviRJS2gKJ1mkqZW3SKv1kAAyj5PQywwz5mTCfjd6BrhJhppHstFal0RQNwnpT2zgvvMtWpmgJkG3OOe2uZPQ74HOVGsrq+lcSJa5T6aWpSk8655QKCS0CYsAnMBOcBOhpfdAtv2GTiRgxtXaQgDFHfjI/GaguzesNtSyQWB5BAXlwWqYHJGpf5cYWvFyRiS1tOR0vpP4LM3JIq+vMoBwjlaD admin@eucalyptus
[Fri Sep 25 20:18:30 2009][019952][EUCAINFO ] asking for virtual hardware (mem/disk/cores): 128/2/1
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] time left for op: 60
[Fri Sep 25 20:18:30 2009][018979][EUCAINFO ] call complete (pid/rc): 19952/0
[Fri Sep 25 20:18:30 2009][018979][EUCADEBUG ] RunInstances(): done
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] scheduler using ROUNDROBIN policy to find next resource
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] scheduler state starting at resource 0
[Fri Sep 25 20:18:30 2009][019078][EUCADEBUG ] scheduler state finishing at resource 0
[Fri Sep 25 20:18:30 2009][019078][EUCAINFO ] scheduler decided to run instance 'i-35E9072D' on resource 'http://eucabuild29:8775/axis2/services/EucalyptusNC'
[Fri Sep 25 20:18:30 2009][019953][EUCAINFO ] client (http://eucabuild29:8775/axis2/services/EucalyptusNC) running instance: i-35E9072D emi-E750108E d0:0d:35:...

Read more...

summary: - SYSTEM mode, clc does not send runinstances.
+ SYSTEM mode, instances do not run
Neil Soman (neilsoman) wrote :

Revno 897 fixes this.

Changed in eucalyptus:
status: New → Fix Committed
Matt Zimmerman (mdz) on 2009-09-28
Changed in eucalyptus (Ubuntu):
importance: High → Medium
Daniel Nurmi (nurmi) on 2009-10-05
Changed in eucalyptus (Ubuntu):
status: Triaged → In Progress
assignee: nobody → Dustin Kirkland (kirkland)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package eucalyptus - 1.6~bzr912-0ubuntu1

---------------
eucalyptus (1.6~bzr912-0ubuntu1) karmic; urgency=low

  * Merge upstream revision 912
  * tools/eucalyptus-cc.in: resolve conflict, ensuring that we maintain
    the 'ThreadsPerChild 1' sed
  * clc/modules/www/src/main/java/edu/ucsb/eucalyptus/admin/server/ServletUtils.java:
    resolve conflict, drop our diff, as upstream has solved the smtp
    hostname issue
  * debian/patches/boot-order.patch: dropped, applies to a file we don't
    use anymore so it can be removed
  * debian/eucalyptus-walrus.eucalyptus-walrus-registration.upstart: source
    conf file and use $WALRUS_IP_ADDR variable, like the other scripts
  * This snapshot is expected to fix the following bugs:
    - LP: #388934 - apply authorization checks
    - LP: #430226 - fix display of image permissions
    - LP: #430957 - fix running of instances in SYSTEM mode
    - LP: #436276 - sc should run as eucalyptus, instead of root
    - LP: #436313 - fix sc registration through web ui
    - LP: #436407 - fix cc segfaults with apache-mpm-worker, rampart
    - LP: #436885 - fix database corruption, c3p0 deadlock on CLC
    - LP: #437014 - handle execessive CLC sockets
    - LP: #439251 - fix restart-required after autoregistration issue
    - LP: #440744 - handle external command
    - LP: #440817 - fail gracefully with volume deletion on sc
    - LP: #443125 - fix loss of admin credentials

 -- Dustin Kirkland <email address hidden> Mon, 05 Oct 2009 15:02:11 -0500

Changed in eucalyptus (Ubuntu):
status: In Progress → Fix Released
Changed in eucalyptus:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers