Update doesn't work when upgrading from devel r145 to r383 devel-proposed

Bug #1444500 reported by Kick In on 2015-04-15
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Snappy
Critical
James Hunt

Bug Description

kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./start_it.sh dev
Les ports de cette machine seront les suivants:
SSH: 8022
HTTP: 8080
HTTPS: 8443
Un rappel des ports est disponible: dev.ports
kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
ssh_exchange_identification: read: Connection reset by peer
kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
Warning: Permanently added '[127.0.0.1]:8022' (ECDSA) to the list of known hosts.
Welcome to Ubuntu Vivid Vervet (development branch) (GNU/Linux 3.19.0-12-generic x86_64)

 * Documentation: https://help.ubuntu.com/
Welcome to the Ubuntu Core rolling development release.

 * See https://ubuntu.com/snappy

It's a brave new world here in snappy Ubuntu Core! This machine
does not use apt-get or deb packages. Please see 'snappy --help'
for app installation and transactional updates.

Last login: Tue Apr 14 15:46:36 2015
ubuntu@localhost:~$ cat /etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel
ubuntu@localhost:~$ cat /writable/cache/system/etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel
ubuntu@localhost:~$ sudo mount -o remount rw /
ubuntu@localhost:~$ sudo mount -o remount rw /writable/cache/system
ubuntu@localhost:~$ sudo sed -i s/devel/devel-proposed/ /etc/system-image/channel.ini
ubuntu@localhost:~$ cat /etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel-proposed
ubuntu@localhost:~$ sudo sed -i s/devel/devel-proposed/ /writable/cache/system/etc/system-image/channel.ini
ubuntu@localhost:~$ cat /writable/cache/system/etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel-proposed
ubuntu@localhost:~$ sudo mount -o remount ro /
ubuntu@localhost:~$ sudo mount -o remount ro /writable/cache/system
ubuntu@localhost:~$ sudo reboot
Connection to 127.0.0.1 closed by remote host.
Connection to 127.0.0.1 closed.
kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
Warning: Permanently added '[127.0.0.1]:8022' (ECDSA) to the list of known hosts.
Welcome to Ubuntu Vivid Vervet (development branch) (GNU/Linux 3.19.0-12-generic x86_64)

 * Documentation: https://help.ubuntu.com/
Welcome to the Ubuntu Core rolling development release.

 * See https://ubuntu.com/snappy

It's a brave new world here in snappy Ubuntu Core! This machine
does not use apt-get or deb packages. Please see 'snappy --help'
for app installation and transactional updates.

Last login: Wed Apr 15 14:08:18 2015 from 10.0.2.2
ubuntu@localhost:~$ cat /writable/cache/system/etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel-proposed
ubuntu@localhost:~$ cat /writable/cache/system/etc/system-image/channel.ini
[service]
base: system-image.ubuntu.com
http_port: 80
https_port: 443
channel: ubuntu-core/devel-proposed
device: generic_amd64
build_number: 145
version_detail: ubuntu=20150223.2,raw-device=20150223.2,version=145
ubuntu@localhost:~$ cat /etc/system-image/channel.ini | grep devel
channel: ubuntu-core/devel-proposed
ubuntu@localhost:~$ cat /etc/system-image/channel.ini
[service]
base: system-image.ubuntu.com
http_port: 80
https_port: 443
channel: ubuntu-core/devel-proposed
device: generic_amd64
build_number: 146
version_detail: ubuntu=20150410.3,raw-device=20150410.3,version=146
ubuntu@localhost:~$ sudo snappy update
Installing ubuntu-core (384)
Starting download of ubuntu-core
135.00 MB / 135.00 MB [=============================================================================================================================================================================================] 100.00 % 1.55 MB/s
Done
error from system-image-cli: ('%s exited with %s:\n%s', ['ubuntu-core-upgrade'], 1, 'I: preparing root filesystem\nTraceback (most recent call last):\n File "/usr/bin/ubuntu-core-upgrade", line 182, in <module>\n main()\n File "/usr/bin/ubuntu-core-upgrade", line 155, in main\n prepare_upgrade(options)\n File "/usr/bin/ubu')

Journalctl is not very verbose:

Apr 15 14:11:06 localhost.localdomain systemd[731]: Starting Default.
Apr 15 14:11:36 localhost.localdomain sudo[766]: ubuntu : TTY=pts/0 ; PWD=/home/ubuntu ; USER=root ; COMMAND=/usr/bin/snappy update
Apr 15 14:11:36 localhost.localdomain sudo[766]: pam_unix(sudo:session): session opened for user root by ubuntu(uid=0)
Apr 15 14:12:35 localhost.localdomain kernel: EXT4-fs (sda3): re-mounted. Opts: (null)
Apr 15 14:12:35 localhost.localdomain kernel: EXT4-fs (sda4): mounted filesystem with ordered data mode. Opts: (null)
Apr 15 14:12:35 localhost.localdomain kernel: EXT4-fs (sda4): mounted filesystem with ordered data mode. Opts: (null)
Apr 15 14:13:04 localhost.localdomain sudo[766]: pam_unix(sudo:session): session closed for user root
Apr 15 14:17:01 localhost.localdomain CRON[908]: pam_unix(cron:session): session opened for user root by (uid=0)
Apr 15 14:17:01 localhost.localdomain CRON[909]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Apr 15 14:17:01 localhost.localdomain CRON[908]: pam_unix(cron:session): session closed for user root
Apr 15 14:17:13 localhost.localdomain sudo[911]: ubuntu : TTY=pts/0 ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/journalctl -l
Apr 15 14:17:13 localhost.localdomain sudo[911]: pam_unix(sudo:session): session opened for user root by ubuntu(uid=0)

Related branches

Kick In (kick-d) on 2015-04-15
Changed in snappy-ubuntu:
importance: Undecided → High
Kick In (kick-d) wrote :

Additional traces:

root@localhost:~# ubuntu-core-upgrade
ERROR: need command file
root@localhost:~# ubuntu-core-upgrade /writable/
cache/ lost+found/ system-data/ user-data/
root@localhost:~# ubuntu-core-upgrade /writable/cache/
assets/ system/
device-dd02f728677fa1ccdf2646ca5bbdb77d747a1c180a10d0e80e0c9419d2eea53d.tar.xz system-ro-788/
device-dd02f728677fa1ccdf2646ca5bbdb77d747a1c180a10d0e80e0c9419d2eea53d.tar.xz.asc ubuntu-ff327cc343626daccb831f8a87034ed975c7be73a02cc3d264a738f8ff5791e0.tar.xz
image-master.tar.xz ubuntu-ff327cc343626daccb831f8a87034ed975c7be73a02cc3d264a738f8ff5791e0.tar.xz.asc
image-master.tar.xz.asc ubuntu_command.applying
image-signing.tar.xz version-384.tar.xz
image-signing.tar.xz.asc version-384.tar.xz.asc
root@localhost:~# ubuntu-core-upgrade /writable/cache/ubuntu_command.applying
I: preparing root filesystem
Traceback (most recent call last):
  File "/usr/bin/ubuntu-core-upgrade", line 182, in <module>
    main()
  File "/usr/bin/ubuntu-core-upgrade", line 155, in main
    prepare_upgrade(options)
  File "/usr/bin/ubuntu-core-upgrade", line 98, in prepare_upgrade
    upgrade(options, commands, remove_list)
  File "/usr/bin/ubuntu-core-upgrade", line 75, in upgrade
    upgrader.run()
  File "/usr/lib/python3/dist-packages/ubuntucoreupgrader/upgrader.py", line 579, in run
    self.dispatcher[cmd](args)
  File "/usr/lib/python3/dist-packages/ubuntucoreupgrader/upgrader.py", line 879, in _cmd_update
    self.options.root_dir))
  File "/usr/lib/python3.4/tarfile.py", line 1973, in extractall
    for tarinfo in members:
  File "/usr/lib/python3/dist-packages/ubuntucoreupgrader/upgrader.py", line 453, in tar_generator
    os.unlink(path)
IsADirectoryError: [Errno 21] Is a directory: '/writable/cache/system/usr/share/apparmor/easyprof/policygroups/ubuntu-snappy'
root@localhost:~# ls -l /writable/cache/system/usr/share/apparmor/easyprof/policygroups/ubuntu-snappy
total 4
drwxr-xr-x 2 root root 4096 Apr 10 16:54 1.3
root@localhost:~#

Michael Vogt (mvo) on 2015-04-15
Changed in snappy-ubuntu:
importance: High → Critical
James Hunt (jamesodhunt) on 2015-04-15
Changed in snappy-ubuntu:
assignee: nobody → James Hunt (jamesodhunt)
Steve Langasek (vorlon) on 2015-04-15
summary: - Update doesn't work when upgrading from devel r146 to r383 devel-
+ Update doesn't work when upgrading from devel r145 to r383 devel-
proposed
James Hunt (jamesodhunt) wrote :

This problem was caused by the partition sync code being called even when a full image was being applied to "other".

I'll add further tests to the branch.

Thought: maybe we should add an option to snappy to request s-i perform a full image upgrade since (once this bug is fixed), that would allow another route in case of further upgrade issues. All we'd need to do is pass '-b 0' to system-image-cli via some means (see bug 1410370). We could even semi-automate this - if a delta update fails, retry by performing a full image update.

James Hunt (jamesodhunt) on 2015-04-16
Changed in snappy-ubuntu:
status: New → Fix Committed
Kick In (kick-d) wrote :

Thanks James, but it seems to be crashing again, do I make a mistake in my commands ?

http://pastebin.ubuntu.com/10835049/

Kick In (kick-d) wrote :

Or maybe I need a newer image than r397 ?

James Hunt (jamesodhunt) wrote :

The fix is in ubuntu-core-upgrader 0.7.10 so if you are starting from a revision with an older version of the upgrader (unclear from your pb), it will still crash of course.

Download full text (6.3 KiB)

Sorry, I missed this obvious reason :( .

2015-04-16 15:57 GMT-05:00 James Hunt <email address hidden>:

> The fix is in ubuntu-core-upgrader 0.7.10 so if you are starting from a
> revision with an older version of the upgrader (unclear from your pb),
> it will still crash of course.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1444500
>
> Title:
> Update doesn't work when upgrading from devel r145 to r383 devel-
> proposed
>
> Status in Snappy Ubuntu:
> Fix Committed
>
> Bug description:
> kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./start_it.sh dev
> Les ports de cette machine seront les suivants:
> SSH: 8022
> HTTP: 8080
> HTTPS: 8443
> Un rappel des ports est disponible: dev.ports
> kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
> ssh_exchange_identification: read: Connection reset by peer
> kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
> Warning: Permanently added '[127.0.0.1]:8022' (ECDSA) to the list of
> known hosts.
> Welcome to Ubuntu Vivid Vervet (development branch) (GNU/Linux
> 3.19.0-12-generic x86_64)
>
> * Documentation: https://help.ubuntu.com/
> Welcome to the Ubuntu Core rolling development release.
>
> * See https://ubuntu.com/snappy
>
> It's a brave new world here in snappy Ubuntu Core! This machine
> does not use apt-get or deb packages. Please see 'snappy --help'
> for app installation and transactional updates.
>
> Last login: Tue Apr 14 15:46:36 2015
> ubuntu@localhost:~$ cat /etc/system-image/channel.ini | grep devel
> channel: ubuntu-core/devel
> ubuntu@localhost:~$ cat
> /writable/cache/system/etc/system-image/channel.ini | grep devel
> channel: ubuntu-core/devel
> ubuntu@localhost:~$ sudo mount -o remount rw /
> ubuntu@localhost:~$ sudo mount -o remount rw /writable/cache/system
> ubuntu@localhost:~$ sudo sed -i s/devel/devel-proposed/
> /etc/system-image/channel.ini
> ubuntu@localhost:~$ cat /etc/system-image/channel.ini | grep devel
> channel: ubuntu-core/devel-proposed
> ubuntu@localhost:~$ sudo sed -i s/devel/devel-proposed/
> /writable/cache/system/etc/system-image/channel.ini
> ubuntu@localhost:~$ cat
> /writable/cache/system/etc/system-image/channel.ini | grep devel
> channel: ubuntu-core/devel-proposed
> ubuntu@localhost:~$ sudo mount -o remount ro /
> ubuntu@localhost:~$ sudo mount -o remount ro /writable/cache/system
> ubuntu@localhost:~$ sudo reboot
> Connection to 127.0.0.1 closed by remote host.
> Connection to 127.0.0.1 closed.
> kick@kick-Lenovo-Yoga-2-13:~/work/sprint/base_work$ ./ssh_it dev
> Warning: Permanently added '[127.0.0.1]:8022' (ECDSA) to the list of
> known hosts.
> Welcome to Ubuntu Vivid Vervet (development branch) (GNU/Linux
> 3.19.0-12-generic x86_64)
>
> * Documentation: https://help.ubuntu.com/
> Welcome to the Ubuntu Core rolling development release.
>
> * See https://ubuntu.com/snappy
>
> It's a brave new world here in snappy Ubuntu Core! This machine
> does not use apt-get or deb packages. Please see 'snappy --help'
> for app installation and transact...

Read more...

Steve Langasek (vorlon) wrote :

does this bug also explain a failure I'm seeing trying to update on devel-proposed from image 373?

Steve Langasek (vorlon) wrote :

(also with an ubuntu-core-upgrader backtrace, to be clear)

James Hunt (jamesodhunt) wrote :

Hi Steve - looks like r373 was one of the 2-3 images that somehow lost the ubuntu-core-upgrader package entirely prior to the promotion.

James Hunt (jamesodhunt) on 2015-04-17
Changed in snappy-ubuntu:
status: Fix Committed → Won't Fix
status: Won't Fix → Fix Released
Michael Terry (mterry) on 2015-05-18
affects: snappy-ubuntu → snappy
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers