Drives and Images sizes issues in Ubuntu 14.04

[xjesus.net]

First of all, thank you Kendek for the good work.

I see you are publishing a lot of updates lately and I was hoping that my issues would solve because since version 1.0.0.31 I have been facing 3 issues in Ubuntu 14.04:

1) The sizes of the drives are wrong. They always show a lot less space than they really have. For example one empty 16G pendrive displays 2,5G, but another empty 8G pendrive displays 3,5G and so on... The available images display 0,01GB when they actually are 2,7GB...
2) There is no way to convert from sblive to ISO
*** 3) When I restore a copy to another PC, the /home partition is not restored correctly, it misses most of the files and configs even though I checked the option to include user data when I created the image.

Hope you can find the way to solve all these things. If I can help you somehow troubleshoot and test this, please just ask me.

Thanks and good luck!

[Kendek (nemh)]

1) I not see any problems with drives sizes on my 64-bit computers with TB size devices. But I using unsigned long instead of unsigned long long. Now I checked this, and now I see, ulong isn't enough on 32-bit operating system.
I fix this problem soon.

2) Maybe the converting isn't possible because previous size problems.
(But otherwise, there is a need to double the size of free space.)

3) I testing this soon...

[Kendek (nemh)]

3) I tested this, but I couldn't find any problem.
Don't remember, if you want to copy all the files, use 'System copy' function instead of 'System install'. System install is copying only with user configuration files with 8 MB file size limit.

[Kendek (nemh)]

The size detection bug is fixed in version 1.0.0.211.

[xjesus.net (xjesus.net)]

The size issue is solved. Thanks a lot Kendek.

I didn't know about the file size limit for user configuration files. It's good to know it. But anyway it didn't copy anything at all.

Now I tested with the last version 1.0.0.211 and I can create the pendrive and the ISO.

But when I boot with the pendrive in another PC, and I go through the process of restoring the image in 3 partitions (root and home with ext4 and swap, which I have to unmount before), then I receive an error message after the whole process is completed:
" La instalación del sistema se abortó!
No han habido cambios significativos en el sistema de archivos durante esta operación."

That means something like "The install process is aborted! No significant changes have been made to the file system during this process".

I see that the root partition "/" is filled with the content, but then I cannot boot the system. I see the Grub menu and after selecting to boot with Ubuntu it takes just a couple of seconds to reboot again in a loop.

The /home partition only creates the common folders (downloads, documents, etc. translated to Spanish, i.e., Descargas, Documentos, ...) but with no files at all. The strange thing is that I can see that there is some "space" occupied because the partition is 105GB in total but there is only 100GB free.

I am working with Ubuntu (with XFCE) 14.04 32 bits. I am working with "System Install" because I want to create an ISO to create some king of new "distro" that people can download and install.

Do you need some logs or more info?

Should I open another bug?

Thanks

[Kendek (nemh)]

If possible, please share with me this ISO, and I will checking it. That would be the best.

[Kendek (nemh)]

I tested again with 32-bit operating system (virtualized Ubuntu), but the Live create and the Live installation processes are works correctly. So please share your Live system with me, I will testing it on my real machine. If anything is wrong with it, I promise that I will fixing. :-)

[Kendek (nemh)]

I think I found the cause of this problem. If you use the same username at installation time as the source system, the installation process may cause an error at the very end. Tomorrow I test this thing.

[xjesus.net (xjesus.net)]

It might be the cause of that problem because I tried again with another image and it didn't give me the error. Just by chance I changed the users and might be that's why it began to work.

Now, I still have the problem with the /home partition when it's separated in another different partition than /.
I think I also have some issues with the swap partition that doesn't mount automatically in the restored image.

I've uploaded my ISO where I face issues with both /home and swap.

https://drive.google.com/file/d/0B7RTzR1724zgY1h3a0VFcnVjcGM/edit?usp=sharing

When I boot the pendrive I can see all the files in the /home folder, but when I do the "recovery" something is wrong with the file system in /home because I can only see folders but no files, even though there is a lot of space "occupied".

I've workarounded it by creating the source image from an installation with a /home folder in the same partition as /, and then it seems to work right.

[Kendek (nemh)]

Hm. I try to installing this system, but an error occured while reading the /usr/lib/firefox/omni.ja file. This is a filesystem error, and fully reproducable with cp command too. So far I haven't seen one.

I fixed username error, and I add a debug message which show item name which cause the error. Just running Systemback in terminal emulator, and read printed error message.

I tested Live system creation and installation with seperated /home partition, but I not found any problem. User configuration files are copied correctly, the created fstab file is contain SWAP partition, and that is automatically activated after the installed system is booted.

Definition of user configuration files:
Hidden files and files in the hidden directories, with 8 MB size limit.

[xjesus.net (xjesus.net)]

Thanks for all the tips and fixes. Now I can get a log when there is some corrupt file. I don't know why but I have experienced it with other files... It would be nice to have the log message even if you don't run it from a terminal emulator. I don't know if I understood you right if you already made it.

I think you have really deserved some Paypal donation ;) Which by the way was hard for me to find, so another suggestion would be to have all the menus in only one page and one big button with the Paypal.

[Kendek (nemh)]

Thanks the donation, I really appreciate it! ;-)
These corrupted files are really strange. This issue affects the original, source system too, or just the created Live?
The Systemback doesn't make log, these debug messages only printing in the terminal. You have to manually redirect the error output if you want. I don't yet plan to graphically displays the affected file name.
Unfortunately, the menu items don't fit on a single page. :-( But not too hard switch between them. :-)

[xjesus.net (xjesus.net)]

This is what I get when a create the Live System:

usu@x61:~$ update-initramfs: Generating /boot/initrd.img-3.13.0-32-generic
Cannot stat exclude dir/file /etc/udev/rules.d/70-persistent-cd.rules because No such file or directory, ignoring
File /home/.sbuserdata/home/usu/.cache/upstart/startxfce4.log changed size while reading filesystem, attempting to re-read

Come on, I'm sure you can rearrange the menus and make this app as cool-looking in the outside as it is in the inside ;)

[xjesus.net (xjesus.net)]

I'm still facing the issue with /home partition. It's not restored and don't know why. I changed the /home partition to be in the same partition as everything else. I thought it was working right but it isn't. And the only error logs I find are those I sent you in the message before.

Can you try again and see if you see the cause with my image?

You can find it here:
https://docs.google.com/file/d/0B7RTzR1724zgRnZneFVPRjVhRm8/edit

user and password are both "usu"

[Kendek (nemh)]

Excluded /etc/udev/rules.d/70-persistent-cd.rules file doesn't exist, ignoring, not problem.
The /home/usu/.cache/upstart/startxfce4.log file has changed, re-readed, not problem.

Yeah, but if I add more menu items, they are in the end it didn't fit, not at all. Cannot indefinitely increase the number of elements on a page. And I hope I will add more features to the application.

[xjesus.net (xjesus.net)]

I've tried to check if changing users during the systemback restore solves restoring the /home partition but it's not the case.

I think my problem is that I manually copied the /home partition from one computer with user "usuario" to another computer with user "usu". And then there must be some inconsistencies like the bookmarks in file managers pointing to /home/usuario/* instead of /home/usu/* ... and other important things...

I'm trying to solve this myself but if you can give some hint I'd appreciate it...

I'm trying to change the user name from "usu" to "usuario" but it doesn't seem to be that easy...
http://ubuntuforums.org/showthread.php?t=1742602

[xjesus.net (xjesus.net)]

But I wonder why when I boot from the Live pendrive I can see all the files in /home/usu but then when I restore it to the computer they are not copied.

[Kendek (nemh)]

I tested this new image (0.7.7) with Systemback version 1.0.1.103 (upgrading on Live). I didn't find any problem, the user configuration files have been copied successfully. I take two screenshot as well:

http://logout.hu/dl/upc/2014-07/180556_systemback_installing_1.png
http://logout.hu/dl/upc/2014-07/180556_systemback_installing_2.png

And the installed system has started without problems.
But of course the desktop items are missing (not configuration files), and the bookmarks in the PCManFM are show the wrong place (usuario vs. usu), but you already mentioned it.

And again:
If you install the system, copying only (one) user configuration files, or items from /etc/skel directory. If you copy the system, copying all items from all user dirs, or copying users configuration files. It depends on what is unchecked.
The installation process doesn't copy data files. Example, data files (icons) on the user desktop.

[xjesus.net (xjesus.net)]

Then I understood it wrong. I thought that only hidden files bigger than 8MB would be excluded. And I am pretty sure that in previous releases I've been able to restore the desktop icons and folders doing this same process with an .ISO image.

Do you know some way to workaround this? I would like to be able to create an image where I can install some folders and icons in the desktop.

I'm getting confused and it doesn't make sense for me. Maybe I'm lost in translation :)
In Spanish I do:
- "Crear Sistema Live"
- Check box "Incluir archivos de datos de usuario".
- I Create the image, convert it to an .ISO file and burn it with Unetbootin. (to check that others can do it without previously installing systemback).

Afterwards, I go to another PC, I boot with the Pendrive, I run systemback and I check the box "Transferir archivos de configuación de usuario".

I think the problem is that there is a mismatch between the two checkboxes ;) when you create the image, and when you install it. Could you add some other check box to say "Incluir archivos de datos de usuario" when you install the image?

Sorry if I am being a little "pain in the ass" :)
I am dreaming of the day that I can use your tool to let my students (and friends) have a complete Linux installation easily. At home and in the School PCs. And I think it's a useful thing to have the desktop files, folders, launchers, etc. at hand.

[Kendek (nemh)]

Maybe I will enable .desktop files copying from ~/Desktop directory at installation time *. But no other files. Personal documents and stuff are not installing over a foreign people.
The Live system can be used not only for installation. This is just a function, and it doesn't copy the data, but installing the system.
Check System copy function on Live system, you will find same "user data files copy" check box. This function will have to copy your personal data files.

* Some people are not happy about this. They placed the install icon on the desktop. And after installation, this install icon is disappears. In a way, it's so good. But this may be easier to be solved.

[xjesus.net (xjesus.net)]

Ok, now I've seen and understood your "workaround", but you have to pay attention and press the "Back" botton after the initial form to install asking you the user/password, name of the PC, etc. Otherwise you can't see the "Copy function".

The problem is that you lose the chance to change user/password/name of the PC, etc. and might confuse users to select the right partitions. By the way, it would be helpful to see the label of the partitions.

[xjesus.net (xjesus.net)]

Another thing you lose is that if you change the swap partition from one PC to other, then you can't automatically mount swap, don't you?

[Kendek (nemh)]

The partition label thing is solvable, and it makes sense too.
And I thinking on the user data files...

The SWAP partitions aren't formatting, only if not mounted. But SWAP partition are mounting automatically at boot time (with Casper), so just not umount. If a SWAP partition isn't formatted, then UUID characters aren't changing. In this case, each system will be good for automatic SWAP activation. But this rule doesn't stem from the Live system. If you install your installed (so not Live) system to another partition, they will using common SWAP.

[Kendek (nemh)]

I added 'Label' column and user data files transferring possibility to version 1.0.2.100.

[xjesus.net (xjesus.net)]

Wow! Thanks a lot Kendek. This is amazing.

I was thinking that after all, If you want to round it completely 100% you should add some shortcut to Gparted in your application or integrate some kind of basic partition funcionality. Then it would really be the perfect personalized-distro tool.

I mean, if you compare the installation process from Ubuntu for example, that's what it does. First it takes you though a process where you choose the locale, etc. and then after it takes you to the partition procedure before installing the distro.

But this is just a thought... I don't want to ask you so much.... Well, maybe only a little shortcut to Gparted would be quite easy, wouldn't it?

Thanks, thanks, thanks

[Kendek (nemh)]

Yeah, but now Systemback is a pure Qt application, the GParted icon isn't going to happen.
But of course I have been thinking about a basic partition manager.

But above all, remember that the system install isn't a main function. The primary goal of the system restoration. Although this is probably the least used function??

[Kendek (nemh)]

I fixed the checkbox text displacement problem, but the PartiallyChecked state is poorly visible on Xubuntu with tristate checkbox. So, the default state is PartiallyChecked, and that means the user configuration files transferring option. If you would like to copy all the files, you should be mark fully (pipe visible, and not striked text).

[xjesus.net (xjesus.net)]

I'm afraid it's not clear for the average user. I think 2 check boxes would be clearer. And in my opinion they should be both checked by default, but I understand that you don't want to change the behaviour of the program by default.

Also I think people would like to see some short and proper explanation about what is "config files" and "data files". Something like "/home/user/.* [smaller than 8MB]" in the first box, and "/home/user/*" in the second box.

I've been thinking that maybe what I really need is a fork of systemback called something like sysinstall ;) But I don't like forks and I don't know how to do it :) The idea is that there are too many options already in systemback and like you said before, I am only using the "Create Live System" and installing it. There should be some piece of software that doesn't need a user guide to make the normal install. People in general hate user guides and don't read them. Or even worse, they just don't install something that first they have to read a user guide.

About Gparted, it doesn't matter if there is no proper icon. The only thing needed is a botton to do the partitioning. If you are going to create something more beautiful and easier like the Ubuntu install with automatic resizing of the Windows partition, etc. then it's OK, but if you are going to develop something with less features than GParted... I'd just prefer to see a link to Gparted for the moment.

[Kendek (nemh)]

Yeah, this is the current solution is a bit unique. Of course I've been thinking a second checkbox, but technically it would have been a little more obstacles. It would be nice if the layout be consistent with the system copy function.

If you like, you can just rename the launcher. If you place the icon called "Install system" on the desktop, users can find him sooner. The same is true for the partition manager. That's what I said, that is normally good because these icons disappear after installation.
But anyway I think you cannot do in C++, so the major rewrite would be a bit more difficult. But of course I can help you if you would need some data in this regard though.

And this option just a click, and I think with no great trouble if the icons are missing from the desktop. What other relevant data would be needed for default installation?
In the worst case, of course, the user must have read that it would be good if he mark the check box.

[HansBKK (hansbkk)]

Not to hijack but - on the "separate 'create distro' 'cold-metal restore' " idea.

Create a bootable system backup/rescue ISO like Clonezilla's and Parted Magic in addition to the installed package.

That way 'create distro' 'cold-metal restore' users can feel more secure that their system is quiescent rather than running while taking the snapshot.

Especially important for advanced users & developers that may have web/mail/DB services running.

This would also enable integrating with parted (and maybe even clonezilla, sysresccd?), would be a killer sysadmin tool.

[xjesus.net (xjesus.net)]

Thanks for your comments Kendek but I don't think I can accomplish such a task for the moment.