[docs] Building fails on Ubuntu 12.04 following developer documentation

[Pete Birley]

I am trying to set up a dev enviroment for Fuel/nailgun.

I have have been follwing the documentation at (http://docs.mirantis.com/fuel-dev/) and from a clean install of ubuntu 13.04 (as guied buy the documentation) I performed the following commands: (I used Ubuntu server and access the box via ssh)

 sudo apt-get update && \
 sudo apt-get upgrade -y && \
 sudo apt-get install -y git && \
 sudo reboot # to load new kernel if required

 #following reboot
 git clone https://github.com/stackforge/fuel-main && \
 git clone https://github.com/stackforge/fuel-web && \
 git clone https://github.com/stackforge/fuel-astute && \
 git clone https://github.com/stackforge/fuel-ostf && \
 git clone https://github.com/stackforge/fuel-library && \
 git clone https://github.com/stackforge/fuel-docs

 sudo apt-get install -y postgresql postgresql-server-dev-9.1 && \
 sudo -u postgres createuser -SDRP nailgun # enter password 'nailgun'

 sudo -u postgres createdb nailgun && \
 sudo apt-get install -y python-dev python-pip python-yaml && \
 sudo pip install virtualenv virtualenvwrapper && \
 source /usr/local/bin/virtualenvwrapper.sh && \
 cd fuel-web && \
 mkvirtualenv fuel && \
 workon fuel && \
 pip install ./shotgun && \
 pip install --allow-all-external -r nailgun/test-requirements.txt && \
 sudo mkdir /var/log/nailgun && \
 sudo chown -R `whoami`.`whoami` /var/log/nailgun && \
 ./run_tests.sh --no-jslint --no-webui && \
 ./run_tests.sh --flake8 && \
 sudo apt-get remove -y nodejs nodejs-legacy && \
 sudo apt-get install -y software-properties-common && \
 sudo add-apt-repository ppa:chris-lea/node.js

 sudo apt-get update && \
 sudo apt-get install -y nodejs && \
 sudo npm install -g grunt-cli && \
 cd nailgun && \
 sudo npm install

 sudo npm install -g phantomjs && \
 cd ~ && \
 git clone git://github.com/n1k0/casperjs.git && \
 cd casperjs && \
 git checkout tags/1.0.0-RC4 && \
 sudo ln -sf `pwd`/bin/casperjs /usr/local/bin/casperjs

 cd ../fuel-web && \
 ./run_tests.sh --jslint && \
 sudo apt-get install -y fontconfig && \
 ./run_tests.sh --webui

 cd nailgun && \
 sudo npm install && \
 grunt bower

 ./manage.py syncdb && \
 ./manage.py loaddefault && \
 ./manage.py loaddata nailgun/fixtures/sample_environment.json && \
 grunt build --static-dir=static_compressed && \
 cd ../../fuel-main && \
 make test-integration

This initially failed until I installed yum, createrepo & multistrap

There are some failures during the nailgun unit tests - though these appear to be to do with removing nodes: so I'm assuming with an empty database this is to be expected.

The build fails reporting:

 mkdir -p /home/user/fuel-main/build/packages/rpm/RPMS/x86_64
 sudo sh -c "${SANDBOX_UP}"
 Starting SANDBOX up
 rpm: RPM should not be used directly install RPM packages, use Alien instead!
 rpm: However assuming you know what you are doing...
 rpm: no packages given for install
 centos-release already installed
 Rebuilding RPM DB
 Installing packages for Sandbox
 warning: Generating 12 missing index(es), please wait...
 Config time: 1.580
 Yum Version: 3.4.3
 Setting up Package Sacks
 mirror | 1.1 kB 00:00 ...
 mirror/primary | 208 B 00:00 ...
 rpmdb time: 0.002
 Setting up Install Process
 Setting up Package Sacks
 Checking for virtual provide or file-provide for bash
 Setting up Package Sacks
 No package bash available.
 Setting up Package Sacks
 Error: Nothing to do
 mount: mount point /home/user/fuel-main/build/packages/rpm/SANDBOX/proc does not exist
 mount: mount point /home/user/fuel-main/build/packages/rpm/SANDBOX/dev does not exist
 make: *** [/home/user/fuel-main/build/packages/rpm/fencing-agent.done] Error 32

I presume this must mean there is an issue with the repo? Though I'd appreciate any advice on how to fix this bug - I'm very keen to get involved in this project, and do not want to progress further toward building an iso until i can get this worked out.

Cheers

Pete

[Vladimir Kuklin (vkuklin)]

Hi, Pete, which environment do you actually want to setup?

The one for nailgun or the one for full ISO building? There is a difference because nailgun requires only python-related stuff and ISO building requires a lot more.

If you want to build ISO, then follow the following instructions:

http://docs.mirantis.com/fuel-dev/develop/env.html#building-the-fuel-iso

Also, be aware that you need to have kernel with version >= 3.8, so I suggest you to install latest Ubuntu 12.04.4 LTS update.

Feel free to ask any questions here, in openstack-dev mailing list with [Fuel] prefix in subject and in our fuel-dev IRC chat on freenode.irc.net

[Pete Birley (petebirley)]

Vladimir,

I was trying to set up the nailgun environment, before moving onto full iso building. The docs for nailgun do not make clear that the Fuel System Tests (http://docs.mirantis.com/fuel-dev/develop/nailgun/development/env.html#running-the-fuel-system-tests) require additional packages - I’m getting them now and seeing how things go.

Also in ubuntu 13.04, lxc-docker-0.10.0 is not available but should be installed using the following I believe:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9 && \
sudo sh -c "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list" && \
sudo apt-get update && \
sudo apt-get install -y linux-image-extra-`uname -r` && \
sudo apt-get install -y lxc-docker

What environment is preferred for development: The documentation refers to 13.04 but I’d prefer 12.04.4 if thats the way to go? I’m writing up all of my work - and would love to help contribute to the documentation once I’ve got a bit deeper.

Cheers

Pete

On 4 May 2014, at 17:58, Vladimir Kuklin <email address hidden> wrote:

> Hi, Pete, which environment do you actually want to setup?
>
> The one for nailgun or the one for full ISO building? There is a
> difference because nailgun requires only python-related stuff and ISO
> building requires a lot more.
>
> If you want to build ISO, then follow the following instructions:
>
> http://docs.mirantis.com/fuel-dev/develop/env.html#building-the-fuel-iso
>
> Also, be aware that you need to have kernel with version >= 3.8, so I
> suggest you to install latest Ubuntu 12.04.4 LTS update.
>
> Feel free to ask any questions here, in openstack-dev mailing list with
> [Fuel] prefix in subject and in our fuel-dev IRC chat on
> freenode.irc.net
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1315904
>
> Title:
> Building fails on Ubuntu 12.04 following developer documentation
>
> Status in Fuel: OpenStack installer that works:
> New
>
> Bug description:
> I am trying to set up a dev enviroment for Fuel/nailgun.
>
> I have have been follwing the documentation at
> (http://docs.mirantis.com/fuel-dev/) and from a clean install of
> ubuntu 13.04 (as guied buy the documentation) I performed the
> following commands: (I used Ubuntu server and access the box via ssh)
>
> sudo apt-get update && \
> sudo apt-get upgrade -y && \
> sudo apt-get install -y git && \
> sudo reboot # to load new kernel if required
>
> #following reboot
> git clone https://github.com/stackforge/fuel-main && \
> git clone https://github.com/stackforge/fuel-web && \
> git clone https://github.com/stackforge/fuel-astute && \
> git clone https://github.com/stackforge/fuel-ostf && \
> git clone https://github.com/stackforge/fuel-library && \
> git clone https://github.com/stackforge/fuel-docs
>
> sudo apt-get install -y postgresql postgresql-server-dev-9.1 && \
> sudo -u postgres createuser -SDRP nailgun # enter password 'nailgun'
>
> sudo -u postgres createdb nailgun && \
> sudo apt-get install -y python-dev python-pip python-yaml...

[Aleksandra Fedorova (bookwar)]

> What environment is preferred for development

We use 12.04.4 Ubuntu for iso builds now, slowly moving towards Ubuntu 14.04 but there are certain issues like this one https://bugs.launchpad.net/ubuntu/+source/multistrap/+bug/1313787

Thus Ubuntu 12.04 is recommended now. ( We probably need to update our docs so they will be consistent and use one LTS version of Ubuntu as a reference architecture )

As for the issue mentioned in the pot post, there is sandbox.mk file in fuel-main repo, which is responsible for the mounting sandbox folders. With a little bit of digging in there you can find a way to enter those SANDBOXes and run yum checks interactively: something like 'yum repolist' and 'yum search bash' from inside the sandbox will show if your centos mirror is configured properly.

If this doesn't help, please attach the full log of the build to the bug.

[Dmitry Pyzhov (dpyzhov)]

Actually, it should work on 13.x Ubuntu.
Sorry for unclear documentation. 'make test-integration' is big master task. It includes build of iso and running lots of system tests for several hours. So just skip it for now.
For ISO build use this instruction: http://docs.mirantis.com/fuel-dev/develop/env.html#building-the-fuel-iso

[Dmitry Pyzhov (dpyzhov)]

Could you attach full output of 'make iso' command if it fails, please? Looks like build is failed in your case, but we need more information for investigation.

[Mike Scherbakov (mihgen)]

Section http://docs.mirantis.com/fuel-dev/develop/nailgun/development/env.html#running-the-fuel-system-tests has to be moved from Nailgun to corresponding tests section.
I believe we need to create sections:
- System Tests (top level)
--- About
--- http://docs.mirantis.com/fuel-dev/devops.html
--- How to run Fuel system tests (content from http://docs.mirantis.com/fuel-dev/develop/nailgun/development/env.html#running-the-fuel-system-tests)

[Nikolay Markov (nmarkov)]

Created corresponding bug: https://bugs.launchpad.net/fuel/+bug/1321682

[Mike Scherbakov (mihgen)]

This bug hit many folks. Increasing this to High. Let's sort it out.

[Christopher Aedo (docaedo)]

I just created a completely clean Ubuntu 12.04 VM, and successfully built an ISO by executing the following (from the docs)

apt-get install git
sudo apt-get install git
git clone https://github.com/stackforge/fuel-main
cd fuel-main/
./prepare-build-env.sh
make iso

I'm going to close this bug, as the docs (and prepare-build-env.sh) are accurate. Please re-open the bug if you find an issue we can reproduce.

[Christopher Aedo (docaedo)]

Also to be clear, when this bug was initially noted, following the docs did NOT result in an environment which would build the ISO. Since then we've taken steps to update the docs and also created the "prepare-build-env.sh" script to automate that step, making it much easier to get the environment ready.

These efforts are part of this blueprint:
https://blueprints.launchpad.net/fuel/+spec/fuel-web-docs-dev-env-restructure