replace symlinks in bzr repo

[Jan Katins]

I try to checkout the repo on windows (in eclipse) and bzr has problems with symlinks and refuses to checkout the branch:

--
Executing: branch lp:igraph file:///c:/data/js/workspace/igraph

    Error while executing branch
    Unable to create symlink 'doc/book/devhelp.xsl' on this platform----
--

Would it be possible to replace symlinks with actual files?

[Gábor Csárdi (gabor.csardi)]

That would duplicate the files, which is not a very good solution.

But actually most symlinks are in doc/book, and we could remove that directory, it is outdated and not used, anyway. And then reorganize the other symlinks in doc/presentations.

[Tamás Nepusz (ntamas)]

Alternatively, we can leave out the symlinks from the repo and add them in the bootstrap.sh script, which we have to run after checkout anyway. People running bootstrap.sh on Wndows are likely to run it from Cygwin where there is a replacement mechanism for symlinks (AFAIK), and people not running bootstrap.sh at all on WIndows would then be responsible for copying the files manually if they need them.

[Jan Katins (jasc)]

Jikes... :-/ Actually I was planning to compile with MS compilers. In Cygwin symlinks shouldn't be a problem, so it's only for normal windows bzr users.

Is it actually possible to compile igraph+python in an MS Environment (and not cygwin)?

[Tamás Nepusz (ntamas)]

It is possible with a few caveats. The problem is that several files in the igraph source code are generated by GNU tools; typically the lexers and parsers of foreign graph formats are generated by bison. We do not store these generated C source files in the repository - we store the original input files instead. The consequence is that if you check out the repo, you won't have all the required C files in the repo, some of them still have to be generated. Our makefiles do that automatically, but since you plan to compile it using MS tools, this is not an option.

We usually do the following when we want to compile igraph using MS tools (which is required for the Windows version of the Python interface anyway):

1. We check out the repo in a GNU system, run ./bootstrap.sh, then run "make msvc". This generates a ZIP file that contains all the necessary stuff to compile igraph in the Microsoft Visual Studio - it even has a project file, which is usually slightly out-of-date, but we usually fix that before release ;)

2. We take the generated ZIP file, copy it to a Windows virtual machine, extract it, fire up VIsual Studio and then compile igraph there.

So, the bottom line is that it works but it requires some extra legwork and a working GNU environment.

[Gábor Csárdi (gabor.csardi)]

Alternatively we could provide an MSVC nightly download, that would be relatively easy IMHO. I am currently writing a script that creates an MSVC project file based on the unix Makefile, so we can wait until I finish that, and then add MSVC to the nightly builds.

But if you really want to follow the bzr repo, then the only solution is to switch to a unix-like system.

[Jan Katins (jasc)]

A MSVC donwload made by skripts would be nice, it would make compiling for python on windows much easier :-)

As I'm only interested in the python part, a description how to compile this on windows would be nice. I only have the free VS compiler, not the full VS GUI. I will look into how to compile it with this tools, lets see how that works...

[Jan Katins (jasc)]

Just for the record:
I installed cygwin and installed gcc, gcc-g++, make, flex, bison, libtool, libxml, gmp... in cygwin (one after another as diverse things failed... :-( )
ran bzr branch lp:igraph and ./bootstrap.sh
I had to insert an additional ./configure to get the makefile to appear
Running make msvc results will compile the whole thing it seems: could that be changed to only create the generated files? (or check this files into the repository so that one could get the without regenerating them...)

And then it stoped with this error:
--
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -I/usr/include/libxml2 -I../include -I../include -Wall -I../optional/glpk -g -O2 -I/usr/include/libxml2 -MT libigraph_la-matching.lo -MD -MP -MF .deps/libigraph_la-matching.Tpo -c matching.c -o libigraph_la-matching.o >/dev/null 2>&1
mv -f .deps/libigraph_la-matching.Tpo .deps/libigraph_la-matching.Plo
/bin/sh ../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -I/usr/include/libxml2 -I../include -I../include -Wall -I../optional/glpk -g -O2 -I/usr/include/libxml2 -MT libigraph_la-lad.lo -MD -MP -MF .deps/libigraph_la-lad.Tpo -c -o libigraph_la-lad.lo `test -f 'lad.c' || echo './'`lad.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I.. -g -O2 -I/usr/include/libxml2 -I../include -I../include -Wall -I../optional/glpk -g -O2 -I/usr/include/libxml2 -MT libigraph_la-lad.lo -MD -MP -MF .deps/libigraph_la-lad.Tpo -c lad.c -DDLL_EXPORT -DPIC -o .libs/libigraph_la-lad.o
In file included from lad.c:58:
igraph_interrupt_internal.h:43: warning: `thread' attribute directive ignored
lad.c: In function `igraph_i_lad_createGraph':
lad.c:769: error: prior parameter's size depends on 'nbU'
lad.c:769: error: prior parameter's size depends on 'nbV'
lad.c:795: error: prior parameter's size depends on 'nbU'
lad.c:795: error: prior parameter's size depends on 'nbV'
lad.c:795: error: prior parameter's size depends on 'nbV'
lad.c:795: error: prior parameter's size depends on 'nbU'
lad.c:786: confused by earlier errors, bailing out
Makefile:5169: recipe for target `libigraph_la-lad.lo' failed
make[3]: *** [libigraph_la-lad.lo] Error 1
make[3]: Leaving directory `/cygdrive/c/temp/igraph/src'
Makefile:1153: recipe for target `all' failed
make[2]: *** [all] Error 2
make[2]: Leaving directory `/cygdrive/c/temp/igraph/src'
Makefile:418: recipe for target `all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/cygdrive/c/temp/igraph'
Makefile:318: recipe for target `all' failed
make: *** [all] Error 2
--

Is that a bug?

[Gábor Csárdi (gabor.csardi)]

Yes, you need to do

./bootstrap.sh
./configure
make msvc

This is intended. A good point is that it would not be strictly needed to compile the files, it was just simpler to write the Makefile that way. But I can update that.

On a second note, I wrote the script to generate the project file, so we can add MSVC files to the nightly builder.

I'll look into the lad.c error.

Stay tuned!

[Jan Katins (jasc)]

Ok, tried again with the new github source (branch develop), but still the same error. Is there a newer source for the msvc download?

My goal is still the same: build igraph for python under windows

BTW: should a go with this bug to the issue list of github?

[Gábor Csárdi (gabor.csardi)]

We didn't really do anything about this, so no wonder that there is no change.

But just to be sure that we are on the same page. The released version of igraph can be easily built on MSVC, you just need to download the source zip file specifically created for MSVC. Is that not good? Do you really want the development version?

Bugs are still on Launchpad, it is somewhat a pain to transfer them to github, I'll do it at some point in the future.

[Jan Katins (jasc)]

What I want is to hack on igraph to include things like https://bugs.launchpad.net/bugs/1090728 and plots shown inline in ipython. :-)

right now I tried to build the msvs pack directly from source (error as above) and then compile via the msvc download. The last resulted right now in:

--
# In igraph-0.6.5-msvc
call "%VS90COMNTOOLS%"\\vsvars32.bat
vcbuild igraph.vcproj "release|win32"
# This succeeds. yay!
--
And the compile the python source:
--
# edit the path in setup.py
python setup.py bdist_wininst
--
This fails with
--
[...]
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I..\igraph-0.6.5-msvc\include -I../../build/include:../../include:/usr/local/include/igraph:/usr/include/igraph -IC:\portabel\Python27\include -IC:\portabel\Python27\PC /Tcsrc\vertexseqobject.c /Fobuild\temp.win-amd64-2.7\Release\src\vertexseqobject.obj
vertexseqobject.c
src\vertexseqobject.c(252) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'long', possible loss of data
src\vertexseqobject.c(354) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'long', possible loss of data
src\vertexseqobject.c(572) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'long', possible loss of data
src\vertexseqobject.c(594) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'long', possible loss of data
C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN\amd64\link.exe /DLL /nologo /INCREMENTAL:NO /LIBPATH:C:\data\external\igraph-msvc\igraph-0.6.5-msvc\Release /LIBPATH:../../build/src/.libs:../../src/.libs:/usr/local/lib:/usr/lib /LIBPATH:C:\portabel\Python27\libs /LIBPATH:C:\portabel\Python27\PCbuild\amd64 igraph.lib /EXPORT:init_igraph build\temp.win-amd64-2.7\Release\src\arpackobject.obj build\temp.win-amd64-2.7\Release\src\attributes.obj build\temp.win-amd64-2.7\Release\src\bfsiter.obj build\temp.win-amd64-2.7\Release\src\common.obj build\temp.win-amd64-2.7\Release\src\convert.obj build\temp.win-amd64-2.7\Release\src\edgeobject.obj build\temp.win-amd64-2.7\Release\src\edgeseqobject.obj build\temp.win-amd64-2.7\Release\src\error.obj build\temp.win-amd64-2.7\Release\src\filehandle.obj build\temp.win-amd64-2.7\Release\src\graphobject.obj build\temp.win-amd64-2.7\Release\src\igraphmodule.obj build\temp.win-amd64-2.7\Release\src\indexing.obj build\temp.win-amd64-2.7\Release\src\py2compat.obj build\temp.win-amd64-2.7\Release\src\pyhelpers.obj build\temp.win-amd64-2.7\Release\src\random.obj build\temp.win-amd64-2.7\Release\src\vertexobject.obj build\temp.win-amd64-2.7\Release\src\vertexseqobject.obj /OUT:build\lib.win-amd64-2.7\igraph\_igraph.pyd /IMPLIB:build\temp.win-amd64-2.7\Release\src\_igraph.lib /MANIFESTFILE:build\temp.win-amd64-2.7\Release\src\_igraph.pyd.manifest /MANIFEST
igraphmodule.obj : warning LNK4197: export 'init_igraph' specified multiple times; using first specification
   Creating library build\temp.win-amd64-2.7\Release\src\_igraph.lib and object build\temp.win-amd64-2.7\Release\src\_igraph.exp
arpackobject.obj : error LNK2019: unresolved external symbol igraph_arpack_options_init referenced in function igraphmodule_ARPACKOptions_new
graphobject.obj : error LNK2001: unresolv...

[Gábor Csárdi (gabor.csardi)]

I am not sure what you mean by hacking it, but if you just want to write python code as suggested in the other bug report you refer to, you don't need to recompile igraph at all.

If you want to modify the C code, then I suggest that you download the MSVC zip file of the current release, change the C code in that and then compile it. Probably you can just recompile the C library and then use it with the binary Python igraph installer. You just need to replace the library, that's all. Assuming you don't change the API, of course.

If you also change the C API, then you do need to recompile the Python extension as well, maybe Tamas can help you with that, he does it for the igraph releases.

But whichever of these you want, you don't need the development version of igraph.

[Jan Katins (jasc)]

Actually I just learned it that way: if you want to contribute: download source, change, build and test and finally submit a pull request.

Anyway, right now I can't build neither the real git source nor the msvc+Python source :-(

[Tamás Nepusz (ntamas)]

Hi Jan,

Sorry, I missed this thread in my inbox. Anyway, what you do with the Python interface seems to be okay for me. What did you write in setup.cfg? I have something like this:

[build_ext]
include_dirs=c:\dev\igraph\igraph-0.6.5-msvc\include
library_dirs=c:\dev\igraph\igraph-0.6.5-msvc\Release

And then I simply run python setup.py build bdist_msi and it works.

[Gábor Csárdi (gabor.csardi)]

Started removing the symbolic links:
https://github.com/igraph/igraph/commit/e004595b8844deee2cdddefa3d1897dc7fd7e8f2

[Gábor Csárdi (gabor.csardi)]

OK, final bit done in 6e8fdc5f14e20f6c8879d7b8b590a8f4bfedcaef.

[Gábor Csárdi (gabor.csardi)]

Since the bug itself is solved now, if you have other compilation issues, please open another bug(s) and link them to this one if it is related to something discussed here. Thanks!

[Gábor Csárdi (gabor.csardi)]

The development of igraph has moved to github, so please do not comment on this bug here. You are of course welcome to comment on github, here:
https://github.com/igraph/igraph/issues/303