/usr/bin/go-5 does not recursively go get packages

[Dave Cheney]

The version of the Go tool shipped with gccgo 5.0 does not correctly recursively download dependencies of a package

dfc@ubuntu:~$ dpkg -S $(which go-5)
gccgo-5: /usr/bin/go-5
dfc@ubuntu:~$ go get -u -v -d github.com/juju/juju/...
github.com/juju/juju (download)

This is wrong. The correct operation would be to inspect all the code in $GOPATH/src/github.com/juju/juju/ and recursively fetch the dependencies.

The proper operation should look like

lucky(~/devel/FlameGraph) % go get -u -v -d github.com/juju/juju/...
github.com/juju/juju (download)
code.google.com/p/go.crypto (download)
code.google.com/p/go.net (download)
github.com/coreos/go-systemd (download)
github.com/godbus/dbus (download)
github.com/dustin/go-humanize (download)
github.com/juju/blobstore (download)
github.com/juju/errors (download)
github.com/juju/loggo (download)
.. list continues for hundreds more lines

[Matthias Klose (doko)]

is this different than with golang 1.4.2? If yes, please could you file a bug report upstream? Also please recheck with the version in vivid.

[Dave Cheney (dave-cheney)]

Yes this is different to go 1.4.2.

Yes, I will file a bug upstream.

Yes, this was tested in a march 11 2015 vivid server install

On Mon, Mar 16, 2015 at 9:55 PM, Matthias Klose <email address hidden> wrote:
> is this different than with golang 1.4.2? If yes, please could you file
> a bug report upstream? Also please recheck with the version in vivid.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1432497
>
> Title:
> /usr/bin/go-5 does not recursively go get packages
>
> Status in gccgo-5 package in Ubuntu:
> New
>
> Bug description:
> The version of the Go tool shipped with gccgo 5.0 does not correctly
> recursively download dependencies of a package
>
> dfc@ubuntu:~$ dpkg -S $(which go-5)
> gccgo-5: /usr/bin/go-5
> dfc@ubuntu:~$ go get -u -v -d github.com/juju/juju/...
> github.com/juju/juju (download)
>
> This is wrong. The correct operation would be to inspect all the code
> in $GOPATH/src/github.com/juju/juju/ and recursively fetch the
> dependencies.
>
> The proper operation should look like
>
> lucky(~/devel/FlameGraph) % go get -u -v -d github.com/juju/juju/...
> github.com/juju/juju (download)
> code.google.com/p/go.crypto (download)
> code.google.com/p/go.net (download)
> github.com/coreos/go-systemd (download)
> github.com/godbus/dbus (download)
> github.com/dustin/go-humanize (download)
> github.com/juju/blobstore (download)
> github.com/juju/errors (download)
> github.com/juju/loggo (download)
> .. list continues for hundreds more lines
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/gccgo-5/+bug/1432497/+subscriptions

[Boger-e (boger-e)]

When using the go tool provided with gccgo in gcc 5.0, the use of 'go get' doesn't always find and build the dependent packages correctly.

Here is an example:

go get -u github.com/mitchellh/gox

There is a dependent package github.com/mitchellh/iochan but that never gets downloaded or built and as a result neither does github.com/mitchellh/gox. I can make it work by downloading them in separate steps this way:

go get -u github.com/mitchellh/uchan
go get -u github.com/mitchellh/gox

After further debugging I found the problem has to do with the code in libgo/go/cmd/go/pkg.go in the load function where it does this check:

....

  p1 := loadImport(path, p.Dir, stk, p.build.ImportPos[path])
  if !reqPkgSrc && p1.Root == "" {
    continue
  }

....

When it tries to load the github.com/mitchellh/uchan package, p1.Root == "" so it skips the rest of the code in the loop that loads the package. Isn't this a case where p1.Root should have a non-null string and therefore not continue/skip the rest of the body of the loop and get it loaded?

[Dave Cheney (dave-cheney)]

After a lot of assing about today I have been able to verify that the
problem exists in the upstream gccgo branch (from the gcc svn server).
This isn't a bug in out packaging.

That said, I have no idea what the bug is yet, but at least I know
where to look.

On Mon, Mar 16, 2015 at 10:28 PM, David Cheney
<email address hidden> wrote:
> Yes this is different to go 1.4.2.
>
> Yes, I will file a bug upstream.
>
> Yes, this was tested in a march 11 2015 vivid server install
>
> On Mon, Mar 16, 2015 at 9:55 PM, Matthias Klose <email address hidden> wrote:
>> is this different than with golang 1.4.2? If yes, please could you file
>> a bug report upstream? Also please recheck with the version in vivid.
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1432497
>>
>> Title:
>> /usr/bin/go-5 does not recursively go get packages
>>
>> Status in gccgo-5 package in Ubuntu:
>> New
>>
>> Bug description:
>> The version of the Go tool shipped with gccgo 5.0 does not correctly
>> recursively download dependencies of a package
>>
>> dfc@ubuntu:~$ dpkg -S $(which go-5)
>> gccgo-5: /usr/bin/go-5
>> dfc@ubuntu:~$ go get -u -v -d github.com/juju/juju/...
>> github.com/juju/juju (download)
>>
>> This is wrong. The correct operation would be to inspect all the code
>> in $GOPATH/src/github.com/juju/juju/ and recursively fetch the
>> dependencies.
>>
>> The proper operation should look like
>>
>> lucky(~/devel/FlameGraph) % go get -u -v -d github.com/juju/juju/...
>> github.com/juju/juju (download)
>> code.google.com/p/go.crypto (download)
>> code.google.com/p/go.net (download)
>> github.com/coreos/go-systemd (download)
>> github.com/godbus/dbus (download)
>> github.com/dustin/go-humanize (download)
>> github.com/juju/blobstore (download)
>> github.com/juju/errors (download)
>> github.com/juju/loggo (download)
>> .. list continues for hundreds more lines
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/ubuntu/+source/gccgo-5/+bug/1432497/+subscriptions

[Boger-e (boger-e)]

Created attachment 35077
Fix dependencies in go tool for gccgo

The code to determine when to import packages was not quite catching the cases it should. It really needs to know what packages are part of the GO standard packages and not try to load those -- if they aren't then they should be loaded.

This makes a few changes to the previous:
- change the name of reqPkgSrc to reqStdPkgSrc to indicate when we should require the source for GO standard packages when loading packages.
- generate the list of standard packages found in libgo and make that available to build into the go tool
- set the flag Standard in the Package structure correctly for gccgo, based on the list of expected libgo standard packages
- use the reqStdPkgSrc flag and the package.Standard flag to determine if the load of the package should be attempted

[Boger-e (boger-e)]

Created attachment 35081
Updated patch to add unsafe to the list of std package names

[Ian Lance Taylor (ianlancetaylor)]

Can you send the libgo part of this patch through the codereview site?

[Boger-e (boger-e)]

Do you mean as if submitting to gofrontend-dev?

[Ian Lance Taylor (ianlancetaylor)]

Yes, I did mean for gofrontend-dev.

Or if you want to submit to the master sources, that is even better; guidelines are at http://golang.org/doc/contribute.html .

[Boger-e (boger-e)]

I decided that you meant the gofrontend so here it is (just did the hg mail)

https://codereview.appspot.com/213570043/

[Dave Cheney (dave-cheney)]

Upstream issue https://github.com/golang/go/issues/10217

[Matthias Klose (doko)]

you are testing with a very old version 5.0.0 20150116, which even doesn't have the final Go 1.4.2 release pulled it. Please make sure to test with recent versions. The distro packages can be used for that, they don't have any specific Go patches.

Please recheck.

[I-ian-1 (i-ian-1)]

Author: ian
Date: Tue Mar 24 19:50:31 2015
New Revision: 221643

URL: https://gcc.gnu.org/viewcvs?rev=221643&root=gcc&view=rev
Log:
 PR go/65462
cmd: Fix dependencies for 'go get' with gccgo

Problem described in GCC BZ 65462.
Generate the list of the standard GO package names based on what was built into libgo in the libgo Makefile.
Change the var name from reqPkgSrc to reqStdPkgSrc to clarify it only affects standard GO packages.
Skip the attempted loading of a package only if it is a standard GO package and the flag is set indicating its source is not required to be available.
This requires a corresponding change to gotools to build and link in the new file containing the list of standard GO package names that was generated by the libgo Makefile.

gotools/:
 PR go/65462
 * Makefile.am (go_cmd_go_files): Add $(libgodir)/zstdpkglist.go.
 * Makefile.in: Rebuild.

Modified:
    trunk/gotools/ChangeLog
    trunk/gotools/Makefile.am
    trunk/gotools/Makefile.in
    trunk/libgo/Makefile.am
    trunk/libgo/Makefile.in
    trunk/libgo/go/cmd/go/build.go
    trunk/libgo/go/cmd/go/pkg.go
    trunk/libgo/go/cmd/go/test.go

[Ian Lance Taylor (ianlancetaylor)]

Fixed now, I hope.

[Dave Cheney (dave-cheney)]

I have confirmed that the bug was fixed upstream with revision

https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=221643

@doko, is it possible to get this patch into V ?

On Wed, Mar 25, 2015 at 10:32 AM, Bug Watch Updater
<email address hidden> wrote:
> Launchpad has imported 9 comments from the remote bug at
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65462.
>
> If you reply to an imported comment from within Launchpad, your comment
> will be sent to the remote bug automatically. Read more about
> Launchpad's inter-bugtracker facilities at
> https://help.launchpad.net/InterBugTracking.
>
> ------------------------------------------------------------------------
> On 2015-03-18T19:37:59+00:00 Boger-e wrote:
>
> When using the go tool provided with gccgo in gcc 5.0, the use of 'go
> get' doesn't always find and build the dependent packages correctly.
>
> Here is an example:
>
> go get -u github.com/mitchellh/gox
>
> There is a dependent package github.com/mitchellh/iochan but that never
> gets downloaded or built and as a result neither does
> github.com/mitchellh/gox. I can make it work by downloading them in
> separate steps this way:
>
> go get -u github.com/mitchellh/uchan
> go get -u github.com/mitchellh/gox
>
>
> After further debugging I found the problem has to do with the code in libgo/go/cmd/go/pkg.go in the load function where it does this check:
>
> ....
>
>
> p1 := loadImport(path, p.Dir, stk, p.build.ImportPos[path])
> if !reqPkgSrc && p1.Root == "" {
> continue
> }
>
> ....
>
>
> When it tries to load the github.com/mitchellh/uchan package, p1.Root == "" so it skips the rest of the code in the loop that loads the package. Isn't this a case where p1.Root should have a non-null string and therefore not continue/skip the rest of the body of the loop and get it loaded?
>
> Reply at:
> https://bugs.launchpad.net/ubuntu/+source/gccgo-5/+bug/1432497/comments/3
>
> ------------------------------------------------------------------------
> On 2015-03-20T16:58:23+00:00 Boger-e wrote:
>
> Created attachment 35077
> Fix dependencies in go tool for gccgo
>
> The code to determine when to import packages was not quite catching the
> cases it should. It really needs to know what packages are part of the
> GO standard packages and not try to load those -- if they aren't then
> they should be loaded.
>
> This makes a few changes to the previous:
> - change the name of reqPkgSrc to reqStdPkgSrc to indicate when we should require the source for GO standard packages when loading packages.
> - generate the list of standard packages found in libgo and make that available to build into the go tool
> - set the flag Standard in the Package structure correctly for gccgo, based on the list of expected libgo standard packages
> - use the reqStdPkgSrc flag and the package.Standard flag to determine if the load of the package should be attempted
>
> Reply at:
> https://bugs.launchpad.net/ubuntu/+source/gccgo-5/+bug/1432497/comments/5
>
> ------------------------------------------------------------------------
> On 2015-03-20T20:08:18+00:00 Boger-e wrote:
>
> Created attachment 35081
> Updated patch to add unsafe to the list of st...

[Launchpad Janitor (janitor)]

This bug was fixed in the package gccgo-5 - 5-20150321-1ubuntu4

---------------
gccgo-5 (5-20150321-1ubuntu4) vivid; urgency=medium

  * Fix PR go/65587, complete support for 32-bit PPC relocations.
 -- Matthias Klose <email address hidden> Thu, 26 Mar 2015 19:30:10 +0100