Linaro Android Infrastructure

Android builds failing to repo sync after git .linaro.org migration

Bug #1179406 reported by Paul Sokolovsky on 2013-05-13

This bug affects 1 person

Affects		Status	Importance	Assigned to	Milestone
	Linaro Android Infrastructure	Fix Released	Critical	Paul Sokolovsky	Linaro Android Infrastructure 2013.05

Bug Description

Vishal Bhoj wrote:

> Android syncs the code using git over http. Recently the Android
> builds are not able to sync the code and I think its related to the
> configuration in the new git.linaro.org . Here is the error:
>
> curl: (60) SSL certificate problem, verify that the CA cert is OK.
> Details: error:14090086:SSL
> routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
> More details here: http://curl.haxx.se/docs/sslcerts.html
>
> curl performs SSL certificate verification by default, using a
> "bundle" of Certificate Authority (CA) public keys (CA certs). If the
> default bundle file isn't adequate, you can specify an alternate file
> using the --cacert option.
> If this HTTPS server uses a certificate signed by a CA represented in
> the bundle, the certificate verification probably failed due to a
> problem with the certificate (it might be expired, or the name might
> not match the domain name in the URL).
> If you'd like to turn off curl's verification of the certificate, use
> the -k (or --insecure) option.
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/people/kejunzhou/tools_for_pm-qa.git/info/refs
>
> fatal: HTTP request failed
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/people/tomgall/libjpeg-turbo/libjpeg-turbo.git/info/refs
>
> fatal: HTTP request failed
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/tools/powertop-2.0.git/info/refs
>
> fatal: HTTP request failed
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/tools/pm-qa.git/info/refs
>
> fatal: HTTP request failed
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/tools/powerdebug.git/info/refs
>
> fatal: HTTP request failed
> error: server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none while accessing
> http://git.linaro.org/git-ro/people/kejunzhou/tools_for_pm-qa.git/info/refs
>
> fatal: HTTP request failed
>
>
> Here is the link to the complete log:
> https://android-build.linaro.org/jenkins/job/linaro-android-member-hisilicon-restricted_hi4511/7/console

Revision history for this message

Paul Sokolovsky (pfalcon) wrote on 2013-05-13:

This affects most Android builds - they are failing with NOT_BUILT status after ~20min.

Changed in linaro-android-infrastructure:
importance:	Undecided → Critical
assignee:	nobody → Paul Sokolovsky (pfalcon)
milestone:	none → 2013.05
status:	New → Confirmed

Revision history for this message

Paul Sokolovsky (pfalcon) wrote on 2013-05-13:

What happens here is that http://git.linaro.org forces redirect to https://

wget doesn't complain about https://git.linaro.org, but curl does, as we see.

Asking ITS to change remove forced redirect for normal repos.

Revision history for this message

Paul Sokolovsky (pfalcon) wrote on 2013-05-13:

I cannot reproduce issue as of now on a build slave:

ubuntu@ip-10-38-113-42:~$ curl -L http://git.linaro.org/git-ro/tools/powertop-2.0.git/info/refs
439cc8e4c0080f7487510b0309afbf853e3fb7a0 refs/heads/linaro
a322b28a9580b245c29c128a425c320d34c78821 refs/heads/linaro-2012.12
9ad1994a5f076976fcc3dc249abad964fe1b2944 refs/heads/linaro-dev
537ed08e18c1f8667e19bc3abfc827e7d5e720df refs/heads/master
ubuntu@ip-10-38-113-42:~$ curl http://git.linaro.org/git-ro/tools/powertop-2.0.git/info/refs
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://git.linaro.org/git-ro/tools/powertop-2.0.git/info/refs">here</a>.</p>
<hr>
<address>Apache/2.2.22 (Ubuntu) Server at git.linaro.org Port 80</address>
</body></html>

Revision history for this message

Paul Sokolovsky (pfalcon) wrote on 2013-05-13:

Ok, Philip turned off redirect, but we're still experiencing random failures trying to clone some repos:

<pfalcon> pfalcon@x34f:~/tmp$ git clone http://git.linaro.org/git-ro/people/tomgall/libjpeg-turbo/libjpeg-turbo.git
<pfalcon> Cloning into 'libjpeg-turbo'...
<pfalcon> WARNING: gnome-keyring:: couldn't connect to: /tmp/keyring-Jho9e7/pkcs11: No such file or directory
<pfalcon> error: Unable to find 95ed21e84965f859da0792558742f9cbf9d4ac7a under http://git.linaro.org/git-ro/people/tomgall/libjpeg-turbo/libjpeg-turbo.git
<pfalcon> Cannot obtain needed object 95ed21e84965f859da0792558742f9cbf9d4ac7a
<pfalcon> while processing commit 52969c9b2cf9db0162bc1c1e9b9f27bd362908ba.
<pfalcon> error: Fetch failed.
<pfalcon> pfalcon@x34f:~/tmp$ git clone http://git.linaro.org/git-ro/people/tomgall/libjpeg-turbo/libjpeg-turbo.git
<pfalcon> Cloning into 'libjpeg-turbo'...
<pfalcon> WARNING: gnome-keyring:: couldn't connect to: /tmp/keyring-Jho9e7/pkcs11: No such file or directory
<pfalcon> error: Unable to find 52969c9b2cf9db0162bc1c1e9b9f27bd362908ba under http://git.linaro.org/git-ro/people/tomgall/libjpeg-turbo/libjpeg-turbo.git
<pfalcon> Cannot obtain needed object 52969c9b2cf9db0162bc1c1e9b9f27bd362908ba
<pfalcon> while processing commit 8413b888ffb5b86ad5143fbfb6957e54e50f807b.
<pfalcon> error: Fetch failed.
<pfalcon> etc