optimization of a recursive function cause segmentation fault during execution (in very rare cases)

[Shkarnikov Sergey]

OS: Ubuntu 12.04.3 LTS
gcc version: 4.6.3-1ubuntu5

It's a modificated test case from gcc-4.6.3 testsuite (20000412-2.c):

int f(int a){
  int x = (( & a)[(((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a % 10))) % 10))) * 314160879)]);
 if (x==0)
  return 1;
  return f(x-1);
}

int main(int argc,char **argv)
{
  if (f (2) != 1)
    abort ();
  exit (0);
}

"((((( a % 10) * ( a % 10)) % 10) > (((10 - ( a % 10)) * (10 - ( a % 10))) % 10)))" is a predicate and always equal to 0. So "x" is actually equal to "a".

As you can see this test should exit successfully, but if it was compiled with optimization (O2 and higher, O1 works fine) segmentation fault happens during execution.