ppc64_cpu --frequency fails with sched_setaffinity: Invalid argument

[bugproxy]

---Problem Description---
ppc64_cpu --frequency does not work in Ubuntu 15.04

---uname output---
Linux lep8d 3.18.0-11-generic #12-Ubuntu SMP Fri Jan 23 22:45:52 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux

Machine Type = P8

---Steps to Reproduce---
Install Ubuntu 15.04 DVD on a local hard disk. Then execute the below command to determine the cpu frequency.

root@lep8d:~# ppc64_cpu --frequency
sched_setaffinity: Invalid argument
root@lep8d:~# echo $?
1

root@lep8d:~# lsmcode
Version of System Firmware is FW830.00 (TV830_015) (t) FW830.00 (TV830_014) (p) FW830.00 (TV830_015) (b)
root@lep8d:~# update_flash -d
Current firwmare version :
  P side : FW830.00 (TV830_014)
  T side : FW830.00 (TV830_015)
  Boot side : FW830.00 (TV830_015)
root@lep8d:~#

> The behavior is not consistent. Some times the command fails while in other
> cases returns success and displays proper information
>
> Even with 3.18.0-12-generic I can recreate this failure.
>
> root@lop824:/home/sachin/dlpar# ppc64_cpu --frequency
> sched_setaffinity: Invalid argument
> sched_setaffinity: Invalid argument
> root@lop824:/home/sachin/dlpar# uname -a
> Linux lop824 3.18.0-12-generic #13-Ubuntu SMP Thu Jan 29 13:44:26 UTC 2015
> ppc64le ppc64le ppc64le GNU/Linux
> root@lop824:/home/sachin/dlpar#
>
> The other behavior I have observed is first time the command display error
> message twice. On subsequent invocation the same message is displayed only
> once.
>
> root@lop824:/home/sachin/dlpar# ppc64_cpu --frequency
> sched_setaffinity: Invalid argument
> root@lop824:/home/sachin/dlpar# ppc64_cpu --frequency
> sched_setaffinity: Invalid argument
> root@lop824:/home/sachin/dlpar# ppc64_cpu --frequency
> sched_setaffinity: Invalid argument
> root@lop824:/home/sachin/dlpar# ppc64_cpu --frequency
> sched_setaffinity: Invalid argument
> root@lop824:/home/sachin/dlpar

I ran in a tight loop and didn't see this once. What else is going on when you are doing this?

Can you, next time, run under strace and save a log of the output (strace -f -o log ppc64_cpu --frequency)?

-Nish

I ran few CPU hotplug tests before executing this command.

> Can you, next time, run under strace and save a log of the output (strace -f
> -o log ppc64_cpu --frequency)?

I have already attached strace log in comment #3.

I have captured the log again and have attached here.

root@lep8d:~# strace -f -o strace-ppc64_cpu.log ppc64_cpu --frequency
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument
sched_setaffinity: Invalid argument

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2015-02-24 16:47 EDT-------
Relevant bits of ppc64_cpu:

static void *soak(void *arg)
{
unsigned int cpu = (long)arg;
cpu_set_t cpumask;
CPU_ZERO(&cpumask);
CPU_SET(cpu, &cpumask);
if (sched_setaffinity(0, sizeof(cpumask), &cpumask)) {
perror("sched_setaffinity");
exit(1);
}
while (1)
; /* Do Nothing */
}

/* Start a soak thread on each CPU */
for (i = 0; i < threads_in_system; i++) {
pthread_t tid;
if (!cpu_online(i)) {
cpu_freq[i] = CPU_OFFLINE;
continue;
}
if (pthread_create(&tid, NULL, soak, (void *)(long)i)) {
perror("pthread_create");
return -1;
}
}

So if cpus are offline, they should be ignored.

Are you doing CPU hotplug *while* running this script? Seems like that could easily break things (no locking that I can see).

Nathan, since this is running in a pthread, and there is error checking later:

if (cpu_freq[i] == CPU_OFFLINE)
continue;
/* No result - Couldn't schedule on that cpu */
if (cpu_freq[i] == 0) {
printf("WARNING: couldn't run on cpu %d\n", i);
continue;
}

can soak() just return if sched_setaffinity fails rather than exit()'ing?

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

------- Comment on attachment From <email address hidden> 2015-05-13 22:48 EDT-------

The pthread handling for checking the cpu frequency in the ppc64_cpu
command does not properly handle when soak threads exit from a failed
sched_setaffinity call.

We need to update the number of threads handled to be dynamically set,
instead of define MAX_NR_CPUS which is out of date, and check for
threads that have exited before reading the perf counters.