Failing EPICS 7 libCom tests on VxWorks 6.7 and 6.9

[Dirk Zimoch]

epicsThreadTest

# System has 1 CPUs
ok 1 - ncpus > 0
# main() thread 0x581810
not ok 2 - Join delayed parent (0 seconds) # TODO Thread join doesn't work
ok 3 - Join tests #1 completed
not ok 4 - Join delayed parent (0 seconds) # TODO Thread join doesn't work
ok 5 - Join tests #2 completed
not ok 12 - infoB.didSomething
not ok 13 - infoA.didSomething
not ok 14 - threadA epicsThreadIsOkToBlock() = 88
not ok 15 - threadB epicsThreadIsOkToBlock() = 7001716

    Results
    =======
       Tests: 15
      Passed: 11 = 73.33%
      Failed: 4 = 26.67%

Behaves differently when called from :

# System has 1 CPUs
ok 1 - ncpus > 0
# main() thread 0x539d10
not ok 2 - Join delayed parent (0 seconds) # TODO Thread join doesn't work
ok 3 - Join tests #1 completed
not ok 4 - Join delayed parent (0 seconds) # TODO Thread join doesn't work
ok 5 - Join tests #2 completed
ok 6 - pget == pset
ok 7 - thread.getPriority() == epicsThreadGetPriority(self)
ok 8 - pget == pset
ok 9 - thread.getPriority() == epicsThreadGetPriority(self)
ok 10 - pget == pset
ok 11 - thread.getPriority() == epicsThreadGetPriority(self)
not ok 12 - infoB.didSomething
not ok 13 - infoA.didSomething
ok 14 - threadA epicsThreadIsOkToBlock() = 0

Planned 15 tests but only ran 14

    Results
    =======
       Tests: 14
      Passed: 12 = 85.71%
      Failed: 2 = 14.29%
ok 15 - threadB epicsThreadIsOkToBlock() = 1

Test results are counted before the test completes!
Why do test 14 and 15 only succeed when called from epicsRunLibComTests but fail when running epicsThreadTest directly? Because the test passes a pointer to a local variable which already went out of scope before being used (because epicsThreadMustJoin did not work).

--------------

epicsEventTest (TODO)

not ok 16 - epicsEventWaitWithTimeout(0.125000) delay error -0.008332 sec # TODO Known issue with delay calculation
not ok 17 - epicsEventWaitWithTimeout(0.062500) delay error -0.012501 sec # TODO Known issue with delay calculation
not ok 18 - epicsEventWaitWithTimeout(0.031250) delay error -0.014585 sec # TODO Known issue with delay calculation

    Results
    =======
       Tests: 37
      Passed: 37 = 100.00%
 Todo Passes: 19 = 51.35%

--------------

epicsSockResolveTest

not ok 26 - aToIPAddr("127.0.0.test", 4000) -> 0
# IP=0x7f000000, port=4000
not ok 27 - aToIPAddr("127.0.0.test:42", 4000) -> 0
# IP=0x7f000000, port=42
not ok 30 - aToIPAddr("1.2.3.4.5", 4000) -> 0
# IP=0x1020304, port=4000
not ok 31 - aToIPAddr("1.2.3.4.5:6", 4000) -> 0
# IP=0x1020304, port=6

    Results
    =======
       Tests: 37
      Passed: 33 = 89.19%
      Failed: 4 = 10.81%

This fails because the VxWorks 6.7 implementation of hostGetByName happily reads rubbish as the 4th numeric component as 0 and ignores any further components. Fixed in vxWorks 6.9.

--------------

epicsStdioTest (not really a bug: r/o NFS mount)

not ok 152 - (stream = fopen(report, "w")) != NULL
# 'report' could not be opened for writing: S_nfsLib_NFSERR_ROFS
ok 153 # SKIP Can't create stream file
ok 154 # SKIP Can't create stream file
ok 155 # SKIP Can't create stream file
ok 156 # SKIP Can't create stream file
ok 157 # SKIP Can't create stream file
ok 158 # SKIP Can't create stream file
ok 159 # SKIP Can't create stream file
ok 160 # SKIP Can't create stream file
ok 161 # SKIP Can't create stream file
ok 162 # SKIP Can't create stream file
ok 163 # SKIP Can't create stream file

    Results
    =======
       Tests: 163
      Passed: 162 = 99.39%
      Failed: 1 = 0.61%
     Skipped: 11 = 6.75%

--------------

macDefExpandTest

# Got "", expected "BAR".

not ok 27 - ${=BAR}
# Got "xy", expected "xBARy".

not ok 28 - x${=BAR}y

    Results
    =======
       Tests: 97
      Passed: 95 = 97.94%
      Failed: 2 = 2.06%

This is a result of previous epicsEnvUnset calls because vxWorks has no way to unset enviroment variables and thus the implementation deletes the name. This effectively creates a variable with empty name and thus ${} and ${=BAR} return an empty string.

Running this test a second time fails more often because some environment variables are now defined, e.g.
not ok 77 - Macro F is 6, expected undefined

    Results
    =======
       Tests: 97
      Passed: 79 = 81.44%
      Failed: 18 = 18.56%

[Andrew Johnson (anj)]

I don't have access to any VxWorks 6.x installations earlier than 6.8 and I don't generally run the EPICS tests on that because we don't use it any more here at APS. PSI is the only EPICS site that I know of which is actively working with VxWorks < 6.9 now.

Re epicsStdioTest: You may be able to get this test to pass if you use ftp to boot and your ftp server has access to a writable directory. I have an 'incoming' directory which is world-writable and I can cd to "server:/path/to/incoming" before running the test to get that it to pass.

Re macDefExpandTest: Thanks for that explanation, that makes sense (I was seeing these failures too). I can modify macCore.c's lookup() routine to check for a NULL or empty name before it calls getenv() and return NULL in that case — that's a 1-line change:

--- a/modules/libcom/src/macLib/macCore.c
+++ b/modules/libcom/src/macLib/macCore.c
@@ -588,7 +588,7 @@ static MAC_ENTRY *lookup( MAC_HANDLE *handle, const char *name, int special )
     }
     if ( (special == FALSE) && (entry == NULL) &&
          (handle->flags & FLAG_USE_ENVIRONMENT) ) {
- char *value = getenv(name);
+ char *value = name && *name ? getenv(name) : NULL;
         if (value) {
             entry = create( handle, name, FALSE );
             if ( entry ) {

It looks like we shouldn't be passing either into getenv() anyway, so this is a legitimate change for the other targets too. I just committed that change along with my fix to add redirect support for vprintf() to epicsStdio.h which fixed those failures.

[Dirk Zimoch (dirk.zimoch)]

epicsStdioTest: This one succeeds when running in a writable directory. However:
Ran 165 tests but only planned for 163!

Looking for more test count mismatch...
epicsMessageQueueTest: Planned 74 tests but only ran 0
epicsMutexTest: Ran 51 tests but only planned for 20!
epicsSpinTest: Ran 6 tests but only planned for 2!
epicsStringTest: Ran 431 tests but only planned for 427!
epicsThreadOnceTest: Ran 12 tests but only planned for 11!
Amazing.

[Dirk Zimoch (dirk.zimoch)]

epicsErrlogTest: This tests prints to stderr "errlog: lost zu messages"

The "zu" originates in the fact that the vxWorks version printf() does not understand the format z modifier (for size_t). Windows may show similar behavior as it uses I for the same purpose.

[Dirk Zimoch (dirk.zimoch)]

This tests fails only on vxWorks 6.9 but not on vxWorks 6.7.

osiSockTest:
# udpSockFanoutTest()
# Interface 172.20.255.255:5064
# Not LO
# RX1 start
# RX2 start
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# RX ignore n=32 cmd=0 size=0 dtype=1 dcnt=13 body=
# Result: RX1 0:0 RX2 0:0
ok 19 - Found non-loopback interface
not ok 20 - Successes 0

VxWorks 6.7 shows:
ok 20 - Successes 1

[Dirk Zimoch (dirk.zimoch)]

The osiSockTest looks like the IOC cannot hear its own transmissions (SIMPLEX). I have confirmed with tcpdump on a different machine that the test sends the correct UDP messages and it obviously receives messages from other IOCs in the same network.
However, running VxWorks 6.7, the ifconfig tool shows:
fei0 Link type:Ethernet HWaddr 00:01:af:30:3e:ea Queue:none
 inet 172.20.5.11 mask 255.255.0.0 broadcast 172.20.255.255
 UP RUNNING SIMPLEX BROADCAST MULTICAST
 MTU:1500 metric:1 VR:0 ifindex:2
 RX packets:3836 mcast:2505 errors:0 dropped:0
 TX packets:549 mcast:0 errors:0
 collisions:0 unsupported proto:0
 RX bytes:2271k TX bytes:49k
while on vxWorks 6.9 it shows:
fei0 Link type:Ethernet HWaddr 00:01:af:30:3e:ea Queue:none
 inet 172.20.5.11 mask 255.255.0.0 broadcast 172.20.255.255
 UP RUNNING BROADCAST MULTICAST
 MTU:1500 metric:1 VR:0 ifindex:2
 RX packets:2424 mcast:0 errors:0 dropped:0
 TX packets:379 mcast:0 errors:0
 collisions:0 unsupported proto:0
 RX bytes:1936k TX bytes:40k

Exactly the opposite SIMPLEX flags settings I would expect from the behavior. On the othe hand VxWorks often behaves unexpectedly