openWNS UnitTest

Test suites disabled and not found

Bug #610400 reported by Re Hova Andre Rove
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
openWNS UnitTest
New
Undecided
Unassigned

Bug Description

Hi All,

I want to require about the unit tests that have been disabled in my OpenWNS installation.
I install OpenWNS with opt option and later run ./playground.py runtest.

As you can see below, I failed 8 test because two of the required suites are disabled.
Is there a solution for this?

-----------------------------------------------------------------------------------------------------------------------
Ran 15743 tests in 1720.084s

FAILED (failures=8)
Warning: You had 2 suites disabled.

         TestSuite: /home/sly/OpenWNS/myOpenWNS/tests/system/tcp-tests
         Reason: Currently no applications available

         TestSuite: /home/sly/OpenWNS/myOpenWNS/tests/system/wimac-tests/configBypass
         Reason: Problem with multiple CIDs for QoS.
----------------------------------------------------------------------------------------------------------------------

Revision history for this message
Maciej Muehleisen (mue-comnets) wrote : Re: [openWNS-develop] [Bug 610400] [NEW] Test suites disabled and not found

Hi,

a disabled TestSuite is not an error. The actual error messages should
be visible above the lines you have pasted here. Please be sure you have
a current openWNS version by typing ./playground.py missing. If there
are available updates do a ./playground.py upgrade and then
./playground.py install and ./playground.py install --flavour=opt. Then
run the tests again.

Greats,

Mac

Re Hova Andre Rove wrote:
> Public bug reported:
>
> Hi All,
>
> I want to require about the unit tests that have been disabled in my OpenWNS installation.
> I install OpenWNS with opt option and later run ./playground.py runtest.
>
> As you can see below, I failed 8 test because two of the required suites are disabled.
> Is there a solution for this?
>
> -----------------------------------------------------------------------------------------------------------------------
> Ran 15743 tests in 1720.084s
>
> FAILED (failures=8)
> Warning: You had 2 suites disabled.
>
> TestSuite: /home/sly/OpenWNS/myOpenWNS/tests/system/tcp-tests
> Reason: Currently no applications available
>
> TestSuite: /home/sly/OpenWNS/myOpenWNS/tests/system/wimac-tests/configBypass
> Reason: Problem with multiple CIDs for QoS.
> ----------------------------------------------------------------------------------------------------------------------
>
> ** Affects: openwns-unittest
> Importance: Undecided
> Status: New
>
>

Revision history for this message
Re Hova Andre Rove (reandre8503) wrote :
Download full text (6.1 KiB)

Hi Mac,

I have already tried ./playground.py missing but there is no missing packages and there are no new items to upgrade.
I tried again ./playground.py runtests and the failures are:

======================================================================
FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs output_dbg_configITUUMa.py/MinPathloss_PDF.dat: variance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
    self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
AssertionError:
Maximum relative error of 0.06 exceeded.
Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)

======================================================================
FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MaxRxPower_PDF.dat vs output_dbg_configITUUMa.py/MaxRxPower_PDF.dat: variance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
    self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
AssertionError:
Maximum relative error of 0.06 exceeded.
Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)

======================================================================
FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs output_opt_configITUUMa.py/MinPathloss_PDF.dat: variance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
    self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
AssertionError:
Maximum relative error of 0.06 exceeded.
Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)

======================================================================
FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MaxRxPower_PDF.dat vs output_opt_configITUUMa.py/MaxRxPower_PDF.dat: variance
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
    self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
AssertionError:
Maximum relative error of 0.06 exceeded.
Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)

======================================================================
FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUSMa.py/MinPathloss_PDF.dat vs output_dbg_configITUSMa.py/MinPathloss_PDF.dat: variance
--------------------------------------------...

Read more...

Revision history for this message
Maciej Muehleisen (mue-comnets) wrote : Re: [openWNS-bugsquad] [Bug 610400] Re: Test suites disabled and not found
Download full text (6.9 KiB)

Hi,

I have already seen this kind of error. Looks like different hardware
architectures produce different results because of computational
accuracy. What does "cat /etc/lsb-release" tell you (Linux version), is
it a 32bit or 64bit installation?

The inaccuracy only seems to affect the IMT-Advanced channel models. If
you don't plan to use them, there is no problem at all. If you depend on
the IMT-Advanced channel models please tell me and I provide some
reference results from the WiMAX IMTA tests to check how far your system
is off.

Greats,

Mac

Re Hova Andre Rove wrote:
> Hi Mac,
>
> I have already tried ./playground.py missing but there is no missing packages and there are no new items to upgrade.
> I tried again ./playground.py runtests and the failures are:
>
> ======================================================================
> FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs output_dbg_configITUUMa.py/MinPathloss_PDF.dat: variance
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
> self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
> AssertionError:
> Maximum relative error of 0.06 exceeded.
> Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)
>
> ======================================================================
> FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MaxRxPower_PDF.dat vs output_dbg_configITUUMa.py/MaxRxPower_PDF.dat: variance
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
> self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
> AssertionError:
> Maximum relative error of 0.06 exceeded.
> Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)
>
> ======================================================================
> FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs output_opt_configITUUMa.py/MinPathloss_PDF.dat: variance
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py", line 941, in runTest
> self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
> AssertionError:
> Maximum relative error of 0.06 exceeded.
> Relative error was: 0.0603340182558 (reference: 158.615043, actual: 149.0451601)
>
> ======================================================================
> FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests) referenceOutput_configITUUMa.py/MaxRxPower_PDF.dat vs output_opt_configITUUMa.py/MaxRxPower_PDF.dat: variance
> ----------------------------------------------------------------------
> Tr...

Read more...

Revision history for this message
Re Hova Andre Rove (reandre8503) wrote : Re: [openWNS-bugsquad] [Bug 610400] Re: Test suites disabled and not found
Download full text (9.2 KiB)

Hi Mac,

I really need to use the IMT-Advanced channels. Here are the results.
cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=10.04
DISTRIB_CODENAME=lucid
DISTRIB_DESCRIPTION="Ubuntu 10.04.1 LTS"

file /usr/bin/file
/usr/bin/file: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped

uname -m
i686

uname -ar
Linux sly-desktop 2.6.32-24-generic #38-Ubuntu SMP Mon Jul 5 09:22:14 UTC
2010 i686 GNU/Linux

Thanks Mac and the whole team : )

On Wed, Jul 28, 2010 at 4:51 PM, Maciej Muehleisen <
<email address hidden>> wrote:

> Hi,
>
> I have already seen this kind of error. Looks like different hardware
> architectures produce different results because of computational
> accuracy. What does "cat /etc/lsb-release" tell you (Linux version), is
> it a 32bit or 64bit installation?
>
> The inaccuracy only seems to affect the IMT-Advanced channel models. If
> you don't plan to use them, there is no problem at all. If you depend on
> the IMT-Advanced channel models please tell me and I provide some
> reference results from the WiMAX IMTA tests to check how far your system
> is off.
>
> Greats,
>
> Mac
>
> Re Hova Andre Rove wrote:
> > Hi Mac,
> >
> > I have already tried ./playground.py missing but there is no missing
> packages and there are no new items to upgrade.
> > I tried again ./playground.py runtests and the failures are:
> >
> > ======================================================================
> > FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests)
> referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs
> output_dbg_configITUUMa.py/MinPathloss_PDF.dat: variance
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File
> "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py",
> line 941, in runTest
> > self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
> > AssertionError:
> > Maximum relative error of 0.06 exceeded.
> > Relative error was: 0.0603340182558 (reference: 158.615043, actual:
> 149.0451601)
> >
> > ======================================================================
> > FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests)
> referenceOutput_configITUUMa.py/MaxRxPower_PDF.dat vs
> output_dbg_configITUUMa.py/MaxRxPower_PDF.dat: variance
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File
> "/home/sly/OpenWNS/myOpenWNS/sandbox/default/lib/python2.4/site-packages/pywns/WNSUnit.py",
> line 941, in runTest
> > self.assertTrue( abs(relError) <= abs(self.maxRelError), errorMsg )
> > AssertionError:
> > Maximum relative error of 0.06 exceeded.
> > Relative error was: 0.0603340182558 (reference: 158.615043, actual:
> 149.0451601)
> >
> > ======================================================================
> > FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests)
> referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs
> output_opt_configITUUMa.py/MinPathloss_PDF.dat: variance
> > ---------------------------------------------------...

Read more...

Revision history for this message
Maciej Muehleisen (mue-comnets) wrote : Re: [openWNS-bugsquad] [Bug 610400] Re: Test suites disabled and not found
Download full text (10.9 KiB)

Hi,

then we should have a look at some "real" system simulation results:
Below ist the output of my WiMAC IMT-Advanced Indoor Hotspot test. If
your results match there should be no problem.

openwns-sdk/tests/system/wimac-tests/configIMTA/output_dbg_configInH.py/wimac.cirSDMA_BS_Moments.dat

# PROBE RESULTS (THIS IS A MAGIC LINE)
#
---------------------------------------------------------------------------
# Evaluation: Moments
#
---------------------------------------------------------------------------
# Name: no name available
# Description: no description available
#
---------------------------------------------------------------------------
# Evaluation terminated successfully!
#
---------------------------------------------------------------------------
# Common Statistics
# Minimum: -2.2592984
# Maximum: 80.9203689
# Trials: 84
# Mean: 36.0785463
#
# Variance: 743.4617021
# Relative variance: 0.5711636
# Coefficient of variation: 0.7557537
# Standard deviation: 27.2664941
# Relative standard deviation: 0.7557537
#
# Skewness: 0.2006351
#
# 2nd moment: 2036.2724742
# 3rd moment: 130540.3363567
#
# Sum of all values: 3030.5978933
# (Sum of all values)^2: 171046.8878308
# (Sum of all values)^3: 10965388.2539660

Greats,

Mac

Re Hova Andre Rove wrote:
> Hi Mac,
>
> I really need to use the IMT-Advanced channels. Here are the results.
> cat /etc/lsb-release
> DISTRIB_ID=Ubuntu
> DISTRIB_RELEASE=10.04
> DISTRIB_CODENAME=lucid
> DISTRIB_DESCRIPTION="Ubuntu 10.04.1 LTS"
>
> file /usr/bin/file
> /usr/bin/file: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
> dynamically linked (uses shared libs), for GNU/Linux 2.6.15, stripped
>
> uname -m
> i686
>
> uname -ar
> Linux sly-desktop 2.6.32-24-generic #38-Ubuntu SMP Mon Jul 5 09:22:14 UTC
> 2010 i686 GNU/Linux
>
> Thanks Mac and the whole team : )
>
>
> On Wed, Jul 28, 2010 at 4:51 PM, Maciej Muehleisen <
> <email address hidden>> wrote:
>
>
>> Hi,
>>
>> I have already seen this kind of error. Looks like different hardware
>> architectures produce different results because of computational
>> accuracy. What does "cat /etc/lsb-release" tell you (Linux version), is
>> it a 32bit or 64bit installation?
>>
>> The inaccuracy only seems to affect the IMT-Advanced channel models. If
>> you don't plan to use them, there is no problem at all. If you depend on
>> the IMT-Advanced channel models please tell me and I provide some
>> reference results from the WiMAX IMTA tests to check how far your system
>> is off.
>>
>> Greats,
>>
>> Mac
>>
>> Re Hova Andre Rove wrote:
>>
>>> Hi Mac,
>>>
>>> I have already tried ./playground.py missing but there is no missing
>>>
>> packages and there are no new items to upgrade.
>>
>>> I tried again ./playground.py runtests and the failures are:
>>>
>>> ======================================================================
>>> FAIL: (/home/sly/OpenWNS/myOpenWNS/tests/system/ofdma-tests)
>>>
>> referenceOutput_configITUUMa.py/MinPathloss_PDF.dat vs
>> output_dbg_configITUUMa.py/MinPathloss_PDF.dat: variance
>>
>>> ----------------------------------------------------------------------
>>> Traceback (most rec...

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.