Counter dependent comparison of output in regression tests is unsafe

[Martin Sandve Alnæs]

The global counter in the regression test causes two types of problems:

1) Lots of false negatives if a new line is introduced (will be common when updating ufc)

2) Possible confusing errors if two lines with the same variable name accidentally get the same count (probably rare but actually happened to me once not so long ago)

The first issue is naturally the most serious. How can we improve the output comparison?

[Martin Sandve Alnæs (martinal)]

What does the counter add of value? I only see the problems it causes.

[Anders Logg (logg)]

On Fri, Feb 01, 2013 at 02:38:13PM -0000, Martin Sandve Alnæs wrote:
> What does the counter add of value? I only see the problems it causes.

I don't remember why it was added. Likely as extra data to ensure
unique signatures. If it works without it, then feel free to remove it.

--
Anders

[Martin Sandve Alnæs (martinal)]

The problem is that it makes all signatures different if you add a new
line, changing the subsequent counters.

I have sketched some code that should give more stable and readable
hierarchial diffs, I'll come back to this.

Btw, are your geometry merges coming soon?

Martin
Den 2. feb. 2013 13:15 skrev "Anders Logg" <email address hidden> følgende:

> On Fri, Feb 01, 2013 at 02:38:13PM -0000, Martin Sandve Alnæs wrote:
> > What does the counter add of value? I only see the problems it causes.
>
> I don't remember why it was added. Likely as extra data to ensure
> unique signatures. If it works without it, then feel free to remove it.
>
> --
> Anders
>
> --
> You received this bug notification because you are a member of FFC Core
> Team, which is subscribed to FFC.
> https://bugs.launchpad.net/bugs/1112508
>
> Title:
> Counter dependent comparison of output in regression tests is unsafe
>
> Status in FEniCS Form Compiler:
> New
>
> Bug description:
> The global counter in the regression test causes two types of
> problems:
>
> 1) Lots of false negatives if a new line is introduced (will be common
> when updating ufc)
>
> 2) Possible confusing errors if two lines with the same variable name
> accidentally get the same count (probably rare but actually happened
> to me once not so long ago)
>
> The first issue is naturally the most serious. How can we improve the
> output comparison?
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ffc/+bug/1112508/+subscriptions
>

[Anders Logg (logg)]

B1;3400;0cOn Sat, Feb 02, 2013 at 12:25:02PM -0000, Martin Sandve Alnæs wrote:
> The problem is that it makes all signatures different if you add a new
> line, changing the subsequent counters.
>
> I have sketched some code that should give more stable and readable
> hierarchial diffs, I'll come back to this.
>
> Btw, are your geometry merges coming soon?

It compiles, runs and does no longer segfault... So I'm getting
closer.

The current obstacle is getting the tests in FFC running again. Some
tests were failing for evaluate_basis. Teaching and other deadlines
has not left many hours/week to work on it.

--
Anders

> Martin
> Den 2. feb. 2013 13:15 skrev "Anders Logg" <email address hidden> følgende:
>
> > On Fri, Feb 01, 2013 at 02:38:13PM -0000, Martin Sandve Alnæs wrote:
> > > What does the counter add of value? I only see the problems it causes.
> >
> > I don't remember why it was added. Likely as extra data to ensure
> > unique signatures. If it works without it, then feel free to remove it.
> >
> >
> >
> > Title:
> > Counter dependent comparison of output in regression tests is unsafe
> >
> > Status in FEniCS Form Compiler:
> > New
> >
> > Bug description:
> > The global counter in the regression test causes two types of
> > problems:
> >
> > 1) Lots of false negatives if a new line is introduced (will be common
> > when updating ufc)
> >
> > 2) Possible confusing errors if two lines with the same variable name
> > accidentally get the same count (probably rare but actually happened
> > to me once not so long ago)
> >
> > The first issue is naturally the most serious. How can we improve the
> > output comparison?
> >
> > To manage notifications about this bug go to:
> > https://bugs.launchpad.net/ffc/+bug/1112508/+subscriptions
> >
>