No data in Report Tables generated from Pivot Tables with DataElement - Details (categories)

Bug #1240399 reported by Fábio Ramos on 2013-10-16
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
DHIS
High
Jan Henrik Øverland

Bug Description

Problem with Pivot Tables and generated Report Tables:

  Create one Pivot Table that show data, but the Report Table generated from the Pivot Table don’t show anything, the pivot table is created with Data Element - Details (Categories).

(the report table don´t show any data, even after the Analytics and Datamart update tables)

(The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build date: 2013-10-14 16:04 , I have this behavior at the snapshot [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version 2.13, see attach https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files/pivot-table_generated-report-table.png)

Fábio Ramos (fd-ramos) wrote :
description: updated
Paulo Grácio (pgracio) wrote :

Hi, also did a test with the sample data used in http://apps.dhis2.org/demo and I could reproduce the problem.

Using the Pivot table sent by Lars announcing this new functionality

"Pivot table supports "details", meaning selection of individual category option combos for data elements. Under "data elements", select "details" and then your desired items."

http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=AOphCIgmjeN

When running Report table no data is displayed

http://apps.dhis2.org/demo/dhis-web-reporting/exportTable.action?uid=AOphCIgmjeN&type=html&

Leandro Soares (lssoares) wrote :

dhis-2\dhis-api\src\main\java\org\hisp\dhis\reporttable\ReportTable.java

Method:
public Grid getGrid( Grid grid, Map<String, Double> valueMap, boolean paramColumns )
....

             // -----------------------------------------------------------------
            // Row data values
            // -----------------------------------------------------------------

            for ( List<NameableObject> column : gridColumns )
            {
                String key = getIdentifer( column, row );

                Double value = valueMap.get( key );

                grid.addValue( value );
            }
....

On the routine above, key has values like: "2013-FIbM6WouPGs.PGn5wAmodfX"
and valueMap has values like "2013-FIbM6WouPGs-PGn5wAmodfX"

Matching tokens is not possible as one uses the '-' as separator and the other a '.'.

This is a duplicate of this one: https://bugs.launchpad.net/dhis2/+bug/1230596

Changed in dhis2:
assignee: nobody → Lars Helge Øverland (larshelge)
milestone: none → 2.13
importance: Undecided → High
status: New → Confirmed

Was caused by a missing "co" item in download URL from pivot table client. Fixed in trunk and 2.13.

Changed in dhis2:
status: Confirmed → Fix Committed
assignee: Lars Helge Øverland (larshelge) → Jan Henrik Øverland (janhenrik-overland)
Paulo Grácio (pgracio) wrote :

Hi Lars, was this fix already merged back to version 2.13?

Yes, rev 12886: https://code.launchpad.net/~dhis2-devs-core/dhis2/2.13

On Mon, Dec 2, 2013 at 11:33 AM, Paulo Grácio
<email address hidden>wrote:

> Hi Lars, was this fix already merged back to version 2.13?
>
> --
> You received this bug notification because you are a bug assignee.
> https://bugs.launchpad.net/bugs/1240399
>
> Title:
> No data in Report Tables generated from Pivot Tables with DataElement
> - Details (categories)
>
> Status in DHIS 2:
> Fix Committed
>
> Bug description:
> Problem with Pivot Tables and generated Report Tables:
>
> Create one Pivot Table that show data, but the Report Table
> generated from the Pivot Table don’t show anything, the pivot table is
> created with Data Element - Details (Categories).
>
> (the report table don´t show any data, even after the Analytics and
> Datamart update tables)
>
> (The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build
> date: 2013-10-14 16:04 , I have this behavior at the snapshot
> [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version
> 2.13, see attach
> https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files
> /pivot-table_generated-report-table.png)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1240399/+subscriptions
>

Yes.

On Mon, Dec 2, 2013 at 11:33 AM, Paulo Grácio
<email address hidden>wrote:

> Hi Lars, was this fix already merged back to version 2.13?
>
> --
> You received this bug notification because you are a member of DHIS 2
> developers, which is subscribed to DHIS.
> https://bugs.launchpad.net/bugs/1240399
>
> Title:
> No data in Report Tables generated from Pivot Tables with DataElement
> - Details (categories)
>
> Status in DHIS 2:
> Fix Committed
>
> Bug description:
> Problem with Pivot Tables and generated Report Tables:
>
> Create one Pivot Table that show data, but the Report Table
> generated from the Pivot Table don’t show anything, the pivot table is
> created with Data Element - Details (Categories).
>
> (the report table don´t show any data, even after the Analytics and
> Datamart update tables)
>
> (The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build
> date: 2013-10-14 16:04 , I have this behavior at the snapshot
> [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version
> 2.13, see attach
> https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files
> /pivot-table_generated-report-table.png)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1240399/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>

Paulo Grácio (pgracio) wrote :

I was trying to reproduce de bug scenario that I described and it seems that the problem still persists.

Using the Pivot table sent by Lars announcing this new functionality

"Pivot table supports "details", meaning selection of individual category option combos for data elements. Under "data elements", select "details" and then your desired items."

http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=AOphCIgmjeN

When running Report table no data is displayed

http://apps.dhis2.org/demo/dhis-web-reporting/exportTable.action?uid=AOphCIgmjeN&type=html&

Maybe I'm doing something wrong...

Hi Paulo, it seems to be okay, have a look for instance at the
"Immunization: doses given under 1 year last 12 months" at the demo server
running 2.13, which uses "data element details" on the columns and produces
this download:

http://apps.dhis2.org/demo/api/analytics.html?dimension=dx:s46m5MS0hxu;UOlfIjgN8X6;YtbsuPPo010;x3Do5e7g4Qo;O05mAByOgAv;pikOziyCXbM;vI2csg55S9C;L2kxa2IA2cs;l6byfWFUGaP;n6aMJNLdvep&dimension=pe:201212;201301;201302;201303;201304;201305;201306;201307;201308;201309;201310;201311&dimension=ou:jUb8gELQApl;PMa2VCrupOd;kJq2mPyFEHo;O6uvpzGd5pu&dimension=co&tableLayout=true&columns=dx;co&rows=pe;ou

On Mon, Dec 2, 2013 at 11:51 AM, Paulo Grácio
<email address hidden>wrote:

> I was trying to reproduce de bug scenario that I described and it seems
> that the problem still persists.
>
> Using the Pivot table sent by Lars announcing this new functionality
>
> "Pivot table supports "details", meaning selection of individual
> category option combos for data elements. Under "data elements", select
> "details" and then your desired items."
>
> http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=AOphCIgmjeN
>
> When running Report table no data is displayed
>
> http://apps.dhis2.org/demo/dhis-web-
> reporting/exportTable.action?uid=AOphCIgmjeN&type=html&
>
> Maybe I'm doing something wrong...
>
> --
> You received this bug notification because you are a member of DHIS 2
> developers, which is subscribed to DHIS.
> https://bugs.launchpad.net/bugs/1240399
>
> Title:
> No data in Report Tables generated from Pivot Tables with DataElement
> - Details (categories)
>
> Status in DHIS 2:
> Fix Committed
>
> Bug description:
> Problem with Pivot Tables and generated Report Tables:
>
> Create one Pivot Table that show data, but the Report Table
> generated from the Pivot Table don’t show anything, the pivot table is
> created with Data Element - Details (Categories).
>
> (the report table don´t show any data, even after the Analytics and
> Datamart update tables)
>
> (The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build
> date: 2013-10-14 16:04 , I have this behavior at the snapshot
> [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version
> 2.13, see attach
> https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files
> /pivot-table_generated-report-table.png)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1240399/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>

Okay you are right, we can fix. But the old report table UI is going away
soon. It works when you go through pivot table -> download as table layout:

http://apps.dhis2.org/demo/dhis-web-pivot/app/index.html?id=AOphCIgmjeN

On Tue, Dec 3, 2013 at 2:00 PM, Paulo Grácio
<email address hidden>wrote:

> Thanks using API directly, wasn't expected to have the same result when
> executing Table Report?
>
> http://apps.dhis2.org/demo/dhis-web-reporting/displayManageTableForm.action
>
> http://apps.dhis2.org/demo/dhis-web-reporting/exportTable.action?uid=AOphCIgmjeN&type=html&
>
> --
> You received this bug notification because you are a member of DHIS 2
> developers, which is subscribed to DHIS.
> https://bugs.launchpad.net/bugs/1240399
>
> Title:
> No data in Report Tables generated from Pivot Tables with DataElement
> - Details (categories)
>
> Status in DHIS 2:
> Fix Committed
>
> Bug description:
> Problem with Pivot Tables and generated Report Tables:
>
> Create one Pivot Table that show data, but the Report Table
> generated from the Pivot Table don’t show anything, the pivot table is
> created with Data Element - Details (Categories).
>
> (the report table don´t show any data, even after the Analytics and
> Datamart update tables)
>
> (The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build
> date: 2013-10-14 16:04 , I have this behavior at the snapshot
> [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version
> 2.13, see attach
> https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files
> /pivot-table_generated-report-table.png)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1240399/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>

Paulo Grácio (pgracio) wrote :

Hi Lars,

does that mean that Standard Report of type 'Jasper report with report table data source' will also be deprecated? I'm asking this because this bug also causes data unavailable at Jasper Reports.

Do you have any idea of how long will it take to have a fix? Any idea of where the bug might be so we can help with a fix?

Thanks for all the support.
Paulo Grácio

Hi no we will not remove standard reports.

Will fix ASAP.

Lars

On Tue, Dec 3, 2013 at 7:46 PM, Paulo Grácio
<email address hidden>wrote:

> Hi Lars,
>
> does that mean that Standard Report of type 'Jasper report with report
> table data source' will also be deprecated? I'm asking this because
> this bug also causes data unavailable at Jasper Reports.
>
> Do you have any idea of how long will it take to have a fix? Any idea of
> where the bug might be so we can help with a fix?
>
> Thanks for all the support.
> Paulo Grácio
>
> --
> You received this bug notification because you are a member of DHIS 2
> developers, which is subscribed to DHIS.
> https://bugs.launchpad.net/bugs/1240399
>
> Title:
> No data in Report Tables generated from Pivot Tables with DataElement
> - Details (categories)
>
> Status in DHIS 2:
> Fix Committed
>
> Bug description:
> Problem with Pivot Tables and generated Report Tables:
>
> Create one Pivot Table that show data, but the Report Table
> generated from the Pivot Table don’t show anything, the pivot table is
> created with Data Element - Details (Categories).
>
> (the report table don´t show any data, even after the Analytics and
> Datamart update tables)
>
> (The Version I’m using is: 2.13-SNAPSHOT, Build revision: 12639, Build
> date: 2013-10-14 16:04 , I have this behavior at the snapshot
> [http://apps.dhis2.org/dev/dhis-web-pivot/app/index.html], version
> 2.13, see attach
> https://bugs.launchpad.net/dhis2/+bug/1240399/+attachment/3879065/+files
> /pivot-table_generated-report-table.png)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/dhis2/+bug/1240399/+subscriptions
>
> _______________________________________________
> Mailing list: https://launchpad.net/~dhis2-devs
> Post to : <email address hidden>
> Unsubscribe : https://launchpad.net/~dhis2-devs
> More help : https://help.launchpad.net/ListHelp
>

Now fixed for report table too in trunk and 2.13.

Paulo Grácio (pgracio) wrote :

From: Paulo Grácio [mailto:<email address hidden>]
Sent: terça-feira, 10 de Dezembro de 2013 14:58
To: 'Lars Helge Øverland';
Subject: RE: Community Features in DHIS2

Hi Lars,

Pivot Table Data, and;
Lars: “This has been corrected - now both for pivot table and for standard reports (report table). Fixed in trunk and version 2.13.”

This still has a problem once data on Jasper Reports is not being rendered correctly. Data appears in a random way. After analysing the code our conclusions are:

- Data is being displayed in a random way because on DataElementCategoryOptionCombo.hbm.xml categoryOptions are mapped to a Set, using HashSet implementation. It makes no guarantees as to the iteration order of the set; in particular, it does not guarantee that the order will remain constant over time.
- When calling getName() it has a random behaviour on Jasper Report expect a fixed name like:

<field name="escolaridade (nenhuma, ccr)" class="java.lang.Double"/>
<field name="escolaridade (cpppf, nenhuma)" class="java.lang.Double"/>
<field name="escolaridade (nenhuma, clínica de tb)" class="java.lang.Double"/>

And sometimes we get

escolaridade (nenhuma, ccr)
escolaridade (nenhuma, cpppf)
escolaridade (nenhuma, clínica de tb)

others

escolaridade (ccr , nenhuma)
escolaridade (nenhuma, cpppf)
escolaridade (clínica de tb, nenhuma)

is this rational correct?

Kind regards,
Paulo Grácio

Okay. I assume this happens when downloading a table as JRXML from the report table UI in reports module?

Paulo Grácio (pgracio) wrote :

Hi Lars,

this is happening when we execute standard reports based on Jasper Reports with Report Table Datasource.

When executing report table and downloading a table as JRXML I guess it executes the same block of code.

Regards,
Paulo Grácio

Pascal Brandt (pascal-v) wrote :

Hi @larshelge,

Any chance you've managed to take another look at this?

Ciao,
Pascal

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers