Prune period and automated aggregation of tracker data fails in 2.16

Bug #1378893 reported by Simon Ndira on 2014-10-08
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
DHIS
High
Thu Tran

Bug Description

I have upgraded my database to 2.16. Everything is working fine, apart from two critical functionalities:

1. Automated aggregate in Programs / Attributes for tracked instances to aggregate data raises the following error:
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "periodid" violates not-null constraint
  Detail: Failing row contains (null, 1, 2013-10-08, 2013-10-08).
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
        at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:395)
        at com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:1006)
        at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:421)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396)
        ... 29 more

********************************
So the only way I can aggregate now is through manual aggregation in individual records.

2. I cannot execute prune periods as the script hangs with 28 more .....
 at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:312)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
Caused by: java.lang.NullPointerException
        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.maybeDirtyTransaction(NewProxyPreparedStatement.java:2069)
        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.clearBatch(NewProxyPreparedStatement.java:1695)
        ... 27 more

*********************************

Might someone have figured a way out of this?

Thanks and looking forward.
Simon.

Changed in dhis2:
assignee: nobody → Thu Tran (tran-hispvietnam)
importance: Undecided → High
milestone: none → 2.16
Thu Tran (tran-hispvietnam) wrote :

Hi Simon,

Which revision do you use for testing ?

For 2nd issue, could you talk to me what is the functionality got this issue ?

Download full text (4.1 KiB)

Hi Thu Trun,
I am using this version:
Version:
2.16
Build revision:
16506
Build date:
2014-10-08 13:15

Regarding the issue. It has to do with Pruning periods and Tracker.

Pruning periods: When I go to 'Data Administration' -> 'Maintenance' -> Perform Maintenance -> 'only for prune period'. This hangs with errors further below.
Aggregation: When I go to 'Program / Attributes' -> 'Automated aggregation' -> 'Execute'. This also hangs with errors further below.

Please let me know if you have any further questions.
Thanks;
Simon.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Thu Tran
Sent: Saturday, October 11, 2014 12:35 PM
To: <email address hidden>
Subject: [Bug 1378893] Re: Prune period and automated aggregation of tracker data fails in 2.16

Hi Simon,

Which revision do you use for testing ?

For 2nd issue, could you talk to me what is the functionality got this
issue ?

--
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1378893

Title:
  Prune period and automated aggregation of tracker data fails in 2.16

Status in DHIS 2:
  New

Bug description:
  I have upgraded my database to 2.16. Everything is working fine, apart
  from two critical functionalities:

  1. Automated aggregate in Programs / Attributes for tracked instances to aggregate data raises the following error:
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
          at java.lang.Thread.run(Thread.java:724)
  Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "periodid" violates not-null constraint
    Detail: Failing row contains (null, 1, 2013-10-08, 2013-10-08).
          at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
          at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
          at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
          at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
          at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
          at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:395)
          at com.mchange.v2.c3p0.impl.NewProxyStatement.execute(NewProxyStatement.java:1006)
          at org.springframework.jdbc.core.JdbcTemplate$1ExecuteStatementCallback.doInStatement(JdbcTemplate.java:421)
          at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:396)
          ... 29 more

  ********************************
  So the only way I can aggregate now is through manual aggregation in individual records.

  2. I cannot execute prune periods as the script hangs with 28 more .....
   at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
          at org.ap...

Read more...

Download full text (8.3 KiB)

Hi Simon,

The bug [1] is fixed in version 2.16, revision 16513. Please take a look at
it on your side.

For the bug [2], I did run on the demo server and my local server. It
doesn't show the same issue.
Maybe giving me the full log file instead of the partial log file will help
me to analyze the issue.
Note that even looking at the full log file might not give me information I
need, but let's try this first.

Best regards,
Tran

On Sat, Oct 11, 2014 at 7:53 PM, Simon Ndira <email address hidden> wrote:

> Hi Thu Trun,
> I am using this version:
> Version:
> 2.16
> Build revision:
> 16506
> Build date:
> 2014-10-08 13:15
>
> Regarding the issue. It has to do with Pruning periods and Tracker.
>
> Pruning periods: When I go to 'Data Administration' -> 'Maintenance' ->
> Perform Maintenance -> 'only for prune period'. This hangs with errors
> further below.
> Aggregation: When I go to 'Program / Attributes' -> 'Automated
> aggregation' -> 'Execute'. This also hangs with errors further below.
>
> Please let me know if you have any further questions.
> Thanks;
> Simon.
>
>
> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf Of
> Thu Tran
> Sent: Saturday, October 11, 2014 12:35 PM
> To: <email address hidden>
> Subject: [Bug 1378893] Re: Prune period and automated aggregation of
> tracker data fails in 2.16
>
> Hi Simon,
>
> Which revision do you use for testing ?
>
> For 2nd issue, could you talk to me what is the functionality got this
> issue ?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1378893
>
> Title:
> Prune period and automated aggregation of tracker data fails in 2.16
>
> Status in DHIS 2:
> New
>
> Bug description:
> I have upgraded my database to 2.16. Everything is working fine, apart
> from two critical functionalities:
>
> 1. Automated aggregate in Programs / Attributes for tracked instances to
> aggregate data raises the following error:
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:724)
> Caused by: org.postgresql.util.PSQLException: ERROR: null value in
> column "periodid" violates not-null constraint
> Detail: Failing row contains (null, 1, 2013-10-08, 2013-10-08).
> at
> org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2161)
> at
> org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1890)
> at
> org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:255)
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:560)
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:403)
> at
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java...

Read more...

Simon Ndira (simon-ndira) wrote :
Download full text (12.3 KiB)

Hi Tran,
Thanks for the effort. I have not seen build 16513 online. The last is 16512.
As soon as it is online, I will run tests and let you know.
It might well be that after fixing bug 1; bug 2 resolves.
So, I will get back to you re bug 2 after testing build 16513.
I appreciate.
With kind regards,
Simon.

Simon Ndira, PhD
Health Information Systems
Planning, Monitoring, Evaluation
Mobile: +213 554 745 448 (currently in Algeria)
Skype: sndira

“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—Goethe

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Thu Tran
Sent: Sunday, October 12, 2014 5:11 AM
To: <email address hidden>
Subject: Re: [Dhis2-devs] [Bug 1378893] Re: Prune period and automated aggregation of tracker data fails in 2.16

Hi Simon,

The bug [1] is fixed in version 2.16, revision 16513. Please take a look at
it on your side.

For the bug [2], I did run on the demo server and my local server. It
doesn't show the same issue.
Maybe giving me the full log file instead of the partial log file will help
me to analyze the issue.
Note that even looking at the full log file might not give me information I
need, but let's try this first.

Best regards,
Tran

On Sat, Oct 11, 2014 at 7:53 PM, Simon Ndira <email address hidden>
wrote:

> Hi Thu Trun,
> I am using this version:
> Version:
> 2.16
> Build revision:
> 16506
> Build date:
> 2014-10-08 13:15
>
> Regarding the issue. It has to do with Pruning periods and Tracker.
>
> Pruning periods: When I go to 'Data Administration' -> 'Maintenance' ->
> Perform Maintenance -> 'only for prune period'. This hangs with errors
> further below.
> Aggregation: When I go to 'Program / Attributes' -> 'Automated
> aggregation' -> 'Execute'. This also hangs with errors further below.
>
> Please let me know if you have any further questions.
> Thanks;
> Simon.
>
>
> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf Of
> Thu Tran
> Sent: Saturday, October 11, 2014 12:35 PM
> To: <email address hidden>
> Subject: [Bug 1378893] Re: Prune period and automated aggregation of
> tracker data fails in 2.16
>
> Hi Simon,
>
> Which revision do you use for testing ?
>
> For 2nd issue, could you talk to me what is the functionality got this
> issue ?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1378893
>
> Title:
> Prune period and automated aggregation of tracker data fails in 2.16
>
> Status in DHIS 2:
> New
>
> Bug description:
> I have upgraded my database to 2.16. Everything is working fine, apart
> from two critical functionalities:
>
> 1. Automated aggregate in Programs / Attributes for tracked instances to
> aggregate data raises the following error:
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run...

Simon Ndira (simon-ndira) wrote :
Download full text (12.6 KiB)

Dear Lars, Edem, Thu Tran,
Thanks so much for the intervention. I would like to confirm that the following are running without error:
1. Automated aggregate query building to aggregating tracked entity data to aggregate level
2. Generation of resource tables
3. Running of analytics.

The only thing that freezes upon execution is pruning periods; but I do not think it is of great consequence.

This analysis is based on the latest build of 2.16 - Build134.

Perhaps you might want to advise the rest of the community experiencing the same problem to migrate to the latest build.

This will be of great help in generating real time data at MIFUMI.

With kind regards,
Simon.

Simon Ndira, PhD
Health Information Systems
Planning, Monitoring, Evaluation
Mobile: +213 554 745 448 (currently in Algeria)
Skype: sndira

“Knowing is not enough; we must apply.
Willing is not enough; we must do.”
—Goethe

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Thu Tran
Sent: Sunday, October 12, 2014 5:11 AM
To: <email address hidden>
Subject: Re: [Dhis2-devs] [Bug 1378893] Re: Prune period and automated aggregation of tracker data fails in 2.16

Hi Simon,

The bug [1] is fixed in version 2.16, revision 16513. Please take a look at
it on your side.

For the bug [2], I did run on the demo server and my local server. It
doesn't show the same issue.
Maybe giving me the full log file instead of the partial log file will help
me to analyze the issue.
Note that even looking at the full log file might not give me information I
need, but let's try this first.

Best regards,
Tran

On Sat, Oct 11, 2014 at 7:53 PM, Simon Ndira <email address hidden>
wrote:

> Hi Thu Trun,
> I am using this version:
> Version:
> 2.16
> Build revision:
> 16506
> Build date:
> 2014-10-08 13:15
>
> Regarding the issue. It has to do with Pruning periods and Tracker.
>
> Pruning periods: When I go to 'Data Administration' -> 'Maintenance' ->
> Perform Maintenance -> 'only for prune period'. This hangs with errors
> further below.
> Aggregation: When I go to 'Program / Attributes' -> 'Automated
> aggregation' -> 'Execute'. This also hangs with errors further below.
>
> Please let me know if you have any further questions.
> Thanks;
> Simon.
>
>
> -----Original Message-----
> From: <email address hidden> [mailto:<email address hidden>] On Behalf Of
> Thu Tran
> Sent: Saturday, October 11, 2014 12:35 PM
> To: <email address hidden>
> Subject: [Bug 1378893] Re: Prune period and automated aggregation of
> tracker data fails in 2.16
>
> Hi Simon,
>
> Which revision do you use for testing ?
>
> For 2nd issue, could you talk to me what is the functionality got this
> issue ?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1378893
>
> Title:
> Prune period and automated aggregation of tracker data fails in 2.16
>
> Status in DHIS 2:
> New
>
> Bug description:
> I have upgraded my database to 2.16. Everything is working fine, apart
> from two critical functionalities:
>
> 1. Automated aggregate in Programs / Attributes for tracked instances to
> aggregate data raises the follow...

Changed in dhis2:
milestone: 2.16 → 2.18
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers