xtrabackup-1.2 is slower than 1.0

[Vadim Tkachenko]

xtrabackup-1.2 is slower in copying files from idle server.

For the same database:

xtrabackup-1.0 (build against 5.0.84):

real 8m49.151s
user 5m43.654s
sys 1m21.868s

xtrabackup-1.2 ( built against 5.1.45)
real 21m21.623s
user 14m58.821s
sys 1m16.553s

[Vadim Tkachenko (vadim-tk)]

original procedure was incorrect, marking as "Invalid" for now

[Jonas Bygdén (jbygden)]

Since I was the one asking the question both here at launchpad and in the google group. What does the last update mean? What is the "original procedure" that was incorrect?

[Vadim Tkachenko (vadim-tk)]

Jonas,

We did not test it yet so I can't answer definitely.

2010/10/18 Jonas Bygdén <email address hidden>:
> Since I was the one asking the question both here at launchpad and in
> the google group. What does the last update mean? What is the "original
> procedure" that was incorrect?
>
> --
> xtrabackup-1.2 is slower than 1.0
> https://bugs.launchpad.net/bugs/645581
> You received this bug notification because you are a member of Percona
> developers, which is the registrant for Percona-XtraBackup.
>
> Status in Open source backup tool for InnoDB and XtraDB: Invalid
>
> Bug description:
> xtrabackup-1.2 is slower in copying files from idle server.
>
> For the same database:
>
> xtrabackup-1.0 (build against 5.0.84):
>
> real    8m49.151s
> user    5m43.654s
> sys     1m21.868s
>
> xtrabackup-1.2 ( built against 5.1.45)
> real    21m21.623s
> user    14m58.821s
> sys     1m16.553s
>
>
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone [phone],  Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

[Vadim Tkachenko (vadim-tk)]

I tested on idle server and I can't repeat regression

xtrabackup 1.0

real 4m30.253s
user 1m31.995s
sys 0m4.588s

xtrabackup 1.2

real 4m59.405s
user 1m28.927s
sys 0m3.341s

[Jonas Bygdén (jbygden)]

For some reason you chose to mark this bug as invalid, with the comment "original procedure was incorrect".

What does that mean?

[Vadim Tkachenko (vadim-tk)]

Jonas,

That means when I reported this bug I initially tested it wrong ( with
different datasets).

It does not mean you use it incorrectly.

If you still have this performance issue please show me more details.

2010/10/19 Jonas Bygdén <email address hidden>:
> For some reason you chose to mark this bug as invalid, with the comment
> "original procedure was incorrect".
>
> What does that mean?
>
> --
> xtrabackup-1.2 is slower than 1.0
> https://bugs.launchpad.net/bugs/645581
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Open source backup tool for InnoDB and XtraDB: Invalid
>
> Bug description:
> xtrabackup-1.2 is slower in copying files from idle server.
>
> For the same database:
>
> xtrabackup-1.0 (build against 5.0.84):
>
> real    8m49.151s
> user    5m43.654s
> sys     1m21.868s
>
> xtrabackup-1.2 ( built against 5.1.45)
> real    21m21.623s
> user    14m58.821s
> sys     1m16.553s
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/percona-xtrabackup/+bug/645581/+subscribe
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone [phone],  Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

[Jonas Bygdén (jbygden)]

No problem, what do you need?

[Vadim Tkachenko (vadim-tk)]

Jonas,

I would need repeatable procedure to investigate this problem.

2010/10/19 Jonas Bygdén <email address hidden>:
> No problem, what do you need?
>
> --
> xtrabackup-1.2 is slower than 1.0
> https://bugs.launchpad.net/bugs/645581
> You received this bug notification because you are a member of Percona
> developers, which is the registrant for Percona-XtraBackup.
>
> Status in Open source backup tool for InnoDB and XtraDB: Invalid
>
> Bug description:
> xtrabackup-1.2 is slower in copying files from idle server.
>
> For the same database:
>
> xtrabackup-1.0 (build against 5.0.84):
>
> real    8m49.151s
> user    5m43.654s
> sys     1m21.868s
>
> xtrabackup-1.2 ( built against 5.1.45)
> real    21m21.623s
> user    14m58.821s
> sys     1m16.553s
>
>
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone [phone],  Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

[Jonas Bygdén (jbygden)]

These are the (sanitized) logs from two similar servers, replicas of the same database, one running xtrabackup 1.0 and the other running xtrabackup 1.3. The one running 1.0 took about 3 hours, the one running 1.3 took more than 5,5 hours.

[Jonas Bygdén (jbygden)]

These are the (sanitized) logs from two similar servers, replicas of the same database, one running xtrabackup 1.0 and the other running xtrabackup 1.3. The one running 1.0 took about 3 hours, the one running 1.3 took more than 5,5 hours.

[Jonas Bygdén (jbygden)]

Don't know if our logs will give you any information that you can use in troubleshooting this. Let me know if you have anything specific you'd like me to do/run/test.

[Vadim Tkachenko (vadim-tk)]

Jonas,

Can you show the same log for 1.0 ?

2010/10/22 Jonas Bygdén <email address hidden>:
> Don't know if our logs will give you any information that you can use in
> troubleshooting this. Let me know if you have anything specific you'd
> like me to do/run/test.
>
> --
> xtrabackup-1.2 is slower than 1.0
> https://bugs.launchpad.net/bugs/645581
> You received this bug notification because you are a direct subscriber
> of the bug.
>
> Status in Open source backup tool for InnoDB and XtraDB: Invalid
>
> Bug description:
> xtrabackup-1.2 is slower in copying files from idle server.
>
> For the same database:
>
> xtrabackup-1.0 (build against 5.0.84):
>
> real    8m49.151s
> user    5m43.654s
> sys     1m21.868s
>
> xtrabackup-1.2 ( built against 5.1.45)
> real    21m21.623s
> user    14m58.821s
> sys     1m16.553s
>
> To unsubscribe from this bug, go to:
> https://bugs.launchpad.net/percona-xtrabackup/+bug/645581/+subscribe
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone [phone],  Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

[Jonas Bygdén (jbygden)]

Both logs are attached, both 1.0 and 1.3, but I couldn't attach them both to the same comment. They are in the comments after each other.

[Vadim Tkachenko (vadim-tk)]

Jonas,

Ok, I see you use --stream.

I suspect the slowdown is caused by tar4ibd which is used instead of
tar in newest versions.

Can you try 1.3 version with --force-tar option ?

2010/10/22 Jonas Bygdén <email address hidden>:
> Both logs are attached, both 1.0 and 1.3, but I couldn't attach them
> both to the same comment. They are in the comments after each other.
>
> --
> xtrabackup-1.2 is slower than 1.0
> https://bugs.launchpad.net/bugs/645581
> You received this bug notification because you are a member of Percona
> developers, which is the registrant for Percona-XtraBackup.
>
> Status in Open source backup tool for InnoDB and XtraDB: Invalid
>
> Bug description:
> xtrabackup-1.2 is slower in copying files from idle server.
>
> For the same database:
>
> xtrabackup-1.0 (build against 5.0.84):
>
> real    8m49.151s
> user    5m43.654s
> sys     1m21.868s
>
> xtrabackup-1.2 ( built against 5.1.45)
> real    21m21.623s
> user    14m58.821s
> sys     1m16.553s
>
>
>

--
Vadim Tkachenko, CTO, Percona Inc.
Phone [phone],  Skype: vadimtk153
Schedule meeting: http://tungle.me/VadimTkachenko

[Vadim Tkachenko (vadim-tk)]

Yes, it seems the reason is that stream uses tar4ibd in new version

Unfortunately there is not so much we can do, as tar4ibd is recommended for stream, and it is needed
to guarantee no data corruption.

time ./innobackupex-1.5.1 --stream=tar --force-tar --ibbackup=/data/vadim/test-xtrabackup/xtrabackup-1.2/bin/xtrabackup ./ >> /mnt/x25m/test/3.tar

real 3m5.514s
user 0m0.721s
sys 0m50.415s

time ./innobackupex-1.5.1 --stream=tar --ibbackup=/data/vadim/test-xtrabackup/xtrabackup-1.2/bin/xtrabackup ./ >> /mnt/x25m/test/2.tar

real 4m56.655s
user 2m44.840s
sys 1m33.538s

[Shahriyar Rzayev (rzayev-sehriyar)]

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXB-285