Merge lp:~charlesk/keeper/untar into lp:keeper/devel
- untar
- Merge into devel
Status: | Merged |
---|---|
Approved by: | Xavi Garcia |
Approved revision: | 160 |
Merged at revision: | 123 |
Proposed branch: | lp:~charlesk/keeper/untar |
Merge into: | lp:keeper/devel |
Diff against target: |
1362 lines (+984/-108) 18 files modified
debian/control (+5/-1) debian/keeper.install (+2/-1) src/helper/folder-backup.sh.in (+1/-1) src/tar/CMakeLists.txt (+72/-27) src/tar/untar-main.cpp (+169/-0) src/tar/untar.cpp (+144/-0) src/tar/untar.h (+38/-0) tests/CMakeLists.txt (+2/-1) tests/com_canonical_keeper.py (+80/-45) tests/unit/tar/CMakeLists.txt (+104/-29) tests/unit/tar/keeper-untar-test.cpp (+232/-0) tests/unit/tar/ku-invoke-nobus.sh.in (+1/-0) tests/unit/tar/ku-invoke.sh.in (+1/-0) tests/unit/tar/untar-test.cpp (+120/-0) tests/utils/file-utils.cpp (+2/-1) tests/utils/file-utils.h (+1/-1) tests/utils/keeper-dbusmock-fixture.h (+9/-0) tests/utils/main.cpp (+1/-1) |
To merge this branch: | bzr merge lp:~charlesk/keeper/untar |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Xavi Garcia (community) | Approve | ||
unity-api-1-bot | continuous-integration | Approve | |
Review via email: mp+313189@code.launchpad.net |
Commit message
Add untar
Description of the change
Add untar.
* The Untar class itself, which manages a couple of QProcesses for xzcat and tar x. Add tar & xz to debian dependencies.
* Add untar-test unit test for the untar class
* Add keeper-untar executable that gets a restore socket from keeper, then untars from it
* Add keeper-untar-test that tests keeper-untar against the dbusmock service. Implement restore in the dbusmock service.
* Rename keeper-tar-create as keeper-tar for symmetry with keeper-untar
* Not pushed/completed: integration tests
unity-api-1-bot (unity-api-1-bot) wrote : | # |
unity-api-1-bot (unity-api-1-bot) wrote : | # |
FAILED: Continuous integration, rev:148
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
Xavi Garcia (xavi-garcia-mena) wrote : | # |
Thanks, Charles.
It looks good and I've tested it with the service and the command line client and it works fine.
I tried to trigger a rebuild myself but jenkins is still complaining.
I've added a comment inline...
A found a minor issue, non blocking, but that in cases that the restore file is very big we could be trying to restore even we had an error in the step method. (the error in not checked when calling step)
And even more minor: the coding style for the brackets is different in some cases.
There are some cases like:
if (blah) {
blah
}
Shouldn't it be:?
if (blah)
{
blah
}
- 149. By Charles Kerr
-
in debian/control, remove errant newline
- 150. By Charles Kerr
-
in keeper-untar-test, extract out common test code into helper methods
- 151. By Charles Kerr
-
in keeper-untar, exit with failure if Untar::step() fails
- 152. By Charles Kerr
-
in Untar class, add finish() method to check that xz/tar finish and check their return values
- 153. By Charles Kerr
-
in keeper-untar, exit with failure if Untar.step() or Untar.finish() fail
- 154. By Charles Kerr
-
fix end-of-namespace semicolon warning
- 155. By Charles Kerr
-
in untar-test, use Untar.finish()
- 156. By Charles Kerr
-
in keeper-untar-test, add test cases for restoring corrupt archives
- 157. By Charles Kerr
-
in service dbusmock, wait for helpers to exit
unity-api-1-bot (unity-api-1-bot) wrote : | # |
FAILED: Continuous integration, rev:157
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
- 158. By Charles Kerr
-
fix bad whitespace that caused whitespace test to fail
- 159. By Charles Kerr
-
fix flake8 warnings in service dbusmock
- 160. By Charles Kerr
-
fix minor int conversion warning in tests/utils/
main.cpp
unity-api-1-bot (unity-api-1-bot) wrote : | # |
FAILED: Continuous integration, rev:
https:/
Executed test runs:
FAILURE: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
FAILURE: https:/
Click here to trigger a rebuild:
https:/
unity-api-1-bot (unity-api-1-bot) wrote : | # |
PASSED: Continuous integration, rev:160
https:/
Executed test runs:
SUCCESS: https:/
SUCCESS: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
SUCCESS: https:/
deb: https:/
Click here to trigger a rebuild:
https:/
Xavi Garcia (xavi-garcia-mena) wrote : | # |
Looks good to me, thanks Charles
Preview Diff
1 | === modified file 'debian/control' |
2 | --- debian/control 2016-08-26 08:48:05 +0000 |
3 | +++ debian/control 2016-12-16 19:41:21 +0000 |
4 | @@ -34,7 +34,7 @@ |
5 | python3-flake8, |
6 | Standards-Version: 3.9.5 |
7 | Homepage: https://launchpad.net/keeper |
8 | -# If you aren't a member of ~indicator-applet-developers but need to upload |
9 | + If you aren't a member of ~indicator-applet-developers but need to upload |
10 | # packaging changes, just go ahead. ~indicator-applet-developers will notice |
11 | # and sync up the code again. |
12 | Vcs-Bzr: https://code.launchpad.net/keeper/trunk |
13 | @@ -45,6 +45,8 @@ |
14 | Depends: ${shlibs:Depends}, |
15 | ${misc:Depends}, |
16 | systemd | systemd-shim, |
17 | + tar, |
18 | + xz-utils |
19 | Description: Backup Tool |
20 | A backup/restore utility for Ubuntu |
21 | |
22 | @@ -53,6 +55,8 @@ |
23 | Depends: ${shlibs:Depends}, |
24 | ${misc:Depends}, |
25 | systemd | systemd-shim, |
26 | + tar, |
27 | + xz-utils |
28 | Description: Backup Tool |
29 | A backup/restore utility for Ubuntu (client application) |
30 | |
31 | |
32 | === modified file 'debian/keeper.install' |
33 | --- debian/keeper.install 2016-08-10 07:43:36 +0000 |
34 | +++ debian/keeper.install 2016-12-16 19:41:21 +0000 |
35 | @@ -1,6 +1,7 @@ |
36 | /usr/lib/*/keeper/folder-backup.sh |
37 | /usr/lib/*/keeper/keeper-service |
38 | -/usr/lib/*/keeper/keeper-tar-create |
39 | +/usr/lib/*/keeper/keeper-tar |
40 | +/usr/lib/*/keeper/keeper-untar |
41 | /usr/lib/*/ubuntu-app-launch/backup-helper/exec-tool |
42 | /usr/share/keeper/helper-registry.json |
43 | /usr/share/dbus-1/services/com.canonical.keeper.service |
44 | |
45 | === modified file 'src/helper/folder-backup.sh.in' |
46 | --- src/helper/folder-backup.sh.in 2016-08-10 05:01:08 +0000 |
47 | +++ src/helper/folder-backup.sh.in 2016-12-16 19:41:21 +0000 |
48 | @@ -20,4 +20,4 @@ |
49 | # |
50 | |
51 | echo $PWD |
52 | -find ./ -type f -print0 | @CMAKE_INSTALL_FULL_PKGLIBEXECDIR@/keeper-tar-create -a /com/canonical/keeper/helper |
53 | +find ./ -type f -print0 | @CMAKE_INSTALL_FULL_PKGLIBEXECDIR@/keeper-tar -a /com/canonical/keeper/helper |
54 | |
55 | === modified file 'src/tar/CMakeLists.txt' |
56 | --- src/tar/CMakeLists.txt 2016-07-22 04:01:28 +0000 |
57 | +++ src/tar/CMakeLists.txt 2016-12-16 19:41:21 +0000 |
58 | @@ -1,8 +1,14 @@ |
59 | set(LIB_NAME "keepertar") |
60 | -set(APP_NAME "keeper-tar-create") |
61 | +set(KEEPER_TAR_APP_NAME "keeper-tar") |
62 | +set(KEEPER_UNTAR_APP_NAME "keeper-untar") |
63 | + |
64 | +## |
65 | +## library |
66 | +## |
67 | |
68 | set(LIB_SOURCES |
69 | tar-creator.cpp |
70 | + untar.cpp |
71 | ) |
72 | add_library( |
73 | ${LIB_NAME} |
74 | @@ -10,38 +16,76 @@ |
75 | ${LIB_SOURCES} |
76 | ) |
77 | |
78 | -set_property( |
79 | - SOURCE main.cpp |
80 | - PROPERTIES APPEND_STRING PROPERTY COMPILE_DEFINITIONS APP_NAME=\"${APP_NAME}\" |
81 | -) |
82 | - |
83 | -set(APP_SOURCES |
84 | - main.cpp |
85 | -) |
86 | - |
87 | -add_executable( |
88 | - ${APP_NAME} |
89 | - ${APP_SOURCES} |
90 | -) |
91 | - |
92 | link_directories( |
93 | ${SERVICE_DEPS_LIBRARY_DIRS} |
94 | ) |
95 | |
96 | -target_link_libraries( |
97 | - ${APP_NAME} |
98 | - ${LIB_NAME} |
99 | - qdbus-stubs |
100 | - storage-framework |
101 | - backup-helper |
102 | - ${SERVICE_DEPS_LIBRARIES} |
103 | - Qt5::Core |
104 | - Qt5::DBus |
105 | -) |
106 | +## |
107 | +## tar creator |
108 | +## |
109 | + |
110 | +set_property( |
111 | + SOURCE tar-creator-main.cpp |
112 | + PROPERTIES APPEND_STRING PROPERTY COMPILE_DEFINITIONS APP_NAME=\"${KEEPER_TAR_APP_NAME}\" |
113 | +) |
114 | + |
115 | +set(KEEPER_UNTAR_APP_SOURCES |
116 | + untar-main.cpp |
117 | +) |
118 | + |
119 | +add_executable( |
120 | + ${KEEPER_UNTAR_APP_NAME} |
121 | + ${KEEPER_UNTAR_APP_SOURCES} |
122 | +) |
123 | + |
124 | +target_link_libraries( |
125 | + ${KEEPER_UNTAR_APP_NAME} |
126 | + ${LIB_NAME} |
127 | + qdbus-stubs |
128 | + storage-framework |
129 | + backup-helper |
130 | + ${SERVICE_DEPS_LIBRARIES} |
131 | + Qt5::Core |
132 | + Qt5::DBus |
133 | +) |
134 | + |
135 | +## |
136 | +## untar |
137 | +## |
138 | + |
139 | +set_property( |
140 | + SOURCE untar-main.cpp |
141 | + PROPERTIES APPEND_STRING PROPERTY COMPILE_DEFINITIONS APP_NAME=\"${KEEPER_UNTAR_APP_NAME}\" |
142 | +) |
143 | + |
144 | +set(KEEPER_TAR_APP_SOURCES |
145 | + tar-creator-main.cpp |
146 | +) |
147 | + |
148 | +add_executable( |
149 | + ${KEEPER_TAR_APP_NAME} |
150 | + ${KEEPER_TAR_APP_SOURCES} |
151 | +) |
152 | + |
153 | +target_link_libraries( |
154 | + ${KEEPER_TAR_APP_NAME} |
155 | + ${LIB_NAME} |
156 | + qdbus-stubs |
157 | + storage-framework |
158 | + backup-helper |
159 | + ${SERVICE_DEPS_LIBRARIES} |
160 | + Qt5::Core |
161 | + Qt5::DBus |
162 | +) |
163 | + |
164 | +## |
165 | +## |
166 | +## |
167 | |
168 | install( |
169 | TARGETS |
170 | - ${APP_NAME} |
171 | + ${KEEPER_TAR_APP_NAME} |
172 | + ${KEEPER_UNTAR_APP_NAME} |
173 | RUNTIME DESTINATION ${CMAKE_INSTALL_FULL_PKGLIBEXECDIR} |
174 | ) |
175 | |
176 | @@ -49,6 +93,7 @@ |
177 | COVERAGE_REPORT_TARGETS |
178 | ${COVERAGE_REPORT_TARGETS} |
179 | ${LIB_NAME} |
180 | - ${APP_NAME} |
181 | + ${KEEPER_TAR_APP_NAME} |
182 | + ${KEEPER_UNTAR_APP_NAME} |
183 | PARENT_SCOPE |
184 | ) |
185 | |
186 | === renamed file 'src/tar/main.cpp' => 'src/tar/tar-creator-main.cpp' |
187 | === added file 'src/tar/untar-main.cpp' |
188 | --- src/tar/untar-main.cpp 1970-01-01 00:00:00 +0000 |
189 | +++ src/tar/untar-main.cpp 2016-12-16 19:41:21 +0000 |
190 | @@ -0,0 +1,169 @@ |
191 | +/* |
192 | + * Copyright (C) 2016 Canonical, Ltd. |
193 | + * |
194 | + * This program is free software: you can redistribute it and/or modify it |
195 | + * under the terms of the GNU General Public License version 3, as published |
196 | + * by the Free Software Foundation. |
197 | + * |
198 | + * This program is distributed in the hope that it will be useful, but |
199 | + * WITHOUT ANY WARRANTY; without even the implied warranties of |
200 | + * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
201 | + * PURPOSE. See the GNU General Public License for more details. |
202 | + * |
203 | + * You should have received a copy of the GNU General Public License along |
204 | + * with this program. If not, see <http://www.gnu.org/licenses/>. |
205 | + * |
206 | + * Authors: |
207 | + * Charles Kerr <charles.kerr@canonical.com> |
208 | + */ |
209 | + |
210 | +#include "tar/untar.h" |
211 | +#include "qdbus-stubs/dbus-types.h" |
212 | +#include "qdbus-stubs/keeper_helper_interface.h" |
213 | + |
214 | +#include <QCommandLineParser> |
215 | +#include <QCoreApplication> |
216 | +#include <QDebug> |
217 | +#include <QDBusConnection> |
218 | +#include <QDBusConnectionInterface> |
219 | +#include <QDBusUnixFileDescriptor> |
220 | +#include <QFile> |
221 | +#include <QLocalSocket> |
222 | + |
223 | +#include <sys/select.h> |
224 | +#include <unistd.h> |
225 | + |
226 | +#include <cstdio> // fileno() |
227 | +#include <ctime> |
228 | +#include <iostream> |
229 | +#include <type_traits> |
230 | + |
231 | +namespace |
232 | +{ |
233 | + |
234 | +std::tuple<QString> |
235 | +parse_args(QCoreApplication& app) |
236 | +{ |
237 | + // parse the command line |
238 | + QCommandLineParser parser; |
239 | + parser.addHelpOption(); |
240 | + parser.setApplicationDescription( |
241 | + "\n" |
242 | + "The reverse of keeper-tar. Queries Keeper for a socket fd, then pipes\n" |
243 | + "that socket through xzcat and tar to restore the archive data into the current\n" |
244 | + "working directory.\n" |
245 | + "\n" |
246 | + "Helper usage: " APP_NAME " -a /bus/path" |
247 | + ); |
248 | + QCommandLineOption bus_path_option{ |
249 | + QStringList() << "a" << "bus-path", |
250 | + QStringLiteral("Keeper service's DBus path"), |
251 | + QStringLiteral("bus-path") |
252 | + }; |
253 | + parser.addOption(bus_path_option); |
254 | + parser.process(app); |
255 | + const auto bus_path = parser.value(bus_path_option); |
256 | + |
257 | + // gotta have the bus path |
258 | + if (bus_path.isEmpty()) { |
259 | + std::cerr << "Missing required argument: --bus-path" << std::endl; |
260 | + parser.showHelp(EXIT_FAILURE); |
261 | + } |
262 | + |
263 | + return std::make_tuple(bus_path); |
264 | +} |
265 | + |
266 | +QDBusUnixFileDescriptor |
267 | +get_socket_from_keeper(const QString& bus_path) |
268 | +{ |
269 | + QDBusUnixFileDescriptor ret; |
270 | + |
271 | + qDebug() << "asking keeper for a socket"; |
272 | + DBusInterfaceKeeperHelper helperInterface( |
273 | + DBusTypes::KEEPER_SERVICE, |
274 | + bus_path, |
275 | + QDBusConnection::sessionBus() |
276 | + ); |
277 | + |
278 | + auto fd_reply = helperInterface.StartRestore(); |
279 | + fd_reply.waitForFinished(); |
280 | + if (fd_reply.isError()) { |
281 | + qCritical("Call to '%s.StartRestore() at '%s' call failed: %s", |
282 | + DBusTypes::KEEPER_SERVICE, |
283 | + qPrintable(bus_path), |
284 | + qPrintable(fd_reply.error().message()) |
285 | + ); |
286 | + } else { |
287 | + ret = fd_reply.value(); |
288 | + } |
289 | + |
290 | + return ret; |
291 | +} |
292 | + |
293 | +bool |
294 | +untar_from_socket(Untar& untar, int fd) |
295 | +{ |
296 | + bool success = false; |
297 | + static constexpr int STEP_BUFSIZE = 4096*4; // arbitrary |
298 | + char buf[STEP_BUFSIZE]; |
299 | + |
300 | + for (;;) |
301 | + { |
302 | + auto const n_read = read(fd, buf, sizeof(buf)); |
303 | + |
304 | + if (n_read > 0) |
305 | + { |
306 | + if (!untar.step(buf, n_read)) |
307 | + break; |
308 | + } |
309 | + else if (n_read == 0) // eof |
310 | + { |
311 | + qDebug() << Q_FUNC_INFO << "eof reached"; |
312 | + success = true; |
313 | + break; |
314 | + } |
315 | + else if (errno == EAGAIN) |
316 | + { |
317 | + QThread::msleep(100); |
318 | + continue; |
319 | + } |
320 | + else |
321 | + { |
322 | + qCritical() << Q_FUNC_INFO << "read() returned" << strerror(errno); |
323 | + break; |
324 | + } |
325 | + } |
326 | + |
327 | + if (success) |
328 | + success = untar.finish(); |
329 | + |
330 | + return success; |
331 | +} |
332 | + |
333 | +} // anonymous namespace |
334 | + |
335 | +int |
336 | +main(int argc, char **argv) |
337 | +{ |
338 | + QCoreApplication app(argc, argv); |
339 | + |
340 | + // get the inputs |
341 | + QString bus_path; |
342 | + std::tie(bus_path) = parse_args(app); |
343 | + |
344 | + // ask keeper for a socket to read |
345 | + const auto qfd = get_socket_from_keeper(bus_path); |
346 | + if (!qfd.isValid()) { |
347 | + qCritical() << "Can't proceed without a socket from keeper"; |
348 | + return EXIT_FAILURE; |
349 | + } |
350 | + |
351 | + // do it! |
352 | + auto const cwd = QDir::currentPath().toStdString(); |
353 | + Untar untar{cwd}; |
354 | + auto const ret = untar_from_socket(untar, qfd.fileDescriptor()) |
355 | + ? EXIT_SUCCESS |
356 | + : EXIT_FAILURE; |
357 | + qInfo() << Q_FUNC_INFO << "returning" << ret; |
358 | + return ret; |
359 | +} |
360 | |
361 | === added file 'src/tar/untar.cpp' |
362 | --- src/tar/untar.cpp 1970-01-01 00:00:00 +0000 |
363 | +++ src/tar/untar.cpp 2016-12-16 19:41:21 +0000 |
364 | @@ -0,0 +1,144 @@ |
365 | +/* |
366 | + * Copyright (C) 2016 Canonical, Ltd. |
367 | + * |
368 | + * This program is free software: you can redistribute it and/or modify it |
369 | + * under the terms of the GNU General Public License version 3, as published |
370 | + * by the Free Software Foundation. |
371 | + * |
372 | + * This program is distributed in the hope that it will be useful, but |
373 | + * WITHOUT ANY WARRANTY; without even the implied warranties of |
374 | + * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
375 | + * PURPOSE. See the GNU General Public License for more details. |
376 | + * |
377 | + * You should have received a copy of the GNU General Public License along |
378 | + * with this program. If not, see <http://www.gnu.org/licenses/>. |
379 | + * |
380 | + * Authors: |
381 | + * Charles Kerr <charles.kerr@canonical.com> |
382 | + */ |
383 | + |
384 | +#include "tar/untar.h" |
385 | + |
386 | +#include <QDebug> |
387 | +#include <QProcess> |
388 | +#include <QString> |
389 | + |
390 | +#include <string> |
391 | +#include <vector> |
392 | + |
393 | +class Untar::Impl |
394 | +{ |
395 | +public: |
396 | + |
397 | + explicit Impl(std::string const& path) |
398 | + : path_{path} |
399 | + { |
400 | + uncompress_.setStandardOutputProcess(&untar_); |
401 | + uncompress_.start("xz", QStringList{ "--decompress", "--stdout", "--force" }); |
402 | + |
403 | + untar_.start("tar", QStringList{ "-xv", "-C", path_.c_str()}); |
404 | + untar_.setProcessChannelMode(QProcess::ForwardedChannels); |
405 | + } |
406 | + |
407 | + ~Impl() |
408 | + { |
409 | + finish(); |
410 | + } |
411 | + |
412 | + bool step(char const * buf, size_t buflen) |
413 | + { |
414 | + bool success = true; |
415 | + |
416 | + auto n_left = buflen; |
417 | + while (n_left > 0) |
418 | + { |
419 | + auto const n_written_this_pass = uncompress_.write(buf, n_left); |
420 | + if (n_written_this_pass == -1) { |
421 | + qCritical() << Q_FUNC_INFO << strerror(errno); |
422 | + success = false; |
423 | + break; |
424 | + } else { |
425 | + n_left -= n_written_this_pass; |
426 | + buf += n_written_this_pass; |
427 | + } |
428 | + } |
429 | + |
430 | + return success; |
431 | + } |
432 | + |
433 | + bool finish () |
434 | + { |
435 | + bool ok = true; |
436 | + |
437 | + uncompress_.closeWriteChannel(); |
438 | + if (!finish(uncompress_, "xz")) |
439 | + ok = false; |
440 | + |
441 | + if (!finish(untar_, "untar")) |
442 | + ok = false; |
443 | + |
444 | + return ok; |
445 | + } |
446 | + |
447 | +private: |
448 | + |
449 | + bool finish (QProcess& proc, QString const& name) |
450 | + { |
451 | + if (proc.state() != QProcess::NotRunning) |
452 | + { |
453 | + proc.waitForFinished(); |
454 | + } |
455 | + |
456 | + bool ok; |
457 | + |
458 | + if (proc.state() != QProcess::NotRunning) |
459 | + { |
460 | + qCritical() << name << "did not finish"; |
461 | + ok = false; |
462 | + } |
463 | + else if (proc.exitStatus() != QProcess::NormalExit) |
464 | + { |
465 | + qCritical() << name << "exited abnormally"; |
466 | + ok = false; |
467 | + } |
468 | + else if (proc.exitCode() != 0) |
469 | + { |
470 | + qCritical() << name << "exited with error code" << proc.exitCode(); |
471 | + ok = false; |
472 | + } |
473 | + else |
474 | + { |
475 | + qDebug() << name << "finished ok"; |
476 | + ok = true; |
477 | + } |
478 | + |
479 | + return ok; |
480 | + } |
481 | + |
482 | + std::string const path_; |
483 | + QProcess uncompress_; |
484 | + QProcess untar_; |
485 | +}; |
486 | + |
487 | +/** |
488 | +*** |
489 | +**/ |
490 | + |
491 | +Untar::Untar(std::string const& path) |
492 | + : impl_{new Impl{path}} |
493 | +{ |
494 | +} |
495 | + |
496 | +Untar::~Untar() =default; |
497 | + |
498 | +bool |
499 | +Untar::step(char const * buf, size_t buflen) |
500 | +{ |
501 | + return impl_->step(buf, buflen); |
502 | +} |
503 | + |
504 | +bool |
505 | +Untar::finish() |
506 | +{ |
507 | + return impl_->finish(); |
508 | +} |
509 | |
510 | === added file 'src/tar/untar.h' |
511 | --- src/tar/untar.h 1970-01-01 00:00:00 +0000 |
512 | +++ src/tar/untar.h 2016-12-16 19:41:21 +0000 |
513 | @@ -0,0 +1,38 @@ |
514 | +/* |
515 | + * Copyright (C) 2016 Canonical, Ltd. |
516 | + * |
517 | + * This program is free software: you can redistribute it and/or modify it |
518 | + * under the terms of the GNU General Public License version 3, as published |
519 | + * by the Free Software Foundation. |
520 | + * |
521 | + * This program is distributed in the hope that it will be useful, but |
522 | + * WITHOUT ANY WARRANTY; without even the implied warranties of |
523 | + * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
524 | + * PURPOSE. See the GNU General Public License for more details. |
525 | + * |
526 | + * You should have received a copy of the GNU General Public License along |
527 | + * with this program. If not, see <http://www.gnu.org/licenses/>. |
528 | + * |
529 | + * Authors: |
530 | + * Charles Kerr <charles.kerr@canonical.com> |
531 | + */ |
532 | + |
533 | +#pragma once |
534 | + |
535 | +#include <cstddef> // size_t |
536 | +#include <memory> // shared_ptr |
537 | + |
538 | + |
539 | +class Untar |
540 | +{ |
541 | +public: |
542 | + explicit Untar(std::string const& target_path); |
543 | + ~Untar(); |
544 | + bool step(char const * buf, size_t n_bytes); |
545 | + bool finish(); |
546 | + |
547 | +private: |
548 | + class Impl; |
549 | + friend class Impl; |
550 | + std::shared_ptr<Impl> impl_; |
551 | +}; |
552 | |
553 | === modified file 'tests/CMakeLists.txt' |
554 | --- tests/CMakeLists.txt 2016-11-11 14:50:06 +0000 |
555 | +++ tests/CMakeLists.txt 2016-12-16 19:41:21 +0000 |
556 | @@ -29,7 +29,8 @@ |
557 | fake-backup-helper-failure |
558 | ) |
559 | |
560 | -set(KEEPER_TAR_CREATE_BIN ${CMAKE_BINARY_DIR}/src/tar/keeper-tar-create) |
561 | +set(KEEPER_TAR_CREATE_BIN ${CMAKE_BINARY_DIR}/src/tar/keeper-tar) |
562 | +set(KEEPER_UNTAR_BIN ${CMAKE_BINARY_DIR}/src/tar/keeper-untar) |
563 | set(KEEPER_HELPER_TEST_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/helpers-test.sh) |
564 | set(BACKUP_HELPER_FAILURE_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/${BACKUP_HELPER_FAILURE}) |
565 | set(RESTORE_HELPER_TEST_LOCATION ${CMAKE_BINARY_DIR}/tests/fakes/${RESTORE_HELPER}) |
566 | |
567 | === modified file 'tests/com_canonical_keeper.py' |
568 | --- tests/com_canonical_keeper.py 2016-08-03 18:38:12 +0000 |
569 | +++ tests/com_canonical_keeper.py 2016-12-16 19:41:21 +0000 |
570 | @@ -12,6 +12,8 @@ |
571 | '''com.canonical.keeper mock template |
572 | ''' |
573 | |
574 | +# (c) 2016 Canonical Ltd. |
575 | +# |
576 | # This program is free software; you can redistribute it and/or modify it under |
577 | # the terms of the GNU Lesser General Public License as published by the Free |
578 | # Software Foundation; either version 3 of the License, or (at your option) any |
579 | @@ -167,7 +169,8 @@ |
580 | helper_cwd = os.getcwd() |
581 | |
582 | # spawn the helper |
583 | - user.log('starting %s for %s, env %s' % (helper_exec, uuid, henv)) |
584 | + user.log('starting %s for %s in %s, env %s' % |
585 | + (helper_exec, uuid, helper_cwd, henv)) |
586 | user.process = subprocess.Popen( |
587 | [helper_exec, HELPER_PATH], |
588 | env=henv, stdout=sys.stdout, stderr=sys.stderr, |
589 | @@ -180,7 +183,6 @@ |
590 | |
591 | def user_periodic_func(user): |
592 | |
593 | - done = False |
594 | got_data_this_pass = False |
595 | |
596 | if not user.process: |
597 | @@ -189,41 +191,61 @@ |
598 | uuid = user.current_task |
599 | td = user.task_data[uuid] |
600 | |
601 | - # did the helper exit with an error code? |
602 | + # check the socket |
603 | + socket_done = td.n_left == 0 or not td.sock |
604 | + if not socket_done: |
605 | + |
606 | + if td.action == ACTION_SAVING: |
607 | + chunk = td.sock.recv(4096*2) |
608 | + if chunk == '': # eof |
609 | + socket_done = True |
610 | + else: |
611 | + chunk_len = len(chunk) |
612 | + if chunk_len: |
613 | + got_data_this_pass = True |
614 | + td.chunks.append(chunk) |
615 | + td.n_left -= chunk_len |
616 | + |
617 | + if td.action == ACTION_RESTORING: |
618 | + begin = td.n_bytes - td.n_left |
619 | + chunklen = min(td.n_left, 4096*2) |
620 | + end = begin + chunklen |
621 | + chunk = td.blob[begin:end] |
622 | + n_sent = td.sock.send(chunk, socket.MSG_DONTWAIT) |
623 | + if n_sent > 0: |
624 | + got_data_this_pass = True |
625 | + td.n_left -= n_sent |
626 | + |
627 | + user.log('after socket pass, n_left==%s' % (td.n_left)) |
628 | + if td.n_left == 0: |
629 | + user.log('cleaning up socket because no data left') |
630 | + td.sock.shutdown(socket.SHUT_RDWR) |
631 | + td.sock.close() |
632 | + td.sock = None |
633 | + socket_done = True |
634 | + |
635 | + # check the process |
636 | returncode = user.process.poll() |
637 | - if returncode: |
638 | - error = 'helper exited with a returncode of %s' % (str(returncode)) |
639 | - user.log(error) |
640 | - td.error = error |
641 | - td.action = ACTION_FAILED |
642 | - done = True |
643 | - |
644 | - # try to read the socket |
645 | - if td.sock and not td.error: |
646 | - chunk = td.sock.recv(4096*2) |
647 | - chunk_len = len(chunk) |
648 | - if chunk_len: |
649 | - got_data_this_pass = True |
650 | - td.chunks.append(chunk) |
651 | - td.n_left -= chunk_len |
652 | - user.log('got %s more bytes; %s left' % (chunk_len, td.n_left)) |
653 | - if td.n_left <= 0: |
654 | - done = True |
655 | - |
656 | - # if done, clean up the socket |
657 | - if done and td.sock: |
658 | - user.log('cleaning up sock') |
659 | - td.sock.shutdown(socket.SHUT_RDWR) |
660 | - td.sock.close() |
661 | - td.sock = None |
662 | - |
663 | - # if done successfully, save the blob |
664 | - if done and not td.error: |
665 | - user.log('setting blob') |
666 | - blob = b''.join(td.chunks) |
667 | - td.blob = blob |
668 | - user.log('backup %s done; %s bytes' % (uuid, len(blob))) |
669 | - td.action = ACTION_COMPLETE |
670 | + process_done = returncode is not None |
671 | + if process_done: |
672 | + exitmsg = 'helper exited with a returncode of %s' % (str(returncode)) |
673 | + user.log(exitmsg) |
674 | + td.error = exitmsg |
675 | + |
676 | + # are we done yet? |
677 | + done = process_done and socket_done |
678 | + if done: |
679 | + |
680 | + # update td.action |
681 | + if returncode != 0: |
682 | + td.action = ACTION_FAILED |
683 | + else: |
684 | + if td.action == ACTION_SAVING: |
685 | + user.log('setting blob') |
686 | + blob = b''.join(td.chunks) |
687 | + td.blob = blob |
688 | + user.log('backup %s done; %s bytes' % (uuid, len(td.blob))) |
689 | + td.action = ACTION_COMPLETE |
690 | |
691 | # maybe update the task's state |
692 | if done or got_data_this_pass: |
693 | @@ -314,16 +336,13 @@ |
694 | task_state[KEY_PERCENT_DONE] = p |
695 | |
696 | # speed |
697 | - helper = mockobject.objects[HELPER_PATH] |
698 | if uuid == user.current_task: |
699 | n_secs = 2 |
700 | n_bytes = 0 |
701 | too_old = time.time() - n_secs |
702 | for key in td.bytes_per_second: |
703 | - helper.log('key is %s' % (str(key))) |
704 | if key > too_old: |
705 | n_bytes += td.bytes_per_second[key] |
706 | - helper.log('n_bytes is %s' % (str(n_bytes))) |
707 | bytes_per_second = n_bytes / n_secs |
708 | else: |
709 | bytes_per_second = 0 |
710 | @@ -361,7 +380,7 @@ |
711 | |
712 | helper.log("got start_backup request for %s bytes" % (n_bytes)) |
713 | |
714 | - parent, child = socket.socketpair() |
715 | + sock1, sock2 = socket.socketpair() |
716 | |
717 | user = mockobject.objects[USER_PATH] |
718 | uuid = user.current_task |
719 | @@ -369,15 +388,31 @@ |
720 | td = user.task_data[uuid] |
721 | td.n_bytes = n_bytes |
722 | td.n_left = n_bytes |
723 | - td.sock = parent |
724 | + td.sock = sock1 |
725 | |
726 | - return dbus.types.UnixFd(child.fileno()) |
727 | + ret = dbus.types.UnixFd(sock2) |
728 | + sock2.close() |
729 | + return ret |
730 | |
731 | |
732 | def helper_start_restore(helper): |
733 | - helper.parent, child = socket.socketpair() |
734 | - return child |
735 | - |
736 | + |
737 | + user = mockobject.objects[USER_PATH] |
738 | + uuid = user.current_task |
739 | + blob = user.restore_choices[uuid][KEY_BLOB] |
740 | + |
741 | + sock1, sock2 = socket.socketpair() |
742 | + |
743 | + td = user.task_data[uuid] |
744 | + td.blob = bytes([int(byte) for byte in blob]) |
745 | + td.n_bytes = len(td.blob) |
746 | + td.n_left = td.n_bytes |
747 | + td.sock = sock1 |
748 | + td.sock.setblocking(0) |
749 | + |
750 | + ret = dbus.types.UnixFd(sock2) |
751 | + sock2.close() |
752 | + return ret |
753 | |
754 | # |
755 | # Controlling the mock |
756 | |
757 | === modified file 'tests/unit/tar/CMakeLists.txt' |
758 | --- tests/unit/tar/CMakeLists.txt 2016-09-15 16:02:54 +0000 |
759 | +++ tests/unit/tar/CMakeLists.txt 2016-12-16 19:41:21 +0000 |
760 | @@ -26,8 +26,26 @@ |
761 | ${CMAKE_CURRENT_SOURCE_DIR}/ktc-invoke-nofiles.sh.in |
762 | ${KTC_INVOKE_NOFILES} |
763 | ) |
764 | +set( |
765 | + KU_INVOKE |
766 | + ${CMAKE_CURRENT_BINARY_DIR}/ku-invoke.sh |
767 | +) |
768 | +configure_file( |
769 | + ${CMAKE_CURRENT_SOURCE_DIR}/ku-invoke.sh.in |
770 | + ${KU_INVOKE} |
771 | +) |
772 | +set( |
773 | + KU_INVOKE_NOBUS |
774 | + ${CMAKE_CURRENT_BINARY_DIR}/ku-invoke-nobus.sh |
775 | +) |
776 | +configure_file( |
777 | + ${CMAKE_CURRENT_SOURCE_DIR}/ku-invoke-nobus.sh.in |
778 | + ${KU_INVOKE_NOBUS} |
779 | +) |
780 | |
781 | add_definitions( |
782 | + -DKU_INVOKE="${KU_INVOKE}" |
783 | + -DKU_INVOKE_NOBUS="${KU_INVOKE_NOBUS}" |
784 | -DKTC_INVOKE="${KTC_INVOKE}" |
785 | -DKTC_INVOKE_NOBUS="${KTC_INVOKE_NOBUS}" |
786 | -DKTC_INVOKE_NOFILES="${KTC_INVOKE_NOFILES}" |
787 | @@ -56,10 +74,38 @@ |
788 | Qt5::Test |
789 | ) |
790 | |
791 | -#add_test( |
792 | -# ${TAR_CREATOR_TEST} |
793 | -# ${TAR_CREATOR_TEST} |
794 | -#) |
795 | +add_test( |
796 | + ${TAR_CREATOR_TEST} |
797 | + ${TAR_CREATOR_TEST} |
798 | +) |
799 | + |
800 | + |
801 | +# |
802 | +# untar-test |
803 | +# |
804 | + |
805 | +set( |
806 | + UNTAR_TEST |
807 | + untar-test |
808 | +) |
809 | + |
810 | +add_executable( |
811 | + ${UNTAR_TEST} |
812 | + untar-test.cpp |
813 | +) |
814 | + |
815 | +target_link_libraries( |
816 | + ${UNTAR_TEST} |
817 | + ${UNIT_TEST_LIBRARIES} |
818 | + Qt5::Core |
819 | + Qt5::DBus |
820 | + Qt5::Test |
821 | +) |
822 | + |
823 | +add_test( |
824 | + ${UNTAR_TEST} |
825 | + ${UNTAR_TEST} |
826 | +) |
827 | |
828 | |
829 | # |
830 | @@ -102,30 +148,57 @@ |
831 | endforeach(funcname) |
832 | |
833 | # |
834 | -# keeper-tar-create-test |
835 | -# |
836 | - |
837 | -set( |
838 | - KEEPER_TAR_CREATE_TEST |
839 | - keeper-tar-create-test |
840 | -) |
841 | - |
842 | -add_executable( |
843 | - ${KEEPER_TAR_CREATE_TEST} |
844 | - keeper-tar-create-test.cpp |
845 | -) |
846 | - |
847 | -target_link_libraries( |
848 | - ${KEEPER_TAR_CREATE_TEST} |
849 | - ${UNIT_TEST_LIBRARIES} |
850 | - Qt5::Core |
851 | - Qt5::DBus |
852 | - Qt5::Test |
853 | -) |
854 | - |
855 | -add_test( |
856 | - ${KEEPER_TAR_CREATE_TEST} |
857 | - ${KEEPER_TAR_CREATE_TEST} |
858 | +# keeper-tar-test |
859 | +# |
860 | + |
861 | +set( |
862 | + KEEPER_TAR_TEST |
863 | + keeper-tar-test |
864 | +) |
865 | + |
866 | +add_executable( |
867 | + ${KEEPER_TAR_TEST} |
868 | + keeper-tar-test.cpp |
869 | +) |
870 | + |
871 | +target_link_libraries( |
872 | + ${KEEPER_TAR_TEST} |
873 | + ${UNIT_TEST_LIBRARIES} |
874 | + Qt5::Core |
875 | + Qt5::DBus |
876 | + Qt5::Test |
877 | +) |
878 | + |
879 | +add_test( |
880 | + ${KEEPER_TAR_TEST} |
881 | + ${KEEPER_TAR_TEST} |
882 | +) |
883 | + |
884 | +# |
885 | +# keeper-untar-test |
886 | +# |
887 | + |
888 | +set( |
889 | + KEEPER_UNTAR_TEST |
890 | + keeper-untar-test |
891 | +) |
892 | + |
893 | +add_executable( |
894 | + ${KEEPER_UNTAR_TEST} |
895 | + keeper-untar-test.cpp |
896 | +) |
897 | + |
898 | +target_link_libraries( |
899 | + ${KEEPER_UNTAR_TEST} |
900 | + ${UNIT_TEST_LIBRARIES} |
901 | + Qt5::Core |
902 | + Qt5::DBus |
903 | + Qt5::Test |
904 | +) |
905 | + |
906 | +add_test( |
907 | + ${KEEPER_UNTAR_TEST} |
908 | + ${KEEPER_UNTAR_TEST} |
909 | ) |
910 | |
911 | # |
912 | @@ -134,8 +207,10 @@ |
913 | set( |
914 | COVERAGE_TEST_TARGETS |
915 | ${COVERAGE_TEST_TARGETS} |
916 | + ${UNTAR_TEST} |
917 | ${TAR_CREATOR_TEST} |
918 | ${TAR_CREATOR_LIBARCHIVE_FAILURE_TEST} |
919 | - ${KEEPER_TAR_CREATE_TEST} |
920 | + ${KEEPER_TAR_TEST} |
921 | + ${KEEPER_UNTAR_TEST} |
922 | PARENT_SCOPE |
923 | ) |
924 | |
925 | === renamed file 'tests/unit/tar/keeper-tar-create-test.cpp' => 'tests/unit/tar/keeper-tar-test.cpp' |
926 | === added file 'tests/unit/tar/keeper-untar-test.cpp' |
927 | --- tests/unit/tar/keeper-untar-test.cpp 1970-01-01 00:00:00 +0000 |
928 | +++ tests/unit/tar/keeper-untar-test.cpp 2016-12-16 19:41:21 +0000 |
929 | @@ -0,0 +1,232 @@ |
930 | +/* |
931 | + * Copyright 2016 Canonical Ltd. |
932 | + * |
933 | + * This program is free software: you can redistribute it and/or modify it |
934 | + * under the terms of the GNU General Public License version 3, as published |
935 | + * by the Free Software Foundation. |
936 | + * |
937 | + * This program is distributed in the hope that it will be useful, but |
938 | + * WITHOUT ANY WARRANTY; without even the implied warranties of |
939 | + * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
940 | + * PURPOSE. See the GNU General Public License for more details. |
941 | + * |
942 | + * You should have received a copy of the GNU General Public License along |
943 | + * with this program. If not, see <http://www.gnu.org/licenses/>. |
944 | + * |
945 | + * Authors: |
946 | + * Charles Kerr <charles.kerr@canonical.com> |
947 | + */ |
948 | + |
949 | +#include "tests/utils/file-utils.h" |
950 | +#include "tests/utils/keeper-dbusmock-fixture.h" |
951 | + |
952 | +#include "tar/tar-creator.h" |
953 | + |
954 | +#include <gtest/gtest.h> |
955 | + |
956 | +#include <QString> |
957 | +#include <QTemporaryDir> |
958 | + |
959 | + |
960 | +/*** |
961 | +**** |
962 | +***/ |
963 | + |
964 | +class KeeperUntarFixture: public KeeperDBusMockFixture |
965 | +{ |
966 | + using parent = KeeperDBusMockFixture; |
967 | + |
968 | + void SetUp() override |
969 | + { |
970 | + parent::SetUp(); |
971 | + |
972 | + qsrand(unsigned(time(nullptr))); |
973 | + } |
974 | + |
975 | + void TearDown() override |
976 | + { |
977 | + } |
978 | + |
979 | +protected: |
980 | + |
981 | + std::vector<char> tar_directory_into_memory(QString const & in_path) |
982 | + { |
983 | + std::vector<char> blob; |
984 | + |
985 | + auto const currentPath = QDir::currentPath(); |
986 | + EXPECT_TRUE(QDir::setCurrent(in_path)); |
987 | + |
988 | + QDir const indir(in_path); |
989 | + QStringList files; |
990 | + for (auto file : FileUtils::getFilesRecursively(in_path)) |
991 | + files += indir.relativeFilePath(file); |
992 | + |
993 | + TarCreator tar_creator(files, false); |
994 | + std::vector<char> step; |
995 | + while (tar_creator.step(step)) |
996 | + blob.insert(blob.end(), step.begin(), step.end()); |
997 | + |
998 | + EXPECT_TRUE(QDir::setCurrent(currentPath)); |
999 | + |
1000 | + return blob; |
1001 | + } |
1002 | + |
1003 | + QMap<QString,QVariant> build_folder_restore_choice( |
1004 | + QTemporaryDir& source, |
1005 | + QTemporaryDir& target, |
1006 | + char const* helper_exec, |
1007 | + std::vector<char> const& blob) |
1008 | + { |
1009 | + |
1010 | + return QMap<QString,QVariant>{ |
1011 | + { KEY_NAME, QDir(source.path()).dirName() }, |
1012 | + { KEY_TYPE, QStringLiteral("folder") }, |
1013 | + { KEY_SUBTYPE, target.path() }, |
1014 | + { KEY_HELPER, QString::fromUtf8(helper_exec) }, |
1015 | + { KEY_SIZE, quint64(blob.size()) }, |
1016 | + { KEY_CTIME, quint64(time(nullptr)) }, |
1017 | + { KEY_BLOB, QByteArray{&blob.front(), int(blob.size())} } |
1018 | + }; |
1019 | + } |
1020 | + |
1021 | + void restore(QString const& uuid) |
1022 | + { |
1023 | + QDBusReply<void> reply = user_iface_->call("StartRestore", QStringList{uuid}); |
1024 | + ASSERT_TRUE(reply.isValid()) << qPrintable(reply.error().message()); |
1025 | + ASSERT_TRUE(wait_for_tasks_to_finish()); |
1026 | + } |
1027 | +}; |
1028 | + |
1029 | +/*** |
1030 | +**** |
1031 | +***/ |
1032 | + |
1033 | +TEST_F(KeeperUntarFixture, RestoreRun) |
1034 | +{ |
1035 | + static constexpr int n_runs {2}; |
1036 | + |
1037 | + for (int i=0; i<n_runs; ++i) |
1038 | + { |
1039 | + // build a directory full of random files |
1040 | + QTemporaryDir in; |
1041 | + FileUtils::fillTemporaryDirectory(in.path()); |
1042 | + |
1043 | + // tar it up |
1044 | + auto const blob = tar_directory_into_memory(in.path()); |
1045 | + |
1046 | + // tell keeper that's a restore choice |
1047 | + QTemporaryDir out; |
1048 | + const auto uuid = add_restore_choice(build_folder_restore_choice(in, out, KU_INVOKE, blob)); |
1049 | + |
1050 | + // now run the restore |
1051 | + restore(uuid); |
1052 | + |
1053 | + // after restore, the source and restore dirs should match |
1054 | + EXPECT_TRUE(FileUtils::compareDirectories(in.path(), out.path())); |
1055 | + |
1056 | + // if the test failed, keep the artifacts for a human to look at |
1057 | + auto const passed = ::testing::UnitTest::GetInstance()->current_test_info()->result()->Passed(); |
1058 | + in.setAutoRemove(passed); |
1059 | + out.setAutoRemove(passed); |
1060 | + } |
1061 | +} |
1062 | + |
1063 | +/*** |
1064 | +**** |
1065 | +***/ |
1066 | + |
1067 | +TEST_F(KeeperUntarFixture, BadArgNoBus) |
1068 | +{ |
1069 | + static constexpr int n_runs {1}; |
1070 | + |
1071 | + for (int i=0; i<n_runs; ++i) |
1072 | + { |
1073 | + // build a directory full of random files |
1074 | + QTemporaryDir in; |
1075 | + FileUtils::fillTemporaryDirectory(in.path()); |
1076 | + |
1077 | + // tar it up |
1078 | + auto const blob = tar_directory_into_memory(in.path()); |
1079 | + |
1080 | + // tell keeper that's a restore choice |
1081 | + QTemporaryDir out; |
1082 | + const auto uuid = add_restore_choice(build_folder_restore_choice(in, out, KU_INVOKE_NOBUS, blob)); |
1083 | + |
1084 | + // now run the restore |
1085 | + restore(uuid); |
1086 | + |
1087 | + // confirm that the backup ended in error |
1088 | + const auto state = user_iface_->state(); |
1089 | + const auto& properties = state[uuid]; |
1090 | + EXPECT_EQ(QString::fromUtf8("failed"), properties.value(KEY_ACTION)) |
1091 | + << qPrintable(properties.value(KEY_ACTION).toString()); |
1092 | + EXPECT_FALSE(properties.value(KEY_ERROR).toString().isEmpty()); |
1093 | + } |
1094 | +} |
1095 | + |
1096 | +/*** |
1097 | +**** |
1098 | +***/ |
1099 | + |
1100 | +TEST_F(KeeperUntarFixture, BadData) |
1101 | +{ |
1102 | + static constexpr int n_runs {1}; |
1103 | + |
1104 | + for (int i=0; i<n_runs; ++i) |
1105 | + { |
1106 | + // build a directory full of random files |
1107 | + QTemporaryDir in; |
1108 | + FileUtils::fillTemporaryDirectory(in.path()); |
1109 | + |
1110 | + // make a junk blob |
1111 | + std::vector<char> blob { 'n', 'o', 't', 'a', 't', 'a', 'r' }; |
1112 | + |
1113 | + // tell keeper that's a restore choice |
1114 | + QTemporaryDir out; |
1115 | + const auto uuid = add_restore_choice(build_folder_restore_choice(in, out, KU_INVOKE, blob)); |
1116 | + |
1117 | + // now run the restore |
1118 | + restore(uuid); |
1119 | + |
1120 | + // confirm that the backup ended in error |
1121 | + const auto state = user_iface_->state(); |
1122 | + const auto& properties = state[uuid]; |
1123 | + EXPECT_EQ(QString::fromUtf8("failed"), properties.value(KEY_ACTION)) |
1124 | + << qPrintable(properties.value(KEY_ACTION).toString()); |
1125 | + EXPECT_FALSE(properties.value(KEY_ERROR).toString().isEmpty()); |
1126 | + } |
1127 | +} |
1128 | + |
1129 | +/*** |
1130 | +**** |
1131 | +***/ |
1132 | + |
1133 | +TEST_F(KeeperUntarFixture, IncompleteData) |
1134 | +{ |
1135 | + static constexpr int n_runs {1}; |
1136 | + |
1137 | + for (int i=0; i<n_runs; ++i) |
1138 | + { |
1139 | + // build a directory full of random files |
1140 | + QTemporaryDir in; |
1141 | + FileUtils::fillTemporaryDirectory(in.path()); |
1142 | + |
1143 | + // make a truncated backup of it |
1144 | + auto blob = tar_directory_into_memory(in.path()); |
1145 | + blob.resize(511); |
1146 | + |
1147 | + // tell keeper that's a restore choice |
1148 | + QTemporaryDir out; |
1149 | + const auto uuid = add_restore_choice(build_folder_restore_choice(in, out, KU_INVOKE, blob)); |
1150 | + |
1151 | + // now run the restore |
1152 | + restore(uuid); |
1153 | + |
1154 | + // confirm that the backup ended in error |
1155 | + const auto state = user_iface_->state(); |
1156 | + const auto& properties = state[uuid]; |
1157 | + EXPECT_EQ(QString::fromUtf8("failed"), properties.value(KEY_ACTION)) |
1158 | + << qPrintable(properties.value(KEY_ACTION).toString()); |
1159 | + EXPECT_FALSE(properties.value(KEY_ERROR).toString().isEmpty()); |
1160 | + } |
1161 | +} |
1162 | |
1163 | === added file 'tests/unit/tar/ku-invoke-nobus.sh.in' |
1164 | --- tests/unit/tar/ku-invoke-nobus.sh.in 1970-01-01 00:00:00 +0000 |
1165 | +++ tests/unit/tar/ku-invoke-nobus.sh.in 2016-12-16 19:41:21 +0000 |
1166 | @@ -0,0 +1,1 @@ |
1167 | +@KEEPER_UNTAR_BIN@ |
1168 | |
1169 | === added file 'tests/unit/tar/ku-invoke.sh.in' |
1170 | --- tests/unit/tar/ku-invoke.sh.in 1970-01-01 00:00:00 +0000 |
1171 | +++ tests/unit/tar/ku-invoke.sh.in 2016-12-16 19:41:21 +0000 |
1172 | @@ -0,0 +1,1 @@ |
1173 | +@KEEPER_UNTAR_BIN@ -a /com/canonical/keeper/helper |
1174 | |
1175 | === added file 'tests/unit/tar/untar-test.cpp' |
1176 | --- tests/unit/tar/untar-test.cpp 1970-01-01 00:00:00 +0000 |
1177 | +++ tests/unit/tar/untar-test.cpp 2016-12-16 19:41:21 +0000 |
1178 | @@ -0,0 +1,120 @@ |
1179 | +/* |
1180 | + * Copyright 2016 Canonical Ltd. |
1181 | + * |
1182 | + * This program is free software: you can redistribute it and/or modify it |
1183 | + * under the terms of the GNU General Public License version 3, as published |
1184 | + * by the Free Software Foundation. |
1185 | + * |
1186 | + * This program is distributed in the hope that it will be useful, but |
1187 | + * WITHOUT ANY WARRANTY; without even the implied warranties of |
1188 | + * MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR |
1189 | + * PURPOSE. See the GNU General Public License for more details. |
1190 | + * |
1191 | + * You should have received a copy of the GNU General Public License along |
1192 | + * with this program. If not, see <http://www.gnu.org/licenses/>. |
1193 | + * |
1194 | + * Authors: |
1195 | + * Charles Kerr <charles.kerr@canonical.com> |
1196 | + */ |
1197 | + |
1198 | +#include "tests/utils/file-utils.h" |
1199 | + |
1200 | +#include "tar/tar-creator.h" |
1201 | +#include "tar/untar.h" |
1202 | + |
1203 | +#include <gtest/gtest.h> |
1204 | + |
1205 | +#include <QDebug> |
1206 | +#include <QDir> |
1207 | +#include <QFile> |
1208 | +#include <QFileInfo> |
1209 | +#include <QProcess> |
1210 | +#include <QString> |
1211 | +#include <QTemporaryDir> |
1212 | + |
1213 | +#include <algorithm> |
1214 | +#include <array> |
1215 | +#include <cstdio> |
1216 | + |
1217 | +class UntarFixture: public ::testing::Test |
1218 | +{ |
1219 | +protected: |
1220 | + |
1221 | + void SetUp() override |
1222 | + { |
1223 | + qsrand(unsigned(time(nullptr))); |
1224 | + } |
1225 | + |
1226 | + void TearDown() override |
1227 | + { |
1228 | + } |
1229 | + |
1230 | +}; |
1231 | + |
1232 | +/*** |
1233 | +**** |
1234 | +***/ |
1235 | + |
1236 | +TEST_F(UntarFixture, Untar) |
1237 | +{ |
1238 | + static constexpr int n_runs {5}; |
1239 | + static constexpr std::array<size_t,4> step_sizes = { 1024, 2048, 4096, INT_MAX }; |
1240 | + //static constexpr std::array<int,1> step_sizes = { 1024 }; |
1241 | + |
1242 | + for (int i=0; i<n_runs; ++i) |
1243 | + { |
1244 | + // build a directory full of random files |
1245 | + QTemporaryDir in; |
1246 | + QDir indir(in.path()); |
1247 | + //FileUtils::fillTemporaryDirectory(in.path()); |
1248 | + FileUtils::fillTemporaryDirectory(in.path(), 3, 3, 4096, 1); |
1249 | + |
1250 | + // tar it up |
1251 | + std::vector<char> contents; |
1252 | + { |
1253 | + EXPECT_TRUE(QDir::setCurrent(in.path())); |
1254 | + QStringList files; |
1255 | + for (auto file : FileUtils::getFilesRecursively(in.path())) |
1256 | + files += indir.relativeFilePath(file); |
1257 | + TarCreator tar_creator(files, false); |
1258 | + std::vector<char> step; |
1259 | + while (tar_creator.step(step)) |
1260 | + contents.insert(contents.end(), step.begin(), step.end()); |
1261 | + } |
1262 | + |
1263 | + // walk through an untar test for each of the step sizes |
1264 | + for (auto const& step_size : step_sizes) |
1265 | + { |
1266 | + char const * walk = &contents.front(); |
1267 | + auto n_left = contents.size(); |
1268 | + |
1269 | + // untar it |
1270 | + QTemporaryDir out; |
1271 | + QDir outdir(out.path()); |
1272 | + |
1273 | + { |
1274 | + Untar untar(out.path().toStdString()); |
1275 | + do |
1276 | + { |
1277 | + auto const current_step_size = std::min(step_size, n_left); |
1278 | + EXPECT_TRUE(untar.step(walk, current_step_size)); |
1279 | + n_left -= current_step_size; |
1280 | + walk += current_step_size; |
1281 | + } |
1282 | + while(n_left > 0); |
1283 | + EXPECT_TRUE(untar.finish()); |
1284 | + } |
1285 | + |
1286 | + // compare it to the original |
1287 | + EXPECT_TRUE(FileUtils::compareDirectories(in.path(), out.path())); |
1288 | + |
1289 | + // if the test failed, keep the outdir for manual inspection |
1290 | + auto const passed = ::testing::UnitTest::GetInstance()->current_test_info()->result()->Passed(); |
1291 | + out.setAutoRemove(passed); |
1292 | + } |
1293 | + |
1294 | + // if the test failed, keep the indir for manual inspection |
1295 | + auto const passed = ::testing::UnitTest::GetInstance()->current_test_info()->result()->Passed(); |
1296 | + in.setAutoRemove(passed); |
1297 | + } |
1298 | +} |
1299 | |
1300 | === modified file 'tests/utils/file-utils.cpp' |
1301 | --- tests/utils/file-utils.cpp 2016-11-22 09:37:40 +0000 |
1302 | +++ tests/utils/file-utils.cpp 2016-12-16 19:41:21 +0000 |
1303 | @@ -38,7 +38,8 @@ |
1304 | create_dummy_string() |
1305 | { |
1306 | // NB we want to exercise long filenames, but this cutoff length is arbitrary |
1307 | - static constexpr int MAX_BASENAME_LEN {200}; |
1308 | + //static constexpr int MAX_BASENAME_LEN {200}; |
1309 | + static constexpr int MAX_BASENAME_LEN {10}; |
1310 | auto const filename_len = std::max(10, qrand() % MAX_BASENAME_LEN); |
1311 | QString str; |
1312 | for (int i=0; i<filename_len; ++i) |
1313 | |
1314 | === modified file 'tests/utils/file-utils.h' |
1315 | --- tests/utils/file-utils.h 2016-09-12 15:28:06 +0000 |
1316 | +++ tests/utils/file-utils.h 2016-12-16 19:41:21 +0000 |
1317 | @@ -33,4 +33,4 @@ |
1318 | bool checkPathIsDir(QString const & dirPath); |
1319 | |
1320 | QStringList getFilesRecursively(QString const & dirPath); |
1321 | -}; |
1322 | +} |
1323 | |
1324 | === modified file 'tests/utils/keeper-dbusmock-fixture.h' |
1325 | --- tests/utils/keeper-dbusmock-fixture.h 2016-09-15 16:02:54 +0000 |
1326 | +++ tests/utils/keeper-dbusmock-fixture.h 2016-12-16 19:41:21 +0000 |
1327 | @@ -142,6 +142,7 @@ |
1328 | bool all_done = true; |
1329 | for(const auto& properties : state) { |
1330 | const auto action = properties.value(KEY_ACTION); |
1331 | + qInfo() << Q_FUNC_INFO << "action is" << action; |
1332 | bool task_done = (action == ACTION_CANCELLED) || (action == ACTION_FAILED) || (action == ACTION_COMPLETE); |
1333 | if (!task_done) |
1334 | all_done = false; |
1335 | @@ -160,6 +161,14 @@ |
1336 | return uuid; |
1337 | } |
1338 | |
1339 | + QString add_restore_choice(const QMap<QString,QVariant>& properties) |
1340 | + { |
1341 | + const auto uuid = QUuid::createUuid().toString(); |
1342 | + auto msg = mock_iface_->call(QStringLiteral("AddRestoreChoice"), uuid, properties); |
1343 | + EXPECT_NE(QDBusMessage::ErrorMessage, msg.type()) << qPrintable(msg.errorMessage()); |
1344 | + return uuid; |
1345 | + } |
1346 | + |
1347 | void fail_next_helper_start() |
1348 | { |
1349 | auto msg = mock_iface_->call(QStringLiteral("FailNextHelperStart")); |
1350 | |
1351 | === modified file 'tests/utils/main.cpp' |
1352 | --- tests/utils/main.cpp 2016-08-02 00:04:53 +0000 |
1353 | +++ tests/utils/main.cpp 2016-12-16 19:41:21 +0000 |
1354 | @@ -52,7 +52,7 @@ |
1355 | |
1356 | qInstallMessageHandler(util::loggingFunction); |
1357 | |
1358 | - qsrand(time(nullptr)); |
1359 | + qsrand(unsigned(time(nullptr))); |
1360 | |
1361 | QCoreApplication application(argc, argv); |
1362 | DBusMock::registerMetaTypes(); |
FAILED: Continuous integration, rev:148 /jenkins. canonical. com/unity- api-1/job/ lp-keeper- ci/148/ /jenkins. canonical. com/unity- api-1/job/ build/1266/ console /jenkins. canonical. com/unity- api-1/job/ build-0- fetch/1273/ console
https:/
Executed test runs:
FAILURE: https:/
FAILURE: https:/
Click here to trigger a rebuild: /jenkins. canonical. com/unity- api-1/job/ lp-keeper- ci/148/ rebuild
https:/