Please add Userspace Packages for NVDIMM support
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Ubuntu |
Fix Released
|
Medium
|
Andreas Hasenack |
Bug Description
The support for Intel NVDIMM technology requires both kernel and userspace components. The kernel components have landed in 4.15 and will be available in Bionic. In order for users to take advantage of NVDIMMs in their deployments, the following userspace packages are also needed in Bionic:
PMDK 1.4(Formery NVML) has been released (https:/
Updated RPM spec is available here:
http://
Tools for management of NV devices and device pools.
ndctl -- Utility library for managing the libnvdimm (non-volatile memory device) sub-system in the Linux kernel.
Source Code: https:/
Please use this PPA for testing:
https:/
ppa:canonical-
tags: | added: needs-packaging |
Changed in ubuntu: | |
assignee: | nobody → Nish Aravamudan (nacc) |
status: | New → In Progress |
importance: | Undecided → Medium |
Nish Aravamudan (nacc) wrote : | #1 |
pragyansri.pathi@intel.com (pragyan) wrote : | #2 |
Update from NVML Team Testing from Marcin Slusarz:
Several issues found, some critical. List in severity order:
1)Libpmem-dev is not installable, because one of the files also exists in libpmem.
2)Libpmemobj-dev package doesn't install all C header files, making it completely useless.
3)Libpmemobj C++ headers and documentation are not packaged at all.
4)Librpmem is not packaged.
5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do not depend on libpmem-dev (pkg-config files won't work correctly without libpmem-dev).
6)Libraries are installed to /usr/lib instead of /usr/lib/
7)Pkg-config files in all -dev packages point to /usr/local.
This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.
8)"NVML library" (NVM library library) in package descriptions looks a bit silly.
9)Homepage points to github, instead of http://
Nish Aravamudan (nacc) wrote : Re: [Bug 1752378] Re: Please add Userspace Packages for NVDIMM support | #3 |
Thank you for the feedback, I will work on getting this resolved as
soon as I can.
-Nish
On Fri, Mar 2, 2018 at 9:39 AM, <email address hidden>
<email address hidden> wrote:
> Update from NVML Team Testing from Marcin Slusarz:
>
> Several issues found, some critical. List in severity order:
> 1)Libpmem-dev is not installable, because one of the files also exists in libpmem.
>
> 2)Libpmemobj-dev package doesn't install all C header files, making it
> completely useless.
>
> 3)Libpmemobj C++ headers and documentation are not packaged at all.
>
> 4)Librpmem is not packaged.
>
> 5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do
> not depend on libpmem-dev (pkg-config files won't work correctly without
> libpmem-dev).
>
> 6)Libraries are installed to /usr/lib instead of /usr/lib/
> gnu/.
>
> 7)Pkg-config files in all -dev packages point to /usr/local.
> This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.
>
> 8)"NVML library" (NVM library library) in package descriptions looks a
> bit silly.
>
> 9)Homepage points to github, instead of http://
>
> --
> You received this bug notification because you are a bug assignee.
> https:/
>
> Title:
> Please add Userspace Packages for NVDIMM support
>
> To manage notifications about this bug go to:
> https:/
>
> Launchpad-
> Launchpad-Bug: distribution=
> Launchpad-Bug-Tags: needs-packaging
> Launchpad-
> Launchpad-
> Launchpad-
> Launchpad-
> Launchpad-
> Launchpad-
> Launchpad-
> Launchpad-
pragyansri.pathi@intel.com (pragyan) wrote : | #4 |
Update for NDCTL Testing:
1. The ndctl package is missing the man pages.
2. The libraries and the development packages should be split into separate sub-packages i.e. ndctl, libndctl, libndctl-devel, daxctl, libdaxctl, and libdaxctl-devel
Nish Aravamudan (nacc) wrote : | #5 |
On Fri, Mar 2, 2018 at 9:39 AM, <email address hidden>
<email address hidden> wrote:
> Update from NVML Team Testing from Marcin Slusarz:
>
> Several issues found, some critical. List in severity order:
> 1)Libpmem-dev is not installable, because one of the files also exists in libpmem.
Fixed, I think. (all the -dev packages had this problem)
> 2)Libpmemobj-dev package doesn't install all C header files, making it
> completely useless.
Fixed.
> 3)Libpmemobj C++ headers and documentation are not packaged at all.
Fixed for the headers, I need to figure out the documetnation bit still.
> 4)Librpmem is not packaged.
Fixed
> 5)Libpmemblk-dev, libpmemlog-dev, libpmempool-dev and libpmemobj-dev do
> not depend on libpmem-dev (pkg-config files won't work correctly without
> libpmem-dev).
Fixed
> 6)Libraries are installed to /usr/lib instead of /usr/lib/
> gnu/.
I think fixed.
> 7)Pkg-config files in all -dev packages point to /usr/local.
> This is actually a bug in pmdk, which we already fixed on master. I think it doesn't matter much, but it's worth backporting it for 1.3.2.
Can you point me at the commit in master? I'm not seeing it.
> 8)"NVML library" (NVM library library) in package descriptions looks a
> bit silly.
Fixed (what verbiage does Intel actually want?)
> 9)Homepage points to github, instead of http://
Fixed
Nish Aravamudan (nacc) wrote : | #6 |
On Mon, Mar 5, 2018 at 9:30 AM, <email address hidden>
<email address hidden> wrote:
> Update for NDCTL Testing:
> 1. The ndctl package is missing the man pages.
Fixed.
> 2. The libraries and the development packages should be split into separate sub-packages i.e. ndctl, libndctl, libndctl-devel, daxctl, libdaxctl, and libdaxctl-devel
Fixed.
Marcin Ślusarz (mslusarz) wrote : | #7 |
NVML:
1) Nope, for old packages only libpmem had this problem. Now it's even worse - none of the packages contain .so used for linking.
3) "detail" directory is still missing.
7) Already backported to stable-1.3 branch, will be part of 1.3.2: https:/
8) I don't care that much as long as it makes sense :)
You can see what we came up with here: https:/
"NVML library" was a common mistake. In 1.4 this will no longer be a problem, because we renamed NVML to PMDK (Persistent Memory Development Kit)
Nish Aravamudan (nacc) wrote : | #8 |
On Tue, Mar 6, 2018 at 2:50 AM, Marcin Ślusarz
<email address hidden> wrote:
> NVML:
> 1) Nope, for old packages only libpmem had this problem. Now it's even worse - none of the packages contain .so used for linking.
Fixed
> 3) "detail" directory is still missing.
Should be fixed.
> 7) Already backported to stable-1.3 branch, will be part of 1.3.2: https:/
Backported to the package.
> 8) I don't care that much as long as it makes sense :)
> You can see what we came up with here: https:/
> "NVML library" was a common mistake. In 1.4 this will no longer be a problem, because we renamed NVML to PMDK (Persistent Memory Development Kit)
Not sure why "NVML ... library" is better than "NVML library for ..." :)
Marcin Ślusarz (mslusarz) wrote : | #9 |
1) Still nope.
Non-dev packages now also include .so files...
10) New issue: libpmem-dev now contains all libraries in nvml_dbg directory.
Nish Aravamudan (nacc) wrote : | #10 |
On Wed, Mar 7, 2018 at 8:52 AM, Marcin Ślusarz
<email address hidden> wrote:
> 1) Still nope.
> Non-dev packages now also include .so files...
>
> 10) New issue: libpmem-dev now contains all libraries in nvml_dbg
> directory.
I think these are resolved with ~ppa5.
Changed in ubuntu: | |
assignee: | Nish Aravamudan (nacc) → Andreas Hasenack (ahasenack) |
Andreas Hasenack (ahasenack) wrote : | #11 |
Hi,
we switched over to a team-owned PPA:
https:/
ppa:canonical-
Please use this one from now on. At this moment, they contain the latest debs from Nish that I just re-uploaded there.
description: | updated |
Andreas Hasenack (ahasenack) wrote : | #12 |
I'm going over the lintian warnings now, which include some of the issues raised already during testing.
Andreas Hasenack (ahasenack) wrote : | #13 |
New ndctl upload to the canonical-server PPA at https:/
ndctl (59.2-0ubuntu1~
* Install pkgconfig files only in the -dev library packages
* Don't install .la files, pkgconfig is preferred
* lib*.so go into -dev packages, lib*.so.* into non-dev
* Update long description in -dev packages
* Append major soname version to non-dev library packages
* Other lintian fixes
Please test.
I'll work on the nvml package now.
Andreas Hasenack (ahasenack) wrote : | #14 |
I just pushed nvml_1.
nvml (1.3.1-
.
* updated package descriptions
* disable rpath
* remove trigger for man-db
* lintian: change location of bash completion file
Please test.
A big chunk of the remaining lintian errors are about copyrights. I'll work on that next, and any feedback you may have so far.
Andreas Hasenack (ahasenack) wrote : | #15 |
Hi, a question. What are the files under /usr/lib/
$ dpkg --contents librpmem-
drwxr-xr-x root/root 0 2018-03-21 20:00 ./
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/debug/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/lib/
-rw-r--r-- root/root 75336 2018-03-21 20:00 ./usr/lib/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/share/
drwxr-xr-x root/root 0 2018-03-21 20:00 ./usr/share/doc/
lrwxrwxrwx root/root 0 2018-03-21 20:00 ./usr/share/
Marcin Ślusarz (mslusarz) wrote : | #16 |
Files under nvml_dbg are builds with debugging symbols, logging, asserts and expensive checks that we normally don't want users to run with.
Andreas Hasenack (ahasenack) wrote : | #17 |
Would you expect end users to need the files from that nvml_dbg directory?
In the current packaging, these go into the corresponding libfoo-dev packaages, but that's not really correct from a packaging perspective. At most they should be in some new package, something like libfoo-
Ubuntu already produces packages with debugging symbols for all builds, so that aspect is covered and wouldn't be lost. What these won't have is the extra logging, asserts and expensive checks that you mentioned, which I assume cannot be enabled at runtime with an environment variable for example.
Andreas Hasenack (ahasenack) wrote : | #18 |
I guess what I am asking is if we can drop the nvml_dbg directory from the dev packages :)
Marcin Ślusarz (mslusarz) wrote : | #19 |
I understand your concern, but I'd like to ask you to not remove it yet, because:
- it would make responding to bug reports and support requests a bit harder
- shipped man pages mention those libraries
But I think you are right, so I opened an issue with the proposal to get rid of nvml_dbg/pmdk_dbg directory:
https:/
Marcin Ślusarz (mslusarz) wrote : | #20 |
It seems almost all problems were resolved. Thanks.
The only missing part is libpmemobj++ documentation.
Andreas Hasenack (ahasenack) wrote : | #21 |
Ok, adding docs.
Andreas Hasenack (ahasenack) wrote : | #22 |
Just pushed nvml 1.3.1-0ubuntu1~ppa7 to the PPA:
nvml (1.3.1-
* Add a lintian override for the manpages.
* Add a lintian override for dev packages regarding ldconfig because of
the _dbg subdirectory.
* Install docs for libpmemobj and fix its directory to match the package
name
The ldconfig lintian override is about the _dbg directory we talked about. I don't know yet if an Archive Admin will accept that, we will find out. I added links to this bug and to the issue you opened in github.
I also renamed the libpmemobj++ documentation directory. The package is named libpmemobj-dev, but make install was installing the docs to /usr/share/
I'm going to check the debian/copyright file on ndctl now, I think it needs a bit of work. I've seen at least one case where a file has an lgpl license but d/copyright claims it's gpl.
Marcin Ślusarz (mslusarz) wrote : | #23 |
Alternatively you could add separate package for C++ headers, pkg-config file and documentation.
Andreas Hasenack (ahasenack) wrote : | #24 |
You mean, create another set of packages for the *_dbg/ files? That would mean a mirror package for every libfoo<soname> and libfoo-dev package, right?
Marcin Ślusarz (mslusarz) wrote : | #25 |
I suggested it in the context of "libpmemobj++ documentation directory" problem.
Andreas Hasenack (ahasenack) wrote : | #26 |
I pushed another ndctl build to the ppa, but the only change is an update to the debian/copyright file:
ndctl (59.2-0ubuntu1~
* d/copyright update
Andreas Hasenack (ahasenack) wrote : | #27 |
Hi,
it just occurred to me that we never had a comment in this bug about the packaged software actually works correctly with the hardware it was meant for.
Could you please add a comment here saying that these packages are working correctly with your hardware? And that the devel packages are correct in the sense that you can link your applications using them?
Thanks
Dan Williams (djbw) wrote : | #28 |
The ndctl package is working for me, I verified listing namespaces and converting namespaces between a few different modes.
The only cosmetic issue is that the version does not match the package version. By default git-version tries to do the right thing based on the tag, but if you're building without the git tree it may fall back to the default defined version.
E.g.:
$ dpkg-query -l ndctl
Desired=
| Status=
|/ Err?=(none)
||/ Name Version Architecture Description
+++-===
ii ndctl 59.2-0ubuntu1 amd64 Utility for managing the libnvdimm subsy
$ ndctl version
59.1+
Dan Williams (djbw) wrote : | #29 |
Fyi, you can create an emulated persistent memory environment with the memmap=ss!nn command line option. For example I tested this build with: memmap=512M!4G
Andreas Hasenack (ahasenack) wrote : | #30 |
Ok, what about the nvml packages? Any testers out there?
I'll take a look at the version issue, but we should not block on it as that can be fixed later.
Marcin Ślusarz (mslusarz) wrote : | #31 |
I asked some people internally to test those packages. I think we'll get results tomorrow.
While waiting on that I tried to build pmemfile project (https:/
-- Checking for module 'libpmemobj>=1.3'
-- Requested 'libpmemobj >= 1.3' but version of libpmemobj is
This is because "version" field in all .pc files is empty.
Marcin Ślusarz (mslusarz) wrote : | #32 |
This is actually a bug in NVML/PMDK. It seems nobody tried to use pkg-config files with minimum version specified for packages built out of git tree.
Temporarily you can work around it by creating ".version" file in the top level directory with "1.3.1" as content.
Andreas Hasenack (ahasenack) wrote : | #33 |
Could you open a github issue for this and link it here please?
Marcin Ślusarz (mslusarz) wrote : | #34 |
I fixed this today. stable-1.3 fix:
https:/
Testing those packages is taking longer than I expected. I'll report back as soon as I get the results.
However without minimum version check, pmemfile builds fine and its tests pass, so I don't expect full test suite will find anything.
(FYI, I'll be out of office until Wednesday)
Andreas Hasenack (ahasenack) wrote : | #35 |
Hi Marcin,
I'm planning on splitting the *_dbg directories out into their own packages. Including runtime libraries from /usr/lib/
I'm still working out the details, but it should be more or less like this (using libpmem as an example):
libpmem1: continues as is
libpmem-dev: no longer includes /usr/lib/
libpmem1-debug: ships /usr/lib/
libpmem-dev-debug: ships /usr/lib/
Is the separate dev package necessary? I.e., do you need the special debug library build when linking with your applications and wanting the extra logging? Or can apps keep linking against the normal library, and just set LD_LIBRARY_PATH when they need extra logging? I assume the latter, so the dev-debug package wouldn't be necessary.
Enjoy your holidays. I'm off on Friday (March 30th) but back on Monday.
Andreas Hasenack (ahasenack) wrote : | #36 |
Ok, new layout done: https:/
Pushing to the ppa:
nvml (1.3.1-
* Drop the manpage lintian override and fix the problem instead.
* Create new -debug packages for the *_dbg/ libraries, document it
in README.source and drop the ldconfig lintian override.
* Patch the devel manpages explaining that the extra debug libraries
are in the -debug packages.
There was no consideration done for the upgrade from previous versions of these packages, you may have to remove and reinstall if you get conflicts or stray files.
These are the git repositories I've been working with, I don't remember if I mentioned this before:
nvml: https:/
ndctl: https:/
Andreas Hasenack (ahasenack) wrote : | #37 |
And now I uploaded ppa9 with the pkgconfig fix you committed upstream:
nvml (1.3.1-
.
* d/p/SRCVERSION-
the pkgconfig files (see
https:/
https:/
Andreas Hasenack (ahasenack) wrote : | #38 |
I don't think versioning is working correctly when this is built from a git tree that is not upstream. For example, I build this from my packaging branch, and I get:
ubuntu@
pmempool debian/
ubuntu@
version=
libdir=
prefix=/usr
includedir=
Name: libvmmalloc
Description: libvmmalloc library from NVML project
Version: ${version}
URL: http://
Requires:
Libs: -L${libdir} -lvmmalloc
Cflags: -I${includedir}
We'll see what the PPA gets.
Andreas Hasenack (ahasenack) wrote : | #39 |
When installing from the PPA, the version in the pkgconfig file seems fine:
ubuntu@bionic:~$ cat /usr/lib/
version=nvml-1.3.1
libdir=
prefix=/usr
includedir=
Name: libpmem
Description: libpmem library from NVML project
Version: ${version}
URL: http://
Requires:
Libs: -L${libdir} -lpmem
Cflags: -I${includedir}
Marcin Ślusarz (mslusarz) wrote : | #40 |
I don't think there is a need for -dev-debug packages. The interface between debug and release packages is the same by design, so nobody should ever need to link to debug libraries.
Both debian/
Marcin Ślusarz (mslusarz) wrote : | #41 |
We tracked one test failure in ppa7 version to debug libraries not having symbols, but I see ppa10 already have them. I don't see it mentioned in Changelog. What happened?
(obj_convert test injects crashes in some specific places using gdb and see if consecutive open handles such pool correctly)
Andreas Hasenack (ahasenack) wrote : | #42 |
I suspect a dh_* script just did the right thing once I moved the dbg libraries to their own -debug packages.
What should I do to get the right version in the pkgconfig files?
Marcin Ślusarz (mslusarz) wrote : | #43 |
For now you can create .version file in the top level directory. We are evaluating various options.
Andreas Hasenack (ahasenack) wrote : | #44 |
nvml (1.3.1-
* Create missing .version file based on the upstream version listed in
the debian/changelog topmost entry.
That should work for a build from a directory that is not a git repository.
Andreas Hasenack (ahasenack) wrote : | #45 |
I've been working on more lintian fixes raised by the AA review. Here is today's batch:
nvml (1.3.1-
* Install pmemobj docs into a html subdir
* Enable parallel build (dh $@ --parallel)
* Several lintian fixes:
- binary-
- duplicate-
- extended-
- restore lintian override about ldconfig for the debug libraries, but
this time in the -debug packages and not -dev packages.
- using-first-
- extended-
- debian-
- d/p/spelling-
- useless-
- hardening-
That last one, hardening-
CPPFLAGS : -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT
CFLAGS : -g -O2 -fdebug-
LDFLAGS : -Wl,-Bsymbolic-
Before it was:
CPPFLAGS : -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -D_REENTRANT
CFLAGS : -g -O2 -fdebug-
LDFLAGS : -Wl,-Bsymbolic-
(don't mind the version, that's just in my local builds)
There is another flags-related lintian warning that I have to address:
I: libvmem1-debug: hardening-
N:
N: This package provides an ELF binary that lacks the use of fortified libc
N: functions. Either there are no potentially unfortified functions called
N: by any routines, all unfortified calls have already been fully validated
N: at compile-time, or the package was not built with the default Debian
N: compiler flags defined by dpkg-buildflags. If built using
N: dpkg-buildflags directly, be sure to import CPPFLAGS.
N:
N: NB: Due to false-positives, Lintian ignores some unprotected functions
N: (e.g. memcpy).
N:
N: Refer to https:/
N: https:/
N:
N: Severity: normal, Certainty: wild-guess
N:
N: Check: binaries, Type: binary, udeb
But since this is about the debug libraries, I think I will just add a note to READBE.source and/or override it.
Andreas Hasenack (ahasenack) wrote : | #46 |
Another set of lintian fixes:
nvml (1.3.1-
* Switch debhelper compat level to 11:
- d/rules: parallel build is default in dh level 11, so drop it from
the rules file.
* Add doc-base control file for the libpmemobj-dev package to register
its html documentation.
* New doc package for libpmemobj, moving the documentation that was
previously in the dev package (fixes lintian
arch-
Andreas Hasenack (ahasenack) wrote : | #47 |
nvml (1.3.1-
* More lintian fixes:
- package-
- package-
Remaining lintian complaints:
I: nvml source: out-of-
I: nvml source: testsuite-
P: nvml source: debian-
I: libvmem1: no-symbols-
I: libvmem1-debug: hardening-
I: libvmmalloc1-debug: hardening-
I: libpmem1: no-symbols-
I: libpmemobj1: no-symbols-
I: librpmem1: no-symbols-
I: libpmemblk1: no-symbols-
I: libpmemobj1-debug: hardening-
I: libpmemlog1-debug: hardening-
I: librpmem1-debug: hardening-
I: libpmem1-debug: hardening-
I: libpmempool1: no-symbols-
I: libpmemlog1: no-symbols-
I: libpmemblk1-debug: hardening-
I: libvmmalloc1: no-symbols-
Big ones remaining are standards-version, testsuite, no symbols file
Andreas Hasenack (ahasenack) wrote : | #48 |
no-symbols-
nvml (1.3.1-
* Create symbols files for the libraries
Andreas Hasenack (ahasenack) wrote : | #49 |
nvml (1.3.1-
* update README.source regarding the hardening-
check for the -debug packages and why it is not being addressed.
* include all pmempool manpages
* add dep8 tests
There is only one remaining lintian error now that has not been addressed yet in nvml:
N:
N: The source package refers to a Standards-Version older than the one that
N: was current at the time the package was created (according to the
N: timestamp of the latest debian/changelog entry). Please consider
N: updating the package to current Policy and setting this control field
N: appropriately.
N:
N: If the package is already compliant with the current standards, you
N: don't have to re-upload the package just to adjust the Standards-Version
N: control field. However, please remember to update this field next time
N: you upload the package.
N:
N: See /usr/share/
N: debian-policy package for a summary of changes in newer versions of
N: Policy.
N:
N: Refer to
N: https:/
N: details.
N:
N: Severity: wishlist, Certainty: certain
N:
N: Check: standards-version, Type: source
Andreas Hasenack (ahasenack) wrote : | #50 |
Uploaded twice today:
nvml (1.3.1-
* Bump standards-version to 4.1.4. No changes necessary.
-- Andreas Hasenack <email address hidden> Tue, 17 Apr 2018 19:29:25 +0000
nvml (1.3.1-
* Do not build examples. Something changed in the bionic toolchain and the
examples no longer build. See
https:/
-- Andreas Hasenack <email address hidden> Tue, 17 Apr 2018 18:01:50 +0000
There was some toolchain change in ubuntu bionic in the past few days that broke the build:
https:/
I don't know yet what that is about, so I disabled some example building in the meantime. I've seen in the logs that some examples are still built, but at least the failure doesn't happen anymore.
~ppa18 is my new Ubuntu Archive Admin review candidate, which I will submit shortly. It would be great if you could give it another round of testing.
I'll now switch back to ndctl and bring that one into compliance.
David Britton (dpb) wrote : | #51 |
As the upstream has change the project name to `pmdk` we will be changing the source package to match. If there are any objections, please speak now. :)
Reference:
Marcin Ślusarz (mslusarz) wrote : | #52 |
I tested ~ppa18 with PMDK in-tree tests and I realized that removal of .so files from nvml_dbg directory we discussed earlier breaks the assumption that you can just set LD_LIBRARY_PATH and everything will work correctly WHEN libvmmalloc is involved.
This works:
LD_PRELOAD=
This doesn't:
LD_LIBRARY_
Of course you can set LD_PRELOAD to libvmmalloc.so.1, but it's a bit annoying that you have to do that when library built from source works as intended...
Marcin Ślusarz (mslusarz) wrote : | #53 |
One more issue which I haven't noticed earlier - missing Valgrind support.
Bit of background: PMDK supports both regular upstream Valgrind tools (memcheck, helgrind and drd) and our pmemcheck tool (available at https:/
To enable support for all tools you have to use EXTRA_CFLAGS=
EXTRA_CFLAGS=
In 1.4 we fixed this by incorporating Valgrind headers into the tree and enabling support for all tools without any flag, independent of installed Valgrind version. If someone would want to use pmemcheck they would have to build our Valgrind fork, without rebuilding PMDK.
Eventually we will upstream pmemcheck, but that's a long term activity.
In the mean time, for 1.3 packages I'd suggest to go with just upstream Valgrind tools.
Andreas Hasenack (ahasenack) wrote : | #54 |
I'm almost done packaging 1.4, as per comment #51. We would prefer to stick with 1.4 unless you object. Keeping in mind that upgrading to 1.4 after 1.3.1 is released into bionic would be much harder, as it changes the source name.
With that in mind, do I need any changes in 1.4 regarding what you said about valgrind? Of course, you haven't seen my packages yet, but in principle?
About the LD_LIBRARY_PATH issue with debug builds, why would you set LD_PRELOAD to the libfoo.so symlink (instead of libfoo.so.1), even when using pmdk (formelly nvml) installed from source? Just because "it works" and then you don't have to remember the major lib version, or update docs whenever that changes? .so symlinks are not meant to be used as runtime libraries, but I can appreciate it's a bit convenient.
Marcin Ślusarz (mslusarz) wrote : | #55 |
Packaging 1.4 sounds great :). You don't need to do anything for Valgrind support in 1.4.
Nobody had considered that libvmmalloc.so would not be available. Our tests use libvmalloc.so (that's how I found this issue - https:/
I'll fix both issues on PMDK side.
Andreas Hasenack (ahasenack) wrote : | #56 |
First 1.4 package uploaded:
pmdk (1.4-0ubuntu1~ppa1) bionic; urgency=medium
* New upstream release
* Drop debian/
* Drop debian/
* Updated debian/
* Drop debian/
applied
* Updated debian/
* Drop debian/
* d/p/drop-
lintian error for the bash completion file.
* d/t/control: fix test dependency
* d/rules: workaround #1765851 by uncompressing the manpages right after
auto_install so dh_compress and dh_installman don't get confused
-- Andreas Hasenack <email address hidden> Mon, 23 Apr 2018 15:12:50 +0000
I made no attempt at creating an upgrade path from the nvml-named packages. If you were using them before, I suggest to purge them and reinstall these instead.
Andreas Hasenack (ahasenack) wrote : | #57 |
Also just uploaded a new ndctl: 60.1. This one still has lintian warnings that need to be fixed, and I also have to update debian/copyright because some files changed or were removed.
Marcin Ślusarz (mslusarz) wrote : | #58 |
Not a big issue, but I don't think shipping object files and Visual Studio project files with examples makes much sense :). Also shipped Makefiles are not self-contained (they source Makefile.inc, which is not included in the package)
Andreas Hasenack (ahasenack) wrote : | #59 |
Right :/
I was trying to find a way to not build the examples, but even removing the "examples" directory from src/Makefile didn't do it, something else kept triggering its build. Do you have a tip on how I can avoid building the examples?
This is what I was trying:
--- a/src/Makefile
+++ b/src/Makefile
@@ -38,12 +38,12 @@
TARGETS = libpmem libvmem libpmemblk libpmemlog libpmemobj libpmempool\
libvmmalloc tools
-ALL_TARGETS = $(TARGETS) common librpmem examples benchmarks
+ALL_TARGETS = $(TARGETS) common librpmem benchmarks
SCOPE_DIRS = $(TARGETS) common librpmem rpmem_common
DEBUG_
libpmempool libvmmalloc librpmem
-RELEASE_TARGETS = tools examples benchmarks
+RELEASE_TARGETS = tools benchmarks
CLEAN_
CLEAN_TARGETS = $(CLEAN_
Andreas Hasenack (ahasenack) wrote : | #60 |
Question: should I build pmdk with ndctl support, i.e., with the daxio package? Set NDCTL_ENABLE=y?
When I do that, however, for some reason libpmemobj and others get RPATH set again, even though I'm passing NORPATH=1:
(...)
cc -Wl,-Bsymbolic-
(...)
Marcin Ślusarz (mslusarz) wrote : | #61 |
Please don't build pmdk 1.4 with NDCTL_ENABLE=y. This variable enables much more than daxio and we are not ready to support that yet.
Marcin Ślusarz (mslusarz) wrote : | #62 |
If you want to disable building examples you have to remove both examples and benchmarks from src/Makefile ALL_TARGETS/
BTW, why do you even need to that? Why are they included in development packages?
Andreas Hasenack (ahasenack) wrote : | #63 |
They are considered documentation and trigger a lintian warning if not installed: https:/
Thanks for the NDCTL warning, I won't enable it.
Andreas Hasenack (ahasenack) wrote : | #64 |
pmdk (1.4-0ubuntu1~ppa2) bionic; urgency=medium
* Do not build examples and benchmarks. The former are documentation
aids, and benchmarks trigger example builds.
Marcin Ślusarz (mslusarz) wrote : | #65 |
I tested pmdk 1.4 ppa1 with pmdk in-tree tests and with some small changes all tests pass.
I'm in the process of upstreaming those changes (PR #2873 #2875 #2885 #2886) and once they are all on master I'll backport them to stable-1.4 branch.
(next week I'm on vacation)
Andreas Hasenack (ahasenack) wrote : | #66 |
ndctl (60.1-0ubuntu1~
* Added DEP8 tests for ndctl and daxctl.
Could someone please test the ndctl packages while I address the remaining of the lintian checks?
Andreas Hasenack (ahasenack) wrote : | #67 |
ndctl (60.1-0ubuntu1~
* Updated d/copyright
* Packages descriptions updates
* Bump standards-version to 4.1.4. No changes necessary.
Ok, barring any testing problems, I'll submit these two packages to another Archive Admin review next week.
quanxian (quanxian-wang) wrote : | #68 |
NVML is renamed PMDK. the source link is changed. I have updated the description for clear.
description: | updated |
description: | updated |
Andreas Hasenack (ahasenack) wrote : | #69 |
Another review point came up: do you still need the bundled jemalloc library? Your README says that you have changes on top of upstream's 3.6.0. Are these changes not worth pushing back upstream?
Andreas Hasenack (ahasenack) wrote : | #70 |
Would you also recommend a minimal testing.sh file to run at least some of the pmdk unit tests during build? The build environment is usually a VM, but could be an unprivileged lxd container as well. As we speak I'm giving a try with the supplied testing.sh.example one (renamed to testing.sh)
Andreas Hasenack (ahasenack) wrote : | #71 |
Hm, that is taking ages to run, I think I'll stick to the dep8 tests I wrote.
Marcin Ślusarz (mslusarz) wrote : | #72 |
jemalloc: We can't upstream those changes, because jemalloc changed too much since we forked it. pmemcto makes this even harder problem, because any changes to jemalloc on-media layout would have to invalidate pmemcto pools and we don't have any mechanism to automatically do that.
testing: Default configuration uses non-pmem code path, which is slow. You can disable it (unset NON_PMEM_FS_DIR), enable pmem path (PMEM_FS_DIR) and if you don't have DAX available set PMEM_FS_
FYI, yesterday I pushed some changes to the 1.4 branch to make it possible to run in-tree tests against installed libraries. Those changes will be part of 1.4.1 release.
Andreas Hasenack (ahasenack) wrote : | #73 |
I'm getting a timeout in util_file_
(...)
util_file_
util_file_
util_file_
util_file_
util_file_
util_file_
util_file_
util_file_
util_file_
RUNTESTS: stopping: util_file_
Makefile:472: recipe for target 'check' failed
That test results in stuck processes:
30263 pts/0 R 22:28 ./util_file_create 0x4000 0x7FFFFFFFFFFFF
13054 pts/0 R 4:25 ./util_file_create 0x4000 0x7FFFFFFFFFFFF
They are very busy doing this:
pwrite64(7, "\0", 1, 798772768766) = 1
pwrite64(7, "\0", 1, 798772772862) = 1
pwrite64(7, "\0", 1, 798772776958) = 1
pwrite64(7, "\0", 1, 798772781054) = 1
pwrite64(7, "\0", 1, 798772785150) = 1
pwrite64(7, "\0", 1, 798772789246) = 1
pwrite64(7, "\0", 1, 798772793342) = 1
pwrite64(7, "\0", 1, 798772797438) = 1
pwrite64(7, "\0", 1, 798772801534) = 1
I'll try in a VM next (that run was in a lxd container).
Andreas Hasenack (ahasenack) wrote : | #74 |
I used this testing config:
echo "PMEM_FS_DIR=/tmp" > src/test/
echo "PMEM_FS_
echo "TEST_BUILD=\"debug nondebug\"" >> src/test/
echo "TM=1" >> src/test/
Note, there is no pmem or dax hardware here.
Andreas Hasenack (ahasenack) wrote : | #75 |
pcheck would always fail at this stage:
make[6]: Entering directory '/home/
cc -Wl,-Bsymbolic-
mblk_priv.o ../../.
output.o: file not recognized: File truncated
collect2: error: ld returned 1 exit status
../../.
make[6]: *** [pmemspoil.
make[6]: Leaving directory '/home/
Without parallelization it worked:
echo "PMEM_FS_DIR=/tmp" > src/test/
echo "PMEM_FS_
echo "TEST_BUILD=\"debug nondebug\"" >> src/test/
echo "TM=1" >> src/test/
# dh_auto_test calls make test, which only builds the test suite
# make check (and pcheck, for parallel check) actually runs it
make SKIP_SYNC_REMOTES=y check
21min to build with that in a VM. Not too bad. I'll add that to the package and skip the test if it's building in a lxc container.
Marcin Ślusarz (mslusarz) wrote : | #76 |
I'm looking into util_file_create failure.
WRT pcheck, you had hit some bug in our build system. I know what's causing the error you are seeing (make tries to build pmemspoil twice, both processes write to the same file, which means creation of corrupted object file and that leads to failure in linking), but I don't know where is the bug yet. I'll look into this.
To work around it run "make test" first and only then use pcheck with -jX.
Marcin Ślusarz (mslusarz) wrote : | #77 |
util_file_create timed out because you used file system (PMEM/NON_
Marcin Ślusarz (mslusarz) wrote : | #78 |
I finally found why pcheck failed (https:/
Andreas Hasenack (ahasenack) wrote : | #79 |
This was finally accepted into the archive and is available in cosmic:
https:/
https:/
I'll close this bug, and open specific ones against each package for an SRU into Bionic.
Thanks all for the work so far!
Changed in ubuntu: | |
status: | In Progress → Fix Released |
Andreas Hasenack (ahasenack) wrote : | #80 |
Do you have a preference for which version we should backport to bionic? Cosmic has these currently:
pmdk: 1.4 (latest upstream is 1.4.1)
ndctl: 60.1 (latest upstream is 61.2)
In general the latest is usually what people want, but I thought best to check. Maybe there is something super important we are missing, or something experimental that we do not want yet.
Keeping in mind that if want to update these, they need to be updated in cosmic first, and then pushed down to bionic.
Dan Williams (djbw) wrote : | #81 |
Yes, for ndctl, please roll forward to latest. I'll Marcin comment on pmdk.
Thanks for all the support!
Marcin Ślusarz (mslusarz) wrote : | #82 |
Please backport the latest version of pmdk.
Michael Reed (mreed8855) wrote : | #83 |
Hi Andreas, I would say we need to go with the latest versions for both ndctl and pmdk to get back ported to bionic.
Andreas Hasenack (ahasenack) wrote : | #84 |
Ok, work on that has already started.
Andreas Hasenack (ahasenack) wrote : | #85 |
> Please don't build pmdk 1.4 with NDCTL_ENABLE=y. This variable enables much more
> than daxio and we are not ready to support that yet.
Should I enable ndctl now with pmdk 1.4.1 and get the daxio utility, or not yet?
Marcin Ślusarz (mslusarz) wrote : | #86 |
In pmdk 1.4.1 NDCTL_ENABLE controls daxio only, so it's safe to build pmdk with this flag. The default value has been flipped to "detect", so if ndctl is installed in the system, there's no need to set it explicitly.
Andreas Hasenack (ahasenack) wrote : | #87 |
Would you guys be able to give the latest ndctl and pmdk packages from https:/
Marcin Ślusarz (mslusarz) wrote : | #88 |
Yeah, we'll test pmdk packages.
Marcin Ślusarz (mslusarz) wrote : | #89 |
PMDK packages look good.
I have updated the packages in my PPA at https:/ /launchpad. net/~nacc/ +archive/ ubuntu/ nvdimm
Please test and report back.