Summary: | slurm configure doesn't detect HDF5 if HDF5 was built using cmake | ||
---|---|---|---|
Product: | Slurm | Reporter: | Dr. Christoph Pospiech <pospiech-HD> |
Component: | Configuration | Assignee: | David Bigagli <david> |
Status: | RESOLVED INFOGIVEN | QA Contact: | |
Severity: | 4 - Minor Issue | ||
Priority: | --- | CC: | brian, da, linstead |
Version: | 15.08.0 | ||
Hardware: | Linux | ||
OS: | Linux | ||
Site: | PIK | Alineos Sites: | --- |
Atos/Eviden Sites: | --- | Confidential Site: | --- |
Coreweave sites: | --- | Cray Sites: | --- |
DS9 clusters: | --- | HPCnow Sites: | --- |
HPE Sites: | --- | IBM Sites: | --- |
NOAA SIte: | --- | OCF Sites: | --- |
Recursion Pharma Sites: | --- | SFW Sites: | --- |
SNIC sites: | --- | Linux Distro: | --- |
Machine Name: | CLE Version: | ||
Version Fixed: | Target Release: | --- | |
DevPrio: | --- | Emory-Cloud Sites: | --- |
Attachments: | List of installed HDF5 files |
Description
Dr. Christoph Pospiech
2015-09-29 23:59:07 MDT
Do you have the hdf5 header and libraries installed locally (hdf5.h, libhdf5 and libhdf5_hl)? Slurm definitely needs those to build the job profiling plugin and "sh5util" tool. Created attachment 2267 [details]
List of installed HDF5 files
Result of running the following two commands.
cp@helios:~$ module load hdf5/1.8.15p1/serial
cp@helios:~$ find $HDF5ROOT > HDF5_file.list
I just added an attachment that was the result of the following two commands. cp@helios:~$ module load hdf5/1.8.15p1/serial cp@helios:~$ find $HDF5ROOT > HDF5_file.list But even if I add --with-hdf5=$HDF5ROOT to configure the HDF5 is not properly detected. configure:20503: checking Using provided HDF5 C wrapper configure:20505: result: /home/cp/tools/ctt/packages/hdf5/1.8.15p1/serial configure:20508: checking for HDF5 libraries configure:20511: result: no configure:20517: WARNING: Unable to locate HDF5 compilation helper scripts 'h5cc' or 'h5pcc'. Please specify --with-hdf5=<LOCATION> as the full path to h5cc or h5pcc. HDF5 support is being disabled (equivalent to --with-hdf5=no). Hi Ciaron, Christoph open this ticket but he is not on the list so I added you in CC I have also decreased the severity of this ticket as I think this is something that Christoph tried on his laptop. We will have a look at it in any case. David David, correct, I was seeing that on my Laptop, but it is/was an issue with the PIK cluster as well. When Herbert Mehlhose built the current slurm installation on the cluster, he couldn't make use of the existing HDF5 modules - as these were built with cmake and slurm wouldn't recognize them. Even if the version 15.08 will be built in the future on the PIK cluster, it will suffer from this problem again. But I agree, it is a minor issue, as there is a work around - download another copy of HDF5, and build a slurm-special version using configure rather than cmake. It only itches as this is somewhat duplicate work. Christoph Hi Christop, thanks for clarifying that. I will build the latest hdf5 with cmake and see what we can do. This may actually apply to other packages as well if they decide to migrate to cmake. David Hi, I have build the hdf5 using cmake, the final install contains all the tools except h5cc. However since h5cc is a wrapper script around gcc it is possible to edit it and specify the location of the HDF5 installation. In may case: prefix="/tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1" then copy the script into /tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1/bin and configure specifying the full path to h5cc. --with-hdf5=/tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1/bin/h5cc Both cmake and configure generate the same installation tree, so the above method works. Let us know if you have any other question. David Did I get the solution correctly ? First HDF5 is built and installed using cmake. Then another run with configure is used to create h5cc which is then to be installed manually ? That is certainly doable (albeit tedious). Perhaps the HDF5 developers should be notified that there is a requirement to create and install h5cc for the cmake build as well ? Christoph Hi Christoph, yes you are right that would be the procedure. h5cc is created by configure so you would have to run it. It is tedious indeed.... I think this can be viewed as hdf5 bug since they broke the backward compatibility by removing a binary. David |