Ticket 1990 - slurm configure doesn't detect HDF5 if HDF5 was built using cmake
Summary: slurm configure doesn't detect HDF5 if HDF5 was built using cmake
Status: RESOLVED INFOGIVEN
Alias: None
Product: Slurm
Classification: Unclassified
Component: Configuration (show other tickets)
Version: 15.08.0
Hardware: Linux Linux
: --- 4 - Minor Issue
Assignee: David Bigagli
QA Contact:
URL:
Depends on:
Blocks:
 
Reported: 2015-09-29 23:59 MDT by Dr. Christoph Pospiech
Modified: 2015-10-21 01:41 MDT (History)
3 users (show)

See Also:
Site: PIK
Alineos Sites: ---
Atos/Eviden Sites: ---
Confidential Site: ---
Coreweave sites: ---
Cray Sites: ---
DS9 clusters: ---
HPCnow Sites: ---
HPE Sites: ---
IBM Sites: ---
NOAA SIte: ---
OCF Sites: ---
Recursion Pharma Sites: ---
SFW Sites: ---
SNIC sites: ---
Linux Distro: ---
Machine Name:
CLE Version:
Version Fixed:
Target Release: ---
DevPrio: ---
Emory-Cloud Sites: ---

Attachments
List of installed HDF5 files (14.80 KB, text/plain)
2015-09-30 20:59 MDT, Dr. Christoph Pospiech 		Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this ticket.
Description Dr. Christoph Pospiech 2015-09-29 23:59:07 MDT 
If HDF5 was built using cmake (rather than configure), there is no
h5cc or h5pcc. Hence there is no way to specify --with-hdf5=PATH in
slurm configure.
Comment 1 Moe Jette 2015-09-30 05:09:30 MDT 
Do you have the hdf5 header and libraries installed locally (hdf5.h, libhdf5 and libhdf5_hl)? Slurm definitely needs those to build the job profiling plugin and "sh5util" tool.
Comment 2 Dr. Christoph Pospiech 2015-09-30 20:59:23 MDT 
Created attachment 2267 [details]
List of installed HDF5 files

Result of running the following two commands.
cp@helios:~$ module load hdf5/1.8.15p1/serial
cp@helios:~$ find $HDF5ROOT > HDF5_file.list
Comment 3 Dr. Christoph Pospiech 2015-09-30 21:04:27 MDT 
I just added an attachment that was the result 
of the following two commands.
cp@helios:~$ module load hdf5/1.8.15p1/serial
cp@helios:~$ find $HDF5ROOT > HDF5_file.list

But even if I add --with-hdf5=$HDF5ROOT to configure
the HDF5 is not properly detected.

configure:20503: checking Using provided HDF5 C wrapper
configure:20505: result: /home/cp/tools/ctt/packages/hdf5/1.8.15p1/serial
configure:20508: checking for HDF5 libraries
configure:20511: result: no
configure:20517: WARNING:
Unable to locate HDF5 compilation helper scripts 'h5cc' or 'h5pcc'.
Please specify --with-hdf5=<LOCATION> as the full path to h5cc or h5pcc.
HDF5 support is being disabled (equivalent to --with-hdf5=no).
Comment 4 David Bigagli 2015-10-07 21:07:13 MDT 
Hi Ciaron,
          Christoph open this ticket but he is not on the list so I added you
in CC I have also decreased the severity of this ticket as I think this is
something that Christoph tried on his laptop. We will have a look at it
in any case.

David
Comment 5 Dr. Christoph Pospiech 2015-10-07 21:57:29 MDT 
David,

correct, I was seeing that on my Laptop, but it is/was an issue with the PIK cluster as well. When Herbert Mehlhose built the current slurm installation on the cluster, he couldn't make use of the existing HDF5 modules - as these were built with cmake and slurm wouldn't recognize them.
Even if the version 15.08 will be built in the future on the PIK cluster, it will suffer from this problem again.
But I agree, it is a minor issue, as there is a work around - download another copy of HDF5, and build a slurm-special version using configure rather than cmake. It only itches as this is somewhat duplicate work.

Christoph
Comment 6 David Bigagli 2015-10-07 22:01:58 MDT 
Hi Christop,
            thanks for clarifying that. I will build the latest hdf5 with
cmake and see what we can do. This may actually apply to other packages
as well if they decide to migrate to cmake.

David
Comment 7 David Bigagli 2015-10-18 21:47:33 MDT 
Hi,
   I have build the hdf5 using cmake, the final install contains all the tools
except h5cc. However since h5cc is a wrapper script around gcc it is possible
to edit it and specify the location of the HDF5 installation.

In may case:

prefix="/tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1"

then copy the script into /tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1/bin and configure specifying the full path to h5cc.

--with-hdf5=/tmp/HDF5-1.8.15-patch1-Linux/HDF_Group/HDF5/1.8.15-patch1/bin/h5cc

Both cmake and configure generate the same installation tree, so the above
method works.

Let us know if you have any other question.

David
Comment 8 Dr. Christoph Pospiech 2015-10-21 01:10:03 MDT 
Did I get the solution correctly ? First HDF5 is built and installed using
cmake. Then another run with configure is used to create h5cc which is then to be installed manually ?

That is certainly doable (albeit tedious). Perhaps the HDF5 developers
should be notified that there is a requirement to create and install
h5cc for the cmake build as well ?

Christoph
Comment 9 David Bigagli 2015-10-21 01:41:19 MDT 
Hi Christoph,
             yes you are right that would be the procedure. h5cc is created by configure so you would have to run it. It is tedious indeed.... 
I think this can be viewed as hdf5 bug since they broke the backward compatibility
by removing a binary.

David