[Date Prev][Date Next] [Thread Prev][Thread Next] [Date Index] [Thread Index]

ROCm build script and meeting agenda

Hello everyone,
I've successfully built {roc,hip}{BLAS,SOLVER,SPARSE,FFT}, rocThrust, rocPRIM, rocALUTION and hipCUB from tarballs on Debian unstable. A Bash script is attached (as a text file). I've also created a gist, which I will update with any improvements I make to that script: https://gist.github.com/cgmb/7cd9a481c42ce132b5d6420380becef3
If you don't have at least 64GB of RAM, you might need to reduce the job count (and perhaps hide cores with something like `docker --cpuset-cpus=0,1`; Tensile in ROCm 4.5 will use all cores available).
As noted in my previous update in the packaging session followup thread, there's a lot components that will require work to conform to the Filesystem Hierarchy Standard (FHS). Now is a great time to collect a list of everything wrong, because there are already proposals to change the ROCm installation layout. Let's ensure your needs are heard.
I'm sure there's much to discuss, so it seems appropriate to arrange another meeting. Given how much interest there has been from Europe, it seems like I should choose a different time. I've been terrible at converting dates and times, so I'm not going to try. If you have at time you'd like, this is my availability: https://calendly.com/cgmb/debian
I'll be in the meeting room tomorrow at the same time as last week, but don't feel obligated to show up for that one. I'll do a second session this week at whatever time seems to work for other people, and then we'll figure out a better schedule going forward. 

On the agenda for the next meeting:

1. What components need changes to be properly packaged?
2. Is there any other information I need to provide to facilitate packaging the components in the script? 

3. What components should I work on next?

Sincerely,

Cory Bloor


set -exuo pipefail

apt-get -qq update
apt-get -qq upgrade
apt-get -qq install build-essential cmake wget

DEB_WORKSPACE=$HOME # where to download and build the sources
DEB_HIP_ARCHITECTURES='gfx906:xnack-' # https://llvm.org/docs/AMDGPUUsage.html

# llvm-amdgpu
cd "$DEB_WORKSPACE"
apt-get -qq install python3
wget -qO- https://github.com/RadeonOpenCompute/llvm-project/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd llvm-project-rocm-4.5.0
# backport patch to remove linux/cyclades.h
wget -qO- https://github.com/RadeonOpenCompute/llvm-project/commit/68d5235cb58f988c71b403334cd9482d663841ab.patch | patch -p1
cmake -Sllvm -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_PROJECTS="clang;lld;clang-tools-extra;compiler-rt" \
  -DLLVM_TARGETS_TO_BUILD="AMDGPU;X86" \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocm-cmake
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/RadeonOpenCompute/rocm-cmake/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocm-cmake-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocm-device-libs
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/RadeonOpenCompute/ROCm-Device-Libs/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd ROCm-Device-Libs-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/clang \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# roct-thunk-interface
cd "$DEB_WORKSPACE"
apt-get -qq install libnuma-dev pkg-config libdrm-dev
wget -qO- https://github.com/RadeonOpenCompute/ROCT-Thunk-Interface/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd ROCT-Thunk-Interface-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocr-runtime
cd "$DEB_WORKSPACE"
apt-get -qq install libelf-dev xxd
wget -qO- https://github.com/RadeonOpenCompute/ROCR-Runtime/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd ROCR-Runtime-rocm-4.5.0
cmake -Ssrc -Bbuild \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocminfo
cd "$DEB_WORKSPACE"
apt-get -qq install kmod python3
wget -qO- https://github.com/RadeonOpenCompute/rocminfo/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocminfo-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# comgr
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/RadeonOpenCompute/ROCm-CompilerSupport/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd ROCm-CompilerSupport-rocm-4.5.0
cmake -Slib/comgr -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hip
cd "$DEB_WORKSPACE"
apt-get -qq install mesa-common-dev
wget -qO- https://github.com/ROCm-Developer-Tools/hipamd/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
wget -qO- https://github.com/ROCm-Developer-Tools/ROCclr/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
wget -qO- https://github.com/RadeonOpenCompute/ROCm-OpenCL-Runtime/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
wget -qO- https://github.com/ROCm-Developer-Tools/HIP/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipamd-rocm-4.5.0
mkdir build
cd build
cmake -S.. -B. -DCMAKE_BUILD_TYPE=Release \
  -DHIP_COMMON_DIR="$DEB_WORKSPACE/HIP-rocm-4.5.0" \
  -DAMD_OPENCL_PATH="$DEB_WORKSPACE/ROCm-OpenCL-Runtime-rocm-4.5.0" \
  -DROCCLR_PATH="$DEB_WORKSPACE/ROCclr-rocm-4.5.0" \
  -DCMAKE_HIP_ARCHITECTURES=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr/hip # can't be /usr due to 
make -j32
#touch /usr/include/.nodelete
#find /usr/include -type f -exec chattr +i {} \+
make install # warning: will delete $CMAKE_INSTALL_PREFIX/include !
#chattr -i /usr/include/.nodelete
#find /usr/include -type f -exec chattr -i {} \+
#rm /usr/include/.nodelete
apt-get -qq install perl file # used for hipcc

# variables used for projects built with hipcc
export HIP_PLATFORM=amd
export HIP_RUNTIME=rocclr
export HIP_COMPILER=clang
export HIP_CLANG_PATH=/usr/bin
export DEVICE_LIB_PATH=/usr/amdgcn/bitcode
export HIP_DEVICE_LIB_PATH=/usr/amdgcn/bitcode
export HSA_PATH=/usr/hsa
export LLVM_PATH=/usr
export HIP_PATH=/usr/hip
export ROCM_PATH=/usr/hip
export HCC_AMDGPU_TARGET=$DEB_HIP_ARCHITECTURES # workaround for hipcc path issues

# optional hipcc variables
export HIPCC_COMPILE_FLAGS_APPEND='-O3 -Wno-format-nonliteral -parallel-jobs=4'
export HIPCC_LINK_FLAGS_APPEND='-O3 -parallel-jobs=4'
export HIP_CLANG_HCC_COMPAT_MODE=1

# rocrand (with tests)
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran git libgtest-dev
wget -qO- https://github.com/ROCmSoftwarePlatform/rocRAND/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocRAND-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DBUILD_TEST=ON \
  -DBUILD_BENCHMARK=ON \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
# make -C build test
make -C build install

# rocblas
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran libmsgpack-dev python3 python3-pip python3-virtualenv python3.9-venv python3-yaml python3-msgpack
wget -qO- https://github.com/ROCmSoftwarePlatform/rocBLAS/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
wget -qO- https://github.com/ROCmSoftwarePlatform/Tensile/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocBLAS-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DTensile_TEST_LOCAL_PATH="$DEB_WORKSPACE/Tensile-rocm-4.5.0" \
  -DTensile_LOGIC=asm_full \
  -DTensile_CODE_OBJECT_VERSION=V3 \
  -DTensile_LIBRARY_FORMAT=msgpack \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DRUN_HEADER_TESTING=OFF \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocsolver
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran libfmt-dev
wget -qO- https://github.com/ROCmSoftwarePlatform/rocSOLVER/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocSOLVER-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hipblas (AMD)
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran
wget -qO- https://github.com/ROCmSoftwarePlatform/hipBLAS/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipBLAS-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hipsolver (AMD)
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran
wget -qO- https://github.com/ROCmSoftwarePlatform/hipSOLVER/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipSOLVER-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocprim
cd "$DEB_WORKSPACE"
apt-get -qq install git
wget -qO- https://github.com/ROCmSoftwarePlatform/rocPRIM/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocPRIM-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocsparse
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran
wget -qO- https://github.com/ROCmSoftwarePlatform/rocSPARSE/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocSPARSE-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hipsparse (AMD)
cd "$DEB_WORKSPACE"
apt-get -qq install gfortran
wget -qO- https://github.com/ROCmSoftwarePlatform/hipSPARSE/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipSPARSE-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocalution
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/ROCmSoftwarePlatform/rocALUTION/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocALUTION-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_MODULE_PATH=/usr/hip/cmake \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DBUILD_CLIENTS_SAMPLES=OFF \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hipcub
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/ROCmSoftwarePlatform/hipCUB/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipCUB-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_MODULE_PATH=/usr/hip/cmake \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DBUILD_CLIENTS_SAMPLES=OFF \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocfft
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/ROCmSoftwarePlatform/rocFFT/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocFFT-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# hipfft
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/ROCmSoftwarePlatform/hipFFT/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd hipFFT-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DCMAKE_MODULE_PATH=/usr/hip/cmake \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install

# rocthrust
cd "$DEB_WORKSPACE"
wget -qO- https://github.com/ROCmSoftwarePlatform/rocThrust/archive/refs/tags/rocm-4.5.0.tar.gz | tar xz
cd rocThrust-rocm-4.5.0
cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_CXX_COMPILER=/usr/hip/bin/hipcc \
  -DAMDGPU_TARGETS=$DEB_HIP_ARCHITECTURES \
  -DCMAKE_INSTALL_PREFIX=/usr
make -j32 -C build
make -C build install
Reply to: