Invalid SPIR-V module: Casts from private/local/global address space are allowed only to generic #1728

AlekseiSudar · 2025-02-22T02:10:28Z

AlekseiSudar
Feb 22, 2025

Hi!
I am just starting to learn SYCL and trying to set up the environment.
I use laptop with Intel Core i5-1135G7 and integrated GPU Intel Iris Xe Graphics.
I am running a Docker container with Fedora (I am using Fedora arbitrarily and can switch to another distribution if needed):

docker run -itd \
--device=/dev/dri:/dev/dri \
--name fedoraSYCL \
-v ${PWD}:/home:Z \
fedora

I have installed the Intel oneAPI repository to use the OpenCL runtime for the CPU and the necessary dependencies for AdaptiveCPP:

tee > /tmp/oneAPI.repo << EOF
[oneAPI]
name=Intel® oneAPI repository
baseurl=https://yum.repos.intel.com/oneapi
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
EOF
sudo mv /tmp/oneAPI.repo /etc/yum.repos.d

dnf install \
python3 \
clang clang-devel \
llvm llvm-devel \
boost boost-devel \
git cmake \
clinfo \
intel-compute-runtime \
intel-oneapi-runtime-opencl \
oneapi-level-zero-devel \
OpenCL-ICD-Loader-devel

Then I installed AdaptiveCPP as follows:

git clone https://github.com/AdaptiveCpp/AdaptiveCpp
cd AdaptiveCpp
mkdir build && cd build

cmake \
DOpenCL_LIBRARY=/usr/lib64/libOpenCL.so \
-DWITH_OPENCL_BACKEND=ON \
-DWITH_LEVEL_ZERO_BACKEND=ON \
..

make install

acpp-info -l shows:

=================Backend information===================
Loaded backend 0: Level Zero
  Found device: Intel(R) Iris(R) Xe Graphics
Loaded backend 1: OpenCL
  Found device: Intel(R) Iris(R) Xe Graphics
  Found device: 11th Gen Intel(R) Core(TM) i5-1135G7 @ 2.40GHz
Loaded backend 2: OpenMP
  Found device: AdaptiveCpp OpenMP host device

Next, I create a simple program and compile using acpp main.cpp -o main

#include <sycl/sycl.hpp>
#include <iostream>
#include <vector>

using namespace sycl;

int main() {
    const size_t N = 1024;
    std::vector<float> vecA(N, 1);
    std::vector<float> vecB(N, 2);
    std::vector<float> vecC(N, 0);


    queue q;
    {
        buffer<float, 1> bufA(vecA.data(), range<1>(N));
        buffer<float, 1> bufB(vecB.data(), range<1>(N));
        buffer<float, 1> bufC(vecC.data(), range<1>(N));

        q.submit([&](handler &h) {
            auto accA = bufA.get_access<access::mode::read>(h);
            auto accB = bufB.get_access<access::mode::read>(h);
            auto accC = bufC.get_access<access::mode::write>(h);

            h.parallel_for(range<1>(N), [=](id<1> i) {
                accC[i] = accA[i] + accB[i];
            });
        });
    }

    std::cout << "First ten 10 elements:\n";
    for (size_t i = 0; i < 10; ++i) {
        std::cout << vecC[i] << " ";
    }
    std::cout << std::endl;

    return 0;
}

When I run the program, I get an error

InvalidModule: Invalid SPIR-V module: Casts from private/local/global address space are allowed only to generic

  %19 = addrspacecast ptr addrspace(1) %10 to ptr
[AdaptiveCpp Error] from /home/AdaptiveCpp/include/hipSYCL/glue/llvm-sscp/jit.hpp:320 @ compile(): jit::compile: Encountered errors:
0: LLVMToSpirv: llvm-spirv invocation failed with exit code 11

[AdaptiveCpp Error] from /home/AdaptiveCpp/src/runtime/ze/ze_queue.cpp:552 @ submit_sscp_kernel_from_code_object(): ze_queue: Code object construction failed
First ten 10 elements:
0 0 0 0 0 0 0 0 0 0 
============== AdaptiveCpp error report ============== 
AdaptiveCpp has caught the following unhandled asynchronous errors: 

   0. from /home/AdaptiveCpp/include/hipSYCL/glue/llvm-sscp/jit.hpp:320 @ compile(): jit::compile: Encountered errors:
0: LLVMToSpirv: llvm-spirv invocation failed with exit code 11

   1. from /home/AdaptiveCpp/src/runtime/ze/ze_queue.cpp:552 @ submit_sscp_kernel_from_code_object(): ze_queue: Code object construction failed
The application will now be terminated.
terminate called without an active exception
Aborted (core dumped)

I also tried to use queue q(cpu_selector_v); to use OpenCL backend.
Based on my tests, this always happens when I create any event in the queue.
However, the code works if I add --acpp-targets=omp flag.

I found a similar problem, but it occurred while compiling a project using DPC++.

Maybe someone knows what’s going on? Does it matter that I’m using Docker?
Thanks in advance for your help.

Answered by illuhad

Feb 23, 2025

I’m reviewing the CMake logs.
I think that CMake is unable to find the translator.

No, this output refers to spirv-tools which is an unnecessary optional dependency of the llvm-spirv translator.
AdaptiveCpp builds its own llvm-spirv translator during the build process. The fact that you get this output shows that AdaptiveCpp is building the llvm-spirv translator.

Please do not install your own llvm-spirv packages, as those will not have the necessary patch applied! AdaptiveCpp will look for llvm-spirv in its own internal directories anyway (which is to where it will install its patched llvm-spirv), so external packages should not make a difference anyway.

I do not see in your cmake outp…

View full answer

illuhad · 2025-02-22T03:02:39Z

illuhad
Feb 22, 2025
Maintainer

Hi!

I don't think this is related to docker. This is a JIT compilation failure, so there's something wrong with compilation, not with kernel execution. The fact that this is in a container is unlikely to play a role here.

When building AdaptiveCpp in a clean cmake build directory, does cmake say anything about being unable to apply a patch to llvm-spirv? AdaptiveCpp uses a patched version of the llvm-spirv translator. I suspect that either its not using its own translator, or patch application went wrong.
Please provide the output when running with the environment variable ACPP_DEBUG_LEVEL=3
Run with the environment variable ACPP_S2_DUMP_IR_FINAL=1. This should create a file main.cpp.ll, which is the final optimized LLVM IR code for your kernel. Please post this file.

I'd recommend using OpenCL over L0. The OpenCL backend is much better tested, optimized and more feature-complete compared to L0.

EDIT: For JIT-compilation, it needs to be able to write temporary files to disk. Is the directory in your container writable?
EDIT2: It will likely also work if you just set ACPP_VISIBILITY_MASK="omp" - no need to play with --acpp-targets. Setting ACPP_VISIBILITY_MASK="omp" will hide all backends except for the OpenMP one, thus forcing the JIT compiler to target the host CPU directly instead of going through OpenCL/L0 and SPIR-V.

0 replies

AlekseiSudar · 2025-02-22T22:58:36Z

AlekseiSudar
Feb 22, 2025
Author

Thank you for the response!

I’m reviewing the CMake logs.
I think that CMake is unable to find the translator.

CMake Warning at CMakeLists.txt:119 (find_package):
  By not providing "FindSPIRV-Tools.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "SPIRV-Tools", but CMake did not find one.

  Could not find a package configuration file provided by "SPIRV-Tools" with
  any of the following names:

    SPIRV-ToolsConfig.cmake
    spirv-tools-config.cmake

  Add the installation prefix of "SPIRV-Tools" to CMAKE_PREFIX_PATH or set
  "SPIRV-Tools_DIR" to a directory containing one of the above files.  If
  "SPIRV-Tools" provides a separate development package or SDK, be sure it
  has been installed.

I tried installing spirv-llvm-translator-devel, spirv-llvm-translator-tools, spirv-llvm-translator, spirv-tools-devel, and spirv-tools, but CMake still shows this warnings. I reinstalled acpp, but the problem persists.
I also tried installing version 19 of the SPIRV-LLVM-Translator manually, but the warnings still don't go away. The problem persists.

It seems that this is an issue with my packages and possibly Fedora.
I will continue experimenting.

cmake_log.txt

DEBUG_LEVEL_3_output.txt

main.cpp.ll:

;---------------- Begin AdaptiveCpp IR dump --------------
; AdaptiveCpp SSCP S2 IR dump; Compiling kernels: <Kernel:_Z18__acpp_sscp_kernelIN7hipsycl4glue15__sscp_dispatch18basic_parallel_forIZZ4mainENK3$_0clERNS0_4sycl7handlerEEUlNS5_2idILi1EEEE_Li1EEEEvRKT_>, stage: final
source_filename = "main.cpp"
target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-A4-n8:16:32:64"
target triple = "spir64-unknown-unknown"

%__acpp_sscp_pointer_wrapper.1 = type { ptr addrspace(1) }

; Function Attrs: mustprogress nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: write)
declare spir_func void @llvm.assume(i1 noundef) #0

declare dso_local spir_func noundef i64 @_Z32__spirv_BuiltInLocalInvocationIdi(i32 noundef) local_unnamed_addr #1

declare dso_local spir_func noundef i64 @_Z26__spirv_BuiltInWorkgroupIdi(i32 noundef) local_unnamed_addr #1

; Function Attrs: nounwind
define spir_kernel void @"_Z18__acpp_sscp_kernelIN7hipsycl4glue15__sscp_dispatch18basic_parallel_forIZZ4mainENK3$_0clERNS0_4sycl7handlerEEUlNS5_2idILi1EEEE_Li1EEEEvRKT_"(ptr addrspace(4) nocapture readonly byval(%__acpp_sscp_pointer_wrapper.1) %0, ptr addrspace(4) nocapture readonly byval(%__acpp_sscp_pointer_wrapper.1) %1, ptr addrspace(4) nocapture readonly byval(%__acpp_sscp_pointer_wrapper.1) %2, i64 %3) #2 !reqd_work_group_size !6 {
  %5 = load ptr addrspace(1), ptr addrspace(4) %0, align 8
  %6 = load ptr addrspace(1), ptr addrspace(4) %1, align 8
  %7 = load ptr addrspace(1), ptr addrspace(4) %2, align 8
  %8 = call spir_func noundef i64 @_Z26__spirv_BuiltInWorkgroupIdi(i32 noundef 0) #2
  %9 = icmp ult i64 %8, 16777215
  call spir_func void @llvm.assume(i1 %9)
  %10 = shl nuw nsw i64 %8, 7
  %11 = call spir_func noundef i64 @_Z32__spirv_BuiltInLocalInvocationIdi(i32 noundef 0) #2
  %12 = icmp ult i64 %11, 128
  call spir_func void @llvm.assume(i1 %12)
  %13 = or disjoint i64 %11, %10
  %.not.i.i = icmp ult i64 %13, %3
  br i1 %.not.i.i, label %_ZN7hipsycl4glue15__sscp_dispatch16item_is_in_rangeILi1ELb0EEEbRKNS_4sycl4itemIXT_EXT0_EEERKNS3_5rangeIXT_EEERKNS3_2idIXT_EEE.exit.i.i, label %"_Z18__acpp_sscp_kernelIN7hipsycl4glue15__sscp_dispatch18basic_parallel_forIZZ4mainENK3$_0clERNS0_4sycl7handlerEEUlNS5_2idILi1EEEE_Li1EEEEvRKT_.old.exit"

_ZN7hipsycl4glue15__sscp_dispatch16item_is_in_rangeILi1ELb0EEEbRKNS_4sycl4itemIXT_EXT0_EEERKNS3_5rangeIXT_EEERKNS3_2idIXT_EEE.exit.i.i: ; preds = %4
  %14 = addrspacecast ptr addrspace(1) %7 to ptr
  %15 = addrspacecast ptr addrspace(1) %6 to ptr
  %16 = addrspacecast ptr addrspace(1) %5 to ptr
  %17 = getelementptr inbounds float, ptr %15, i64 %13
  %18 = load float, ptr %17, align 4
  %19 = getelementptr inbounds float, ptr %14, i64 %13
  %20 = load float, ptr %19, align 4
  %21 = fadd float %18, %20
  %22 = getelementptr inbounds float, ptr %16, i64 %13
  store float %21, ptr %22, align 4
  br label %"_Z18__acpp_sscp_kernelIN7hipsycl4glue15__sscp_dispatch18basic_parallel_forIZZ4mainENK3$_0clERNS0_4sycl7handlerEEUlNS5_2idILi1EEEE_Li1EEEEvRKT_.old.exit"

"_Z18__acpp_sscp_kernelIN7hipsycl4glue15__sscp_dispatch18basic_parallel_forIZZ4mainENK3$_0clERNS0_4sycl7handlerEEUlNS5_2idILi1EEEE_Li1EEEEvRKT_.old.exit": ; preds = %4, %_ZN7hipsycl4glue15__sscp_dispatch16item_is_in_rangeILi1ELb0EEEbRKNS_4sycl4itemIXT_EXT0_EEERKNS3_5rangeIXT_EEERKNS3_2idIXT_EEE.exit.i.i
  ret void
}

attributes #0 = { mustprogress nocallback nofree nosync nounwind willreturn memory(inaccessiblemem: write) }
attributes #1 = { "frame-pointer"="all" "no-trapping-math"="true" "stack-protector-buffer-size"="8" }
attributes #2 = { nounwind }

!llvm.module.flags = !{!0, !1, !2}
!llvm.ident = !{!3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3, !3}
!hipsycl.sscp.annotations = !{!4, !5}

!0 = !{i32 1, !"wchar_size", i32 4}
!1 = !{i32 7, !"uwtable", i32 2}
!2 = !{i32 7, !"frame-pointer", i32 2}
!3 = !{!"clang version 19.1.7 (Fedora 19.1.7-2.fc41)"}
!4 = distinct !{null, !"hipsycl_kernel_dimension", i32 1}
!5 = !{ptr undef, !"hipsycl_kernel_dimension", i32 1}
!6 = !{i32 128, i32 1, i32 1}
;----------------- End AdaptiveCpp IR dump ---------------

EDIT:
Yes, directory in my container is writable and temporary files are writing.
Yes, OMP is working. At the build stage of ACPP, I can disable L0 and OCL.
But then I won't be able to use the graphics card. It's not that I need it at this stage, but I'm curious as to why it doesn't work.

0 replies

illuhad · 2025-02-23T03:35:31Z

illuhad
Feb 23, 2025
Maintainer

I’m reviewing the CMake logs.
I think that CMake is unable to find the translator.

No, this output refers to spirv-tools which is an unnecessary optional dependency of the llvm-spirv translator.
AdaptiveCpp builds its own llvm-spirv translator during the build process. The fact that you get this output shows that AdaptiveCpp is building the llvm-spirv translator.

Please do not install your own llvm-spirv packages, as those will not have the necessary patch applied! AdaptiveCpp will look for llvm-spirv in its own internal directories anyway (which is to where it will install its patched llvm-spirv), so external packages should not make a difference anyway.

I do not see in your cmake output that it attempts to apply the patch.
Please check whether you have the patch command installed and available.
If not, install patch, clean your build directory, and try reinstalling AdaptiveCpp!

main.cpp.ll

Thank you, this IR looks fine. With this IR, the behavior you are getting is consistent with what you'd get if the llvm-spirv translator does not have the patch installed.

Yes, directory in my container is writable and temporary files are writing.

Okay, thanks for confirming!

0 replies

AlekseiSudar · 2025-02-23T18:40:30Z

AlekseiSudar
Feb 23, 2025
Author

Correct! It turned out that the patch command was missing. After installing it, everything works as expected.

Thank you for your help!
And thanks for developing this project!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Invalid SPIR-V module: Casts from private/local/global address space are allowed only to generic #1728

Uh oh!

{{title}}

Uh oh!

Replies: 4 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Invalid SPIR-V module: Casts from private/local/global address space are allowed only to generic #1728

Uh oh!

AlekseiSudar Feb 22, 2025

Replies: 4 comments

Uh oh!

Uh oh!

illuhad Feb 22, 2025 Maintainer

Uh oh!

Uh oh!

AlekseiSudar Feb 22, 2025 Author

Uh oh!

Uh oh!

illuhad Feb 23, 2025 Maintainer

Uh oh!

AlekseiSudar Feb 23, 2025 Author

AlekseiSudar
Feb 22, 2025

illuhad
Feb 22, 2025
Maintainer

AlekseiSudar
Feb 22, 2025
Author

illuhad
Feb 23, 2025
Maintainer

AlekseiSudar
Feb 23, 2025
Author