o j9:jR@sddlZddlZddlmZddlZz ddlmZdZWn e y)dZdZYnwddl m Z dej fdd Zdej fd d Zdej fd d ZdeddfddZdedefddZdej fddZdej fddZdej fddZdeefddZ    d0dedededBdedBdedBd edeeeffd!d"ZGd#d$d$ZGd%d&d&Z d1d'eeBd(eedBdee ed&fBfd)d*Z! d2d+ed,ed-edefd.d/Z dS)3N)Any)runtimeTF)_get_device_indexreturnc Cszddl}tt|dd}Wn#ttfy5tjdkr.tdt j j dd}ntd}Ynw|j |_ |j|_|j|_|j|_|j|_|S)Nramdhip64win32 amdhip64_.dllzlibamdhip64.so)rocm_sdkctypesCDLLstrfind_libraries ImportError IndexErrorsysplatformtorchversionhiphipGetErrorStringcuGetErrorStringhipModuleLoadDatacuModuleLoadDatahipModuleGetFunctioncuModuleGetFunctionhipModuleLaunchKernelcuLaunchKernelhipFuncSetAttributecuFuncSetAttribute)r libr!X/home/nk/hobo-godmode/plappi-mvp/.venv/lib/python3.10/site-packages/torch/cuda/_utils.py_get_hip_runtime_librarys  r#cCstjdkr tdStdS)Nrz nvcuda.dllz libcuda.so.1)rrr r r!r!r!r"_get_cuda_library-s   r$cCtjjrtStSN)rrrr#r$r!r!r!r"_get_gpu_runtime_library5sr'resultcCsR|dkrdSt}t}||t||jdur |jnd}td|)NrUnknown CUDA error CUDA error: )r c_char_pr'rbyrefvaluedecode RuntimeError)r(err_strlibcuda error_messager!r!r" _check_cuda=sr3cCs~tstd|^}}|tjjkr+t|\}}t|tr |}td|d|dt |dkr3dSt |dkr=|dS|S)aCheck a cuda.bindings (cuda-python) call result for errors. All cuda.bindings runtime calls return ``(error, *outputs)``. This helper unpacks the tuple, raises on non-success, and returns the outputs (``None`` for zero outputs, scalar for one, tuple otherwise). zcuda.bindings is not availabler*z ()rN) _HAS_CUDA_BINDINGSr/_cuda_bindings_runtime cudaError_t cudaSuccesscudaGetErrorString isinstancebytesr.len)r(errout_r0r!r!r"_check_cuda_bindingsIs$   rAc Cszddl}tt|dd}Wn0ttfyBtjdkr;d dt j j ddt j j dg}td|d}ntd}Ynw|j |_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_|j|_ |S) Nrhiprtcr0r z libhiprtc.so)!r r r r rrrrrjoinrrrhiprtcGetErrorStringnvrtcGetErrorStringhiprtcCreateProgramnvrtcCreateProgramhiprtcDestroyProgramnvrtcDestroyProgramhiprtcCompileProgramnvrtcCompileProgramhiprtcGetCodeSizenvrtcGetCUBINSize hiprtcGetCode nvrtcGetCUBINhiprtcGetProgramLogSizenvrtcGetProgramLogSizehiprtcGetProgramLognvrtcGetProgramLoghiprtcAddNameExpressionnvrtcAddNameExpressionhiprtcGetLoweredNamenvrtcGetLoweredName)r r version_strr!r!r"_get_hiprtc_libraryfs.   r\c Csrttjjdd}tjdkrd|dg}nd|dg}|D]}zt|WSt y4Yq!wt d) N.rrnvrtc64_z0_0.dllz libnvrtc.so.z libnvrtc.soz Could not find any NVRTC library) intrrcudasplitrrr r OSError) major_version nvrtc_libslib_namer!r!r"_get_nvrtc_librarys   rfcCr%r&)rrrr\rfr!r!r!r"_get_gpu_rtc_librarysrgcs>ddlm}m}dhfdd|D}tjjr|||S)z Get HIPCC/NVCC flags that are compatible with NVRTC compilation. Returns: List of HIPCC/NVCC flags that can be safely used with NVRTC. r)COMMON_HIPCC_FLAGSCOMMON_NVCC_FLAGSz--expt-relaxed-constexprcsg|]}|vr|qSr!r!.0flagnvrtc_unsupported_flagsr!r" sz1_get_gpu_rtc_compatible_flags..)torch.utils.cpp_extensionrhrirrrextend)rhricompatible_flagsr!rmr"_get_gpu_rtc_compatible_flagss  rs kernel_source kernel_namecompute_capabilitycuda_include_dirs nvcc_optionsauto_pchc sddl}tddtddffdd }|d}|dur8|j|j} |jjr0| j }n| j | j }g} |jjrI| d|n | d |dd l m} | d } | D] } | d | q_|r}|D] }| d |qp|rt|jjd krtd|jj|durg}| d|r|D] }| |dqt}| dd|Dt| }tj|| }t}|t|||dddd|d}||||||}|krt}|t|t|j}||t d|j!t}|"|t|t|j}|#||t}|$||t||jdurO|j!}nd}%t||j&|fS)a Compiles a CUDA kernel using NVRTC and returns the PTX code. Args: kernel_source (str): The CUDA kernel source code as a string kernel_name (str): The name of the kernel function to compile compute_capability (str, None): The compute capability to target (e.g., "86"). If None, will detect from current device. cuda_include_dirs (list, None): List of directories containing CUDA headers nvcc_options (list, None): Additional options to pass to NVRTC auto_pch (bool): Enable automatic precompiled headers (CUDA 12.8+) Returns: Tuple[bytes, str]: The compiled PTX code and mangled kernel name rNr(rcsL|kr$t}|t||jdur|jnd}td|dS)Nr)r*)r r+rHr,r-r.r/)r(r0r2 NVRTC_SUCCESSlibnvrtcr!r" check_nvrtcs  z#_nvrtc_compile..check_nvrtcutf-8z--offload-arch=z--gpu-architecture=sm_) include_pathsr`z-Iz12.8zPCH requires CUDA 12.8+, got z--pchcSsg|]}|dqS)r~)encoderjr!r!r"rosz"_nvrtc_compile..z.cuzKernel compilation failed: rC)' torch.cudargr_rr`get_device_propertiescurrent_devicerr gcnArchNamemajorminorappendrprr AssertionErrorrsrqr=r r+c_void_prJr,rXrNc_size_trTcreate_string_bufferr-rVr/r.rPrRrZrLraw)rtrurvrwrxryrr} source_bytespropsoptionsrcuda_include_paths cuda_path directoryoptionnvrtc_compatible_flags num_options options_arrayprog c_kernel_namereslog_sizelog binary_sizebinaryc_mangled_name mangled_namer!rzr"_nvrtc_compiles             rc@s2eZdZdejddfddZdeddfdd ZdS) _CudaModulemodulerNcCs||_i|_dSr&)_module_kernels)selfrr!r!r"__init__Js z_CudaModule.__init__name _CudaKernelc Cs||jvr |j|Sddlm}|}t}zt|t||j| dt ||j}||j|<|WSt yJ}z t d|d|d}~ww)Nr)r'r~zNo kernel named 'z' in this module) rtorch.cuda._utilsr'r rr3rr,rrrr/AttributeError)rrr'r1funckernelr>r!r!r" __getattr__Ns$     z_CudaModule.__getattr__)__name__ __module__ __qualname__r rrr rr!r!r!r"rIsrc @seZdZdZdejdejddfddZ    dd eeeefd eeeefd e dBd ede dBddf ddZ deddfddZ dS)rzT Represents a compiled CUDA kernel that can be called with PyTorch tensors. rrrNcCs||_||_d|_dS)Nr)rr_max_shared_mem_bytes)rrrr!r!r"rls z_CudaKernel.__init__r5r5r5rgridblockargs shared_memstreamcCsddl}|jj}|sg}g}g} |D]Y} t| |jr?| js*| jr&| s*t dt | } | | | t | qt| trRt | } | t | qt| tret | } | t | qtdt| t j t| }t| D] \}} t | t j ||<qz|durddl}|j}|dkr|jdks||jkr|jdkrdnd|jd}td |d |d t||j|d|d |d |d|d |d ||j|d dS)a Call the compiled CUDA kernel Args: grid (tuple): Grid dimensions (grid_x, grid_y, grid_z) block (tuple): Block dimensions (block_x, block_y, block_z) args (list): List of arguments to pass to the kernel. PyTorch tensor arguments will be automatically converted to pointers. shared_mem (int): Shared memory size in bytes stream (torch.cuda.Stream): CUDA stream to use. If None, uses current stream. rNz?All tensor arguments must be CUDA tensors or pinned CPU tensorszUnsupported argument type: znot configuredzonly z bytes configuredzKernel requires z' bytes of shared memory (>= 48KB), but ze. Call kernel.set_shared_memory_config(shared_mem) after compilation and before launching the kernel.r5rE) rr`_utilsr'r;Tensoris_cudais_cpu is_pinned ValueErrorr rdata_ptrrr,r_c_intfloatc_double TypeErrortyper= enumeratecastrcurrent_streamrr/r3rr_as_parameter_)rrrrrrrr1processed_argsc_argsargptrrr c_args_arrayiconfigured_msgr!r!r"__call__qsl          z_CudaKernel.__call__shared_mem_bytescCs|dkr ||_dSt}tj}tjjr|jdkrdnd}nt|dd}||kr4t d|d|dd }t | |j ||||_dS) Nrgfx950iishared_memory_per_block_optinzRequested shared memory (z bytes) exceeds device limit (z= bytes). Consider reducing block size or shared memory usage.) rr'rr`rrrrgetattrr/r3rr)rrr1 device_propsmax_shared_mem+cudaFuncAttributeMaxDynamicSharedMemorySizer!r!r"set_shared_memory_configs4  z$_CudaKernel.set_shared_memory_config)rrNrN) rrr__doc__r rrtupler_listrrrr!r!r!r"rgs,   arptx kernel_namesc Csddl}t}t|tr|d}t}|j}|t | t ||Wdn1s2wY|s=t |Si}|D]}t}t | t |||dt||||<qA|S)a, Loads a CUDA module from PTX code and returns a module object that can access kernels. Args: ptx (bytes or str): The PTX code to load kernel_names (list, optional): List of kernel names to extract from the module. If None, will return a module object with __getattr__. Returns: object: If kernel_names is None, returns a module object with __getattr__ to access kernels. If kernel_names is provided, returns a dict mapping kernel names to _CudaKernel objects. rNr~)rr'r;r rr rr`rr3rr,rrr) rrrr1rrkernelsrrr!r!r"_cuda_load_modules*   rdeviceoptional allow_cpucCst|tr|St|trt|}t|tjr2|r&|jdvr%td|n |jdkr2td|tjsAt|tj jrA|j St |||S)aGet the device index from :attr:`device`, which can be a torch.device object, a Python integer, or ``None``. If :attr:`device` is a torch.device object, returns the device index if it is a CUDA device. Note that for a CUDA device without a specified index, i.e., ``torch.device('cuda')``, this will return the current default CUDA device if :attr:`optional` is ``True``. If :attr:`allow_cpu` is ``True``, CPU devices will be accepted and ``-1`` will be returned in this case. If :attr:`device` is a Python integer, it is returned as is. If :attr:`device` is ``None``, this will return the current default CUDA device if :attr:`optional` is ``True``. )r`cpuz(Expected a cuda or cpu device, but got: r`z!Expected a cuda device, but got: ) r;r_r rrrrjit is_scriptingr`idx_torch_get_device_index)rrrr!r!r"r-s        r)NNNFr&)FF)"r rtypingrr cuda.bindingsrr7r6r torch._utilsrrr r#r$r'r_r3rAr\rfrgrr rsboolrr<rrrdictrr!r!r!r"s|          1