Very low/odd performance and segfault when using large buffers in OpenCL on RX580
System information
inxi -GCS -xx
output:
System: Host: (omitted) Kernel: 5.4.0-4-amd64 x86_64 bits: 64 compiler: gcc v: 9.2.1 Console: N/A dm: N/A
Distro: Debian GNU/Linux bullseye/sid
CPU: Topology: 8-Core model: AMD Ryzen 7 3700X bits: 64 type: MT MCP arch: Zen L2 cache: 4096 KiB
flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm bogomips: 114984
Speed: 2196 MHz min/max: 2200/3600 MHz Core speeds (MHz): 1: 2196 2: 2198 3: 2196 4: 2184 5: 2189 6: 2185 7: 2196
8: 2195 9: 2196 10: 2194 11: 2207 12: 2196 13: 2200 14: 2195 15: 2192 16: 2195
Graphics: Device-1: Advanced Micro Devices [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590]
vendor: Sapphire Limited driver: amdgpu v: kernel bus ID: 07:00.0 chip ID: 1002:67df
Display: server: X.org 1.20.7 driver: modesetting unloaded: fbdev,vesa tty: 238x64
Message: Advanced graphics data unavailable in console. Try -G --display
Describe the issue
I've just configured a new machine and am testing OpenCL on this Radeon RX 580 GPU (clinfo
output at the end, for clarity). I'm running the reduction
test from this repository.
With no options, the tool tries a reduction with 1717986918 elements requiring ~6GB of VRAM (the maximum allocation size advertised by the platform). Device allocations are a CL_MEM_READ_ONLY
OpenCL buffer (no *_HOST_PTR
flags) to hold the input data, and a much smaller (256 elements) CL_MEM_READ_WRITE
OpenCL buffer to be used as scratch during the first reduction step and to hold the final result as well.
The data is initialized on host in a calloc()
ed area, and then uploaded to the device through an async clEnqueueWriteBuffer
(using a synchronous upload doesn't change the results, though). The reduction kernel is run multiple times with different configurations, and finally the result is downloaded and all resources are released, in inverse allocation order. The unexpected behavior has three symptoms, that seem related (hence the single bug report):
- the initial upload reports impossibly short times: around 0.00064ms, for an effective bandwidth of millions of GB/sec, as if the data wasn't actually being copied;
- the reduction kernel runtimes show abysmally low performance, around 13GB/sec;
- when the
calloc()
ed host data pointer isfree()
ed, the program segfaults.
Additionally, running the program with a smaller dataset (up to 2^25) seems to work better: while the upload timing is still wrong (same value), the reduction kernel runs an order of magnitude faster (> 100GB/sec), and there is no segfault at the end. With a data size of 2^26 performance drops to the tens of GB/sec again, but there is no segfault. With a data size of 2 ^30 I get a segfault during the clEnqueueWriteBuffer
somewhere deep in Mesa. I'm still trying to find if I can get debug symbols for the Debian sid version of Mesa so I can provide a meaningful backtrace. All other logs are appeneded.
Regression
Unknown.
Log files as attachment
clinfo
output:
Number of platforms 3
Platform Name Clover
Platform Vendor Mesa
Platform Version OpenCL 1.1 Mesa 20.0.2
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd
Platform Extensions function suffix MESA
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
Platform Version OpenCL 2.0 AMD-APP (1912.5)
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_amd_event_callback cl_amd_offline_devices
Platform Extensions function suffix AMD
Platform Name Intel(R) OpenCL
Platform Vendor Intel(R) Corporation
Platform Version OpenCL 1.2 LINUX
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_intel_exec_by_local_thread cl_khr_depth_images cl_khr_3d_image_writes cl_khr_fp64
Platform Extensions function suffix INTEL
Platform Name Clover
Number of devices 1
Device Name Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
Device Vendor AMD
Device Vendor ID 0x1002
Device Version OpenCL 1.1 Mesa 20.0.2
Driver Version 20.0.2
Device OpenCL C Version OpenCL C 1.1
Device Type GPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Max compute units 36
Max clock frequency 1366MHz
Max work item dimensions 3
Max work item sizes 256x256x256
Max work group size 256
Preferred work group size multiple 64
Preferred / native vector sizes
char 16 / 16
short 8 / 8
int 4 / 4
long 2 / 2
half 8 / 8 (cl_khr_fp16)
float 4 / 4
double 2 / 2 (cl_khr_fp64)
Half-precision Floating-point support (cl_khr_fp16)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Single-precision Floating-point support (core)
Denormals No
Infinity and NANs Yes
Round to nearest Yes
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Correctly-rounded divide and sqrt operations No
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 8589934592 (8GiB)
Error Correction support No
Max memory allocation 6871947673 (6.4GiB)
Unified memory for Host and Device No
Minimum alignment for any data type 128 bytes
Alignment of base address 32768 bits (4096 bytes)
Global Memory cache type None
Image support No
Local memory type Local
Local memory size 32768 (32KiB)
Max number of constant args 16
Max constant buffer size 2147483647 (2GiB)
Max size of kernel argument 1024
Queue properties
Out-of-order execution No
Profiling Yes
Profiling timer resolution 0ns
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
Device Extensions cl_khr_byte_addressable_store cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_fp64 cl_khr_fp16
Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name AMD Ryzen 7 3700X 8-Core Processor
Device Vendor AuthenticAMD
Device Vendor ID 0x1002
Device Version OpenCL 1.2 AMD-APP (1912.5)
Driver Version 1912.5 (sse2,avx)
Device OpenCL C Version OpenCL C 1.2
Device Type CPU
Device Board Name (AMD) (n/a)
Device PCI-e ID (AMD) <printDeviceInfo:9: get CL_DEVICE_PCIE_ID_AMD : error -30>
Device Topology (AMD) (n/a)
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 16
Max clock frequency 2192MHz
Device Partition (core, cl_ext_device_fission)
Max number of sub-devices 16
Supported partition types equally, by counts, by affinity domain
Supported affinity domains L3 cache, L2 cache, L1 cache, next partitionable
Supported partition types (ext) equally, by counts, by affinity domain
Supported affinity domains (ext) L3 cache, L2 cache, L1 cache, next fissionable
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 1024
Preferred work group size multiple 1
Preferred / native vector sizes
char 16 / 16
short 8 / 8
int 4 / 4
long 2 / 2
half 4 / 4 (n/a)
float 8 / 8
double 4 / 4 (cl_khr_fp64)
Half-precision Floating-point support (n/a)
Single-precision Floating-point support (core)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 67431702528 (62.8GiB)
Error Correction support No
Max memory allocation 16857925632 (15.7GiB)
Unified memory for Host and Device Yes
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Global Memory cache type Read/Write
Global Memory cache size 32768 (32KiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 16
Max size for 1D images from buffer 65536 pixels
Max 1D or 2D image array size 2048 images
Max 2D image size 8192x8192 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 64
Local memory type Global
Local memory size 32768 (32KiB)
Max number of constant args 8
Max constant buffer size 65536 (64KiB)
Max size of kernel argument 4096 (4KiB)
Queue properties
Out-of-order execution No
Profiling Yes
Prefer user sync for interop Yes
Profiling timer resolution 1ns
Profiling timer offset since Epoch (AMD) 1585428104663123418ns (Sat Mar 28 21:41:44 2020)
Execution capabilities
Run OpenCL kernels Yes
Run native kernels Yes
SPIR versions 1.2
printf() buffer size 65536 (64KiB)
Built-in kernels (n/a)
Device Extensions cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_ext_device_fission cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_spir cl_khr_gl_event
Platform Name Intel(R) OpenCL
Number of devices 1
Device Name AMD Ryzen 7 3700X 8-Core Processor
Device Vendor Intel(R) Corporation
Device Vendor ID 0x8086
Device Version OpenCL 1.2 (Build 83073)
Driver Version 1.2.0.83073
Device OpenCL C Version OpenCL C 1.2
Device Type CPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 16
Max clock frequency 0MHz
Device Partition (core)
Max number of sub-devices 16
Supported partition types by counts, equally, by names (Intel)
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 8192x8192x8192
Max work group size 8192
Preferred work group size multiple 128
Preferred / native vector sizes
char 1 / 32
short 1 / 16
int 1 / 8
long 1 / 4
half 0 / 0 (n/a)
float 1 / 8
double 1 / 4 (cl_khr_fp64)
Half-precision Floating-point support (n/a)
Single-precision Floating-point support (core)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Correctly-rounded divide and sqrt operations No
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 67431702528 (62.8GiB)
Error Correction support No
Max memory allocation 16857925632 (15.7GiB)
Unified memory for Host and Device Yes
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Global Memory cache type Read/Write
Global Memory cache size 524288 (512KiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 480
Max size for 1D images from buffer 1053620352 pixels
Max 1D or 2D image array size 2048 images
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 480
Max number of write image args 480
Local memory type Global
Local memory size 32768 (32KiB)
Max number of constant args 480
Max constant buffer size 131072 (128KiB)
Max size of kernel argument 3840 (3.75KiB)
Queue properties
Out-of-order execution Yes
Profiling Yes
Local thread execution (Intel) Yes
Prefer user sync for interop No
Profiling timer resolution 1ns
Execution capabilities
Run OpenCL kernels Yes
Run native kernels Yes
SPIR versions 1.2
printf() buffer size 1048576 (1024KiB)
Built-in kernels (n/a)
Device Extensions cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_spir cl_intel_exec_by_local_thread cl_khr_depth_images cl_khr_3d_image_writes cl_khr_fp64
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) Clover
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [MESA]
clCreateContext(NULL, ...) [default] Success [MESA]
clCreateContext(NULL, ...) [other] Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name Clover
Device Name Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name Clover
Device Name Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name Clover
Device Name Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.2.12
ICD loader Profile OpenCL 2.2
AMD_DEBUG=info,compute,vm ./reduction
output:
VM start=0x100000000 end=0x100000200 | Buffer 512 bytes
VM start=0x100020000 end=0x100030000 | Buffer 65536 bytes
VM start=0x100000200 end=0x100000400 | Buffer 512 bytes
VM start=0x100000400 end=0x100000600 | Buffer 512 bytes
VM start=0x100000600 end=0x100000800 | Buffer 512 bytes
VM start=0x100030000 end=0x100040000 | Buffer 65536 bytes
VM start=0x100000800 end=0x100000A00 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100400000 end=0x199D9A000 | Buffer 2576982016 bytes
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
Will use device 0 on platform 0
Device info:
pci (domain:bus:dev.func): 0000:07:00.0
name = POLARIS10
marketing_name = Radeon RX 580 Series
is_pro_graphics = 0
pci_id = 0x67df
family = 63
chip_class = 10
family_id = 130
chip_external_rev = 81
clock_crystal_freq = 25000
Features:
has_graphics = 1
num_rings[RING_GFX] = 1
num_rings[RING_DMA] = 2
num_rings[RING_COMPUTE] = 4
num_rings[RING_UVD] = 1
num_rings[RING_VCE] = 1
num_rings[RING_UVD_ENC] = 1
num_rings[RING_VCN_DEC] = 0
num_rings[RING_VCN_ENC] = 0
num_rings[RING_VCN_JPEG] = 0
has_clear_state = 1
has_distributed_tess = 1
has_dcc_constant_encode = 0
has_rbplus = 0
rbplus_allowed = 0
has_load_ctx_reg_pkt = 1
has_out_of_order_rast = 1
cpdma_prefetch_writes_memory = 1
has_gfx9_scissor_bug = 0
has_tc_compat_zrange_bug = 1
has_msaa_sample_loc_bug = 1
has_ls_vgpr_init_bug = 0
Display features:
use_display_dcc_unaligned = 0
use_display_dcc_with_retile_blit = 0
Memory info:
pte_fragment_size = 2097152
gart_page_size = 4096
gart_size = 8192 MB
vram_size = 8192 MB
vram_vis_size = 256 MB
gds_size = 64 kB
gds_gfx_partition_size = 0 kB
max_alloc_size = 6554 MB
min_alloc_size = 512
address32_hi = 0
has_dedicated_vram = 1
num_sdp_interfaces = 0
num_tcc_blocks = 8
tcc_cache_line_size = 64
tcc_harvested = 0
pc_lines = 0
CP info:
gfx_ib_pad_with_type2 = 0
ib_start_alignment = 256
me_fw_version = 167
me_fw_feature = 49
pfp_fw_version = 254
pfp_fw_feature = 49
ce_fw_version = 140
ce_fw_feature = 49
Multimedia info:
has_hw_decode = 1
uvd_enc_supported = 1
uvd_fw_version = 25300992
vce_fw_version = 890897152
vce_harvest_config = 2
Kernel & winsys capabilities:
drm = 3.35.0
has_userptr = 1
has_syncobj = 1
has_syncobj_wait_for_submit = 1
has_fence_to_handle = 1
has_ctx_priority = 1
has_local_buffers = 0
kernel_flushes_hdp_before_ib = 1
htile_cmask_support_1d_tiling = 1
si_TA_CS_BC_BASE_ADDR_allowed = 1
has_bo_metadata = 1
has_gpu_reset_status_query = 1
has_eqaa_surface_allocator = 1
has_format_bc1_through_bc7 = 1
kernel_flushes_tc_l2_after_ib = 1
has_indirect_compute_dispatch = 1
has_unaligned_shader_loads = 1
has_sparse_vm_mappings = 1
has_2d_tiling = 1
has_read_registers_query = 1
has_gds_ordered_append = 1
has_scheduled_fence_dependency = 1
Shader core info:
max_shader_clock = 1366
num_good_compute_units = 36
num_good_cu_per_sh = 9
max_se = 4
max_sh_per_se = 1
max_wave64_per_simd = 8
num_physical_sgprs_per_simd = 800
num_physical_wave64_vgprs_per_simd = 256
Render backend info:
pa_sc_tile_steering_override = 0x0
num_render_backends = 8
num_tile_pipes = 8
pipe_interleave_bytes = 256
enabled_rb_mask = 0xff
max_alignment = 524288
pbb_max_alloc_count = 0
GB_ADDR_CONFIG: 0x22011003
num_pipes = 8
pipe_interleave_size = 256
bank_interleave_size = 1
num_shader_engines = 2
shader_engine_tile_size = 32
num_gpus = 0 (raw)
multi_gpu_tile_size = 2 (raw)
row_size = 4096
num_lower_pipes = 0 (raw)
3 OpenCL platforms found:
platform 0: Clover (Mesa)
platform 1: AMD Accelerated Parallel Processing (Advanced Micro Devices, Inc.)
platform 2: Intel(R) OpenCL (Intel(R) Corporation)
using platform 0
1 devs found:
dev 0: Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
using device 0
Device has 36 compute units, 32768 local memory, 8589934592 (8192MB) separate memory with max alloc of 6871947673 (6553MB)
Reduction will use 216 groups (6 groups/CU)
Reduction style: interleaved
will process 1717986918 (1638M) elements
1717986918 elements generated, sum 3092376453, data size 6871947672 (6553MB)
compiler options: -Isrc/ -DREDUCTION_STYLE=0 -DVECSIZE=1 -DUNROLL=1 -DTYPE=int -DOP=ADD_OP -DOP_NULL=ADD_OP_NULL
loading src/reduction_kernels.ocl
read 5862 bytes from rVM start=0x199E00000 end=0x199E20000 | Buffer 131072 bytes
VM start=0x100000A00 end=0x100000C00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x002c0041 COMPUTE_PGM_RSRC2: 0x00000090
VM start=0x400000 end=0x440000 | Buffer 262144 bytes
VM start=0x100200400 end=0x100200600 | Buffer 512 bytes
VM start=0x100002000 end=0x100002400 | Buffer 1024 bytes
VM start=0x100002400 end=0x100002800 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
VM start=0x440000 end=0x480000 | Buffer 262144 bytes
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 1717986918
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100002800 end=0x100002C00 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
VM start=0x100200800 end=0x100200A00 | Buffer 512 bytes
VM start=0x100200A00 end=0x100200C00 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100002C00 end=0x100003000 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 1717986918
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100003000 end=0x100003400 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100201800 end=0x100201A00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 1717986918
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201A00 end=0x100201C00 | Buffer 512 bytes
VM start=0x100201C00 end=0x100201E00 | Buffer 512 bytes
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201E00 end=0x100202000 | Buffer 512 bytes
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
eduction_kernels.ocl
kernel prefers multiples of 64 up to 256
memory upload runtime: 0.00096ms
Upload effective bandwidth: 7158278.83 GB/s
Group size: 64, 256
Kernel pass #1 runtime: 503.581ms
Kernel pass #2 runtime: 0.0048ms
Total runtime: 503.586ms
Bandwidth: 13.65 GB/s
Reduction performance: 3.412 GE/s
SUMMARY: 64 Ã 216 + 256 Ã 1 => 503.59ms | 13.65 GB/s | 3.41 GE/s
Group size: 128, 256
Kernel pass #1 runtime: 504.348ms
Kernel pass #2 runtime: 0.00464ms
Total runtime: 504.354ms
Bandwidth: 13.63 GB/s
Reduction performance: 3.406 GE/s
SUMMARY: 128 Ã 216 + 256 Ã 1 => 504.35ms | 13.63 GB/s | 3.41 GE/s
Group size: 256, 256
Kernel pass #1 runtime: 535.717ms
Kernel pass #2 runtime: 0.00048ms
Total runtime: 535.718ms
Bandwidth: 12.83 GB/s
Reduction performance: 3.207 GE/s
SUMMARY: 256 Ã 216 + 256 Ã 1 => 535.72ms | 12.83 GB/s | 3.21 GE/s
memory download runtime: 13.7378ms
total download runtime: 13.7378ms for 3.8147e-06MB (2.71172e-07 GB/s)
Corresponding dmesg
:
[Mar29 12:36] gmc_v8_0_process_interrupt: 246106 callbacks suppressed
[ +0.000005] amdgpu 0000:07:00.0: GPU fault detected: 146 0x0cd0880c for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.003692] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00199D9A
[ +0.001861] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C08800C
[ +0.001837] amdgpu 0000:07:00.0: VM fault (0x0c, vmid 6, pasid 32769) at page 1678746, read from 'TC6' (0x54433600) (136)
[ +0.001850] amdgpu 0000:07:00.0: GPU fault detected: 147 0x00588802 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001850] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x00199D9E
[ +0.001860] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C08400C
[ +0.001839] amdgpu 0000:07:00.0: VM fault (0x0c, vmid 6, pasid 32769) at page 1678750, read from 'TC7' (0x54433700) (132)
[ +0.001850] amdgpu 0000:07:00.0: GPU fault detected: 147 0x03480402 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001852] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001A1E39
[ +0.001862] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C088002
[ +0.001839] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1711673, read from 'TC6' (0x54433600) (136)
[ +0.001851] amdgpu 0000:07:00.0: GPU fault detected: 147 0x0268c402 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001853] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001A8353
[ +0.001864] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C044002
[ +0.001839] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1737555, read from 'TC5' (0x54433500) (68)
[ +0.001851] amdgpu 0000:07:00.0: GPU fault detected: 147 0x06d84802 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001854] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001AE790
[ +0.001863] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C008002
[ +0.001840] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1763216, read from 'TC0' (0x54433000) (8)
[ +0.001850] amdgpu 0000:07:00.0: GPU fault detected: 147 0x06984802 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001851] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001B4B52
[ +0.001862] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C0C8002
[ +0.001839] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1788754, read from 'TC2' (0x54433200) (200)
[ +0.001848] amdgpu 0000:07:00.0: GPU fault detected: 147 0x0a188402 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001841] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001BB502
[ +0.001824] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C088002
[ +0.001825] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1815810, read from 'TC6' (0x54433600) (136)
[ +0.001840] amdgpu 0000:07:00.0: GPU fault detected: 147 0x0e30c802 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001839] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001C133B
[ +0.001826] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C008002
[ +0.001824] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1839931, read from 'TC0' (0x54433000) (8)
[ +0.001840] amdgpu 0000:07:00.0: GPU fault detected: 147 0x00b0c802 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001839] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001C769B
[ +0.001825] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C0C8002
[ +0.001824] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1865371, read from 'TC2' (0x54433200) (200)
[ +0.001840] amdgpu 0000:07:00.0: GPU fault detected: 147 0x02f8c402 for process reduction pid 129486 thread reduction:cs0 pid 129488
[ +0.001840] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_ADDR 0x001CDE3E
[ +0.001825] amdgpu 0000:07:00.0: VM_CONTEXT1_PROTECTION_FAULT_STATUS 0x0C044002
[ +0.001825] amdgpu 0000:07:00.0: VM fault (0x02, vmid 6, pasid 32769) at page 1891902, read from 'TC5' (0x54433500) (68)
AMD_DEBUG=info,compute,vm ./reduction --elements $((32*1024*1024))
output (“working” case):
VM start=0x100000000 end=0x100000200 | Buffer 512 bytes
VM start=0x100020000 end=0x100030000 | Buffer 65536 bytes
VM start=0x100000200 end=0x100000400 | Buffer 512 bytes
VM start=0x100000400 end=0x100000600 | Buffer 512 bytes
VM start=0x100000600 end=0x100000800 | Buffer 512 bytes
VM start=0x100030000 end=0x100040000 | Buffer 65536 bytes
VM start=0x100000800 end=0x100000A00 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100400000 end=0x108400000 | Buffer 134217728 bytes
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
Will use device 0 on platform 0
Device info:
pci (domain:bus:dev.func): 0000:07:00.0
name = POLARIS10
marketing_name = Radeon RX 580 Series
is_pro_graphics = 0
pci_id = 0x67df
family = 63
chip_class = 10
family_id = 130
chip_external_rev = 81
clock_crystal_freq = 25000
Features:
has_graphics = 1
num_rings[RING_GFX] = 1
num_rings[RING_DMA] = 2
num_rings[RING_COMPUTE] = 4
num_rings[RING_UVD] = 1
num_rings[RING_VCE] = 1
num_rings[RING_UVD_ENC] = 1
num_rings[RING_VCN_DEC] = 0
num_rings[RING_VCN_ENC] = 0
num_rings[RING_VCN_JPEG] = 0
has_clear_state = 1
has_distributed_tess = 1
has_dcc_constant_encode = 0
has_rbplus = 0
rbplus_allowed = 0
has_load_ctx_reg_pkt = 1
has_out_of_order_rast = 1
cpdma_prefetch_writes_memory = 1
has_gfx9_scissor_bug = 0
has_tc_compat_zrange_bug = 1
has_msaa_sample_loc_bug = 1
has_ls_vgpr_init_bug = 0
Display features:
use_display_dcc_unaligned = 0
use_display_dcc_with_retile_blit = 0
Memory info:
pte_fragment_size = 2097152
gart_page_size = 4096
gart_size = 8192 MB
vram_size = 8192 MB
vram_vis_size = 256 MB
gds_size = 64 kB
gds_gfx_partition_size = 0 kB
max_alloc_size = 6554 MB
min_alloc_size = 512
address32_hi = 0
has_dedicated_vram = 1
num_sdp_interfaces = 0
num_tcc_blocks = 8
tcc_cache_line_size = 64
tcc_harvested = 0
pc_lines = 0
CP info:
gfx_ib_pad_with_type2 = 0
ib_start_alignment = 256
me_fw_version = 167
me_fw_feature = 49
pfp_fw_version = 254
pfp_fw_feature = 49
ce_fw_version = 140
ce_fw_feature = 49
Multimedia info:
has_hw_decode = 1
uvd_enc_supported = 1
uvd_fw_version = 25300992
vce_fw_version = 890897152
vce_harvest_config = 2
Kernel & winsys capabilities:
drm = 3.35.0
has_userptr = 1
has_syncobj = 1
has_syncobj_wait_for_submit = 1
has_fence_to_handle = 1
has_ctx_priority = 1
has_local_buffers = 0
kernel_flushes_hdp_before_ib = 1
htile_cmask_support_1d_tiling = 1
si_TA_CS_BC_BASE_ADDR_allowed = 1
has_bo_metadata = 1
has_gpu_reset_status_query = 1
has_eqaa_surface_allocator = 1
has_format_bc1_through_bc7 = 1
kernel_flushes_tc_l2_after_ib = 1
has_indirect_compute_dispatch = 1
has_unaligned_shader_loads = 1
has_sparse_vm_mappings = 1
has_2d_tiling = 1
has_read_registers_query = 1
has_gds_ordered_append = 1
has_scheduled_fence_dependency = 1
Shader core info:
max_shader_clock = 1366
num_good_compute_units = 36
num_good_cu_per_sh = 9
max_se = 4
max_sh_per_se = 1
max_wave64_per_simd = 8
num_physical_sgprs_per_simd = 800
num_physical_wave64_vgprs_per_simd = 256
Render backend info:
pa_sc_tile_steering_override = 0x0
num_render_backends = 8
num_tile_pipes = 8
pipe_interleave_bytes = 256
enabled_rb_mask = 0xff
max_alignment = 524288
pbb_max_alloc_count = 0
GB_ADDR_CONFIG: 0x22011003
num_pipes = 8
pipe_interleave_size = 256
bank_interleave_size = 1
num_shader_engines = 2
shader_engine_tile_size = 32
num_gpus = 0 (raw)
multi_gpu_tile_size = 2 (raw)
row_size = 4096
num_lower_pipes = 0 (raw)
3 OpenCL platforms found:
platform 0: Clover (Mesa)
platform 1: AMD Accelerated Parallel Processing (Advanced Micro Devices, Inc.)
platform 2: Intel(R) OpenCL (Intel(R) Corporation)
using platform 0
1 devs found:
dev 0: Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
using device 0
Device has 36 compute units, 32768 local memory, 8589934592 (8192MB) separate memory with max alloc of 6871947673 (6553MB)
Reduction will use 216 groups (6 groups/CU)
Reduction style: interleaved
will process 33554432 (32M) elements
33554432 elements generated, sum 16777216, data size 134217728 (128MB)
compiler options: -Isrc/ -DREDUCTION_STYLE=0 -DVECSIZE=1 -DUNROLL=1 -DTYPE=int -DOP=ADD_OP -DOP_NULL=ADD_OP_NULL
loading src/reduction_kernels.ocl
read 5862 bytes from reduction_kVM start=0x108400000 end=0x108420000 | Buffer 131072 bytes
VM start=0x100000A00 end=0x100000C00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x002c0041 COMPUTE_PGM_RSRC2: 0x00000090
VM start=0x400000 end=0x440000 | Buffer 262144 bytes
VM start=0x100200400 end=0x100200600 | Buffer 512 bytes
VM start=0x100002000 end=0x100002400 | Buffer 1024 bytes
VM start=0x100002400 end=0x100002800 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
VM start=0x440000 end=0x480000 | Buffer 262144 bytes
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 33554432
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100002800 end=0x100002C00 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
VM start=0x100200800 end=0x100200A00 | Buffer 512 bytes
VM start=0x100200A00 end=0x100200C00 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100002C00 end=0x100003000 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 33554432
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100003000 end=0x100003400 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100201800 end=0x100201A00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 33554432
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201A00 end=0x100201C00 | Buffer 512 bytes
VM start=0x100201C00 end=0x100201E00 | Buffer 512 bytes
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201E00 end=0x100202000 | Buffer 512 bytes
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
ernels.ocl
kernel prefers multiples of 64 up to 256
memory upload runtime: 0.00048ms
Upload effective bandwidth: 279620.27 GB/s
Group size: 64, 256
Kernel pass #1 runtime: 1.21808ms
Kernel pass #2 runtime: 0.00672ms
Total runtime: 1.22544ms
Bandwidth: 109.5 GB/s
Reduction performance: 27.38 GE/s
SUMMARY: 64 × 216 + 256 × 1 => 1.23ms | 109.53 GB/s | 27.38 GE/s
Group size: 128, 256
Kernel pass #1 runtime: 1.1616ms
Kernel pass #2 runtime: 0.00672ms
Total runtime: 1.16896ms
Bandwidth: 114.8 GB/s
Reduction performance: 28.7 GE/s
SUMMARY: 128 × 216 + 256 × 1 => 1.17ms | 114.82 GB/s | 28.70 GE/s
Group size: 256, 256
Kernel pass #1 runtime: 1.1536ms
Kernel pass #2 runtime: 0.00432ms
Total runtime: 1.1584ms
Bandwidth: 115.9 GB/s
Reduction performance: 28.97 GE/s
SUMMARY: 256 × 216 + 256 × 1 => 1.16ms | 115.87 GB/s | 28.97 GE/s
memory download runtime: 0.06064ms
total download runtime: 0.06064ms for 3.8147e-06MB (6.14329e-05 GB/s)
Parallel sum: 16777216 vs 16777216 (expected: 16777216)
Deltas: 0 vs 0
(with no new messages in dmesg
)
AMD_DEBUG=info,compute,vm ./reduction --elements $((64*1024*1024))
output (poor performance, no segfault):
VM start=0x100000000 end=0x100000200 | Buffer 512 bytes
VM start=0x100020000 end=0x100030000 | Buffer 65536 bytes
VM start=0x100000200 end=0x100000400 | Buffer 512 bytes
VM start=0x100000400 end=0x100000600 | Buffer 512 bytes
VM start=0x100000600 end=0x100000800 | Buffer 512 bytes
VM start=0x100030000 end=0x100040000 | Buffer 65536 bytes
VM start=0x100000800 end=0x100000A00 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100400000 end=0x110400000 | Buffer 268435456 bytes
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
Will use device 0 on platform 0
Device info:
pci (domain:bus:dev.func): 0000:07:00.0
name = POLARIS10
marketing_name = Radeon RX 580 Series
is_pro_graphics = 0
pci_id = 0x67df
family = 63
chip_class = 10
family_id = 130
chip_external_rev = 81
clock_crystal_freq = 25000
Features:
has_graphics = 1
num_rings[RING_GFX] = 1
num_rings[RING_DMA] = 2
num_rings[RING_COMPUTE] = 4
num_rings[RING_UVD] = 1
num_rings[RING_VCE] = 1
num_rings[RING_UVD_ENC] = 1
num_rings[RING_VCN_DEC] = 0
num_rings[RING_VCN_ENC] = 0
num_rings[RING_VCN_JPEG] = 0
has_clear_state = 1
has_distributed_tess = 1
has_dcc_constant_encode = 0
has_rbplus = 0
rbplus_allowed = 0
has_load_ctx_reg_pkt = 1
has_out_of_order_rast = 1
cpdma_prefetch_writes_memory = 1
has_gfx9_scissor_bug = 0
has_tc_compat_zrange_bug = 1
has_msaa_sample_loc_bug = 1
has_ls_vgpr_init_bug = 0
Display features:
use_display_dcc_unaligned = 0
use_display_dcc_with_retile_blit = 0
Memory info:
pte_fragment_size = 2097152
gart_page_size = 4096
gart_size = 8192 MB
vram_size = 8192 MB
vram_vis_size = 256 MB
gds_size = 64 kB
gds_gfx_partition_size = 0 kB
max_alloc_size = 6554 MB
min_alloc_size = 512
address32_hi = 0
has_dedicated_vram = 1
num_sdp_interfaces = 0
num_tcc_blocks = 8
tcc_cache_line_size = 64
tcc_harvested = 0
pc_lines = 0
CP info:
gfx_ib_pad_with_type2 = 0
ib_start_alignment = 256
me_fw_version = 167
me_fw_feature = 49
pfp_fw_version = 254
pfp_fw_feature = 49
ce_fw_version = 140
ce_fw_feature = 49
Multimedia info:
has_hw_decode = 1
uvd_enc_supported = 1
uvd_fw_version = 25300992
vce_fw_version = 890897152
vce_harvest_config = 2
Kernel & winsys capabilities:
drm = 3.35.0
has_userptr = 1
has_syncobj = 1
has_syncobj_wait_for_submit = 1
has_fence_to_handle = 1
has_ctx_priority = 1
has_local_buffers = 0
kernel_flushes_hdp_before_ib = 1
htile_cmask_support_1d_tiling = 1
si_TA_CS_BC_BASE_ADDR_allowed = 1
has_bo_metadata = 1
has_gpu_reset_status_query = 1
has_eqaa_surface_allocator = 1
has_format_bc1_through_bc7 = 1
kernel_flushes_tc_l2_after_ib = 1
has_indirect_compute_dispatch = 1
has_unaligned_shader_loads = 1
has_sparse_vm_mappings = 1
has_2d_tiling = 1
has_read_registers_query = 1
has_gds_ordered_append = 1
has_scheduled_fence_dependency = 1
Shader core info:
max_shader_clock = 1366
num_good_compute_units = 36
num_good_cu_per_sh = 9
max_se = 4
max_sh_per_se = 1
max_wave64_per_simd = 8
num_physical_sgprs_per_simd = 800
num_physical_wave64_vgprs_per_simd = 256
Render backend info:
pa_sc_tile_steering_override = 0x0
num_render_backends = 8
num_tile_pipes = 8
pipe_interleave_bytes = 256
enabled_rb_mask = 0xff
max_alignment = 524288
pbb_max_alloc_count = 0
GB_ADDR_CONFIG: 0x22011003
num_pipes = 8
pipe_interleave_size = 256
bank_interleave_size = 1
num_shader_engines = 2
shader_engine_tile_size = 32
num_gpus = 0 (raw)
multi_gpu_tile_size = 2 (raw)
row_size = 4096
num_lower_pipes = 0 (raw)
3 OpenCL platforms found:
platform 0: Clover (Mesa)
platform 1: AMD Accelerated Parallel Processing (Advanced Micro Devices, Inc.)
platform 2: Intel(R) OpenCL (Intel(R) Corporation)
using platform 0
1 devs found:
dev 0: Radeon RX 580 Series (POLARIS10, DRM 3.35.0, 5.4.0-4-amd64, LLVM 9.0.1)
using device 0
Device has 36 compute units, 32768 local memory, 8589934592 (8192MB) separate memory with max alloc of 6871947673 (6553MB)
Reduction will use 216 groups (6 groups/CU)
Reduction style: interleaved
will process 67108864 (64M) elements
67108864 elements generated, sum 33554432, data size 268435456 (256MB)
compiler options: -Isrc/ -DREDUCTION_STYLE=0 -DVECSIZE=1 -DUNROLL=1 -DTYPE=int -DOP=ADD_OP -DOP_NULL=ADD_OP_NULL
loading src/reduction_kernels.ocl
read 5862 bytes from reduction_kVM start=0x110400000 end=0x110420000 | Buffer 131072 bytes
VM start=0x100000A00 end=0x100000C00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x002c0041 COMPUTE_PGM_RSRC2: 0x00000090
VM start=0x400000 end=0x440000 | Buffer 262144 bytes
VM start=0x100200400 end=0x100200600 | Buffer 512 bytes
VM start=0x100002000 end=0x100002400 | Buffer 1024 bytes
VM start=0x100002400 end=0x100002800 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
VM start=0x440000 end=0x480000 | Buffer 262144 bytes
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 67108864
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200200 end=0x100200400 | Buffer 512 bytes
VM start=0x100200000 end=0x100200200 | Buffer 512 bytes
VM start=0x100002800 end=0x100002C00 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
VM start=0x100200800 end=0x100200A00 | Buffer 512 bytes
VM start=0x100200A00 end=0x100200C00 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100002C00 end=0x100003000 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00008090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 67108864
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100003000 end=0x100003400 | Buffer 1024 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100201800 end=0x100201A00 | Buffer 512 bytes
COMPUTE_PGM_RSRC1: 0x00ac0081 COMPUTE_PGM_RSRC2: 0x00010090
input 0 : 4194304
input 1 : 1
input 2 : 0
input 3 : 67108864
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201A00 end=0x100201C00 | Buffer 512 bytes
VM start=0x100201C00 end=0x100201E00 | Buffer 512 bytes
input 0 : 8192
input 1 : 1
input 2 : 0
input 3 : 216
input 4 : 8192
input 5 : 1
input 6 : 1
input 7 : 0
input 8 : 0
input 9 : 0
VM start=0x100201E00 end=0x100202000 | Buffer 512 bytes
VM start=0x100201200 end=0x100201400 | Buffer 512 bytes
VM start=0x100201000 end=0x100201200 | Buffer 512 bytes
VM start=0x100200E00 end=0x100201000 | Buffer 512 bytes
VM start=0x100200C00 end=0x100200E00 | Buffer 512 bytes
VM start=0x100201400 end=0x100201600 | Buffer 512 bytes
VM start=0x100201600 end=0x100201800 | Buffer 512 bytes
VM start=0x100200600 end=0x100200800 | Buffer 512 bytes
ernels.ocl
kernel prefers multiples of 64 up to 256
memory upload runtime: 0.00272ms
Upload effective bandwidth: 98689.51 GB/s
Group size: 64, 256
Kernel pass #1 runtime: 23.7778ms
Kernel pass #2 runtime: 0.00672ms
Total runtime: 23.785ms
Bandwidth: 11.29 GB/s
Reduction performance: 2.821 GE/s
SUMMARY: 64 × 216 + 256 × 1 => 23.78ms | 11.29 GB/s | 2.82 GE/s
Group size: 128, 256
Kernel pass #1 runtime: 23.7803ms
Kernel pass #2 runtime: 0.00656ms
Total runtime: 23.7875ms
Bandwidth: 11.28 GB/s
Reduction performance: 2.821 GE/s
SUMMARY: 128 × 216 + 256 × 1 => 23.79ms | 11.28 GB/s | 2.82 GE/s
Group size: 256, 256
Kernel pass #1 runtime: 23.7331ms
Kernel pass #2 runtime: 0.004ms
Total runtime: 23.7378ms
Bandwidth: 11.31 GB/s
Reduction performance: 2.827 GE/s
SUMMARY: 256 × 216 + 256 × 1 => 23.74ms | 11.31 GB/s | 2.83 GE/s
memory download runtime: 0.07744ms
total download runtime: 0.07744ms for 3.8147e-06MB (4.81055e-05 GB/s)
Parallel sum: 33554432 vs 33554432 (expected: 33554432)
Deltas: 0 vs 0
(no new dmesg
messages)