950基础矩阵乘法TLA示例

950 Basic Matmul TLA Example Readme

【免费下载链接】catlass本项目是CANN的算子模板库，提供NPU上高性能矩阵乘及其相关融合类算子模板样例。项目地址: https://gitcode.com/cann/catlass

Note: The community package does not currently support 950 capabilities. Stay tuned for a future supported version.

Code Organization

├── 43_ascend950_basic_matmul │ ├── CMakeLists.txt # CMake build file │ ├── README.md │ └── basic_matmul_tla.cpp # Main file

Usage Example

• After obtaining the code, build the corresponding operator executable. See Template Library Quick Start. This case is a 950 operator, and-DCATLASS_ARCH=3510must be added during build.
• Run the operator.

# Build the specified case bash scripts/build.sh 43_ascend950_basic_matmul -DCATLASS_ARCH=3510 cd output/bin # Executable file name | matrix m axis | n axis | k axis | Device ID # Device ID is optional and defaults to 0 ./43_ascend950_basic_matmul 256 512 1024 0

The execution result is as follows, indicating that the precision comparison succeeds.

Compare success.

Usage Notes

The DispatchPolicy MmadPingpong used by BasicMatmul by default supports the following template parameters:

Assume the matrix Shape isM N K, the tile size on L1 ism1 n1 k1, the number of tiles in the M direction ismTiles = CeilDiv(M, m1), the number of tiles in the N direction isnTiles = CeilDiv(N, n1), and the total number of tasks istaskBlocks = mTiles * nTiles. enableL1Resident can be enabled in the following two cases:

1. mTiles = 1,nTiles > CoreNum, andK < 2 * k1. In this case,l0CStages=2can also be set (enableUnitFlag must be disabled). If there is not enough space andl0CStages=2cannot be set, setn1to half of the original value.

2. nTiles = 1,mTiles > CoreNum, andK < 2 * k1. In this case,l0CStages=2can also be set (enableUnitFlag must be disabled). If there is not enough space andl0CStages=2cannot be set, setm1to half of the original value.

BasicMatmul also supports DispatchPolicy MmadPreloadAsyncWithCallback, which supports the following template parameters:

Compared withMmadPingpong,MmadPreloadAsyncWithCallbackhas two more template parameters. One ispreloadStages. This parameter is usually set to 1 and specifies the number of preloads. When this parameter is set to 1, the first loop only loads data and does not perform matmul computation. The second loop first loads the data for the second loop, and then completes the Matmul computation of the previous loop, and so on. After the final loop ends, one additional Matmul computation is performed. The benefit is that the data required for the current Matmul computation has already been moved in the previous loop. Therefore, instruction issue is advanced, which reduces the performance loss caused by instruction issue latency.

The second parameter isenableShuffleK. This parameter is mainly used to avoid bandwidth loss caused by same-address access conflicts. The main principle is to stagger the data read addresses of each core. This parameter does not need to be enabled on 950.

Compared withMmadPingpong,MmadPreloadAsyncWithCallbackhas more optimization points, but its logic is also more complex and has higher Scalar overhead. Use it based on the scenario, especially for small Shape scenarios.

【免费下载链接】catlass本项目是CANN的算子模板库，提供NPU上高性能矩阵乘及其相关融合类算子模板样例。项目地址: https://gitcode.com/cann/catlass