CANN / hcomm 实验性NIC插件指南

HCOMM Experimental NIC Plugin Guide

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm

Overview

The HCOMM Experimental NIC Plugin extends the host-side NIC communication implementation in general server scenarios. The plugin is built and deployed as an independent.sofile. When creating a HOST endpoint or channel, HCOMM searches for registered plugins based onEndpointDesc.protocol. If a matching plugin protocol is found, the plugin implementation is used. Otherwise, the original built-in implementation is used.

This directory provides two example plugins:

Building and Packaging

Before compiling, complete the dependency installation, CANN software package installation, and environment variable configuration as described indocs/en/build/build.md.

After navigating to the repository root directory, execute the following command to compile the host package with experimental plugins enabled:

bash build.sh --pkg --experimental

To compile both the host and device packages:

bash build.sh --pkg --full --experimental

If the compilation environment does not have network access, download the third-party dependency packages in a networked environment, upload them to the compilation environment, and specify the dependency package path using--cann_3rd_lib_path:

bash build.sh --pkg --experimental --cann_3rd_lib_path={your_3rd_party_path}

After compilation, the HCOMM software package is generated in thebuild_outdirectory at the repository root:

./build_out/cann-hcomm_<version>_linux-<arch>.run

<version>is the software version number, and<arch>is the system architecture, for example,x86_64oraarch64.

Installation and Deployment

Install the compiled HCOMM software package:

bash ./build_out/cann-hcomm_<version>_linux-<arch>.run --full

Replace the package name in the command with the actual generated file name. After installation, the experimental plugins are installed in the HCOMM plugin directory:

${ASCEND_HOME_PATH}/hcomm_plugin/libhcomm_cpu_roce_plugin.so ${ASCEND_HOME_PATH}/hcomm_plugin/libhcomm_cpu_ub_plugin.so

For manual debugging, you can also copy the plugin.sofiles directly to the${ASCEND_HOME_PATH}/hcomm_plugin/directory.

Signature Verification Notes

Host-only plugin usage does not involve device package signature verification. If you compile and install the--fullpackage and run on-board tests, refer todocs/en/build/build.mdfor the steps to disable signature verification.

Enabling at Runtime

Before running, load the CANN or HCOMM environment variables. For the default installation path, execute:

source /usr/local/Ascend/cann/set_env.sh

For a specified installation path, execute:

source ${install_path}/cann/set_env.sh

The plugin loading rules are as follows:

• WhenASCEND_HOME_PATHis not empty, HCOMM scans${ASCEND_HOME_PATH}/hcomm_plugin/*.so.
• WhenASCEND_HOME_PATHis empty, HCOMM reads the plugin path specified byHCOMM_NIC_PLUGIN_SO.
• HCOMM_NIC_PLUGIN_SOsupports multiple.sopaths separated by colons.

Example:

export HCOMM_NIC_PLUGIN_SO=/path/to/libhcomm_cpu_roce_plugin.so:/path/to/libhcomm_cpu_ub_plugin.so

Business code does not need to call plugin interfaces directly. When HCOMM creates a HOST endpoint throughHcommEndpointCreate, it searches for plugins based onEndpointDesc.protocol. If the protocol is registered by a plugin, subsequent endpoint, channel, and data plane interfaces are dispatched to the pluginops. If no matching plugin is found, the original built-in path is used.

Usage Conditions and Limitations

• The plugin only applies to endpoints whereEndpointDesc.loc.locType == ENDPOINT_LOC_TYPE_HOST.
• The current example plugins target general server host-only scenarios.
• The current loader skips plugin loading when it detects that the number of runtime devices is non-zero.
• When multiple plugins register the same protocol, the later-loaded plugin overrides the earlier one.
• The plugin ABI must match the magic word, version, and size defined inHcommNicPluginInfo,HcommNicEndpointOps, andHcommNicChannelOps.

Custom Plugin Development Entry Points

Custom plugins need to export the following C ABI symbols:

const HcommNicPluginInfo *HcommNicPluginGetInfo(void); int32_t HcommNicPluginCreateEndpoint( const EndpointDesc *endpointDesc, void **outCtx, HcommNicEndpointOps **outOps); int32_t HcommNicPluginCreateChannel( void *epCtx, const HcommChannelDesc *channelDesc, void **outCtx, HcommNicChannelOps **outOps);

Development requirements:

• Declare the plugin name and supportedCommProtocolinHcommNicPluginInfo.
• FillHcommNicEndpointOpsfor the endpoint, implementing interfaces such as initialization, memory registration, export, import, and destroy.
• FillHcommNicChannelOpsfor the channel, implementing interfaces such as initialization, status query, read, write, notify, fence, and destroy.
• Ensure the magic word, version, and size in the ABI header match the definitions inhcomm_nic_plugin.h.

For reference, see the following files:

• host_roce_plugin.cc: HOST RoCE plugin entry point.
• host_ub_plugin.cc: HOST UB plugin entry point.
• nic_plugin_ops.h: Example plugin ops adaptation template.
• plugin_core.cc: Plugin common auxiliary logic.

Troubleshooting

If the plugin does not take effect, check the following items in order:

1. Confirm that--experimentalwas added during the build and that the plugin.sois included in the software package.
2. Confirm that the plugin.sois installed in${ASCEND_HOME_PATH}/hcomm_plugin/.
3. Confirm thatASCEND_HOME_PATHorHCOMM_NIC_PLUGIN_SOis set according to the current loading rules.
4. Confirm that the business creates a HOST endpoint, that is,EndpointDesc.loc.locType == ENDPOINT_LOC_TYPE_HOST.
5. Confirm thatEndpointDesc.protocolis registered in the pluginHcommNicPluginInfo.protocols.
6. Check the runtime log for the[NicPlugin]keyword to see whether the plugin was scanned, loaded, and its protocol registered.

【免费下载链接】hcommHCOMM（Huawei Communication）是HCCL的通信基础库，提供通信域以及通信资源的管理能力。项目地址: https://gitcode.com/cann/hcomm