CANN/runtime模型流切换示例

2_model_switch

【免费下载链接】runtime本项目提供CANN运行时组件和维测功能组件。项目地址: https://gitcode.com/cann/runtime

Description

This sample demonstrates how to use aclmdlRIBuildBegin interface to create a model instance, and implements Stream jumping and Stream activation in tasks.

Product Support

Key interfaces in this sample have the following support status on different products:

Build and Run

For environment installation details and running details, see README in the example directory.

Run steps:

# Replace ${install_root} with CANN installation root directory, default installation at /usr/local/Ascend source ${install_root}/cann/set_env.sh export ASCEND_INSTALL_PATH=${install_root}/cann # Replace ${ascend_name} with Ascend AI processor model, obtained by checking Name field using npu-smi info and removing spaces, for example ascend910b3 export SOC_VERSION=${ascend_name} # Some samples involve calling AscendC operators, need to configure AscendC compiler ascendc.cmake path, for example ${install_root}/cann/aarch64-linux/tikcpp/ascendc_kernel_cmake # Find ascendc_kernel_cmake under CANN package installation path, for example find ./ -name ascendc_kernel_cmake, and replace ${cmake_path} with ascendc_kernel_cmake directory path export ASCENDC_CMAKE_DIR=${cmake_path} # Build and run bash run.sh

CANN RUNTIME API

Key features and interfaces in this sample:

• Initialization
  • Call aclInit interface to initialize AscendCL configuration.
  • Call aclFinalize interface to deinitialize AscendCL.
• Device Management
  • Call aclrtSetDevice interface to specify Device for computation.
  • Call aclrtResetDeviceForce interface to forcibly reset current computation Device and reclaim Device resources.
• Context Management
  • Call aclrtCreateContext interface to create Context.
  • Call aclrtDestroyContext interface to destroy Context.
• Stream Management
  • Call aclrtCreateStream interface to create Stream.
  • Call aclrtSynchronizeStream interface to block waiting for Stream task completion.
  • Call aclrtDestroyStreamForce interface to forcibly destroy Stream, discarding all tasks.
  • Call aclrtCreateStreamWithConfig interface to create Stream with special config.
  • Call aclrtSwitchStream interface to jump between Streams based on condition.
  • Call aclrtActiveStream interface to activate Stream.
• Model Management
  • Call aclmdlRIBuildBegin interface to start building a model runtime instance.
  • Call aclmdlRIBindStream interface to bind model runtime instance to Stream.
  • Call aclmdlRIEndTask interface to mark task dispatch end.
  • Call aclmdlRIBuildEnd interface to finish building model runtime instance.
  • Call aclmdlRIUnbindStream interface to unbind model runtime instance from Stream.
  • Call aclmdlRIExecuteAsync interface to asynchronously execute model inference.
  • Call aclmdlRIDestroy interface to destroy model runtime instance.
• Memory Management
  • Call aclrtMalloc interface to allocate Device memory.
  • Call aclrtFree interface to release Device memory.
• Data Transfer
  • Call aclrtMemcpy interface to implement data transfer by memory copy.
  • Call aclrtMemcpyAsync interface to perform asynchronous memory copy.

Known Issues

None

【免费下载链接】runtime本项目提供CANN运行时组件和维测功能组件。项目地址: https://gitcode.com/cann/runtime