企业官网建设流程全解析

Ascend C Custom Operator Runtime Compilation and Offline Model Sinking Sample

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge

Sample Overview

• Graph construction entry:GE graph construction + ATC offline compilation
• Operator programming language:Ascend C
• Compilation method:RTC operator runtime compilation
• Model sinking capability:Supports sinking to om offline model
• Core pipeline:Ascend C kernel source code RTC operator runtime compilation -> GE deliverables -> AIR -> ATC offline compilation -> om offline model -> ACL execution
• Difference from other samples: This sample completes Ascend C kernel runtime compilation inCompilecallback and serializes compilation products into om offline model, which can be deserialized and used in subsequent model execution phases.
• Multi-shape note: This sample mainly provides a multi-shape/multi-kernel processing framework that manages and sinks multiple kernel binaries by shape key.
• Compile-time input note: The sample reads input byTensorlayout inCompilephase, so it can simultaneously obtain metadata such asShape,DataType,Format; current key is still only generated by shape.

This sample demonstrates a minimum runnable GE custom operator sinking pipeline: first construct graph to generateAIR, then generateom offline modelthroughATC offline compilation, finally load and execute byACLprogram. The AddCustom in the sample simultaneously inheritsCompilableOp,PortableOpandShapeInferOp, so it can not only complete Ascend C kernel runtime compilation through RTC inCompilecallback, but also serialize compilation results to final model, and complete Shape and DataType inference in the same implementation class.

Applicable Scenarios

• Want to see Ascend C custom operator sample inGE graph construction + ATC offline compilation + om offline modelpipeline.
• Want to reference complete flow of custom operator RTC runtime compilation, model sinking and final model execution.
• Want to see minimum runnable pipeline ofAIR -> ATC offline compilation -> om offline model -> ACL.
• Not suitable for understandingPyTorch/TorchAirorTensorFlow/Tritonscenarios.

Prerequisites

CANN

• CANN environment has been correctly installed and configured, e.g., executedsource ${ASCEND_HOME_PATH}/set_env.sh.
• atccommand is available.
• Current environment hasACL,GE,Graphrelated headers and libraries.
• Follow installation guide to complete toolkit and ops package installation.

Framework and Plugins

• This sample does not depend on PyTorch or TensorFlow.

Environment Variables

• ASCEND_HOME_PATH
• ASCEND_CUSTOM_OPP_PATHwill be automatically appended withoutput/inrun.sh, and custom operator deliverables will be placed inoutput/op_graph/lib/linux/x86_64/oroutput/op_graph/lib/linux/aarch64/

Additional Dependencies

• cmake
• g++

Quick Run

Execute inexamples/custom_op/compilable_add_customdirectory:

Recommended Method

bash run.sh

run.shwill automatically appendoutput/toASCEND_CUSTOM_OPP_PATHafter generatingoutput/op_graph/lib/<os>/<arch>/libcust_opapi.so. If successful,output/single_add.air,output/single_add.omwill be generated, and terminal will printModel executed successfully!andFirst element of output: 2.000000.

Step-by-step Method

If you want to manually observe artifacts at each stage, you can execute:

cmake -S . -B build -DCMAKE_BUILD_TYPE=Release cmake --build build -j"$(nproc)" cmake --install build export ASCEND_CUSTOM_OPP_PATH="$(pwd)/output:$ASCEND_CUSTOM_OPP_PATH" cd output ../build/compilable_add_graph_build cd .. atc \ --model="$(pwd)/output/single_add.air" \ --framework=1 \ --output="$(pwd)/output/single_add" \ --soc_version=Ascend910B1 cd build ./compilable_add_model_exec ../output/single_add.om cd ..

Whereexport ASCEND_CUSTOM_OPP_PATH="$(pwd)/output:$ASCEND_CUSTOM_OPP_PATH"is used to add custom operator package root directory to environment variable, then GE/ATC will load deliverables according tooutput/op_graph/lib/<os>/<arch>/libcust_opapi.sorule.

Directory Structure and Key Files

compilable_add_custom ├── CMakeLists.txt ├── README.md ├── run.sh ├── ge │ ├── add_custom_kernel.cpp // Ascend C kernel source code read during operator runtime compilation │ ├── add_custom.h // AddCustom unique proto definition, shared by deliverables and graph construction side │ ├── custom_op.cpp // CompilableOp/PortableOp/ShapeInferOp main process implementation │ └── utils │ ├── compile_utils.cpp // Compile-time input Tensor reading, platform info, kernel path and shape key helper logic │ └── kernel_binary_map_utils.cpp // Multi-bin serialization/deserialization logic ├── graph_build │ └── main.cc // Construct graph and export single_add.air └── model_exec └── main.cc // Load om offline model and execute

Key files:

• ge/custom_op.cppCustom operator core main process, responsible for chaining operator runtime compilation, serialization, deserialization, Shape/DataType inference and execution; compile-time readsShape/DataType/Formatmetadata by input Tensor.
• ge/add_custom.hAddCustomunique proto definition; graph construction side uses the same name header file generated tooutput/op_graph/include/.
• ge/add_custom_kernel.cppAscend C kernel source code read during operator runtime compilation phase.
• ge/utils/compile_utils.cppEncapsulates compile-time input Tensor reading, platform info reading, kernel source path locating, and binary key generation logic based on shape, used to demonstrate binary management method in multi-shape scenarios.
• ge/utils/kernel_binary_map_utils.cppResponsible for serialization and deserialization of multiple kernel binaries, highlighting multi-key/multi-binary persistence and recovery.
• graph_build/main.ccBuild minimum Add graph and exportsingle_add.air.
• model_exec/main.ccLoad om offline model, prepare input/output and execute model.
• run.shChain configure, build, install, ATC offline compilation and model execution.

Core Pipeline

1. ge/custom_op.cppfirst reads input TensorShape/DataType/Formatmetadata inCompilecallback, then readsge/add_custom_kernel.cpp, completes Add kernel operator runtime compilation viaaclrtc.
2. graph_build/main.ccconstructs minimum Add graph and exportssingle_add.air.
3. atcoffline compilessingle_add.airtosingle_add.om.
4. model_exec/main.ccloadssingle_add.omand executes via ACL, outputs first result for verification.

Note: Sample compile-time input is already organized by Tensor method, so shape, data type, format can be directly obtained; current still only uses input shape to generate key, and based on this caches, serializes and deserializes kernel binary, overall providing multi-shape/multi-kernel processing framework.

Build Artifacts

• output/op_graph/lib/linux/x86_64/libcust_opapi.soCustom operator deliverables used by GE/ATC offline compilation in Linux x86_64 environment; aarch64 environment corresponds tooutput/op_graph/lib/linux/aarch64/libcust_opapi.so.
• output/op_graph/include/add_custom.hAddCustom proto header file that can be directly used by graph construction side, generated by copying fromge/add_custom.hduring build phase.
• output/op_graph/lib/linux/x86_64/add_custom_kernel.cppKernel source file output together with deliverables, read during operator runtime compilation phase; aarch64 environment corresponds tooutput/op_graph/lib/linux/aarch64/add_custom_kernel.cpp.
• output/single_add.airAIR file exported by graph construction program, as ATC offline compilation input.
• output/single_add.om.omoffline model file generated by ATC offline compilation.
• build/compilable_add_graph_buildgraph_build program.
• build/compilable_add_model_execom offline model execution program.

Result Verification

When successful, you can observe:

• output/single_add.airhas been generated.
• output/single_add.omhas been generated.
• Terminal output containsModel executed successfully!.
• Terminal output containsFirst element of output: 2.000000.

If failed, priority checks:

• WhetherASCEND_HOME_PATHhas been set and CANN environment has been correctlysourced.
• Whetheratcis available.
• Whetheroutput/op_graph/lib/<os>/<arch>/libcust_opapi.so,output/op_graph/lib/<os>/<arch>/add_custom_kernel.cppandoutput/op_graph/include/add_custom.hhave been generated.
• Whether currentsoc_versionmatches actual environment,run.shdefaults toAscend910B1.

Notes / Limitations

• run.shcurrently fixessoc_version=Ascend910B1, if you need to adapt to other products please adjust according to actual environment.
• build/compilable_add_model_execneeds to pass model path as positional parameter.
• run.shwill automatically appendASCEND_CUSTOM_OPP_PATHasoutput/, so thatATC offline compilationloadslibcust_opapi.soaccording toop_graph/lib/<os>/<arch>/directory specification.
• Current sample organizes multiple kernel binaries by shape key, mainly providing multi-shape/multi-kernel processing framework.
• Current sample depends on installed CANN toolkit headers and directly reads compile-time input viaOpCompileContext::GetInputTensor.

Appendix

Reference Documentation

• RTC operator runtime compilation reference: Ascend C Operator JIT Compilation
• ATC offline compilation reference: ATC Tool Usage Guide

Operator Runtime Compilation and Sinking Mechanism

• CompilableOpis responsible for reading kernel source code inCompilecallback and callingaclrtcto complete operator runtime compilation.
• Compilephase input reading is already organized byTensor, so in addition to shape, data type and format can also be directly accessed.
• PortableOpis responsible for serializing compilation products to final model and deserializing for recovery in execution phase.
• ShapeInferOpis responsible for completing output shape and data type inference inInferShapeandInferDataTypecallbacks.
• After these parts cooperate, the om offline model generated by ATC offline compilation will contain device-side binaries needed for sample custom operator execution; if later extended to scenarios where key corresponds to different kernels, this sample already provides management and sinking method for multiple binaries.

Key Implementation Responsibilities

• Compile: Read source code and trigger operator runtime compilation.
• InferShape / InferDataType: Derive output description based on input shape and data type.
• Serialize / Deserialize: Write compilation products to model and recover in execution phase.
• Execute: Load device-side binary and initiate kernel call in model execution phase.

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge