Why Aidge?#
Aidge is a generic, multi-paradigms compute graph manipulation, quantization, mapping, scheduling and code generation tool. Its primary targets are embedded systems with specialized hardware accelerators, especially dataflow or restricted instruction set architectures. It is highly interoperable thanks to built-in ONNX import/export and direct PyTorch interface, and its modularity allows to use any of its features in standalone or in conjunction with other tools along the deployment path of a model to the embedded system.
Please check below some reasons that might encourage you to use Aidge:
Well-defined dataflow graph IR model#
In Aidge, the computing graph is always explicitly specified by the user, who knows exactly which operators are instantiated. The notions of graph (a set of nodes), mathematical operator and actual implementation are separated: a node contains an operator, which itself points to an implementation. Furthermore, the graph only defines a topology, without any assumption on execution priority.
Here are some interesting Aidge features:
A view can be created on any set or subset of nodes, which itself can constitute a (meta) node;
Graphs can be hierarchical (to any depth), thanks to meta operators that contain any graph view. It is trivial to (recursively) replace a set of nodes by a meta node or inversely, flatten a meta node to a set of nodes;
It is possible to specify any type of operator, even without a corresponding implementation, thanks to generic operators. No need to write a single line of code to handle custom operators at the graph level;
Nodes inputs and outputs are ordered, and any set of connected nodes can be ranked in a unique and deterministic way, making graph isomorphism identification trivial.
Cyclic graphs are supported. Unlike ONNX, any kind of graph can be fully flattened in Aidge and thus optimized globally.
Below is an example of a cyclic graph (an LSTM) in Aidge:
Powerful graph search & replace engine#
Aidge introduces a simple and efficient DSL for graph matching, sometimes called “graph regex”. It is possible to write complex textual queries to find a quantified or unquantified set of nodes with specific types, attributes and/or relationships between them. This is particularly useful to implement sophisticated pattern-matching heuristics with no effort!
Here is an example of a query that you can do in Aidge:
graph_regex = aidge_core.GraphRegex()
graph_regex.add_query("(Pad#?->Conv#|Deconv#->Pad#?)->ReLU#?->Add#1?->ReLU#?->MaxPool#?->ReLU#?")
graph_regex.add_query(".->Add#1?")
for match in graph_regex.match(model):
aidge_core.GraphView.replace(match, MyCustomIPOperator())
You can define your own node test function as well:
def myNodeTestingFunc(node):
...
return predicate
graph_regex = aidge_core.GraphRegex()
graph_regex.set_node_key("test", myNodeTestingFunc)
graph_regex.add_query("Conv->test")
...
See the tutorial Perform advanced graph matching with the Graph Regular Expression tool for more information.
Generic, compiler-agnostic tiling methods#
Operator tiling is an important operation in high-level compilers such as Halide or TVM, but usually implies an IR lowering step, such as C code generation, and a standard compiler backend, such as LLVM. In Aidge, tiling does not make any assumption on the programming paradigm. Thus, tiling is done at the same graph IR level, by just expanding the compute graph with tiled operators. It is up to the user to choose the right tiling granularity, depending on the type of operators he uses. The tiled operator implementation may be a C kernel, a call to a specific hardware accelerator or HLS-generated operator, or none of that, if the new tiled graph is just made to be exported back in ONNX, ready to be fed to another tool.
See the tutorial Optimize the inference of your neural network with Tiling for more information.
Well-defined consumer-producer model and scheduling#
Aidge introduces a well-defined consumer-producer (C-P) model for operator implementations, similar to transaction-level modeling (TLM) for electronic design. A generic, default implementation is provided as well. C-P model can be specified as precise amounts of data or arbitrary data quantity (token), for each operator and dynamically at each execution step. The C-P model execution path is decoupled from the data execution path, thus allowing to statically schedule the graph execution without providing the actual operator’s implementation.
For example, for a 2D convolution implementation that only processes one input line at each execution step, it is trivial to build a pipelined dataflow and get the right execution order, by overloading two C-P methods in the implementation:
Elts_t MyCustomPipelinedConvImpl::getNbRequiredData(IOIndex_t inputIdx) {
// Consume a single input line at each execution
return Elts_t::DataElts(getInputLineSize(inputIdx));
}
Elts_t MyCustomPipelinedConvImpl::getRequiredMemory(IOIndex_t outputIdx) {
if (enoughDataToComputeOutputLine()) {
// Produce an output line only if there is enough input data
return Elts_t::DataElts(getOutputLineSize(outputIdx));
}
else {
return Elts_t::DataElts(0);
}
}
Thanks to Aidge’s C-P model, arbitrary complex cyclic and acyclic dataflow graphs can be statically scheduled. Generic sequential and parallel schedulers are available, and custom schedulers can be built using static scheduling data (logical early and late execution steps and associated dependencies for each scheduled node).
Generic hardware model and mapping heuristics#
🚧 This is planned for the next major Aidge release, stay tuned!
Simple and effective code generation engine#
Aidge uses the Jinja template engine to easily generate any type of code/programming model from a graph. It provides facilities to easily define what should be generated for each operator type. Beyond that, optimized static scheduling and memory mapping can be generated as well. Eventually, the full compute graph can be generated in an entirely static configuration, with minimal to no control overhead whatsoever during the execution of the dataflow on the intended hardware target, even in multi-threaded environments.
See the tutorial Add a custom implementation for a cpp export for more information.
Seamless interoperability with ONNX, PyTorch and TensorFlow#
Aidge has native built-in ONNX import and export capabilities, even for custom ONNX operators.
🚧 As for now, Aidge only implements a restricted set of ONNX operators (unsupported operators are loaded as generic operators), but the list is growing! Advanced PyTorch and Tensorflow interoperability are planned for the next major Aidge release.
Well-characterized, state-of-the-art PTQ and QAT methods#
🚧 This is planned for the next major Aidge release, stay tuned!