← Writing
AI Compiler

From Relay to Relax to Unity: The Historical Evolution of Apache TVM’s High-Level Compilation Stack

Publication Date: 1 May 2026
Author: Ailven LIU 刘翔
All Rights Reserved

1. Overview

The evolution from Relay to Relax and then to Unity in Apache TVM is not merely a sequence of syntactic replacements. It reflects a deeper shift in the compiler architecture of TVM: from a graph-level optimisation system centred on a high-level neural-network IR, towards a modular, cross-level machine learning compilation infrastructure in which graph programs, tensor programs, runtime objects, external libraries, and hardware-specific code generation can be represented and optimised together.

In concise terms:

NNVM Graph
  |
  V
Relay
  |
  V
Relax + TensorIR + TVMScript + MetaSchedule + Runtime / Backend Integration
  |
  V
TVM Unity Modular Compilation Flow

Relay was TVM’s first mature high-level graph IR. Relax is the next-generation high-level IR designed for dynamic shapes, modern model structures, LLM workloads, Python-first workflows, and cross-level optimisation with TensorIR. Unity is not a single IR, but a broader architectural direction that aims to unify Relax, TensorIR, TVMScript, MetaSchedule, the runtime, libraries, and target backends into a coherent and composable compilation system.

Therefore, the essential transformation can be summarised as follows:

Relay:
    high-level graph IR for neural-network compilation

Relax:
    symbolic-shape-aware dataflow/program IR integrated with TensorIR

Unity:
    modular cross-layer ML compilation architecture

The transition from Relay to Relax should not be interpreted as a superficial renaming. It represents a change in the underlying compilation philosophy: from a largely staged pipeline of high-level graph optimisation followed by low-level tensor-program lowering, towards an IRModule-centred system in which high-level graph programs and low-level tensor programs coexist, interact, and can be transformed together.


2. Before Relay: The NNVM Graph Era

Before Relay became the central graph-level IR in TVM, early TVM workflows relied on NNVM as the high-level graph representation. In that period, the overall compilation pipeline could be roughly described as:

Framework Model
  |
  V
NNVM Graph
  |
  V
Operator Lowering
  |
  V
TVM Tensor Expression / Schedule
  |
  V
Code Generation

This approach was effective for many early deep learning workloads. Models such as CNNs, MLPs, and relatively regular RNN variants could often be represented as static operator DAGs:

Conv2D -> BatchNorm -> ReLU -> Pooling -> Dense

However, NNVM-style graph IRs had several structural limitations.

First, their expressive power was limited. A conventional computation graph is well suited to representing a DAG of tensor operators, but it is not a natural representation for control flow, subgraphs, higher-order functions, recursion, or dynamic computation.

Second, the boundary between compile-time representation and runtime representation was insufficiently clean. The graph was both an optimisation object and a structure close to runtime execution, which restricted the compiler’s ability to perform deeper program transformations.

Third, the interaction between high-level graph optimisation and low-level tensor-program optimisation was weak. A graph-level pass could identify that a subgraph contained a convolution, a matrix multiplication, or an elementwise operation, but it had limited access to the details that matter at the tensor-program level: tiling, vectorisation, memory layout, shared-memory usage, tensorisation, cache locality, and target-specific scheduling.

These limitations motivated the design of Relay.


3. Relay: TVM’s First Mature High-Level IR

3.1 Historical Role of Relay

Relay was introduced around 2018 as the successor to NNVM. It was designed as a new high-level IR for machine learning frameworks and was often described as NNVM v2.0. Its purpose was to provide a more expressive and principled representation for machine learning programs than a traditional computation graph.

Relay can be characterised as:

statically typed
purely functional
graph-level / high-level
machine-learning-oriented
intermediate representation

Unlike a simple DAG representation, Relay introduced concepts from programming language design: functions, variables, let-bindings, function calls, operator calls, tuples, conditional expressions, and a type system. This made Relay significantly more expressive than NNVM-style graphs.

Relay was therefore not merely a graph container. It was a typed functional IR designed to represent machine learning models as programs.


3.2 What Relay Solved

Relay solved several important problems in the earlier TVM stack.

First, it introduced a stronger type system. Relay could represent tensor types, tuple types, function types, and other structured types, enabling more robust static checking, type inference, and program transformation.

Second, Relay provided functional abstraction. Models no longer had to be represented only as flat operator DAGs. They could include function definitions, let-bindings, control-flow constructs, and nested computation structures.

Third, Relay allowed graph optimisation to be expressed as program transformation. Passes such as constant folding, operator fusion, layout rewriting, dead-code elimination, common subexpression elimination, and quantisation rewriting could be implemented as transformations over Relay programs.

Fourth, Relay became the main graph-level IR in TVM for a substantial period. It served as the central representation for frontend import, high-level graph optimisation, operator fusion, and the initial stages of lowering towards tensor-level computation.

A typical Relay-based compilation flow looked like this:

PyTorch / TensorFlow / ONNX / TFLite / MXNet
      |
      V   Frontend Import
Relay IRModule
      |
      V   Relay Graph-Level Optimisation
Optimised Relay
      |
      V   Operator Lowering / Fusion
TE / TIR
      |
      V   Scheduling / Auto-Tuning / Code Generation
Runtime Module

From an engineering perspective, Relay played four major roles:

model-structure representation
graph-level optimisation
unified frontend import
operator fusion and lowering decision layer

This made Relay one of the most important components in the maturation of TVM as a deep learning compiler.


3.3 Limitations of Relay

Relay was not a failed design. On the contrary, it was a key stage in the evolution of TVM. However, as model architectures, hardware targets, and deployment requirements changed, Relay began to expose structural limitations.

3.3.1 Limited Support for Dynamic Shape Workloads

Relay was designed during a period when many important workloads were still dominated by relatively static shapes:

fixed batch size
fixed image resolution
often fixed sequence length
regular operator structure

Modern workloads are different. Transformers, LLMs, VLMs, diffusion models, dynamic batching, variable sequence lengths, KV cache management, ragged tensors, and prefill/decode separation all require stronger support for symbolic and runtime-dependent shapes.

For example, an LLM inference workload naturally contains two different execution regimes:

Prefill:
    input shape = [batch, prompt_length]

Decode:
    input shape = [batch, 1]
    KV cache shape = [num_layers, batch, num_heads, sequence_length, head_dim]
    sequence_length grows token by token

Relay could support certain dynamic-shape cases, but it was not designed around first-class symbolic shape as a central organising principle. As dynamic workloads became increasingly important, this became a fundamental limitation.

3.3.2 A Hard Boundary Between Graph IR and TensorIR

In the Relay-era compilation stack, the relationship between the high-level graph and low-level tensor program was usually staged:

Relay Graph Passes
  |
  V
Lowering to TE / TIR
  |
  V
TIR Scheduling / Code Generation

This structure created a relatively hard boundary between high-level and low-level optimisation.

A graph-level pass could decide whether to fuse operators, rewrite layouts, or transform subgraphs, but it often lacked precise information about low-level scheduling consequences. For example, whether a fusion decision is beneficial depends not only on graph topology, but also on:

register pressure
shared-memory usage
tensor-core-friendly layout
memory coalescing
occupancy
vectorisation feasibility
target-specific code-generation behaviour

A purely graph-level view is insufficient for such decisions. Conversely, low-level scheduling may reveal constraints or opportunities that should influence graph-level transformation. Relay’s architecture made this kind of bidirectional feedback difficult.

3.3.3 Limited Modularity of the Pass Pipeline

Relay supported custom passes, external code generation, BYOC, custom operators, and multiple frontend paths. However, many practical workflows still felt like a relatively fixed pipeline:

frontend import
  |
  V
relay.transform.Sequential([...])
  |
  V
relay.build(...)

This was acceptable for conventional model deployment, but less suitable for compiler engineers, hardware vendors, and AI infrastructure developers who needed to insert custom transformations, replace lowering paths, integrate external libraries, mix hand-written kernels with generated kernels, and inspect intermediate IR in a disciplined way.

As TensorIR, TVMScript, and MetaSchedule matured, TVM required a more modular, inspectable, and programmable compilation flow.

3.3.4 Insufficient Natural Fit for LLM Workloads

Relay was not originally designed around modern LLM inference. LLM workloads impose a set of requirements that are difficult to express elegantly in a traditional graph-centric compiler flow:

symbolic sequence length
dynamic KV cache
paged attention
prefill/decode separation
shape-dependent memory planning
external kernel integration
quantisation-aware graph rewriting
tensor parallelism
pipeline parallelism
runtime shape dispatch

These are not marginal extensions to ordinary graph execution. They require a representation that can naturally express graph structure, symbolic shapes, runtime objects, external function calls, TIR kernels, device placement, and memory/runtime behaviour within a single composable system.

This requirement led to Relax.


4. TVM Unity: The Architectural Vision Behind Relax

4.1 Unity Is Not a Single IR

TVM Unity should not be understood as a replacement IR. It is a broader architectural direction for TVM.

The central goal of Unity is to break the rigid boundaries between several layers of the machine learning system stack:

computational graph
tensor programme
library / runtime
hardware primitive

Historically, machine learning systems often separated these concerns into relatively isolated layers. Graph compilers optimised graph-level structure; tensor compilers generated kernels; vendor libraries supplied high-performance primitives; runtimes handled execution and memory management. These divisions simplified system design, but they also constrained cross-layer optimisation.

Unity’s central judgement is that future ML compilation cannot be a one-way lowering pipeline from graph to kernel. It must allow graph IR, tensor programs, external libraries, runtime systems, and hardware primitives to interact.

Thus, Unity should be understood as:

not one IR to rule them all
but interoperable IRs in one modular system

In practical terms, Unity consists of several mutually connected components:

Relax:
    graph-level and programme-level abstraction

TensorIR:
    tensor-program abstraction

TVMScript:
    inspectable and writable IR surface

MetaSchedule:
    automatic search-based scheduling

DLight and rule-based scheduling:
    practical default scheduling automation

Runtime / VM / external codegen:
    execution and backend substrate

Unity does not attempt to collapse all levels of abstraction into a single monolithic representation. Instead, it attempts to make multiple representations interoperable within one coherent compilation framework.


5. Relax: Relay Next

5.1 Historical Position of Relax

Relax emerged as ‘Relay Next’ under the TVM Unity vision. Its purpose was not to discard Relay’s achievements, but to evolve the high-level IR in response to new model and hardware requirements.

Relax inherits Relay’s role as the graph-level IR, but redesigns the abstraction around several new requirements:

dynamic shape
symbolic shape
TensorIR integration
cross-level optimisation
runtime object representation
external kernel integration
modular transformation
Python-first inspectability

The result is a high-level IR that is better suited to contemporary machine learning workloads, especially Transformer- and LLM-based systems.


5.2 Core Objectives of Relax

The essential objectives of Relax can be summarised as follows:

G0: bridge graph IR and TensorIR
G1: provide first-class symbolic shape support
G2: represent dynamic workloads naturally
G3: improve customisation and composability
G4: integrate cleanly with TVMScript and IRModule
G5: support emerging workloads such as Transformers, LLMs, and dynamic models

Relax was therefore designed not merely as a cleaner graph IR, but as the high-level program representation required by the Unity architecture.


5.3 Basic Design of Relax

The key design idea of Relax is that high-level graph programs and low-level tensor programs can coexist inside the same IRModule.

A simplified Relax + TensorIR module may look conceptually like this:

@tvm.script.ir_module
class Module:
    @T.prim_func
    def matmul(...):
        ...

    @R.function
    def main(x, w):
        y = R.call_tir(matmul, (x, w), ...)
        return y

This example illustrates the essential relationship:

Relax functions:
    high-level dataflow, graph structure, runtime-level composition

TIR PrimFuncs:
    low-level tensor programmes and kernels

IRModule:
    shared container that holds both Relax and TIR

R.call_tir:
    explicit bridge from Relax-level programme to TIR-level kernel

This is a major change from the older model in which a Relay graph was lowered into tensor-level code through a relatively staged pipeline. In Relax/Unity, the IRModule becomes the central object of transformation.


6. Core Differences Between Relay and Relax

6.1 IR Style: Functional Graph IR versus Dataflow/Binding-Oriented Program IR

Relay is strongly functional in style. It represents programs using expressions such as:

let-binding
function
call
tuple
if-expression
recursion

Relax is more explicitly structured around dataflow blocks, bindings, symbolic shape information, StructInfo, and explicit boundaries between pure dataflow and effectful/runtime operations.

A simplified comparison is:

Relay:
    machine learning programme as typed functional expression

Relax:
    machine learning programme as structured dataflow with bindings,
    symbolic shape, TIR calls, and runtime integration

Relax is closer to a modern SSA-like dataflow program representation while still preserving the high-level graph structure needed for model optimisation.


6.2 Type System: Relay Type versus Relax StructInfo

Relay uses a type system to describe expressions. Examples include tensor types, tuple types, and function types.

Relax introduces the concept of StructInfo. StructInfo describes the structural information of expressions, including:

TensorStructInfo
ShapeStructInfo
TupleStructInfo
FuncStructInfo
ObjectStructInfo

StructInfo is not merely a conventional type system. It is better suited to representing partially known and partially dynamic information:

dtype
number of dimensions
symbolic shape
known and unknown dimensions
runtime object structure
shape expressions

This is crucial for dynamic-shape workloads. For example, in LLM inference:

input_ids:
    Tensor[(batch, sequence_length), int32]

kv_cache:
    runtime object / tuple / opaque cache structure

logits:
    Tensor[(batch, sequence_length, vocab_size), float16]

Here, sequence_length may be symbolic rather than compile-time constant. Relax’s StructInfo model is designed to represent this kind of partial static knowledge cleanly.


6.3 Shape: Auxiliary Type Information versus First-Class Symbolic Shape

In Relay, shape information is largely attached to tensor types. Dynamic-shape support exists, but symbolic shape is not the central organising principle of the IR.

Relax treats symbolic shape as a first-class design concern. This is essential for workloads such as:

Transformer variable sequence length
LLM prefill/decode execution
dynamic batching
ragged input
vision-language models with variable token counts
diffusion models with dynamic resolution
shape-polymorphic library dispatch

From an AI infrastructure perspective, Relax makes it natural to represent partially dynamic programs such as:

n = T.int64()
m = T.int64()

x: R.Tensor((n, m), "float16")

This is fundamentally different from forcing all shapes to be fixed at compile time.


6.4 High-Low Level Relationship: Lowering Boundary versus IRModule-Level Co-Design

In the Relay-era flow:

Relay graph
  |
  V
lowering to TE / TIR
  |
  V
TIR scheduling and code generation

In the Relax/Unity flow:

IRModule {
    Relax functions
    TIR PrimFuncs
    external functions
    runtime calls
}
      |
      V
IRModule-to-IRModule transformations
      |
      V
build

This is a major architectural change.

The compiler can now examine and transform high-level dataflow, low-level TIR kernels, external calls, device information, symbolic shape, and runtime calls within the same module-level representation.

This enables cross-level optimisation: high-level graph transformations can be informed by tensor-program details, and tensor-program generation can be shaped by high-level graph structure.


6.5 Pass Structure: Graph Passes versus Modular IRModule Transformations

Relay passes are often expressed as transformations over Relay functions or Relay expressions.

Relax places stronger emphasis on:

IRModule -> IRModule

This design fits naturally with TVMScript, structural equality, and testable compiler passes. A pass can be developed and tested by comparing:

before IRModule
  |
  V   pass
after IRModule

This improves engineering discipline. It becomes easier to inspect intermediate IR, write unit tests, build custom flows, and reason about transformation correctness.


6.6 Runtime: Graph Executor / Relay VM versus Relax VM and Modular Runtime Integration

Relay workflows commonly used the Graph Executor or Relay VM.

Relax naturally aligns with a more flexible runtime model, especially the Relax VM and runtime objects. This is important because modern ML inference is often not just static graph execution. LLM inference, for example, must manage:

KV cache
paged attention metadata
sampling
token-by-token generation loops
dynamic sequence length
batch scheduling
runtime shape
external fused kernels

Relax is therefore better positioned as an interface between compiler IR and runtime execution.


7. Unity Connection and the Integration of Relax

7.1 External Development Before Mainline Integration

Relax was initially developed partly outside the main TVM tree in order to reduce disruption to existing Relay users and production deployments. This explains why, for a period, users encountered references to:

apache/tvm main branch
apache/tvm unity branch
mlc-ai-related documentation
Relax-related forks and experimental branches

This separation was not accidental. It allowed the community to experiment with the new architecture while preserving the stability of existing Relay-based workflows.


7.2 Relax as the Future Graph IR

Over time, the community direction became clearer. Relax moved from an exploratory ‘Relay Next’ effort towards the intended graph IR for new TVM development. Meanwhile, legacy components such as Relay, TE schedule, AutoTVM, and AutoScheduler remained important for existing users and historical workflows, but the new modular direction increasingly centred on:

Relax as graph/program IR
TensorIR as tensor-program IR
MetaSchedule as the primary automatic scheduling infrastructure
TVMScript as the inspectable IR surface

This does not mean Relay disappears immediately. Relay still has historical importance, existing production users, mature frontend paths, and a large body of accumulated compiler logic. The more accurate interpretation is:

new development:
    favour Relax and the Unity modular flow

legacy production:
    Relay remains relevant where existing pipelines depend on it

long-term direction:
    move from Relay towards Relax as the principal graph IR

8. The Essential Transformation: Relay to Relax to Unity

8.1 From Graph IR to Program IR

Although Relay was more expressive than a traditional computation graph, it largely served as TVM’s graph-level IR.

Relax shifts the abstraction closer to a complete machine learning program IR. It can represent not only high-level dataflow, but also:

shape computation
runtime-dependent structure
tensor computation
runtime objects
external calls
device placement
memory planning boundaries
control and dataflow boundaries

This is increasingly necessary because modern model deployment is no longer just graph execution. LLM inference, for example, is graph computation plus runtime loops, cache management, memory scheduling, sampling, dynamic batching, and external kernel invocation.

Relax therefore represents a move from:

neural network graph IR

towards:

machine learning programme IR

8.2 From Staged Lowering to Cross-Level Co-Optimisation

The Relay-era mental model was often:

perform high-level graph optimisation first
then lower to low-level tensor computation
then schedule and generate kernels

The Unity-era mental model is:

represent high-level and low-level programmes together
perform transformations across abstraction boundaries
allow graph and tensor-level decisions to inform each other

This distinction is critical for performance.

Consider attention optimisation. A graph-level representation may see:

matmul -> softmax -> matmul

A Relax + TensorIR + external-kernel system can represent several alternatives:

call_tir(fused_attention_tir)
call_external("flashinfer.paged_attention")
rewrite pattern into a vendor-specific kernel
preserve symbolic sequence length and dispatch different kernel variants

This makes it significantly easier to integrate FlashAttention, FlashInfer, CUTLASS, cuBLASLt, TensorRT plugins, custom CUDA kernels, NPU backends, and other specialised execution paths.


8.3 From Static Model Compilation to Dynamic Workload Compilation

Early TVM workflows were often optimised for the following scenario:

known model
known input shape
known target
compile once
run many times

Modern AI infrastructure often faces a different scenario:

partially dynamic shape
dynamic batch size
runtime dispatch
hybrid compiler/library execution
model-specific kernels
target-specific graph rewriting
quantisation variants
multi-device execution

Relax’s symbolic shape, StructInfo system, runtime object support, call_tir, and external call mechanisms make it more suitable for these workloads.


8.4 From Operator-Centric Compilation to IRModule-Centric Compilation

Relay-era workflows often felt like:

import model
  |
  V
optimise graph
  |
  V
lower operators

Unity-era workflows are more naturally expressed as:

construct IRModule
inspect IRModule
transform IRModule
schedule TIR
replace subgraphs
call external libraries
tune kernels
build runtime module

The IRModule becomes the central object of the compilation process.

This is particularly valuable for AI infrastructure engineers. With TVMScript and Relax, one can inspect and modify the program at each stage:

print(mod.script())
apply transformation pass
print(mod.script())
insert custom PrimFunc
replace Relax call with call_tir
lower to target-specific code

This is far more controllable than a black-box frontend-to-build pipeline.


9. Significance for LLM and MLC-LLM

9.1 LLM Inference Is Not a Conventional Relay Workload

LLM inference contains structures that are awkward to express in a traditional static graph compiler:

prefill/decode separation
KV cache read/write
sequence length that grows dynamically
dynamic attention masks
sampling, top-k, and top-p
paged KV cache
tensor parallelism
pipeline parallelism
quantised weights
activation and weight layout transformations
external fused kernels

Several questions arise naturally:

Should the KV cache be represented as a tensor or a runtime object?
Should the decode loop be inside or outside the graph?
Is sequence length a compile-time constant or runtime value?
How should paged-attention metadata be represented?
How should different kernel variants be dispatched?

Relax provides a more suitable framework for these questions because it can represent graph structure, symbolic shape, TIR kernels, runtime objects, and external calls within the same system.


9.2 Value of Relax for MLC-LLM

MLC-LLM aims to compile LLMs across multiple backends, including CUDA, Metal, Vulkan, WebGPU, and CPU. Such a system requires:

dynamic-shape support
TIR-level kernel optimisation
runtime-level model execution
cross-platform lowering
quantisation rewriting
library and external kernel integration

These requirements align naturally with Relax and Unity.

A typical LLM compilation flow can be understood as:

Relax graph:
    model architecture, high-level dataflow, module composition

TIR PrimFunc:
    matmul kernel
    decode attention kernel
    RMSNorm kernel
    rotary embedding kernel
    sampling kernel

External call:
    FlashInfer
    cuBLAS / cuBLASLt
    vendor library
    runtime packed function

Relax makes it possible to represent these elements within one IRModule instead of scattering logic across frontend import, Relay passes, TE schedules, runtime glue code, and external backend hooks.


9.3 Relax and Model-Level Rewriting

Modern LLM deployment often relies on model-level rewriting. For example, a floating-point matrix multiplication may be rewritten into a quantised path.

At a high level, a graph expression such as:

matmul(x, w)

may become:

dequantize(w_q, scale, zero_point)
matmul(x, dequantized_w)

or, more efficiently:

call_tir(quantized_matmul_kernel, x, w_q, scale, ...)

or even:

call_external("cutlass_int4_gemm", ...)

Relax is better suited to these transformations because it can express graph-level patterns and then lower selected subgraphs either into TIR or into external libraries.


10. Significance for Hardware Vendors and AI Engine Engineering

Relax and Unity are also important for hardware vendors and AI Engine engineers.

A hardware backend often needs to express or control:

custom memory hierarchy
custom tensor core or matrix engine
custom DMA behaviour
custom layout
custom vector ISA
custom runtime integration
custom graph partitioning

Relay-era TVM supported many of these through BYOC, external codegen, and custom operators. However, the engineering experience could be heavy and fragmented.

Unity aims to make this integration more natural:

graph partition
  |
  V
external library or accelerator call
  |
  V
TIR scheduling or external code generation
  |
  V
runtime integration

Relax + TensorIR allows hardware vendors to participate simultaneously in:

high-level pattern rewriting
layout propagation
operator fusion
TIR scheduling
tensorisation
runtime calls
device placement

This is particularly relevant for heterogeneous execution across CPU, GPU, and specialised accelerators.


11. Common Misinterpretations

11.1 Relax Is Not Merely a New Name for Relay

Incorrect interpretation:

Relay was renamed Relax.

More accurate interpretation:

Relax inherits Relay's role as the high-level graph IR,
but redesigns the abstraction around symbolic shape,
TensorIR integration, modular transformation, and the Unity architecture.

11.2 Unity Is Not Identical to Relax

Incorrect interpretation:

Unity = Relax.

More accurate interpretation:

Unity:
    the next-generation modular compilation architecture of TVM

Relax:
    the high-level graph/programme IR within Unity

TensorIR:
    the tensor-program IR within Unity

TVMScript:
    the inspectable and writable representation layer

MetaSchedule / DLight:
    automatic and rule-based scheduling infrastructure

Runtime / VM / external codegen:
    execution and backend integration layer

11.3 Relay Does Not Disappear Immediately

Relay has a large amount of historical code, mature frontend infrastructure, many passes, and existing production users.

A realistic view is:

Relay:
    still important for historical understanding and existing deployments

Relax:
    preferred direction for new graph-level development

Unity:
    broader system direction for future TVM compilation workflows

12. Historical Timeline

2017-2018: From NNVM to Relay

TVM’s early graph-level representation was NNVM. Around 2018, Relay was introduced as a more expressive high-level IR, intended to solve the limitations of conventional graph representations by adding typed functional program structure.

2018-2021: Relay as the Core Graph IR

Relay became the main graph-level IR in TVM. It supported frontend import, graph optimisation, operator fusion, layout rewriting, quantisation workflows, and BYOC-style integration.

2021: The TVM Unity Vision

The Unity vision was proposed as a new direction for TVM. Its goal was to break boundaries between computational graphs, tensor programs, libraries, runtimes, and hardware primitives, enabling deeper cross-layer optimisation.

2022: Relax Appears as Relay Next

Relax emerged as the high-level IR designed for Unity. It aimed to evolve Relay around symbolic shape, dynamic workload representation, TensorIR integration, modular transformation, and improved portability.

2022: Unity Connection

The Unity connection work aimed to bring Relax into the broader TVM ecosystem as a key component of the new end-to-end modular flow, together with TensorIR, MetaSchedule, and TVMScript.

2023: Relax Remains Actively Developed

Relax continued to develop actively in the Unity-oriented workflow. The community direction became increasingly clear: Relax was no longer merely an experiment, but the intended successor direction for high-level graph compilation.

2024 and After: Transition from Legacy Components to the New Modular Flow

The TVM community increasingly encouraged new development around Relax, TensorIR, MetaSchedule, TVMScript, and modular IRModule-level transformation. Legacy components such as Relay, TE schedule, AutoTVM, and AutoScheduler remained relevant for existing workflows, but the long-term direction moved towards the Unity stack.


13. How a Relay Engineer Should Understand Relax

For engineers familiar with Relay, the following conceptual mapping is useful:

Relay Concept Relax / Unity Interpretation
Relay Function Relax Function
Relay Expr Relax Expr / Binding
Relay Type StructInfo
Relay graph pass Relax / IRModule transformation
Relay operator fusion Relax pattern rewriting plus call_tir or external call
Relay build Relax / Unity build flow
TE lowering TensorIR PrimFunc and call_tir
Relay VM Relax VM
BYOC Relax external call / codegen integration / modular backend
Static-shape graph Symbolic-shape-aware program
Graph-only optimisation Cross-level graph + TIR optimisation

However, this table only captures surface-level correspondence. The deeper shift is that Relax encourages engineers to treat model compilation as a sequence of transformations over an IRModule, rather than as a fixed frontend-import-to-build pipeline.


14. A Practical Mental Model of the Relax/Unity Flow

A typical Relax/Unity compilation flow can be understood as:

1. Import or construct the model
   PyTorch / ONNX / TensorFlow / NNModule / manual TVMScript
      |
      V

2. Produce a Relax IRModule
   Relax functions + symbolic shape + high-level operations
      |
      V

3. Apply graph-level transformations
   legalisation, fusion, pattern rewriting, quantisation, layout rewriting
      |
      V

4. Lower selected parts to TensorIR
   call_tir, PrimFunc generation, external calls
      |
      V

5. Schedule TensorIR
   manual schedule, DLight, MetaSchedule, tensorisation
      |
      V

6. Integrate with runtime
   Relax VM, external libraries, packed functions, device placement
      |
      V

7. Build deployable module
   target-specific code generation and runtime module

Compared with the Relay-era pipeline, the key difference is that steps 3, 4, and 5 are no longer separated by rigid one-way boundaries. They can interact through IRModule-level transformations.


15. Why This Matters for AI Infrastructure Engineers

From the perspective of AI infrastructure, the transition from Relay to Relax and Unity means that TVM is evolving from a neural-network compiler into a programmable ML systems compilation infrastructure.

For embedded deployment, autonomous driving, LLM inference, NPU/GPU backends, model compression, runtime optimisation, and custom kernel integration, Relax/Unity offers several practical advantages.

First, model structure becomes more controllable. Engineers can perform structured rewrites at the Relax level rather than relying on runtime hacks or low-level kernel replacement alone.

Second, dynamic shape becomes a first-class concern. This is essential for Transformers, BEV models, VLA models, multimodal models, and LLM workloads.

Third, integration with TIR becomes more direct. Hand-written TIR, generated TIR, scheduled TIR, and Relax graph programs can coexist in the same IRModule.

Fourth, external high-performance kernels become easier to integrate. Examples include FlashInfer, CUTLASS, cuBLASLt, TensorRT plugins, vendor libraries, and custom CUDA kernels.

Fifth, end-to-end performance experiments become more systematic. One can compare different implementation paths:

Relax operator path
call_tir path
external library path
manual TIR schedule path
MetaSchedule-tuned path

Sixth, compiler engineering becomes more maintainable. TVMScript, structural equality, and IRModule-level transformation provide a more inspectable and testable workflow than opaque build pipelines.


16. Conclusion

Relay was the decisive step that transformed TVM from an early tensor compiler with graph support into a mature deep learning compiler. It introduced a typed functional high-level IR and enabled robust graph-level optimisation.

Relax is the successor direction of Relay under the Unity vision. It does not reject Relay’s achievements; it reconstructs the high-level abstraction to address the requirements of modern workloads: symbolic shape, dynamic execution, LLM inference, runtime objects, external kernels, and cross-level optimisation with TensorIR.

Unity is the broader architectural framework. It attempts to integrate Relax, TensorIR, TVMScript, MetaSchedule, runtime systems, libraries, external code generation, and hardware-specific primitives into a modular and composable compilation system.

The historical evolution can therefore be summarised as:

NNVM:
    computation graph as model representation

Relay:
    typed functional IR as graph-level compiler IR

Relax:
    symbolic-shape-aware dataflow/programme IR integrated with TensorIR

Unity:
    modular cross-layer ML compilation system

For current and future work in TVM, MLC-LLM, AI Engine engineering, and high-performance model deployment, the most important direction is not merely to understand Relay passes or traditional TE schedules. Those remain historically and technically valuable. However, the main trajectory is now Relax + TensorIR + TVMScript + MetaSchedule + runtime integration.

Relay explains how TVM became a mature graph compiler. Relax and Unity explain where TVM is going.