<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Ailven LIU — Writing</title>
  <link href="https://www.ailven.cc/" rel="alternate" type="text/html"/>
  <link href="https://www.ailven.cc/feed.xml" rel="self" type="application/atom+xml"/>
  <id>https://www.ailven.cc/</id>
  <updated>2026-11-12T00:00:00Z</updated>
  <author><name>Ailven LIU</name><uri>https://www.ailven.cc/</uri></author>
  <entry>
    <title>NaN Propagation and Constant Bloat in a Flash Attention Compilation Path</title>
    <link href="https://www.ailven.cc/posts/ai-compiler-260930.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/ai-compiler-260930.html</id>
    <published>2026-09-30T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A post-mortem of two independent failures in one compiled Flash Attention model – NaN from -inf minus -inf in the online softmax rescale, and a causal mask materialised twenty-four times – traced through the export, tuning, CSE, constant-pool and runtime stages, each of which made one copy.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 30 September 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;abstract&quot;&gt;Abstract&lt;/h2&gt;
&lt;p&gt;An end-to-end autonomous-driving planner – a 12-layer Transformer trunk whose attention subgraphs are compiled into fused Flash Attention kernels – failed in two independent ways once Flash Attention was switched on. It emitted NaN, and it shipped a constant data segment and a device memory footprint far larger than the model&amp;#39;s logical requirements. The same sources lowered through batch matmul plus softmax were numerically clean, which placed the numerical fault inside the Flash kernel rather than in the model.&lt;/p&gt;
&lt;p&gt;The two failures share a compilation path but not a root cause. The NaN comes from one subtraction in the online softmax rescale step, where &lt;code&gt;-inf&lt;/code&gt; minus &lt;code&gt;-inf&lt;/code&gt; is an invalid operation under IEEE 754 and yields NaN. The bloat comes from a causal mask that is logically a single 68.9 MB tensor and physically twenty-four device buffers, produced by a chain of stages each of which duplicated what the previous one handed it.&lt;/p&gt;
&lt;p&gt;The repair is three changes at three layers of the stack: a compile-time &lt;code&gt;-inf&lt;/code&gt; guard in the kernel, a content-level constant deduplication pass in the compiler, and an existing but disabled cross-function constant sharing switch in the VM compiler. Together they remove the NaNs, cut the constant data segment by 63% and the peak device memory by 39%, and leave inference latency unchanged.&lt;/p&gt;
&lt;h2 id=&quot;1-two-failures-one-compilation-path&quot;&gt;1. Two failures, one compilation path&lt;/h2&gt;
&lt;p&gt;The model is an end-to-end autonomous-driving planner. A 12-layer Transformer trunk consumes a sequence of historical feature frames together with the current token sequence, and produces trajectory deltas and classification logits. Its attention subgraphs are compiled into fused Flash Attention kernels.&lt;/p&gt;
&lt;p&gt;Switching Flash Attention on introduced two independent faults. The first is numerical: the model emits NaN, while the same weights and the same sources lowered through batch matmul plus softmax are numerically clean. Only the compilation configuration and the kernel path differ between the two builds, so the fault has to lie inside the Flash kernel&amp;#39;s numerical path. The second is volumetric: both the constant data segment of the compiled artefact and the device memory the runtime reserves are far larger than the model&amp;#39;s logical requirements, and nothing in the model explains the excess.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Numerical correctness&lt;/td&gt;
&lt;td&gt;NaN in the outputs&lt;/td&gt;
&lt;td&gt;zero NaN; 2,586 kernel outputs byte-identical to an fp32 reference&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Constant data segment&lt;/td&gt;
&lt;td&gt;1,357.1 MB&lt;/td&gt;
&lt;td&gt;526.65 MB, down 63%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Peak device memory&lt;/td&gt;
&lt;td&gt;2,039.68 MB&lt;/td&gt;
&lt;td&gt;1,247.7 MB, down 39%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Inference latency&lt;/td&gt;
&lt;td&gt;39.0 ms / 44.8 ms&lt;/td&gt;
&lt;td&gt;38.7 ms / 45.0 ms, unchanged&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The repair is three changes, each at a different layer of the stack: a numerical guard in the kernel&amp;#39;s online softmax, content-level constant deduplication at compile time, and cross-function constant sharing at load time. Each addresses a separate root cause, and the last two turn out to be upstream and downstream of each other.&lt;/p&gt;
&lt;h2 id=&quot;2-what-the-compiler-has-to-materialise&quot;&gt;2. What the compiler has to materialise&lt;/h2&gt;
&lt;p&gt;The trunk has 12 layers, sequence length L = 1467, 16 heads, head dimension 64, and hidden size 1024. Each layer contains two Flash Attention kernel calls with identical input shapes – a Q, a K, a V and one attention mask – whose outputs are concatenated back into the hidden dimension. That is 24 Flash kernels. The two kernels within a layer share a causal mask, and before subgraph partitioning they reference the same bias input.&lt;/p&gt;
&lt;p&gt;The mask is where the arithmetic becomes interesting. The model is autoregressive, so token t may only attend to positions before t. That constraint is expressed as an additive attention mask built from a lower-triangular matrix: zeros on and below the diagonal, a blocking value above it. The blocking value is written as an fp16 extreme and is semantically -inf. Added to the logits before softmax, it drives &lt;code&gt;exp(-inf)&lt;/code&gt; to zero, removing future positions from the attention weights without disturbing the normalising denominator over past positions.&lt;/p&gt;
&lt;p&gt;The mask is stored as &lt;code&gt;[1, 16, 1467, 1467]&lt;/code&gt; in fp16. One copy is 16 × 1467 × 1467 × 2 bytes, or 68,866,944 bytes – about 68.9 MB.&lt;/p&gt;
&lt;p&gt;The model needs one copy. What it gets is 24 copies in device memory, about 1.65 GB, and 12 copies in the constant data segment, about 826 MB. That expansion from one to twenty-four is the whole of the second problem.&lt;/p&gt;
&lt;p&gt;Two details matter for what follows. The mask keeps its head dimension rather than being collapsed to &lt;code&gt;[1, 1, 1467, 1467]&lt;/code&gt;, because the interface has to accommodate per-head biases; the sixteen heads repeat the same causal pattern, so the mask is redundant along that axis, but the conclusion that one copy is logically sufficient is unaffected. And the compiler folds the softmax scale into the bias, dividing the mask constant by &lt;code&gt;scale = 1/sqrt(64) = 1/8&lt;/code&gt;. Numerically that multiplies the mask by eight, and the fp16 extreme on the upper triangle – whether written as the smallest finite fp16 value, -65504, or as a literal -inf – overflows to a literal -inf. The kernel therefore sees a genuine -inf after the bias addition rather than a large negative number, and the NaN analysis that follows has to be carried out under IEEE 754 semantics rather than as a question of magnitude.&lt;/p&gt;
&lt;p&gt;The trunk is not the only large operator. A convolutional stage produces a BEV feature map of roughly 201 MB in fp16, and it dominates the non-attention memory and arithmetic. There are KV cache reads and writes, twelve &lt;code&gt;[1, 1467, 2048]&lt;/code&gt; outputs, plus rotary embeddings, layer norms, and a large population of small fused graph operators – reshape, strided_slice, take, concatenate – with an argmax at the output. A dump of the 2,586 kernel directories shows the distribution: almost everything is a &lt;code&gt;fused_cast_reshape&lt;/code&gt;, &lt;code&gt;fused_layer_norm&lt;/code&gt; or &lt;code&gt;fused_strided_slice&lt;/code&gt;, alongside the 24 Flash kernels and the convolution kernels. The model is overwhelmingly made of small operators. The only tensors large enough to distort the constant segment are the ones the Flash kernels share.&lt;/p&gt;
&lt;p&gt;The weights account for about 577 MB, and several groups within them are byte-identical, layer norm weights shared across layers being the clearest case. The largest single group is duplicated eleven times, so roughly 112 MB is recoverable in principle. Those duplicates have the same origin as the mask duplicates: graph-level copying that the compilation path never recognises or reclaims.&lt;/p&gt;
&lt;h2 id=&quot;3-how-the-pipeline-makes-the-copies&quot;&gt;3. How the pipeline makes the copies&lt;/h2&gt;
&lt;p&gt;The stack is a graph compiler plus operator fusion plus a heterogeneous operator library, in the lineage of TVM and its CUTLASS backend. It runs in eight stages, and the expansion is what those stages produce when each one does something locally reasonable and none of them is responsible for spotting the repetition.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://www.ailven.cc/assets/posts/ai-engine-260930/compilation-pipeline.png&quot; alt=&quot;The eight-stage compilation path from PyTorch model to device execution, with the tuning stage highlighted and content deduplication shown alongside CSE&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 1 – the compilation path. The tuning stage is where bias-scale folding materialises new constants; content deduplication runs alongside structural CSE after it.&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&quot;export-and-partitioning&quot;&gt;Export and partitioning&lt;/h3&gt;
&lt;p&gt;PyTorch exports the model to a graph IR, and the export separates computation from data. Operators become graph nodes, constants become &lt;code&gt;ConstantNode&lt;/code&gt; objects, weights become parameters.&lt;/p&gt;
&lt;p&gt;On the torch side the mask is computed twelve times, once per layer. Export therefore produces twelve mutually independent &lt;code&gt;ConstantNode&lt;/code&gt;s holding byte-identical data. Those twelve are fixed by the model&amp;#39;s graph structure, and they set the base for everything that follows.&lt;/p&gt;
&lt;p&gt;The graph compiler does not emit code per operator. It partitions subgraphs matching a known pattern into standalone global subfunctions and hands each one to the operator library. Flash Attention is one such pattern: &lt;code&gt;QK^T&lt;/code&gt;, the mask addition, online softmax and the PV weighting are recognised as a single unit. After partitioning, Flash Attention is a global subfunction and the causal mask is one of its parameters. The two kernels within a layer shared a bias input before partitioning and still reference the same mask constant node afterwards. This stage invents no new duplication; it preserves the one-per-layer structure of twelve.&lt;/p&gt;
&lt;h3 id=&quot;the-tuning-rewrite&quot;&gt;The tuning rewrite&lt;/h3&gt;
&lt;p&gt;This stage is where both problems originate. It runs several passes in sequence: a quantisation scale rewrite, type inference, bias-scale folding to push the softmax scale into the bias, and a bias shape rewrite to a broadcast-friendly layout. An operator annotator then labels each subgraph with its op type, shapes and dtypes, and a profiler plus an automatic search selects kernel configurations.&lt;/p&gt;
&lt;p&gt;Bias-scale folding is the one that matters. A Flash kernel would rather not multiply by the softmax scale inside the kernel, so the scale is pre-multiplied into the bias. The identity is&lt;/p&gt;
&lt;p&gt;$$\text{softmax}\left((QK^{T} + \text{mask}) \cdot s\right) = \text{softmax}\left(QK^{T} \cdot s + \text{mask} \cdot s\right)$$&lt;/p&gt;
&lt;p&gt;so dividing the mask by &lt;code&gt;1/s&lt;/code&gt; is equivalent. This is textbook constant folding, and it is necessary: subfunction parameters bind directly to kernel arguments at codegen, so leaving the scale unfolded would either keep a divide operator in the graph – an extra elementwise division over a 68.9 MB tensor, roughly 138 MB of read and write traffic – or change the kernel signature. Folding converts a runtime cost into a compile-time constant.&lt;/p&gt;
&lt;p&gt;The defect is in how it folds. The pass folds independently at each call site, and each fold genuinely evaluates &lt;code&gt;divide(mask, scale)&lt;/code&gt; and allocates a fresh &lt;code&gt;ConstantNode&lt;/code&gt; for the result. Twenty-four call sites produce twenty-four independent &lt;code&gt;ConstantNode&lt;/code&gt;s holding byte-identical data. After tuning, the graph carries twenty-four mask constants where it carried twelve. Nothing in the IR layer can tell that the new objects have the same contents.&lt;/p&gt;
&lt;h3 id=&quot;why-structural-cse-cannot-see-it&quot;&gt;Why structural CSE cannot see it&lt;/h3&gt;
&lt;p&gt;The pipeline runs structural common subexpression elimination after tuning. Its criterion is structural equality: two expressions merge only when their operator, their attributes and all their subexpressions are structurally identical.&lt;/p&gt;
&lt;p&gt;Consider &lt;code&gt;divide(mask_A, scale_1)&lt;/code&gt; and &lt;code&gt;divide(mask_B, scale_2)&lt;/code&gt;. Even when &lt;code&gt;mask_A&lt;/code&gt; and &lt;code&gt;mask_B&lt;/code&gt; hold byte-identical data, they are different &lt;code&gt;ConstantNode&lt;/code&gt; objects, and the structural comparison of a &lt;code&gt;ConstantNode&lt;/code&gt; compares object identity rather than content. The two expressions are therefore not structurally equal, CSE leaves them alone, and the twenty-four folded mask constants survive. Enabling the option that hoists constants into function parameters to make them shareable changes how the constants are parameterised, not the criterion by which they are compared.&lt;/p&gt;
&lt;p&gt;The general point is that CSE removes redundant computation – the same expression evaluated twice – and not redundant data. When duplication takes the form of identical content under different structure, every structural tool in the pipeline is blind to it.&lt;/p&gt;
&lt;h3 id=&quot;codegen-the-constant-pool-and-the-runtime&quot;&gt;Codegen, the constant pool and the runtime&lt;/h3&gt;
&lt;p&gt;After CSE the IR enters codegen. Each subgraph becomes C++/CUDA source, and nvcc compiles it into a dynamic library that carries all kernel machine code and determines the arithmetic cost and the latency.&lt;/p&gt;
&lt;p&gt;The graph functions are then compiled to virtual machine bytecode, and the data behind every &lt;code&gt;ConstantNode&lt;/code&gt; is collected into a global constant pool, which is serialised into the data segment. Constant handling does deduplicate, but only within a narrow scope. The first layer is a per-function hash table keyed on the &lt;code&gt;ConstantNode&lt;/code&gt; pointer and scoped to the function currently being compiled: a function that references the same &lt;code&gt;ConstantNode&lt;/code&gt; twice pools it once. The twenty-four masks live in twenty-four different subfunctions, so across functions the table does nothing at all. The second layer is the global pool itself, which appends every &lt;code&gt;ConstantNode&lt;/code&gt; it has not seen before.&lt;/p&gt;
&lt;p&gt;Twenty-four functions therefore pool the mask twenty-four times. The data segment ends up with twelve entries rather than twenty-four, and the reason is this. VM compilation walks functions one at a time; the two call sites in a layer share the same folded result object after the rewrite, so compiling the paths through those two call sites hits one &lt;code&gt;ConstantNode&lt;/code&gt;, while the twelve layers are twelve distinct objects. Twelve or twenty-four makes no difference to the mechanism, which is that the pool keys on object identity and charges separately for objects whose contents agree.&lt;/p&gt;
&lt;p&gt;At runtime the VM loads the dynamic library and the constant pool, with the whole pool entering device memory, and executes the bytecode. The load-constant instruction means &amp;quot;take entry &lt;code&gt;index&lt;/code&gt; of the pool, ensure it has a device buffer, point the register at it&amp;quot;, and the implementation caches by index: the first execution of a given index allocates the buffer, and later ones reuse it.&lt;/p&gt;
&lt;p&gt;Had the twenty-four kernels all referenced one index, device memory would hold one buffer. VM compilation has already assigned them different indices, because it pooled twelve distinct objects, and a cache keyed on index cannot hit across them. Each index gets its own allocation, and the mask&amp;#39;s device footprint becomes twenty-four copies.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://www.ailven.cc/assets/posts/ai-engine-260930/causal-mask-lifecycle.png&quot; alt=&quot;The life cycle of the causal mask through export, tuning, structural CSE, the VM constant pool and the runtime, growing from one logical copy to twenty-four device buffers&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 2 – the same tensor at each stage. The logical requirement is one copy; the data segment holds twelve entries; device memory holds twenty-four buffers.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;Five stages, five copies, and no stage recognises the copy the previous one made. That is the whole of the expansion from one to twenty-four.&lt;/p&gt;
&lt;h2 id=&quot;4-where-the-nan-comes-from&quot;&gt;4. Where the NaN comes from&lt;/h2&gt;
&lt;p&gt;Three conditions have to hold at once for the NaN: the kernel takes the non-causal path, the upper triangle of the mask is a literal -inf, and online softmax rescales its running accumulators.&lt;/p&gt;
&lt;p&gt;The first is a property of the kernel as compiled. The original Flash kernel does not enable its built-in causality test, so it walks every column block – block size 128, twelve blocks over a sequence of 1467. Causality is supplied entirely by the external mask. The kernel has no idea which positions lie in the future, and reads only the values in the mask.&lt;/p&gt;
&lt;p&gt;The second follows from the scale folding described above. After bias-scale folding, the extreme on the upper triangle has overflowed to a literal fp16 -inf. For a query at an early position t, every column block past the causal boundary – every key position greater than t – is uniformly -inf.&lt;/p&gt;
&lt;p&gt;The third is the structure of online softmax. Flash Attention maintains a running maximum &lt;code&gt;scores_max&lt;/code&gt; and an accumulated output &lt;code&gt;acc_o&lt;/code&gt;, block by block, rescaling the accumulated values by the ratio of the new maximum to the old one so that the result matches a global softmax.&lt;/p&gt;
&lt;h3 id=&quot;the-rescale&quot;&gt;The rescale&lt;/h3&gt;
&lt;p&gt;The fault is in the rescale function, whose signature takes a &lt;code&gt;Check_inf&lt;/code&gt; template parameter defaulting to false.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cpp&quot;&gt;float scores_max_cur = !Check_inf
    ? scores_max(mi)
    : (scores_max(mi) == -INFINITY ? 0.0f : scores_max(mi));
float scores_scale = exp2f((scores_max_prev(mi) - scores_max_cur) * softmax_scale_log2);
scores_sum(mi) *= scores_scale;
// ... acc_o_rowcol(mi, ni) *= scores_scale;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;scores_max_prev&lt;/code&gt; is the previous round&amp;#39;s maximum; &lt;code&gt;scores_max&lt;/code&gt; is the maximum over the current column block after &lt;code&gt;reduce_max&lt;/code&gt;. The subtraction of &lt;code&gt;scores_max_cur&lt;/code&gt; from &lt;code&gt;scores_max_prev&lt;/code&gt; is the whole of the bug.&lt;/p&gt;
&lt;h3 id=&quot;the-trace&quot;&gt;The trace&lt;/h3&gt;
&lt;p&gt;Take a query row whose causal boundary lies early enough that two consecutive column blocks are entirely in its future, so both are uniformly -inf.&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://www.ailven.cc/assets/posts/ai-engine-260930/nan-trigger.png&quot; alt=&quot;The NaN trigger sequence: two consecutive fully masked tiles drive prev minus cur to -inf minus -inf, which contaminates the accumulated output&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 3 – the trigger sequence. Two consecutive fully masked tiles are required; one alone is harmless.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The first of the two blocks leaves &lt;code&gt;scores_max&lt;/code&gt; at -inf, since the maximum of a set of -inf values is still -inf. Entering the second block, the kernel copies the previous maximum into &lt;code&gt;scores_max_prev = -inf&lt;/code&gt;, reduces the new block to &lt;code&gt;scores_max = -inf&lt;/code&gt;, and with &lt;code&gt;Check_inf&lt;/code&gt; false sets &lt;code&gt;scores_max_cur&lt;/code&gt; to -inf. The scale is then &lt;code&gt;exp2f((-inf - (-inf)) * scale_log2)&lt;/code&gt;. Under IEEE 754, -inf minus -inf is an invalid operation and yields NaN; NaN multiplied by anything is NaN; &lt;code&gt;exp2f(NaN)&lt;/code&gt; is NaN. &lt;code&gt;scores_sum&lt;/code&gt; and &lt;code&gt;acc_o&lt;/code&gt; are multiplied by NaN, and the NaN enters the accumulated output.&lt;/p&gt;
&lt;h3 id=&quot;why-it-stays-hidden&quot;&gt;Why it stays hidden&lt;/h3&gt;
&lt;p&gt;A single fully masked column block triggers nothing. &lt;code&gt;prev = -inf&lt;/code&gt; together with &lt;code&gt;cur = -inf&lt;/code&gt; requires two consecutive fully masked blocks, and if &lt;code&gt;prev = -inf&lt;/code&gt; while &lt;code&gt;cur&lt;/code&gt; is finite then (-inf) minus a finite value is -inf, &lt;code&gt;exp2f(-inf)&lt;/code&gt; is zero, and the scale clears the accumulated history, which is mathematically correct. The bug therefore depends on where the causal boundary falls relative to the column-block boundaries. It fires only at particular sequence positions, which is why it never surfaced in a simple unit test and yet appeared reliably on real long sequences.&lt;/p&gt;
&lt;p&gt;The upstream code does guard one step of this computation: the &lt;code&gt;exp2&lt;/code&gt; applied to the scores themselves is written as &lt;code&gt;max == -INFINITY ? 0 : exp2(scores)&lt;/code&gt;. That guard covers the &lt;code&gt;exp2&lt;/code&gt; of the scores, not the rescale of the accumulated output. &lt;code&gt;scores_scale&lt;/code&gt; is computed independently inside the rescale function, from an unguarded copy of &lt;code&gt;scores_max_prev&lt;/code&gt;, so the protection sits at a different point in the formula from the one where -inf meets -inf.&lt;/p&gt;
&lt;h3 id=&quot;propagation&quot;&gt;Propagation&lt;/h3&gt;
&lt;p&gt;Once NaN is in the accumulated output, the rest is determined. The epilogue normalises by &lt;code&gt;1/scores_sum&lt;/code&gt;, which propagates it, and the result is cast to fp16 and written to global memory as the kernel&amp;#39;s output. That output is added back into the residual stream, &lt;code&gt;x = x + attn_out&lt;/code&gt;, and the addition contaminates the whole hidden state. Every subsequent layer norm spreads it further, since the mean and variance of a vector containing NaN are themselves NaN, and the MLP and the next attention follow. After twelve layers, every output – logits, trajectory deltas, KV cache – is NaN.&lt;/p&gt;
&lt;p&gt;Residual connections and layer norms do not absorb NaN; they broadcast it. That is why one kernel producing NaN at one point takes down the entire output.&lt;/p&gt;
&lt;h2 id=&quot;5-the-bloat-accounted-for&quot;&gt;5. The bloat, accounted for&lt;/h2&gt;
&lt;p&gt;The original data segment is 1,357.1 MB, and the causal mask accounts for twelve copies of it: one per layer, 68.9 MB each, about 826 MB, or 61% of the segment. Peak device memory is 2,039.68 MB, of which the mask&amp;#39;s buffers account for twenty-four copies – one independent buffer per kernel, about 1,652 MB. One copy would have been 69 MB.&lt;/p&gt;
&lt;p&gt;The instructive part is that no single stage is a bug. Each behaves correctly inside its own abstraction.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Stage&lt;/th&gt;
&lt;th&gt;What it did&lt;/th&gt;
&lt;th&gt;Why that is defensible&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Export&lt;/td&gt;
&lt;td&gt;one independent &lt;code&gt;ConstantNode&lt;/code&gt; per layer, twelve in total&lt;/td&gt;
&lt;td&gt;it reflects the model&amp;#39;s graph structure faithfully, and has no reason to compare two tensors byte by byte&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Tuning fold&lt;/td&gt;
&lt;td&gt;materialised a new &lt;code&gt;ConstantNode&lt;/code&gt; per call site, twenty-four objects&lt;/td&gt;
&lt;td&gt;folding exists to remove runtime work, and materialising per call site is the direct implementation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Structural CSE&lt;/td&gt;
&lt;td&gt;failed to match the content duplicates&lt;/td&gt;
&lt;td&gt;its criterion is structural equivalence, and it carries no data-comparison duty&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Constant pool&lt;/td&gt;
&lt;td&gt;pooled by object pointer, no cross-function reuse&lt;/td&gt;
&lt;td&gt;a compiler does not assume that two distinct objects agree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Runtime&lt;/td&gt;
&lt;td&gt;indices differed, so the cache never hit&lt;/td&gt;
&lt;td&gt;the index cache already allocates once per index; it was handed indices that never coincide&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The bloat is what those correct local behaviours add up to. Every stage discharges its own responsibility, and none of them owns the cross-stage question of whether two constants hold the same bytes. Fixing any single stage in isolation would not close the chain.&lt;/p&gt;
&lt;h3 id=&quot;twelve-entries-twenty-four-allocations&quot;&gt;Twelve entries, twenty-four allocations&lt;/h3&gt;
&lt;p&gt;There is a second copy, and it is not the same as the first. The data segment is counted in constant pool entries, of which there are twelve. Device memory is counted in allocations, which follow the load instruction indices. One pool entry referenced from several functions under different indices is allocated once per index. Twelve entries, each referenced by two kernels under two different indices, give twenty-four buffers.&lt;/p&gt;
&lt;p&gt;Data is therefore duplicated twice, once at serialisation and once at allocation, and repairing only the serialisation stage leaves the runtime duplicating as before. The two stages need separate repairs because they key on different things: the pool on object identity, the runtime cache on index.&lt;/p&gt;
&lt;h2 id=&quot;6-the-three-fixes&quot;&gt;6. The three fixes&lt;/h2&gt;
&lt;h3 id=&quot;a-compile-time-guard-in-the-kernel&quot;&gt;A compile-time guard in the kernel&lt;/h3&gt;
&lt;p&gt;The smallest change sits in the kernel. A compile-time constant decides whether -inf protection is enabled, and it is passed to the rescale function&amp;#39;s &lt;code&gt;Check_inf&lt;/code&gt; parameter.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cpp&quot;&gt;// An external additive bias (position bias / swin attention mask) may contain
// -inf, and those paths need the same protection in the softmax_rescale stage;
// otherwise two consecutive fully masked key tiles compute
// exp2f(-inf - (-inf)) = NaN.
constexpr bool kCheckInf = Is_causal
                        || Kernel_traits::Has_Position_Bias
                        || Kernel_traits::Has_Swin_Attn_Mask;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;All three rescale call sites pass &lt;code&gt;Check_inf = kCheckInf&lt;/code&gt;, and with &lt;code&gt;Check_inf&lt;/code&gt; true the evaluation of &lt;code&gt;scores_max_cur&lt;/code&gt; folds -inf to zero:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cpp&quot;&gt;float scores_max_cur = scores_max(mi) == -INFINITY ? 0.0f : scores_max(mi);
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Now &lt;code&gt;scores_max_prev&lt;/code&gt; (-inf) minus &lt;code&gt;scores_max_cur&lt;/code&gt; (0) is -inf, &lt;code&gt;exp2f(-inf)&lt;/code&gt; is zero, the scale is zero, and the accumulated history is cleared correctly instead of being multiplied by NaN.&lt;/p&gt;
&lt;p&gt;Making the guard a compile-time constant is the right choice because whether a kernel instance carries an external bias is a static fact about the instantiation rather than a runtime condition. A constant lets the compiler eliminate the irrelevant branch during instantiation, so no runtime &lt;code&gt;if&lt;/code&gt; is added to kernels that do not need it.&lt;/p&gt;
&lt;p&gt;One alternative was rejected. Giving the kernel a causal-mode template parameter, in the manner of standard Flash Attention, would clip the column-block loop to the causal region and never enter two fully masked blocks. The change was line-for-line equivalent to a known-correct fallback at source level, yet measured byte-identical inputs into the first Flash kernel produced outputs correlating at 0.087 with the reference. The root cause was never pinned down; the plausible explanation is that adding and removing template parameters perturbs C++ template instantiation and compiler optimisation – register allocation, shared-memory layout, instruction scheduling – into a numerically different and wrong result. The minimal change was taken instead: one compile-time boolean, no alteration to the traversal, the smallest possible disturbance to instantiation.&lt;/p&gt;
&lt;h3 id=&quot;content-level-deduplication-at-compile-time&quot;&gt;Content-level deduplication at compile time&lt;/h3&gt;
&lt;p&gt;The second change is a new compile-time pass that deduplicates constants by content. Its core is under twenty lines. It removes redundant data – constants whose bytes agree but whose object identities differ – and leaves redundant computation alone.&lt;/p&gt;
&lt;p&gt;Three things matter in it. The content fingerprint is a key of &lt;code&gt;(dtype, shape tuple, sha256 of the bytes)&lt;/code&gt;, a triple that determines a constant&amp;#39;s content identity. A size filter drops constants below a threshold, 4096 elements say, which avoids hashing a large population of small scale and bias vectors that would save nothing if merged. And the representative table is shared across functions: an &lt;code&gt;ExprMutator&lt;/code&gt; rewrites &lt;code&gt;visit_constant&lt;/code&gt; so that a constant whose content key is already registered is replaced by the representative, and one that is not registers itself.&lt;/p&gt;
&lt;p&gt;The sharing is the point. The twenty-four Flash kernels are twenty-four independent global functions. With one representative table per function, deduplication can only happen inside a function, and none of the twelve masks can merge with any other. With a single table for the module, the mask encountered in the first function registers as the representative and the remaining twenty-three references are replaced by that same &lt;code&gt;ConstantNode&lt;/code&gt;: twenty-four references collapse to one object and one copy of the data. How wide the table is decides the ceiling on what deduplication can recover.&lt;/p&gt;
&lt;p&gt;The pass runs after the tuning rewrite, alongside structural CSE, with CSE first. Running it after tuning matters because the duplicate &lt;code&gt;ConstantNode&lt;/code&gt;s are exactly what bias-scale folding produced; run before tuning, deduplication sees twelve shared bias parameters after partitioning and finds nothing to merge, and tuning then materialises twenty-four new objects. CSE first matters because it merges the structurally equivalent expressions and leaves content deduplication to handle the structurally different but byte-identical remainder. The two are complementary, and doing CSE first hands content deduplication a smaller graph.&lt;/p&gt;
&lt;h3 id=&quot;cross-function-constant-sharing-at-runtime&quot;&gt;Cross-function constant sharing at runtime&lt;/h3&gt;
&lt;p&gt;The third change sits at the boundary between VM compilation and runtime: enabling a cross-function constant sharing switch that already exists and is off by default.&lt;/p&gt;
&lt;p&gt;The VM compiler handles a constant node through three branches. The first reuses within a function, reusing the register with a single Move when the current function has already handled this &lt;code&gt;ConstantNode&lt;/code&gt;. The second reuses across functions, and this is the branch the fix enables: when the underlying data has already been pooled by any earlier function, it emits an instruction loading the existing index instead of appending a new constant. The third is the default path, appending a new constant to the pool and recording both the in-function mapping and the data-to-index mapping. Only the first and third are active by default; the second is behind an environment variable, and the fix turns it on at the compilation entry point.&lt;/p&gt;
&lt;p&gt;The second branch is backed by a global map from the underlying data object, an &lt;code&gt;NDArray&lt;/code&gt;, to a constant pool index. Its header comment already describes its purpose as sharing constants across VM functions, and it compares data objects by pointer. The third branch registers data against index when it pools, and the second branch looks the data up before pooling.&lt;/p&gt;
&lt;p&gt;The switch on its own fixes nothing, because the map compares pointers. If the twenty-four masks are twenty-four distinct data objects, identical bytes notwithstanding, the pointers differ and the lookup never hits. Content deduplication has to unify the twenty-four references onto one &lt;code&gt;ConstantNode&lt;/code&gt; and one copy of the data first, and only then do the pointers agree and the lookup succeed. In the other direction, content deduplication without the switch shrinks the data segment, since some paths benefit once the objects are unified, but VM compilation still pools per function through the third branch, the runtime still allocates per index, and the memory saving is discounted.&lt;/p&gt;
&lt;p&gt;With both in place, every kernel&amp;#39;s load instruction points at the same index, the runtime caches by index, the first allocation is reused by all the rest, and all the kernels share one device buffer. The mask&amp;#39;s device footprint falls from twenty-four copies, about 1,652 MB, to one, about 69 MB. Content deduplication unifies object identity; cross-function sharing unifies the index. The first gives the second something to share, and the second makes the sharing real in device memory.&lt;/p&gt;
&lt;h3 id=&quot;how-the-three-fit-together&quot;&gt;How the three fit together&lt;/h3&gt;
&lt;p&gt;&lt;img src=&quot;https://www.ailven.cc/assets/posts/ai-engine-260930/three-fixes.png&quot; alt=&quot;The three fixes positioned against the compilation path: constant deduplication at compile time, the kernel guard at codegen, and constant sharing at runtime&quot;&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure 4 – where each fix attaches to the path in Figure 1.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;The first fix restores numerical correctness, closing the one place where -inf minus -inf leaks a NaN. The second fixes the data segment, collapsing twenty-four identical objects into one. The third fixes device memory, letting different functions share one pool entry and one device buffer.&lt;/p&gt;
&lt;p&gt;There is a causal chain between the last two: deduplication creates the pointer identity that sharing depends on, and sharing turns unified objects into shared memory. The first fix is independent of that chain and acts on correctness. Deduplication and cross-function sharing both happen at compile time and load time, so neither adds per-token cost, and the kernel guard is evaluated at compile time and eliminated at instantiation. Latency is unchanged, and the dynamic library grows by nothing measurable.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Numerical correctness&lt;/td&gt;
&lt;td&gt;zero NaN; 2,586 kernel outputs byte-identical to an fp32 reference&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Constant data segment&lt;/td&gt;
&lt;td&gt;down 63%, 830 MB saved&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Peak device memory&lt;/td&gt;
&lt;td&gt;down 39%, 792 MB saved&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Latency&lt;/td&gt;
&lt;td&gt;unchanged, no regression&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Dynamic library size&lt;/td&gt;
&lt;td&gt;essentially unchanged&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;h2 id=&quot;7-two-kinds-of-redundancy&quot;&gt;7. Two kinds of redundancy&lt;/h2&gt;
&lt;p&gt;The two failures came from three gaps in one compilation path, and the gaps are not of the same kind. The NaN is a missing guard in the online softmax rescale, at the single operation where -inf meets -inf. The data segment bloat is a missing content-level comparison in the IR, which leaves the constant pool keying on object identity. The device memory bloat is a missing index reuse in the VM compiler, which leaves the runtime cache keyed on indices that never coincide. The last two sit upstream and downstream of each other, and neither works alone.&lt;/p&gt;
&lt;p&gt;Diagnosing the NaN meant refusing to stop at the observation that the output contains NaN, and following IEEE 754 semantics line by line until the one operation capable of producing it was isolated. Diagnosing the bloat meant separating structural equivalence from content equivalence. They are different kinds of redundancy: structural CSE tests for the first, and byte-identical data behind distinct objects falls outside its criterion altogether. Those two facts between them explain why the numerical fix belongs in the kernel and the volumetric fix belongs in the compiler&amp;#39;s constant handling.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Agent Foundry: A Repository Contract for Agentic Development</title>
    <link href="https://www.ailven.cc/posts/ai-agent-260930.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/ai-agent-260930.html</id>
    <published>2026-09-30T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>How Agent Foundry turns reusable skills, project profiles, session checks and executable policy into a development workflow that can survive a change of agent or session.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 30 September 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;An agent can follow a detailed instruction in one conversation and lose it in the next. It can also describe a quality gate accurately while never running it. The useful unit of agentic development is therefore the repository: a place where the procedure is discoverable, the project state is explicit and the checks still work after the conversation has ended. I built &lt;a href=&quot;https://github.com/AilvenLiu/agent-foundry&quot;&gt;Agent Foundry&lt;/a&gt; around that boundary and have kept using it as my own skill hub for agentic development.&lt;/p&gt;
&lt;p&gt;I keep procedures in &lt;code&gt;.agents/skills/&lt;/code&gt; and obligations in &lt;code&gt;.agents/constraints/&lt;/code&gt;. The project profile selects the relevant rules; wrappers and hooks check conditions an agent might otherwise skip. When a build goes wrong, I can trace the failure to a procedure, a policy decision or an executable check instead of trying to reconstruct what the agent was told.&lt;/p&gt;
&lt;h2 id=&quot;a-procedure-needs-a-durable-home&quot;&gt;A procedure needs a durable home&lt;/h2&gt;
&lt;p&gt;A recurring task gets a canonical procedure at &lt;code&gt;.agents/skills/&amp;lt;name&amp;gt;/SKILL.md&lt;/code&gt;. The collection covers session initialisation, dependency changes, builds, pre-commit validation and code navigation, alongside longer workflows such as roadmaps and deployment. The &lt;code&gt;pre-commit&lt;/code&gt; skill, for example, directs the agent to &lt;code&gt;agent-precommit&lt;/code&gt;. That command selects tools for the project profile and explicitly collects tests from both the application and the agent infrastructure. Its exit status carries more weight than the agent’s account of what it intended to check.&lt;/p&gt;
&lt;p&gt;Policy sits beside the procedures in &lt;code&gt;.agents/constraints/&lt;/code&gt;. Common files govern the session and Git workflow; language files cover matters such as Python dependencies and C++ memory safety. The hybrid profile assigns native build ownership to CMake. That rule applies whether the agent arrived through &lt;code&gt;build&lt;/code&gt;, &lt;code&gt;dependency&lt;/code&gt; or a request to add a binding. Keeping it outside any one procedure lets the same constraint govern all three.&lt;/p&gt;
&lt;p&gt;A policy change should have one source edit. Codex reads the skill body under &lt;code&gt;.agents/skills/&lt;/code&gt;; Claude Code discovers a short delegate under &lt;code&gt;.claude/skills/&lt;/code&gt; that points back to it. The two agents enter through &lt;code&gt;AGENTS.md&lt;/code&gt; and &lt;code&gt;CLAUDE.md&lt;/code&gt; respectively, then reach the same &lt;code&gt;.agents/bin/agent-*&lt;/code&gt; commands. Tests verify the delegate links and reject skills placed in the obsolete &lt;code&gt;.codex/skills/&lt;/code&gt; location. Without those checks, two apparently equivalent platform copies could diverge quietly as the foundry evolves.&lt;/p&gt;
&lt;p&gt;The adapters cannot erase platform differences. Hooks are registered separately, and repository instructions remain subject to each platform’s tool and permission requirements. An earlier version of Agent Foundry made this boundary painfully clear: its instruction wording interfered with normal Claude Code sessions. The fix scoped precedence to repository-controlled guidance and made initialisation print a bounded list of constraint paths. The rules remained enforceable without asking the agent to treat a local file as higher authority than its platform.&lt;/p&gt;
&lt;h2 id=&quot;session-state-is-a-checkable-precondition&quot;&gt;Session state is a checkable precondition&lt;/h2&gt;
&lt;p&gt;The &lt;code&gt;init&lt;/code&gt; procedure resolves a project profile, audits declared capabilities and prints a manifest of relevant constraint paths. It records the result in &lt;code&gt;.agents/session_state.json&lt;/code&gt;, with a compatibility copy for Claude Code. The manifest is deliberately bounded. It selects common constraints and the language rules needed for the current profile and files, leaving the agent to read the listed bodies before the relevant work. Filling the context window with every policy file would make initialisation expensive while giving little evidence that the right rule was applied.&lt;/p&gt;
&lt;p&gt;The capability list lives in &lt;code&gt;.agents/capabilities.yml&lt;/code&gt;. It declares shared skills and repository commands, then adds platform requirements where they differ. A Python project requires the Python environment procedure; a CUDA-targeted project needs the GPU CI surface. The audit turns a missing wrapper or skill into a reported failure. The session hook then uses the recorded result as a gate for mutation. In the Claude path, the pre-tool hook blocks writes before initialisation and blocks mutation after a failed capability audit. The Codex adapter is registered separately, with shared policy logic underneath. This does not prove that an agent understood every instruction; it makes an important prerequisite observable.&lt;/p&gt;
&lt;p&gt;In a hybrid repository, an agent editing packaging configuration can accidentally move compiler policy out of CMake. The project profile identifies both languages, so initialisation selects the C++ and hybrid constraints before the edit. The pre-tool hook can stop work that begins without initialisation. The native targets, tests, benchmarks, compile options and ABI-sensitive settings remain CMake’s responsibility; scikit-build-core only bridges them into a wheel. A later constraint check can catch some forbidden patterns. None of these checks establishes that the agent read every selected file or understood the build graph, so the changed targets still need review.&lt;/p&gt;
&lt;p&gt;For longer work, &lt;code&gt;agent-roadmap&lt;/code&gt; records phase and task dependencies under &lt;code&gt;agent_roadmaps/&lt;/code&gt;. The active phase and current task are explicit, and the workflow can create a handoff for another session. These files are temporary execution state, not permanent product documentation. Once the roadmap is complete, the workspace is cleared back to its placeholder. This avoids treating a stale session note as if it were a durable rule. It also gives an agent resuming work a narrower question than ‘What were we doing?’: which dependency-ready task is active, and which invariants govern it?&lt;/p&gt;
&lt;h2 id=&quot;the-project-profile-travels-with-the-code&quot;&gt;The project profile travels with the code&lt;/h2&gt;
&lt;p&gt;Agent Foundry also generates repositories. Its &lt;code&gt;create-project&lt;/code&gt; script copies the shared &lt;code&gt;.agents/&lt;/code&gt;, &lt;code&gt;.claude/&lt;/code&gt;, &lt;code&gt;.codex/&lt;/code&gt;, &lt;code&gt;.github/&lt;/code&gt; and roadmap structure, then overlays one of three profiles: Python, C++/CUDA or hybrid. The overlay provides that project’s &lt;code&gt;AGENTS.md&lt;/code&gt;, &lt;code&gt;CLAUDE.md&lt;/code&gt;, &lt;code&gt;CONTRIBUTING.md&lt;/code&gt; and &lt;code&gt;.agents/project.yml&lt;/code&gt;. The generated profile is an input to later constraint selection and validation, not merely a label attached to the template.&lt;/p&gt;
&lt;p&gt;The distinction is clearest in the hybrid profile. CMake is responsible for the native build graph and CPM for lightweight C++ dependency acquisition. Poetry manages the Python environment; scikit-build-core is a packaging bridge. Those choices appear in the generated layout and rules, so an agent has an answer when it encounters a Python package with CUDA code. The Python profile, by contrast, does not inherit C++-specific build obligations. Capability audits filter their requirements by profile as well. A copied template that demanded every possible skill in every project would soon teach agents to ignore failed audits.&lt;/p&gt;
&lt;p&gt;The generator intentionally omits template-only material and historical roadmap state, then creates an initial Git commit. That gives a new project the current operating contract without inheriting the foundry’s own temporary work. It also sets a maintenance cost: generated repositories are copies, and later foundry changes do not propagate automatically. The compatibility guide describes a migration process that preserves project-specific customisations, updates the copied instruction surfaces and checks a freshly generated project. Cross-project reuse here works by deliberate distribution; the copied rules have to be maintained.&lt;/p&gt;
&lt;h2 id=&quot;two-scales-of-engineering-work&quot;&gt;Two scales of engineering work&lt;/h2&gt;
&lt;p&gt;An empty repository offers little context for the first build decision. Agent Foundry supplies the project profile, initial directory layout and entrypoints, then routes Python dependencies through Poetry and native ones through pinned CPM declarations. In a hybrid project, the build wrapper runs direct CMake configuration, compilation and CTest before the Python packaging step. A roadmap records dependencies and current focus when the work spans sessions. Those files give the next agent a branch, a task and a build command to inspect; the application’s domain architecture still has to be designed for the system at hand.&lt;/p&gt;
&lt;p&gt;Adding a small feature to an established codebase poses a different problem. The requested change may be ten lines, while its effect crosses a public interface, a build target and several call sites. The &lt;code&gt;navigate&lt;/code&gt; skill offers definition, reference, dependency and call-graph searches for Python and C++, with &lt;code&gt;rg&lt;/code&gt; as the fallback when an agent lacks the native skill. Python analysis uses the AST; C++ lookup combines regex and clang-based tools and is more reliable when &lt;code&gt;compile_commands.json&lt;/code&gt; is present. Dynamic imports and metaprogramming can escape that analysis. The output is a map for review, with completeness still requiring an engineer’s judgement.&lt;/p&gt;
&lt;p&gt;The next step is to establish the existing repository’s authority before changing a module. An agent reads its entrypoint, resolves its project profile and checks the active roadmap, if there is one. It can then identify the smallest edit surface and the governing build and dependency rules. A new C++ component in a hybrid repository belongs in the CMake target graph; a Python binding may expose it, but must not silently replace the native test or installation path. After the change, the build and pre-commit procedures exercise the affected languages, while constraint checks inspect prohibited patterns. A small module can be treated as a local edit without pretending the surrounding engineering system is small.&lt;/p&gt;
&lt;p&gt;Agent Foundry does not automatically retrofit every existing repository. Its generator was built for new projects, and its compatibility guide treats downstream migration as a review of local customisations and copied files. In an established codebase, adopting the hub means reconciling its contract with the repository’s current build graph and policies. That friction is useful information. If a skill’s preferred command cannot be run without overriding the host project’s design, the mismatch should be resolved explicitly before the feature is merged into it.&lt;/p&gt;
&lt;h2 id=&quot;validation-must-reach-the-workflow-not-just-the-prose&quot;&gt;Validation must reach the workflow, not just the prose&lt;/h2&gt;
&lt;p&gt;&lt;code&gt;agent-check-constraints&lt;/code&gt; runs structural, instruction-safety and forbidden-pattern checks. The &lt;code&gt;agent-commit&lt;/code&gt; wrapper takes explicit file paths, rejects some sensitive path shapes and calls the pre-commit and branch-policy gates before staging. For Python, pre-commit selects Ruff, mypy and pytest; C++ projects use the native toolchain. A failed command leaves an exit code and a finding for the next session to inspect.&lt;/p&gt;
&lt;p&gt;The tests create projects with the actual template function, add source files and invoke the wrappers copied into those projects. They inspect exit codes and findings after seeding known violations. A scanner unit test alone would not catch a broken wrapper path or a skill missing from the generated tree.&lt;/p&gt;
&lt;p&gt;For example, a generated Python fixture contains a hard-coded secret, a mutable default argument, a bare &lt;code&gt;except&lt;/code&gt;, an &lt;code&gt;eval&lt;/code&gt; call and a shell-based &lt;code&gt;pip install&lt;/code&gt;. The scenario suite checks which of those violations the distributed tooling reports. Its results describe detection of those fixtures, not the frequency of mistakes in ordinary agent work. I still read the resulting diff and run validation for the code that actually changed.&lt;/p&gt;
&lt;p&gt;The Git history starts in January 2026. Later changes addressed release promotion, instruction compatibility and downstream hardening; the Claude Code failure, in particular, changed both the authority wording and the output of &lt;code&gt;init&lt;/code&gt;. Those revisions came from using the hub and finding where its contract broke. A skill count would not show them.&lt;/p&gt;
&lt;p&gt;Deployment choices and code review still demand judgement. After a session change, though, I can inspect &lt;code&gt;.agents/project.yml&lt;/code&gt;, run &lt;code&gt;agent-init&lt;/code&gt;, read the selected constraints and see which gate passed or failed. The next edit begins from those records and the code itself.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The Value Boundary of Specialised Operators for High-Performance Transformer Inference in Modern AI Compiler Stacks</title>
    <link href="https://www.ailven.cc/posts/ai-compiler-260911.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/ai-compiler-260911.html</id>
    <published>2026-09-11T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>An examination of how much value remains in specialised HPC kernel engineering for Transformer inference once modern AI compiler stacks – TVM Relax, TorchInductor, CUTLASS/CuTe and Triton – have absorbed conventional optimisations, assessed through compiler capability boundaries, hardware utilisation, Roofline and Amdahl reasoning, and service-level SLOs.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 11 September 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;abstract&quot;&gt;Abstract&lt;/h2&gt;
&lt;p&gt;High-performance Transformer inference has historically relied heavily on expert-written CUDA kernels, operator fusion, and bespoke implementations tuned to particular model shapes. Over the past few years, however, the technical environment has changed materially. Low-level libraries such as cuBLASLt, CUTLASS/CuTe and cuDNN have continued to broaden their coverage; compilers such as TVM Relax and TorchInductor can now perform graph-level pattern matching, backend dispatch, autotuning and operator fusion; and lower-level DSLs such as CuTe DSL and Triton have exposed to compilers forms of hardware specialisation that previously required substantial CUDA C++ template engineering. By 2026, PyTorch Inductor could place CuTeDSL/NVGEMM, ATen and Triton within a single autotuning process, while FlashAttention-4 itself was implemented in CuTe DSL. This sharpens a longstanding question: once general-purpose compilers and high-performance operator libraries have become sufficiently capable, how much value remains in developing specialised HPC kernels for Transformer inference?&lt;/p&gt;
&lt;p&gt;There is no meaningful universal answer such as ‘5%’, ‘20%’ or ‘2×’. The reason is that the remaining performance headroom has progressively migrated away from isolated arithmetic kernels towards data layout, quantisation representation, dynamic scheduling, KV-cache access, sparsity, cross-operator dataflow, and communication–computation co-ordination. For standard dense GEMMs with large, regular shapes and common numerical formats, the benefit of reimplementing the main loop from a strong baseline is narrowing. By contrast, in low-effective-batch inference, dynamic attention, MoE, model-specific low-precision formats, cross-operator fusion and multi-GPU execution, specialisation can still alter the amount of data moved, the synchronisation structure and the critical path itself; in such cases the benefit may be far larger than what would traditionally be described as ‘operator optimisation’.&lt;/p&gt;
&lt;p&gt;This article re-examines the issue from five perspectives: compiler capability boundaries, hardware utilisation, Roofline-style reasoning, Amdahl’s law, and service-level SLOs. Its central argument is that specialised HPC optimisation has not lost its value, but that its economic value is shifting away from ‘owning more private kernels’ towards ‘identifying workload structure that the general stack does not yet express well, and specialising that structure at the lowest sustainable maintenance cost’.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Keywords:&lt;/strong&gt; Transformer inference; AI compiler; CUTLASS; CuTe; cuBLASLt; operator fusion; attention; MoE; kernel specialisation; GPU performance engineering&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Software status and published experimental results discussed in this article are current to September 2026. Hardware, models, baselines and benchmarking methodology differ across the cited work. Performance figures are therefore used only to illustrate the nature of remaining optimisation headroom; they are not averaged across papers and should not be extrapolated directly to unrelated workloads.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&quot;1-reframing-the-problem-from-operator-specialisation-to-workload-specialisation&quot;&gt;1. Reframing the Problem: From Operator Specialisation to Workload Specialisation&lt;/h2&gt;
&lt;p&gt;In this article, a specialised HPC operator is not restricted to hand-written CUDA C++. Any implementation may be regarded as specialised if it is tailored to a particular hardware architecture, model structure, shape distribution, numerical format or runtime state, and if its performance depends materially on those assumptions. Such an implementation may be written in CUDA, CuTe DSL, Triton or another low-level DSL; it may also extend beyond a conventional operator into a persistent kernel, mega-kernel, or communication–computation fused execution unit.&lt;/p&gt;
&lt;p&gt;Likewise, a ‘modern AI compiler’ can no longer be understood merely as a graph compiler that performs constant folding and pointwise fusion. TVM Relax’s Dataflow Pattern Language can identify subgraphs, fuse them, or dispatch them to external backends such as CUTLASS, cuBLAS and cuDNN, while allowing projects to define their own rewrite patterns. TorchInductor, meanwhile, can place Triton, ATen/cuBLAS, CUTLASS and CuTeDSL/NVGEMM into practical candidate-generation and autotuning flows.[1–4]&lt;/p&gt;
&lt;p&gt;CUTLASS itself is also changing in character. CUTLASS 4.8.0, released in August 2026, continued the expansion of CuTe DSL and added support for new FP8/FP4 Tensor Core capabilities on Rubin, alongside enlarged TMEM and shared-memory capabilities. At the same time, the CUTLASS Operator API wraps DSL kernels as discoverable, filterable and instantiable operators, and can use &lt;code&gt;nvMatmulHeuristics&lt;/code&gt; to rank large sets of valid candidates.[1][5]&lt;/p&gt;
&lt;p&gt;The old framing of ‘compiler versus hand-written kernel’ is therefore no longer accurate. A more representative stack now looks like:&lt;/p&gt;
&lt;p&gt;$$
\text{graph compiler}
\rightarrow
\text{pattern / cost model}
\rightarrow
\text{kernel family}
\rightarrow
\text{autotuning}
\rightarrow
\text{CuTe/Triton/CUDA implementation}.
$$&lt;/p&gt;
&lt;p&gt;Tile selection, pipeline design, epilogue construction and architecture-specific specialisation – once largely embedded in the tacit knowledge of individual CUDA experts – are increasingly becoming parts of a parameter space that compilers can search and invoke.&lt;/p&gt;
&lt;p&gt;This does not eliminate low-level optimisation. It changes who owns it.&lt;/p&gt;
&lt;p&gt;From this perspective, the question of whether specialised HPC work remains worthwhile can no longer be reduced to:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Can a particular kernel be made faster than the library implementation?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The more important question is now:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Once the current compiler stack and public kernel libraries have already captured most conventional optimisations, which structures in the real workload remain poorly represented by the public abstractions?&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Those structures may concern shape, but they may equally concern layout, precision, dynamic behaviour, sparsity, communication topology, or the execution order of the model as a whole.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;2-the-strong-baseline-what-modern-compilers-and-high-performance-libraries-already-solve&quot;&gt;2. The Strong Baseline: What Modern Compilers and High-Performance Libraries Already Solve&lt;/h2&gt;
&lt;h3 id=&quot;21-the-value-of-customisation-depends-first-on-the-baseline&quot;&gt;2.1 The Value of Customisation Depends First on the Baseline&lt;/h3&gt;
&lt;p&gt;Before deciding whether in-house optimisation is worthwhile, the baseline must be defined correctly.&lt;/p&gt;
&lt;p&gt;A common but misleading benchmark compares a hand-tuned kernel against an untuned framework default. The observed performance gap may be real, yet it does not answer whether maintaining a private implementation is justified. A significant part of that gap may disappear simply by enabling the correct library, increasing workspace, caching algorithm-selection results, enabling fusion, or adopting an existing high-performance backend.&lt;/p&gt;
&lt;p&gt;A more useful hierarchy is:&lt;/p&gt;
&lt;p&gt;$$
\begin{aligned}
B_0 &amp;amp;= \text{current production implementation}, \\
B_1 &amp;amp;= \text{best deployable public implementation under production constraints}, \\
B_2 &amp;amp;= \text{implementation with private specialisation added on top of }B_1.
\end{aligned}
$$&lt;/p&gt;
&lt;p&gt;The quantity that matters for an investment decision is $\Delta_{\text{custom}} = B_2-B_1,$ not $B_2-B_0.$&lt;/p&gt;
&lt;p&gt;‘Best public implementation’ does not mean adopting experimental software without constraint. It remains bounded by CUDA and driver versions, AOT requirements, model-quality validation cycles, licensing, deployment stability and release cadence. However, if a team has merely failed to enable an already available high-performance backend, it is misleading to attribute the resulting gap to the long-term value of proprietary HPC capability.&lt;/p&gt;
&lt;h3 id=&quot;22-public-libraries-are-no-longer-just-collections-of-fixed-kernels&quot;&gt;2.2 Public Libraries Are No Longer Just Collections of Fixed Kernels&lt;/h3&gt;
&lt;p&gt;cuBLASLt is a useful example. It is not a single GEMM implementation, but an algorithm space plus a selection interface. Current documentation explicitly describes heuristic selection based on matrix shape and GPU configuration, and notes that a single heuristic query may itself consume tens of microseconds of host time; for stable workloads, results should therefore be cached and reused.[6]&lt;/p&gt;
&lt;p&gt;TorchInductor’s CuTeDSL backend follows a similar principle. It can enumerate a large set of valid kernel configurations, use &lt;code&gt;nvMatmulHeuristics&lt;/code&gt; to reduce that set, and then benchmark a small number of top-ranked candidates against ATen and Triton implementations on real hardware. This already introduces three layers of optimisation:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;kernel family;&lt;/li&gt;
&lt;li&gt;performance model;&lt;/li&gt;
&lt;li&gt;hardware autotuning.[3][5]&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;For standard GEMMs, it is therefore still possible that a library or compiler fails to select the absolute fastest implementation, but it is no longer reasonable to assume this as the default case.&lt;/p&gt;
&lt;p&gt;More importantly, ‘public capability’ no longer means only vendor libraries. FlashInfer, FlashAttention and DeepGEMM began as highly specialised systems, but once such implementations are integrated into vLLM, SGLang, TensorRT-LLM or PyTorch, they should be treated as part of the public baseline rather than counted as value that an internal HPC team can newly create.&lt;/p&gt;
&lt;p&gt;The first principle of modern inference optimisation is therefore not to write a kernel quickly, but to establish accurately where $B_1$ actually lies.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;3-remaining-headroom-in-dense-transformers-gemm-fusion-and-low-batch&quot;&gt;3. Remaining Headroom in Dense Transformers: GEMM, Fusion and Low Batch&lt;/h2&gt;
&lt;h3 id=&quot;31-standard-gemm-the-headroom-has-not-vanished-but-the-competition-is-strong&quot;&gt;3.1 Standard GEMM: The Headroom Has Not Vanished, but the Competition Is Strong&lt;/h3&gt;
&lt;p&gt;Transformer linear layers reduce largely to GEMM. For common dtypes, standard layouts and regular large matrices, this is also one of the most mature areas of vendor-library optimisation.&lt;/p&gt;
&lt;p&gt;CUDA-L2, published in 2025, provides a useful counterexample to the claim that ‘mature GEMM leaves no room for improvement’. On an A100 over 1,000 HGEMM configurations, the paper reports an average 11.4% improvement in offline mode over cuBLASLt results selected by autotuning from as many as 100 candidates; in a server mode intended to simulate intermittent inference calls, it reports 15.9%.[7]&lt;/p&gt;
&lt;p&gt;This shows that the statement ‘mature GEMM has no remaining headroom’ is false. It does not, however, imply that ‘custom GEMM typically offers 10–15%’.&lt;/p&gt;
&lt;p&gt;There are at least four important constraints:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the experiments target A100;&lt;/li&gt;
&lt;li&gt;they cover HGEMM only;&lt;/li&gt;
&lt;li&gt;the shape set is defined by the paper;&lt;/li&gt;
&lt;li&gt;the results are kernel-level GEMM benchmarks, not end-to-end Transformer gains.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;More fundamentally, local GEMM gains are constrained by the hardware resource that is already acting as the bottleneck.&lt;/p&gt;
&lt;p&gt;If a GEMM already achieves a fraction $u$ of the sustainable limit of the relevant bottleneck resource, and a new implementation neither reduces arithmetic work nor data movement nor changes the limiting resource, then the theoretical speed-up available purely from higher utilisation is bounded by $S_{\max}\leq\frac{1}{u}.$&lt;/p&gt;
&lt;p&gt;For example, if $u=0.9$, the upper bound is only about $1.11\times.$&lt;/p&gt;
&lt;p&gt;The 90% figure here is purely illustrative; it is not a measured claim about cuBLASLt or CUTLASS.&lt;/p&gt;
&lt;p&gt;Optimisations that exceed this bound generally do so by changing the problem itself rather than by merely executing the same GEMM more efficiently:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;reducing numerical precision;&lt;/li&gt;
&lt;li&gt;changing layout;&lt;/li&gt;
&lt;li&gt;folding epilogues into GEMM;&lt;/li&gt;
&lt;li&gt;eliminating intermediate tensors;&lt;/li&gt;
&lt;li&gt;keeping data on-chip for subsequent computation;&lt;/li&gt;
&lt;li&gt;adopting a quantisation format that reduces HBM traffic.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is precisely why the centre of gravity in GEMM optimisation has shifted.&lt;/p&gt;
&lt;h3 id=&quot;32-the-operations-around-gemm-are-becoming-the-new-optimisation-centre&quot;&gt;3.2 The Operations Around GEMM Are Becoming the New Optimisation Centre&lt;/h3&gt;
&lt;p&gt;In a highly optimised Transformer block, matrix multiplication may already be close to hardware limits, while normalisation, activation, residual updates, scaling, quantisation and reduction continue to write intermediate values to global memory only for the next kernel to read them again.&lt;/p&gt;
&lt;p&gt;CODA formalised this observation in 2026. It fixes a high-performance GEMM main loop and rewrites normalisation, activation and reduction as GEMM epilogue programs, allowing them to execute while the output tile is still resident on-chip.[8]&lt;/p&gt;
&lt;p&gt;This reveals a more important fact than the familiar statement that ‘fusion reduces kernel launches’:&lt;/p&gt;
&lt;p&gt;$\text{the main value of fusion} \neq \text{fewer API calls},$ and is better understood as fewer ‘HBM round trips + longer on-chip data lifetime + elimination of unnecessary materialisation’.&lt;/p&gt;
&lt;p&gt;The Transformer block can therefore become faster even if the GEMM itself does not.&lt;/p&gt;
&lt;p&gt;These optimisations are also being absorbed into compilers. The NVGEMM/CuTeDSL path in PyTorch 2.14 supports unified autotuning across epilogue fusion, scaled GEMM, NVFP4 GEMM and grouped-reduction epilogues.[3]&lt;/p&gt;
&lt;p&gt;This suggests an important engineering trend: successful ‘specialised kernel techniques’ do not necessarily remain private implementations for long. Once the pattern is sufficiently general, it tends to be re-expressed as a template, epilogue DSL, pattern rewrite or backend capability, and eventually becomes part of the next public baseline.&lt;/p&gt;
&lt;h3 id=&quot;33-why-low-batch-decode-still-encourages-specialisation&quot;&gt;3.3 Why Low-Batch Decode Still Encourages Specialisation&lt;/h3&gt;
&lt;p&gt;Autoregressive LLM decoding has different computational characteristics from large-batch training or prefill.&lt;/p&gt;
&lt;p&gt;Consider a linear layer $Y_{M\times N}=X_{M\times K}W_{K\times N},$ it performs approximately $2MNK$ floating-point operations. If weight reads dominate HBM traffic during low-batch decode, and each weight occupies $q$ bytes, a full weight scan costs roughly $qKN$ bytes.&lt;/p&gt;
&lt;p&gt;Ignoring other traffic, the arithmetic intensity is then approximately $I \approx \frac{2MNK}{qKN} = \frac{2M}{q}$ FLOP/byte.&lt;/p&gt;
&lt;p&gt;For BF16 or FP16 weights, $q=2$, giving $I\approx M.$&lt;/p&gt;
&lt;p&gt;At one token or small effective batch, Tensor Core peak FLOPS may therefore be less relevant than weight bandwidth, kernel launch, synchronisation and other memory-bound operations.&lt;/p&gt;
&lt;p&gt;This is why batch-1 inference deserves separate treatment. FlashFormer explicitly explores whole-model kernels for low-batch Transformer inference, motivated by the rising relative importance of memory bandwidth and launch overhead in this regime.[9]&lt;/p&gt;
&lt;p&gt;That conclusion should not, however, be applied mechanically to every Transformer. An application-level batch size of one does not imply $M=1$ in the underlying GEMM. Vision Transformers, video Transformers and BEV Transformers may still have large $M$ because spatial or temporal token dimensions remain substantial even at a frame batch of one.&lt;/p&gt;
&lt;p&gt;The relevant object of analysis is therefore the actual matrix shape and dataflow, not the application’s batch parameter in isolation.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;4-attention-specialised-optimisation-remains-active-but-is-rapidly-becoming-platform-capability&quot;&gt;4. Attention: Specialised Optimisation Remains Active, but Is Rapidly Becoming Platform Capability&lt;/h2&gt;
&lt;h3 id=&quot;41-new-hardware-continues-to-reopen-kernel-level-headroom&quot;&gt;4.1 New Hardware Continues to Reopen Kernel-Level Headroom&lt;/h3&gt;
&lt;p&gt;Attention is perhaps the clearest domain in which to observe these changes.&lt;/p&gt;
&lt;p&gt;FlashAttention-4 redesigns the pipeline around Blackwell-specific hardware characteristics. Tensor Core throughput on Blackwell has grown faster than several other resources, so the bottleneck structure differs materially from Hopper. FA4 co-designs asynchronous MMA, tile sizes, softmax rescaling, Tensor Memory and 2-CTA MMA accordingly.&lt;/p&gt;
&lt;p&gt;In the authors’ reported B200 BF16 experiments, FA4 reaches up to 1.3× the performance of cuDNN 9.13 and up to 2.7× that of the Triton implementation used for comparison.[10]&lt;/p&gt;
&lt;p&gt;The result has two implications.&lt;/p&gt;
&lt;p&gt;First, even a mature vendor library may leave meaningful optimisation headroom when a new GPU architecture alters the balance between compute, memory and auxiliary units.&lt;/p&gt;
&lt;p&gt;Second, this class of ‘expert kernel’ no longer necessarily requires extensive traditional CUDA C++ template engineering. FlashAttention-4 is implemented entirely in CuTe DSL, and its authors also report significantly faster compilation than the conventional C++ template path.[10]&lt;/p&gt;
&lt;p&gt;The effect of the modern compiler ecosystem on expert HPC work is therefore not to eliminate expert-level optimisation, but to reduce the cost of expressing and integrating it.&lt;/p&gt;
&lt;h3 id=&quot;42-non-standard-attention-is-moving-from-rewrite-the-kernel-to-express-the-semantics&quot;&gt;4.2 Non-Standard Attention Is Moving from ‘Rewrite the Kernel’ to ‘Express the Semantics’&lt;/h3&gt;
&lt;p&gt;PyTorch FlexAttention can now translate user-defined score and mask modifications into CuTe DSL and instantiate FlashAttention-4 through JIT compilation. This allows variants such as ALiBi, sliding-window attention, document masking and soft-capping to reuse the same backend.[11]&lt;/p&gt;
&lt;p&gt;A class of attention variants that once required maintaining a separate CUDA implementation may therefore now require only an expression of the semantic difference.&lt;/p&gt;
&lt;p&gt;Specialisation has not disappeared. It has moved from &lt;em&gt;reimplementing the attention algorithm&lt;/em&gt; towards &lt;em&gt;expressing the model semantics in a form that a high-performance template can consume, and ensuring that the compiler recognises those semantics&lt;/em&gt;.&lt;/p&gt;
&lt;p&gt;This change is important because it reduces the amount of private implementation without necessarily reducing performance.&lt;/p&gt;
&lt;h3 id=&quot;43-dynamic-kv-cache-and-irregular-workloads-are-still-not-ordinary-gemm-problems&quot;&gt;4.3 Dynamic KV-Cache and Irregular Workloads Are Still Not Ordinary GEMM Problems&lt;/h3&gt;
&lt;p&gt;In production LLM serving, the difficult part of attention is often not $QK^\top$ itself.&lt;/p&gt;
&lt;p&gt;Requests have different sequence lengths; KV-cache may be paged or block-sparse; GQA and MQA alter key/value sharing; requests enter and leave batches dynamically; and long contexts create strongly uneven amounts of work between queries.&lt;/p&gt;
&lt;p&gt;FlashInfer is designed around precisely these concerns. It addresses KV-cache representation, JIT attention customisation, dynamic load balancing and the static execution constraints imposed by CUDA Graphs. In the reported experiments, relative to the compiler backend used for comparison, it achieves:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;29–69% lower inter-token latency;&lt;/li&gt;
&lt;li&gt;28–30% lower latency in long-context inference;&lt;/li&gt;
&lt;li&gt;13–17% speed-up in parallel-generation serving.[12]&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These figures should not be interpreted as the expected return from ‘writing a custom attention kernel’.&lt;/p&gt;
&lt;p&gt;FlashInfer itself has already been integrated into public serving stacks including vLLM, SGLang and MLC-Engine.[12]&lt;/p&gt;
&lt;p&gt;Once such an optimisation enters a public backend, it belongs in the new $B_1$. It should no longer be counted as expected value from newly developing a private implementation.&lt;/p&gt;
&lt;p&gt;This is one of the most common valuation errors in high-performance inference: taking the gain reported by an older paper against an older compiler baseline and treating it as the business case for rebuilding the same class of optimisation today.&lt;/p&gt;
&lt;h3 id=&quot;44-automatic-compilation-of-complex-attention-is-also-advancing&quot;&gt;4.4 Automatic Compilation of Complex Attention Is Also Advancing&lt;/h3&gt;
&lt;p&gt;If the CuTeDSL backend demonstrates how expert templates can enter a compiler, Nautilus points in a different direction: automatically discovering complex GPU transformations from a higher-level mathematical description.&lt;/p&gt;
&lt;p&gt;In 2026, Nautilus reported that its auto-scheduler could start from an algebraic description of attention and discover an implementation structure resembling FlashAttention-3. Across multiple models and configurations on GH200 and RTX 5090, the authors report up to 23% and 42% higher throughput, respectively, than the state-of-the-art compiler used for comparison, while matching or exceeding hand-written cuDNN kernels in many long-sequence cases.[13]&lt;/p&gt;
&lt;p&gt;This does not imply that manually engineered kernels are about to disappear. Nautilus remains a research system with explicit bounds on its search space, hardware coverage and operator domain.&lt;/p&gt;
&lt;p&gt;It does, however, alter the long-term judgement. Complex fusion, reduction reordering and attention tiling cannot safely be treated as domains that compilers will never reach.&lt;/p&gt;
&lt;p&gt;An investment in private kernel engineering therefore carries an additional risk: &lt;strong&gt;the half-life of any technical advantage is shrinking&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The more successfully an optimisation can be expressed as a stable pattern, template or scheduler rule, the more likely it is to migrate quickly into the public compiler stack.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;5-what-general-purpose-libraries-struggle-to-flatten-representation-sparsity-and-moe&quot;&gt;5. What General-Purpose Libraries Struggle to Flatten: Representation, Sparsity and MoE&lt;/h2&gt;
&lt;h3 id=&quot;51-in-low-precision-inference-the-hard-problem-is-the-entire-data-path&quot;&gt;5.1 In Low-Precision Inference, the Hard Problem Is the Entire Data Path&lt;/h3&gt;
&lt;p&gt;Low-precision inference is a representative case in which model-specific specialisation can retain substantial value.&lt;/p&gt;
&lt;p&gt;If only the quantised GEMM kernel is benchmarked, it is easy to report very high Tensor Core utilisation. The complete data path, however, also includes:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;activation quantisation;&lt;/li&gt;
&lt;li&gt;scale computation;&lt;/li&gt;
&lt;li&gt;scale layout;&lt;/li&gt;
&lt;li&gt;transpose;&lt;/li&gt;
&lt;li&gt;packing;&lt;/li&gt;
&lt;li&gt;output conversion.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;DeepGEMM’s interface documentation makes this explicit. Certain FP8 paths require scaling factors in a transposed layout compatible with TMA; input transposition and FP8 casting must either be performed separately by the caller or fused into a preceding kernel. The simple PyTorch utility functions supplied by the project are not intended as peak-performance implementations.[14]&lt;/p&gt;
&lt;p&gt;A meaningful performance model is therefore not $T=T_{\text{GEMM}},$ but $T=T_{\text{quant}}+T_{\text{layout}}+T_{\text{scale}}+T_{\text{GEMM}}+T_{\text{postprocess}}.$&lt;/p&gt;
&lt;p&gt;A ‘faster GEMM’ that requires an additional transpose may lose its entire kernel-level advantage once integrated into the model.&lt;/p&gt;
&lt;p&gt;Conversely, this is exactly why model-specific specialisation remains valuable:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;fixed weights can be pre-arranged offline;&lt;/li&gt;
&lt;li&gt;the producer can emit the scale layout required by the downstream GEMM;&lt;/li&gt;
&lt;li&gt;quantisation can be fused into normalisation or another upstream producer.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In such cases, the gain does not come from executing the same instruction sequence faster. It comes from not executing the original transformation at all.&lt;/p&gt;
&lt;p&gt;This kind of optimisation is harder for a general-purpose library to cover completely because it depends on the data contract between neighbouring model stages.&lt;/p&gt;
&lt;h3 id=&quot;52-sparse-attention-derives-its-value-primarily-from-doing-less-work&quot;&gt;5.2 Sparse Attention Derives Its Value Primarily from Doing Less Work&lt;/h3&gt;
&lt;p&gt;FlashMLA provides a concrete example for DeepSeek-style MLA and token-level sparse attention. Its public implementation includes dense and sparse prefill, dense and sparse decode, and FP8 KV-cache support. Its sparse FP8 KV-cache also uses a model-specific layout in which the NoPE component is quantised while the RoPE component retains higher precision where required.[15]&lt;/p&gt;
&lt;p&gt;The distinction between such an implementation and ordinary FlashAttention is no longer merely a question of which tile is better.&lt;/p&gt;
&lt;p&gt;It couples model algorithm semantics, sparse indexing, KV-cache representation, quantisation format and hardware access patterns.&lt;/p&gt;
&lt;p&gt;If sparse attention accesses only a subset of the full KV-cache, the potential gain comes first from &lt;strong&gt;reading less data&lt;/strong&gt; and &lt;strong&gt;performing less computation&lt;/strong&gt;, not from achieving a higher peak FLOPS rate for the same workload.&lt;/p&gt;
&lt;p&gt;Sparsity is not free, however. Index accesses, gather operations, irregular memory transactions, load imbalance and metadata processing all introduce new costs. Whether a specialised implementation is worthwhile can therefore only be decided from the actual sparsity pattern and sequence-length distribution.&lt;/p&gt;
&lt;p&gt;A reduction in theoretical FLOPs is not, by itself, a performance prediction.&lt;/p&gt;
&lt;h3 id=&quot;53-moe-pushes-the-optimisation-boundary-beyond-the-single-kernel&quot;&gt;5.3 MoE Pushes the Optimisation Boundary Beyond the Single Kernel&lt;/h3&gt;
&lt;p&gt;Mixture-of-Experts appears GEMM-heavy, but its actual critical path includes:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;token routing;&lt;/li&gt;
&lt;li&gt;dispatch;&lt;/li&gt;
&lt;li&gt;expert grouping;&lt;/li&gt;
&lt;li&gt;grouped GEMM;&lt;/li&gt;
&lt;li&gt;activation;&lt;/li&gt;
&lt;li&gt;combine;&lt;/li&gt;
&lt;li&gt;multi-GPU expert-parallel communication.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;DeepGEMM’s current Mega MoE path no longer treats these stages as isolated operators. It organises EP dispatch, the first FP8×FP4 linear layer, SwiGLU, the second linear layer and EP combine into a co-operative mega-kernel, while attempting to overlap NVLink communication with Tensor Core computation.[14]&lt;/p&gt;
&lt;p&gt;This introduces a problem that a conventional kernel benchmark cannot capture well:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;The fastest GEMM does not necessarily produce the fastest model.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;Suppose one GEMM occupies all SMs and therefore reaches the highest standalone TFLOPS, but in doing so prevents communication from progressing. Another implementation uses fewer SMs and appears slower in isolation, yet permits communication and computation to overlap. The second implementation may finish the entire MoE layer sooner.&lt;/p&gt;
&lt;p&gt;KTransformers, which targets CPU/GPU hybrid MoE inference, illustrates the point even more clearly: overall performance may be constrained by CPU kernels, CPU–GPU synchronisation and asynchronous scheduling rather than by GPU GEMM peak throughput.&lt;/p&gt;
&lt;p&gt;In its SOSP 2025 evaluation of large hybrid MoE scenarios, the paper reports, relative to the compared systems:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;4.62–19.74× prefill speed-up;&lt;/li&gt;
&lt;li&gt;1.25–4.09× decode speed-up.[16]&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These large factors should not be compared directly with GPU-only GEMM tuning. They arise at a different optimisation layer: the system changes data placement and heterogeneous execution.&lt;/p&gt;
&lt;p&gt;The broader conclusion is nevertheless clear:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Once individual kernels are already fast, much of the remaining performance headroom shifts into the relationships between kernels.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;hr&gt;
&lt;h2 id=&quot;6-from-operator-optimisation-to-execution-model-optimisation&quot;&gt;6. From Operator Optimisation to Execution-Model Optimisation&lt;/h2&gt;
&lt;h3 id=&quot;61-persistent-kernels-and-mega-kernels&quot;&gt;6.1 Persistent Kernels and Mega-Kernels&lt;/h3&gt;
&lt;p&gt;Conventional deep-learning runtimes typically follow a kernel-per-operator execution model $K_1\rightarrow K_2\rightarrow K_3\rightarrow\cdots$, each operator has its own grid, synchronisation boundary and global state.&lt;/p&gt;
&lt;p&gt;CUDA Graph can substantially reduce host launch overhead, but it does not automatically turn the thread blocks of multiple operators into a single, unified on-device scheduling system.&lt;/p&gt;
&lt;p&gt;Mirage Persistent Kernel (MPK) attempts to change this execution model directly. Its compiler transforms a tensor program into an SM-level task graph and schedules that graph at fine granularity inside a persistent mega-kernel. This enables:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;cross-operator software pipelining;&lt;/li&gt;
&lt;li&gt;computation overlap;&lt;/li&gt;
&lt;li&gt;multi-GPU communication overlap;&lt;/li&gt;
&lt;li&gt;finer-grained device-side scheduling.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The paper reports up to roughly 1.7× lower end-to-end latency in its LLM inference experiments.[17]&lt;/p&gt;
&lt;p&gt;The implication for the value of specialised HPC work is straightforward.&lt;/p&gt;
&lt;p&gt;If ‘specialised operator’ is defined narrowly as a single CUDA function, its remaining value is indeed shrinking.&lt;/p&gt;
&lt;p&gt;If it is understood more broadly as a hardware-level execution program specialised to a fixed model graph, there remains substantial research and engineering headroom.&lt;/p&gt;
&lt;p&gt;The latter, however, is better described as compiler/runtime co-design than as a traditional operator library.&lt;/p&gt;
&lt;h3 id=&quot;62-public-inference-frameworks-are-increasingly-specialised-kernel-integrators&quot;&gt;6.2 Public Inference Frameworks Are Increasingly Specialised-Kernel Integrators&lt;/h3&gt;
&lt;p&gt;TensorRT-LLM currently exposes specialised paths including:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;GPT attention plugins;&lt;/li&gt;
&lt;li&gt;cuBLASLt GEMM plugins;&lt;/li&gt;
&lt;li&gt;FP8 row-wise GEMM;&lt;/li&gt;
&lt;li&gt;low-latency GEMM;&lt;/li&gt;
&lt;li&gt;GEMM+SwiGLU;&lt;/li&gt;
&lt;li&gt;GEMM+AllReduce;&lt;/li&gt;
&lt;li&gt;MoE plugins;&lt;/li&gt;
&lt;li&gt;norm–quant fusion;&lt;/li&gt;
&lt;li&gt;paged KV-cache.[18]&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The boundary between a ‘general framework’ and a ‘specialised kernel stack’ is therefore increasingly blurred.&lt;/p&gt;
&lt;p&gt;A modern inference framework derives much of its value from:&lt;/p&gt;
&lt;p&gt;$$
\text{recognising the workload}
\rightarrow
\text{selecting a specialised implementation}
\rightarrow
\text{composing specialised implementations}.
$$&lt;/p&gt;
&lt;p&gt;For deployment teams, the relevant question is not whether custom kernels exist, but who owns their maintenance.&lt;/p&gt;
&lt;p&gt;If NVIDIA, PyTorch, FlashInfer or DeepSeek already maintain an implementation for the same model pattern at comparable performance, the economic case for a private copy weakens rapidly.&lt;/p&gt;
&lt;p&gt;Conversely, when a production model uses a distinctive attention mechanism, quantisation format, fixed-shape regime or unusually strict latency envelope, public libraries may have little incentive to optimise precisely for that narrow workload. Private specialisation becomes more defensible.&lt;/p&gt;
&lt;h3 id=&quot;63-a-custom-operator-can-also-become-a-new-compiler-boundary&quot;&gt;6.3 A Custom Operator Can Also Become a New Compiler Boundary&lt;/h3&gt;
&lt;p&gt;Compiler visibility introduces another cost that is easy to overlook.&lt;/p&gt;
&lt;p&gt;In PyTorch, &lt;code&gt;torch.library.custom_op()&lt;/code&gt; can expose a user-defined operator to &lt;code&gt;torch.compile&lt;/code&gt; while keeping its implementation opaque: the compiler understands the operator schema, but does not inspect or optimise through the implementation. By contrast, &lt;code&gt;torch.library.triton_op()&lt;/code&gt; can expose the Triton implementation to the compiler for further analysis.[19]&lt;/p&gt;
&lt;p&gt;This creates a perfectly plausible situation in which $T_{\text{custom kernel}} &amp;lt; T_{\text{original kernel}},$ yet $T_{\text{whole graph with custom}} &amp;gt; T_{\text{whole graph before custom}}.$&lt;/p&gt;
&lt;p&gt;Possible causes include:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;loss of cross-boundary fusion;&lt;/li&gt;
&lt;li&gt;additional temporary allocations;&lt;/li&gt;
&lt;li&gt;disruption of layout propagation;&lt;/li&gt;
&lt;li&gt;insertion of new synchronisation points.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;A 20% improvement in a kernel microbenchmark is therefore not an adequate deployment criterion.&lt;/p&gt;
&lt;p&gt;The correct unit of evaluation is the net effect of the custom code on the complete graph.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;7-calculating-the-real-engineering-return-of-specialised-hpc-optimisation&quot;&gt;7. Calculating the Real Engineering Return of Specialised HPC Optimisation&lt;/h2&gt;
&lt;h3 id=&quot;71-published-benchmarks-cannot-be-turned-directly-into-an-expected-return&quot;&gt;7.1 Published Benchmarks Cannot Be Turned Directly into an Expected Return&lt;/h3&gt;
&lt;p&gt;By 2026, published results span a very wide range:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th align=&quot;left&quot;&gt;Work&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;Representative author-reported result&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;What it actually demonstrates&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;CUDA-L2&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;A100 HGEMM, average +11.4% over cuBLASLt autotuning in offline mode&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Mature GEMM still contains configuration-level search space&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;FlashAttention4&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Up to 1.3× over cuDNN 9.13 on B200 BF16&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;New architectures can reopen kernel-level optimisation headroom&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;FlashInfer&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;29–69% lower ITL in specific serving benchmarks&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;KV-cache, dynamic scheduling and the attention runtime retain system-level headroom&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;Nautilus&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Up to +23% on GH200 and +42% on RTX 5090&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Compilers are entering complex tiled-kernel optimisation domains once dominated by manual expertise&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;MPK&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Up to roughly 1.7× lower LLM inference latency&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Cross-operator scheduling and persistent execution can break through the operator-by-operator model&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;KTransformers&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;1.25–4.09× decode speed-up in specific hybrid MoE settings&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Changing heterogeneous execution and data placement can matter far more than single-GPU kernel tuning&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The relevant sources are [7][10][12][13][16][17].&lt;/p&gt;
&lt;p&gt;These values should not be averaged.&lt;/p&gt;
&lt;p&gt;They optimise different layers and use different baselines. Some concern a single kernel, some an attention engine, some the entire GPU execution model, and some CPU/GPU placement.&lt;/p&gt;
&lt;p&gt;The only robust conclusion is:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Remaining performance headroom is not a constant. It is concentrated in workload structures that the current public stack does not yet express well.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h3 id=&quot;72-use-amdahls-law-before-starting-the-project&quot;&gt;7.2 Use Amdahl’s Law Before Starting the Project&lt;/h3&gt;
&lt;p&gt;Suppose a candidate optimisation region accounts for a fraction $p$ of the original critical path, the region becomes $s$ times faster, and additional conversion, dispatch or synchronisation introduces overhead $\epsilon$. The total speed-up is approximately $S=\frac{1}{(1-p)+p/s+\epsilon}.$ If a kernel accounts for 10% of total latency, then even an infinitely fast replacement is bounded by $S_{\max} = \frac{1}{0.9} \approx1.111.$&lt;/p&gt;
&lt;p&gt;The total model latency can fall by at most 10%.&lt;/p&gt;
&lt;p&gt;If the kernel is merely doubled in speed $S= \frac{1}{0.9+0.05} \approx1.053,$ the whole model is only about 5% faster.&lt;/p&gt;
&lt;p&gt;This calculation should be performed before kernel development begins, not after.&lt;/p&gt;
&lt;p&gt;In asynchronous execution, $p$ must represent contribution to the actual critical path. It cannot be obtained by naively summing all CUDA-kernel durations in a profiler. Two kernels that each execute for 100 $\mu s$ in parallel do not create a 200 $\mu s$ critical path.&lt;/p&gt;
&lt;p&gt;This is one reason simple profiler rankings are particularly misleading in MoE, multi-stream and communication-overlap workloads.&lt;/p&gt;
&lt;h3 id=&quot;73-serving-systems-ultimately-need-slo-goodput-not-tflops&quot;&gt;7.3 Serving Systems Ultimately Need SLO Goodput, Not TFLOPS&lt;/h3&gt;
&lt;p&gt;LLM serving has several distinct performance objectives:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;TTFT: time from request submission to first token;&lt;/li&gt;
&lt;li&gt;TPOT: average time per output token after the first;&lt;/li&gt;
&lt;li&gt;ITL: interval between streamed output tokens;&lt;/li&gt;
&lt;li&gt;end-to-end latency;&lt;/li&gt;
&lt;li&gt;request or token goodput under latency constraints.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;vLLM’s current benchmarking tools distinguish TTFT, TPOT and ITL explicitly, and can calculate goodput subject to TTFT, TPOT and end-to-end latency SLOs,[20] therefore $\text{kernel throughput} \neq \text{model throughput} \neq \text{SLO goodput}.$ A kernel may increase peak throughput but require more workspace or higher occupancy, degrading concurrent scheduling.&lt;/p&gt;
&lt;p&gt;Another optimisation may barely alter single-request latency while reducing KV-cache or temporary memory enough to support more concurrent requests, thereby increasing SLO goodput.&lt;/p&gt;
&lt;p&gt;For production deployment, the latter is often economically more important than a higher microbenchmark TFLOPS figure.&lt;/p&gt;
&lt;h3 id=&quot;74-where-custom-optimisation-gains-are-most-commonly-overstated&quot;&gt;7.4 Where Custom-Optimisation Gains Are Most Commonly Overstated&lt;/h3&gt;
&lt;h4 id=&quot;weak-baseline&quot;&gt;Weak baseline&lt;/h4&gt;
&lt;p&gt;Default PyTorch eager execution is not the same baseline as fully autotuned Inductor/CuTeDSL. Likewise, a single cuBLAS heuristic result is not the same as profiling a broad candidate set.&lt;/p&gt;
&lt;p&gt;If the gap comes from failing to enable an existing backend, it is an integration gain, not a custom-kernel gain.&lt;/p&gt;
&lt;h4 id=&quot;format-conversion-excluded-from-the-timed-region&quot;&gt;Format conversion excluded from the timed region&lt;/h4&gt;
&lt;p&gt;An FP8 GEMM benchmark that excludes quantisation, scale layout or transpose will systematically overstate model-level benefit.&lt;/p&gt;
&lt;h4 id=&quot;benchmark-shapes-differ-from-production-shape-distributions&quot;&gt;Benchmark shapes differ from production shape distributions&lt;/h4&gt;
&lt;p&gt;Real serving workloads are distributions of GEMM shapes $(x_i,w_i).$ The correct weighted execution time is $T=\sum_i w_i t_i,$&lt;br&gt;not the arithmetic mean of per-shape speed-up ratios.&lt;/p&gt;
&lt;h4 id=&quot;only-the-mean-is-considered&quot;&gt;Only the mean is considered&lt;/h4&gt;
&lt;p&gt;In dynamic batching, MoE routing and long-context workloads, P95 and P99 frequently expose load imbalance that the mean conceals.&lt;/p&gt;
&lt;h4 id=&quot;numerical-semantics-are-ignored&quot;&gt;Numerical semantics are ignored&lt;/h4&gt;
&lt;p&gt;Performance gains from FP8, FP4, approximate softmax or altered accumulation order must be compared under equivalent model-quality constraints.&lt;/p&gt;
&lt;p&gt;A performance difference between numerical formats cannot be attributed entirely to kernel engineering.&lt;/p&gt;
&lt;h4 id=&quot;the-maintenance-half-life-is-ignored&quot;&gt;The maintenance half-life is ignored&lt;/h4&gt;
&lt;p&gt;An architecture-specific trick that works extremely well on Blackwell may need redesign on the next GPU generation. Conversely, it may be absorbed quickly into CUTLASS, cuDNN or Inductor.&lt;/p&gt;
&lt;p&gt;A more realistic engineering return model is therefore:&lt;/p&gt;
&lt;p&gt;$$ROI= \frac{ \int_0^{T} V(t)\Delta P(t)\,dt}{C_{\text{dev}} + C_{\text{validation}} + C_{\text{maintenance}}},$$&lt;/p&gt;
&lt;p&gt;where $\Delta P(t)$ may decline as upstream implementations catch up.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;8-engineering-boundaries-what-is-losing-value-and-what-still-deserves-investment&quot;&gt;8. Engineering Boundaries: What Is Losing Value, and What Still Deserves Investment&lt;/h2&gt;
&lt;h3 id=&quot;81-work-that-is-losing-investment-value&quot;&gt;8.1 Work That Is Losing Investment Value&lt;/h3&gt;
&lt;p&gt;Once a strong public baseline exists, several categories are increasingly difficult to justify as long-lived proprietary advantages.&lt;/p&gt;
&lt;h4 id=&quot;reimplementing-standard-large-bf16fp16-gemms&quot;&gt;Reimplementing Standard Large BF16/FP16 GEMMs&lt;/h4&gt;
&lt;p&gt;If the shape, layout and epilogue all lie within the mainstream coverage of mature libraries, the competitor is now cuBLASLt, CUTLASS/CuTe plus an autotuning system – not a naïve CUDA implementation.&lt;/p&gt;
&lt;h4 id=&quot;isolated-hand-written-optimisation-of-common-pointwise-operators&quot;&gt;Isolated Hand-Written Optimisation of Common Pointwise Operators&lt;/h4&gt;
&lt;p&gt;These operations are usually better absorbed by compiler fusion. Introducing an additional operator boundary may even reduce whole-graph optimisation potential.&lt;/p&gt;
&lt;h4 id=&quot;standard-mhagqa-already-fully-covered-by-public-attention-backends&quot;&gt;Standard MHA/GQA Already Fully Covered by Public Attention Backends&lt;/h4&gt;
&lt;p&gt;If the target semantics can already be expressed efficiently by FlashAttention, FlexAttention, cuDNN, FlashInfer or TensorRT-LLM, maintaining another complete implementation is difficult to justify unless its performance advantage is both material and persistent.&lt;/p&gt;
&lt;h4 id=&quot;tile-tuning-for-a-single-benchmark-shape&quot;&gt;Tile Tuning for a Single Benchmark Shape&lt;/h4&gt;
&lt;p&gt;If the gain depends on an extremely narrow shape regime while production traffic spans a broader distribution, a local peak is unlikely to translate into system-level value.&lt;/p&gt;
&lt;p&gt;The key point is not that these problems cannot be optimised, but that:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;the advantage is increasingly difficult to sustain.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h3 id=&quot;82-areas-where-strong-specialised-hpc-capability-still-matters&quot;&gt;8.2 Areas Where Strong Specialised HPC Capability Still Matters&lt;/h3&gt;
&lt;h4 id=&quot;model-specific-data-layouts&quot;&gt;Model-Specific Data Layouts&lt;/h4&gt;
&lt;p&gt;Where a producer and consumer have a fixed relationship, intermediate representations can be designed directly to eliminate transpose, packing and layout conversion.&lt;/p&gt;
&lt;h4 id=&quot;new-or-irregular-attention&quot;&gt;New or Irregular Attention&lt;/h4&gt;
&lt;p&gt;This includes sparse attention, MLA, unusual KV-cache formats, non-standard positional transformations, dynamic top-k patterns and irregular page access.&lt;/p&gt;
&lt;h4 id=&quot;low-batch-strict-latency-inference&quot;&gt;Low-Batch, Strict-Latency Inference&lt;/h4&gt;
&lt;p&gt;Where launch cost, HBM traffic and small-GEMM efficiency dominate latency, whole-block or whole-model specialisation retains substantial potential.&lt;/p&gt;
&lt;h4 id=&quot;moe&quot;&gt;MoE&lt;/h4&gt;
&lt;p&gt;Performance depends simultaneously on token routing, grouped GEMM, load balancing, expert-parallel communication and resource contention. Optimising GEMM alone is rarely sufficient.&lt;/p&gt;
&lt;h4 id=&quot;low-precision-execution-pipelines&quot;&gt;Low-Precision Execution Pipelines&lt;/h4&gt;
&lt;p&gt;The important object is not an isolated FP8 or FP4 GEMM, but the complete quantise–layout–GEMM–dequantise/epilogue path.&lt;/p&gt;
&lt;h4 id=&quot;cross-operator-and-cross-gpu-fusion&quot;&gt;Cross-Operator and Cross-GPU Fusion&lt;/h4&gt;
&lt;p&gt;When the objective is to remove synchronisation, overlap communication or construct persistent execution, the traditional operator abstraction itself may become the limiting factor.&lt;/p&gt;
&lt;p&gt;These problems share a common property that &lt;strong&gt;the specialised implementation changes the problem being executed&lt;/strong&gt;, rather than merely &lt;strong&gt;executing the same problem faster&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;The latter is increasingly easy for the public compiler stack to catch up with. The former still requires joint reasoning about model semantics, runtime behaviour and hardware.&lt;/p&gt;
&lt;h3 id=&quot;83-a-more-rational-layering-of-performance-engineering&quot;&gt;8.3 A More Rational Layering of Performance Engineering&lt;/h3&gt;
&lt;p&gt;In a modern AI compiler stack, performance engineering can be organised into four layers of increasing cost.&lt;/p&gt;
&lt;h4 id=&quot;layer-1-use-the-public-stack-correctly&quot;&gt;Layer 1: Use the Public Stack Correctly&lt;/h4&gt;
&lt;p&gt;This includes correct backend dispatch, autotuning, workspace settings, CUDA Graphs, existing fused operators, numerical precision and layout.&lt;/p&gt;
&lt;p&gt;There is little reason to create private kernels at this layer.&lt;/p&gt;
&lt;h4 id=&quot;layer-2-specialise-the-graph-and-dataflow&quot;&gt;Layer 2: Specialise the Graph and Dataflow&lt;/h4&gt;
&lt;p&gt;This includes pattern rewrites, layout propagation, offline weight transformation, producer–consumer fusion and better composition of existing high-performance kernels.&lt;/p&gt;
&lt;p&gt;This layer often has a particularly attractive maintenance-to-benefit ratio.&lt;/p&gt;
&lt;h4 id=&quot;layer-3-develop-local-specialised-kernels&quot;&gt;Layer 3: Develop Local Specialised Kernels&lt;/h4&gt;
&lt;p&gt;Only where profiling and modelling reveal a genuine gap should specialised CuTe, Triton or CUDA implementations be added for attention, quantisation, small GEMM, sparse or grouped workloads, with a public fallback retained.&lt;/p&gt;
&lt;h4 id=&quot;layer-4-change-the-execution-model&quot;&gt;Layer 4: Change the Execution Model&lt;/h4&gt;
&lt;p&gt;Persistent kernels, mega-kernels, communication–computation fusion, SM partitioning and heterogeneous scheduling belong here.&lt;/p&gt;
&lt;p&gt;This layer may offer the largest system-level gains, but it also carries the highest verification cost and strongest coupling to hardware and runtime details.&lt;/p&gt;
&lt;p&gt;A mature team does not need to begin at Layer 4 in order to demonstrate technical sophistication.&lt;/p&gt;
&lt;p&gt;A healthier process is to progress from Layer 1 downwards until the expected marginal gain no longer justifies the maintenance burden.&lt;/p&gt;
&lt;h3 id=&quot;84-a-quantitative-answer-to-how-much-value-is-left&quot;&gt;8.4 A Quantitative Answer to ‘How Much Value Is Left?’&lt;/h3&gt;
&lt;p&gt;If the entire discussion must be compressed into an engineering judgement, the following is defensible.&lt;/p&gt;
&lt;p&gt;For &lt;strong&gt;standard dense Transformers with common shapes and dtypes, already running on a current, high-quality autotuned backend&lt;/strong&gt;, it is no longer reasonable to assume that rewriting ordinary GEMMs by hand will yield large double-digit end-to-end gains. Published work shows kernel-level improvements above 10% in specific cases, but after Amdahl scaling, the model-level impact depends entirely on the share of the critical path occupied by that region.[7]&lt;/p&gt;
&lt;p&gt;For &lt;strong&gt;attention and other complex single operators&lt;/strong&gt;, new algorithms, GPU architectures or non-standard semantics can still produce differences of tens of percentage points at kernel level. FlashAttention-4 demonstrates that the headroom is real on Blackwell; the evolution of FlexAttention shows that a substantial portion of such specialisation can also be absorbed into general interfaces over time.[10][11]&lt;/p&gt;
&lt;p&gt;For &lt;strong&gt;dynamic KV-cache access, low batch, sparse execution, specialised quantisation formats and irregular MoE&lt;/strong&gt;, the remaining headroom often comes not from arithmetic micro-optimisation but from reducing data movement and unnecessary work. In such cases, double-digit end-to-end improvements remain entirely plausible.[12][14][15]&lt;/p&gt;
&lt;p&gt;For &lt;strong&gt;cross-operator, cross-communication-boundary or execution-model-changing optimisation&lt;/strong&gt;, published research still reports system-level gaps approaching 1.5× or more. At that point, however, the work is no longer well described as ‘writing a faster operator’. It is compiler/runtime/hardware co-design.[16][17]&lt;/p&gt;
&lt;p&gt;What is declining, therefore, is not HPC performance engineering itself, but this development model:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;maintaining a long-lived private fork of a standard operator in order to remain a few percentage points ahead of a mature public implementation.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;What is becoming more valuable is a different capability:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;identifying model structure, dataflow and execution constraints that the current compiler stack does not yet express well, and converting those structures into less data movement, less synchronisation, less redundant computation, or better communication–computation overlap.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;The long-term trend can therefore be described more accurately as $\text{micro-kernel specialisation}\;\longrightarrow\;\text{workload specialisation}$&lt;/p&gt;
&lt;p&gt;The capability worth accumulating is no longer the ability to maintain as many private CUDA kernels as possible, but the ability to construct a complete performance-reasoning chain:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;workload characterisation
  -&amp;gt; roofline / critical-path analysis
  -&amp;gt; strong public baseline
  -&amp;gt; specialisation hypothesis
  -&amp;gt; kernel or dataflow implementation
  -&amp;gt; end-to-end SLO verification
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Within this chain, CuTe, CUTLASS, cuBLASLt and the compiler are not competitors to specialised performance engineering. They are its infrastructure.&lt;/p&gt;
&lt;p&gt;The mistake to avoid is equating &lt;strong&gt;being able to write a faster kernel&lt;/strong&gt; with &lt;strong&gt;the system ought to own a private kernel&lt;/strong&gt;. The difference between those two statements is where the real value of modern Transformer inference engineering now lies.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;references&quot;&gt;References&lt;/h2&gt;
&lt;p&gt;[1] NVIDIA. &lt;em&gt;CUTLASS Documentation: CUTLASS 4.8.0 Changelog&lt;/em&gt; [EB/OL]. 2026-08-25.&lt;/p&gt;
&lt;p&gt;[2] Apache TVM. &lt;em&gt;Dataflow Pattern Language (DPL); Operator Fusion&lt;/em&gt; [EB/OL]. Apache TVM Documentation.&lt;/p&gt;
&lt;p&gt;[3] PATEL N, LAZOS M, GUESSOUS D, et al. &lt;em&gt;Generating State-of-the-Art GEMMs with TorchInductor’s CuteDSL Backend&lt;/em&gt; [EB/OL]. PyTorch Blog, 2026-04-07.&lt;/p&gt;
&lt;p&gt;[4] PyTorch Foundation. &lt;em&gt;PyTorch 2.14 Release Blog&lt;/em&gt; [EB/OL]. 2026-09-02.&lt;/p&gt;
&lt;p&gt;[5] NVIDIA. &lt;em&gt;NVIDIA Matmul Heuristics; CUTLASS Operator API: Heuristics&lt;/em&gt; [EB/OL]. NVIDIA Documentation, 2026.&lt;/p&gt;
&lt;p&gt;[6] NVIDIA. &lt;em&gt;cuBLAS 13.4 Documentation: cuBLASLt Matmul and Heuristics&lt;/em&gt; [EB/OL]. CUDA Toolkit Documentation, 2026.&lt;/p&gt;
&lt;p&gt;[7] SU S, SUN X, LI X, et al. &lt;em&gt;CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning&lt;/em&gt; [J/OL]. arXiv:2512.02551, 2025.&lt;/p&gt;
&lt;p&gt;[8] GUO H, ZHANG J, MENON A, et al. &lt;em&gt;CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs&lt;/em&gt; [J/OL]. arXiv:2605.19269, 2026. DOI: 10.48550/arXiv.2605.19269.&lt;/p&gt;
&lt;p&gt;[9] NRUSIMHA A, BRANDON W, MISHRA M, et al. &lt;em&gt;FlashFormer: Whole-Model Kernels for Efficient Low-Batch Inference&lt;/em&gt; [J/OL]. arXiv:2505.22758, 2025. DOI: 10.48550/arXiv.2505.22758.&lt;/p&gt;
&lt;p&gt;[10] ZADOURI T, HOEHNERBACH M, SHAH J, et al. &lt;em&gt;FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling&lt;/em&gt; [J/OL]. arXiv:2603.05451, 2026.&lt;/p&gt;
&lt;p&gt;[11] GUESSOUS D, STERN R, HOEHNERBACH M, et al. &lt;em&gt;FlexAttention + FlashAttention-4: Fast and Flexible&lt;/em&gt; [EB/OL]. PyTorch Blog, 2026-03-05.&lt;/p&gt;
&lt;p&gt;[12] YE Z, CHEN L, LAI R, et al. &lt;em&gt;FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving&lt;/em&gt; [J/OL]. arXiv:2501.01005, 2025.&lt;/p&gt;
&lt;p&gt;[13] ZHAO Y, YANG Y, BUDIU M, MISAILOVIC S. &lt;em&gt;Nautilus: An Auto-Scheduling Tensor Compiler for Efficient Tiled GPU Kernels&lt;/em&gt; [J/OL]. arXiv:2604.14825, 2026.&lt;/p&gt;
&lt;p&gt;[14] DeepSeek-AI. &lt;em&gt;DeepGEMM: High-Performance Tensor Core Kernels for Modern LLMs&lt;/em&gt; [EB/OL]. GitHub, 2025–2026.&lt;/p&gt;
&lt;p&gt;[15] DeepSeek-AI. &lt;em&gt;FlashMLA: Efficient Multi-head Latent Attention Kernels&lt;/em&gt; [EB/OL]. GitHub, 2025–2026.&lt;/p&gt;
&lt;p&gt;[16] CHEN H, XIE W, ZHANG B, et al. &lt;em&gt;KTransformers: Unleashing the Full Potential of CPU/GPU Hybrid Inference for MoE Models&lt;/em&gt; [C]//&lt;em&gt;Proceedings of the ACM SIGOPS 31st Symposium on Operating Systems Principles&lt;/em&gt;. ACM, 2025. DOI: 10.1145/3731569.3764843.&lt;/p&gt;
&lt;p&gt;[17] CHENG X, ZHANG Z, ZHOU Y, et al. &lt;em&gt;Mirage Persistent Kernel: A Compiler and Runtime for Mega-Kernelizing Tensor Programs&lt;/em&gt; [J/OL]. arXiv:2512.22219, 2025.&lt;/p&gt;
&lt;p&gt;[18] NVIDIA. &lt;em&gt;TensorRT-LLM Plugin Configuration and Performance Tuning Documentation&lt;/em&gt; [EB/OL]. TensorRT-LLM Documentation, 2026.&lt;/p&gt;
&lt;p&gt;[19] PyTorch. &lt;em&gt;torch.library; Using User-Defined Triton Kernels with torch.compile&lt;/em&gt; [EB/OL]. PyTorch Documentation, 2026.&lt;/p&gt;
&lt;p&gt;[20] vLLM Project. &lt;em&gt;Benchmark CLI: TTFT, TPOT, ITL and Goodput&lt;/em&gt; [EB/OL]. vLLM Documentation, 2026.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Building a Dual-Stack WireGuard Full-Tunnel Gateway with Dynamic Multi-Network Routing</title>
    <link href="https://www.ailven.cc/posts/devops-260719a.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-260719a.html</id>
    <published>2026-07-19T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A complete Hetzner and macOS deployment guide for dual-stack WireGuard egress, private recursive DNS, Apple On-Demand connections, and dynamic routing between an office network, Feilian, and a home LAN.</summary>
    <content type="html">&lt;p&gt;A personal VPN gateway becomes more complicated as soon as it must do more than carry ordinary Internet traffic. In this deployment, every personal device needs a stable Finnish IPv4 and IPv6 identity, DNS must remain inside the tunnel, and a corporate MacBook must still reach private services through either the office network or Feilian without disrupting its home LAN.&lt;/p&gt;
&lt;p&gt;This article documents the resulting design on a Hetzner server running Ubuntu 26.04 LTS. It covers the server, Apple clients, recursive DNS, split DNS, and the three-state macOS routing policy that ties the system together.&lt;/p&gt;
&lt;h2 id=&quot;designing-the-gateway&quot;&gt;Designing the gateway&lt;/h2&gt;
&lt;p&gt;The deployment starts from a set of required behaviours rather than from a chosen topology. The architecture and the address plan follow from those requirements, not the other way round.&lt;/p&gt;
&lt;h3 id=&quot;objectives&quot;&gt;Objectives&lt;/h3&gt;
&lt;p&gt;The gateway must provide the following behaviour:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;All IPv4, IPv6, and DNS traffic from an iPhone, personal MacBook, and Mac mini passes through the Hetzner server.&lt;/li&gt;
&lt;li&gt;Every personal device presents the same stable Finnish egress addresses to the public Internet.&lt;/li&gt;
&lt;li&gt;The server runs a local recursive resolver, preventing client DNS traffic from escaping the tunnel.&lt;/li&gt;
&lt;li&gt;Apple devices use system-level On-Demand VPN rules for automatic start-up, background operation, and reconnection after a network change.&lt;/li&gt;
&lt;li&gt;A corporate MacBook can run the Hetzner full tunnel while continuing to reach the corporate network.&lt;/li&gt;
&lt;li&gt;In the office, corporate traffic goes directly through the physical gateway.&lt;/li&gt;
&lt;li&gt;At home, when Feilian is connected, corporate traffic uses the Feilian tunnel.&lt;/li&gt;
&lt;li&gt;At home without Feilian, no stale corporate routes remain.&lt;/li&gt;
&lt;li&gt;Devices on the home LAN continue to communicate directly rather than through Hetzner or Feilian.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The design therefore has three paths:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Public traffic:
Apple client
    --&amp;gt; WireGuard full tunnel
    --&amp;gt; Hetzner Finland
    --&amp;gt; Internet

Corporate traffic in the office:
Corporate MacBook
    --&amp;gt; en0
    --&amp;gt; corporate gateway
    --&amp;gt; corporate network

Corporate traffic at home with Feilian:
Corporate MacBook
    --&amp;gt; Feilian utunN
    --&amp;gt; corporate network
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;architecture&quot;&gt;Architecture&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;                              Internet
                                  ^
                                  |
                  +---------------+----------------+
                  | Hetzner Finland server         |
                  | Ubuntu 26.04 LTS               |
                  |                                |
                  | Public IPv4                    |
                  | 204.168.142.215                |
                  |                                |
                  | Public IPv6                    |
                  | 2a01:4f9:c014:e6d2::1          |
                  |                                |
                  | wg0                            |
                  | 10.8.0.1/24                    |
                  | fd42:8::1/64                   |
                  |                                |
                  | NAT44 + NAT66                  |
                  | Unbound DNS                    |
                  +---------------+----------------+
                                  |
                    WireGuard IPv4/IPv6 full tunnel
                                  |
             +--------------------+--------------------+
             |                    |                    |
      iPhone Personal      MacBook Personal     Mac mini Personal
      10.8.0.3             10.8.0.4             10.8.0.5
      fd42:8::3            fd42:8::4            fd42:8::5

                                  |
                           MacBook NIO
                           10.8.0.2
                           fd42:8::2
                                  |
               +------------------+-------------------+
               |                  |                   |
         Public traffic      Office network     Home + Feilian
         --&amp;gt; Hetzner WG      --&amp;gt; en0 gateway    --&amp;gt; Feilian utunN
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;address-plan&quot;&gt;Address plan&lt;/h3&gt;
&lt;p&gt;The WireGuard network uses these internal ranges:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;IPv4 subnet: 10.8.0.0/24
IPv6 subnet: fd42:8::/64
&lt;/code&gt;&lt;/pre&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Device&lt;/th&gt;
&lt;th&gt;WireGuard IPv4&lt;/th&gt;
&lt;th&gt;WireGuard IPv6&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Hetzner server&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.8.0.1/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;fd42:8::1/64&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MacBook NIO&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.8.0.2/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;fd42:8::2/64&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;iPhone Personal&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.8.0.3/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;fd42:8::3/64&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MacBook Personal&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.8.0.4/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;fd42:8::4/64&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mac mini Personal&lt;/td&gt;
&lt;td&gt;&lt;code&gt;10.8.0.5/24&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;fd42:8::5/64&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Each server-side peer receives exact host routes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;10.8.0.X/32
fd42:8::X/128
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Every device needs its own private key, public key, IPv4 address, IPv6 address, and server-side peer entry. Never share a WireGuard key pair or an internal address between devices.&lt;/p&gt;
&lt;h2 id=&quot;building-the-server&quot;&gt;Building the server&lt;/h2&gt;
&lt;p&gt;The server side is built in a single pass, from a bare Ubuntu install to a running tunnel with its own recursive resolver.&lt;/p&gt;
&lt;h3 id=&quot;server-environment&quot;&gt;Server environment&lt;/h3&gt;
&lt;p&gt;The server used here has the following properties:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Operating system: Ubuntu 26.04 LTS
Public interface: eth0
Public IPv4: 204.168.142.215
Public IPv6: 2a01:4f9:c014:e6d2::1/64
WireGuard UDP port: 51820
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Before beginning, the server should already have:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;a normal administrative user named &lt;code&gt;ailven&lt;/code&gt;, which is the only non-root user with &lt;code&gt;sudo&lt;/code&gt; access;&lt;/li&gt;
&lt;li&gt;direct root SSH login disabled;&lt;/li&gt;
&lt;li&gt;SSH key authentication enabled;&lt;/li&gt;
&lt;li&gt;UDP port 51820 allowed by the Hetzner firewall or the host firewall; and&lt;/li&gt;
&lt;li&gt;working public IPv4 and IPv6 connectivity.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Install the required packages:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo apt update
sudo apt install -y wireguard qrencode unbound dnsutils tcpdump
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;enable-ipv4-and-ipv6-forwarding&quot;&gt;Enable IPv4 and IPv6 forwarding&lt;/h3&gt;
&lt;p&gt;Create &lt;code&gt;/etc/sysctl.d/99-wireguard.conf&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-conf&quot;&gt;net.ipv4.ip_forward=1
net.ipv6.conf.all.forwarding=1
net.ipv6.conf.default.forwarding=1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Apply and verify the settings:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo sysctl --system

sysctl net.ipv4.ip_forward
sysctl net.ipv6.conf.all.forwarding
sysctl net.ipv6.conf.default.forwarding
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The expected values are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
net.ipv6.conf.default.forwarding = 1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;WireGuard creates an encrypted layer-three interface; it does not, by itself, make the server a router. Successful Internet access still depends on kernel forwarding, FORWARD-chain rules, IPv4 NAT, and either IPv6 NAT or native IPv6 routing.&lt;/p&gt;
&lt;h3 id=&quot;manage-wireguard-keys&quot;&gt;Manage WireGuard keys&lt;/h3&gt;
&lt;p&gt;Generate the server keys:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo -s
cd /etc/wireguard

umask 077

wg genkey | tee server_private.key |
    wg pubkey &amp;gt; server_public.key
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Generate a separate pair for every client:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;wg genkey | tee macbook_nio_private.key |
    wg pubkey &amp;gt; macbook_nio_public.key

wg genkey | tee iphone_personal_private.key |
    wg pubkey &amp;gt; iphone_personal_public.key

wg genkey | tee macbook_personal_private.key |
    wg pubkey &amp;gt; macbook_personal_public.key

wg genkey | tee macmini_personal_private.key |
    wg pubkey &amp;gt; macmini_personal_public.key
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Set restrictive permissions:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo chmod 600 /etc/wireguard/*private.key
sudo chmod 644 /etc/wireguard/*public.key
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Only &lt;code&gt;server_private.key&lt;/code&gt; is required for normal server operation. Once each client configuration has been imported, its private key should not remain on the server.&lt;/p&gt;
&lt;h3 id=&quot;configure-the-wireguard-server&quot;&gt;Configure the WireGuard server&lt;/h3&gt;
&lt;p&gt;Create &lt;code&gt;/etc/wireguard/wg0.conf&lt;/code&gt; with the following structure:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;[Interface]
Address = 10.8.0.1/24, fd42:8::1/64
ListenPort = 51820
PrivateKey = &amp;lt;SERVER_PRIVATE_KEY&amp;gt;

# IPv4 forwarding and NAT44
PostUp = iptables -A FORWARD -i wg0 -j ACCEPT
PostUp = iptables -A FORWARD -o wg0 -j ACCEPT
PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

PostDown = iptables -D FORWARD -i wg0 -j ACCEPT
PostDown = iptables -D FORWARD -o wg0 -j ACCEPT
PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE

# IPv6 forwarding and NAT66
PostUp = ip6tables -A FORWARD -i wg0 -j ACCEPT
PostUp = ip6tables -A FORWARD -o wg0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
PostUp = ip6tables -t nat -A POSTROUTING -s fd42:8::/64 -o eth0 -j MASQUERADE

PostDown = ip6tables -D FORWARD -i wg0 -j ACCEPT
PostDown = ip6tables -D FORWARD -o wg0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
PostDown = ip6tables -t nat -D POSTROUTING -s fd42:8::/64 -o eth0 -j MASQUERADE

[Peer]
# MacBook NIO
PublicKey = &amp;lt;MACBOOK_NIO_PUBLIC_KEY&amp;gt;
AllowedIPs = 10.8.0.2/32, fd42:8::2/128

[Peer]
# iPhone Personal
PublicKey = &amp;lt;IPHONE_PERSONAL_PUBLIC_KEY&amp;gt;
AllowedIPs = 10.8.0.3/32, fd42:8::3/128

[Peer]
# MacBook Personal
PublicKey = &amp;lt;MACBOOK_PERSONAL_PUBLIC_KEY&amp;gt;
AllowedIPs = 10.8.0.4/32, fd42:8::4/128

[Peer]
# Mac mini Personal
PublicKey = &amp;lt;MACMINI_PERSONAL_PUBLIC_KEY&amp;gt;
AllowedIPs = 10.8.0.5/32, fd42:8::5/128
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Use the IPv6 ULA prefix consistently in every location:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Server address:          fd42:8::1/64
Client address:          fd42:8::X/64
Server peer AllowedIPs:  fd42:8::X/128
NAT66 source:            fd42:8::/64
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Although compressed IPv6 forms can be equivalent, adopting one spelling throughout the configuration avoids subtle NAT and route-matching mistakes.&lt;/p&gt;
&lt;h3 id=&quot;start-and-manage-wireguard&quot;&gt;Start and manage WireGuard&lt;/h3&gt;
&lt;p&gt;Enable and start the interface:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo systemctl enable wg-quick@wg0
sudo systemctl start wg-quick@wg0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For a complete restart, use:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo systemctl restart wg-quick@wg0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Inspect the service and the interface:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo systemctl status wg-quick@wg0 --no-pager
sudo wg show
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;An &lt;code&gt;active (exited)&lt;/code&gt; state is normal. &lt;code&gt;wg-quick@.service&lt;/code&gt; is a oneshot unit: it creates &lt;code&gt;wg0&lt;/code&gt;, applies the WireGuard parameters and addresses, sets the MTU, executes &lt;code&gt;PostUp&lt;/code&gt;, and installs routes. The kernel then maintains the interface without a resident user-space process.&lt;/p&gt;
&lt;p&gt;Perform a full restart after changing &lt;code&gt;Address&lt;/code&gt;, &lt;code&gt;ListenPort&lt;/code&gt;, &lt;code&gt;PostUp&lt;/code&gt;, &lt;code&gt;PostDown&lt;/code&gt;, NAT rules, the MTU, or other interface-level settings.&lt;/p&gt;
&lt;h3 id=&quot;verify-the-firewall-and-nat&quot;&gt;Verify the firewall and NAT&lt;/h3&gt;
&lt;p&gt;Inspect IPv4 rules:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo iptables -S FORWARD
sudo iptables -t nat -S POSTROUTING
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;They should include at least:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;-A FORWARD -i wg0 -j ACCEPT
-A FORWARD -o wg0 -j ACCEPT
-A POSTROUTING -o eth0 -j MASQUERADE
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Inspect IPv6 rules:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo ip6tables -S FORWARD
sudo ip6tables -t nat -S POSTROUTING
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;They should include at least:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;-A FORWARD -i wg0 -j ACCEPT
-A FORWARD -o wg0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A POSTROUTING -s fd42:8::/64 -o eth0 -j MASQUERADE
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Confirm that WireGuard is listening:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo ss -lunp | grep 51820
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To observe the outer UDP exchange:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo tcpdump -ni eth0 udp port 51820
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A normal handshake commonly appears as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Client --&amp;gt; Server: UDP length 148
Server --&amp;gt; Client: UDP length 92
Client --&amp;gt; Server: UDP length 32
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;With &lt;code&gt;PersistentKeepalive = 25&lt;/code&gt;, a small keepalive datagram should appear roughly every 25 seconds.&lt;/p&gt;
&lt;h3 id=&quot;run-a-private-recursive-resolver-with-unbound&quot;&gt;Run a private recursive resolver with Unbound&lt;/h3&gt;
&lt;p&gt;Clients use the WireGuard gateway itself as their DNS server:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Client
    --&amp;gt; WireGuard
    --&amp;gt; 10.8.0.1:53
    --&amp;gt; Unbound recursive resolver
    --&amp;gt; authoritative DNS hierarchy
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This keeps DNS on the same path as public traffic, avoids dependence on the client’s local network, and leaves the resolver inaccessible from the public Internet.&lt;/p&gt;
&lt;p&gt;Create &lt;code&gt;/etc/unbound/unbound.conf.d/wireguard.conf&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-conf&quot;&gt;server:
    interface: 10.8.0.1

    access-control: 10.8.0.0/24 allow
    access-control: 0.0.0.0/0 refuse

    hide-identity: yes
    hide-version: yes

    prefetch: yes
    prefetch-key: yes

    serve-expired: yes
    serve-expired-ttl: 3600
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Validate, restart, and inspect the listener:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo unbound-checkconf
sudo systemctl restart unbound
sudo ss -lntup | grep &amp;#39;:53&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Unbound should listen on &lt;code&gt;10.8.0.1:53&lt;/code&gt; over both UDP and TCP, not on &lt;code&gt;0.0.0.0:53&lt;/code&gt; or &lt;code&gt;[::]:53&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Test it directly:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dig @10.8.0.1 google.com A +dnssec +stats
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Even when the server answers from cache, the latency observed by a remote client still includes at least one round trip to Finland.&lt;/p&gt;
&lt;h2 id=&quot;personal-devices&quot;&gt;Personal devices&lt;/h2&gt;
&lt;p&gt;With the server running, each personal device needs a profile and a mechanism for keeping the tunnel up without manual intervention.&lt;/p&gt;
&lt;h3 id=&quot;create-the-full-tunnel-client-profile&quot;&gt;Create the full-tunnel client profile&lt;/h3&gt;
&lt;p&gt;Use this template for each client:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;[Interface]
PrivateKey = &amp;lt;CLIENT_PRIVATE_KEY&amp;gt;
Address = 10.8.0.X/24, fd42:8::X/64
DNS = 10.8.0.1
MTU = 1420

[Peer]
PublicKey = &amp;lt;SERVER_PUBLIC_KEY&amp;gt;
Endpoint = 204.168.142.215:51820
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 25
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;AllowedIPs = 0.0.0.0/0, ::/0&lt;/code&gt; sends all IPv4 and IPv6 traffic through WireGuard. &lt;code&gt;DNS = 10.8.0.1&lt;/code&gt; sends name resolution through the tunnel to Unbound. &lt;code&gt;PersistentKeepalive = 25&lt;/code&gt; preserves NAT state across mobile networks, home NAT, CGNAT, locked devices, network changes, and long idle periods.&lt;/p&gt;
&lt;p&gt;Keepalive and On-Demand solve different problems:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PersistentKeepalive:
preserves an existing NAT mapping

On-Demand:
starts the VPN after a reboot, network change, or tunnel interruption
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;configure-each-personal-device&quot;&gt;Configure each personal device&lt;/h3&gt;
&lt;p&gt;The iPhone profile is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;[Interface]
PrivateKey = &amp;lt;IPHONE_PRIVATE_KEY&amp;gt;
Address = 10.8.0.3/24, fd42:8::3/64
DNS = 10.8.0.1
MTU = 1420

[Peer]
PublicKey = &amp;lt;SERVER_PUBLIC_KEY&amp;gt;
Endpoint = 204.168.142.215:51820
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 25
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If iOS reports the tunnel as connected but the server receives no UDP traffic, the log contains no &lt;code&gt;Sending handshake initiation&lt;/code&gt;, and &lt;code&gt;unsatisfied route&lt;/code&gt; appears repeatedly, delete the old tunnel and recreate it. Begin with restricted routes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;AllowedIPs = 10.8.0.0/24, fd42:8::/64
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Once the server observes a handshake, restore the full-tunnel setting:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;AllowedIPs = 0.0.0.0/0, ::/0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The personal MacBook profile is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;[Interface]
PrivateKey = &amp;lt;MACBOOK_PERSONAL_PRIVATE_KEY&amp;gt;
Address = 10.8.0.4/24, fd42:8::4/64
DNS = 10.8.0.1
MTU = 1420

[Peer]
PublicKey = &amp;lt;SERVER_PUBLIC_KEY&amp;gt;
Endpoint = 204.168.142.215:51820
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 25
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The Mac mini profile is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ini&quot;&gt;[Interface]
PrivateKey = &amp;lt;MACMINI_PERSONAL_PRIVATE_KEY&amp;gt;
Address = 10.8.0.5/24, fd42:8::5/64
DNS = 10.8.0.1
MTU = 1420

[Peer]
PublicKey = &amp;lt;SERVER_PUBLIC_KEY&amp;gt;
Endpoint = 204.168.142.215:51820
AllowedIPs = 0.0.0.0/0, ::/0
PersistentKeepalive = 25
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;enable-background-operation-on-apple-devices&quot;&gt;Enable background operation on Apple devices&lt;/h3&gt;
&lt;p&gt;WireGuard tunnels on Apple platforms are managed by the system’s Network Extension framework. The graphical WireGuard application does not need to remain in the foreground.&lt;/p&gt;
&lt;p&gt;On the iPhone, enable On-Demand activation for both Wi-Fi and cellular networks. This reconnects the tunnel after start-up, transitions between Wi-Fi and cellular, lock-screen periods, and temporary loss of connectivity.&lt;/p&gt;
&lt;p&gt;On the personal MacBook, enable On-Demand for Wi-Fi and Ethernet. Do the same on the Mac mini; Ethernet must be selected if it is the machine’s normal connection.&lt;/p&gt;
&lt;p&gt;There is no need to add the WireGuard GUI to Login Items. A login item merely opens the application and does not replace Network Extension’s automatic connection policy.&lt;/p&gt;
&lt;p&gt;Verify the macOS connection with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;scutil --nc list
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The tunnel should appear as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;(Connected) VPN (com.wireguard.macos)
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;verify-dual-stack-egress&quot;&gt;Verify dual-stack egress&lt;/h3&gt;
&lt;p&gt;Check the public IPv4 address:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;curl -4 https://api.ipify.org
echo
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The expected result is &lt;code&gt;204.168.142.215&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Check the public IPv6 address:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;curl -6 https://api6.ipify.org
echo
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The expected result is &lt;code&gt;2a01:4f9:c014:e6d2::1&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Then test the internal gateway, public IPv6 reachability, and DNS:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ping -c 5 10.8.0.1
ping6 -c 5 fd42:8::1
ping6 -c 5 2606:4700:4700::1111
dig google.com A +stats
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The DNS server reported by &lt;code&gt;dig&lt;/code&gt; should be &lt;code&gt;10.8.0.1#53&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&quot;the-corporate-macbook-and-dynamic-routing&quot;&gt;The corporate MacBook and dynamic routing&lt;/h2&gt;
&lt;p&gt;The corporate MacBook is the part a single full tunnel cannot solve, because its traffic has three different correct destinations depending on where the machine is.&lt;/p&gt;
&lt;h3 id=&quot;account-for-the-corporate-macbook&quot;&gt;Account for the corporate MacBook&lt;/h3&gt;
&lt;p&gt;The corporate MacBook uses all of the following at different times:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;the Hetzner WireGuard full tunnel;&lt;/li&gt;
&lt;li&gt;the physical office network;&lt;/li&gt;
&lt;li&gt;the home network;&lt;/li&gt;
&lt;li&gt;the Feilian corporate VPN;&lt;/li&gt;
&lt;li&gt;internal corporate DNS; and&lt;/li&gt;
&lt;li&gt;an internal SOCKS5 &lt;code&gt;ProxyCommand&lt;/code&gt;.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The Hetzner profile installs full-tunnel routes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;0.0.0.0/0 --&amp;gt; Hetzner WireGuard
::/0      --&amp;gt; Hetzner WireGuard
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Any corporate address without a more specific route will therefore be sent to Hetzner and become unreachable. The MacBook needs a routing policy that changes with its network environment.&lt;/p&gt;
&lt;h3 id=&quot;define-the-corporate-private-ranges&quot;&gt;Define the corporate private ranges&lt;/h3&gt;
&lt;p&gt;The core corporate ranges are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;10.0.0.0/8
100.64.0.0/10
172.16.0.0/12
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The physical office network may additionally require &lt;code&gt;192.168.0.0/16&lt;/code&gt;. That entire range must not be forced through Feilian at home, because home LANs commonly use subnets such as &lt;code&gt;192.168.0.0/24&lt;/code&gt;, &lt;code&gt;192.168.1.0/24&lt;/code&gt;, &lt;code&gt;192.168.31.0/24&lt;/code&gt;, or &lt;code&gt;192.168.123.0/24&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The resulting policy is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Office mode:
10/8          --&amp;gt; corporate physical gateway
100.64/10     --&amp;gt; corporate physical gateway
172.16/12     --&amp;gt; corporate physical gateway
192.168/16    --&amp;gt; corporate physical gateway

Feilian mode:
10/8          --&amp;gt; Feilian utunN
100.64/10     --&amp;gt; Feilian utunN
172.16/12     --&amp;gt; Feilian utunN

192.168/16:
not claimed wholesale by this script;
the home LAN remains local;
Feilian manages the more-specific 192.168.x routes it needs.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There is no need to manage &lt;code&gt;127.0.0.0/8&lt;/code&gt; or &lt;code&gt;169.254.0.0/16&lt;/code&gt;. The former belongs to the loopback interface and the latter is link-local; the operating system installs both routes automatically.&lt;/p&gt;
&lt;h3 id=&quot;configure-split-dns-for-internal-domains&quot;&gt;Configure split DNS for internal domains&lt;/h3&gt;
&lt;p&gt;The corporate resolvers are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;10.110.3.129
10.110.3.130
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The internal domains are &lt;code&gt;nioint.com&lt;/code&gt; and &lt;code&gt;nevint.com&lt;/code&gt;. Create the macOS resolver directory:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo mkdir -p /etc/resolver
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Create both &lt;code&gt;/etc/resolver/nioint.com&lt;/code&gt; and &lt;code&gt;/etc/resolver/nevint.com&lt;/code&gt; with this content:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-conf&quot;&gt;nameserver 10.110.3.129
nameserver 10.110.3.130
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Flush the caches:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo dscacheutil -flushcache
sudo killall -HUP mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Inspect the scoped resolvers:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;scutil --dns | grep -A6 -B2 -E &amp;#39;nioint\.com|nevint\.com&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Test resolution through the macOS system resolver:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dscacheutil -q host -a name nts-proxy.hlidc.nioint.com
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The intended split is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;nioint.com and nevint.com
    --&amp;gt; 10.110.3.129 / 10.110.3.130

All other domains
    --&amp;gt; 10.8.0.1
    --&amp;gt; Hetzner Unbound
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A plain &lt;code&gt;dig hostname&lt;/code&gt; does not necessarily reproduce macOS’s scoped-resolver selection. Prefer &lt;code&gt;dscacheutil&lt;/code&gt; and &lt;code&gt;scutil --dns&lt;/code&gt; when verifying the system path.&lt;/p&gt;
&lt;h3 id=&quot;model-routing-as-three-mutually-exclusive-states&quot;&gt;Model routing as three mutually exclusive states&lt;/h3&gt;
&lt;p&gt;The final script recognises three states.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;State 1: the office network.&lt;/strong&gt; The IPv4 address on &lt;code&gt;en0&lt;/code&gt; belongs to &lt;code&gt;10.110.32.0/20&lt;/code&gt;. Routes for &lt;code&gt;10/8&lt;/code&gt;, &lt;code&gt;100.64/10&lt;/code&gt;, &lt;code&gt;172.16/12&lt;/code&gt;, and &lt;code&gt;192.168/16&lt;/code&gt; use the current physical gateway on &lt;code&gt;en0&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;State 2: an external network with Feilian connected.&lt;/strong&gt; Feilian has installed a route to a known corporate anchor, in this case &lt;code&gt;10.137.10.37/32&lt;/code&gt;, on a &lt;code&gt;utun&lt;/code&gt; interface. Routes for &lt;code&gt;10/8&lt;/code&gt;, &lt;code&gt;100.64/10&lt;/code&gt;, and &lt;code&gt;172.16/12&lt;/code&gt; use that interface. The script deliberately does not add &lt;code&gt;192.168.0.0/16&lt;/code&gt;, thereby preserving the home LAN.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;State 3: an external network without Feilian.&lt;/strong&gt; The script removes every corporate bypass route it manages. Public traffic continues through Hetzner, whilst corporate destinations have no available corporate path. This is safer than sending private corporate traffic to the home gateway.&lt;/p&gt;
&lt;h3 id=&quot;discover-the-feilian-interface-dynamically&quot;&gt;Discover the Feilian interface dynamically&lt;/h3&gt;
&lt;p&gt;macOS does not guarantee a stable &lt;code&gt;utunN&lt;/code&gt; number. Feilian might currently be &lt;code&gt;utun6&lt;/code&gt; and WireGuard &lt;code&gt;utun7&lt;/code&gt;, but a reboot, VPN reconnection, Network Extension rebuild, another VPN, or a sleep/wake cycle can change both numbers.&lt;/p&gt;
&lt;p&gt;Never hard-code this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;FEILIAN_INTERFACE=&amp;quot;utun6&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Instead, ask macOS for the route that Feilian installs to its anchor:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;route -n get 10.137.10.37
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If the result names &lt;code&gt;interface: utun6&lt;/code&gt;, then &lt;code&gt;utun6&lt;/code&gt; is the active Feilian interface. Choose an anchor that exists only while Feilian is working, is installed by Feilian itself, remains stable, and cannot be confused with an exact route created by WireGuard or the physical network.&lt;/p&gt;
&lt;h3 id=&quot;install-the-dynamic-routing-script&quot;&gt;Install the dynamic routing script&lt;/h3&gt;
&lt;p&gt;Create &lt;code&gt;/usr/local/sbin/wireguard-bypass-routes.sh&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;#!/bin/bash
set -euo pipefail

PHYSICAL_INTERFACE=&amp;quot;en0&amp;quot;

CORPORATE_NETWORK=&amp;quot;10.110.32.0/20&amp;quot;

# A route installed by Feilian when its corporate tunnel is active.
FEILIAN_ANCHOR_IP=&amp;quot;10.137.10.37&amp;quot;

# Broad corporate ranges used on both office Wi-Fi and Feilian.
CORE_CORPORATE_NETWORKS=(
  &amp;quot;10.0.0.0/8&amp;quot;
  &amp;quot;100.64.0.0/10&amp;quot;
  &amp;quot;172.16.0.0/12&amp;quot;
)

# Used only on the corporate physical network.
# Do not install this through Feilian because it may cover the home LAN.
OFFICE_ONLY_NETWORKS=(
  &amp;quot;192.168.0.0/16&amp;quot;
)

# Known cloned host routes that may retain a stale gateway/interface.
CLONED_HOSTS=(
  &amp;quot;10.110.3.129&amp;quot;
  &amp;quot;10.110.3.130&amp;quot;
  &amp;quot;172.21.150.89&amp;quot;
)

log() {
  printf &amp;#39;%s %s\n&amp;#39; &amp;quot;$(date &amp;#39;+%Y-%m-%d %H:%M:%S&amp;#39;)&amp;quot; &amp;quot;$*&amp;quot;
}

get_interface_ip() {
  /usr/sbin/ipconfig getifaddr &amp;quot;$PHYSICAL_INTERFACE&amp;quot; 2&amp;gt;/dev/null || true
}

get_physical_gateway() {
  /sbin/route -n get -ifscope &amp;quot;$PHYSICAL_INTERFACE&amp;quot; default 2&amp;gt;/dev/null |
    /usr/bin/awk &amp;#39;/gateway:/{print $2; exit}&amp;#39;
}

ip_in_network() {
  local ip=&amp;quot;$1&amp;quot;
  local network=&amp;quot;$2&amp;quot;

  /usr/bin/python3 - &amp;quot;$ip&amp;quot; &amp;quot;$network&amp;quot; &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
import ipaddress
import sys

try:
    address = ipaddress.ip_address(sys.argv[1])
    network = ipaddress.ip_network(sys.argv[2], strict=False)
except ValueError:
    sys.exit(1)

sys.exit(0 if address in network else 1)
PY
}

get_feilian_interface() {
  local interface

  interface=&amp;quot;$(
    /sbin/route -n get &amp;quot;$FEILIAN_ANCHOR_IP&amp;quot; 2&amp;gt;/dev/null |
      /usr/bin/awk &amp;#39;/interface:/{print $2; exit}&amp;#39;
  )&amp;quot;

  case &amp;quot;$interface&amp;quot; in
    utun*)
      printf &amp;#39;%s\n&amp;#39; &amp;quot;$interface&amp;quot;
      ;;
    *)
      return 1
      ;;
  esac
}

delete_exact_network_route() {
  local network=&amp;quot;$1&amp;quot;

  /sbin/route -n delete -net &amp;quot;$network&amp;quot; &amp;gt;/dev/null 2&amp;gt;&amp;amp;1 || true
}

delete_managed_routes() {
  local network

  for network in \
    &amp;quot;${CORE_CORPORATE_NETWORKS[@]}&amp;quot; \
    &amp;quot;${OFFICE_ONLY_NETWORKS[@]}&amp;quot;
  do
    delete_exact_network_route &amp;quot;$network&amp;quot;
  done
}

clear_cloned_host_routes() {
  local host

  for host in &amp;quot;${CLONED_HOSTS[@]}&amp;quot;; do
    /sbin/route -n delete -host &amp;quot;$host&amp;quot; &amp;gt;/dev/null 2&amp;gt;&amp;amp;1 || true
  done
}

install_routes_via_gateway() {
  local gateway=&amp;quot;$1&amp;quot;
  local network

  delete_managed_routes

  for network in \
    &amp;quot;${CORE_CORPORATE_NETWORKS[@]}&amp;quot; \
    &amp;quot;${OFFICE_ONLY_NETWORKS[@]}&amp;quot;
  do
    /sbin/route -n add -net &amp;quot;$network&amp;quot; &amp;quot;$gateway&amp;quot;
  done

  clear_cloned_host_routes
}

install_routes_via_feilian() {
  local interface=&amp;quot;$1&amp;quot;
  local network

  delete_managed_routes

  for network in &amp;quot;${CORE_CORPORATE_NETWORKS[@]}&amp;quot;; do
    /sbin/route -n add -net &amp;quot;$network&amp;quot; -interface &amp;quot;$interface&amp;quot;
  done

  # Deliberately do not add 192.168.0.0/16 here:
  # - it could capture the home LAN;
  # - Feilian keeps control of its own more specific 192.168.x routes.
  clear_cloned_host_routes
}

main() {
  local physical_ip
  local physical_gateway
  local feilian_interface

  physical_ip=&amp;quot;$(get_interface_ip)&amp;quot;

  # State 1: directly connected to the corporate physical network.
  if [[ -n &amp;quot;$physical_ip&amp;quot; ]] &amp;amp;&amp;amp;
     ip_in_network &amp;quot;$physical_ip&amp;quot; &amp;quot;$CORPORATE_NETWORK&amp;quot;; then

    physical_gateway=&amp;quot;$(get_physical_gateway)&amp;quot;

    if [[ -z &amp;quot;$physical_gateway&amp;quot; ]]; then
      log &amp;quot;Corporate Wi-Fi detected, but no gateway was found on ${PHYSICAL_INTERFACE}&amp;quot; &amp;gt;&amp;amp;2
      exit 1
    fi

    install_routes_via_gateway &amp;quot;$physical_gateway&amp;quot;

    log &amp;quot;Mode=office: corporate routes installed through ${physical_gateway} on ${PHYSICAL_INTERFACE}&amp;quot;
    exit 0
  fi

  # State 2: outside the office, but Feilian has installed its anchor route.
  if feilian_interface=&amp;quot;$(get_feilian_interface)&amp;quot;; then
    install_routes_via_feilian &amp;quot;$feilian_interface&amp;quot;

    log &amp;quot;Mode=feilian: corporate routes installed through ${feilian_interface}&amp;quot;
    exit 0
  fi

  # State 3: neither corporate Wi-Fi nor Feilian is available.
  delete_managed_routes
  clear_cloned_host_routes

  log &amp;quot;Mode=external: corporate bypass routes removed&amp;quot;
}

main &amp;quot;$@&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Set the permissions, check the syntax, and run it manually:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo chmod 755 /usr/local/sbin/wireguard-bypass-routes.sh
bash -n /usr/local/sbin/wireguard-bypass-routes.sh
sudo /usr/local/sbin/wireguard-bypass-routes.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The final line should report one of these modes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Mode=office: corporate routes installed through 10.110.32.1 on en0
Mode=feilian: corporate routes installed through utun6
Mode=external: corporate bypass routes removed
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;persist-routing-with-a-launchdaemon&quot;&gt;Persist routing with a LaunchDaemon&lt;/h3&gt;
&lt;p&gt;Create &lt;code&gt;/Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-xml&quot;&gt;&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot;?&amp;gt;
&amp;lt;!DOCTYPE plist PUBLIC &amp;quot;-//Apple//DTD PLIST 1.0//EN&amp;quot;
  &amp;quot;http://www.apple.com/DTDs/PropertyList-1.0.dtd&amp;quot;&amp;gt;
&amp;lt;plist version=&amp;quot;1.0&amp;quot;&amp;gt;
&amp;lt;dict&amp;gt;
    &amp;lt;key&amp;gt;Label&amp;lt;/key&amp;gt;
    &amp;lt;string&amp;gt;com.ailven.wireguard-bypass-routes&amp;lt;/string&amp;gt;

    &amp;lt;key&amp;gt;ProgramArguments&amp;lt;/key&amp;gt;
    &amp;lt;array&amp;gt;
        &amp;lt;string&amp;gt;/usr/local/sbin/wireguard-bypass-routes.sh&amp;lt;/string&amp;gt;
    &amp;lt;/array&amp;gt;

    &amp;lt;key&amp;gt;RunAtLoad&amp;lt;/key&amp;gt;
    &amp;lt;true/&amp;gt;

    &amp;lt;key&amp;gt;StartInterval&amp;lt;/key&amp;gt;
    &amp;lt;integer&amp;gt;60&amp;lt;/integer&amp;gt;

    &amp;lt;key&amp;gt;StandardOutPath&amp;lt;/key&amp;gt;
    &amp;lt;string&amp;gt;/var/log/wireguard-bypass-routes.log&amp;lt;/string&amp;gt;

    &amp;lt;key&amp;gt;StandardErrorPath&amp;lt;/key&amp;gt;
    &amp;lt;string&amp;gt;/var/log/wireguard-bypass-routes.err&amp;lt;/string&amp;gt;
&amp;lt;/dict&amp;gt;
&amp;lt;/plist&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Set the required ownership and permissions:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo chown root:wheel \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist

sudo chmod 644 \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Validate and load it:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;plutil -lint \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist

sudo launchctl bootstrap system \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;To reload it after an edit:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo launchctl bootout system \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist \
  2&amp;gt;/dev/null || true

sudo launchctl bootstrap system \
  /Library/LaunchDaemons/com.ailven.wireguard-bypass-routes.plist
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Force an immediate run and inspect its status:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo launchctl kickstart -k \
  system/com.ailven.wireguard-bypass-routes

sudo launchctl print \
  system/com.ailven.wireguard-bypass-routes |
grep -E &amp;#39;state =|runs =|last exit code&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For this short-lived periodic job, &lt;code&gt;state = not running&lt;/code&gt; together with &lt;code&gt;last exit code = 0&lt;/code&gt; is normal. Read its logs with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo tail -n 30 \
  /var/log/wireguard-bypass-routes.log \
  /var/log/wireguard-bypass-routes.err
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;understand-why-the-home-lan-remains-directly-reachable&quot;&gt;Understand why the home LAN remains directly reachable&lt;/h3&gt;
&lt;p&gt;Suppose the home network is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MacBook:      192.168.123.166/24
Router:       192.168.123.1
Other device: 192.168.123.20
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;From the DHCP lease and interface address, macOS installs a connected route:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;192.168.123.0/24 --&amp;gt; en0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;WireGuard installs its full-tunnel route:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;0.0.0.0/0 --&amp;gt; utunN
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Routing uses longest-prefix matching. &lt;code&gt;192.168.123.0/24&lt;/code&gt; is more specific than &lt;code&gt;0.0.0.0/0&lt;/code&gt;, so packets for &lt;code&gt;192.168.123.20&lt;/code&gt; use &lt;code&gt;en0&lt;/code&gt;, not the Hetzner tunnel:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MacBook
    --&amp;gt; ARP lookup for the destination MAC address
    --&amp;gt; home access point / layer-two network
    --&amp;gt; 192.168.123.20
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This traffic does not cross the home router at layer three, Hetzner, WireGuard, or Feilian. Verify the decision with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;route -n get 192.168.123.20
netstat -rn -f inet | grep &amp;#39;^192\.168\.123&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The result should identify destination &lt;code&gt;192.168.123.0&lt;/code&gt; and interface &lt;code&gt;en0&lt;/code&gt;. Local reachability follows from the connected route, longest-prefix matching, and layer-two communication within the same subnet.&lt;/p&gt;
&lt;h3 id=&quot;do-not-let-feilian-claim-all-of-1921680016&quot;&gt;Do not let Feilian claim all of &lt;code&gt;192.168.0.0/16&lt;/code&gt;&lt;/h3&gt;
&lt;p&gt;Even though a local &lt;code&gt;/24&lt;/code&gt; normally wins over a &lt;code&gt;/16&lt;/code&gt;, the script should not add this broad route:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;192.168.0.0/16 --&amp;gt; Feilian
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;There are several reasons:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Different homes use different &lt;code&gt;192.168.x.0/24&lt;/code&gt; networks.&lt;/li&gt;
&lt;li&gt;Hotels and mobile hotspots sometimes advertise unusual masks.&lt;/li&gt;
&lt;li&gt;Virtual machines, containers, and development tools may use &lt;code&gt;192.168.x.x&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Feilian already installs the more-specific routes it requires.&lt;/li&gt;
&lt;li&gt;The helper script should not claim private address space that does not belong to the company.&lt;/li&gt;
&lt;li&gt;Broad capture risks interfering with LAN discovery, AirPlay, file sharing, and device management.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Feilian mode therefore adds only &lt;code&gt;10/8&lt;/code&gt;, &lt;code&gt;100.64/10&lt;/code&gt;, and &lt;code&gt;172.16/12&lt;/code&gt;. Feilian remains responsible for exact corporate routes within &lt;code&gt;192.168.x&lt;/code&gt;.&lt;/p&gt;
&lt;h3 id=&quot;verify-feilian-mode&quot;&gt;Verify Feilian mode&lt;/h3&gt;
&lt;p&gt;Connect the home Wi-Fi, Hetzner WireGuard, and Feilian. Inspect the tunnel interfaces and anchor route:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ifconfig | grep &amp;#39;^utun&amp;#39;
route -n get 10.137.10.37
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The anchor route should name a &lt;code&gt;utunN&lt;/code&gt; interface. Then inspect representative destinations:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;for ip in \
  10.60.68.80 \
  10.110.3.129 \
  10.115.16.89 \
  172.21.150.89 \
  172.22.34.62 \
  192.168.123.1
do
  echo &amp;quot;=== ${ip} ===&amp;quot;
  route -n get &amp;quot;${ip}&amp;quot; |
    grep -E &amp;#39;destination:|gateway:|interface:|flags:&amp;#39;
done
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The expected choices are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;10.60.68.80      --&amp;gt; Feilian utunN
10.110.3.129     --&amp;gt; Feilian utunN
10.115.16.89     --&amp;gt; Feilian utunN
172.21.150.89    --&amp;gt; Feilian utunN
172.22.34.62     --&amp;gt; Feilian utunN
192.168.123.1    --&amp;gt; en0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Finally, exercise the actual services:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dscacheutil -q host \
  -a name nts-proxy.hlidc.nioint.com

nc -vz -w 5 \
  nts-proxy.hlidc.nioint.com 8889

ssh workshop_tvm
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;operating-the-gateway&quot;&gt;Operating the gateway&lt;/h2&gt;
&lt;p&gt;What remains is the part that outlives the build: what performance to expect, where the secrets live, and what to check when something drifts.&lt;/p&gt;
&lt;h3 id=&quot;set-realistic-performance-expectations&quot;&gt;Set realistic performance expectations&lt;/h3&gt;
&lt;p&gt;The measured round-trip time from the client to Finland was approximately 225–250 ms, with occasional excursions to about 300 ms. A representative Google HTTPS request broke down as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;DNS:         approximately 10 ms
TCP connect: approximately 250 ms
TLS:         approximately 490 ms
TTFB:        approximately 1.0 s
Total:       approximately 1.5 s
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;WireGuard itself added no significant RTT, MTU 1420 worked correctly, and DNS was not the principal bottleneck. Most latency came from the physical distance and cross-border route between China and Finland. Subscription proxy services can be faster because they may offer an East Asian ingress point, an optimised route, or an intermediate relay. WireGuard tuning cannot remove the lower bound imposed by the physical path.&lt;/p&gt;
&lt;p&gt;The value of this design lies elsewhere: dedicated egress, stable public addresses, a consistent Finnish dual-stack identity, complete DNS control, one egress point for several devices, no dependence on shared proxy nodes, and full control of both ends. It is not a low-latency international private circuit.&lt;/p&gt;
&lt;h3 id=&quot;remove-client-secrets-from-the-server&quot;&gt;Remove client secrets from the server&lt;/h3&gt;
&lt;p&gt;After importing the personal client profiles, delete their private keys from the server:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo rm -f \
  /etc/wireguard/iphone_personal_private.key \
  /etc/wireguard/macbook_personal_private.key \
  /etc/wireguard/macmini_personal_private.key
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Remove any saved client configuration files as well:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo rm -f \
  /etc/wireguard/clients/iphone_personal.conf \
  /etc/wireguard/clients/macbook_personal.conf \
  /etc/wireguard/clients/macmini_personal.conf
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Retain only what the server needs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/etc/wireguard/wg0.conf
/etc/wireguard/server_private.key
/etc/wireguard/server_public.key
/etc/wireguard/*_public.key
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Protect the sensitive files and encrypt all backups:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo chmod 600 /etc/wireguard/wg0.conf
sudo chmod 600 /etc/wireguard/server_private.key
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;perform-routine-operational-checks&quot;&gt;Perform routine operational checks&lt;/h3&gt;
&lt;p&gt;The following commands cover the principal failure points:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# WireGuard state and service
sudo wg show
sudo systemctl status wg-quick@wg0 --no-pager

# Recursive DNS
sudo systemctl status unbound --no-pager

# Listeners
sudo ss -lunp | grep 51820
sudo ss -lntup | grep &amp;#39;:53&amp;#39;

# NAT
sudo iptables -t nat -S POSTROUTING
sudo ip6tables -t nat -S POSTROUTING

# Peer handshakes and transfer counters
sudo wg show |
grep -E &amp;#39;peer:|endpoint:|latest handshake:|transfer:&amp;#39;

# Dynamic routing logs on the corporate MacBook
sudo tail -n 50 \
  /var/log/wireguard-bypass-routes.log \
  /var/log/wireguard-bypass-routes.err
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;resulting-network-behaviour&quot;&gt;Resulting network behaviour&lt;/h3&gt;
&lt;p&gt;Personal devices always behave as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;IPv4 Internet
    --&amp;gt; Hetzner WireGuard
    --&amp;gt; 204.168.142.215

IPv6 Internet
    --&amp;gt; Hetzner WireGuard
    --&amp;gt; NAT66
    --&amp;gt; 2a01:4f9:c014:e6d2::1

DNS
    --&amp;gt; 10.8.0.1
    --&amp;gt; Unbound
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On the corporate MacBook in the office:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Public IPv4/IPv6
    --&amp;gt; Hetzner WireGuard

Corporate private IPv4
    --&amp;gt; en0
    --&amp;gt; corporate physical gateway

nioint.com / nevint.com
    --&amp;gt; corporate DNS
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On the corporate MacBook at home with Feilian connected:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Public IPv4/IPv6
    --&amp;gt; Hetzner WireGuard

10/8, 100.64/10, and 172.16/12
    --&amp;gt; Feilian utunN

Corporate 192.168.x resources
    --&amp;gt; Feilian&amp;#39;s own specific routes

Home LAN
    --&amp;gt; en0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;On the corporate MacBook at home without Feilian:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Public IPv4/IPv6
    --&amp;gt; Hetzner WireGuard

Home LAN
    --&amp;gt; en0

Script-managed corporate bypass routes
    --&amp;gt; removed
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The completed system provides stable Finnish IPv4 and IPv6 egress, private recursive DNS, automatic connections across several Apple devices, direct office routing, Feilian routing at home, direct home-LAN access, dynamic VPN-interface discovery, and automatic management of all three routing states.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Deploying a Static Service to a Fresh Server: Nginx, Cloudflare, GitHub Actions, and WireGuard</title>
    <link href="https://www.ailven.cc/posts/devops-260719.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-260719.html</id>
    <published>2026-07-19T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A practical deployment pattern for a fresh Ubuntu server: atomic static-site releases owned by a deploy user, Cloudflare Full (strict), GitHub Actions over restricted SSH, and the firewall rules a WireGuard gateway must retain.</summary>
    <content type="html">&lt;p&gt;Moving a site to a new server is not a &lt;code&gt;git pull&lt;/code&gt; problem. It is a small systems-design exercise: DNS, TLS, the web server, deployment identity, firewall policy, and any pre-existing network services all need to agree.&lt;/p&gt;
&lt;p&gt;This note records a working pattern for deploying a static service to a fresh Ubuntu server. The examples use &lt;code&gt;zsh&lt;/code&gt;, nginx, Cloudflare, GitHub Actions, and a WireGuard gateway. The domain and addresses below are deliberately placeholders.&lt;/p&gt;
&lt;h2 id=&quot;1-start-with-roles-and-boundaries&quot;&gt;1. Start with roles and boundaries&lt;/h2&gt;
&lt;p&gt;Give each concern one owner:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Concern&lt;/th&gt;
&lt;th&gt;Owner&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Public HTTP/HTTPS entry point&lt;/td&gt;
&lt;td&gt;nginx&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Site files and release process&lt;/td&gt;
&lt;td&gt;&lt;code&gt;deploy&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;nginx configuration and private TLS key&lt;/td&gt;
&lt;td&gt;&lt;code&gt;root&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;DNS, edge TLS, proxying&lt;/td&gt;
&lt;td&gt;Cloudflare&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Build metadata and deployment trigger&lt;/td&gt;
&lt;td&gt;GitHub Actions&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The deployment account should own application files, not the entire machine. It does not need unrestricted &lt;code&gt;sudo&lt;/code&gt;. nginx runs as its usual unprivileged worker user and only needs read/traverse access to published files.&lt;/p&gt;
&lt;p&gt;For a server expected to host several services, a dedicated data filesystem such as &lt;code&gt;/data/www&lt;/code&gt; is entirely reasonable. The traditional &lt;code&gt;/var/www&lt;/code&gt; is a convention, not a requirement. A useful layout is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/data/www/
|-- homepage/
|   |-- repo/       # deployment checkout; never exposed by nginx
|   |-- releases/   # immutable releases, named by commit SHA
|   `-- current -&amp;gt; releases/&amp;lt;commit-sha&amp;gt;
|-- app-one/
`-- app-two/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Keeping &lt;code&gt;repo/&lt;/code&gt; outside the nginx root means &lt;code&gt;.git&lt;/code&gt; and deployment helpers are never accidentally served.&lt;/p&gt;
&lt;h2 id=&quot;2-publish-immutable-releases-instead-of-editing-the-live-tree&quot;&gt;2. Publish immutable releases instead of editing the live tree&lt;/h2&gt;
&lt;p&gt;An in-place &lt;code&gt;git reset --hard&lt;/code&gt; is simple but can briefly expose a mixture of old and new assets. A release directory plus a symlink makes publication atomic.&lt;/p&gt;
&lt;p&gt;The deployment script is a zsh program owned by &lt;code&gt;deploy&lt;/code&gt;. Its essential algorithm is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;#!/usr/bin/env zsh
emulate -LR zsh
setopt ERR_EXIT NO_UNSET PIPE_FAIL

readonly site=/data/www/homepage
readonly repo=$site/repo
readonly releases=$site/releases

git -C &amp;quot;$repo&amp;quot; fetch --prune origin master
revision=$(git -C &amp;quot;$repo&amp;quot; rev-parse origin/master)
target=&amp;quot;$releases/$revision&amp;quot;

if [[ ! -d &amp;quot;$target&amp;quot; ]]; then
  staging=&amp;quot;$releases/.${revision}.${RANDOM}.tmp&amp;quot;
  mkdir -p &amp;quot;$staging&amp;quot;
  git -C &amp;quot;$repo&amp;quot; archive &amp;quot;$revision&amp;quot; | tar -xf - -C &amp;quot;$staging&amp;quot;
  mv &amp;quot;$staging&amp;quot; &amp;quot;$target&amp;quot;
fi

next=&amp;quot;$site/.current-${revision}&amp;quot;
ln -s &amp;quot;releases/$revision&amp;quot; &amp;quot;$next&amp;quot;
mv -Tf &amp;quot;$next&amp;quot; &amp;quot;$site/current&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;mv -T&lt;/code&gt; replaces the symlink in one filesystem operation. A request therefore reaches the old complete release or the new complete release. Keeping a few old release directories also makes rollback a symlink change rather than another build.&lt;/p&gt;
&lt;p&gt;nginx should point only at the symlinked release:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-nginx&quot;&gt;root /data/www/homepage/current;
index index.html;

location / {
    try_files $uri $uri/ =404;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;3-put-https-in-place-before-changing-dns&quot;&gt;3. Put HTTPS in place before changing DNS&lt;/h2&gt;
&lt;p&gt;When Cloudflare proxies a hostname, Cloudflare handles browser-facing TLS and connects separately to the origin. The origin should still use TLS. Create a Cloudflare Origin Certificate with both the apex and first-level wildcard names:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;example.com
*.example.com
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Store the certificate and key in a root-owned directory such as &lt;code&gt;/etc/ssl/cloudflare/&lt;/code&gt;, with the private key mode &lt;code&gt;0600&lt;/code&gt;. Configure nginx on both IPv4 and IPv6, then use Cloudflare’s &lt;strong&gt;Full (strict)&lt;/strong&gt; mode. Never use Flexible mode: it leaves the Cloudflare-to-origin hop unencrypted and often causes redirect loops.&lt;/p&gt;
&lt;p&gt;Before changing the DNS record, test the new origin directly while preserving the real hostname for SNI:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;curl -kI --resolve www.example.com:443:NEW_SERVER_IPV4 https://www.example.com/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;-k&lt;/code&gt; is appropriate only for this direct test because a Cloudflare Origin Certificate is not trusted by browsers. Through Cloudflare, the browser receives Cloudflare’s publicly trusted certificate.&lt;/p&gt;
&lt;p&gt;Once that succeeds, update only the web records in Cloudflare. A typical proxied setup is:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Type&lt;/th&gt;
&lt;th&gt;Name&lt;/th&gt;
&lt;th&gt;Content&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;A&lt;/td&gt;
&lt;td&gt;&lt;code&gt;@&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;new server IPv4&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AAAA&lt;/td&gt;
&lt;td&gt;&lt;code&gt;@&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;new server IPv6&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;CNAME&lt;/td&gt;
&lt;td&gt;&lt;code&gt;www&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;apex domain&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Do not disturb MX, SPF, DKIM, DMARC, or other unrelated records. Keep the old origin running until &lt;code&gt;curl -I https://www.example.com/&lt;/code&gt; returns the expected page through Cloudflare.&lt;/p&gt;
&lt;h2 id=&quot;4-let-github-actions-build-then-ask-the-server-to-publish&quot;&gt;4. Let GitHub Actions build, then ask the server to publish&lt;/h2&gt;
&lt;p&gt;This homepage derives &lt;code&gt;posts/index.json&lt;/code&gt;, &lt;code&gt;assets/site-meta.json&lt;/code&gt;, and &lt;code&gt;sitemap.xml&lt;/code&gt; from Markdown frontmatter and Git history. The GitHub workflow must therefore generate and commit those artifacts before it asks the server to deploy.&lt;/p&gt;
&lt;p&gt;The deployment step itself is intentionally small:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-yaml&quot;&gt;- name: Deploy release
  uses: appleboy/ssh-action@v1.0.0
  with:
    host: ${{ secrets.SERVER_HOST }}
    username: ${{ secrets.SERVER_USER }}
    key: ${{ secrets.SSH_PRIVATE_KEY }}
    port: ${{ secrets.SERVER_PORT }}
    script: |
      exec zsh /data/www/homepage/deploy-homepage.zsh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The SSH host must be the server’s direct IP address, not the proxied web domain: Cloudflare does not proxy normal SSH traffic.&lt;/p&gt;
&lt;p&gt;Use a dedicated deployment key. Its public half belongs in &lt;code&gt;/home/deploy/.ssh/authorized_keys&lt;/code&gt; with a forced command and OpenSSH restrictions:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;restrict,command=&amp;quot;/data/www/homepage/deploy-homepage.zsh&amp;quot; ssh-ed25519 AAAA... github-actions-homepage
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That key cannot open an interactive shell, create a tunnel, or execute arbitrary commands. It can only run the release publisher as the unprivileged &lt;code&gt;deploy&lt;/code&gt; user.&lt;/p&gt;
&lt;p&gt;After a push, verify both sides:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;readlink -f /data/www/homepage/current
curl -I https://www.example.com/
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The symlink target should be the commit that GitHub Actions deployed, and the edge response should be successful.&lt;/p&gt;
&lt;h2 id=&quot;5-firewall-changes-must-preserve-existing-vpn-services&quot;&gt;5. Firewall changes must preserve existing VPN services&lt;/h2&gt;
&lt;p&gt;This server also acted as a WireGuard gateway on &lt;code&gt;wg0&lt;/code&gt;, with the gateway at &lt;code&gt;10.8.0.1&lt;/code&gt; and a local DNS resolver on port 53. Enabling UFW with only SSH, HTTP, and HTTPS rules silently broke the VPN:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;macOS preferred &lt;code&gt;10.8.0.1&lt;/code&gt; as its DNS resolver.&lt;/li&gt;
&lt;li&gt;UFW blocked the WireGuard UDP socket and/or DNS packets arriving on &lt;code&gt;wg0&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Browsers appeared unable to reach Google and the newly deployed website.&lt;/li&gt;
&lt;li&gt;A &lt;code&gt;curl --resolve&lt;/code&gt; request succeeded, proving that HTTPS worked and isolating the failure to DNS over the VPN.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Before enabling a firewall, inventory existing listeners and routes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;ss -lntup
ip -br address
ip route show
systemctl --type=service --state=running
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For a WireGuard gateway, preserve these categories of traffic. Replace the port with the &lt;code&gt;ListenPort&lt;/code&gt; from &lt;code&gt;wg0.conf&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;sudo ufw allow &amp;lt;WIREGUARD_PORT&amp;gt;/udp
sudo ufw allow in on wg0 to 10.8.0.1 port 53 proto udp
sudo ufw allow in on wg0 to 10.8.0.1 port 53 proto tcp
sudo ufw route allow in on wg0 out on eth0 from 10.8.0.0/24
sudo ufw reload
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The final route rule is needed in addition to &lt;code&gt;net.ipv4.ip_forward=1&lt;/code&gt;: forwarding enabled in the kernel does not override a firewall’s forward policy. If the VPN provides full internet egress, also ensure its existing NAT/MASQUERADE rule remains present.&lt;/p&gt;
&lt;p&gt;From a client, test the gateway explicitly before blaming the public DNS:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-zsh&quot;&gt;dig @10.8.0.1 www.google.com +time=2 +tries=1
dig www.example.com
curl -I https://www.example.com/
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;6-operate-the-next-service-the-same-way&quot;&gt;6. Operate the next service the same way&lt;/h2&gt;
&lt;p&gt;For each future &lt;code&gt;service.example.com&lt;/code&gt;:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Give it a separate directory under &lt;code&gt;/data/www&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Run its application process as &lt;code&gt;deploy&lt;/code&gt;, bound to &lt;code&gt;127.0.0.1&lt;/code&gt; or a Unix socket.&lt;/li&gt;
&lt;li&gt;Give nginx a dedicated &lt;code&gt;server_name&lt;/code&gt; and reverse-proxy rule.&lt;/li&gt;
&lt;li&gt;Add a proxied Cloudflare DNS record.&lt;/li&gt;
&lt;li&gt;Make origin TLS work before exposing the hostname.&lt;/li&gt;
&lt;li&gt;Add the service’s required ports, routes, and health checks to the firewall plan.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;HSTS with &lt;code&gt;includeSubDomains&lt;/code&gt; means every future subdomain must be HTTPS-ready from its first public use. This is a useful safety property, but it makes an HTTP-only test deployment under the production domain a bad idea.&lt;/p&gt;
&lt;h2 id=&quot;final-checklist&quot;&gt;Final checklist&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; nginx serves only a release directory, never the deployment checkout.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; &lt;code&gt;deploy&lt;/code&gt; owns service files but has no broad sudo access.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Cloudflare is set to Full (strict).&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; Both IPv4 and IPv6 origin paths have been tested.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; GitHub Actions uses a dedicated, restricted SSH key.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; DNS, HTTPS, and the deployed commit have been verified after a push.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; UFW rules preserve WireGuard, DNS, forwarding, and NAT behaviour.&lt;/li&gt;
&lt;li&gt;&lt;input disabled=&quot;&quot; type=&quot;checkbox&quot;&gt; The previous server remains available until rollback is no longer needed.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;With these boundaries in place, deploying another static site or introducing a reverse-proxied service becomes a repeatable operation rather than a fresh migration project.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The Machine of Innocence in Love</title>
    <link href="https://www.ailven.cc/posts/moment-260615.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260615.html</id>
    <published>2026-06-15T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>On Moral Laundering, Techniques of Withdrawal, and the Liquidation of Personhood in Modern Intimacy.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 15 June 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;i-people-calculate-and-then-insist-that-their-calculations-are-not-calculations&quot;&gt;I. People Calculate, and Then Insist That Their Calculations Are Not Calculations&lt;/h2&gt;
&lt;p&gt;The crudest critique of a failed intimate relationship is to say that one person leaves because the other has lost value; because the relationship is no longer worth the cost; because, in some brutal sense, the abandoned party is deemed no longer good enough. This judgement is not necessarily false. Yet it remains at the surface of relational economics. It identifies valuation, but not the process by which valuation is legitimised. It identifies withdrawal, but not the way withdrawal is recast as moral conduct. And it misses the more sinister fact: human beings calculate, and then insist on preserving the self-image of someone who has never calculated at all.&lt;/p&gt;
&lt;p&gt;The real object of critique lies deeper than her eventual decision that you were insufficiently valuable. That is too simple, and too close to the ordinary anger of heartbreak. The deeper question is this: when a person leaves another, by what means do they translate private interest, emotional exhaustion, class aspiration, optimisation of life-path, and avoidance of risk into a language of purity and innocence? She does not say, ‘Having recalculated the matter, I no longer find you worth the investment.’ She says, ‘I have finally learnt to love myself.’ She does not say, ‘I no longer wish to carry responsibility for your pain.’ She says, ‘I need boundaries.’ A plain ‘I want a life with greater possibilities’ becomes ‘I need to grow’; a plain ‘you have depreciated within my imagined future’ becomes ‘we are not compatible’; a plain ‘I am no longer willing to bear the costs of this relationship’ becomes ‘I no longer want to be drained’.&lt;/p&gt;
&lt;p&gt;Here lies something more elusive than mere utility: the moral laundering of utility.&lt;/p&gt;
&lt;p&gt;Naked utility is not especially frightening; at least it admits its own calculation. It admits that it is choosing according to interest, prospects, emotional return, and opportunity cost. Utility purified by contemporary language is more dangerous. It no longer appears in its crude form. It wears the polished garments of growth, freedom, independence, clarity, boundaries, ease, female subjectivity, life-stage difference, energetic mismatch, and self-realisation. Departure no longer looks like departure; breach no longer looks like breach; divestment no longer looks like divestment. Every cold decision is relocated within a narrative of spiritual progress. One no longer abandons another person; one ‘finally becomes oneself’.&lt;/p&gt;
&lt;p&gt;This is one of the bleakest mechanisms in modern intimacy: people do not merely want to win; they want to win innocently. They do not merely want to leave; they want to leave nobly. They want the deletion of someone from the future to resemble an awakening of the self.&lt;/p&gt;
&lt;p&gt;If a person were to say plainly, ‘I no longer love you because being with you no longer satisfies my expectations of the future, of class position, of emotional experience, and of the image I wish to have of myself,’ the sentence would be cruel, but at least honest. It would place the knife on the table and admit that the knife is a knife. Yet modern people rarely speak in this way. To do so would damage the image they need to retain of themselves. A person wants to pursue their own interests and to feel clean while doing so. She must believe that she is not cold, not calculating, not someone who crosses the bridge and burns it behind her. She therefore requires a machine of innocence capable of processing all concrete judgements of interest into abstract vocabularies of value.&lt;/p&gt;
&lt;p&gt;‘We are not compatible’ is one of the most commonly used components of this machine. It sounds gentle. In reality, it is a highly compressed verdict. It requires no evidence, no responsibility, no account of when the incompatibility began, where it lies, why compatibility once seemed possible, or why it is now impossible. It swallows the whole history in a single vague concept. Past promises, shared plans, acts of devotion, reliance, mutual making, all those moments once described as fate, are lightly covered over by the phrase: ‘later, we realised we were not compatible.’ The force of this phrase lies in its simultaneous accomplishment of three things: it cancels the binding force of the past, rationalises present withdrawal, and exempts the speaker from future responsibility.&lt;/p&gt;
&lt;p&gt;‘I have lost the feeling’ works in the same way. Feeling is presented as an unaccountable natural phenomenon. Love and its disappearance are made to resemble changes in the weather: inexplicable in their arrival, unanswerable in their departure. Yet feeling within a relationship has no isolated natural existence. It is continuously shaped by comparison, future expectation, external pressure, identity fantasy, willingness to communicate, and frames of interpretation. When a person begins repeatedly to interpret your flaws as fatal defects, your attempts to speak as disturbances, your devotion as burden, and your existence as depletion, feeling will of course disappear. So-called loss of feeling is often the product of a sustained reinterpretation. The feeling rarely goes first. Often, because she needs to leave, she gradually trains herself not to feel, and the devaluation follows.&lt;/p&gt;
&lt;p&gt;This does not mean that all departures are wrong. A person may of course cease to love, may choose a different life, may refuse to endure a relationship any longer. The problem is that modern intimacy has no shortage of freedom to leave. What it lacks is honesty about leaving. Too many people enjoy freedom while refusing responsibility; exercise choice while denying its cost; revise their life-path while refusing to admit that a concrete human being has been injured in the process. The abandoned party loses a person, and is compelled to accept the explanatory system supplied by the one who leaves: she has not withdrawn; she has grown. She has not changed her heart; she has awakened. Her coldness is clear-sightedness; the erasure of you is finally a form of self-respect.&lt;/p&gt;
&lt;p&gt;This is the second injury. The first injury is the departure itself. The second is the narration of that departure as correctness, and the simultaneous narration of the abandoned person’s pain as immaturity, entanglement, control, or inability to accept reality.&lt;/p&gt;
&lt;h2 id=&quot;ii-what-modern-people-excel-at-is-the-rewriting-of-betrayal-as-self-protection&quot;&gt;II. What Modern People Excel at Is the Rewriting of Betrayal as Self-Protection&lt;/h2&gt;
&lt;p&gt;Betrayal, in its traditional moral register, is too heavy a word. Modern people dislike it. It sounds old, rigid, patriarchal, accusatory, insufficiently psychological, insufficiently contemporary. In its place, a softer, more sophisticated, and harder-to-refute vocabulary has developed: I am only protecting myself; I am only setting boundaries; I am only ending depletion; I am only taking responsibility for my own life; I simply cannot continue betraying myself.&lt;/p&gt;
&lt;p&gt;None of these sentences is inherently false. In genuinely oppressive relationships, they may be entirely justified. When a person is subjected to control, humiliation, violence, exploitation, or persistent negation, they should indeed protect themselves, set boundaries, end depletion. The problem is that every powerful language of liberation, once it enters the ordinary terrain of interest-conflict, becomes available to the opportunist. The more morally unassailable a vocabulary is, the more suitable it becomes for concealing responsibility.&lt;/p&gt;
&lt;p&gt;‘Protecting myself’ may be true. It may also mean refusing responsibility for the pain one has caused. ‘Boundaries’ may be real. They may also mean the severing of all communication. ‘Taking responsibility for myself’ may be admirable. It may also mean transferring all shared commitments back into the private account of individual interest.&lt;/p&gt;
&lt;p&gt;One of the deep pathologies of modern intimacy is the psychologisation of responsibility. Older moral language would ask: Did you make a promise? Did you accept the other person’s devotion? Did you once include them in your future? Did you change the rules at a decisive point? Did you preserve their dignity while leaving? Modern psychologised language rewrites these questions: Are you comfortable? Are you nourished? Are you being depleted? Are you becoming a better version of yourself? Do you feel free?&lt;/p&gt;
&lt;p&gt;Ethical questions are thereby converted into questions of feeling. If I feel uncomfortable, I have reason to leave. If I feel depleted, I have reason to stop. If I feel that the relationship no longer nourishes me, I have reason to withdraw. The promises, histories, real debts of mutual care, and shared imaginings of the future that once belonged to the relationship lose their weight before ‘how I feel’.&lt;/p&gt;
&lt;p&gt;This conversion is exceedingly dangerous. Feeling is real, but feeling is not a supreme court. To feel uncomfortable does not automatically prove that the other person is guilty. To feel suffocated does not automatically prove that the other person is oppressing you. Feelings require interpretation, and interpretation is contaminated by interest. The error of modern people is to treat their own feeling as self-validating truth. If she feels irritated, you become disturbance. If she feels tired, you become burden. If she wants to leave, you become incompatibility.&lt;/p&gt;
&lt;p&gt;This is the despotism of feeling. It is more elusive than traditional moral despotism, precisely because it appears anti-oppressive, anti-controlling, respectful of the individual. In practice, it merely transfers judgement from external norms to immediate feeling, while immediate feeling is itself manipulated by desire, fear, comparison, boredom, opportunity, and class imagination. A person need only say, ‘I feel bad,’ and moral immunity for withdrawal is obtained. Whether that bad feeling has been caused by the other person, by one’s own change of heart, by the arrival of new opportunities, or by one’s unwillingness to bear relational cost, becomes secondary.&lt;/p&gt;
&lt;p&gt;This is also why many who leave appear so strangely calm at the point of separation. It is not that they have no emotion. It is that within their own explanatory system they have already been acquitted. She is not indebted; she is saving herself. She is not withdrawing investment; she is clear-sighted. Rewriting a commitment becomes a form of being true to herself. She need not grieve, because she has placed the other person in the category of ‘one who makes me feel bad’. Once that category is established, all injury can be explained as necessary severance.&lt;/p&gt;
&lt;p&gt;The abandoned person is then placed in a position of extreme passivity. He cannot refute her feeling. He cannot say, ‘You do not feel suffocated,’ because feeling is unfalsifiable. He cannot say, ‘You should not protect yourself,’ because that sounds oppressive. He is therefore forced to plead within the very language that has already condemned him. The more he explains, the more he seems to violate boundaries; the more he asks, the more he resembles an emotional blackmailer; the more he suffers, the more he appears to confirm the wisdom of her departure.&lt;/p&gt;
&lt;p&gt;This is the most precise mechanism of silencing in contemporary break-up narratives: the other person first uses feeling to declare you guilty, then uses boundaries to forbid your defence, and finally uses your attempt at defence as evidence of the original guilt.&lt;/p&gt;
&lt;h2 id=&quot;iii-what-the-departing-person-murders-is-not-merely-the-relationship-but-the-right-to-interpret-the-shared-past&quot;&gt;III. What the Departing Person Murders Is Not Merely the Relationship, but the Right to Interpret the Shared Past&lt;/h2&gt;
&lt;p&gt;When a relationship ends, the most basic ethical demand is an acknowledgement of history. To acknowledge that there was love; that there was reliance; that a future was once imagined together; that the other person’s devotion once genuinely supported one’s life; that the act of leaving now wounds a concrete human being. Such acknowledgement does not prevent departure, but it preserves the minimum dignity of personhood.&lt;/p&gt;
&lt;p&gt;Many people, however, are incapable of this at the moment of leaving. To acknowledge history is to accept discomfort. If one admits that there was once genuine love, present coldness appears cruel. If one admits that promises were made, present change carries the taint of breach. If one admits that the other remains worthy of respect, one cannot so easily narrate one’s departure as escape from a swamp.&lt;/p&gt;
&lt;p&gt;The departing person therefore often needs to rewrite history.&lt;/p&gt;
&lt;p&gt;There are many ways of doing this. The first is the denial of singularity: the relationship was not so special; any two people could have experienced similar sweetness. The second is the denial of love: it was not love, merely responsibility, habit, dependence, or a stage of companionship. Then come the denials of contribution, of seriousness in shared futures, and of the other’s pain: whatever you did, you did voluntarily, perhaps even to satisfy your own desire to give; what was said in the past was merely what one felt at the time, and people change; break-ups are normal, and if you cannot get over them you are simply too fragile.&lt;/p&gt;
&lt;p&gt;On the surface, these statements explain the relationship. In reality, they liquidate the other person’s narrative rights. She does not merely refuse a future with you; she prescribes how the past is to be understood. She withdraws, and rewrites the relationship into a story she can leave without weight. What you remember as devotion, she calls a phase. What you remember as promise, she calls emotion. What you remember as mutual support, she calls your own choice.&lt;/p&gt;
&lt;p&gt;The injury inflicted here is deeper than ordinary separation. A person can bear the disappearance of the future more readily than the deprivation of the past. The disappearance of the future says: there will be no ‘us’ henceforth. The deprivation of the past says: perhaps there was never truly an ‘us’ at all. The first is loss. The second is a kind of spiritual murder. It leaves the abandoned person unsure whether his memories remain valid, whether those moments of happiness were real, whether he was ever loved at all. He is then compelled to remember, to produce evidence, to write down the details again and again – not merely because he refuses to let go, but because he is defending the right of his own history to have existed.&lt;/p&gt;
&lt;p&gt;This is why, years later, a similar figure glimpsed in the street may still pierce him. On the surface, he has not let go of the person. What he has not resolved is the legitimacy of the history. So long as her interpretation retains sovereignty, so long as he waits for her to admit ‘it was real’, ‘you once mattered’, ‘I did injure you’, he remains inside the tribunal she established.&lt;/p&gt;
&lt;p&gt;The real cruelty is this: the person who leaves may no longer care about the relationship, yet the narrative structure she leaves behind may continue to govern the one abandoned. She has gone, but the bench remains. She no longer speaks, yet the judgement she once delivered continues to echo. She may have entered another life, while the one left behind continues to petition for the legitimacy of an old history.&lt;/p&gt;
&lt;p&gt;Recovery, therefore, is not the forgetting of her. It is the abolition of her monopoly over the interpretation of the shared past: no longer requiring her acknowledgement, no longer treating her version as the final text. One must become able to say: you may deny it, but your denial does not cancel my experience; you may leave, but your leaving does not rewrite the whole past; you may no longer love me, but you have no authority to declare that I was never loved; you may remove me from your future, but you cannot delete me from history.&lt;/p&gt;
&lt;p&gt;What the abandoned person must truly recover is narrative sovereignty. Love, compensation, and belated apology are lesser things.&lt;/p&gt;
&lt;h2 id=&quot;iv-the-greatest-danger-for-the-abandoned-person-is-to-mistake-trauma-for-insight-and-resentment-for-truth&quot;&gt;IV. The Greatest Danger for the Abandoned Person Is to Mistake Trauma for Insight, and Resentment for Truth&lt;/h2&gt;
&lt;p&gt;The critique of the departing person must continue inwards; otherwise it decays into refined accusation. When a person has been priced, divested from, negated, and rewritten, anger is inevitable. That anger has a factual basis. Yet anger does not automatically produce truth. Trauma may open fissures in reality; it may also generate new blindness. To see the other person’s coldness is not yet to have seen oneself.&lt;/p&gt;
&lt;p&gt;The abandoned person is especially prone to a moral illusion: because I was injured, I am closer to love; because she chose reality, I represent sincerity; because she calculated, I am pure. This self-image is seductive, and dangerous. It converts failure into nobility, abandonment into clarity, helplessness into moral superiority.&lt;/p&gt;
&lt;p&gt;But the facts may not be so simple. To hate being priced does not prove that one truly rejects the price system. It may only prove that one cannot bear having been priced low. To declare that one values life, sincerity, peace, companionship, and quiet devotion does not prove that one has rebuilt one’s values. It may only mean that, after frustration in ascent, competition, class, income, and social recognition, one has used the phrase ‘I understand life more deeply’ to preserve self-esteem.&lt;/p&gt;
&lt;p&gt;This is the harsher self-judgement: are you criticising the assetisation of human beings, or are you resenting the fact that you were no longer held as a high-value asset? Are you opposed to value exchange in intimacy, or are you in pain because you ceased to be regarded as an investment worth maintaining? Are you seeking a love unpolluted by utility, or are you yearning for someone to prove, unconditionally, that you are more valuable than the path she later chose?&lt;/p&gt;
&lt;p&gt;If these questions cannot be answered, the abandoned person’s critique remains governed by the one who left. His entire thought still revolves around her valuation. She says he is not worthy, so he spends his life proving that he is. She says he is immature, so he is desperate to prove maturity. She says he lacks value, so he uses career, income, growth, writing, depth, and suffering to prove that value exists. He appears to resist her; in fact, he has accepted her examination paper.&lt;/p&gt;
&lt;p&gt;This is merely another form of being defined by her.&lt;/p&gt;
&lt;p&gt;More than this, the abandoned person must also examine his own violence within the relationship. Devotion is not automatically innocent. Giving is not automatically innocent. Wanting to communicate is not automatically innocent. When the other person has already withdrawn, continued explanation may become pressure. When the other person no longer accepts relational obligation, repeated appeal to past devotion may become debt-collection. When the other person seeks an individual path, insistence upon the common life may become moral binding. One may have been genuinely wronged, and yet have injured the other through one’s own conduct. These truths do not cancel each other.&lt;/p&gt;
&lt;p&gt;A mature critique must be able to hold both facts at once: her mode of departure may have been cold, opportunistic, depersonalising; your mode of trying to recover the relationship may also have been pressurising, debt-laden, uncontrolled, and possessive. She used freedom to evade responsibility; you used love to exert pressure. She assetised you; you indebted her. She dissolved the common world; you attempted to force its restoration. In the end, both parties concealed their own violence beneath beautiful words: she spoke of boundaries, you spoke of devotion; she spoke of growth, you spoke of promises; she spoke of self-love, you spoke of cherishing the relationship.&lt;/p&gt;
&lt;p&gt;This must be stated coldly enough: the late stage of many relationships is no longer a struggle between love and lovelessness, but a war between two systems of self-acquittal. Each side fights to secure the right to be the injured party, the right to define reality, the right to describe their own conduct as unavoidable. After love dies, what remains is a struggle over narrative authority.&lt;/p&gt;
&lt;p&gt;For this reason, if the abandoned person is to recover in any serious sense, he must dismantle not only the other person’s machine of innocence, but his own. He must ask not only how she called betrayal growth, but how he called control love. How did she call coldness a boundary, and how did he call entanglement communication? How did she call divestment clarity, and how did he call failure nobility?&lt;/p&gt;
&lt;p&gt;Only at this point does critique acquire force. Otherwise, it is merely wounded self-esteem raised to the level of rhetoric.&lt;/p&gt;
&lt;h2 id=&quot;v-the-real-problem-is-how-modern-intimacy-trains-people-not-to-bear-the-weight-of-relationship&quot;&gt;V. The Real Problem Is How Modern Intimacy Trains People Not to Bear the Weight of Relationship&lt;/h2&gt;
&lt;p&gt;To reduce the problem to the badness, egoism, opportunism, ingratitude, or fickleness of one particular person may be emotionally satisfying, but it is not sufficiently deep. The concrete person has responsibility, certainly. Yet she is not a monster who appeared from nowhere. She is a symptom of a modern relational structure. This age continually trains people to keep options open, optimise personal trajectories, minimise sunk cost, beware of depletion, stop losses in time, respect feelings, become themselves, and avoid sacrificing too much for anyone.&lt;/p&gt;
&lt;p&gt;At the macro level, these principles are not simply wrong. They have helped many people escape oppressive relationships, reduce meaningless sacrifice, and resist old forms of moral coercion that devoured the individual in the name of love. But when universalised without restraint, they strip intimacy of its most basic weight. Love ceases to be a form of shared bearing and becomes an experience-product subject to continuous reassessment. A relationship ceases to be the construction of a common life and becomes a stage configuration within an individual growth-path. Departure ceases to require recognition as a serious ethical event and becomes a routine update in the version history of the self.&lt;/p&gt;
&lt;p&gt;Modern people therefore become ever more skilled at beginning relationships, and ever less capable of bearing them. They can begin with intensity, because intensity does not require long-term structure. They can promise quickly, because promises in contemporary culture increasingly resemble emotional expressions rather than ethical contracts. They can also withdraw rapidly, because withdrawal, once needed, is called boundary.&lt;/p&gt;
&lt;p&gt;The ontological status of commitment has declined, and that is a deeper matter than love growing shallow. In older forms of life, commitment was often oppressive, but it at least possessed weight. Today, commitment is increasingly psychologised: I did mean it at the time, but I have changed; I did love you then, but I no longer feel it; I did once imagine a future with you, but people must be allowed to change. None of this is entirely false. People do change. Yet if every commitment can be dissolved by the statement ‘people change’, then commitment is no longer commitment. It is merely a screenshot of emotion.&lt;/p&gt;
&lt;p&gt;Intimacy thus enters a state of low responsibility and high expression. People speak expansively, but bear lightly; they sound eternal in affection and provisional in withdrawal; they invoke the common life when they need you and individual freedom when they do not. The hypocrisy of modern love is not the absence of genuine feeling, but the increasing refusal of genuine feeling to accept structural consequence.&lt;/p&gt;
&lt;p&gt;What should alarm us, then, is that this manner of leaving is becoming normalised, rationalised, even ennobled. That one particular person left is the smaller fact. With sufficient command of contemporary psychological language, one can convert almost every relational responsibility into a matter of personal feeling. One need not say ‘I owe you’; one need only say ‘I cannot continue depleting myself’. One need not say ‘I injured you’; one need only say ‘You need to learn to let go’.&lt;/p&gt;
&lt;p&gt;The cruelty of this language lies in the fact that it is not entirely false. Precisely because it is half true, it is difficult to refute. She may indeed have been depleted; she may indeed need her own life; she may indeed have the right to let go; she should indeed not be imprisoned by a relationship. But when half-truth conceals the other half of truth, it becomes a lie. The other half is this: you did receive the other person’s love; you did once let them enter your future; you did wound a concrete person by withdrawing. You have the right to leave, but not the right to translate all the cost of leaving into the other person’s ‘immaturity’.&lt;/p&gt;
&lt;p&gt;At this point the critique is no longer directed solely at the one who left, but at the ethical condition of modern intimacy itself: it protects choice intensely, but undervalues commitment; it honours feeling intensely, but trivialises responsibility; it venerates clarity, but does not ask whether clarity is sometimes only the beautified name of coldness.&lt;/p&gt;
&lt;h2 id=&quot;vi-conclusion-true-severity-lies-not-in-accusing-the-other-of-heartlessness-but-in-ending-all-narratives-of-innocence&quot;&gt;VI. Conclusion: True Severity Lies Not in Accusing the Other of Heartlessness, but in Ending All Narratives of Innocence&lt;/h2&gt;
&lt;p&gt;After a relationship ends, the easiest things to say are: she stopped loving me; she changed; she became realistic; she decided I was not good enough. These statements may all be true, but they are insufficient. The colder conclusion is this: the most frightening human capacity in intimacy is the capacity to organise everything after a change of heart into a truth favourable to oneself – to make calculation resemble awakening, withdrawal resemble victimhood, injury appear necessary.&lt;/p&gt;
&lt;p&gt;By the same logic, the most dangerous self-deception of the abandoned person is his beautification of inability as the nobility of devotion, of suffering as a licence for insight, of resentment as final knowledge of human nature, of continued definition as resistance.&lt;/p&gt;
&lt;p&gt;For this reason, the essay cannot end merely by saying that she has no right to define your value. That remains too easy, too close to the literature of recovery. The more exact conclusion is this: no party should be permitted a monopoly on innocence.&lt;/p&gt;
&lt;p&gt;The one who leaves has no right to call all divestment growth. The one left behind has no right to call all loss of control devotion. The one who leaves has no right to use boundaries to erase responsibility. The one left behind has no right to use promises to erase freedom. The one who leaves has no right to rewrite the entire past. The one left behind has no right to use the past as a claim upon the future.&lt;/p&gt;
&lt;p&gt;True clarity means refusing to let any beautiful word automatically acquit a person. Growth does not automatically acquit coldness. Freedom does not automatically acquit breach. Devotion does not automatically acquit control. Pain does not automatically acquit harm. Every person must answer for concrete conduct, rather than hide behind a noble concept.&lt;/p&gt;
&lt;p&gt;The human problem in intimacy reveals its hardest outline here. People find an explanation that makes their selfishness look uncruel; that, more than the selfishness itself, is what makes them cruel. People interpret weakness as a higher love; that is what turns a wound into a claim. People know that they calculate, and cannot bear to know that they are merely calculators. People know that they demand, and want their demand to look like giving.&lt;/p&gt;
&lt;p&gt;What must finally be abolished, therefore, is her authority to price you, and with it your own authority to acquit yourself falsely. You must no longer allow her judgement to determine your value; nor may you allow your pain automatically to prove your correctness. The task is to leave the whole machinery of judgement: her valuation, your appeal, the endless litigation over the verdict of the past, the interminable struggle over who was truly innocent. Rescuing yourself from one person’s tribunal only to ascend another bench is not the same thing.&lt;/p&gt;
&lt;p&gt;Love happened. That is true. Injury followed. That is also true. She had her coldness; you had your loss of control. She had her machine of innocence; you had your self-sanctification. The death of the relationship cannot be explained by the phrase ‘not good enough’, nor summarised by saying ‘she did not know how to cherish what she had’. It was a ruin jointly produced by two people within a modern value-system of personal optimisation, depreciated commitment, sovereign feeling, narrative struggle, and self-protective instinct.&lt;/p&gt;
&lt;p&gt;The real ending is not forgetting her. Forgetting is too low a bar. The real ending is no longer needing her to be guilty in order to know that you have value; no longer needing yourself to be innocent in order to admit that you were injured; no longer needing to reduce her to a villain in order to recover narrative sovereignty from her.&lt;/p&gt;
&lt;p&gt;That is the coldest form of freedom.&lt;/p&gt;
&lt;p&gt;She has not finally been seen through.&lt;/p&gt;
&lt;p&gt;You no longer require any narrative of innocence in order to live.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Building a Stable macOS Remote Development Infrastructure with Tailscale, ShadowRocket, SMB, SSH, and Screen Sharing</title>
    <link href="https://www.ailven.cc/posts/devops-260603.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-260603.html</id>
    <published>2026-06-03T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A systematic macOS remote-development guide covering how to separate private networking from internet proxying, so that Tailscale, SMB, SSH, and Screen Sharing remain stable while GitHub, Google, Hugging Face, PyPI, and Homebrew stay accelerated.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 3 June 2026&lt;br&gt;&lt;strong&gt;Latest Update Date&lt;/strong&gt;: 9 June 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;overview&quot;&gt;Overview&lt;/h2&gt;
&lt;p&gt;This memo targets a setup with:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;MacBook Air as client&lt;/li&gt;
&lt;li&gt;Mac Mini as remote host / NAS&lt;/li&gt;
&lt;li&gt;Tailscale for private networking&lt;/li&gt;
&lt;li&gt;ShadowRocket for outbound acceleration&lt;/li&gt;
&lt;li&gt;SMB, SSH, and Screen Sharing for access&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Target state:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;LAN remains direct&lt;/li&gt;
&lt;li&gt;Tailnet remains stable&lt;/li&gt;
&lt;li&gt;SSH / SMB / VNC are not captured by proxy logic&lt;/li&gt;
&lt;li&gt;GitHub / Google / Hugging Face / PyPI / Homebrew remain accelerated&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Single principle:&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Tailscale should own private networking, and ShadowRocket should own internet acceleration.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;1-architecture-and-final-recommendation&quot;&gt;1. Architecture and final recommendation&lt;/h2&gt;
&lt;p&gt;The topology below is the stable end state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;                          Internet
                             │
                   ┌─────────┴─────────┐
                   │                   │
                   │     Tailscale     │
                   │   Overlay Mesh    │
                   │                   │
                   └─────────┬─────────┘
                             │
               ┌─────────────┴─────────────┐
               │                           │
          MacBook Air                 Mac Mini (NAS)
          Client Node                 Development Host
               │                           │
          ShadowRocket               ShadowRocket
         Full-Network Client        Restricted-Network Client
               │                           │
          SSH / SMB / VNC            SSH / SMB / VNC
            via LAN or                 via LAN or
             Tailnet                    Tailnet
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This yields four concrete rules:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;Both machines join the same Tailnet.&lt;/li&gt;
&lt;li&gt;Both machines use ShadowRocket with a local proxy listener at &lt;code&gt;127.0.0.1:1082&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The MacBook Air enables full-network capture, while the Mac Mini NAS keeps private networking outside the tunnel.&lt;/li&gt;
&lt;li&gt;LAN ranges, Tailscale ranges, loopback ranges, and internal domains must bypass proxy handling explicitly.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Typical failure symptoms when the boundary is broken:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;SSH intermittently stalls&lt;/li&gt;
&lt;li&gt;SMB mounts reconnect poorly&lt;/li&gt;
&lt;li&gt;Screen Sharing becomes erratic&lt;/li&gt;
&lt;li&gt;Tailnet hostnames resolve inconsistently&lt;/li&gt;
&lt;li&gt;public internet remains fast, which hides the root cause&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;2-why-the-two-machines-use-different-tunnel-scope&quot;&gt;2. Why the two machines use different tunnel scope&lt;/h2&gt;
&lt;p&gt;The stable design is not ‘proxy on one machine, no proxy on the other.’ It is ‘different tunnel scope for different roles.’&lt;/p&gt;
&lt;h3 id=&quot;21-macbook-air-primary-device&quot;&gt;2.1 MacBook Air: primary device&lt;/h3&gt;
&lt;p&gt;The MacBook Air is the primary interactive client. It:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;initiates SSH sessions&lt;/li&gt;
&lt;li&gt;mounts SMB shares&lt;/li&gt;
&lt;li&gt;opens browser sessions&lt;/li&gt;
&lt;li&gt;starts Screen Sharing&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For that role, ShadowRocket can capture all non-excluded traffic:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;Include All Networks: ON&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;Include Local Networks: OFF&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;private ranges and Tailnet ranges still excluded explicitly&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;22-mac-mini-nas-and-remote-host&quot;&gt;2.2 Mac Mini: NAS and remote host&lt;/h3&gt;
&lt;p&gt;The Mac Mini must remain continuously reachable as:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;an SSH server&lt;/li&gt;
&lt;li&gt;an SMB server&lt;/li&gt;
&lt;li&gt;a Screen Sharing host&lt;/li&gt;
&lt;li&gt;a persistent node expected to remain reachable after sleep, wake, and network changes&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Because it is a service host, the safer configuration is to keep private addressing outside full-network capture:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;Include All Networks: OFF&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;local, Tailnet, and loopback destinations bypassed in both ShadowRocket and macOS System Settings&lt;/li&gt;
&lt;li&gt;local proxy still exposed at &lt;code&gt;127.0.0.1:1082&lt;/code&gt; for outbound accelerated traffic&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Practical rule:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;MacBook Air: full-network client with explicit exclusions&lt;/li&gt;
&lt;li&gt;Mac Mini: restricted-network client with the same exclusions plus private-network preservation by default&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;3-configuration-that-actually-works&quot;&gt;3. Configuration that actually works&lt;/h2&gt;
&lt;h3 id=&quot;31-mac-mini-tailscale-first-proxy-second&quot;&gt;3.1 Mac Mini: Tailscale first, proxy second&lt;/h3&gt;
&lt;p&gt;Recommended startup order:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;1. Start Tailscale
2. Wait until the Tailnet is established
3. Start ShadowRocket
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;32-mac-mini-shadowrocket-settings&quot;&gt;3.2 Mac Mini: ShadowRocket settings&lt;/h3&gt;
&lt;p&gt;Reusable ShadowRocket config file:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Direct download: &lt;a href=&quot;https://www.ailven.cc/assets/posts/sr_top500_whitelist_ad.conf&quot;&gt;sr_top500_whitelist_ad.conf&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;Browser path: &lt;code&gt;/assets/posts/sr_top500_whitelist_ad.conf&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Repo path: &lt;code&gt;assets/posts/sr_top500_whitelist_ad.conf&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Recommended ShadowRocket settings on the &lt;strong&gt;Mac Mini NAS&lt;/strong&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Always On: ON
On Demand: ON
Enforce Route: OFF
Compatibility: ON
Proxy Type: HTTP
Proxy Address: 127.0.0.1
Proxy Port: 1082
Include All Networks: OFF
Include Local Networks: N/A
Include APNs: N/A
Include Cellular Services: N/A
Enable IPv6: OFF
Prefer IPv6: OFF
Private IP Answer: OFF
Fallback DNS: 1.1.1.1, 8.8.8.8, 8.8.4.4, 100.100.100.100, 192.168.123.1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Key point:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Include All Networks: OFF
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;33-private-ranges-that-must-be-excluded&quot;&gt;3.3 Private ranges that must be excluded&lt;/h3&gt;
&lt;p&gt;These entries should be repeated in:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;ShadowRocket &lt;code&gt;Skip Proxy&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;ShadowRocket &lt;code&gt;TUN Excluded Routes&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;macOS &lt;code&gt;Bypass proxy settings for these hosts and domains&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Recommended list:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;10.0.0.0/8
100.64.0.0/10
169.254.0.0/16
172.16.0.0/12
224.0.0.0/4
127.0.0.0/8
192.168.0.0/16
*.local
localhost
*.tailnet
*.tail.net
*m-team.cc
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Most important:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;100.64.0.0/10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Reason: Tailscale CGNAT range.&lt;/p&gt;
&lt;p&gt;If a default exclusion list already exists, keep it and append the entries above.&lt;/p&gt;
&lt;h3 id=&quot;34-macos-network-proxy-and-bypass-list&quot;&gt;3.4 macOS network proxy and bypass list&lt;/h3&gt;
&lt;p&gt;On both the Mac Mini and the MacBook Air, the active network service should use:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Auto Proxy Discovery: OFF
Automatic Proxy Configuration: OFF
Web Proxy (HTTP): OFF
Secure Web Proxy (HTTPS): OFF
SOCKS Proxy: 127.0.0.1:1082
Exclude simple hostnames: ON
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The bypass list in System Settings should match the exclusion list in Section 3.3.&lt;/p&gt;
&lt;h4 id=&quot;bypass-list&quot;&gt;Bypass list&lt;/h4&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;localhost
*.local

192.168.0.0/16
192.168.123.0/24
172.16.0.0/12
100.64.0.0/10
10.0.0.0/8
127.0.0.0/8
*.local
*.m-team.cc
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Optionally:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;*.ts.net
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;for Tailnet hostnames.&lt;/p&gt;
&lt;h3 id=&quot;35-macbook-air-shadowrocket-settings&quot;&gt;3.5 MacBook Air: ShadowRocket settings&lt;/h3&gt;
&lt;p&gt;Recommended ShadowRocket settings on the &lt;strong&gt;MacBook Air primary device&lt;/strong&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Always On: ON
On Demand: ON
Enforce Route: OFF
Compatibility: ON
Proxy Type: HTTP
Proxy Address: 127.0.0.1
Proxy Port: 1082
Include All Networks: ON
Include Local Networks: OFF
Include APNs: ON
Include Cellular Services: ON
Enable IPv6: OFF
Prefer IPv6: OFF
Private IP Answer: OFF
Fallback DNS: 1.1.1.1, 8.8.8.8, 8.8.4.4, 100.100.100.100, 192.168.123.1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This keeps the MacBook Air convenient for daily outbound traffic while preserving direct access to LAN and Tailnet destinations through the shared exclusion set.&lt;/p&gt;
&lt;h3 id=&quot;36-copy-and-paste-configuration-reference&quot;&gt;3.6 Copy-and-paste configuration reference&lt;/h3&gt;
&lt;p&gt;The following block preserves the current working configuration exactly as recorded:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MacMini as NAS: 

    -&amp;gt; ShadowRockets:     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; On Demand --&amp;gt; Always On            : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; On Demand --&amp;gt; On Demand            : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Enforce Routes          : OFF    
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include All Networks    : OFF     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include Local Networks  : N/A (When Include All Networks is OFF, it is inapplicable)    
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include APNs            : N/A (When Include All Networks is OFF, it is inapplicable)     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include Cellular Services : N/A (When Include All Networks is OFF, it is inapplicable)     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Compatibility Mode       : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Type               : HTTP (Do not use TUN mode)      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Port               : 1082
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Address            : 127.0.0.1       
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Enable IPv6                         : OFF    
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Prefer IPv6                         : OFF    
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Private IP Answer                   : OFF     
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Fallback DNS                        : 1.1.1.1, 8.8.8.8, 8.8.4.4, 100.100.100.100, 192.168.123.1     
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Skip Proxy                          : [existed default], 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16, *.local, localhost, *.tailnet, *.tail.net, *m-team.cc      
    -&amp;gt; Config --&amp;gt; General --&amp;gt; TUN Excluded Routes                 : [existed default], 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16     

    -&amp;gt; System Settings:       
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Auto Proxy Discovery          : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Automatic Proxy Configuration : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Web Proxy (HTTP)              : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Secure Web Proxy (HTTPS)      : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; SOCKS Proxy                   : 127.0.0.1:1082      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Exclude simple hostnames      : ON     
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Bypass proxy settings for these hosts and domains : 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16, *.local, localhost, *.tailnet, *.tail.net, *m-team.cc     

MacBookAir as primary device:      

    -&amp;gt; ShadowRockets:     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; On Demand --&amp;gt; Always On            : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; On Demand --&amp;gt; On Demand            : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Enforce Routes          : OFF    
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include All Networks    : ON     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include Local Networks  : OFF     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include APNs            : ON     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Tunnel --&amp;gt; Include Cellular Services : ON     
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Compatibility Mode       : ON      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Type               : HTTP (Do not use TUN mode)      
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Port               : 1082
    -&amp;gt; Settings --&amp;gt; Tunnel --&amp;gt; Proxy --&amp;gt; Proxy Address            : 127.0.0.1       
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Enable IPv6                         : OFF    
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Prefer IPv6                         : OFF    
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Private IP Answer                   : OFF     
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Fallback DNS                        : 1.1.1.1, 8.8.8.8, 8.8.4.4, 100.100.100.100, 192.168.123.1     
    -&amp;gt; Config --&amp;gt; General --&amp;gt; Skip Proxy                          : [existed default], 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16, *.local, localhost, *.tailnet, *.tail.net, *m-team.cc      
    -&amp;gt; Config --&amp;gt; General --&amp;gt; TUN Excluded Routes                 : [existed default], 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16     

    -&amp;gt; System Settings:       
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Auto Proxy Discovery          : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Automatic Proxy Configuration : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Web Proxy (HTTP)              : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Secure Web Proxy (HTTPS)      : OFF      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; SOCKS Proxy                   : 127.0.0.1:1082      
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Exclude simple hostnames      : ON     
    -&amp;gt; Networks --&amp;gt; The Network connected current --&amp;gt; Bypass proxy settings for these hosts and domains : 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 172.16.0.0/12, 224.0.0.0/4, 127.0.0.0/8, 192.168.0.0/16, *.local, localhost, *.tailnet, *.tail.net, *m-team.cc
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;37-an-alternative-macbook-air-setup-tun-mode&quot;&gt;3.7 An alternative MacBook Air setup: TUN mode&lt;/h3&gt;
&lt;p&gt;This is the alternative recorded for the MacBook Air. It differs from 3.5 in using TUN capture rather than the system SOCKS proxy, and in leaving &lt;code&gt;Include Local Networks&lt;/code&gt; on.&lt;/p&gt;
&lt;h4 id=&quot;tailscale&quot;&gt;Tailscale&lt;/h4&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Enabled
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;shadowrocket&quot;&gt;ShadowRocket&lt;/h4&gt;
&lt;p&gt;A TUN-based client configuration is acceptable.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Include All Networks: ON
Include Local Networks: ON
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;works correctly when the remote Mac Mini follows the server-side configuration described above.&lt;/p&gt;
&lt;p&gt;No system SOCKS proxy configuration is required.&lt;/p&gt;
&lt;h2 id=&quot;4-verification-and-fast-diagnosis&quot;&gt;4. Verification and fast diagnosis&lt;/h2&gt;
&lt;h3 id=&quot;41-first-verify-tailscale-itself&quot;&gt;4.1 First verify Tailscale itself&lt;/h3&gt;
&lt;p&gt;Run:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;tailscale status
tailscale ping &amp;lt;mac-mini-tailnet-name&amp;gt;
ping &amp;lt;mac-mini-tailnet-name&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;the Mac Mini appears in &lt;code&gt;tailscale status&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;tailscale ping&lt;/code&gt; succeeds&lt;/li&gt;
&lt;li&gt;MagicDNS resolution works consistently&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;If &lt;code&gt;tailscale status&lt;/code&gt; is healthy but &lt;code&gt;tailscale ping&lt;/code&gt; is unstable, inspect &lt;code&gt;100.64.0.0/10&lt;/code&gt; exclusions first.&lt;/p&gt;
&lt;h3 id=&quot;42-then-verify-local-resolution-and-route-choice&quot;&gt;4.2 Then verify local resolution and route choice&lt;/h3&gt;
&lt;p&gt;Run:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;scutil --dns
route -n get &amp;lt;mac-mini-lan-ip&amp;gt;
route -n get &amp;lt;mac-mini-tailnet-ip&amp;gt;
netstat -rn | grep 100.64
ifconfig | grep -A3 utun
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;is MagicDNS visible in the resolver chain&lt;/li&gt;
&lt;li&gt;is LAN traffic using the LAN interface&lt;/li&gt;
&lt;li&gt;is Tailnet traffic using the Tailscale path&lt;/li&gt;
&lt;li&gt;has an unexpected &lt;code&gt;utun&lt;/code&gt; route captured traffic that should not be proxied&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;43-verify-ssh-smb-and-screen-sharing-separately&quot;&gt;4.3 Verify SSH, SMB, and Screen Sharing separately&lt;/h3&gt;
&lt;p&gt;SSH:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ssh &amp;lt;mac-mini-lan-ip&amp;gt;
ssh &amp;lt;mac-mini-tailnet-name&amp;gt;
nc -vz &amp;lt;mac-mini-lan-ip&amp;gt; 22
nc -vz &amp;lt;mac-mini-tailnet-ip&amp;gt; 22
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;SMB and Screen Sharing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;smb://&amp;lt;mac-mini-lan-ip&amp;gt;
smb://&amp;lt;mac-mini-tailnet-name&amp;gt;
vnc://&amp;lt;mac-mini-lan-ip&amp;gt;
vnc://&amp;lt;mac-mini-tailnet-name&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;if LAN works but Tailnet fails, inspect Tailscale routing and &lt;code&gt;100.64.0.0/10&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;if Tailnet works but LAN fails, inspect local subnet exclusions&lt;/li&gt;
&lt;li&gt;if SSH works but VNC is unstable, suspect deeper private-traffic interference rather than a VNC-only bug&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;44-verify-that-internet-acceleration-still-works&quot;&gt;4.4 Verify that internet acceleration still works&lt;/h3&gt;
&lt;p&gt;Run:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;curl -I https://github.com
curl -I https://pypi.org
curl -I https://huggingface.co
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;45-end-to-end-checklist&quot;&gt;4.5 End-to-end checklist&lt;/h3&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Check&lt;/th&gt;
&lt;th&gt;Command&lt;/th&gt;
&lt;th&gt;Expected&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;LAN connectivity&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ping 192.168.123.20&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;0% packet loss&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SSH&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ssh mac_local&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Successful login&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SMB&lt;/td&gt;
&lt;td&gt;Finder ⌘+K, then &lt;code&gt;smb://192.168.123.20&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Successful authentication&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Screen Sharing&lt;/td&gt;
&lt;td&gt;&lt;code&gt;open vnc://192.168.123.20&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Session established&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Tailnet connectivity&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ping mac-mini-nas.tailxxxx.ts.net&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Successful response&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;h2 id=&quot;5-shadowrocket-and-tailscale-troubleshooting-commands&quot;&gt;5. ShadowRocket and Tailscale troubleshooting commands&lt;/h2&gt;
&lt;h3 id=&quot;51-tailscale-focused-checks&quot;&gt;5.1 Tailscale-focused checks&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;tailscale status
tailscale ip -4
tailscale ping &amp;lt;peer&amp;gt;
tailscale netcheck
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;whether the peer is visible&lt;/li&gt;
&lt;li&gt;what Tailscale IPv4 address the local node currently owns&lt;/li&gt;
&lt;li&gt;whether end-to-end Tailnet connectivity is healthy&lt;/li&gt;
&lt;li&gt;whether the local network environment is hostile to direct peer communication&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;52-route-and-interface-checks&quot;&gt;5.2 Route and interface checks&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;route -n get &amp;lt;peer-tailnet-ip&amp;gt;
route -n get &amp;lt;peer-lan-ip&amp;gt;
netstat -rn
ifconfig
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;whether traffic is choosing the intended interface&lt;/li&gt;
&lt;li&gt;whether a &lt;code&gt;utun&lt;/code&gt; interface has become unexpectedly dominant&lt;/li&gt;
&lt;li&gt;whether route ownership changed after enabling ShadowRocket&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;53-dns-checks&quot;&gt;5.3 DNS checks&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;scutil --dns
dscacheutil -q host -a name &amp;lt;peer-tailnet-name&amp;gt;
ping &amp;lt;peer-tailnet-name&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;ul&gt;
&lt;li&gt;whether MagicDNS is in the resolver chain&lt;/li&gt;
&lt;li&gt;whether the hostname resolves to the expected address&lt;/li&gt;
&lt;li&gt;whether name resolution and actual packet forwarding agree&lt;/li&gt;
&lt;/ul&gt;
&lt;h3 id=&quot;54-service-port-checks&quot;&gt;5.4 Service port checks&lt;/h3&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;nc -vz &amp;lt;peer-lan-ip&amp;gt; 22
nc -vz &amp;lt;peer-tailnet-ip&amp;gt; 22
nc -vz &amp;lt;peer-lan-ip&amp;gt; 5900
nc -vz &amp;lt;peer-tailnet-ip&amp;gt; 5900
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This separates path failure from service failure.&lt;/p&gt;
&lt;h3 id=&quot;55-fault-mapping&quot;&gt;5.5 Fault mapping&lt;/h3&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;tailscale ping&lt;/code&gt; fails, but public internet is fine: inspect Tailscale range exclusions&lt;/li&gt;
&lt;li&gt;LAN SSH fails, but Tailnet SSH works: inspect local subnet exclusions&lt;/li&gt;
&lt;li&gt;both SSH paths fail after enabling TUN on the Mac Mini: disable server-side TUN first&lt;/li&gt;
&lt;li&gt;SSH works, but SMB/VNC are unstable: suspect proxy interference with richer private traffic flows&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;6-smb-bonjour-and-finder-fault-isolation&quot;&gt;6. SMB, Bonjour, and Finder fault isolation&lt;/h2&gt;
&lt;p&gt;When SMB appears unstable on macOS, separate the problem into three independent layers:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Finder UI
    │
Bonjour / DNS-SD
    │
SMB Transport
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Many troubleshooting attempts fail because symptoms from different layers are mixed together.&lt;/p&gt;
&lt;h3 id=&quot;61-verify-smb-independently-of-finder&quot;&gt;6.1 Verify SMB independently of Finder&lt;/h3&gt;
&lt;p&gt;Check active SMB mounts:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;mount | grep smb
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;... on /Volumes/&amp;lt;share&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If mounted volumes still exist, SMB remains operational even if Finder no longer displays them.&lt;/p&gt;
&lt;p&gt;Inspect SMB session state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;smbutil statshares -a
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Pay particular attention to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;SESSION_RECONNECT_COUNT
SESSION_RECONNECT_TIME
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A reconnect event indicates a transport interruption but does not necessarily indicate a persistent SMB failure.&lt;/p&gt;
&lt;h3 id=&quot;62-verify-bonjour-independently-of-smb&quot;&gt;6.2 Verify Bonjour independently of SMB&lt;/h3&gt;
&lt;p&gt;Check SMB service discovery:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dns-sd -B _smb._tcp
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Healthy output:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Mac mini (NAS)
MacBook Air
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If Bonjour discovery works but Finder does not display the NAS, the problem is no longer in the networking layer.&lt;/p&gt;
&lt;h3 id=&quot;63-verify-dns-sd-infrastructure&quot;&gt;6.3 Verify DNS-SD infrastructure&lt;/h3&gt;
&lt;p&gt;Check DNS-SD socket state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;stat /var/run/mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;srw-rw-rw- ... /var/run/mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A missing socket may correlate with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;DNSServiceBrowse failed -65563 (Service Not Running)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;and simultaneously cause:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Finder Network discovery failures&lt;/li&gt;
&lt;li&gt;Bonjour browsing failures&lt;/li&gt;
&lt;li&gt;intermittent SMB visibility issues&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Additional verification:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ps aux | grep mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo lsof -nP -p $(pgrep -x mDNSResponder)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Healthy output should include:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;UDP *:5353
/var/run/mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;64-separate-finder-problems-from-network-problems&quot;&gt;6.4 Separate Finder problems from network problems&lt;/h3&gt;
&lt;p&gt;Verify mounted shares:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ls -la /Volumes
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If a share exists under:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/Volumes/NAS
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;but does not appear in Finder Sidebar, the SMB connection is already established.&lt;/p&gt;
&lt;p&gt;Restart Finder:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;killall Finder
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If the mounted share immediately reappears in Sidebar after Finder restarts, the problem is a Finder UI state issue rather than an SMB issue.&lt;/p&gt;
&lt;h3 id=&quot;65-fault-mapping&quot;&gt;6.5 Fault mapping&lt;/h3&gt;
&lt;h4 id=&quot;finder-sidebar-missing-nas&quot;&gt;Finder sidebar missing NAS&lt;/h4&gt;
&lt;p&gt;Symptoms:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Finder Sidebar missing NAS
mount shows active SMB volumes
dns-sd works
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Likely cause:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Finder Sidebar cache/state inconsistency
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Recommended action:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;killall Finder
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;finder-network-browser-missing-nas&quot;&gt;Finder network browser missing NAS&lt;/h4&gt;
&lt;p&gt;Symptoms:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;dns-sd -B _smb._tcp
DNSServiceBrowse failed -65563 (Service Not Running)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Likely cause:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Bonjour / DNS-SD failure
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Recommended investigation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;stat /var/run/mDNSResponder
ps aux | grep mDNSResponder
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;smb-failure&quot;&gt;SMB failure&lt;/h4&gt;
&lt;p&gt;Symptoms:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;No active SMB mounts
Cannot mount via smb://&amp;lt;ip&amp;gt;
Infuse SMB access also fails
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Likely cause:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;SMB transport or network path failure
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This should be investigated separately from Bonjour and Finder.&lt;/p&gt;
&lt;h3 id=&quot;66-the-lesson&quot;&gt;6.6 The lesson&lt;/h3&gt;
&lt;p&gt;Do not treat:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Finder cannot see NAS
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;as equivalent to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;SMB is broken
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Always verify:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;SMB
Bonjour
Finder
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;independently before making configuration changes.&lt;/p&gt;
&lt;h2 id=&quot;7-reusing-the-socks-endpoint-from-development-tools&quot;&gt;7. Reusing the SOCKS endpoint from development tools&lt;/h2&gt;
&lt;p&gt;The local SOCKS endpoint:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;can be reused by:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Git&lt;/li&gt;
&lt;li&gt;pip&lt;/li&gt;
&lt;li&gt;uv&lt;/li&gt;
&lt;li&gt;Homebrew&lt;/li&gt;
&lt;li&gt;Hugging Face&lt;/li&gt;
&lt;li&gt;curl&lt;/li&gt;
&lt;li&gt;wget&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;export ALL_PROXY=socks5h://127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Git:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git config --global http.proxy socks5://127.0.0.1:1082
git config --global https.proxy socks5://127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;8-conclusion&quot;&gt;8. Conclusion&lt;/h2&gt;
&lt;p&gt;Recommended model:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Tailscale owns private connectivity&lt;/li&gt;
&lt;li&gt;ShadowRocket owns outbound internet acceleration&lt;/li&gt;
&lt;li&gt;the client may use TUN mode&lt;/li&gt;
&lt;li&gt;the server should avoid TUN mode&lt;/li&gt;
&lt;li&gt;private and Tailnet ranges must be excluded explicitly&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This is easier to reason about, easier to debug, and materially more stable in daily use.&lt;/p&gt;
&lt;p&gt;The architecture provides stable LAN and Tailnet communication, reliable SSH, SMB and Screen Sharing services, accelerated access to external development resources, and minimal interaction between Tailscale and the proxy infrastructure.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>当时只道是寻常</title>
    <link href="https://www.ailven.cc/posts/moment-260523.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260523.html</id>
    <published>2026-05-23T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>一次与父亲、祖父同行的江南短途旅行，在归家后的空虚中显出真正分量：它记录的不是风景，而是三代人尚能共同远行的短暂时间窗口。</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 23 May 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;这次旅行结束以后，我把车还了，回到家里。&lt;/p&gt;
&lt;p&gt;屋子里很静。前几天放在车上的水、纸巾、充电线、临时买的小东西，都一样一样拿了下来。租来的车已经交还，酒店已经退房，行李也空了。手机里的导航不再响起，高速出口不再需要提前提醒，停车场的位置也不必再记着了。茶叶还在袋子里，照片还在手机里，微信上偶尔还有一两条消息，可人已经走了。&lt;/p&gt;
&lt;p&gt;父亲和祖父坐上了回济南的火车。我从虹桥回来，又去还车。办完这些事的时候，并没有立刻觉得怎样。人忙着的时候，心是钝的。要看路，要找停车位，要搬行李，要提醒他们拿好证件。等这一切都停下来，门一关，房间里只剩下我一个人，那空荡才慢慢从身后追了上来。&lt;/p&gt;
&lt;p&gt;我忽然觉得，这三天三夜并不只是一次上海周边的小旅行。&lt;/p&gt;
&lt;p&gt;它时间不长，也不算短；行程不满，也没有松散。我们在上海相聚，去了苏州，又去了安吉。昨天上午在溪龙乡茶山附近的一个乡村小院里吃饭、买茶、拍照，下午去了竹博园，看竹子，也看熊猫。晚上又回到上海，吃了一顿饭。今早从南翔出发，把父亲和祖父送到虹桥。他们都很开心。父亲开心，祖父也开心。我知道他们是真的开心，不是为了让我安心才说好。&lt;/p&gt;
&lt;p&gt;按旅游来说，这并不是一条多么惊人的路线。没有名山大川，没有很远的远方，也没有什么非去不可的奇观。可是它结束以后，我才知道，它留在心里的分量，远不是一个短途游所能解释的。&lt;/p&gt;
&lt;p&gt;因为这次旅行的好处，本来就不全在旅行本身。&lt;/p&gt;
&lt;p&gt;它的珍贵处，是它发生了。&lt;/p&gt;
&lt;p&gt;而且是在这样一个时候发生了：我二十八岁，父亲还开得动长途车，祖父八十岁了，却还耳聪目明，身体硬朗，能坐车，能走路，能吃饭，能看茶山，能在竹林里停下来拍照片。我们三代人，正好都还能够出门，能够在同一辆车里待上几天。&lt;/p&gt;
&lt;p&gt;这件事，说起来寻常，细想却并不寻常。&lt;/p&gt;
&lt;p&gt;小时候，我大约不会想到这些。父亲会开车，祖父会坐在那里，家里人到了节日会相聚，下一次总会来。这些事在孩子眼里像天色一样自然，像饭桌一样自然。后来长大了，人忙着读书、工作、赶路，仍旧以为许多东西会照旧。可时间并不照旧。它只是不出声，不惊动人，一点一点改着每个人的身体、步子、胃口、精力，也改着一家人之间的位置。&lt;/p&gt;
&lt;p&gt;许多事情不会郑重其事地告诉你：这是最后一次。它只会装成一个普通的周末，一顿普通的饭，一段普通的高速，一张随手拍下的照片。等许多年后再回头看，人才知道，那里面已经藏着不可再来的东西。&lt;/p&gt;
&lt;p&gt;当时只道是寻常。&lt;/p&gt;
&lt;h2 id=&quot;接站&quot;&gt;接站&lt;/h2&gt;
&lt;p&gt;我现在很清楚地记得浦东机场那一晚。父亲和祖父落地，我开车去接他们。那时候我想着航班有没有晚点，停车场怎么走，浦东回南翔会不会堵，晚上吃什么，老人累不累。上海的高架、机场的灯、路上的车流，都只是那一晚的背景。可现在想起来，它却像这次旅行真正的开头。&lt;/p&gt;
&lt;p&gt;父亲和祖父从北方来，落在上海夜色里。我开着租来的帕萨特去接他们。车门关上以后，后备箱里放着行李，前方是回南翔的路。那一刻，我好像把他们从一个地方接进了我的生活里。过去是父亲带我出门，是大人安排路、安排饭、安排住处。现在轮到我看导航，轮到我想哪里停车方便，轮到我问他们饿不饿、累不累。&lt;/p&gt;
&lt;p&gt;这种变化没有人宣布。它不是某一句话，也不是某一个明显的仪式。它藏在许多小事里。&lt;/p&gt;
&lt;p&gt;我开始想着：祖父上下车慢不慢，饭菜要不要清淡一点，午后是不是该休息，今天还要不要多去一个地方；父亲开车是不是累了，要不要换一换路线，要不要少走山路；住的地方有没有电梯，停车是不是方便，明天去火车站要留多少时间。&lt;/p&gt;
&lt;p&gt;这些都是旅行中很琐碎的事。可是琐碎事最能说明人的位置。小时候，是他们替我想这些。如今，是我开始替他们想这些。&lt;/p&gt;
&lt;p&gt;我以前总觉得父亲就是父亲。他会开车，会判断路，会处理麻烦，会做决定。他坐在驾驶座上，我就觉得稳妥。祖父也像家中一直存在的人。他坐在那里，说话，吃饭，笑，讲过去的事。很久以来，我并没有认真把他们看作“正在老去的人”。他们只是父亲，只是祖父，是家的一部分，好像本来就该在那里。&lt;/p&gt;
&lt;p&gt;这次旅行却让我忽然看见：祖父已经老了，父亲也正在慢慢变老，而我已经开始站到了这个家庭中间的位置。&lt;/p&gt;
&lt;p&gt;这话一想到，心里便酸。&lt;/p&gt;
&lt;p&gt;不是因为有哪一件事特别悲伤。恰恰相反，这几天都很好，很圆满，大家都很高兴。可是这种圆满里，有一种很轻的疼。它不锋利，却深。它使我明白，我已经不只是那个被照顾的孩子了。我开始成为安排者，成为连接者，成为看路的人，成为想着大家体力和时间的人。&lt;/p&gt;
&lt;h2 id=&quot;安吉&quot;&gt;安吉&lt;/h2&gt;
&lt;p&gt;父亲仍在前面开车。祖父仍坐在后排看窗外。我坐在副驾，低头看导航，提醒前方出口，看看路况，偶尔回头问祖父累不累。这个画面太平常了。每一个自驾家庭都可能有这样的画面。可它并不是可以无限重复的。&lt;/p&gt;
&lt;p&gt;它要许多条件一起成立。父亲还愿意开车，也开得动；祖父还能够远行，也愿意同行；我有时间，有心力，也有一点能力把这趟路安排好。我们三个人都还在，都还健康，都还能够在同一辆车里，把三天三夜过成共同的生活。&lt;/p&gt;
&lt;p&gt;这不是理所当然的。&lt;/p&gt;
&lt;p&gt;我后来反复想起溪龙乡。那天上午到了茶山，天色、空气、山路，都有一种江南山里的湿润。我们没有去什么很大的景点，只是到了一个乡村小院。饭菜摆上来，都是当地菜。笋、鸡、青菜、茶水。不是大馆子的精致，倒有一种家常的实在。父亲看茶叶，看得很认真。祖父也跟着坐在一旁，慢慢喝茶。院子里有光，有风，有人说话，茶杯放在桌上，饭菜热着。那一刻并没有什么豪华处，却很舒服。&lt;/p&gt;
&lt;p&gt;安吉白茶这个名字，过去只是一个茶名。那天以后，它就不是单单一个茶名了。它里面有山路，有小院，有父亲挑茶的神情，有祖父坐着喝茶的样子，有我们拍下的照片，也有那顿并不隆重却很妥帖的午饭。&lt;/p&gt;
&lt;p&gt;后来去了竹博园。竹子很多，青得安静。熊猫也看了。其实熊猫在哪里都有人看，竹林也不是没有别处可看。可三个人一起慢慢走、慢慢看，停下来拍照，觉得累了就歇一歇，这样的下午，反而比许多“著名景点”更留得住。它不热闹，不紧张，不贪多。快乐不是被推到高处的，是散在路边、树下、竹影里的一点一点轻松。&lt;/p&gt;
&lt;p&gt;我现在想，好的旅行大约就是这样：不把人赶着走，也不让人空着；不叫人疲倦，也不叫人遗憾。我们这一次便是如此。时间刚好，路程刚好，风景刚好，老人也刚好能承受，父亲也有兴致。它没有一处惊天动地，却处处合适。&lt;/p&gt;
&lt;p&gt;晚上回到上海吃饭，父亲和祖父都很开心。我当时也开心，可那时的开心还在事情里面，没来得及沉下来。真正的后劲，是他们走了以后才来的。&lt;/p&gt;
&lt;p&gt;今早送他们去虹桥。南翔到虹桥并不远，三四十分钟的车程。早晨的上海还没有完全热闹起来。路上车不算少，但也顺。到了虹桥，又是停车、拿行李、看时间、找进站口。人到车站，总是忙的。忙着安顿，忙着提醒，忙着说“到了发消息”。看他们往里走，我也没有立刻伤感，只觉得事情办完了，可以松一口气了。&lt;/p&gt;
&lt;p&gt;后来一个人转身离开，才觉得这一松，松得有些空。&lt;/p&gt;
&lt;h2 id=&quot;送站&quot;&gt;送站&lt;/h2&gt;
&lt;p&gt;送站是很奇怪的事。人在一起的时候，总觉得分别还远。到了车站，分别就变成一套程序：停车、下车、取行李、进站、挥手。程序很清楚，情绪却滞后。等走回车里，等路上没有人说话，等车还了，等回到家，情绪才慢慢赶来。&lt;/p&gt;
&lt;p&gt;原来刚刚那几天，是真的在生活。&lt;/p&gt;
&lt;p&gt;这句话在我心里停了很久。&lt;/p&gt;
&lt;p&gt;平时我当然也在生活。工作、吃饭、睡觉、读书、回消息、想未来，这些都是生活。可那种生活常常被许多任务切碎，被屏幕围住，被目标推着走。它真实，却常常缺少人与人之间连续的温度。这几天不一样。我们一起起床，一起出发，一起坐车，一起等菜，一起看路，一起决定下一站，一起买茶，一起拍照，一起回酒店。那些事都很小，可小事连在一起，就成了一段完整的生活。&lt;/p&gt;
&lt;p&gt;也许我所怀念的，并不是苏州、安吉、上海这些地名本身，而是三代人连续几天共同生活的状态。&lt;/p&gt;
&lt;p&gt;现代人的家，常常是散的。成年子女在一个城市工作，父母在另一个地方生活，老人又在更远的时间里慢慢老去。我们平时靠电话、微信、照片维系联系。联系是有的，但共同生活少了。真正坐在同一张桌子旁吃饭，坐在同一辆车里看路，晚上回到同一个住处，第二天早晨再一起出门，这样的机会并不多。&lt;/p&gt;
&lt;p&gt;所以这次旅行不像只是出去玩。&lt;/p&gt;
&lt;p&gt;它更像短短几天里，我们又重新成为了一家人。&lt;/p&gt;
&lt;p&gt;这句话也许有些奇怪。我们本来就是一家人，何来“重新”？可人到成年以后，亲人之间往往只剩身份，少了相处。身份还在，日常却不在。父亲仍是父亲，祖父仍是祖父，可如果没有共同经历，亲情也会慢慢变成节日里的问候、手机里的头像、通话里的几句叮嘱。亲情不是空的名词。它要落在具体事情上：一顿饭，一段路，一个停车场，一杯茶，一张照片，一次送站。&lt;/p&gt;
&lt;p&gt;这次旅行给这些关系重新添了实体。&lt;/p&gt;
&lt;h2 id=&quot;然后呢&quot;&gt;然后呢&lt;/h2&gt;
&lt;p&gt;以后再说起溪龙乡，说起安吉白茶，说起竹博园，说起虹桥，说起那天的饭，它们都不再只是地名、茶名、景点名。它们会成为我们家里能共同想起的东西。一提到那天的茶山，就会想起小院；一看到茶叶，就会想起父亲挑茶；一说到竹博园，就会想起祖父的照片；一想到上海，就会想起那天夜里的灯和最后一上午的路。&lt;/p&gt;
&lt;p&gt;这些留下来的，已经不是旅行记忆。&lt;/p&gt;
&lt;p&gt;而是家还完整存在过的证据。&lt;/p&gt;
&lt;p&gt;所谓完整，并不是说家里没有遗憾，也不是说所有事情都圆满无缺。完整只是说，在那几天里，祖父还走得动，父亲还开得动，我还接得住这件事；我们还能一起吃饭，一起出门，一起坐车，一起说笑。这样的完整很安静，不声张，也不耀眼。可是人一旦看见它，就知道它极其珍贵。&lt;/p&gt;
&lt;p&gt;我从前大约太少这样想。人年轻时，总觉得更重要的是未来：工作、事业、技术、成长、远方、更大的城市、更好的机会。这些当然也重要。可是这次旅行让我明白，人生里还有一些东西，不能用未来来替代。父亲现在能开车，不等于永远能开；祖父现在能远行，不等于以后仍能远行；一家人现在能在一起三天三夜，不等于这样的日子会自然再来。&lt;/p&gt;
&lt;p&gt;幸福有时候不是多得了什么，而是在一个很普通的时刻忽然知道：我现在拥有的，已经足够珍贵。&lt;/p&gt;
&lt;p&gt;这种知道会带来疼痛。因为看见珍贵，也就是看见它不可久留。时间并不因为我们觉察就停下。父亲还会老，祖父还会老，我也会继续向前走。家族里的位置会变化，人的体力会变化，见面的次数会变化，能一起远行的机会会变少。想到这些，我并不轻松。可是也正因为如此，已经发生的这一次，才显得更深。&lt;/p&gt;
&lt;p&gt;它提醒我，不要等到一切都不能重来，才开始怀念普通日子。&lt;/p&gt;
&lt;p&gt;我现在仍然有些空。可我不急着把它填满。这个空不是坏东西。它说明这几天不是被我随手消费掉的时间，而是真正进入我心里的时间。若一件事结束以后毫无痕迹，那它大约只是经过了；若结束以后人觉得空，觉得酸，觉得想哭，那说明人曾经在里面真切地活过。&lt;/p&gt;
&lt;h2 id=&quot;当时只道是寻常&quot;&gt;当时只道是寻常&lt;/h2&gt;
&lt;p&gt;我想起朱砂印落在纸上，颜色未必很深，却不会轻易擦去。这次旅行也像这样，在我的心里盖了一个小小的印。没有谁特意用力，可它已经在那里。&lt;/p&gt;
&lt;p&gt;我会记得浦东机场那晚的车流。&lt;br&gt;我会记得父亲坐在驾驶座上。&lt;br&gt;我会记得祖父坐在后排看窗外。&lt;br&gt;我会记得苏州那一天。&lt;br&gt;我会记得溪龙乡的茶山和乡村小院。&lt;br&gt;我会记得杯子里的白茶。&lt;br&gt;我会记得竹博园的竹子和熊猫。&lt;br&gt;我会记得上海夜里的灯光。&lt;br&gt;我会记得南翔清晨去虹桥的路。&lt;br&gt;我会记得送站后，一个人还车回家的空荡。&lt;/p&gt;
&lt;p&gt;很多年以后，酒店名字、饭店名字、具体路线，也许都会淡下去。可是那个画面，我想我不会忘：父亲在前面开车，祖父在后排，而我坐在副驾导航。&lt;/p&gt;
&lt;p&gt;那是一次旅行里的画面，也是一种人生里的秩序。前面是正在慢慢老去的父亲，后面是已经年迈的祖父，而我在中间看路、提醒、安排、陪伴。那几天里，我清楚地站在了家族时间的中间。不是被推上去的，也不是自己宣告的，只是在一段普通旅程里，自然而然地站到了那里。&lt;/p&gt;
&lt;p&gt;当时只道是寻常。&lt;/p&gt;
&lt;p&gt;如今才知道，那一点也不寻常。&lt;/p&gt;
&lt;p&gt;那是时间在日常生活里露出的一瞬。&lt;br&gt;那是家人仍能共同远行的一段短暂窗口。&lt;br&gt;那是父亲仍然能开车、祖父仍然能同行、我仍然能陪伴他们的几天。&lt;br&gt;那是一个儿子和孙子，第一次如此清楚地看见自己的位置。&lt;/p&gt;
&lt;p&gt;我确认父亲仍在。&lt;br&gt;我确认祖父仍在。&lt;br&gt;我确认我们曾经一起走过这几天。&lt;br&gt;我确认这些普通日子，已经成了我生命里很深的一部分。&lt;/p&gt;
&lt;p&gt;所以此刻若想哭，我不愿忍住。&lt;/p&gt;
&lt;p&gt;这不是脆弱。&lt;br&gt;这是人终于看见时间、亲情和不可重复的生活以后，心里自然生出的泪。&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Building the MLC-LLM Compilation Stack from Source on NVIDIA H800</title>
    <link href="https://www.ailven.cc/posts/devops-260504.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-260504.html</id>
    <published>2026-05-04T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>A step-by-step technical guide for establishing a complete, source-built machine learning compilation environment on an NVIDIA H800 system under Kubernetes – covering container configuration, read-only filesystem remediation, LLVM and CUDA toolchain installation, and TVM compilation.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 4 May 2026&lt;br&gt;&lt;strong&gt;Last Update Date&lt;/strong&gt;: 7 May 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;overview&quot;&gt;Overview&lt;/h2&gt;
&lt;p&gt;This guide is intended for practitioners who wish to build the complete machine learning compilation stack entirely from source on an NVIDIA H800 system (24-core CPU, 220 GB memory) within a Linux container environment, as referenced in the &lt;code&gt;mlc-ai/mlc-llm&lt;/code&gt; source repository. The stack in question comprises MLC-LLM, XGrammar, Apache TVM, FlashInfer, and CUDATile.&lt;/p&gt;
&lt;p&gt;The guide covers the full process: pulling a base Docker image, configuring the container, resolving system-level read-only filesystem and dependency issues, accelerating package management (APT and PyPI), and configuring and compiling TVM. Each step adheres strictly to a ‘one step, one verification’ principle to ensure robustness and reproducibility.&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Critical Note&lt;/strong&gt;: In the target environment described in this guide, the container is managed by a Kubernetes orchestration platform, with the root filesystem configured as &lt;strong&gt;read-only&lt;/strong&gt; by default. This causes the system package manager (dpkg/APT) to fail when attempting to write to &lt;code&gt;/etc/group&lt;/code&gt;, blocking all subsequent operations.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;This guide not only addresses how to resolve such read-only container constraints gracefully, but treats the issue as an opportunity to establish an independent, writable, and stable high-performance compilation environment from first principles.&lt;/p&gt;
&lt;h2 id=&quot;1-preparing-the-container&quot;&gt;1. Preparing the container&lt;/h2&gt;
&lt;h3 id=&quot;1-container-environment-preparation&quot;&gt;1. Container Environment Preparation&lt;/h3&gt;
&lt;h4 id=&quot;11-pulling-the-base-image&quot;&gt;1.1 Pulling the Base Image&lt;/h4&gt;
&lt;p&gt;MLC-LLM requires CUDA version 12.0 or later to correctly support its compilation and runtime operators. We select &lt;code&gt;nvidia/cuda:13.2.0-cudnn-devel-ubuntu22.04&lt;/code&gt; as the base container. The &lt;code&gt;devel&lt;/code&gt; tag is built atop the &lt;code&gt;runtime&lt;/code&gt; image and includes the CUDA compilation toolchain, header files, and development libraries – precisely what is required for building CUDA applications from source.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Base pull command (exact invocation depends on your container runtime)
docker pull docker.io/nvidia/cuda:13.2.0-cudnn-devel-ubuntu22.04

# If operating within a Chinese network where docker.io access is slow or unavailable,
# a domestic mirror may be used instead:
# swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/nvidia/cuda:13.2.0-cudnn-devel-ubuntu22.04
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;12-launching-an-interactive-container&quot;&gt;1.2 Launching an Interactive Container&lt;/h4&gt;
&lt;p&gt;The following command launches a persistent interactive container from the Docker image. To ensure that compiled artefacts are not lost when the container exits, the host-side source directory must be mounted into the container. In a Kubernetes environment in particular, &lt;strong&gt;the root filesystem must be explicitly declared as writable&lt;/strong&gt;; failure to do so will trigger the dpkg errors described later in this guide.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Launch an interactive container with host directory mount and writable root filesystem
docker run -it \
  --gpus all \
  --shm-size=32g \
  --mount type=bind,source=/path/to/your/local/repos,target=/workspace/repos \
  --read-only=false \     # Critical in Kubernetes environments
  docker.io/nvidia/cuda:13.2.0-cudnn-devel-ubuntu22.04 /bin/bash
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If using a company-managed Kubernetes-based experimental platform (such as KTP or an equivalent), the image address may be entered directly, provided that the platform is capable of pulling from the specified registry.&lt;/p&gt;
&lt;h4 id=&quot;13-base-directory-structure&quot;&gt;1.3 Base Directory Structure&lt;/h4&gt;
&lt;p&gt;Upon entering the container, it is advisable to create the source code directory in advance before proceeding with subsequent steps.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;mkdir -p /workspace/repos
cd /workspace/repos
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;2-system-configuration-and-package-mirror-acceleration&quot;&gt;2. System Configuration and Package Mirror Acceleration&lt;/h3&gt;
&lt;p&gt;When provisioning a Pod on a corporate Kubernetes-based experimental platform, the resulting environment will very likely be a read-write–restricted Pod whose root filesystem is read-only. This can cause &lt;code&gt;apt&lt;/code&gt; configuration failures or system package corruption. Under any circumstances, a writable system environment is a prerequisite for compiling software. Attempting to run advanced compilation tasks on an unstable read-only filesystem is inadvisable.&lt;/p&gt;
&lt;h4 id=&quot;21-configuring-apt-and-pip-mirror-sources&quot;&gt;2.1 Configuring APT and PIP Mirror Sources&lt;/h4&gt;
&lt;p&gt;To ensure the fastest possible download speeds during compilation, it is recommended to switch to high-speed domestic mirrors.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;APT acceleration (Tsinghua University mirror)&lt;/strong&gt;: &lt;a href=&quot;https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu/&quot;&gt;https://mirrors.tuna.tsinghua.edu.cn/help/ubuntu/&lt;/a&gt;&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;PyPI acceleration (Tsinghua University mirror)&lt;/strong&gt;: &lt;a href=&quot;https://mirrors.tuna.tsinghua.edu.cn/help/pypi/&quot;&gt;https://mirrors.tuna.tsinghua.edu.cn/help/pypi/&lt;/a&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Configure a trusted host to suppress SSL-related warnings
pip config set install.trusted-host mirrors.tuna.tsinghua.edu.cn
&lt;/code&gt;&lt;/pre&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;h4 id=&quot;22-installing-essential-build-tools&quot;&gt;2.2 Installing Essential Build Tools&lt;/h4&gt;
&lt;p&gt;The following installs the foundational components required for building TVM:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Install core build dependencies
sudo apt install -y \
    vim \
    build-essential \
    cmake \
    curl \
    wget \ 
    git \
    ninja-build \
    python3 \
    python3-pip \
    python3-dev

# Install version control and additional dependencies
sudo apt install -y git-lfs

# Verify installations
cmake --version          # Should be &amp;gt;= 3.22
ninja --version          # Should be &amp;gt;= 1.10
python3 --version        # Should be &amp;gt;= 3.10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;If the system has entered a &lt;code&gt;dpkg&lt;/code&gt; deadlock – for instance, if &lt;code&gt;groupadd: cannot open /etc/group&lt;/code&gt; or &lt;code&gt;E: Sub-process /usr/bin/dpkg returned an error code (1)&lt;/code&gt; is reported – the obstructing post-installation scripts may be safely removed as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Remove the corrupted package configuration script (openssh-client shown as an example)
sudo rm -f /var/lib/dpkg/info/openssh-client.postinst

# Force completion of the dpkg database repair
sudo dpkg --configure -a
sudo apt install -f
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;2-installing-the-toolchains&quot;&gt;2. Installing the toolchains&lt;/h2&gt;
&lt;h3 id=&quot;3-installing-the-llvm-and-cuda-toolchains&quot;&gt;3. Installing the LLVM and CUDA Toolchains&lt;/h3&gt;
&lt;h4 id=&quot;31-installing-llvm-18&quot;&gt;3.1 Installing LLVM 18&lt;/h4&gt;
&lt;p&gt;LLVM 18 is installed via the ISCAS (Institute of Software, Chinese Academy of Sciences) mirror, which provides one of the most stable and accessible open-source LLVM repositories.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;set -e

# Basic Dependencies
sudo apt update
sudo apt install -y ca-certificates wget gnupg

# Remove personal pre-existing LLVM source list files
sudo rm -f /etc/apt/sources.list.d/llvm-iscas.list

# Add the ISCAS LLVM mirror
wget -qO- https://apt.llvm.org/llvm-snapshot.gpg.key \
  | sudo gpg --dearmor -o /usr/share/keyrings/llvm-archive-keyring.gpg

# Verify fingerprint 
gpg --show-keys --fingerprint /usr/share/keyrings/llvm-archive-keyring.gpg

# Add ISCAS LLVM mirror
echo &amp;quot;deb [signed-by=/usr/share/keyrings/llvm-archive-keyring.gpg] https://mirror.iscas.ac.cn/llvm-apt/jammy/ llvm-toolchain-jammy-18 main&amp;quot; \
  | sudo tee /etc/apt/sources.list.d/llvm-iscas.list

sudo apt update

sudo apt install -y \
  clang-18 \
  lld-18 \
  lldb-18 \
  clangd-18 \
  clang-format-18 \
  clang-tidy-18 \
  llvm-18-dev \
  llvm-18-tools

# Verify the installation
llvm-config-18 --version   # Expected output: 18.1.8
clang-18 --version
which clang-18
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;em&gt;Outcome at this stage: the ISCAS mirror provides &lt;code&gt;llvm-config-18&lt;/code&gt; at version &lt;code&gt;18.1.8&lt;/code&gt;, supplying TVM with a complete LLVM backend for JIT compilation.&lt;/em&gt;&lt;/p&gt;
&lt;h4 id=&quot;32-configuring-cuda-and-cudnn&quot;&gt;3.2 Configuring CUDA and cuDNN&lt;/h4&gt;
&lt;p&gt;This guide uses NVIDIA CUDA 13.2.0. As the container image already contains the full CUDA toolchain, no additional installation is necessary. Should the toolchain be absent, installation instructions are available at the &lt;a href=&quot;https://developer.nvidia.com/cuda-downloads&quot;&gt;NVIDIA CUDA download page&lt;/a&gt;.&lt;/p&gt;
&lt;p&gt;When compiling TVM, CMake must be able to locate the cuDNN library files precisely. If the default path does not exist – for example, if &lt;code&gt;/usr/lib/x86_64-linux-gnu/libcudnn.so&lt;/code&gt; is a broken symbolic link or is absent – it must be specified manually.&lt;/p&gt;
&lt;p&gt;Confirm the actual location of the cuDNN libraries first:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;find /usr -name &amp;quot;libcudnn*&amp;quot; 2&amp;gt;/dev/null
find /usr -name &amp;quot;cudnn*.h&amp;quot; 2&amp;gt;/dev/null
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Adjust &lt;code&gt;CUDNN_LIBRARY&lt;/code&gt; and the cuDNN &lt;code&gt;INCLUDE_DIR&lt;/code&gt; within &lt;code&gt;config.cmake&lt;/code&gt; according to the paths returned by these commands.&lt;/p&gt;
&lt;h3 id=&quot;4-configuring-the-proxy-for-github&quot;&gt;4. Configuring the proxy for GitHub&lt;/h3&gt;
&lt;p&gt;In some cases, especially you are operating on a domestic machine, you may encounter problem that github unaccessible. Please make sure you have a proxy server in your server for github clone:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo apt update
sudo apt install shadowsocks-libev
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Create a configuration file, link &lt;code&gt;~/shadowsockets.json&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
    &amp;quot;server&amp;quot;: &amp;quot;your.ss.node.ip&amp;quot;,
    &amp;quot;server_port&amp;quot;: {port},
    &amp;quot;local_address&amp;quot;: &amp;quot;127.0.0.1&amp;quot;,
    &amp;quot;local_port&amp;quot;: 1082,
    &amp;quot;password&amp;quot;: &amp;quot;your ss node password&amp;quot;,
    &amp;quot;timeout&amp;quot;: 300,
    &amp;quot;method&amp;quot;: &amp;quot;entropy method&amp;quot;,
    &amp;quot;fast_open&amp;quot;: false,
    &amp;quot;workers&amp;quot;: 1
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check where &lt;code&gt;ss-local&lt;/code&gt; is: &lt;code&gt;which ss-local&lt;/code&gt;, it always is /usr/bin/ss-local&lt;/p&gt;
&lt;h4 id=&quot;self-start&quot;&gt;self-start&lt;/h4&gt;
&lt;p&gt;Create systemd: &lt;code&gt;sudo vim /etc/systemd/system/ss-local.service&lt;/code&gt;, paste content below:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;[Unit]
Description=Shadowsocks Local Proxy
After=network.target

[Service]
Type=simple
User=ailven.liu
ExecStart=/usr/bin/ss-local -c /home/ailven.liu/shadowsockets.json
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: replace &lt;code&gt;ExecStart&lt;/code&gt; and &lt;code&gt;User&lt;/code&gt; to what they really are.&lt;/p&gt;
&lt;p&gt;Reload configuration and set to self-start:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;sudo systemctl daemon-reload
sudo systemctl enable --now ss-local.service
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check status and log:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;systemctl status ss-local.service
journalctl -u ss-local.service -f
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;run-background&quot;&gt;Run background&lt;/h4&gt;
&lt;pre&gt;&lt;code&gt;nohup ss-local -c ~/shadowsockets.json &amp;gt; ~/ss-local.log 2&amp;gt;&amp;amp;1 &amp;amp;
ps aux | grep ss-local # check status
tail -f ~/ss-local.log # check logs
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;setting-the-proxy-for-github-only&quot;&gt;Setting the proxy for GitHub only&lt;/h4&gt;
&lt;p&gt;Then run command building proxy only for github:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git config --global http.https://github.com/.proxy socks5://127.0.0.1:1082
git config --global https.https://github.com/.proxy socks5://127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;or you can also edit file &lt;code&gt;~/.gitconfig&lt;/code&gt; directly:&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;[http &amp;quot;https://github.com&amp;quot;]
    proxy = socks5://127.0.0.1:1082
[https &amp;quot;https://github.com&amp;quot;]
    proxy = socks5://127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check it:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git config --global --list --show-origin | grep github
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;You can also delete the proxy config from github:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git config --global --unset http.https://github.com/.proxy
git config --global --unset https.https://github.com/.proxy
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;3-cloning-and-pinning-the-sources&quot;&gt;3. Cloning and pinning the sources&lt;/h2&gt;
&lt;h3 id=&quot;5-cloning-mlc-llm-and-its-pinned-source-dependencies&quot;&gt;5. Cloning MLC-LLM and Its Pinned Source Dependencies&lt;/h3&gt;
&lt;p&gt;All source code is placed under &lt;code&gt;~/repos&lt;/code&gt;.&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;mkdir -p ~/repos
cd ~/repos
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Before cloning the full repository, verify that GitHub access is working through the configured Git proxy:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;git config --global --list --show-origin | grep -Ei &amp;#39;proxy|github&amp;#39; || true

GIT_CURL_VERBOSE=1 git ls-remote https://github.com/mlc-ai/mlc-llm.git HEAD
GIT_CURL_VERBOSE=1 git ls-remote https://github.com/apache/tvm.git HEAD
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The GitHub-specific SOCKS5 proxy used in this environment is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;socks5://127.0.0.1:1082
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Clone &lt;code&gt;mlc-llm&lt;/code&gt; recursively:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos

git clone --recursive https://github.com/mlc-ai/mlc-llm.git mlc-llm

cd ~/repos/mlc-llm
git rev-parse HEAD
git submodule status --recursive
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this run, the checked-out &lt;code&gt;mlc-llm&lt;/code&gt; commit was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;d1ea69a87280e821611643958bfec385b62dafd3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A transient network failure occurred while cloning one nested CUTLASS submodule:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;error: RPC failed; curl 56 GnuTLS recv error (-9): Error decoding the received TLS packet.
fatal: early EOF
fatal: fetch-pack: invalid index-pack output
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, Git automatically retried the submodule clone and succeeded. The final &lt;code&gt;git submodule status --recursive&lt;/code&gt; output showed all submodules checked out cleanly, with no leading &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;+&lt;/code&gt;, or &lt;code&gt;U&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Important observation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;is not a direct checkout of &lt;code&gt;apache/tvm.git&lt;/code&gt;. It is pinned by &lt;code&gt;mlc-llm&lt;/code&gt; to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;https://github.com/mlc-ai/relax.git
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The checked-out TVM commit was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;c2028da5ac03055f30a52b492260f37b9650ac03
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For MLC-LLM development, this pinned TVM source tree should be built and used. Building an unrelated &lt;code&gt;apache/tvm&lt;/code&gt; HEAD separately may cause API or ABI mismatches.&lt;/p&gt;
&lt;p&gt;Verify repository cleanliness:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm

git status --short
git rev-parse HEAD

git submodule foreach --recursive &amp;#39;
  echo &amp;quot;----- $name :: $path -----&amp;quot;
  git status --short
  git rev-parse HEAD
&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A clean checkout should produce no &lt;code&gt;git status --short&lt;/code&gt; output in the main repository or in submodules.&lt;/p&gt;
&lt;h3 id=&quot;6-configuring-the-pinned-tvm-source-tree&quot;&gt;6. Configuring the Pinned TVM Source Tree&lt;/h3&gt;
&lt;p&gt;Enter the pinned TVM source tree:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The TVM source tree contains a default CMake configuration file at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;cmake/config.cmake
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, TVM does not automatically read this file from &lt;code&gt;cmake/config.cmake&lt;/code&gt;. Its top-level &lt;code&gt;CMakeLists.txt&lt;/code&gt; loads configuration in the following order:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cmake&quot;&gt;if(EXISTS ${CMAKE_BINARY_DIR}/config.cmake)
  include(${CMAKE_BINARY_DIR}/config.cmake)
else()
  if(EXISTS ${CMAKE_SOURCE_DIR}/config.cmake)
    include(${CMAKE_SOURCE_DIR}/config.cmake)
  endif()
endif()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Therefore, after modifying &lt;code&gt;cmake/config.cmake&lt;/code&gt;, it must be copied into the build directory as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;build/config.cmake
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;61-confirm-cuda-cudnn-and-nccl-state&quot;&gt;6.1 Confirm CUDA, cuDNN, and NCCL State&lt;/h4&gt;
&lt;p&gt;CUDA root:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;readlink -f /usr/local/cuda
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected result in this environment:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/usr/local/cuda-13.2
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check key CUDA libraries:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ls -lh \
  /usr/local/cuda/include/cuda.h \
  /usr/local/cuda/lib64/libcudart.so \
  /usr/local/cuda/lib64/libcublas.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;cuDNN 9 development files were installed through the CUDA repository:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dpkg -l | grep -Ei &amp;#39;cudnn|libcudnn&amp;#39; || true
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Installed packages:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libcudnn9-cuda-13         9.20.0.48-1
libcudnn9-dev-cuda-13     9.20.0.48-1
libcudnn9-headers-cuda-13 9.20.0.48-1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;cuDNN headers and libraries are located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/usr/include/x86_64-linux-gnu/cudnn.h
/usr/include/x86_64-linux-gnu/cudnn_version.h
/usr/lib/x86_64-linux-gnu/libcudnn.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check via:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dpkg -L libcudnn9-headers-cuda-13 libcudnn9-dev-cuda-13 \
  | grep -Ei &amp;#39;include|header|cudnn|\.h$&amp;#39; \
  | sort
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;NCCL was not installed in this environment:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;dpkg -l | grep -Ei &amp;#39;nccl|libnccl&amp;#39; || true
find /usr /opt -type f \( -name &amp;#39;nccl.h&amp;#39; -o -name &amp;#39;libnccl.so*&amp;#39; \) 2&amp;gt;/dev/null | sort
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Because the build policy is source-only and refuses precompiled binary packages, &lt;code&gt;USE_NCCL&lt;/code&gt; was kept &lt;code&gt;OFF&lt;/code&gt; for the TVM build. NCCL can be built later from NVIDIA’s NCCL source tree if required.&lt;/p&gt;
&lt;h4 id=&quot;62-patch-tvm-cmake-configuration&quot;&gt;6.2 Patch TVM CMake Configuration&lt;/h4&gt;
&lt;p&gt;Back up the original TVM CMake configuration:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

cp -av cmake/config.cmake cmake/config.cmake.bak.$(date +%Y%m%d_%H%M%S)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Apply the local configuration:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
from pathlib import Path

p = Path(&amp;quot;cmake/config.cmake&amp;quot;)
s = p.read_text()

repls = {
    &amp;quot;set(USE_CUDA OFF)&amp;quot;: &amp;quot;set(USE_CUDA /usr/local/cuda)&amp;quot;,
    &amp;quot;set(USE_NCCL OFF)&amp;quot;: &amp;quot;set(USE_NCCL OFF)&amp;quot;,
    &amp;quot;set(USE_ROCM OFF)&amp;quot;: &amp;quot;set(USE_ROCM OFF)&amp;quot;,
    &amp;quot;set(USE_OPENCL OFF)&amp;quot;: &amp;quot;set(USE_OPENCL OFF)&amp;quot;,
    &amp;quot;set(USE_METAL OFF)&amp;quot;: &amp;quot;set(USE_METAL OFF)&amp;quot;,
    &amp;quot;set(USE_VULKAN OFF)&amp;quot;: &amp;quot;set(USE_VULKAN OFF)&amp;quot;,
    &amp;quot;set(USE_LLVM OFF)&amp;quot;: &amp;quot;set(USE_LLVM /usr/bin/llvm-config-18)&amp;quot;,
    &amp;quot;set(USE_CUDNN OFF)&amp;quot;: &amp;quot;set(USE_CUDNN ON)&amp;quot;,
    &amp;quot;set(USE_CUDNN_FRONTEND OFF)&amp;quot;: &amp;quot;set(USE_CUDNN_FRONTEND OFF)&amp;quot;,
    &amp;quot;set(USE_CUBLAS OFF)&amp;quot;: &amp;quot;set(USE_CUBLAS ON)&amp;quot;,
    &amp;quot;set(TVM_FFI_USE_LIBBACKTRACE ON)&amp;quot;: &amp;quot;set(TVM_FFI_USE_LIBBACKTRACE ON)&amp;quot;,
    &amp;quot;set(USE_CUTLASS OFF)&amp;quot;: &amp;quot;set(USE_CUTLASS ON)&amp;quot;,
    &amp;quot;set(USE_FLASHINFER OFF)&amp;quot;: &amp;quot;set(USE_FLASHINFER ON)&amp;quot;,
}

missing = []
for old, new in repls.items():
    if old not in s:
        missing.append(old)
    else:
        s = s.replace(old, new)

if missing:
    print(&amp;quot;MISSING_PATTERNS:&amp;quot;)
    for x in missing:
        print(x)
    raise SystemExit(1)

extra = &amp;quot;&amp;quot;&amp;quot;
# ---- Local explicit paths for CUDA/cuDNN on this H800 CUDA 13.2 environment ----
set(CUDNN_INCLUDE_DIR /usr/include/x86_64-linux-gnu)
set(CUDNN_LIBRARY /usr/lib/x86_64-linux-gnu/libcudnn.so)
&amp;quot;&amp;quot;&amp;quot;

if &amp;quot;Local explicit paths for CUDA/cuDNN&amp;quot; not in s:
    s += extra

p.write_text(s)
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify the relevant options:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -nE &amp;#39;USE_CUDA|USE_NCCL|USE_LLVM|USE_CUDNN|USE_CUBLAS|USE_CUTLASS|USE_FLASHINFER|TVM_FFI_USE_LIBBACKTRACE|CUDNN_INCLUDE_DIR|CUDNN_LIBRARY|USE_OPENCL|USE_METAL|USE_ROCM|USE_VULKAN&amp;#39; \
  cmake/config.cmake
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected important entries:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;set(USE_CUDA /usr/local/cuda)
set(USE_NCCL OFF)
set(USE_ROCM OFF)
set(USE_OPENCL OFF)
set(USE_METAL OFF)
set(USE_VULKAN OFF)
set(USE_LLVM /usr/bin/llvm-config-18)
set(USE_CUDNN ON)
set(USE_CUDNN_FRONTEND OFF)
set(USE_CUBLAS ON)
set(TVM_FFI_USE_LIBBACKTRACE ON)
set(USE_CUTLASS ON)
set(USE_FLASHINFER ON)
set(CUDNN_INCLUDE_DIR /usr/include/x86_64-linux-gnu)
set(CUDNN_LIBRARY /usr/lib/x86_64-linux-gnu/libcudnn.so)
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;4-configuring-tvm&quot;&gt;4. Configuring TVM&lt;/h2&gt;
&lt;h3 id=&quot;7-configuring-tvm-for-nvidia-h800--hopper&quot;&gt;7. Configuring TVM for NVIDIA H800 / Hopper&lt;/h3&gt;
&lt;p&gt;The first CMake configure attempt showed that, if &lt;code&gt;cmake/config.cmake&lt;/code&gt; is not copied into the build directory, TVM silently keeps important options disabled:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;USE_CUDA=OFF
USE_LLVM=OFF
USE_CUDNN=OFF
USE_CUBLAS=OFF
USE_CUTLASS=OFF
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The correct procedure is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

rm -rf build
mkdir -p build
cp -av cmake/config.cmake build/config.cmake

cd build

cmake .. \
  -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/gcc \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++ \
  -DCMAKE_CUDA_ARCHITECTURES=90
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The explicit CUDA architecture is important. H800 is a Hopper GPU and should target:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;sm_90 / compute_90
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Without this override, CMake detected or defaulted to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;CMAKE_CUDA_ARCHITECTURES=75
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;which is inappropriate for H800.&lt;/p&gt;
&lt;p&gt;A successful configure should include messages similar to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;-- Custom CUDA_PATH=/usr/local/cuda
-- Found CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda
-- Found CUDA_CUDART_LIBRARY=/usr/local/cuda/lib64/libcudart.so
-- Found CUDA_CUDNN_LIBRARY=/usr/lib/x86_64-linux-gnu/libcudnn.so
-- Found CUDA_CUBLAS_LIBRARY=/usr/local/cuda/lib64/libcublas.so
-- Found CUDA_CUBLASLT_LIBRARY=/usr/local/cuda/lib64/libcublasLt.so
-- Build with CUDA  support
-- Build with cuDNN support
-- Build with cuBLAS support
-- Found CMAKE_CUDA_ARCHITECTURES=90
-- Use llvm-config=/usr/bin/llvm-config-18
-- Found TVM_LLVM_VERSION=181
-- Build with LLVM
-- Build with CUTLASS
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify CUDA architecture in &lt;code&gt;build.ninja&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -nE &amp;#39;compute_90|sm_90&amp;#39; build.ninja | sed -n &amp;#39;1,80p&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected form:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--generate-code=arch=compute_90,code=[compute_90,sm_90]
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;8-patching-libflash_attn-cuda-architecture-from-sm_80-to-sm_90&quot;&gt;8. Patching libflash_attn CUDA Architecture from sm_80 to sm_90&lt;/h3&gt;
&lt;p&gt;After configuring TVM with &lt;code&gt;CMAKE_CUDA_ARCHITECTURES=90&lt;/code&gt;, &lt;code&gt;build.ninja&lt;/code&gt; still contained some &lt;code&gt;compute_80 / sm_80&lt;/code&gt; flags.&lt;/p&gt;
&lt;p&gt;The source was identified as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;3rdparty/libflash_attn/src/CMakeLists.txt
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relevant original content:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-cmake&quot;&gt;set(CMAKE_CUDA_FLAGS &amp;quot;${CMAKE_CUDA_FLAGS} --expt-relaxed-constexpr --expt-relaxed-constexpr --use_fast_math -t 8 \
                      -gencode=arch=compute_80,code=\\\&amp;quot;sm_80,compute_80\\\&amp;quot; \
                      &amp;quot;)

set_target_properties(flash_attn PROPERTIES CUDA_ARCHITECTURES &amp;quot;80&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Patch it for H800 / Hopper:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

cp -av 3rdparty/libflash_attn/src/CMakeLists.txt \
  3rdparty/libflash_attn/src/CMakeLists.txt.bak.h800_sm90.$(date +%Y%m%d_%H%M%S)

python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
from pathlib import Path

p = Path(&amp;quot;3rdparty/libflash_attn/src/CMakeLists.txt&amp;quot;)
s = p.read_text()

repls = {
    &amp;quot;compute_80&amp;quot;: &amp;quot;compute_90&amp;quot;,
    &amp;quot;sm_80&amp;quot;: &amp;quot;sm_90&amp;quot;,
    &amp;#39;CUDA_ARCHITECTURES &amp;quot;80&amp;quot;&amp;#39;: &amp;#39;CUDA_ARCHITECTURES &amp;quot;90&amp;quot;&amp;#39;,
}

for old, new in repls.items():
    if old not in s:
        print(f&amp;quot;WARNING: pattern not found: {old}&amp;quot;)
    s = s.replace(old, new)

p.write_text(s)
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Reconfigure TVM after this patch:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

rm -rf build
mkdir -p build
cp -av cmake/config.cmake build/config.cmake

cd build

cmake .. \
  -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/gcc \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++ \
  -DCMAKE_CUDA_ARCHITECTURES=90
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify that &lt;code&gt;compute_80 / sm_80&lt;/code&gt; no longer appear in generated build rules:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;grep -nE &amp;#39;compute_80|sm_80&amp;#39; build.ninja | sed -n &amp;#39;1,80p&amp;#39; || true
grep -nE &amp;#39;compute_90|sm_90&amp;#39; build.ninja | sed -n &amp;#39;1,80p&amp;#39; || true
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;No compute_80 / sm_80 compile flags remain.
compute_90 / sm_90 compile flags are present.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Note: several source filenames still contain &lt;code&gt;sm80&lt;/code&gt;, for example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flash_fwd_hdim128_fp16_sm80.cu
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is only a filename. The actual NVCC target architecture is controlled by &lt;code&gt;-gencode&lt;/code&gt; and &lt;code&gt;CUDA_ARCHITECTURES&lt;/code&gt;.&lt;/p&gt;
&lt;h2 id=&quot;5-building-tvm-and-its-bindings&quot;&gt;5. Building TVM and its bindings&lt;/h2&gt;
&lt;h3 id=&quot;9-building-tvm-native-libraries&quot;&gt;9. Building TVM Native Libraries&lt;/h3&gt;
&lt;p&gt;Build TVM with Ninja. A conservative parallelism level is used first to keep failures easier to debug:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm/build

ninja -j64
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Successful final link messages include:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;[634/634] Linking CXX shared library libtvm.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Generated libraries include:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;./lib/libtvm_ffi_testing.so
./lib/libtvm_ffi.so
./libtvm_runtime.so
./libtvm.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this run:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libtvm_runtime.so  6.5M
libtvm.so          86M
libtvm_ffi.so      2.0M
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Check linked dependencies:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm/build

ldd ./libtvm.so | grep -Ei &amp;#39;cuda|cudnn|cublas|llvm|stdc\+\+|not found&amp;#39; || true
ldd ./libtvm_runtime.so | grep -Ei &amp;#39;cuda|cudnn|cublas|llvm|stdc\+\+|not found&amp;#39; || true
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected important dependencies for &lt;code&gt;libtvm.so&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libLLVM-18.so.18.1
libcudart.so.13
libcuda.so.1
libcudnn.so.9
libcublas.so.13
libcublasLt.so.13
libstdc++.so.6
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected important dependencies for &lt;code&gt;libtvm_runtime.so&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libcudart.so.13
libcuda.so.1
libcudnn.so.9
libcublas.so.13
libcublasLt.so.13
libstdc++.so.6
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No &lt;code&gt;not found&lt;/code&gt; entries should appear.&lt;/p&gt;
&lt;h3 id=&quot;10-building-and-installing-apache-tvm-ffi-from-source&quot;&gt;10. Building and Installing apache-tvm-ffi from Source&lt;/h3&gt;
&lt;p&gt;After TVM native libraries were built, directly importing TVM from source initially failed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;ModuleNotFoundError: No module named &amp;#39;tvm_ffi&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The correct &lt;code&gt;tvm_ffi&lt;/code&gt; source path is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;After adding that path, import proceeded further but failed because package metadata was missing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;importlib.metadata.PackageNotFoundError: No package metadata was found for apache-tvm-ffi
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;101-local-development-patch-for-tvm_ffilibinfopy&quot;&gt;10.1 Local Development Patch for tvm_ffi/libinfo.py&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;tvm_ffi/libinfo.py&lt;/code&gt; attempted to call:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;dist = im.distribution(package)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;before entering its fallback search logic. In a source-tree development build, &lt;code&gt;apache-tvm-ffi&lt;/code&gt; package metadata may not exist yet, even though &lt;code&gt;libtvm_ffi.so&lt;/code&gt; has already been built.&lt;/p&gt;
&lt;p&gt;Patch &lt;code&gt;3rdparty/tvm-ffi/python/tvm_ffi/libinfo.py&lt;/code&gt; so that &lt;code&gt;PackageNotFoundError&lt;/code&gt; falls through to the existing fallback search:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

cp -av 3rdparty/tvm-ffi/python/tvm_ffi/libinfo.py \
  3rdparty/tvm-ffi/python/tvm_ffi/libinfo.py.bak.dev_fallback.$(date +%Y%m%d_%H%M%S)

python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
from pathlib import Path

p = Path(&amp;quot;3rdparty/tvm-ffi/python/tvm_ffi/libinfo.py&amp;quot;)
s = p.read_text()

old = &amp;#39;&amp;#39;&amp;#39;    # Use `importlib.metadata` is the most reliable way to find package data files
    dist: im.PathDistribution = im.distribution(package)  # ty: ignore[invalid-assignment]
    record = dist.read_text(&amp;quot;RECORD&amp;quot;) or &amp;quot;&amp;quot;
    for line in record.splitlines():
        partial_path, *_ = line.split(&amp;quot;,&amp;quot;)
        if partial_path.endswith(lib_dll_names):
            try:
                path = (dist._path.parent / partial_path).resolve()
            except OSError:
                continue
            if path.name in lib_dll_names:
                return path
&amp;#39;&amp;#39;&amp;#39;

new = &amp;#39;&amp;#39;&amp;#39;    # Use `importlib.metadata` is the most reliable way to find package data files.
    # In a source-tree development build, package metadata may not exist yet.
    # In that case, continue to the fallback search below.
    try:
        dist: im.PathDistribution = im.distribution(package)  # ty: ignore[invalid-assignment]
    except im.PackageNotFoundError:
        dist = None

    if dist is not None:
        record = dist.read_text(&amp;quot;RECORD&amp;quot;) or &amp;quot;&amp;quot;
        for line in record.splitlines():
            partial_path, *_ = line.split(&amp;quot;,&amp;quot;)
            if partial_path.endswith(lib_dll_names):
                try:
                    path = (dist._path.parent / partial_path).resolve()
                except OSError:
                    continue
                if path.name in lib_dll_names:
                    return path
&amp;#39;&amp;#39;&amp;#39;

if old not in s:
    raise SystemExit(&amp;quot;ERROR: target block not found; do not patch blindly&amp;quot;)

p.write_text(s.replace(old, new))
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This patch allows &lt;code&gt;tvm_ffi&lt;/code&gt; to continue searching:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;$PROJECT_ROOT/build/lib
$PROJECT_ROOT/lib
$DEV_TOP/build/lib
$DEV_TOP/lib
LD_LIBRARY_PATH
PATH
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;instead of failing immediately when Python package metadata is absent.&lt;/p&gt;
&lt;h4 id=&quot;102-install-python-build-tools-from-source&quot;&gt;10.2 Install Python Build Tools from Source&lt;/h4&gt;
&lt;p&gt;&lt;code&gt;tvm_ffi.core&lt;/code&gt; is a Cython native extension. It is not produced by the TVM C++ Ninja build alone.&lt;/p&gt;
&lt;p&gt;The source files include:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;3rdparty/tvm-ffi/python/tvm_ffi/cython/core.pyx
3rdparty/tvm-ffi/python/tvm_ffi/core.pyi
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The required Python build tools are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;scikit-build-core
Cython
setuptools-scm
setuptools
ninja
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this environment, &lt;code&gt;scikit-build-core&lt;/code&gt;, &lt;code&gt;tomli&lt;/code&gt;, &lt;code&gt;packaging&lt;/code&gt;, and &lt;code&gt;ninja&lt;/code&gt; were already installed in user-site, but &lt;code&gt;Cython&lt;/code&gt; and &lt;code&gt;setuptools-scm&lt;/code&gt; were missing.&lt;/p&gt;
&lt;p&gt;Correct import name for scikit-build-core:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;import scikit_build_core
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;not:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;import skbuild_core
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify Python build tools:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
mods = [
    &amp;quot;scikit_build_core&amp;quot;,
    &amp;quot;Cython&amp;quot;,
    &amp;quot;setuptools_scm&amp;quot;,
    &amp;quot;tomli&amp;quot;,
    &amp;quot;packaging&amp;quot;,
    &amp;quot;setuptools&amp;quot;,
    &amp;quot;wheel&amp;quot;,
    &amp;quot;ninja&amp;quot;,
]
for m in mods:
    try:
        mod = __import__(m)
        print(f&amp;quot;{m}: OK -&amp;gt; {getattr(mod, &amp;#39;__version__&amp;#39;, &amp;#39;unknown&amp;#39;)} -&amp;gt; {getattr(mod, &amp;#39;__file__&amp;#39;, &amp;#39;built-in&amp;#39;)}&amp;quot;)
    except Exception as e:
        print(f&amp;quot;{m}: MISSING/ERROR -&amp;gt; {type(e).__name__}: {e}&amp;quot;)
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Upgrade &lt;code&gt;setuptools&lt;/code&gt; from source:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m pip install \
  --user \
  --no-binary=:all: \
  --upgrade \
  &amp;quot;setuptools&amp;gt;=77.0.3&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In this run, &lt;code&gt;setuptools&lt;/code&gt; was built from source and installed as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;setuptools 82.0.1
/home/ailven.liu/.local/lib/python3.10/site-packages/setuptools
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Warning observed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;torch 2.11.0 requires setuptools&amp;lt;82, but you have setuptools 82.0.1 which is incompatible.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This warning was not blocking for TVM/tvm-ffi. If later Torch-related tooling fails, isolate this stack in a virtual environment or adjust &lt;code&gt;setuptools&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Install Cython from source:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m pip install \
  --user \
  --no-binary=:all: \
  --upgrade \
  &amp;quot;cython&amp;gt;=3.0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
import Cython
print(&amp;quot;Cython:&amp;quot;, Cython.__version__, Cython.__file__)
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected result in this run:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Cython 3.2.4
/home/ailven.liu/.local/lib/python3.10/site-packages/Cython
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Installing &lt;code&gt;setuptools-scm&lt;/code&gt; directly from source initially failed because its build isolation environment lacked &lt;code&gt;tomli&lt;/code&gt; while building &lt;code&gt;vcs-versioning&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The successful procedure was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m pip install \
  --user \
  --no-binary=:all: \
  --no-build-isolation \
  --upgrade \
  &amp;quot;vcs-versioning&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Then:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m pip install \
  --user \
  --no-binary=:all: \
  --no-build-isolation \
  --upgrade \
  &amp;quot;setuptools-scm&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
import vcs_versioning
import setuptools_scm

print(&amp;quot;vcs_versioning:&amp;quot;, getattr(vcs_versioning, &amp;quot;__version__&amp;quot;, &amp;quot;unknown&amp;quot;), vcs_versioning.__file__)
print(&amp;quot;setuptools_scm:&amp;quot;, getattr(setuptools_scm, &amp;quot;__version__&amp;quot;, &amp;quot;unknown&amp;quot;), setuptools_scm.__file__)
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;103-editable-source-build-of-apache-tvm-ffi&quot;&gt;10.3 Editable Source Build of apache-tvm-ffi&lt;/h4&gt;
&lt;p&gt;Build and install &lt;code&gt;apache-tvm-ffi&lt;/code&gt; from the local source tree in editable mode:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi

export LD_LIBRARY_PATH=$PWD/../../build/lib:$PWD/../../build:/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}

python3 -m pip install \
  --user \
  --no-build-isolation \
  --no-binary=:all: \
  -e .
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Although pip reports a locally created wheel during editable installation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Created wheel for apache-tvm-ffi: ...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;this is a PEP 517 editable-install build artefact produced from the local source tree. It is not a downloaded precompiled binary package.&lt;/p&gt;
&lt;p&gt;Verify:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
import tvm_ffi
import tvm_ffi.core
import importlib.metadata as im

print(&amp;quot;tvm_ffi:&amp;quot;, tvm_ffi.__file__)
print(&amp;quot;tvm_ffi.core:&amp;quot;, tvm_ffi.core.__file__)
print(&amp;quot;dist version:&amp;quot;, im.version(&amp;quot;apache-tvm-ffi&amp;quot;))
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Successful result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm_ffi: /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python/tvm_ffi/__init__.py
tvm_ffi.core: /home/ailven.liu/.local/lib/python3.10/site-packages/tvm_ffi/core.cpython-310-x86_64-linux-gnu.so
dist version: 0.1.11.dev0+g1fed0ae04.d20260505
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;11-verifying-tvm-python-binding&quot;&gt;11. Verifying TVM Python Binding&lt;/h3&gt;
&lt;p&gt;Set the TVM environment for the current shell:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/mlc-llm/3rdparty/tvm

export TVM_HOME=$PWD
export PYTHONPATH=$TVM_HOME/python
export LD_LIBRARY_PATH=$TVM_HOME/build:$TVM_HOME/build/lib:/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify &lt;code&gt;tvm&lt;/code&gt;, &lt;code&gt;tvm_ffi&lt;/code&gt;, and CUDA visibility:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 - &amp;lt;&amp;lt;&amp;#39;PY&amp;#39;
import os
import tvm
import tvm_ffi
import tvm_ffi.core

print(&amp;quot;tvm module:&amp;quot;, tvm.__file__)
print(&amp;quot;tvm_ffi module:&amp;quot;, tvm_ffi.__file__)
print(&amp;quot;tvm_ffi.core:&amp;quot;, tvm_ffi.core.__file__)

from tvm.support import libinfo
info = libinfo()
for k in sorted(info):
    if k.startswith(&amp;quot;USE_&amp;quot;) or k in (&amp;quot;GIT_COMMIT_HASH&amp;quot;, &amp;quot;LLVM_VERSION&amp;quot;):
        print(f&amp;quot;{k}={info[k]}&amp;quot;)

print(&amp;quot;cuda exist:&amp;quot;, tvm.cuda().exist)
print(&amp;quot;cuda device:&amp;quot;, tvm.cuda(0))
PY
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Successful result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm module: /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/python/tvm/__init__.py
tvm_ffi module: /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python/tvm_ffi/__init__.py
tvm_ffi.core: /home/ailven.liu/.local/lib/python3.10/site-packages/tvm_ffi/core.cpython-310-x86_64-linux-gnu.so

GIT_COMMIT_HASH=c2028da5ac03055f30a52b492260f37b9650ac03
LLVM_VERSION=18.1.8
USE_CUBLAS=ON
USE_CUDA=/usr/local/cuda
USE_CUDNN=ON
USE_CUTLASS=ON
USE_LLVM=/usr/bin/llvm-config-18
USE_NCCL=OFF
USE_RPC=ON
USE_THREADS=ON

cuda exist: True
cuda device: cuda:0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point, the source-built TVM stack is functional from Python and can see the CUDA device.&lt;/p&gt;
&lt;h3 id=&quot;12-persisting-the-tvm-source-environment&quot;&gt;12. Persisting the TVM Source Environment&lt;/h3&gt;
&lt;p&gt;To avoid manually exporting environment variables every time, create a sourceable environment script:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cat &amp;gt; ~/repos/mlc-llm/env-tvm-source.sh &amp;lt;&amp;lt;&amp;#39;EOF&amp;#39;
# Source-built TVM environment for this mlc-llm checkout.
# Usage:
#   source ~/repos/mlc-llm/env-tvm-source.sh

export MLC_LLM_HOME=&amp;quot;$HOME/repos/mlc-llm&amp;quot;
export TVM_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/tvm&amp;quot;

export PYTHONPATH=&amp;quot;$TVM_HOME/python:$MLC_LLM_HOME/python:${PYTHONPATH:-}&amp;quot;
export LD_LIBRARY_PATH=&amp;quot;$TVM_HOME/build:$TVM_HOME/build/lib:/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}&amp;quot;
export PATH=&amp;quot;$HOME/.local/bin:/usr/local/cuda/bin:${PATH:-}&amp;quot;
EOF

chmod +x ~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Use it with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verify from a fresh subshell:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;bash -lc &amp;#39;
  source ~/repos/mlc-llm/env-tvm-source.sh
  python3 - &amp;lt;&amp;lt;PY
import tvm
print(&amp;quot;tvm:&amp;quot;, tvm.__file__)
print(&amp;quot;cuda:&amp;quot;, tvm.cuda().exist)
PY
&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Expected result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm: /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/python/tvm/__init__.py
cuda: True
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;6-building-the-remaining-dependencies&quot;&gt;6. Building the remaining dependencies&lt;/h2&gt;
&lt;h3 id=&quot;13-continuing-from-tvm-to-xgrammar&quot;&gt;13. Continuing from TVM to XGrammar&lt;/h3&gt;
&lt;p&gt;After the source-built TVM stack had been verified, the next stage was to build and install XGrammar from the &lt;code&gt;mlc-llm&lt;/code&gt; pinned submodule.&lt;/p&gt;
&lt;p&gt;The XGrammar source tree is located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/xgrammar
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The checked-out XGrammar commit in this environment is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;d4f57c440f3da8e7330a1e5d50bba9c31f9433ea
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Its &lt;code&gt;dlpack&lt;/code&gt; submodule was already initialised:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;3rdparty/dlpack: bbd2f4d32427e548797929af08cfe2a9cbb3cf12
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The relevant build entry points are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/xgrammar/CMakeLists.txt
~/repos/mlc-llm/3rdparty/xgrammar/cpp/pybind/CMakeLists.txt
~/repos/mlc-llm/3rdparty/xgrammar/python/setup.py
~/repos/mlc-llm/3rdparty/xgrammar/python/pyproject.toml
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A key observation is that &lt;code&gt;python/setup.py&lt;/code&gt; does &lt;strong&gt;not&lt;/strong&gt; invoke CMake by itself. It expects the native binding library to have already been built and placed under:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.*.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Therefore, the correct build order is:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;configure and build the native C++/pybind11 extension with CMake/Ninja;&lt;/li&gt;
&lt;li&gt;install the Python package from the local source tree.&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The available toolchain used for XGrammar was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;cmake: 3.22.1
ninja: 1.13.0.git.kitware.jobserver-pipe-1
gcc/g++: 11.4.0
Python: 3.10.12
pybind11: 3.0.4
torch: 2.11.0+cu130
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The Torch CMake prefix path was available at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/.local/lib/python3.10/site-packages/torch/share/cmake
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The pybind11 CMake directory was available at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/.local/lib/python3.10/site-packages/pybind11/share/cmake/pybind11
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;131-configuring-xgrammar&quot;&gt;13.1 Configuring XGrammar&lt;/h4&gt;
&lt;p&gt;Configure XGrammar with CMake:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/xgrammar

rm -rf build
mkdir -p build

cmake -S . -B build \
  -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/gcc \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The configure stage completed successfully and generated Ninja build files under:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/xgrammar/build
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;During configuration, CMake emitted the following non-blocking Torch warning:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;library kineto not found
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This warning did not prevent configuration. Torch was still found successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Found Torch: ~/.local/lib/python3.10/site-packages/torch/lib/libtorch.so
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;132-building-xgrammar-native-binding&quot;&gt;13.2 Building XGrammar Native Binding&lt;/h4&gt;
&lt;p&gt;Build the native binding:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/xgrammar

cmake --build build -j64
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The build completed successfully and produced:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.cpython-310-x86_64-linux-gnu.so
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;133-installing-xgrammar-from-local-source&quot;&gt;13.3 Installing XGrammar from Local Source&lt;/h4&gt;
&lt;p&gt;Install the Python package from the local source tree in editable mode:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/xgrammar/python

python3 -m pip install \
  --user \
  --no-build-isolation \
  --no-binary=:all: \
  -e .
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The installation completed successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Successfully built xgrammar
Successfully installed xgrammar-0.0.3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Although pip created an editable wheel locally, this was a local PEP 660 editable-install artefact, not a downloaded precompiled binary package:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;xgrammar-0.0.3-0.editable-cp310-cp310-linux_x86_64.whl
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The final import locations were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;xgrammar:
~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/__init__.py

xgrammar_bindings:
~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.cpython-310-x86_64-linux-gnu.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The visible public API included:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;BNFGrammar
BuiltinGrammar
CachedGrammarCompiler
CompiledGrammar
GrammarMatcher
TokenizerInfo
VocabType
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point, XGrammar was successfully built and installed from local source.&lt;/p&gt;
&lt;h3 id=&quot;14-updating-the-source-environment-script-for-xgrammar&quot;&gt;14. Updating the Source Environment Script for XGrammar&lt;/h3&gt;
&lt;p&gt;The environment script was extended so that XGrammar is explicitly part of the source development environment.&lt;/p&gt;
&lt;p&gt;The script is located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following variable was added:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;export XGRAMMAR_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/xgrammar&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;PYTHONPATH&lt;/code&gt; was updated to include:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;$XGRAMMAR_HOME/python
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The relevant environment entries became:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;export MLC_LLM_HOME=&amp;quot;$HOME/repos/mlc-llm&amp;quot;
export TVM_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/tvm&amp;quot;
export XGRAMMAR_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/xgrammar&amp;quot;

export PYTHONPATH=&amp;quot;$TVM_HOME/python:$MLC_LLM_HOME/python:$XGRAMMAR_HOME/python:${PYTHONPATH:-}&amp;quot;
export LD_LIBRARY_PATH=&amp;quot;$TVM_HOME/build:$TVM_HOME/build/lib:/usr/local/cuda/lib64:${LD_LIBRARY_PATH:-}&amp;quot;
export PATH=&amp;quot;$HOME/.local/bin:/usr/local/cuda/bin:${PATH:-}&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A fresh shell confirmed that the source-built TVM, XGrammar, and CUDA visibility remained valid.&lt;/p&gt;
&lt;h3 id=&quot;15-flashinfer-source-state-in-this-checkout&quot;&gt;15. FlashInfer Source State in This Checkout&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;mlc-llm/3rdparty&lt;/code&gt; directory in this checkout does &lt;strong&gt;not&lt;/strong&gt; contain an independent &lt;code&gt;flashinfer&lt;/code&gt; source tree.&lt;/p&gt;
&lt;p&gt;The available immediate third-party directories were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;argparse
googletest
stb
tokenizers-cpp
tvm
xgrammar
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The only FlashInfer or FlashAttention-related source discovered in the current checkout was under the pinned TVM source tree:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/3rdparty/libflash_attn
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The TVM build had already incorporated this internal flash attention component, and the MLC-LLM native libraries later linked against:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libflash_attn.so
libfpA_intB_gemm.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, the MLC-LLM Python package metadata declares a Linux dependency on:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This appears in:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/pyproject.toml
~/repos/mlc-llm/python/requirements.txt
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The relevant dependency entry is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python; sys_platform == &amp;#39;linux&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The current environment did not have &lt;code&gt;flashinfer-python&lt;/code&gt; installed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python: NOT_FOUND
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The MLC-LLM Python source tree did not contain a direct Python import of FlashInfer, such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;import flashinfer
from flashinfer import ...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Instead, FlashInfer references were concentrated in compiler/runtime paths and optimisation flags, including:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;python/mlc_llm/compiler_pass
python/mlc_llm/interface/compiler_flags.py
python/mlc_llm/op/extern.py
python/mlc_llm/op/attention.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Therefore, &lt;code&gt;flashinfer-python&lt;/code&gt; was deliberately not installed at this stage. To avoid downloading a precompiled wheel, MLC-LLM itself was later installed with dependency resolution disabled.&lt;/p&gt;
&lt;p&gt;For model compilation before building &lt;code&gt;flashinfer-python&lt;/code&gt; from source, FlashInfer can be explicitly disabled via the optimisation string, for example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;--opt &amp;quot;O2;flashinfer=0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;16-building-tokenizers-cpp-from-source&quot;&gt;16. Building tokenizers-cpp from Source&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;tokenizers-cpp&lt;/code&gt; source tree is located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tokenizers-cpp
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The checked-out commit was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;34885cfd7b9ef27b859c28a41e71413dd31926f5
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following submodules were already initialised:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;msgpack:       092bc69b6e815980bce7808595c914dd3a29f905
sentencepiece: 11051e3b73b3a6222a52acd720e39805dc7545ab
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The project is a CMake-based C++ library with a Rust/Cargo component. It produces static libraries rather than a Python package.&lt;/p&gt;
&lt;p&gt;Important build characteristics:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;CMake project: tokenizers_cpp
Rust crate: tokenizers-c
Rust crate type: staticlib
SentencePiece tokenizer: enabled
msgpack: local submodule
sentencepiece: local submodule
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The Rust dependency declaration is located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tokenizers-cpp/rust/Cargo.toml
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The key dependencies were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-toml&quot;&gt;tokenizers = { version = &amp;quot;0.21.2&amp;quot;, default-features = false, features = [&amp;quot;onig&amp;quot;] }
serde = { version = &amp;quot;1.0&amp;quot;, features = [ &amp;quot;derive&amp;quot; ] }
serde_json = &amp;quot;1.0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The Rust toolchain available in this environment was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;rustc 1.95.0
cargo 1.95.0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Cargo was configured to use the USTC crates.io mirror:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-toml&quot;&gt;[source.crates-io]
replace-with = &amp;quot;ustc&amp;quot;

[source.ustc]
registry = &amp;quot;https://mirrors.ustc.edu.cn/crates.io-index&amp;quot;

[net]
git-fetch-with-cli = true
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;161-generating-cargolock&quot;&gt;16.1 Generating Cargo.lock&lt;/h4&gt;
&lt;p&gt;The repository did not include &lt;code&gt;rust/Cargo.lock&lt;/code&gt;. Generate it first:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/tokenizers-cpp/rust

cargo generate-lockfile
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This successfully generated:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tokenizers-cpp/rust/Cargo.lock
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Cargo locked 78 packages. The resolved &lt;code&gt;tokenizers&lt;/code&gt; crate version was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tokenizers v0.21.4
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The generated &lt;code&gt;Cargo.lock&lt;/code&gt; is ignored by the repository’s Git configuration and appears as an ignored file:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;!! rust/Cargo.lock
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;162-configuring-tokenizers-cpp&quot;&gt;16.2 Configuring tokenizers-cpp&lt;/h4&gt;
&lt;p&gt;Configure the build:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/tokenizers-cpp

rm -rf build
mkdir -p build

cmake -S . -B build \
  -G Ninja \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_COMPILER=/usr/bin/gcc \
  -DCMAKE_CXX_COMPILER=/usr/bin/g++ \
  -DMLC_ENABLE_SENTENCEPIECE_TOKENIZER=ON
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The configure stage completed successfully and wrote build files to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tokenizers-cpp/build
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;163-building-tokenizers-cpp&quot;&gt;16.3 Building tokenizers-cpp&lt;/h4&gt;
&lt;p&gt;Build the static libraries:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm/3rdparty/tokenizers-cpp

cmake --build build -j64
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The build completed successfully and generated:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/libtokenizers_c.a
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/libtokenizers_cpp.a
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/release/libtokenizers_c.a
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/sentencepiece/src/libsentencepiece.a
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point, &lt;code&gt;tokenizers-cpp&lt;/code&gt; had been successfully built from local source.&lt;/p&gt;
&lt;h2 id=&quot;7-installing-and-validating-mlc-llm&quot;&gt;7. Installing and validating MLC-LLM&lt;/h2&gt;
&lt;h3 id=&quot;17-installing-mlc-llm-from-local-source&quot;&gt;17. Installing MLC-LLM from Local Source&lt;/h3&gt;
&lt;p&gt;After TVM, &lt;code&gt;apache-tvm-ffi&lt;/code&gt;, XGrammar, and &lt;code&gt;tokenizers-cpp&lt;/code&gt; had been built, MLC-LLM itself was installed from the local checkout.&lt;/p&gt;
&lt;p&gt;The source tree is located at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The checked-out MLC-LLM commit was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;d1ea69a87280e821611643958bfec385b62dafd3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The project metadata declares the following relevant dependencies:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache-tvm-ffi
flashinfer-python; sys_platform == &amp;#39;linux&amp;#39;
requests
torch
tqdm
transformers
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Since &lt;code&gt;flashinfer-python&lt;/code&gt; was not available as a local source tree and should not be pulled as a prebuilt wheel, MLC-LLM was installed with dependency resolution disabled:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh

cd ~/repos/mlc-llm

python3 -m pip install \
  --user \
  --no-build-isolation \
  --no-binary=:all: \
  --no-deps \
  -e .
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The installation completed successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Successfully built mlc_llm
Successfully installed mlc_llm-0.20.0.dev0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The editable wheel created during installation was a local build artefact:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;mlc_llm-0.20.0.dev0-py3-none-linux_x86_64.whl
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No &lt;code&gt;flashinfer-python&lt;/code&gt; wheel was downloaded.&lt;/p&gt;
&lt;p&gt;The installed package was confirmed to point back to the local source tree:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;mlc_llm:
~/repos/mlc-llm/python/mlc_llm/__init__.py

distribution:
mlc_llm 0.20.0.dev0
editable: true
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The current editable install path did not generate a &lt;code&gt;mlc_llm&lt;/code&gt; console script under:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/.local/bin
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, the module CLI is available and should be used as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm ...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm --help
python3 -m mlc_llm compile --help
python3 -m mlc_llm chat --help
python3 -m mlc_llm serve --help
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;18-mlc-llm-native-runtime-libraries&quot;&gt;18. MLC-LLM Native Runtime Libraries&lt;/h3&gt;
&lt;p&gt;The MLC-LLM native runtime libraries were present under the local build directory:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/build/libmlc_llm.so
~/repos/mlc-llm/build/libmlc_llm_module.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;They were discoverable through &lt;code&gt;mlc_llm.libinfo&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;mlc_llm:
~/repos/mlc-llm/build/libmlc_llm.so

mlc_llm_module:
~/repos/mlc-llm/build/libmlc_llm_module.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The dynamic linker resolved their dependencies successfully, including:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libtvm.so
libtvm_runtime.so
libtvm_ffi.so
libflash_attn.so
libfpA_intB_gemm.so
libcudart.so.13
libcuda.so.1
libcudnn.so.9
libcublas.so.13
libcublasLt.so.13
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No missing shared-library dependency was observed.&lt;/p&gt;
&lt;p&gt;Representative resolved paths included:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;libtvm.so:
~/repos/mlc-llm/3rdparty/tvm/build/libtvm.so

libtvm_runtime.so:
~/repos/mlc-llm/3rdparty/tvm/build/libtvm_runtime.so

libtvm_ffi.so:
~/repos/mlc-llm/3rdparty/tvm/build/lib/libtvm_ffi.so

libflash_attn.so:
~/repos/mlc-llm/3rdparty/tvm/build/3rdparty/libflash_attn/src/libflash_attn.so

libfpA_intB_gemm.so:
~/repos/mlc-llm/3rdparty/tvm/build/3rdparty/cutlass_fpA_intB_gemm/cutlass_kernels/libfpA_intB_gemm.so
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;19-mlc-llm-python-and-runtime-smoke-validation&quot;&gt;19. MLC-LLM Python and Runtime Smoke Validation&lt;/h3&gt;
&lt;p&gt;The following Python modules loaded successfully from the local source-built stack:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm
tvm_ffi
tvm_ffi.core
xgrammar
xgrammar.xgrammar_bindings
mlc_llm
mlc_llm.base
mlc_llm.support
mlc_llm.interface.compiler_flags
mlc_llm.interface.compile
mlc_llm.op.extern
mlc_llm.op.attention
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The resolved source paths were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm:
~/repos/mlc-llm/3rdparty/tvm/python/tvm/__init__.py

tvm_ffi:
~/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python/tvm_ffi/__init__.py

tvm_ffi.core:
~/.local/lib/python3.10/site-packages/tvm_ffi/core.cpython-310-x86_64-linux-gnu.so

xgrammar:
~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/__init__.py

xgrammar_bindings:
~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.cpython-310-x86_64-linux-gnu.so

mlc_llm:
~/repos/mlc-llm/python/mlc_llm/__init__.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;CUDA was visible through TVM:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;cuda: True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The MLC-LLM module CLI loaded successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm --help
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following subcommands also loaded their help pages successfully without traceback:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;compile
convert_weight
gen_config
chat
serve
package
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;compile&lt;/code&gt; command exposed the expected model compilation interface, including:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--quantization
--model-type
--device
--host
--opt
--output
--overrides
--debug-dump
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The compiler option string can be used to control optimisation flags, for example:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;--opt &amp;quot;O2;flashinfer=0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is useful while &lt;code&gt;flashinfer-python&lt;/code&gt; remains intentionally uninstalled.&lt;/p&gt;
&lt;h3 id=&quot;20-preventing-environment-variable-duplication&quot;&gt;20. Preventing Environment Variable Duplication&lt;/h3&gt;
&lt;p&gt;The first version of &lt;code&gt;env-tvm-source.sh&lt;/code&gt; prepended paths directly. Repeatedly sourcing it caused &lt;code&gt;PATH&lt;/code&gt;, &lt;code&gt;PYTHONPATH&lt;/code&gt;, and &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt; to grow with duplicate entries.&lt;/p&gt;
&lt;p&gt;The script was rewritten to be idempotent.&lt;/p&gt;
&lt;p&gt;The final environment script is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Source-built MLC-LLM environment for this checkout.
# Usage:
#   source ~/repos/mlc-llm/env-tvm-source.sh
#
# This script is intentionally sourceable and idempotent.
# It does not modify shell startup files.

export MLC_LLM_HOME=&amp;quot;$HOME/repos/mlc-llm&amp;quot;
export TVM_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/tvm&amp;quot;
export XGRAMMAR_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/xgrammar&amp;quot;
export TOKENIZERS_CPP_HOME=&amp;quot;$MLC_LLM_HOME/3rdparty/tokenizers-cpp&amp;quot;

_path_prepend_once() {
  case &amp;quot;:${PATH:-}:&amp;quot; in
    *&amp;quot;:$1:&amp;quot;*) ;;
    *) export PATH=&amp;quot;$1:${PATH:-}&amp;quot; ;;
  esac
}

_pythonpath_prepend_once() {
  case &amp;quot;:${PYTHONPATH:-}:&amp;quot; in
    *&amp;quot;:$1:&amp;quot;*) ;;
    *) export PYTHONPATH=&amp;quot;$1:${PYTHONPATH:-}&amp;quot; ;;
  esac
}

_ldpath_prepend_once() {
  case &amp;quot;:${LD_LIBRARY_PATH:-}:&amp;quot; in
    *&amp;quot;:$1:&amp;quot;*) ;;
    *) export LD_LIBRARY_PATH=&amp;quot;$1:${LD_LIBRARY_PATH:-}&amp;quot; ;;
  esac
}

_pythonpath_prepend_once &amp;quot;$XGRAMMAR_HOME/python&amp;quot;
_pythonpath_prepend_once &amp;quot;$MLC_LLM_HOME/python&amp;quot;
_pythonpath_prepend_once &amp;quot;$TVM_HOME/python&amp;quot;

_ldpath_prepend_once &amp;quot;$TVM_HOME/build/lib&amp;quot;
_ldpath_prepend_once &amp;quot;$TVM_HOME/build&amp;quot;
_ldpath_prepend_once &amp;quot;/usr/local/cuda/lib64&amp;quot;

_path_prepend_once &amp;quot;/usr/local/cuda/bin&amp;quot;
_path_prepend_once &amp;quot;$HOME/.local/bin&amp;quot;

unset -f _path_prepend_once
unset -f _pythonpath_prepend_once
unset -f _ldpath_prepend_once
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The script passed shell syntax validation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;bash -n ~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Repeatedly sourcing the script no longer duplicates the relevant paths.&lt;/p&gt;
&lt;p&gt;The final effective variables are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MLC_LLM_HOME=~/repos/mlc-llm
TVM_HOME=~/repos/mlc-llm/3rdparty/tvm
XGRAMMAR_HOME=~/repos/mlc-llm/3rdparty/xgrammar
TOKENIZERS_CPP_HOME=~/repos/mlc-llm/3rdparty/tokenizers-cpp
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The script remains local and does not modify &lt;code&gt;.bashrc&lt;/code&gt; or any system-wide shell configuration.&lt;/p&gt;
&lt;h3 id=&quot;21-verifying-that-no-critical-prebuilt-package-was-accidentally-installed&quot;&gt;21. Verifying That No Critical Prebuilt Package Was Accidentally Installed&lt;/h3&gt;
&lt;p&gt;The key installed Python distributions were:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache-tvm-ffi:
0.1.11.dev0+g1fed0ae04.d20260505
editable local source:
~/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi

xgrammar:
0.0.3
editable local source:
~/repos/mlc-llm/3rdparty/xgrammar/python

mlc_llm:
0.20.0.dev0
editable local source:
~/repos/mlc-llm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following packages were not installed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python
apache-tvm
tlcpack
mlc-ai-nightly
mlc-llm-nightly
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This confirms that the stack did not accidentally switch to an unrelated TVM wheel or MLC nightly package.&lt;/p&gt;
&lt;h2 id=&quot;8-local-modifications-and-build-state&quot;&gt;8. Local modifications and build state&lt;/h2&gt;
&lt;h3 id=&quot;22-local-source-modifications-introduced-during-the-build&quot;&gt;22. Local Source Modifications Introduced During the Build&lt;/h3&gt;
&lt;p&gt;The build required several local source changes. These changes are intentional and should be preserved or documented before any reset or clean checkout.&lt;/p&gt;
&lt;h4 id=&quot;221-tvm-cmake-configuration&quot;&gt;22.1 TVM CMake Configuration&lt;/h4&gt;
&lt;p&gt;Modified file:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/cmake/config.cmake
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The important local changes are:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;USE_CUDA=/usr/local/cuda
USE_LLVM=/usr/bin/llvm-config-18
USE_CUDNN=ON
USE_CUBLAS=ON
USE_CUTLASS=ON
USE_FLASHINFER=ON
CUDNN_INCLUDE_DIR=/usr/include/x86_64-linux-gnu
CUDNN_LIBRARY=/usr/lib/x86_64-linux-gnu/libcudnn.so
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;222-libflash_attn-h800-patch&quot;&gt;22.2 libflash_attn H800 Patch&lt;/h4&gt;
&lt;p&gt;Modified file:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/3rdparty/libflash_attn/src/CMakeLists.txt
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The local patch changed the hard-coded Ampere target to Hopper:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;compute_80 / sm_80 -&amp;gt; compute_90 / sm_90
CUDA_ARCHITECTURES &amp;quot;80&amp;quot; -&amp;gt; &amp;quot;90&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This was required for the H800 environment.&lt;/p&gt;
&lt;h4 id=&quot;223-tvm-ffi-source-development-fallback-patch&quot;&gt;22.3 tvm-ffi Source-Development Fallback Patch&lt;/h4&gt;
&lt;p&gt;Modified file:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python/tvm_ffi/libinfo.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The patch allows &lt;code&gt;apache-tvm-ffi&lt;/code&gt; to continue to its existing fallback search if &lt;code&gt;importlib.metadata.distribution(&amp;quot;apache-tvm-ffi&amp;quot;)&lt;/code&gt; raises &lt;code&gt;PackageNotFoundError&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;The purpose is to support a source-tree development build where the native library already exists but Python package metadata may not yet exist.&lt;/p&gt;
&lt;h4 id=&quot;224-local-backup-files&quot;&gt;22.4 Local Backup Files&lt;/h4&gt;
&lt;p&gt;The following local backup files were created during patching:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/3rdparty/libflash_attn/src/CMakeLists.txt.bak.h800_sm90.20260505_072157
~/repos/mlc-llm/3rdparty/tvm/cmake/config.cmake.bak.20260505_071332
~/repos/mlc-llm/env-tvm-source.sh.bak.20260505_090200
~/repos/mlc-llm/env-tvm-source.sh.bak.idempotent.20260505_085858
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are local recovery artefacts and are not required for normal runtime use.&lt;/p&gt;
&lt;h3 id=&quot;23-local-build-state-snapshot&quot;&gt;23. Local Build-State Snapshot&lt;/h3&gt;
&lt;p&gt;A local build-state summary file was created at:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/BUILD_STATE_LOCAL.md
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;It records:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;source roots
verified commits
environment script
built TVM state
apache-tvm-ffi state
XGrammar state
tokenizers-cpp state
MLC-LLM state
FlashInfer state
local patches
known non-blocking issues
fresh-shell smoke test result
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The file was also normalised to use standard markdown code fences.&lt;/p&gt;
&lt;h3 id=&quot;24-current-completed-state&quot;&gt;24. Current Completed State&lt;/h3&gt;
&lt;p&gt;The following parts of the stack have now been successfully built or configured from source:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MLC-LLM source tree:
~/repos/mlc-llm

Pinned TVM source:
~/repos/mlc-llm/3rdparty/tvm

TVM native libraries:
built with Ninja

TVM Python source import:
working

apache-tvm-ffi:
editable local source build installed

tvm_ffi.core native extension:
built from Cython source

XGrammar:
native binding built and Python package installed from editable local source

tokenizers-cpp:
C++ / Rust / SentencePiece static libraries built from local source

MLC-LLM:
editable local source install completed with --no-deps

MLC-LLM native runtime:
libmlc_llm.so and libmlc_llm_module.so available and linkable

CUDA visibility from TVM:
working

LLVM backend:
LLVM 18.1.8

CUDA architecture:
sm_90 / compute_90 for H800

cuDNN:
ON

cuBLAS:
ON

CUTLASS:
ON

NCCL:
OFF

flashinfer-python:
not installed
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Important built artefacts:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/3rdparty/tvm/build/libtvm.so
~/repos/mlc-llm/3rdparty/tvm/build/libtvm_runtime.so
~/repos/mlc-llm/3rdparty/tvm/build/lib/libtvm_ffi.so
~/.local/lib/python3.10/site-packages/tvm_ffi/core.cpython-310-x86_64-linux-gnu.so
~/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.cpython-310-x86_64-linux-gnu.so
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/libtokenizers_c.a
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/libtokenizers_cpp.a
~/repos/mlc-llm/3rdparty/tokenizers-cpp/build/sentencepiece/src/libsentencepiece.a
~/repos/mlc-llm/build/libmlc_llm.so
~/repos/mlc-llm/build/libmlc_llm_module.so
~/repos/mlc-llm/env-tvm-source.sh
~/repos/mlc-llm/BUILD_STATE_LOCAL.md
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The correct environment entry point is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The correct CLI invocation form is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm ...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;rather than:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;mlc_llm ...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;because the editable install path did not generate a &lt;code&gt;~/.local/bin/mlc_llm&lt;/code&gt; console script.&lt;/p&gt;
&lt;p&gt;A final fresh-shell smoke test passed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;bash -lc &amp;#39;
  source ~/repos/mlc-llm/env-tvm-source.sh
  python3 - &amp;lt;&amp;lt;PY
import tvm
import tvm_ffi
import tvm_ffi.core
import xgrammar
import xgrammar.xgrammar_bindings
import mlc_llm
print(&amp;quot;cuda:&amp;quot;, tvm.cuda().exist)
PY
  python3 -m mlc_llm --help
&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Successful result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;cuda: True
MLC-LLM CLI help loads successfully
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At this point, the local source-built MLC-LLM development stack is functional for Python import, native runtime loading, CLI entry, and subsequent model compilation work. The remaining unresolved item is whether to build &lt;code&gt;flashinfer-python&lt;/code&gt; from source or to continue compiling models with FlashInfer explicitly disabled via:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;--opt &amp;quot;O2;flashinfer=0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;9-appendix-session-log--flashinfer-source-integration&quot;&gt;9. Appendix: session log — FlashInfer source integration&lt;/h2&gt;
&lt;p&gt;Date: 2026-05-05&lt;/p&gt;
&lt;h3 id=&quot;scope&quot;&gt;Scope&lt;/h3&gt;
&lt;p&gt;This session continued the source-build setup of the MLC-LLM development stack on the Ubuntu/H800 machine.&lt;/p&gt;
&lt;p&gt;The explicit objective was:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Integrate flashinfer-python from source into the existing MLC-LLM source-built environment.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The session did not download model weights, did not run a large model compilation, and did not modify global shell configuration such as &lt;code&gt;.bashrc&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;All source repositories remain under:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The main MLC-LLM repository remains:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;1-initial-fresh-shell-validation&quot;&gt;1. Initial Fresh-Shell Validation&lt;/h3&gt;
&lt;p&gt;A fresh-shell smoke test was run first by sourcing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;source ~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following components were verified to remain valid in a fresh shell:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;TVM
TVM-FFI
XGrammar
MLC-LLM
MLC-LLM native libraries
MLC-LLM CLI
CUDA visibility
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Observed paths:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm:
  /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/python/tvm/__init__.py

tvm_ffi:
  /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/3rdparty/tvm-ffi/python/tvm_ffi/__init__.py

tvm_ffi.core:
  /home/ailven.liu/.local/lib/python3.10/site-packages/tvm_ffi/core.cpython-310-x86_64-linux-gnu.so

xgrammar:
  /home/ailven.liu/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/__init__.py

xgrammar_bindings:
  /home/ailven.liu/repos/mlc-llm/3rdparty/xgrammar/python/xgrammar/xgrammar_bindings.cpython-310-x86_64-linux-gnu.so

mlc_llm:
  /home/ailven.liu/repos/mlc-llm/python/mlc_llm/__init__.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;MLC-LLM native libraries were located successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/home/ailven.liu/repos/mlc-llm/build/libmlc_llm.so
/home/ailven.liu/repos/mlc-llm/build/libmlc_llm_module.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;CUDA was visible to TVM:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm.cuda().exist: True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;At the beginning of the session, FlashInfer was confirmed as absent:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python: NOT_FOUND
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;2-mlc-llm-source-inspection-for-flashinfer-usage&quot;&gt;2. MLC-LLM Source Inspection for FlashInfer Usage&lt;/h3&gt;
&lt;p&gt;The current MLC-LLM commit was confirmed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;d1ea69a87280e821611643958bfec385b62dafd3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The local repository state contained expected local changes/untracked state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;m 3rdparty/tvm
?? BUILD_STATE_LOCAL.md
?? env-tvm-source.sh
?? env-tvm-source.sh.bak.*
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A local grep over the MLC-LLM and pinned TVM source tree found FlashInfer references in:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;pyproject.toml
python/requirements.txt
python/mlc_llm/interface/compiler_flags.py
python/mlc_llm/compiler_pass/dispatch_kv_cache_creation.py
python/mlc_llm/compiler_pass/pipeline.py
python/mlc_llm/op/attention.py
python/mlc_llm/op/extern.py
cpp/serve/function_table.cc
cpp/serve/sampler/gpu_sampler.cc
3rdparty/tvm/python/tvm/relax/backend/cuda/flashinfer.py
3rdparty/tvm/python/tvm/relax/frontend/nn/llm/kv_cache.py
3rdparty/tvm/src/runtime/vm/attn_backend.*
3rdparty/tvm/src/runtime/vm/paged_kv_cache.cc
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The MLC-LLM dependency declaration was confirmed as unpinned:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-toml&quot;&gt;&amp;quot;flashinfer-python; sys_platform == &amp;#39;linux&amp;#39;&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;and:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;python/requirements.txt:
flashinfer-python; sys_platform == &amp;quot;linux&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No exact FlashInfer version was pinned by MLC-LLM.&lt;/p&gt;
&lt;h3 id=&quot;3-flashinfer-api-requirements-identified&quot;&gt;3. FlashInfer API Requirements Identified&lt;/h3&gt;
&lt;p&gt;The pinned TVM FlashInfer backend requires these Python APIs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.jit.gen_customize_batch_prefill_module
flashinfer.jit.gen_customize_batch_decode_module
flashinfer.jit.gen_batch_mla_module
flashinfer.gemm.gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The main H800 attention and paged KV cache paths require the first three APIs.&lt;/p&gt;
&lt;p&gt;The &lt;code&gt;flashinfer.gemm.gen_gemm_sm100_module&lt;/code&gt; path is only used by the grouped GEMM generation path for compute capability 100 in the current TVM wrapper logic. The current machine is H800 / SM90, so this is not an immediate blocker for H800 FlashInfer attention, prefill, decode, paged KV cache, or MLA.&lt;/p&gt;
&lt;h3 id=&quot;4-python-pytorch-cuda-and-build-tool-state&quot;&gt;4. Python, PyTorch, CUDA, and Build Tool State&lt;/h3&gt;
&lt;p&gt;The Python environment was checked before building FlashInfer.&lt;/p&gt;
&lt;p&gt;Observed state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;python3:
  /usr/bin/python3

Python:
  3.10.12

pip:
  22.0.2

setuptools:
  82.0.1

wheel:
  0.37.1

ninja:
  1.13.0

cmake Python package:
  NOT_FOUND

python3 -m build:
  NOT_FOUND

system cmake:
  /usr/bin/cmake
  cmake version 3.22.1

system git:
  /usr/bin/git
  git version 2.34.1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;PyTorch/CUDA state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;torch:
  2.11.0+cu130

torch.version.cuda:
  13.0

torch.cuda.is_available:
  True

torch.cuda.device_count:
  8

device0:
  NVIDIA H800

capability0:
  (9, 0)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;System CUDA compiler:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/usr/local/cuda/bin/nvcc

nvcc:
  CUDA 13.2
  V13.2.51
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Important compatibility note:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PyTorch is built for CUDA 13.0, while the system nvcc is CUDA 13.2.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This combination was later validated successfully by FlashInfer decode, prefill, and MLA JIT build-and-load probes.&lt;/p&gt;
&lt;h3 id=&quot;5-flashinfer-version-selection&quot;&gt;5. FlashInfer Version Selection&lt;/h3&gt;
&lt;p&gt;The official FlashInfer repository was queried:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;https://github.com/flashinfer-ai/flashinfer.git
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Remote HEAD was accessible:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;HEAD -&amp;gt; refs/heads/main
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following tag was selected:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;v0.6.10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Tag commit:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;943d0ff0dfdbb5019b9733bd78ac730c5bff6f92
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Reason for selecting &lt;code&gt;v0.6.10&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;It is a current stable FlashInfer tag.
It exposes the required flashinfer.jit APIs used by the pinned TVM backend.
Its pyproject.toml declares package name flashinfer-python.
It supports Python &amp;gt;=3.10,&amp;lt;4.0.
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;6-remote-source-api-check-before-clone&quot;&gt;6. Remote Source API Check Before Clone&lt;/h3&gt;
&lt;p&gt;Before cloning, the tag contents were inspected through remote source reads and a temporary one-tag fetch.&lt;/p&gt;
&lt;p&gt;Confirmed in &lt;code&gt;v0.6.10&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.jit.gen_customize_batch_prefill_module: present
flashinfer.jit.gen_customize_batch_decode_module: present
flashinfer.jit.gen_batch_mla_module: present
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Confirmed layout:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.gemm is a package, not a single flashinfer/gemm.py file.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relevant files:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer/gemm/__init__.py
flashinfer/gemm/gemm_base.py
flashinfer/jit/gemm/__init__.py
flashinfer/jit/gemm/core.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The SM100 GEMM generator exists here:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.jit.gemm.gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;but is not exported from:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.gemm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Therefore:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;from flashinfer.gemm import gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;is incompatible with FlashInfer v0.6.10.&lt;/p&gt;
&lt;p&gt;However:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;from flashinfer.jit.gemm import gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;works.&lt;/p&gt;
&lt;p&gt;This was recorded as a known non-blocking compatibility issue for H800.&lt;/p&gt;
&lt;h3 id=&quot;7-flashinfer-v0610-requirements-check&quot;&gt;7. FlashInfer v0.6.10 Requirements Check&lt;/h3&gt;
&lt;p&gt;The &lt;code&gt;v0.6.10&lt;/code&gt; requirements were inspected:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache-tvm-ffi&amp;gt;=0.1.6,!=0.1.8,!=0.1.8.post0,&amp;lt;0.2
click
cuda-tile
einops
ninja
numpy
nvidia-cudnn-frontend&amp;gt;=1.13.0
nvidia-cutlass-dsl&amp;gt;=4.4.2
nvidia-ml-py
packaging&amp;gt;=24.2
requests
tabulate
torch
tqdm
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The current Python environment already contained these distributions:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache-tvm-ffi: 0.1.11.dev0+g1fed0ae04.d20260505
click: 8.3.3
cuda-tile: 1.3.0
einops: 0.8.2
ninja: 1.13.0
numpy: 2.2.6
nvidia-cudnn-frontend: 1.23.0
nvidia-cutlass-dsl: 4.4.2
nvidia-ml-py: 13.595.45
packaging: 26.2
requests: 2.33.1
tabulate: 0.10.0
torch: 2.11.0
tqdm: 4.67.3
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Non-obvious import checks:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;cutlass: IMPORT_OK
cuda: IMPORT_OK
cuda.bindings: IMPORT_OK
cuda_tile: IMPORT_FAIL
cudnn: IMPORT_OK
nvidia.cudnn_frontend: IMPORT_FAIL
pynvml: IMPORT_OK
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The failed import checks were not treated as immediate blockers because the distribution metadata was present, and import names do not always match distribution names.&lt;/p&gt;
&lt;h3 id=&quot;8-flashinfer-source-clone&quot;&gt;8. FlashInfer Source Clone&lt;/h3&gt;
&lt;p&gt;FlashInfer was cloned under the unified source root:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/flashinfer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Clone command shape:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos
git clone --recursive --branch v0.6.10 https://github.com/flashinfer-ai/flashinfer.git flashinfer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Verified state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;repository:
  /home/ailven.liu/repos/flashinfer

tag:
  v0.6.10

commit:
  943d0ff0dfdbb5019b9733bd78ac730c5bff6f92
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Submodules:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;3rdparty/cccl:
  876867684f7fac130e0f5911236e0a92a970d4fd
  v3.3.2

3rdparty/cutlass:
  da5e086dab31d63815acafdac9a9c5893b1c69e2
  v4.4.2

3rdparty/spdlog:
  c3aed4b68373955e1cc94307683d44dca1515d2b
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The detached HEAD state is expected because &lt;code&gt;v0.6.10&lt;/code&gt; is a tag.&lt;/p&gt;
&lt;h3 id=&quot;9-flashinfer-build-metadata-probe&quot;&gt;9. FlashInfer Build Metadata Probe&lt;/h3&gt;
&lt;p&gt;Before installation, PEP 517 editable metadata generation was tested manually.&lt;/p&gt;
&lt;p&gt;The source contains:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;pyproject.toml
build_backend.py
build_utils.py
requirements.txt
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Initially, this generated file did not exist:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer/_build_meta.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The metadata probe successfully created it:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;&amp;quot;&amp;quot;&amp;quot;Build metadata for flashinfer package.&amp;quot;&amp;quot;&amp;quot;
__version__ = &amp;quot;0.6.10&amp;quot;
__git_version__ = &amp;quot;943d0ff0dfdbb5019b9733bd78ac730c5bff6f92&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Editable metadata output:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer_python-0.6.10.dist-info
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Metadata content confirmed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Name: flashinfer-python
Version: 0.6.10
Requires-Python: &amp;lt;4.0,&amp;gt;=3.10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No CUDA compilation, wheel download, dependency resolution, or installation was performed during the metadata-only probe.&lt;/p&gt;
&lt;p&gt;Warnings observed during metadata generation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;pyproject.toml does not contain setuptools_scm / vcs-versioning tool sections
Pattern &amp;#39;LICENSE*.txt&amp;#39; did not match any files
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These warnings did not block metadata generation.&lt;/p&gt;
&lt;h3 id=&quot;10-flashinfer-editable-source-installation&quot;&gt;10. FlashInfer Editable Source Installation&lt;/h3&gt;
&lt;p&gt;FlashInfer was installed as an editable local source package.&lt;/p&gt;
&lt;p&gt;Command used:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;cd ~/repos/flashinfer

python3 -m pip install \
  --user \
  --no-build-isolation \
  --no-binary=:all: \
  --no-deps \
  -e . \
  -v
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Rationale:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--no-build-isolation:
  Preserve the current source-built Python environment, especially local apache-tvm-ffi.
  Avoid creating an isolated build environment that might resolve packages from PyPI.

--no-deps:
  Prevent pip from downloading, replacing, or resolving dependencies such as torch, cuda-tile,
  nvidia-cudnn-frontend, nvidia-cutlass-dsl, or apache-tvm-ffi.

--no-binary=:all:
  Preserve the source-build policy and reject prebuilt binary packages.

-e .:
  Install the local FlashInfer source tree in editable mode.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Installation result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Successfully installed flashinfer-python-0.6.10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The install log only contained a locally generated PEP 660 editable wheel:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer_python-0.6.10-0.editable-py3-none-any.whl
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This was not a downloaded binary wheel. It was a local editable-install metadata artifact.&lt;/p&gt;
&lt;p&gt;Installed metadata:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer-python: 0.6.10
location: /home/ailven.liu/.local/lib/python3.10/site-packages
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Editable import path:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/home/ailven.liu/repos/flashinfer/flashinfer/__init__.py
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;11-flashinfer-python-import-and-api-validation&quot;&gt;11. FlashInfer Python Import and API Validation&lt;/h3&gt;
&lt;p&gt;After editable installation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.__file__:
  /home/ailven.liu/repos/flashinfer/flashinfer/__init__.py

flashinfer.__version__:
  0.6.10

flashinfer.jit.__file__:
  /home/ailven.liu/repos/flashinfer/flashinfer/jit/__init__.py
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Required JIT APIs were available:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer.jit.gen_customize_batch_prefill_module: True
flashinfer.jit.gen_customize_batch_decode_module: True
flashinfer.jit.gen_batch_mla_module: True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Known non-blocking incompatibility:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;from flashinfer.gemm import gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;failed with:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;ImportError: cannot import name &amp;#39;gen_gemm_sm100_module&amp;#39; from &amp;#39;flashinfer.gemm&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;but:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;from flashinfer.jit.gemm import gen_gemm_sm100_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;succeeded.&lt;/p&gt;
&lt;p&gt;This remains non-blocking for the current H800 / SM90 path.&lt;/p&gt;
&lt;h3 id=&quot;12-flashinfer-jit-spec-generation-probe&quot;&gt;12. FlashInfer JIT Spec Generation Probe&lt;/h3&gt;
&lt;p&gt;Before compiling JIT kernels, FlashInfer JIT spec generation was tested.&lt;/p&gt;
&lt;p&gt;Direct FlashInfer JIT generation succeeded for:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;prefill
decode
MLA
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Generated source locations:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/home/ailven.liu/.cache/flashinfer/0.6.10/90a/generated/...
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Observed source counts:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;prefill: 10 CUDA source files
decode:  3 CUDA source files
MLA:     3 CUDA source files
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The &lt;code&gt;90a&lt;/code&gt; cache path indicates that FlashInfer generated Hopper-oriented &lt;code&gt;sm_90a&lt;/code&gt; JIT artifacts for the H800 environment.&lt;/p&gt;
&lt;p&gt;No CUDA compilation was performed in this spec-generation probe.&lt;/p&gt;
&lt;h3 id=&quot;13-flashinfer-direct-jit-build-and-load-probes&quot;&gt;13. FlashInfer Direct JIT Build-and-Load Probes&lt;/h3&gt;
&lt;p&gt;The following environment variables were set for JIT compilation probes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;export CUDA_HOME=/usr/local/cuda
export CUDA_PATH=/usr/local/cuda
export CUDACXX=/usr/local/cuda/bin/nvcc
export TORCH_CUDA_ARCH_LIST=&amp;quot;9.0a&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;decode-jit-probe&quot;&gt;Decode JIT Probe&lt;/h4&gt;
&lt;p&gt;Direct FlashInfer decode JIT:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;gen_customize_batch_decode_module(...).build_and_load()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;build_and_load: OK
module_type: tvm_ffi.module.Module
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;prefill-jit-probe&quot;&gt;Prefill JIT Probe&lt;/h4&gt;
&lt;p&gt;Direct FlashInfer prefill JIT:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;gen_customize_batch_prefill_module(...).build_and_load()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;build_and_load: OK
module_type: tvm_ffi.module.Module
&lt;/code&gt;&lt;/pre&gt;
&lt;h4 id=&quot;mla-jit-probe&quot;&gt;MLA JIT Probe&lt;/h4&gt;
&lt;p&gt;Direct FlashInfer MLA JIT:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;gen_batch_mla_module(...).build_and_load()
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;build_and_load: OK
module_type: tvm_ffi.module.Module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Conclusion:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;FlashInfer direct JIT generation, CUDA compilation, linking, and runtime loading work for decode, prefill, and MLA on this H800 environment.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Validated toolchain combination:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;FlashInfer: 0.6.10
PyTorch: 2.11.0+cu130
torch.version.cuda: 13.0
System nvcc: CUDA 13.2
GPU: NVIDIA H800
GPU capability: (9, 0)
TORCH_CUDA_ARCH_LIST: 9.0a
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;14-environment-script-patch-for-flashinfer-jit&quot;&gt;14. Environment Script Patch for FlashInfer JIT&lt;/h3&gt;
&lt;p&gt;The local source environment script was patched:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/env-tvm-source.sh
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A backup was created:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/home/ailven.liu/repos/mlc-llm/env-tvm-source.sh.bak.flashinfer.20260505_100638
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following block was appended:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# FlashInfer JIT CUDA toolchain for Hopper/H800.
# Keep these in the source-able local environment script instead of .bashrc.
# PyTorch is cu130, while system nvcc is CUDA 13.2; this combination has been
# validated by FlashInfer decode/prefill/MLA JIT build_and_load probes.
export CUDA_HOME=&amp;quot;${CUDA_HOME:-/usr/local/cuda}&amp;quot;
export CUDA_PATH=&amp;quot;${CUDA_PATH:-/usr/local/cuda}&amp;quot;
export CUDACXX=&amp;quot;${CUDACXX:-/usr/local/cuda/bin/nvcc}&amp;quot;
export TORCH_CUDA_ARCH_LIST=&amp;quot;${TORCH_CUDA_ARCH_LIST:-9.0a}&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Fresh-shell validation after patch:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;CUDA_HOME=/usr/local/cuda
CUDA_PATH=/usr/local/cuda
CUDACXX=/usr/local/cuda/bin/nvcc
TORCH_CUDA_ARCH_LIST=9.0a
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;CUDACXX&lt;/code&gt; was executable:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/usr/local/cuda/bin/nvcc
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No &lt;code&gt;.bashrc&lt;/code&gt; or global system shell configuration was modified.&lt;/p&gt;
&lt;h3 id=&quot;15-fresh-shell-full-stack-validation-after-flashinfer-integration&quot;&gt;15. Fresh-Shell Full Stack Validation After FlashInfer Integration&lt;/h3&gt;
&lt;p&gt;After patching the environment script, a fresh-shell full stack check was run.&lt;/p&gt;
&lt;p&gt;Command pattern:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;bash -lc &amp;#39;
  source ~/repos/mlc-llm/env-tvm-source.sh
  ...
&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Confirmed environment:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;CUDA_HOME=/usr/local/cuda
CUDA_PATH=/usr/local/cuda
CUDACXX=/usr/local/cuda/bin/nvcc
TORCH_CUDA_ARCH_LIST=9.0a
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Confirmed stack imports:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;torch: 2.11.0+cu130
torch.version.cuda: 13.0
torch.cuda.is_available: True
device0: NVIDIA H800
capability0: (9, 0)

tvm:
  /home/ailven.liu/repos/mlc-llm/3rdparty/tvm/python/tvm/__init__.py

mlc_llm:
  /home/ailven.liu/repos/mlc-llm/python/mlc_llm/__init__.py

flashinfer:
  /home/ailven.liu/repos/flashinfer/flashinfer/__init__.py

flashinfer.__version__:
  0.6.10

flashinfer.jit:
  /home/ailven.liu/repos/flashinfer/flashinfer/jit/__init__.py

tvm.cuda().exist:
  True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;MLC-LLM native libraries remained discoverable:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;/home/ailven.liu/repos/mlc-llm/build/libmlc_llm.so
/home/ailven.liu/repos/mlc-llm/build/libmlc_llm_module.so
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Distribution metadata:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache-tvm-ffi: 0.1.11.dev0+g1fed0ae04.d20260505
xgrammar: 0.0.3
mlc_llm: 0.20.0.dev0
flashinfer-python: 0.6.10
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;MLC-LLM CLI still worked:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm --help
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;16-tvm-flashinfer-backend-wrapper-validation&quot;&gt;16. TVM FlashInfer Backend Wrapper Validation&lt;/h3&gt;
&lt;p&gt;After validating direct FlashInfer JIT paths, the actual TVM wrapper path was tested:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;from tvm.relax.backend.cuda import flashinfer as tvm_flashinfer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The following TVM wrapper APIs were compiled and loaded successfully:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;tvm.relax.backend.cuda.flashinfer.gen_flashinfer_decode_module
tvm.relax.backend.cuda.flashinfer.gen_flashinfer_prefill_module
tvm.relax.backend.cuda.flashinfer.gen_flashinfer_mla_module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Results:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;decode_mods_len: 1
prefill_mods_len: 1
mla_mods_len: 1
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Returned module types:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;&amp;lt;class &amp;#39;tvm.runtime.module.Module&amp;#39;&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Final wrapper result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;TVM_BACKEND_FLASHINFER_WRAPPER_BUILD: OK
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Conclusion:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;The TVM backend wrapper path used by MLC-LLM can call FlashInfer, generate CUDA code, compile it, load it, and return TVM runtime modules.
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;17-mlc-llm-optimisation-flag-behaviour&quot;&gt;17. MLC-LLM Optimisation Flag Behaviour&lt;/h3&gt;
&lt;p&gt;The compile optimisation flag behaviour was inspected through Python API and CLI help.&lt;/p&gt;
&lt;p&gt;Detected target:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;target: cuda
TVM target arch: sm_90
detected_cuda_arch_list: [90]
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Quantisation used for check:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;q0f16
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Observed preset behaviour:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;O0 -&amp;gt; flashinfer=False
O1 -&amp;gt; flashinfer=False
O2 -&amp;gt; flashinfer=True
O3 -&amp;gt; flashinfer=True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Therefore, for this H800 machine:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;The default --opt O2 enables FlashInfer.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Explicit detailed knobs were tested:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;flashinfer=0 -&amp;gt; flashinfer=False
flashinfer=1 -&amp;gt; flashinfer=True
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Important parser constraint:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;O2 is a preset and cannot be mixed with detailed knobs in the same semicolon-separated string.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Invalid:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;--opt &amp;quot;O2;flashinfer=0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Observed failure:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;unrecognized arguments: --O2
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Reason:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;When the string is not exactly one of O0/O1/O2/O3, the parser treats each semicolon-separated segment as a detailed knob and prepends &amp;quot;--&amp;quot;.
Therefore &amp;quot;O2;flashinfer=0&amp;quot; becomes &amp;quot;--O2 --flashinfer=0&amp;quot;, but &amp;quot;--O2&amp;quot; is not a recognised detailed knob.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Valid examples:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;--opt O2
--opt O3
--opt &amp;quot;flashinfer=0&amp;quot;
--opt &amp;quot;flashinfer=1&amp;quot;
--opt &amp;quot;cublas_gemm=1;cudagraph=0&amp;quot;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Note:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--opt &amp;quot;flashinfer=1&amp;quot; is not equivalent to &amp;quot;O2 plus only changing FlashInfer&amp;quot;.
It explicitly sets only the provided detailed knobs and leaves other optimisation flags at their detailed-knob defaults.
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;18-mlc-llm-compile-cli-help-check&quot;&gt;18. MLC-LLM Compile CLI Help Check&lt;/h3&gt;
&lt;p&gt;The compile subcommand help was checked:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;python3 -m mlc_llm compile --help
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Confirmed important options:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--quantization
--model-type
--device
--host
--enable-subgroups
--opt
--system-lib-prefix
--output / -o
--overrides
--debug-dump
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The CLI help states:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;--opt OPT

Optimization flags. MLC LLM maintains a predefined set of optimization flags,
denoted as O0, O1, O2, O3, where O0 means no optimization, O2 means majority
of them, and O3 represents extreme optimization that could potentially break
the system. Meanwhile, optimization flags could be explicitly specified via
details knobs, e.g. --opt=&amp;quot;cublas_gemm=1;cudagraph=0&amp;quot;.
(default: &amp;quot;O2&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;19-build_state_localmd-update&quot;&gt;19. BUILD_STATE_LOCAL.md Update&lt;/h3&gt;
&lt;p&gt;The FlashInfer integration state and compile option semantics were appended to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;~/repos/mlc-llm/BUILD_STATE_LOCAL.md
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The file was then validated.&lt;/p&gt;
&lt;p&gt;Observed file state:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;-rw-r--r-- 1 ailven.liu ailven.liu 11K 5 May 10:29 BUILD_STATE_LOCAL.md
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Key sections found:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;MLC-LLM Compile CLI and FlashInfer Integration Update
Current FlashInfer Integration Status
Next Recommended Step
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Tail validation confirmed that the appended section was not truncated.&lt;/p&gt;
&lt;h3 id=&quot;20-local-model-config-search&quot;&gt;20. Local Model Config Search&lt;/h3&gt;
&lt;p&gt;A local search was performed for existing MLC model configs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;find ~/repos \
  -path &amp;quot;*/.git&amp;quot; -prune -o \
  -name &amp;quot;mlc-chat-config.json&amp;quot; \
  -type f \
  -print
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Result:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;No mlc-chat-config.json was found under ~/repos.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Python recursive summary over &lt;code&gt;~/repos&lt;/code&gt; also found no local &lt;code&gt;mlc-chat-config.json&lt;/code&gt;.&lt;/p&gt;
&lt;p&gt;Conclusion:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;There is currently no local MLC model config available for the next minimal compile-path probe.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;No model was downloaded, no compile was run, and no model artifact was generated.&lt;/p&gt;
&lt;h3 id=&quot;21-current-end-of-session-state&quot;&gt;21. Current End-of-Session State&lt;/h3&gt;
&lt;p&gt;Current completed status:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;TVM source build: OK
TVM Python import: OK
TVM CUDA visibility: OK
apache-tvm-ffi source/editable integration: OK
XGrammar source/editable integration: OK
tokenizers-cpp source build: OK
MLC-LLM source/editable integration: OK
MLC-LLM native libraries: OK
MLC-LLM CLI: OK
FlashInfer source clone: OK
FlashInfer submodules: OK
FlashInfer metadata generation: OK
FlashInfer editable source install: OK
FlashInfer Python import: OK
FlashInfer required JIT APIs: OK
FlashInfer direct decode JIT build/load: OK
FlashInfer direct prefill JIT build/load: OK
FlashInfer direct MLA JIT build/load: OK
TVM FlashInfer wrapper decode build/load: OK
TVM FlashInfer wrapper prefill build/load: OK
TVM FlashInfer wrapper MLA build/load: OK
env-tvm-source.sh FlashInfer JIT variables: OK
MLC-LLM --opt O2 FlashInfer enablement on H800: OK
Local mlc-chat-config.json availability: NOT_FOUND
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Current unresolved item:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;No local mlc-chat-config.json exists under ~/repos.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Known non-blocking compatibility issue:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;FlashInfer v0.6.10 does not export gen_gemm_sm100_module from flashinfer.gemm.
The symbol exists under flashinfer.jit.gemm.
This affects the SM100 grouped GEMM wrapper path only and is not currently blocking H800 / SM90 attention, paged KV cache, decode, prefill, or MLA paths.
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;22-recommended-next-step&quot;&gt;22. Recommended Next Step&lt;/h3&gt;
&lt;p&gt;Since no local &lt;code&gt;mlc-chat-config.json&lt;/code&gt; exists under &lt;code&gt;~/repos&lt;/code&gt;, the next step should not be a normal model compilation yet.&lt;/p&gt;
&lt;p&gt;Recommended next action:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Create or locate a minimal local MLC model config suitable for a compiler-path probe,
without downloading model weights and without running a large compilation.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Possible next directions:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;1. Search the MLC-LLM test tree for minimal model config examples.
2. Inspect MLC-LLM compiler tests for synthetic config construction.
3. Build a minimal local test config only if it can exercise the FlashInfer paged KV cache creation path.
4. Run a minimal compile-path probe with --opt O2 only after a suitable local config is available.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Do not proceed directly to downloading a HuggingFace model or compiling a large model until the minimal compile-path probe strategy is selected.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>From Relay to Relax to Unity: The Historical Evolution of Apache TVM’s High-Level Compilation Stack</title>
    <link href="https://www.ailven.cc/posts/ai-compiler-260501.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/ai-compiler-260501.html</id>
    <published>2026-05-01T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>A deep-dive into Apache TVM’s compiler architecture evolution – from the NNVM graph era through Relay’s graph IR, to Relax’s symbolic-shape-aware dataflow IR and TensorIR integration, culminating in the Unity modular cross-layer ML compilation system.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 1 May 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;1-overview&quot;&gt;1. Overview&lt;/h2&gt;
&lt;p&gt;The evolution from Relay to Relax and then to Unity in Apache TVM is not merely a sequence of syntactic replacements. It reflects a deeper shift in the compiler architecture of TVM: from a graph-level optimisation system centred on a high-level neural-network IR, towards a modular, cross-level machine learning compilation infrastructure in which graph programs, tensor programs, runtime objects, external libraries, and hardware-specific code generation can be represented and optimised together.&lt;/p&gt;
&lt;p&gt;In concise terms:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;NNVM Graph
  |
  V
Relay
  |
  V
Relax + TensorIR + TVMScript + MetaSchedule + Runtime / Backend Integration
  |
  V
TVM Unity Modular Compilation Flow
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relay was TVM’s first mature high-level graph IR. Relax is the next-generation high-level IR designed for dynamic shapes, modern model structures, LLM workloads, Python-first workflows, and cross-level optimisation with TensorIR. Unity is not a single IR, but a broader architectural direction that aims to unify Relax, TensorIR, TVMScript, MetaSchedule, the runtime, libraries, and target backends into a coherent and composable compilation system.&lt;/p&gt;
&lt;p&gt;Therefore, the essential transformation can be summarised as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay:
    high-level graph IR for neural-network compilation

Relax:
    symbolic-shape-aware dataflow/program IR integrated with TensorIR

Unity:
    modular cross-layer ML compilation architecture
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The transition from Relay to Relax should not be interpreted as a superficial renaming. It represents a change in the underlying compilation philosophy: from a largely staged pipeline of high-level graph optimisation followed by low-level tensor-program lowering, towards an IRModule-centred system in which high-level graph programs and low-level tensor programs coexist, interact, and can be transformed together.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;2-before-relay-the-nnvm-graph-era&quot;&gt;2. Before Relay: The NNVM Graph Era&lt;/h2&gt;
&lt;p&gt;Before Relay became the central graph-level IR in TVM, early TVM workflows relied on NNVM as the high-level graph representation. In that period, the overall compilation pipeline could be roughly described as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Framework Model
  |
  V
NNVM Graph
  |
  V
Operator Lowering
  |
  V
TVM Tensor Expression / Schedule
  |
  V
Code Generation
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This approach was effective for many early deep learning workloads. Models such as CNNs, MLPs, and relatively regular RNN variants could often be represented as static operator DAGs:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Conv2D -&amp;gt; BatchNorm -&amp;gt; ReLU -&amp;gt; Pooling -&amp;gt; Dense
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;However, NNVM-style graph IRs had several structural limitations.&lt;/p&gt;
&lt;p&gt;First, their expressive power was limited. A conventional computation graph is well suited to representing a DAG of tensor operators, but it is not a natural representation for control flow, subgraphs, higher-order functions, recursion, or dynamic computation.&lt;/p&gt;
&lt;p&gt;Second, the boundary between compile-time representation and runtime representation was insufficiently clean. The graph was both an optimisation object and a structure close to runtime execution, which restricted the compiler’s ability to perform deeper program transformations.&lt;/p&gt;
&lt;p&gt;Third, the interaction between high-level graph optimisation and low-level tensor-program optimisation was weak. A graph-level pass could identify that a subgraph contained a convolution, a matrix multiplication, or an elementwise operation, but it had limited access to the details that matter at the tensor-program level: tiling, vectorisation, memory layout, shared-memory usage, tensorisation, cache locality, and target-specific scheduling.&lt;/p&gt;
&lt;p&gt;These limitations motivated the design of Relay.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;3-relay-tvms-first-mature-high-level-ir&quot;&gt;3. Relay: TVM’s First Mature High-Level IR&lt;/h2&gt;
&lt;h3 id=&quot;31-historical-role-of-relay&quot;&gt;3.1 Historical Role of Relay&lt;/h3&gt;
&lt;p&gt;Relay was introduced around 2018 as the successor to NNVM. It was designed as a new high-level IR for machine learning frameworks and was often described as NNVM v2.0. Its purpose was to provide a more expressive and principled representation for machine learning programs than a traditional computation graph.&lt;/p&gt;
&lt;p&gt;Relay can be characterised as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;statically typed
purely functional
graph-level / high-level
machine-learning-oriented
intermediate representation
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Unlike a simple DAG representation, Relay introduced concepts from programming language design: functions, variables, let-bindings, function calls, operator calls, tuples, conditional expressions, and a type system. This made Relay significantly more expressive than NNVM-style graphs.&lt;/p&gt;
&lt;p&gt;Relay was therefore not merely a graph container. It was a typed functional IR designed to represent machine learning models as programs.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;32-what-relay-solved&quot;&gt;3.2 What Relay Solved&lt;/h3&gt;
&lt;p&gt;Relay solved several important problems in the earlier TVM stack.&lt;/p&gt;
&lt;p&gt;First, it introduced a stronger type system. Relay could represent tensor types, tuple types, function types, and other structured types, enabling more robust static checking, type inference, and program transformation.&lt;/p&gt;
&lt;p&gt;Second, Relay provided functional abstraction. Models no longer had to be represented only as flat operator DAGs. They could include function definitions, let-bindings, control-flow constructs, and nested computation structures.&lt;/p&gt;
&lt;p&gt;Third, Relay allowed graph optimisation to be expressed as program transformation. Passes such as constant folding, operator fusion, layout rewriting, dead-code elimination, common subexpression elimination, and quantisation rewriting could be implemented as transformations over Relay programs.&lt;/p&gt;
&lt;p&gt;Fourth, Relay became the main graph-level IR in TVM for a substantial period. It served as the central representation for frontend import, high-level graph optimisation, operator fusion, and the initial stages of lowering towards tensor-level computation.&lt;/p&gt;
&lt;p&gt;A typical Relay-based compilation flow looked like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;PyTorch / TensorFlow / ONNX / TFLite / MXNet
      |
      V   Frontend Import
Relay IRModule
      |
      V   Relay Graph-Level Optimisation
Optimised Relay
      |
      V   Operator Lowering / Fusion
TE / TIR
      |
      V   Scheduling / Auto-Tuning / Code Generation
Runtime Module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;From an engineering perspective, Relay played four major roles:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;model-structure representation
graph-level optimisation
unified frontend import
operator fusion and lowering decision layer
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This made Relay one of the most important components in the maturation of TVM as a deep learning compiler.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;33-limitations-of-relay&quot;&gt;3.3 Limitations of Relay&lt;/h3&gt;
&lt;p&gt;Relay was not a failed design. On the contrary, it was a key stage in the evolution of TVM. However, as model architectures, hardware targets, and deployment requirements changed, Relay began to expose structural limitations.&lt;/p&gt;
&lt;h4 id=&quot;331-limited-support-for-dynamic-shape-workloads&quot;&gt;3.3.1 Limited Support for Dynamic Shape Workloads&lt;/h4&gt;
&lt;p&gt;Relay was designed during a period when many important workloads were still dominated by relatively static shapes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;fixed batch size
fixed image resolution
often fixed sequence length
regular operator structure
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Modern workloads are different. Transformers, LLMs, VLMs, diffusion models, dynamic batching, variable sequence lengths, KV cache management, ragged tensors, and prefill/decode separation all require stronger support for symbolic and runtime-dependent shapes.&lt;/p&gt;
&lt;p&gt;For example, an LLM inference workload naturally contains two different execution regimes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Prefill:
    input shape = [batch, prompt_length]

Decode:
    input shape = [batch, 1]
    KV cache shape = [num_layers, batch, num_heads, sequence_length, head_dim]
    sequence_length grows token by token
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relay could support certain dynamic-shape cases, but it was not designed around first-class symbolic shape as a central organising principle. As dynamic workloads became increasingly important, this became a fundamental limitation.&lt;/p&gt;
&lt;h4 id=&quot;332-a-hard-boundary-between-graph-ir-and-tensorir&quot;&gt;3.3.2 A Hard Boundary Between Graph IR and TensorIR&lt;/h4&gt;
&lt;p&gt;In the Relay-era compilation stack, the relationship between the high-level graph and low-level tensor program was usually staged:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay Graph Passes
  |
  V
Lowering to TE / TIR
  |
  V
TIR Scheduling / Code Generation
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This structure created a relatively hard boundary between high-level and low-level optimisation.&lt;/p&gt;
&lt;p&gt;A graph-level pass could decide whether to fuse operators, rewrite layouts, or transform subgraphs, but it often lacked precise information about low-level scheduling consequences. For example, whether a fusion decision is beneficial depends not only on graph topology, but also on:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;register pressure
shared-memory usage
tensor-core-friendly layout
memory coalescing
occupancy
vectorisation feasibility
target-specific code-generation behaviour
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A purely graph-level view is insufficient for such decisions. Conversely, low-level scheduling may reveal constraints or opportunities that should influence graph-level transformation. Relay’s architecture made this kind of bidirectional feedback difficult.&lt;/p&gt;
&lt;h4 id=&quot;333-limited-modularity-of-the-pass-pipeline&quot;&gt;3.3.3 Limited Modularity of the Pass Pipeline&lt;/h4&gt;
&lt;p&gt;Relay supported custom passes, external code generation, BYOC, custom operators, and multiple frontend paths. However, many practical workflows still felt like a relatively fixed pipeline:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;frontend import
  |
  V
relay.transform.Sequential([...])
  |
  V
relay.build(...)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This was acceptable for conventional model deployment, but less suitable for compiler engineers, hardware vendors, and AI infrastructure developers who needed to insert custom transformations, replace lowering paths, integrate external libraries, mix hand-written kernels with generated kernels, and inspect intermediate IR in a disciplined way.&lt;/p&gt;
&lt;p&gt;As TensorIR, TVMScript, and MetaSchedule matured, TVM required a more modular, inspectable, and programmable compilation flow.&lt;/p&gt;
&lt;h4 id=&quot;334-insufficient-natural-fit-for-llm-workloads&quot;&gt;3.3.4 Insufficient Natural Fit for LLM Workloads&lt;/h4&gt;
&lt;p&gt;Relay was not originally designed around modern LLM inference. LLM workloads impose a set of requirements that are difficult to express elegantly in a traditional graph-centric compiler flow:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;symbolic sequence length
dynamic KV cache
paged attention
prefill/decode separation
shape-dependent memory planning
external kernel integration
quantisation-aware graph rewriting
tensor parallelism
pipeline parallelism
runtime shape dispatch
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These are not marginal extensions to ordinary graph execution. They require a representation that can naturally express graph structure, symbolic shapes, runtime objects, external function calls, TIR kernels, device placement, and memory/runtime behaviour within a single composable system.&lt;/p&gt;
&lt;p&gt;This requirement led to Relax.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;4-tvm-unity-the-architectural-vision-behind-relax&quot;&gt;4. TVM Unity: The Architectural Vision Behind Relax&lt;/h2&gt;
&lt;h3 id=&quot;41-unity-is-not-a-single-ir&quot;&gt;4.1 Unity Is Not a Single IR&lt;/h3&gt;
&lt;p&gt;TVM Unity should not be understood as a replacement IR. It is a broader architectural direction for TVM.&lt;/p&gt;
&lt;p&gt;The central goal of Unity is to break the rigid boundaries between several layers of the machine learning system stack:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;computational graph
tensor programme
library / runtime
hardware primitive
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Historically, machine learning systems often separated these concerns into relatively isolated layers. Graph compilers optimised graph-level structure; tensor compilers generated kernels; vendor libraries supplied high-performance primitives; runtimes handled execution and memory management. These divisions simplified system design, but they also constrained cross-layer optimisation.&lt;/p&gt;
&lt;p&gt;Unity’s central judgement is that future ML compilation cannot be a one-way lowering pipeline from graph to kernel. It must allow graph IR, tensor programs, external libraries, runtime systems, and hardware primitives to interact.&lt;/p&gt;
&lt;p&gt;Thus, Unity should be understood as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;not one IR to rule them all
but interoperable IRs in one modular system
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In practical terms, Unity consists of several mutually connected components:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax:
    graph-level and programme-level abstraction

TensorIR:
    tensor-program abstraction

TVMScript:
    inspectable and writable IR surface

MetaSchedule:
    automatic search-based scheduling

DLight and rule-based scheduling:
    practical default scheduling automation

Runtime / VM / external codegen:
    execution and backend substrate
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Unity does not attempt to collapse all levels of abstraction into a single monolithic representation. Instead, it attempts to make multiple representations interoperable within one coherent compilation framework.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;5-relax-relay-next&quot;&gt;5. Relax: Relay Next&lt;/h2&gt;
&lt;h3 id=&quot;51-historical-position-of-relax&quot;&gt;5.1 Historical Position of Relax&lt;/h3&gt;
&lt;p&gt;Relax emerged as ‘Relay Next’ under the TVM Unity vision. Its purpose was not to discard Relay’s achievements, but to evolve the high-level IR in response to new model and hardware requirements.&lt;/p&gt;
&lt;p&gt;Relax inherits Relay’s role as the graph-level IR, but redesigns the abstraction around several new requirements:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;dynamic shape
symbolic shape
TensorIR integration
cross-level optimisation
runtime object representation
external kernel integration
modular transformation
Python-first inspectability
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The result is a high-level IR that is better suited to contemporary machine learning workloads, especially Transformer- and LLM-based systems.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;52-core-objectives-of-relax&quot;&gt;5.2 Core Objectives of Relax&lt;/h3&gt;
&lt;p&gt;The essential objectives of Relax can be summarised as follows:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;G0: bridge graph IR and TensorIR
G1: provide first-class symbolic shape support
G2: represent dynamic workloads naturally
G3: improve customisation and composability
G4: integrate cleanly with TVMScript and IRModule
G5: support emerging workloads such as Transformers, LLMs, and dynamic models
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax was therefore designed not merely as a cleaner graph IR, but as the high-level program representation required by the Unity architecture.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;53-basic-design-of-relax&quot;&gt;5.3 Basic Design of Relax&lt;/h3&gt;
&lt;p&gt;The key design idea of Relax is that high-level graph programs and low-level tensor programs can coexist inside the same IRModule.&lt;/p&gt;
&lt;p&gt;A simplified Relax + TensorIR module may look conceptually like this:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-python&quot;&gt;@tvm.script.ir_module
class Module:
    @T.prim_func
    def matmul(...):
        ...

    @R.function
    def main(x, w):
        y = R.call_tir(matmul, (x, w), ...)
        return y
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This example illustrates the essential relationship:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax functions:
    high-level dataflow, graph structure, runtime-level composition

TIR PrimFuncs:
    low-level tensor programmes and kernels

IRModule:
    shared container that holds both Relax and TIR

R.call_tir:
    explicit bridge from Relax-level programme to TIR-level kernel
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is a major change from the older model in which a Relay graph was lowered into tensor-level code through a relatively staged pipeline. In Relax/Unity, the IRModule becomes the central object of transformation.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;6-core-differences-between-relay-and-relax&quot;&gt;6. Core Differences Between Relay and Relax&lt;/h2&gt;
&lt;h3 id=&quot;61-ir-style-functional-graph-ir-versus-dataflowbinding-oriented-programme-ir&quot;&gt;6.1 IR Style: Functional Graph IR versus Dataflow/Binding-Oriented Program IR&lt;/h3&gt;

&lt;p&gt;Relay is strongly functional in style. It represents programs using expressions such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;let-binding
function
call
tuple
if-expression
recursion
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax is more explicitly structured around dataflow blocks, bindings, symbolic shape information, StructInfo, and explicit boundaries between pure dataflow and effectful/runtime operations.&lt;/p&gt;
&lt;p&gt;A simplified comparison is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay:
    machine learning programme as typed functional expression

Relax:
    machine learning programme as structured dataflow with bindings,
    symbolic shape, TIR calls, and runtime integration
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax is closer to a modern SSA-like dataflow program representation while still preserving the high-level graph structure needed for model optimisation.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;62-type-system-relay-type-versus-relax-structinfo&quot;&gt;6.2 Type System: Relay Type versus Relax StructInfo&lt;/h3&gt;
&lt;p&gt;Relay uses a type system to describe expressions. Examples include tensor types, tuple types, and function types.&lt;/p&gt;
&lt;p&gt;Relax introduces the concept of &lt;code&gt;StructInfo&lt;/code&gt;. StructInfo describes the structural information of expressions, including:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;TensorStructInfo
ShapeStructInfo
TupleStructInfo
FuncStructInfo
ObjectStructInfo
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;StructInfo is not merely a conventional type system. It is better suited to representing partially known and partially dynamic information:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;dtype
number of dimensions
symbolic shape
known and unknown dimensions
runtime object structure
shape expressions
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is crucial for dynamic-shape workloads. For example, in LLM inference:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;input_ids:
    Tensor[(batch, sequence_length), int32]

kv_cache:
    runtime object / tuple / opaque cache structure

logits:
    Tensor[(batch, sequence_length, vocab_size), float16]
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Here, &lt;code&gt;sequence_length&lt;/code&gt; may be symbolic rather than compile-time constant. Relax’s StructInfo model is designed to represent this kind of partial static knowledge cleanly.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;63-shape-auxiliary-type-information-versus-first-class-symbolic-shape&quot;&gt;6.3 Shape: Auxiliary Type Information versus First-Class Symbolic Shape&lt;/h3&gt;
&lt;p&gt;In Relay, shape information is largely attached to tensor types. Dynamic-shape support exists, but symbolic shape is not the central organising principle of the IR.&lt;/p&gt;
&lt;p&gt;Relax treats symbolic shape as a first-class design concern. This is essential for workloads such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Transformer variable sequence length
LLM prefill/decode execution
dynamic batching
ragged input
vision-language models with variable token counts
diffusion models with dynamic resolution
shape-polymorphic library dispatch
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;From an AI infrastructure perspective, Relax makes it natural to represent partially dynamic programs such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;n = T.int64()
m = T.int64()

x: R.Tensor((n, m), &amp;quot;float16&amp;quot;)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is fundamentally different from forcing all shapes to be fixed at compile time.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;64-high-low-level-relationship-lowering-boundary-versus-irmodule-level-co-design&quot;&gt;6.4 High-Low Level Relationship: Lowering Boundary versus IRModule-Level Co-Design&lt;/h3&gt;
&lt;p&gt;In the Relay-era flow:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay graph
  |
  V
lowering to TE / TIR
  |
  V
TIR scheduling and code generation
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;In the Relax/Unity flow:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;IRModule {
    Relax functions
    TIR PrimFuncs
    external functions
    runtime calls
}
      |
      V
IRModule-to-IRModule transformations
      |
      V
build
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is a major architectural change.&lt;/p&gt;
&lt;p&gt;The compiler can now examine and transform high-level dataflow, low-level TIR kernels, external calls, device information, symbolic shape, and runtime calls within the same module-level representation.&lt;/p&gt;
&lt;p&gt;This enables cross-level optimisation: high-level graph transformations can be informed by tensor-program details, and tensor-program generation can be shaped by high-level graph structure.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;65-pass-structure-graph-passes-versus-modular-irmodule-transformations&quot;&gt;6.5 Pass Structure: Graph Passes versus Modular IRModule Transformations&lt;/h3&gt;
&lt;p&gt;Relay passes are often expressed as transformations over Relay functions or Relay expressions.&lt;/p&gt;
&lt;p&gt;Relax places stronger emphasis on:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;IRModule -&amp;gt; IRModule
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This design fits naturally with TVMScript, structural equality, and testable compiler passes. A pass can be developed and tested by comparing:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;before IRModule
  |
  V   pass
after IRModule
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This improves engineering discipline. It becomes easier to inspect intermediate IR, write unit tests, build custom flows, and reason about transformation correctness.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;66-runtime-graph-executor--relay-vm-versus-relax-vm-and-modular-runtime-integration&quot;&gt;6.6 Runtime: Graph Executor / Relay VM versus Relax VM and Modular Runtime Integration&lt;/h3&gt;
&lt;p&gt;Relay workflows commonly used the Graph Executor or Relay VM.&lt;/p&gt;
&lt;p&gt;Relax naturally aligns with a more flexible runtime model, especially the Relax VM and runtime objects. This is important because modern ML inference is often not just static graph execution. LLM inference, for example, must manage:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;KV cache
paged attention metadata
sampling
token-by-token generation loops
dynamic sequence length
batch scheduling
runtime shape
external fused kernels
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax is therefore better positioned as an interface between compiler IR and runtime execution.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;7-unity-connection-and-the-integration-of-relax&quot;&gt;7. Unity Connection and the Integration of Relax&lt;/h2&gt;
&lt;h3 id=&quot;71-external-development-before-mainline-integration&quot;&gt;7.1 External Development Before Mainline Integration&lt;/h3&gt;
&lt;p&gt;Relax was initially developed partly outside the main TVM tree in order to reduce disruption to existing Relay users and production deployments. This explains why, for a period, users encountered references to:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;apache/tvm main branch
apache/tvm unity branch
mlc-ai-related documentation
Relax-related forks and experimental branches
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This separation was not accidental. It allowed the community to experiment with the new architecture while preserving the stability of existing Relay-based workflows.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;72-relax-as-the-future-graph-ir&quot;&gt;7.2 Relax as the Future Graph IR&lt;/h3&gt;
&lt;p&gt;Over time, the community direction became clearer. Relax moved from an exploratory ‘Relay Next’ effort towards the intended graph IR for new TVM development. Meanwhile, legacy components such as Relay, TE schedule, AutoTVM, and AutoScheduler remained important for existing users and historical workflows, but the new modular direction increasingly centred on:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax as graph/program IR
TensorIR as tensor-program IR
MetaSchedule as the primary automatic scheduling infrastructure
TVMScript as the inspectable IR surface
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This does not mean Relay disappears immediately. Relay still has historical importance, existing production users, mature frontend paths, and a large body of accumulated compiler logic. The more accurate interpretation is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;new development:
    favour Relax and the Unity modular flow

legacy production:
    Relay remains relevant where existing pipelines depend on it

long-term direction:
    move from Relay towards Relax as the principal graph IR
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h2 id=&quot;8-the-essential-transformation-relay-to-relax-to-unity&quot;&gt;8. The Essential Transformation: Relay to Relax to Unity&lt;/h2&gt;
&lt;h3 id=&quot;81-from-graph-ir-to-programme-ir&quot;&gt;8.1 From Graph IR to Program IR&lt;/h3&gt;

&lt;p&gt;Although Relay was more expressive than a traditional computation graph, it largely served as TVM’s graph-level IR.&lt;/p&gt;
&lt;p&gt;Relax shifts the abstraction closer to a complete machine learning program IR. It can represent not only high-level dataflow, but also:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;shape computation
runtime-dependent structure
tensor computation
runtime objects
external calls
device placement
memory planning boundaries
control and dataflow boundaries
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is increasingly necessary because modern model deployment is no longer just graph execution. LLM inference, for example, is graph computation plus runtime loops, cache management, memory scheduling, sampling, dynamic batching, and external kernel invocation.&lt;/p&gt;
&lt;p&gt;Relax therefore represents a move from:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;neural network graph IR
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;towards:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;machine learning programme IR
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h3 id=&quot;82-from-staged-lowering-to-cross-level-co-optimisation&quot;&gt;8.2 From Staged Lowering to Cross-Level Co-Optimisation&lt;/h3&gt;
&lt;p&gt;The Relay-era mental model was often:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;perform high-level graph optimisation first
then lower to low-level tensor computation
then schedule and generate kernels
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The Unity-era mental model is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;represent high-level and low-level programmes together
perform transformations across abstraction boundaries
allow graph and tensor-level decisions to inform each other
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This distinction is critical for performance.&lt;/p&gt;
&lt;p&gt;Consider attention optimisation. A graph-level representation may see:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;matmul -&amp;gt; softmax -&amp;gt; matmul
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;A Relax + TensorIR + external-kernel system can represent several alternatives:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;call_tir(fused_attention_tir)
call_external(&amp;quot;flashinfer.paged_attention&amp;quot;)
rewrite pattern into a vendor-specific kernel
preserve symbolic sequence length and dispatch different kernel variants
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This makes it significantly easier to integrate FlashAttention, FlashInfer, CUTLASS, cuBLASLt, TensorRT plugins, custom CUDA kernels, NPU backends, and other specialised execution paths.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;83-from-static-model-compilation-to-dynamic-workload-compilation&quot;&gt;8.3 From Static Model Compilation to Dynamic Workload Compilation&lt;/h3&gt;
&lt;p&gt;Early TVM workflows were often optimised for the following scenario:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;known model
known input shape
known target
compile once
run many times
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Modern AI infrastructure often faces a different scenario:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;partially dynamic shape
dynamic batch size
runtime dispatch
hybrid compiler/library execution
model-specific kernels
target-specific graph rewriting
quantisation variants
multi-device execution
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax’s symbolic shape, StructInfo system, runtime object support, &lt;code&gt;call_tir&lt;/code&gt;, and external call mechanisms make it more suitable for these workloads.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;84-from-operator-centric-compilation-to-irmodule-centric-compilation&quot;&gt;8.4 From Operator-Centric Compilation to IRModule-Centric Compilation&lt;/h3&gt;
&lt;p&gt;Relay-era workflows often felt like:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;import model
  |
  V
optimise graph
  |
  V
lower operators
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Unity-era workflows are more naturally expressed as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;construct IRModule
inspect IRModule
transform IRModule
schedule TIR
replace subgraphs
call external libraries
tune kernels
build runtime module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The IRModule becomes the central object of the compilation process.&lt;/p&gt;
&lt;p&gt;This is particularly valuable for AI infrastructure engineers. With TVMScript and Relax, one can inspect and modify the program at each stage:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;print(mod.script())
apply transformation pass
print(mod.script())
insert custom PrimFunc
replace Relax call with call_tir
lower to target-specific code
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is far more controllable than a black-box frontend-to-build pipeline.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;9-significance-for-llm-and-mlc-llm&quot;&gt;9. Significance for LLM and MLC-LLM&lt;/h2&gt;
&lt;h3 id=&quot;91-llm-inference-is-not-a-conventional-relay-workload&quot;&gt;9.1 LLM Inference Is Not a Conventional Relay Workload&lt;/h3&gt;
&lt;p&gt;LLM inference contains structures that are awkward to express in a traditional static graph compiler:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;prefill/decode separation
KV cache read/write
sequence length that grows dynamically
dynamic attention masks
sampling, top-k, and top-p
paged KV cache
tensor parallelism
pipeline parallelism
quantised weights
activation and weight layout transformations
external fused kernels
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Several questions arise naturally:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Should the KV cache be represented as a tensor or a runtime object?
Should the decode loop be inside or outside the graph?
Is sequence length a compile-time constant or runtime value?
How should paged-attention metadata be represented?
How should different kernel variants be dispatched?
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax provides a more suitable framework for these questions because it can represent graph structure, symbolic shape, TIR kernels, runtime objects, and external calls within the same system.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;92-value-of-relax-for-mlc-llm&quot;&gt;9.2 Value of Relax for MLC-LLM&lt;/h3&gt;
&lt;p&gt;MLC-LLM aims to compile LLMs across multiple backends, including CUDA, Metal, Vulkan, WebGPU, and CPU. Such a system requires:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;dynamic-shape support
TIR-level kernel optimisation
runtime-level model execution
cross-platform lowering
quantisation rewriting
library and external kernel integration
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;These requirements align naturally with Relax and Unity.&lt;/p&gt;
&lt;p&gt;A typical LLM compilation flow can be understood as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax graph:
    model architecture, high-level dataflow, module composition

TIR PrimFunc:
    matmul kernel
    decode attention kernel
    RMSNorm kernel
    rotary embedding kernel
    sampling kernel

External call:
    FlashInfer
    cuBLAS / cuBLASLt
    vendor library
    runtime packed function
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax makes it possible to represent these elements within one IRModule instead of scattering logic across frontend import, Relay passes, TE schedules, runtime glue code, and external backend hooks.&lt;/p&gt;
&lt;hr&gt;
&lt;h3 id=&quot;93-relax-and-model-level-rewriting&quot;&gt;9.3 Relax and Model-Level Rewriting&lt;/h3&gt;
&lt;p&gt;Modern LLM deployment often relies on model-level rewriting. For example, a floating-point matrix multiplication may be rewritten into a quantised path.&lt;/p&gt;
&lt;p&gt;At a high level, a graph expression such as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;matmul(x, w)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;may become:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;dequantize(w_q, scale, zero_point)
matmul(x, dequantized_w)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;or, more efficiently:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;call_tir(quantized_matmul_kernel, x, w_q, scale, ...)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;or even:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;call_external(&amp;quot;cutlass_int4_gemm&amp;quot;, ...)
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax is better suited to these transformations because it can express graph-level patterns and then lower selected subgraphs either into TIR or into external libraries.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;10-significance-for-hardware-vendors-and-ai-engine-engineering&quot;&gt;10. Significance for Hardware Vendors and AI Engine Engineering&lt;/h2&gt;
&lt;p&gt;Relax and Unity are also important for hardware vendors and AI Engine engineers.&lt;/p&gt;
&lt;p&gt;A hardware backend often needs to express or control:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;custom memory hierarchy
custom tensor core or matrix engine
custom DMA behaviour
custom layout
custom vector ISA
custom runtime integration
custom graph partitioning
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relay-era TVM supported many of these through BYOC, external codegen, and custom operators. However, the engineering experience could be heavy and fragmented.&lt;/p&gt;
&lt;p&gt;Unity aims to make this integration more natural:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;graph partition
  |
  V
external library or accelerator call
  |
  V
TIR scheduling or external code generation
  |
  V
runtime integration
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Relax + TensorIR allows hardware vendors to participate simultaneously in:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;high-level pattern rewriting
layout propagation
operator fusion
TIR scheduling
tensorisation
runtime calls
device placement
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;This is particularly relevant for heterogeneous execution across CPU, GPU, and specialised accelerators.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;11-common-misinterpretations&quot;&gt;11. Common Misinterpretations&lt;/h2&gt;
&lt;h3 id=&quot;111-relax-is-not-merely-a-new-name-for-relay&quot;&gt;11.1 Relax Is Not Merely a New Name for Relay&lt;/h3&gt;
&lt;p&gt;Incorrect interpretation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay was renamed Relax.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;More accurate interpretation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax inherits Relay&amp;#39;s role as the high-level graph IR,
but redesigns the abstraction around symbolic shape,
TensorIR integration, modular transformation, and the Unity architecture.
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;112-unity-is-not-identical-to-relax&quot;&gt;11.2 Unity Is Not Identical to Relax&lt;/h3&gt;
&lt;p&gt;Incorrect interpretation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Unity = Relax.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;More accurate interpretation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Unity:
    the next-generation modular compilation architecture of TVM

Relax:
    the high-level graph/programme IR within Unity

TensorIR:
    the tensor-program IR within Unity

TVMScript:
    the inspectable and writable representation layer

MetaSchedule / DLight:
    automatic and rule-based scheduling infrastructure

Runtime / VM / external codegen:
    execution and backend integration layer
&lt;/code&gt;&lt;/pre&gt;
&lt;h3 id=&quot;113-relay-does-not-disappear-immediately&quot;&gt;11.3 Relay Does Not Disappear Immediately&lt;/h3&gt;
&lt;p&gt;Relay has a large amount of historical code, mature frontend infrastructure, many passes, and existing production users.&lt;/p&gt;
&lt;p&gt;A realistic view is:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relay:
    still important for historical understanding and existing deployments

Relax:
    preferred direction for new graph-level development

Unity:
    broader system direction for future TVM compilation workflows
&lt;/code&gt;&lt;/pre&gt;
&lt;hr&gt;
&lt;h2 id=&quot;12-historical-timeline&quot;&gt;12. Historical Timeline&lt;/h2&gt;
&lt;h3 id=&quot;2017-2018-from-nnvm-to-relay&quot;&gt;2017-2018: From NNVM to Relay&lt;/h3&gt;
&lt;p&gt;TVM’s early graph-level representation was NNVM. Around 2018, Relay was introduced as a more expressive high-level IR, intended to solve the limitations of conventional graph representations by adding typed functional program structure.&lt;/p&gt;
&lt;h3 id=&quot;2018-2021-relay-as-the-core-graph-ir&quot;&gt;2018-2021: Relay as the Core Graph IR&lt;/h3&gt;
&lt;p&gt;Relay became the main graph-level IR in TVM. It supported frontend import, graph optimisation, operator fusion, layout rewriting, quantisation workflows, and BYOC-style integration.&lt;/p&gt;
&lt;h3 id=&quot;2021-the-tvm-unity-vision&quot;&gt;2021: The TVM Unity Vision&lt;/h3&gt;
&lt;p&gt;The Unity vision was proposed as a new direction for TVM. Its goal was to break boundaries between computational graphs, tensor programs, libraries, runtimes, and hardware primitives, enabling deeper cross-layer optimisation.&lt;/p&gt;
&lt;h3 id=&quot;2022-relax-appears-as-relay-next&quot;&gt;2022: Relax Appears as Relay Next&lt;/h3&gt;
&lt;p&gt;Relax emerged as the high-level IR designed for Unity. It aimed to evolve Relay around symbolic shape, dynamic workload representation, TensorIR integration, modular transformation, and improved portability.&lt;/p&gt;
&lt;h3 id=&quot;2022-unity-connection&quot;&gt;2022: Unity Connection&lt;/h3&gt;
&lt;p&gt;The Unity connection work aimed to bring Relax into the broader TVM ecosystem as a key component of the new end-to-end modular flow, together with TensorIR, MetaSchedule, and TVMScript.&lt;/p&gt;
&lt;h3 id=&quot;2023-relax-remains-actively-developed&quot;&gt;2023: Relax Remains Actively Developed&lt;/h3&gt;
&lt;p&gt;Relax continued to develop actively in the Unity-oriented workflow. The community direction became increasingly clear: Relax was no longer merely an experiment, but the intended successor direction for high-level graph compilation.&lt;/p&gt;
&lt;h3 id=&quot;2024-and-after-transition-from-legacy-components-to-the-new-modular-flow&quot;&gt;2024 and After: Transition from Legacy Components to the New Modular Flow&lt;/h3&gt;
&lt;p&gt;The TVM community increasingly encouraged new development around Relax, TensorIR, MetaSchedule, TVMScript, and modular IRModule-level transformation. Legacy components such as Relay, TE schedule, AutoTVM, and AutoScheduler remained relevant for existing workflows, but the long-term direction moved towards the Unity stack.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;13-how-a-relay-engineer-should-understand-relax&quot;&gt;13. How a Relay Engineer Should Understand Relax&lt;/h2&gt;
&lt;p&gt;For engineers familiar with Relay, the following conceptual mapping is useful:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Relay Concept&lt;/th&gt;
&lt;th&gt;Relax / Unity Interpretation&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Relay Function&lt;/td&gt;
&lt;td&gt;Relax Function&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay Expr&lt;/td&gt;
&lt;td&gt;Relax Expr / Binding&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay Type&lt;/td&gt;
&lt;td&gt;StructInfo&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay graph pass&lt;/td&gt;
&lt;td&gt;Relax / IRModule transformation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay operator fusion&lt;/td&gt;
&lt;td&gt;Relax pattern rewriting plus &lt;code&gt;call_tir&lt;/code&gt; or external call&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay build&lt;/td&gt;
&lt;td&gt;Relax / Unity build flow&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;TE lowering&lt;/td&gt;
&lt;td&gt;TensorIR PrimFunc and &lt;code&gt;call_tir&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Relay VM&lt;/td&gt;
&lt;td&gt;Relax VM&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BYOC&lt;/td&gt;
&lt;td&gt;Relax external call / codegen integration / modular backend&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Static-shape graph&lt;/td&gt;
&lt;td&gt;Symbolic-shape-aware program&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Graph-only optimisation&lt;/td&gt;
&lt;td&gt;Cross-level graph + TIR optimisation&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;However, this table only captures surface-level correspondence. The deeper shift is that Relax encourages engineers to treat model compilation as a sequence of transformations over an IRModule, rather than as a fixed frontend-import-to-build pipeline.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;14-a-practical-mental-model-of-the-relaxunity-flow&quot;&gt;14. A Practical Mental Model of the Relax/Unity Flow&lt;/h2&gt;
&lt;p&gt;A typical Relax/Unity compilation flow can be understood as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;1. Import or construct the model
   PyTorch / ONNX / TensorFlow / NNModule / manual TVMScript
      |
      V

2. Produce a Relax IRModule
   Relax functions + symbolic shape + high-level operations
      |
      V

3. Apply graph-level transformations
   legalisation, fusion, pattern rewriting, quantisation, layout rewriting
      |
      V

4. Lower selected parts to TensorIR
   call_tir, PrimFunc generation, external calls
      |
      V

5. Schedule TensorIR
   manual schedule, DLight, MetaSchedule, tensorisation
      |
      V

6. Integrate with runtime
   Relax VM, external libraries, packed functions, device placement
      |
      V

7. Build deployable module
   target-specific code generation and runtime module
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Compared with the Relay-era pipeline, the key difference is that steps 3, 4, and 5 are no longer separated by rigid one-way boundaries. They can interact through IRModule-level transformations.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;15-why-this-matters-for-ai-infrastructure-engineers&quot;&gt;15. Why This Matters for AI Infrastructure Engineers&lt;/h2&gt;
&lt;p&gt;From the perspective of AI infrastructure, the transition from Relay to Relax and Unity means that TVM is evolving from a neural-network compiler into a programmable ML systems compilation infrastructure.&lt;/p&gt;
&lt;p&gt;For embedded deployment, autonomous driving, LLM inference, NPU/GPU backends, model compression, runtime optimisation, and custom kernel integration, Relax/Unity offers several practical advantages.&lt;/p&gt;
&lt;p&gt;First, model structure becomes more controllable. Engineers can perform structured rewrites at the Relax level rather than relying on runtime hacks or low-level kernel replacement alone.&lt;/p&gt;
&lt;p&gt;Second, dynamic shape becomes a first-class concern. This is essential for Transformers, BEV models, VLA models, multimodal models, and LLM workloads.&lt;/p&gt;
&lt;p&gt;Third, integration with TIR becomes more direct. Hand-written TIR, generated TIR, scheduled TIR, and Relax graph programs can coexist in the same IRModule.&lt;/p&gt;
&lt;p&gt;Fourth, external high-performance kernels become easier to integrate. Examples include FlashInfer, CUTLASS, cuBLASLt, TensorRT plugins, vendor libraries, and custom CUDA kernels.&lt;/p&gt;
&lt;p&gt;Fifth, end-to-end performance experiments become more systematic. One can compare different implementation paths:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Relax operator path
call_tir path
external library path
manual TIR schedule path
MetaSchedule-tuned path
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Sixth, compiler engineering becomes more maintainable. TVMScript, structural equality, and IRModule-level transformation provide a more inspectable and testable workflow than opaque build pipelines.&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;16-conclusion&quot;&gt;16. Conclusion&lt;/h2&gt;
&lt;p&gt;Relay was the decisive step that transformed TVM from an early tensor compiler with graph support into a mature deep learning compiler. It introduced a typed functional high-level IR and enabled robust graph-level optimisation.&lt;/p&gt;
&lt;p&gt;Relax is the successor direction of Relay under the Unity vision. It does not reject Relay’s achievements; it reconstructs the high-level abstraction to address the requirements of modern workloads: symbolic shape, dynamic execution, LLM inference, runtime objects, external kernels, and cross-level optimisation with TensorIR.&lt;/p&gt;
&lt;p&gt;Unity is the broader architectural framework. It attempts to integrate Relax, TensorIR, TVMScript, MetaSchedule, runtime systems, libraries, external code generation, and hardware-specific primitives into a modular and composable compilation system.&lt;/p&gt;
&lt;p&gt;The historical evolution can therefore be summarised as:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;NNVM:
    computation graph as model representation

Relay:
    typed functional IR as graph-level compiler IR

Relax:
    symbolic-shape-aware dataflow/programme IR integrated with TensorIR

Unity:
    modular cross-layer ML compilation system
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For current and future work in TVM, MLC-LLM, AI Engine engineering, and high-performance model deployment, the most important direction is not merely to understand Relay passes or traditional TE schedules. Those remain historically and technically valuable. However, the main trajectory is now Relax + TensorIR + TVMScript + MetaSchedule + runtime integration.&lt;/p&gt;
&lt;p&gt;Relay explains how TVM became a mature graph compiler. Relax and Unity explain where TVM is going.&lt;/p&gt;
&lt;pre&gt;&lt;code&gt;
&lt;/code&gt;&lt;/pre&gt;
</content>
  </entry>
  <entry>
    <title>Systematic Analysis and Engineering Solution for Slow SSH Connections (Terminal + VSCode)</title>
    <link href="https://www.ailven.cc/posts/devops-260415.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-260415.html</id>
    <published>2026-04-15T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A layered diagnosis of slow SSH logins and VSCode Remote-SSH stalls, showing why repeated session setup dominates latency and how ControlMaster-based multiplexing fixes it.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 15 April 2026&lt;br&gt;&lt;strong&gt;Latest Update Date&lt;/strong&gt;: 15 April 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;An SSH connection that takes four to six seconds to open, then behaves perfectly once it is open, is a particular kind of problem. It rules out most of what people reach for first. The delay is not throughput, not the remote machine&amp;#39;s load, and not the login shell. It is the cost of establishing a session, paid again on every connection.&lt;/p&gt;
&lt;h2 id=&quot;the-symptom&quot;&gt;The symptom&lt;/h2&gt;
&lt;p&gt;From a terminal, &lt;code&gt;ssh user@host&lt;/code&gt; sits for four to six seconds before the prompt appears. A command that does no work at all costs the same:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;time ssh host &amp;#39;true&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;That still returns in roughly five seconds. Once an interactive shell is up, however, everything inside it is responsive. That is the first real clue, because it separates session setup from session use — and only the former is broken.&lt;/p&gt;
&lt;p&gt;VSCode Remote-SSH shows the same cause with worse symptoms. The initial connection is slow; the status bar lingers on &lt;code&gt;Installing VSCode Server&lt;/code&gt; and &lt;code&gt;Starting VSCode Server&lt;/code&gt;; extension loading and file-tree initialisation drag; and reconnection after a drop is unreliable.&lt;/p&gt;
&lt;h2 id=&quot;ruling-out-the-usual-suspects&quot;&gt;Ruling out the usual suspects&lt;/h2&gt;
&lt;p&gt;Four explanations get offered first, and each can be tested in a few seconds.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Assumption&lt;/th&gt;
&lt;th&gt;Reality&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Insufficient network bandwidth&lt;/td&gt;
&lt;td&gt;False&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Poor remote server performance&lt;/td&gt;
&lt;td&gt;False&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Heavy shell configuration (&lt;code&gt;.zshrc&lt;/code&gt;)&lt;/td&gt;
&lt;td&gt;False&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Git or proxy-related issues&lt;/td&gt;
&lt;td&gt;Irrelevant&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Bandwidth is ruled out by the shape of the problem: a &lt;code&gt;true&lt;/code&gt; command transfers no payload, so throughput cannot be what costs five seconds. Server load is ruled out by the interactive shell being responsive. The shell configuration is ruled out directly:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;time ssh host &amp;#39;/bin/sh -c &amp;quot;true&amp;quot;&amp;#39;
time ssh host &amp;#39;/usr/bin/env -i /bin/sh -c &amp;quot;true&amp;quot;&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The second form strips the environment entirely. If the delay survives that, no amount of &lt;code&gt;.zshrc&lt;/code&gt; tuning will touch it. It does survive it.&lt;/p&gt;
&lt;h2 id=&quot;where-the-time-actually-goes&quot;&gt;Where the time actually goes&lt;/h2&gt;
&lt;p&gt;A connection can be decomposed into five stages, and each has to be measured on its own before anything can be concluded:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;[1] TCP connection establishment
[2] SSH handshake (KEX, cipher negotiation, authentication)
[3] PAM and session initialisation
[4] Shell startup
[5] Remote command execution
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Stage 1 is healthy. A bare TCP probe returns in under ten milliseconds:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;time nc -vz &amp;lt;host&amp;gt; 22
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Stage 2 is healthy too. Under &lt;code&gt;ssh -vvv &amp;lt;host&amp;gt;&lt;/code&gt;, the three checkpoints that matter — &lt;code&gt;Connection established&lt;/code&gt;, &lt;code&gt;SSH2_MSG_KEXINIT&lt;/code&gt;, &lt;code&gt;Authenticated to ...&lt;/code&gt; — pass without any visible gap. Protocol negotiation is not the problem.&lt;/p&gt;
&lt;p&gt;Stages 3 and 4 are cheap. Checking &lt;code&gt;/etc/update-motd.d&lt;/code&gt;, &lt;code&gt;/etc/pam.d/sshd&lt;/code&gt; and &lt;code&gt;/etc/environment&lt;/code&gt; accounts for under 0.1 seconds in total, and the environment-stripped test above already excluded shell startup.&lt;/p&gt;
&lt;p&gt;That leaves stage 5, and the measurement there is the one that reframes the problem:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;time ssh host &amp;#39;true&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Roughly five seconds for a command whose remote side does nothing. The cost is not in any stage of the work. It is in reaching the stage where work could begin.&lt;/p&gt;
&lt;h2 id=&quot;connection-multiplexing&quot;&gt;Connection multiplexing&lt;/h2&gt;
&lt;p&gt;The fix is to stop paying that setup cost per connection. SSH ControlMaster lets the first connection become a master, with later connections riding on its transport:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;ssh -M -S ~/.ssh/cm-host -fnNT host
ssh -S ~/.ssh/cm-host host &amp;#39;true&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first command opens the master in the background. The second reuses it, and the difference is immediate:&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Operation&lt;/th&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Initial master connection&lt;/td&gt;
&lt;td&gt;2 to 3 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subsequent commands&lt;/td&gt;
&lt;td&gt;Around 1 second&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Made permanent in &lt;code&gt;~/.ssh/config&lt;/code&gt;:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ssh&quot;&gt;Host ali_server
  HostName &amp;lt;your_host&amp;gt;
  User &amp;lt;your_user&amp;gt;

  ControlMaster auto
  ControlPath ~/.ssh/cm-%r@%h:%p
  ControlPersist 10m
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;ControlMaster auto&lt;/code&gt; reuses an existing master when there is one and creates it otherwise, so the same &lt;code&gt;ssh ali_server&lt;/code&gt; invocation works whether or not a master is already up. &lt;code&gt;ControlPersist 10m&lt;/code&gt; keeps the master alive for ten minutes after the last session closes, which is what makes the reuse useful across a working session rather than only within a single command. Verification is a matter of running three commands in sequence and watching the first one pay the cost:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;time ssh ali_server &amp;#39;true&amp;#39;
time ssh ali_server &amp;#39;echo OK&amp;#39;
time ssh ali_server &amp;#39;echo AGAIN&amp;#39;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Two further settings cut the initial negotiation:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-ssh&quot;&gt;Host ali_server
  Compression no
  GSSAPIAuthentication no
  PreferredAuthentications publickey
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Compression buys nothing on a link whose problem is round-trip count rather than volume, and GSSAPI authentication is a negotiation round trip that a public-key setup never uses. Restricting &lt;code&gt;PreferredAuthentications&lt;/code&gt; to &lt;code&gt;publickey&lt;/code&gt; skips the methods that would otherwise be tried and rejected first.&lt;/p&gt;
&lt;p&gt;For VSCode, the important part is making the editor use the same client and therefore the same master:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-json&quot;&gt;{
  &amp;quot;remote.SSH.useLocalServer&amp;quot;: true,
  &amp;quot;remote.SSH.enableDynamicForwarding&amp;quot;: false,
  &amp;quot;remote.SSH.path&amp;quot;: &amp;quot;/usr/bin/ssh&amp;quot;
}
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;useLocalServer&lt;/code&gt; routes Remote-SSH through the local SSH binary rather than a bundled one, which is what lets it find &lt;code&gt;ControlPath&lt;/code&gt;. &lt;code&gt;enableDynamicForwarding&lt;/code&gt; is disabled because VSCode opens a forwarding channel per connection by default, and those channels are exactly the short-lived connections the master exists to avoid.&lt;/p&gt;
&lt;h2 id=&quot;results&quot;&gt;Results&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Scenario&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Single SSH command&lt;/td&gt;
&lt;td&gt;4 to 6 seconds&lt;/td&gt;
&lt;td&gt;1 to 2 seconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Multiple SSH commands&lt;/td&gt;
&lt;td&gt;Each 4 to 6 seconds&lt;/td&gt;
&lt;td&gt;Approximately 1 second&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;VSCode Remote-SSH&lt;/td&gt;
&lt;td&gt;Very slow&lt;/td&gt;
&lt;td&gt;Significantly improved&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;The generalisation worth keeping is about diagnosis order. When a connection is slow to establish but responsive afterwards, the fault is in setup, and setup cost is multiplied by connection count. Multiplexing attacks that product directly, whereas tuning the network or the shell attacks factors that were never the problem. If more is needed, the same logic points at reducing connection frequency — long-lived sessions under &lt;code&gt;tmux&lt;/code&gt;, or a persistent path such as WireGuard — rather than at making each connection marginally faster.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>When Love Degenerates into Valuation</title>
    <link href="https://www.ailven.cc/posts/moment-260414.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260414.html</id>
    <published>2026-04-14T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A philosophical analysis of love, personhood, and instrumental rationality, contrasting fidelity that continues to include the other with valuation that discards them once a better world appears.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;From ‘I Want to Go to a Better World with You’ to ‘You Are No Longer Good Enough for Me’ – An Anatomy of Human Nature&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 14 April 2026&lt;br&gt;&lt;strong&gt;Latest Update Date&lt;/strong&gt;: 14 April 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;True love says: I have seen a better world, and I want to go there with you.&lt;br&gt;False love says: I have seen a better world; you are no longer good enough for me, so do not hold me back.&lt;/p&gt;
&lt;p&gt;The difference between these two statements looks like a difference in temperature. It is in fact the difference between two utterly distinct philosophies of human relationship – more precisely, between two utterly distinct understandings of what a human being is.&lt;/p&gt;
&lt;p&gt;The former acknowledges the symbiotic nature of a relationship; it acknowledges that love entails walking together, supporting one another, waiting, and making room for each other’s incompleteness. The latter treats a relationship as a provisional arrangement, and treats a person as an asset that may appreciate or depreciate in value. The moment a better platform, a superior path, or a more advantageous future appears, the old relationship is instantly demoted from a ‘community of destiny’ to a ‘historical burden’. The former still treats a person as a person. The latter has already reduced a person to a tool; it merely wraps that instrumental rationality in a layer of refined language, mature rhetoric, composure, and an air of seeming transcendence over vulgar reality.&lt;/p&gt;
&lt;p&gt;Thus, when many people part, abandon, betray, or turn their faces away, they speak of growth while calculating returns. They speak of incompatibility while thinking that you are no longer worth the cost. What they mean by wishing each other well is: do not delay my ascent.&lt;/p&gt;
&lt;p&gt;People change; that much is unremarkable. The question is whether the change moves them towards truthfulness, or towards a greater skill in finding respectable narratives for selfishness.&lt;/p&gt;
&lt;h2 id=&quot;i-the-truth-of-love-is-not-first-of-all-an-emotional-question-but-a-question-of-how-one-regards-the-other&quot;&gt;I. The truth of love is not, first of all, an emotional question, but a question of how one regards the Other&lt;/h2&gt;
&lt;p&gt;Every intimate relationship, in the final analysis, cannot avoid one most brutal question: what exactly do you take the other person to be?&lt;/p&gt;
&lt;p&gt;This is a more fundamental question than ‘Do you love me?’ because the word &lt;em&gt;love&lt;/em&gt; is far too easily abused. It may mean desire, dependence, possession, illusion, projection, or a form of self-intoxication that has not yet been tested by reality. Many people claim to love someone deeply, when in fact what they love is the version of themselves reflected beside that person: admired, affirmed, needed, and made radiant.&lt;/p&gt;
&lt;p&gt;Martin Buber distinguished between the ‘I–Thou’ relation and the ‘I–It’ relation. The former regards the other as an irreplaceable being; the latter regards the other as an object, a resource, a tool, a means. One of the tragedies of human life is that many intimate relationships begin in the language of ‘I–Thou’, yet, at decisive moments, reveal themselves to be thoroughly structured by ‘I–It’.&lt;/p&gt;
&lt;p&gt;When all is going well, the other is a soulmate, the only one, destiny, a spiritual companion; but the moment interests, prospects, status, class mobility, living costs, external judgement, and opportunities for upward movement begin to intervene, the way many people treat a partner suddenly becomes as rational as the way a firm handles an underperforming asset: assessment, comparison, weighing, depreciation, divestment, disposal.&lt;/p&gt;
&lt;p&gt;This does not mean that they were lying from the beginning. The more frightening truth is that they may indeed once have been sincere – moved, attached, invested. Yet that sincerity was never unconditional. It existed only so long as you continued to satisfy their expectations; once you no longer met that hidden and severe function of value, what had been called devotion swiftly withdrew its capital.&lt;/p&gt;
&lt;p&gt;This is the first truth of love: many relationships rest upon a provisional recognition of the value a person appears to carry. The person &lt;em&gt;as such&lt;/em&gt; is never the foundation.&lt;/p&gt;
&lt;h2 id=&quot;ii-the-thing-at-which-modern-society-excels-most-is-turning-every-relationship-into-a-transaction&quot;&gt;II. The thing at which modern society excels most is turning every relationship into a transaction&lt;/h2&gt;
&lt;p&gt;If ancient society arranged relationships more directly through lineage, clan order, property, and marriage hierarchy, modern society is cleverer, and colder. It no longer crudely commands you whom you &lt;em&gt;must&lt;/em&gt; marry or &lt;em&gt;must&lt;/em&gt; remain with. Instead, through education, media, professional life, urban stratification, consumerism, and the omnipresent machinery of competition, it continuously instils an almost irresistible standard of judgement:&lt;/p&gt;
&lt;p&gt;Who has the brighter future is more deserving of investment;&lt;br&gt;Who grows faster is more worthy of love;&lt;br&gt;Who occupies the higher platform has the greater right to eliminate others.&lt;br&gt;And whoever stays where they are has earned being left behind.&lt;/p&gt;
&lt;p&gt;Thus, love – the very relationship that ought most to resist the invasion of utility – becomes one of the easiest sites for social discipline to penetrate. Two people appear to be in love, while in fact they are jointly participating in a covert process of valuation. The degree to which a person is loved increasingly depends on whether he or she can provide a respectable life, a trajectory of growth, emotional utility, social gain, class aspiration, and a sense of future security. What is called ‘compatibility’ often means little more than ‘the ratio of returns to costs remains acceptable’; what is called ‘having no future together’ often means nothing more than ‘your valuation no longer looks promising’.&lt;/p&gt;
&lt;p&gt;Hegel pointed out that self-consciousness depends upon recognition. Yet modern society has rendered recognition extraordinarily mercenary: it recognises only your performance, your potential, your visibility, and your capacity for upward mobility. Thus, what many people suffer in intimate relationships is a sentence rather different from ‘I no longer love you as a person’: ‘Given your present configuration, you are no longer worth my continued investment.’&lt;/p&gt;
&lt;p&gt;This is why many separations produce a humiliation almost ontological in its force. What truly pierces is less the loss of one concrete individual than the judgement implied by that person’s departure: you are not enough, you are not worth it, you have fallen behind, you have no right to enter the future I wish to inhabit.&lt;/p&gt;
&lt;p&gt;Modern intimate relationships are so cruel precisely because they inherit the core logic of the market: everything can be compared, everything must justify its rate of return, and everything that cannot continue to appreciate ought to be replaced.&lt;/p&gt;
&lt;h2 id=&quot;iii-what-is-called-mature-lucidity-is-very-often-merely-refined-egoism-in-its-civilised-dress&quot;&gt;III. What is called ‘mature lucidity’ is very often merely refined egoism in its civilised dress&lt;/h2&gt;
&lt;p&gt;What deserves vigilance today is not naked selfishness, but selfishness that has been exquisitely civilised.&lt;/p&gt;
&lt;p&gt;The truly crude person is not the most dangerous, for greed, contempt, opportunism, and the will to exploit are almost written on his face. What is dangerous is the person of cultivated language, rigorous logic, polished taste, respectable bearing, and perhaps even a slight philosophical air. Such people do not say, ‘You are no longer of use to me, so I do not want you any more.’ They say, ‘What I seek is a higher dimension of growth.’ Where a plainer speaker would say ‘you are beneath me’, they say, ‘Our levels of cognition are no longer on the same plane.’ A demand for a stronger provider becomes a wish for someone who can truly stand alongside them; the complaint that you are weighing them down becomes the declaration that they must remain faithful to the rhythm of their own life.&lt;/p&gt;
&lt;p&gt;The most insidious thing about this rhetoric is that it disguises selfishness as self-respect, utility as lucidity, betrayal as growth, abandonment as responsibility towards the future.&lt;/p&gt;
&lt;p&gt;Sartre called this &lt;em&gt;bad faith&lt;/em&gt;. Human beings cannot bear to see themselves as merely vulgar creatures of profit, calculation, snobbery, and submission to advantage, and so they manufacture a noble narrative for their own conduct. In this way, they may enjoy real benefit while escaping the ugliness of moral self-knowledge. More than that, they may come sincerely to believe that they are pursuing a better self, respecting the selection of fate, enacting the evolution of life itself.&lt;/p&gt;
&lt;p&gt;Such people are often especially adept at one thing: when no conflict of interest exists, they appear intensely idealistic; once core interests are touched, they switch instantly to the most thoroughgoing instrumental rationality. One moment they are speaking of tenderness, soul, romance, meaning, and mutual growth; the next they are calculating marginal returns, opportunity costs, future yield, emotional burden, and the timing of exit. More disgusting still, they beautify this switch under the name of ‘maturity’.&lt;/p&gt;
&lt;p&gt;What is called ‘maturity’, in this context, means learning at last to bear no costs for others and to be responsible only for oneself; learning to withdraw affection at the proper moment, lest an inefficient relationship continue to erode one’s path of ascent; learning to subsume all feeling under the logic of cost-benefit analysis.&lt;/p&gt;
&lt;p&gt;This ‘maturity’ is not growth, but the marketisation of personality.&lt;/p&gt;
&lt;h2 id=&quot;iv-later-circumstances-reveal-a-structure-that-was-always-there&quot;&gt;IV. Later circumstances reveal a structure that was always there&lt;/h2&gt;
&lt;p&gt;People like to say: ‘He changed.’ ‘She was not like this before.’&lt;br&gt;Most of the time, this is only half true.&lt;/p&gt;
&lt;p&gt;A more accurate formulation would be: at certain stages, a person’s underlying structure has not yet been fully activated; when new resources, opportunities, status, choices, and avenues of mobility appear, that concealed structure finally acquires room to unfold.&lt;/p&gt;
&lt;p&gt;A person who is weak, dependent, unseen, and not yet seated at the table may appear extremely gentle, loyal, affectionate, humble, and grateful. For at that stage, he needs connection, support, you, the relationship, the stability it offers, the emotional comfort it provides, perhaps even the resources by which it lifts him. His sincerity at that stage is real enough, but it remains contingent upon the ongoing usefulness of the relationship to himself.&lt;/p&gt;
&lt;p&gt;Once circumstances change – educational elevation, professional elevation, rising income, a renewed social circle, a swollen self-estimation, rapidly expanding options – his understanding of the relationship may alter at once. What was previously received as warmth begins to seem cumbersome; what had once been taken as a shared future comes to feel like a constraint. Then old covenants are reinterpreted, old attachments renamed, old investments revalued.&lt;/p&gt;
&lt;p&gt;The most typical gesture is the revision of history.&lt;/p&gt;
&lt;p&gt;He will say that he was simply young and unknowing;&lt;br&gt;That it was never love, only dependence;&lt;br&gt;That any two people could have replicated those sweet moments;&lt;br&gt;That you failed to grow.&lt;/p&gt;
&lt;p&gt;This is a form of moral self-preservation, and it is more than a correction of memory. Unless the past, the other person, and the relationship’s former reality are all diminished, he would have to admit that his departure now contains betrayal, calculation, and a reorientation of value. To avoid facing that, the easiest course is to degrade the past itself: if it was never truly lofty, then casting it aside need incur no guilt.&lt;/p&gt;
&lt;p&gt;Thus, many people have not suddenly grown heartless. They have merely reached the point at which they no longer need your existence in order to stabilise their own. When they can independently obtain confirmation from a higher order of reality, you are transformed from ‘the person of destiny’ into a ‘stage-specific arrangement’.&lt;/p&gt;
&lt;h2 id=&quot;v-what-wounds-most-is-never-departure-itself-but-the-humiliation-of-value-that-departs-with-it&quot;&gt;V. What wounds most is never departure itself, but the humiliation of value that departs with it&lt;/h2&gt;
&lt;p&gt;Departure is not necessarily cruel. Human intentions change, emotions fade, relationships break apart: this is part of reality. What often leaves people unable to recover for years is less the fact of the leaving than the verdict attached to it: ‘before leaving, you proved me inferior’.&lt;/p&gt;
&lt;p&gt;Many people, when ending a relationship, are not content merely to exit; they also require the moral high ground. They wish to go, and to have you believe that the fault was yours: that your inadequacy explains the end, that you were always unworthy of the commitment you received, that you failed to grow into someone worth retaining.&lt;/p&gt;
&lt;p&gt;Thus separation ceases to be the termination of a relationship and becomes a public or private trial of value. What one suffers is not merely the end of the relationship. It is a liquidation of personality.&lt;/p&gt;
&lt;p&gt;This humiliation is so effective because it strikes at one of humanity’s deepest fears: not being chosen, not being needed, not being regarded as a person of worth. In intimacy, that fear is amplified to the utmost. For the denial of a stranger remains a social judgement; the denial of one who was closest is easily mistaken for existential truth. One instinctively thinks: if even the person nearest to me saw me this way, then perhaps I truly am unworthy.&lt;/p&gt;
&lt;p&gt;Many people’s later collapse is, on the surface, caused by heartbreak; in fact it is caused by the loss of narrative authority. They no longer possess the right to interpret that relationship, nor the right to testify to their own value. The whole rhetorical edifice left behind by the departing person – you are childish, low, burdensome, suffocating, inadequate, unworthy – remains within the soul like a judicial sentence, furnishing the grounds for a new round of self-condemnation.&lt;/p&gt;
&lt;p&gt;And this is precisely one of the most sinister aspects of modern relationships: one party leaves, and the other loses the relationship, and with it the right to bear witness on his or her own behalf.&lt;/p&gt;
&lt;h2 id=&quot;vi-the-phrase-i-merely-want-a-better-life-often-means-i-am-prepared-to-sacrifice-others-in-exchange-for-it&quot;&gt;VI. The phrase ‘I merely want a better life’ often means: ‘I am prepared to sacrifice others in exchange for it’&lt;/h2&gt;
&lt;p&gt;A sentence must be spoken here that many people do not wish to hear: the so-called pursuit of ‘a better life’ is, in most cases, not a pure proposition at all.&lt;/p&gt;
&lt;p&gt;For ‘better’ is not itself guilty. The question is what cost one is prepared to pay for this ‘better’, and which costs one is prepared to shift on to others.&lt;/p&gt;
&lt;p&gt;If someone says: I have seen bigger cities, better education, higher platforms, a wider world, and therefore I hope that we may strive together, increase together, and bear together the cost of migration and reconstruction – then this is a joint ascent, an ethic of partnership, the treatment of the relationship as a common enterprise.&lt;/p&gt;
&lt;p&gt;But if someone says: I have seen a larger world, and therefore I realise that you are no longer good enough, that you cannot keep up, that you must not cling to me, that you must not delay my pace – then there is nothing noble here. There is only the plainest, oldest, and commonest feature of human nature: when I possess a better option, I shall immediately reprice you.&lt;/p&gt;
&lt;p&gt;Nietzsche spoke of the ‘revaluation of values’, yet most real-world revaluation creates no higher value; it is only a more efficient submission to the logic of strength. Once a person senses that he has more capital, all the defects of the other which he could once tolerate are swiftly upgraded into unacceptable structural flaws. Once it was ‘we can change together’; later it becomes ‘I do not wish to go through change with you’. Once it was ‘we shall go together’; later it becomes ‘do not obstruct my going’.&lt;/p&gt;
&lt;p&gt;Observe that the change in ethical stance is what matters here, and the gap in capability is not. A true community does not require two people always to grow at the same speed; but it does at least require that the stronger party, before all hope is exhausted, not treat the weaker as a lagging component to be discarded. Modern cruelty lies precisely here: people admit without shame that they do not wish to wait, and wrap the unwillingness in the language of ‘I need not wrong myself’.&lt;/p&gt;
&lt;p&gt;In one sentence: what many people call their ideal life is nothing more than a permit that legitimises the abandonment of lower-value former relationships.&lt;/p&gt;
&lt;h2 id=&quot;vii-love-is-scarce-because-people-need-it-to-prove-their-own-success&quot;&gt;VII. Love is scarce because people need it to prove their own success&lt;/h2&gt;
&lt;p&gt;Modern people are famished for intimacy, more so than ever: to be understood, accompanied, and seen. The problem is that they increasingly struggle to allow love to remain love. They demand that it perform additional functions.&lt;/p&gt;
&lt;p&gt;Love must prove that one is worth envying;&lt;br&gt;Love must prove that one has not chosen badly;&lt;br&gt;Love must become a combined certificate of social display, class ascent, spiritual superiority, and existential correctness.&lt;/p&gt;
&lt;p&gt;Thus, a partner ceases to be merely a partner, and becomes part of one’s curriculum vitae, part of one’s identity, part of one’s imagined future. To be chosen now means demonstrating: &lt;em&gt;my life has turned out rather well&lt;/em&gt;. Conversely, to be abandoned is no longer merely an emotional defeat; it is often experienced as a social devaluation.&lt;/p&gt;
&lt;p&gt;In such a situation, love must inevitably be corrupted. For once love is made to bear the burden of proving one’s value, it can never truly be unconditional. You may accompany me through hardship, provided that hardship still leads to a better future; you may be lost, provided that your confusion does not reduce the valuation of my own life.&lt;/p&gt;
&lt;p&gt;The moment these premises are doubted, what is called feeling changes at once. Their attachment has always been tied to a larger enterprise: through the relationship with you, they seek confirmation of their own correctness.&lt;/p&gt;
&lt;p&gt;Erich Fromm wrote that love is a capacity, not an emotion. Yet modern society is more inclined to treat love as a prize: the more excellent one is, the more entitled one is to it; the brighter one’s future, the more deserving one is of long-term investment. In such an environment, anyone still capable of treating the other as an end rather than a means must be rare indeed. For this requires a person partially to withdraw from the entire social system of valuation, or at least to preserve within intimacy a territory not wholly governed by performance. Most people cannot do it, and the reason is not a lack of intelligence. They are unwilling to bear the risk.&lt;/p&gt;
&lt;p&gt;They would sooner lose love than lose the chance of winning.&lt;/p&gt;
&lt;h2 id=&quot;viii-the-most-widespread-feature-of-human-nature-is-conditional-benevolence&quot;&gt;VIII. The most widespread feature of human nature is conditional benevolence&lt;/h2&gt;
&lt;p&gt;If all this were reduced merely to ‘wickedness’, the matter would become too simple. The truly terrible thing about human nature is something much commoner than great evil: benevolence that lasts only so long as one remains safe, comfortable, and unthreatened; once core interests are involved, benevolence evaporates at once.&lt;/p&gt;
&lt;p&gt;This is not unique to modern society. Countless struggles for power, marriage alliances, calculations of rank, literary circles, court factions, and commercial partnerships throughout history repeatedly demonstrate the same truth: human kindness towards others is often founded upon the condition that no serious sacrifice be demanded of oneself. The moment the price becomes too high, so-called loyalty yields to self-preservation.&lt;/p&gt;
&lt;p&gt;Hobbes observed that competition, diffidence, and the pursuit of glory lie latent in the relation of man to man. Modern society has not abolished these things; it has merely wrapped them in more elegant forms. People today do not directly say, as they might once have done, ‘our families are not of equal standing’; instead they say, ‘our values do not align’. A plain ‘you have lost status’ becomes ‘I no longer feel we are on the same path’; a plain ‘I wish to climb a higher branch’ becomes ‘I must be responsible to my future’.&lt;/p&gt;
&lt;p&gt;Civilisation has advanced, speech has grown elegant, methods gentler, yet the logic remains unchanged. One might even say that, because we have learnt more refined expressions, many cruelties have become harder to identify and harder still to condemn.&lt;/p&gt;
&lt;p&gt;Thus the root of ‘false love’ lies less in the vice of a few rotten individuals than in one of the commonest and most worldly features of human nature: I may love you, provided I continue to believe that loving you will not cost me too much.&lt;/p&gt;
&lt;p&gt;This is conditional benevolence. It is not monstrous in the melodramatic sense, yet it is sufficient to destroy many people’s most basic faith in life. For it arrives clothed as love, yet at the critical moment reveals something more injurious than hatred: precise calculation of value.&lt;/p&gt;
&lt;h2 id=&quot;ix-true-love-takes-reality-into-account-without-revoking-recognition&quot;&gt;IX. True love takes reality into account without revoking recognition&lt;/h2&gt;
&lt;p&gt;One clarification is necessary: to criticise instrumentalised love is not to advocate some completely unreal romantic fantasy detached from the world.&lt;/p&gt;
&lt;p&gt;True love obviously cannot deny material pressures, income disparity, relocation, professional prospects, emotional labour, or family responsibility. A truly mature relationship is certainly not the childish declaration that ‘love alone is enough’. That belongs to cheap fiction, not to life.&lt;/p&gt;
&lt;p&gt;The key question is rather this: once reality enters the relationship, what role does it play?&lt;/p&gt;
&lt;p&gt;In false love, reality becomes the reason for withdrawing recognition. You are not sufficiently excellent, and therefore I no longer regard you as someone worth bearing with or bearing for.&lt;br&gt;In true love, reality becomes a shared problem to be worked upon. The world has become harder, and therefore we must redivide labour, remould ourselves, and determine anew how not to relinquish each other.&lt;/p&gt;
&lt;p&gt;The difference lies less in whether one is rational than in what that rationality serves. In the former case, reason serves self-preservation and valuation-maximisation. In the latter, reason is placed in the service of sustaining the common life. The former asks: ‘Is this relationship still worth it for me?’ The latter asks: ‘How may we continue, within reality, without abandoning one another?’&lt;/p&gt;
&lt;p&gt;This is the fundamental distinction revealed in the opening contrast:&lt;/p&gt;
&lt;p&gt;True love is, after seeing a greater world, still including the other person within the structure of one’s future;&lt;br&gt;False love is, after seeing a greater world, immediately updating the valuation of the other by a new standard, and then calling elimination ‘destiny’.&lt;/p&gt;
&lt;p&gt;In other words, true love is fidelity. It refuses to revoke personhood on account of difference, and it refuses to treat growth as a licence to discard one’s companion.&lt;/p&gt;
&lt;h2 id=&quot;x-conclusion-what-people-most-often-betray-is-not-another-person-but-their-own-former-claim-that-you-are-irreplaceable&quot;&gt;X. Conclusion: what people most often betray is not another person, but their own former claim that ‘you are irreplaceable’&lt;/h2&gt;
&lt;p&gt;Among the cheapest sentences in human life is: ‘I shall always stay by your side.’&lt;br&gt;Among the costliest acts is this: once one has acquired a better option, still refusing to treat the person beside one as a residue of an earlier, lower version of oneself that ought to be cast off.&lt;/p&gt;
&lt;p&gt;So what this essay really wishes to say is something more merciless than the observation that certain people are bad:&lt;/p&gt;
&lt;p&gt;Most people possess no capacity for love; what they possess is the capacity, when conditions are favourable, to display a fervour resembling love.&lt;br&gt;Most people have not yet found sufficient reason to despise the person beside them, and mistake that for respect.&lt;br&gt;What passes for faithfulness is, in most cases, the absence of a higher-grade substitute.&lt;br&gt;What passes for honesty is the absence of pressure: reality has not yet forced them to display their calculations.&lt;/p&gt;
&lt;p&gt;And what is called ‘refined egoism’ is almost the default personality template of modern society. It is not the vice of a few ambitious or clever people. At school people learn to compare; at work they learn to weigh; in social life they learn to display; in intimate relationships they learn to appraise. In the end, they no longer even feel that they are using others; they sincerely believe: I am merely lucid, I am merely growing, I am merely learning at last to love myself.&lt;/p&gt;
&lt;p&gt;Unfortunately, the terminus of much so-called ‘self-love’ is simply the loss of the ability to love anyone else.&lt;/p&gt;
&lt;p&gt;Thus the coldest scene in the world repeats itself again and again:&lt;br&gt;One person once held another’s hand and said, let us go to the future together;&lt;br&gt;Later he sees brighter lights, higher floors, a wider world, and lets go of that hand, saying lightly: you can no longer keep up.&lt;br&gt;Then he turns away and explains his betrayal as lucidity, his heartlessness as maturity, and his instinct for profit and self-preservation as the selection of fate.&lt;/p&gt;
&lt;p&gt;This is the final form of false love:&lt;br&gt;a tenderness that always had a price;&lt;br&gt;a sincerity that never accepted loss;&lt;br&gt;a longing for a better world, indulged alone or with those deemed worthy enough to accompany it.&lt;/p&gt;
&lt;p&gt;And true love is rare, though not because people cannot utter beautiful words.&lt;br&gt;In an age filled with valuation, competition, and universal upward striving,&lt;br&gt;to remain willing, after seeing the world clearly,&lt;br&gt;not first to take the world’s measuring rod and cut apart the person beside one –&lt;br&gt;that, in itself, has already become extraordinarily difficult.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>I Do Not Love Life; I Am Merely, For the Moment, Unable to Win</title>
    <link href="https://www.ailven.cc/posts/moment-260321.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260321.html</id>
    <published>2026-03-21T00:00:00Z</published>
    <updated>2026-03-21T00:00:00Z</updated>
    <summary>A Cold Self-Statement on Failure, Self-Defence, Value Camouflage, and Personality Split.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 21 March 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;A Cold Self-Statement on Failure, Self-Defence, Value Camouflage, and Personality Split.&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;I am increasingly aware that what truly requires judgement is how I deploy such language to preserve a self-image that still refuses to collapse. My work, my income, my emotional history, and even the notion I profess – that ‘life is greater than utility’ – are secondary to that.&lt;/p&gt;
&lt;h2 id=&quot;i-the-suspicion-of-life-first&quot;&gt;I. The Suspicion of ‘Life First’&lt;/h2&gt;
&lt;p&gt;I once found it plausible to believe that I belonged to a certain type of person: one not excessively attached to material gain, one who keeps a distance from utilitarian competition, one who regards leisure, reading, walking, music, urban wandering, purposeless observation and sensation as the true texture of life. This view is hardly rare; indeed, it carries a faint air of spiritual superiority. It allows one to believe that one has not been wholly trained by the world into a machine that merely calculates returns, compares gains and losses, and chases status and wealth. It permits one to think that even if one has not ‘won’ in the worldly sense, it does not follow that one has ‘lost’, because one has retained a higher faculty of judgement: the ability to recognise what truly matters.&lt;/p&gt;
&lt;p&gt;The problem lies precisely here. Once a person begins to emphasise ‘what truly matters’, it often signals that he is already experiencing loss in those worldly domains that are visible, comparable, and quantifiable. The louder the proclamation of value, the more likely it arises from a precarious position in reality, and the less likely from deeper insight. The more a person insists that he does not care about something, the more one must suspect that it is exactly what he cannot calmly admit he cares about deeply.&lt;/p&gt;
&lt;p&gt;Thus, looking back at myself today, I can no longer easily regard ‘life first’ as a naturally noble conviction. I must place it on the operating table and dissect it: is it truly my value, or merely an explanatory shell I have constructed after prolonged failure in order to prevent total collapse of the self?&lt;/p&gt;
&lt;p&gt;I must first admit something I was previously unwilling to acknowledge: I do care about worldly competition. I envy salary increases, promotions, platform transitions; I envy those who have positioned themselves correctly within capital markets, industrial cycles, and upward career trajectories. I am provoked by the progress of my peers, stung by news of others’ wealth, and beneath an apparently calm surface, I carry a persistent and concrete sense of humiliation at my own stagnation. I remain within the market order and accept most of its judgements. My pain itself proves that I have never truly exited this evaluative system. If income, platform, prospects, and social position were genuinely irrelevant to me, I would not repeatedly compare inwardly; I would not experience that instinctive imbalance when hearing of others’ salary rises; I would not construct such cruel and humiliating comparisons between my own income and others’ tax statements.&lt;/p&gt;
&lt;p&gt;In other words, I am merely someone who lacks the capacity to calmly say, ‘I also want to win.’ I seem more willing to interpret myself as someone who values life and dismisses utility, because that at least preserves a degree of personal dignity. To admit that I desire worldly success is to admit that I am not fundamentally different from those I once subtly disdained or tried to distance myself from. I compare, I calculate, I wish to validate myself through income, position, and external recognition. It is only because I have not achieved sufficiently good results in this competition that I require a discourse of greater ‘spiritual elevation’ to offset the embarrassment of this outcome.&lt;/p&gt;
&lt;p&gt;This is precisely what renders ‘life first’ suspect. It is unlikely to be a primary belief; it reads as a secondary rhetoric, an interpretation imposed after unfavourable results rather than an original ordering of values. Its function is damage control, not judgement. It allows me to rewrite ‘I failed to obtain more’ as ‘those things were not that important anyway’; to rewrite ‘I lag behind in competition’ as ‘at least I still possess life’; to rewrite ‘the world has judged me insufficient’ as ‘I have not been alienated by the world’. These reinterpretations are not transparent. They frequently operate under the guise of philosophy while performing the task of psychological defence.&lt;/p&gt;
&lt;h2 id=&quot;ii-the-wound-of-being-judged-unworthy&quot;&gt;II. The Wound of Being Judged Unworthy&lt;/h2&gt;
&lt;p&gt;I must also admit that what my former partner said at the moment of separation has affected me far more deeply than I would like to acknowledge. It was a judgement laden with hierarchy: I am inferior, my horizon is limited, I cannot handle her, I am unworthy of her. The most devastating aspect was that she issued a valuation of my very existence in a language of social status, hierarchy, and capital. It was a fine, sharp blade inserted into the most vulnerable region of male self-esteem: capability, status, value, and worthiness. It forced me to confront an intolerable possibility: that intimacy is at times merely another concealed yet equally brutal mechanism of value selection.&lt;/p&gt;
&lt;p&gt;After that, I could no longer regard income, platform, and prospects as purely professional matters. They became bound to a more humiliating question: am I worthy? Do I deserve to be chosen? Do I have the right to possess better relationships, better life conditions, greater space? My concern for salary, promotion, and success became a concern for the recognisability of my own value. Thus, the more I emphasise ‘life above utility’, the more likely it reflects the depth of the wound inflicted by utilitarian logic, and the less likely any transcendence of it. I am resisting that logic from within the wound.&lt;/p&gt;
&lt;h2 id=&quot;iii-reactive-resistance-not-transcendence&quot;&gt;III. Reactive Resistance, Not Transcendence&lt;/h2&gt;
&lt;p&gt;These are fundamentally different states. The former is transcendence; the latter is reactive resistance. Transcendence means one can perceive the power of worldly evaluation without being fundamentally governed by it. Reaction means one is still governed by it, merely adopting an oppositional posture. What appears as intense anti-utilitarian expression is often the reactive stance of the defeated. It establishes no new stable order; it merely harbours resentment against the old one, taking the old order as its reference point and rewriting the object of pursuit into an object of critique.&lt;/p&gt;
&lt;p&gt;If I am honest, I must admit that I have likely remained in this reactive state for a long time. I have not reordered life and success. What I have done is hold two conflicting evaluative systems simultaneously – one condemning me for insufficient success, the other consoling me that success is unnecessary. The former derives from the training of reality, the filtering of educational systems, the pricing of industry competition, and the final verdict delivered by that woman; the latter derives from my aspiration that ‘a human should be human’, from my instinctive resistance to alienation, instrumentalisation, and capital logic, and from my genuine experiences of leisure, wandering, and purposelessness during earlier periods of life. The problem is that these two systems remain unintegrated within me; they pull against each other, cancelling each other out, leaving me unable either to fully commit to competition or to live with stability.&lt;/p&gt;
&lt;p&gt;Thus emerges an absurd yet accurate scene: I genuinely long for walking, reading, exhibitions, music, bus rides, photography, and flowers – forms of time that are non-utilitarian. Yet the moment I engage in them, another internal judge immediately pronounces: today you have not improved, produced, accumulated, or advanced any clear objective; you are wasting time, you are decaying. On the surface, this appears as value conflict; in essence, it is a conflict of sovereignty. I know what kind of life I desire; I have not granted that life genuine legitimacy. I praise it verbally, yet still allow performance logic to monopolise final judgement. My leisure is merely tolerated as a subordinate process for restoring work capacity, never recognised as life itself. Once detached from the premise of ‘serving better work performance’, it is immediately judged suspicious, inefficient, even guilty.&lt;/p&gt;
&lt;p&gt;This indicates that my so-called ‘life first’ has not become my core value. It is merely a defensive concept mobilised at certain moments to counter external pressure. It resembles a wartime slogan rather than a constitution. It appears when I am provoked, compared, or humiliated, providing psychological cover; yet in actual allocation of time and daily decision-making, it quickly recedes, returning legislative authority to efficiency, competition, measurable returns, and future appreciation. In other words, I merely borrow the name of ‘life’ to temporarily resist competitive order; I am not centred on it.&lt;/p&gt;
&lt;h2 id=&quot;iv-the-hybrid-truth-of-non-utilitarian-life&quot;&gt;IV. The Hybrid Truth of Non-Utilitarian Life&lt;/h2&gt;
&lt;p&gt;Yet to conclude here that ‘life first is merely a defence mechanism’ would still be crude. Human reality is rarely a binary between entirely sincere value and entirely false rationalisation. More often it is hybrid: a concept may possess genuine experiential grounding while also serving defensive functions; it may express inner need while masking incapacity; it may be both value judgement and self-protection. I suspect my ‘life first’ belongs to precisely such a hybrid state.&lt;/p&gt;
&lt;p&gt;It is not wholly false, because I am indeed not someone capable of endlessly tolerating a fully instrumentalised existence. My need for leisure, aesthetics, reading, and wandering is not fabricated. I have genuinely experienced moments of low-purpose fulfilment – moments unrelated to capital accumulation, performance metrics, or status elevation, yet not empty for that reason. Sitting on a bus across the city, accomplishing nothing worth recording on a résumé, yet feeling, through light, wind, streets, trees, buildings, and faces, that one is not entirely dead – this experience is real. It was not taught to me; I lived it. It proves that I did not invent ‘life’ merely as a product of failure.&lt;/p&gt;
&lt;p&gt;But this does not absolve me. Authentic experience does not automatically equal authentic prioritisation. One may genuinely love a certain quality of life, yet elevate it excessively after failure to compensate for what one could not obtain. The real question is this: when life and success are in tension, which am I prepared to place higher? And will this ordering remain stable if my material circumstances improve? If I speak highly of life today due to stagnant income, yet tomorrow, upon obtaining better opportunities, higher pay, and clearer advancement, willingly demote life, then my present philosophical expressions are merely rhetorical products of deprivation. Only if, after improvement of resources, I still preserve inviolable boundaries for leisure, non-utilitarian activity, aesthetics, and inner space, can ‘life first’ possess genuine stability.&lt;/p&gt;
&lt;p&gt;This constitutes a brutally cold test: do I oppose the logic that reduces humans to instruments of capital itself, or merely oppose my low position within it? If the former, then even if I ascend in income and status, I will remain vigilant against its encroachment. If the latter, then my current critique is merely resentment. People rarely admit resentment, for it lacks dignity; it is harder to acknowledge than ambition. Yet I must admit that my thought likely contains such elements. I may simply resent the world of winners from the standpoint of the defeated, rather than criticising competition from a standpoint of truth.&lt;/p&gt;
&lt;p&gt;This does not render the world of winners correct. The evaluative system composed of capital, platforms, salaries, and status is inherently crude and reductive, excluding complex human experience. But the crudeness of a system does not grant automatic insight to those excluded by it. Many who fail have merely been rejected by the system, not seen through it. Rejection alone does not produce insight. If I cannot distinguish between ‘my critique of this world’ and ‘the resentment triggered by its rejection of me’, then all my philosophical posture risks being merely refined camouflage for emotion.&lt;/p&gt;
&lt;h2 id=&quot;v-the-collapse-of-a-flattering-self-image&quot;&gt;V. The Collapse of a Flattering Self-Image&lt;/h2&gt;
&lt;p&gt;Thus the problem converges upon a more embarrassing yet clearer point: what kind of person am I? I once preferred to imagine myself as someone insufficiently successful yet retaining a degree of spiritual independence; someone who had not won much yet had not surrendered to utilitarian logic; someone still capable of responding to literature, philosophy, leisure, and purposeless life. This self-image is not entirely fictitious, but it is evidently selectively edited. It preserves what I am willing to claim, while suppressing what I am not. It highlights my ‘human aspect’ while evading my ‘competitive aspect’; it praises my sensitivity and reflection while concealing vanity, comparison, jealousy, humiliation, and anxiety about worthiness. Thus, while I appear to seek self-knowledge, I have in fact remained within a refined form of self-idealisation.&lt;/p&gt;
&lt;h2 id=&quot;vi-two-impulses-without-integration&quot;&gt;VI. Two Impulses Without Integration&lt;/h2&gt;
&lt;p&gt;True self-knowledge requires admitting that I am not unified. I contain at least two equally real impulses. First, I genuinely desire a life not fully instrumentalised – one in which I can read, walk, think, perceive beauty, wander, and maintain non-utilitarian relations with the world. Second, I equally desire a better position in career and worldly competition – higher income, validated ability, relief from lagging behind, and freedom from being judged unworthy in intimate relations. Both are real. The issue is that I have long attempted to let the former cancel the latter, as though firm belief in the value of life could eliminate my desire for victory, dignity, and social valuation.&lt;/p&gt;
&lt;p&gt;This approach is doomed to fail. What is suppressed does not disappear; it resurfaces elsewhere. Thus, whenever I try to live, another self accuses me; whenever I compete, this self condemns me as vulgar, utilitarian, alienated. I neither allow myself to pursue worldly success directly, nor permit myself to peacefully enjoy life beyond it. I become a courtroom of internal judgement rather than an acting subject. My contradiction persists because I avoid a simple and undignified fact: I want both, yet have neither paid sufficient cost for either, nor established an order capable of integrating them.&lt;/p&gt;
&lt;p&gt;At this point, the question of whether I am ‘naïve’ acquires new meaning. There is nothing naïve in aspiring to life. True naivety lies in believing that articulating a higher value removes the pain of lower status; that reading literature and philosophy elevates one above those concerned with salary and promotion; that imagining oneself ‘not vulgar’ replaces honest examination of one’s desires. A deeper naivety lies in mistaking reluctance to admit ambition for spiritual purity, translating inability to win into voluntary withdrawal, and confusing uncompleted self-integration with achieved maturity.&lt;/p&gt;
&lt;h2 id=&quot;vii-time-allocation-as-the-real-order-of-values&quot;&gt;VII. Time Allocation as the Real Order of Values&lt;/h2&gt;
&lt;p&gt;If I am to be responsible to myself, I must cease using overly elegant slogans such as ‘life is more important than utility’. This statement may not be false, but it is too smooth – so smooth that it conceals real conflict. It makes one appear noble but guides no concrete decision. The real issue is never abstract importance. It is this: to what extent am I willing to accept higher intensity, reduced leisure, and greater risk in exchange for income and security; whether I can refrain from judging time that produces no output; whether I can bear relative lag when others advance while I preserve life boundaries. These are questions of ordering and cost, not slogans.&lt;/p&gt;
&lt;p&gt;And ordering can only be demonstrated through allocation of time. What matters is how one partitions one’s finite life. Claims about value are meaningless. My time is allocated to anxiety, comparison, skill pursuit, and professional unease, and also to fragmented reading, occasional walking, and intermittent aesthetic experience – but the latter has not formed sovereignty; it remains residual. This indicates that I am someone exhausted within competitive order who retains a residual longing for life. This judgement lacks nobility, but is closer to truth than self-praise.&lt;/p&gt;
&lt;p&gt;Thus, a more accurate statement may be: I require both a dignified position within reality and a space of life not entirely consumed by it. I do wish to win, but cannot accept becoming entirely mechanised in doing so; I cannot truly be content with poverty, yet know that winning alone does not constitute life. This formulation lacks elegance and manifesto-like quality, but approaches fact. It acknowledges me as shaped by reality while resisting alienation, rather than elevating me into a transcendent subject. I am neither noble nor pure; I have not seen through the world – I am merely wounded within it, yet unwilling to fully submit.&lt;/p&gt;
&lt;p&gt;This may be the first truth I must accept.&lt;/p&gt;
&lt;h2 id=&quot;viii-a-less-elegant-but-more-honest-conclusion&quot;&gt;VIII. A Less Elegant but More Honest Conclusion&lt;/h2&gt;
&lt;p&gt;The second is harsher: until I admit my genuine desire for worldly success, I will never be able to live calmly. Every attempt at living will be seen by the suppressed competitor as betrayal; every attempt at competition will be judged by the idealised ‘life-self’ as degradation. Only when I admit that I also want to win, fear poverty, and care about status and recognition, can life cease to be a defensive slogan and become life itself. Otherwise, my love of life is always entangled with escape, and my pursuit of success entangled with shame. I remain suspended – failing, theorising, envying, denigrating, desiring, and disguising simultaneously.&lt;/p&gt;
&lt;p&gt;This has been my most authentic mental state in recent years: not balance, but repeated tearing. The cause is my refusal to be sufficiently ruthless towards myself. I wish to preserve dignity – to render desire refined, failure justified, stagnation meaningful. Yet true self-critique demands abandoning such rhetorical dignity. I must admit that at times my values are ornaments after failure; my praise of life a compensatory rhetoric; my discussion of literature and philosophy a temporary concealment of impotence, envy, and defeat.&lt;/p&gt;
&lt;p&gt;Admitting all this does not ease matters; it removes cheap consolation. Once I can no longer say simply ‘I value life more’, I must confront reality: dissatisfaction with my position, unwillingness to stagnate, pain before others’ progress, and the lingering shadow of ‘you are unworthy’. Simultaneously, I require the useless: reading, wandering, scenery, music, exhibitions, photography, streets, dusk, trees, air – time that cannot be converted into performance, yet proves I remain human. Both must coexist. Only then does the problem return from pseudo-philosophy to life.&lt;/p&gt;
&lt;p&gt;And life does not reward elegant conclusions. It demands boundaries, costs, and tolerance of impurity. I may not arrive at any grand statement. I may no longer say ‘life above all’ or ‘success is trivial’. I can only state, more restrained and less dignified: I seek a position where I am not humiliated or displaced; within it, I retain time and experience serving no objective; I no longer disguise ambition as vulgarity, nor life as nobility. This is no philosophical victory. It is a reduction of self-deception.&lt;/p&gt;
&lt;p&gt;If a provisional conclusion must be written, it is this: What I loved was the dignity the slogan ‘life first’ once granted me, not the slogan itself. Nor do I merely envy others’ income and position; I envy the state of no longer needing to justify one’s value. The former elevates me above reality; the latter places me beneath it. Thus I remain suspended between them, unable either to admit worldly desire or to properly enjoy non-worldly time.&lt;/p&gt;
&lt;p&gt;What must be reconstructed is not a grand declaration, but a more honest definition of self: I still care about success and failure while refusing to let them monopolise meaning. I am a critic wounded within capital logic, not one standing outside it. I am not naturally mature; I can simply no longer conceal mixture, resentment, ambition, vulnerability, and shame beneath beautiful language.&lt;/p&gt;
&lt;p&gt;Having said all this, I am at least closer to truth – even if it lacks grace. And that is preferable to mistaking defence for belief, impotence for transcendence, and fragmentation for depth.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Rethinking My First Love</title>
    <link href="https://www.ailven.cc/posts/moment-260316.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260316.html</id>
    <published>2026-03-16T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A reflective essay on first love, mutual harm, narrative authority, and the collapse of a shared future under diverging moral and existential doctrines.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;From the sudden stab of a sharp needle to the heart: a first love, a relationship breakdown, and why I have yet to reclaim my right to the narrative&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 16 March 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;At a particular moment in Beijing I caught sight of a girl who bore an uncanny resemblance to her, accompanied by a boy I did not recognise. It may well have been nobody at all; it might simply have been a similarity of outline, of gait, of atmosphere. Yet my body did not wait for facts to be verified. In that instant it felt as though a razor-sharp needle had suddenly been plunged deep into my heart. What followed was an almost physical agony: a tightening across the chest, the centre of gravity shifting, the world seeming suddenly to reel. Everything I had believed time had buried came flooding back within a single second.&lt;/p&gt;
&lt;p&gt;That scene itself is not complicated, yet it is sufficient to expose a truth. I have not truly completed my reckoning with that relationship. I may have resumed my life, work and daily routines in practical terms, but on a symbolic level – concerning my sense of self, my conception of love, commitment, value and fate – that relationship has never really found its place. It has embedded itself in a more hidden yet sturdier way.&lt;/p&gt;
&lt;p&gt;This essay is therefore not intended to retell a sentimental story, nor to rehearse any facile emotional catharsis. My task is colder and harder: why did a relationship I once believed would not end collapse so completely? What types of responsibility did I and she respectively bear in that collapse? Why, three years on, can a fleeting resemblance still fell me? Why did I slide from believing ‘where there is a will there is a way’ towards a conviction that everything is arranged in some inscrutable destiny? And, perhaps most importantly, why do I continue to retain, on some level, the right to define that relationship with her rather than reclaiming it for myself?&lt;/p&gt;
&lt;h2 id=&quot;i-the-fracture-of-an-entire-life-structure&quot;&gt;I: The fracture of an entire life structure&lt;/h2&gt;
&lt;p&gt;Losing a person alone need not inflict pain of such duration. What makes relationships particularly hard to process is when that person has become woven into the entirety of one’s vision of the future. She was more than a lover: she formed part of my overall projection of where and how I would live, with whom I would spend my later years, and how to understand intimacy itself. At that time I conceived of love as a structure pointing directly to building a common life: mutual support, helping each other, planning together, and through practical constraints gradually creating a life that belonged to ‘us’.&lt;/p&gt;
&lt;p&gt;Precisely because of this, the most catastrophic element was the unilateral demolition of our shared future. In my records I wrote: &lt;strong&gt;‘At a stage in which we both faced infinite choices, I knew that if two people were to stay together long-term, they had to have a future negotiated together.’&lt;/strong&gt; That sentence represented my underlying institutional understanding of intimacy: love was an enterprise in common, not a cluster of feelings. In retrospect, these words constitute my institutional presupposition about relationships rather than an expression of emotional ethics: love is a community project, not a readily revocable affection.&lt;/p&gt;
&lt;p&gt;She later revealed a very different doctrine: personal development trumps all; commitments are provisional; the future can be discussed but should not substantially restrict an individual’s path; when the cost of communication becomes too high, withdrawal becomes legitimate or even necessary. This doctrine is not necessarily wrong, but it is incompatible with mine. The problem is that I only realised she had switched doctrine when our relationship had already been deeply communalised.&lt;/p&gt;
&lt;p&gt;This also explains a key phenomenon: why did I later feel she had become a completely different person? The truth is that the foundation on which she understood the relationship had changed; she herself may not have. Many of our differences and conflicts already existed, but within a communal framework they would have been taken as frictions to work through together. Once the structure shifted to one prioritising individual gain, those differences were reinterpreted as evidence of incompatibility. Our divergence did not suddenly widen; the evaluative regime had shifted.&lt;/p&gt;
&lt;p&gt;In this sense, our relationship was hollowed out over time by a deeper institutional mismatch. No single argument or harsh word did that work. By the time the final rupture came, what appeared to be a sudden break was in fact the expression of a long-running structural split.&lt;/p&gt;
&lt;h2 id=&quot;ii-my-responsibility-lay-in-turning-fear-of-loss-into-controlling-harm&quot;&gt;II: My responsibility lay in turning fear of loss into controlling harm&lt;/h2&gt;
&lt;p&gt;It would be too convenient – and shallow – to conclude that our breakup was wholly due to her altering her priorities and renouncing our shared commitments. In truth, the acceleration into toxicity was fuelled by my own behaviours when confronted with distance, instability and uncertainty, as much as by her shifting stance.&lt;/p&gt;
&lt;p&gt;I have since acknowledged, and need to name plainly, that during that period I often spoke to her in ways that wounded, belittled, blamed and insulted, even employing techniques akin to emotional manipulation. This deserves no beautification. In a state of existential threat to my attachment, I was unable to stabilise my pain as expressions of feeling and need; I transmuted it into pressure, accusation, relentless explanation, and moral debt. Though my stated aim was to repair the relationship, my methods bore the structure of coercion – pressing her to resume the trajectory I understood as ‘right’.&lt;/p&gt;
&lt;p&gt;The boundary between legitimate pain and manipulative harm lies in the structure of the interaction, not the intensity of emotion. To communicate distress – however fiercely – is one thing; to transform that distress into a mechanism for obliging the other to change their behaviour is quite another. When I slid into telling her she ‘didn’t care’, she ‘only used people’, she ‘had no heart’, when I invoked the things I had done for her as a kind of unspoken debt, I was employing shame and guilt as levers, not simply expressing vulnerability. My complaints thus morphed from ‘I feel hurt’ to ‘you are a problem’.&lt;/p&gt;
&lt;p&gt;This manifested in two principal ways. First, I could not tolerate the fact that her decisions were no longer aligned to ‘us’. Consequently, I kept trying to drag her back through analysis, explanation, letters and appeals, hoping to re-enlist her in a communal enterprise. In my own mind I was working to mend something; but to someone who had ceased to participate in the same repair mechanism, these high-frequency interventions were themselves a burden. Second, I transformed my practical support during her study and application processes into an implicit moral debt: because I helped you, you should not leave. Even if I rarely said it outright, the structure implied a bond of obligation that was not freely chosen.&lt;/p&gt;
&lt;p&gt;Thus, my principal responsibility was something more specific than a vague immaturity: under a collapsed attachment system, I transposed fear into high-pressure, invasive patterns of interaction. I wanted to protect a relationship, yet I resorted to tools that would accelerate its destruction.&lt;/p&gt;
&lt;h2 id=&quot;iii-her-responsibility-lay-in-choosing-humiliation-and-de-personification-as-the-mode-of-departure&quot;&gt;III: Her responsibility lay in choosing humiliation and de-personification as the mode of departure&lt;/h2&gt;
&lt;p&gt;Leaving someone is not in itself morally illicit; anyone has the right to decide to cease a relationship. What creates ethical responsibility is the manner in which one leaves.&lt;/p&gt;
&lt;p&gt;What cut deepest was the language and posture she employed in withdrawing. Rather than saying ‘I don’t wish to continue’, ‘we are incompatible’ or ‘I cannot sustain this’, she progressively rewrote me in ways that vacated me of personhood. My efforts were cast as worthless self-dramatics; my sincerity was compressed into petty ‘letters’ that bored and repulsed her; my existence became suffocating to her; my emotions were denied any legitimacy; everything I could offer became highly substitutable; our entire history together was recoded as a banal, universal experience any two people could have had. What might have been a process of saying no became a process of annihilating my standing as someone worth speaking to at all.&lt;/p&gt;
&lt;p&gt;Such de-personification is cruel, and it serves clear functions. To exit from a deep investment without internal self-conflict, it helps if the other can be reframed as someone one has good reason to abandon. If one simply acknowledges, ‘I am changing my own doctrine’, one must shoulder the responsibility of having altered a commitment; it is easier to say, ‘He was never worth it’. By magnifying my faults, cancelling our past significance and positioning me as the source of her pain, she converted a choice to leave into an act of self-liberation. The more degrading the portrayal, the less room for remorse or doubt.&lt;/p&gt;
&lt;p&gt;The better question is how to name the combination of self-preservation, anger, vengeance, rationalisation and status inversion that made humiliation her chosen method, rather than whether she was ‘bad’. Yes, she had every right to leave. But the way she left – through denying my right to be seen and rewriting me into nothingness – carries its own weight of harm that cannot be explained away as mere self-defence.&lt;/p&gt;
&lt;h2 id=&quot;iv-the-deepest-wound-came-not-from-losing-her-but-from-losing-the-legitimacy-of-our-past&quot;&gt;IV: The deepest wound came not from losing her, but from losing the legitimacy of our past&lt;/h2&gt;
&lt;p&gt;Many break-ups hurt, but they are not always annihilating. If both sides can still recognise the period they shared as something genuine, unique and cherished, the pain remains bound to the future: there is no ‘us’ henceforth. In my case, the worst damage lay in the revision of the past. She withdrew from a future together, and then from the very reality of what had been.&lt;/p&gt;
&lt;p&gt;I know she had, at one point, truly loved me. This is not some myth I later invented. She told me she felt she must have done something remarkable in a previous life, that she had exhausted her fortune to meet me. We walked the streets of Qingdao like an old couple, our life skipping over youthful infatuation straight into a retired kind of peace. She spoke of driving across China when we were old, not needing wealth or splendour, only each other. Those were not fantasies; they really happened. And yet later they were recoded. She said our being together was never love, only responsibility; she recast everything I did as mere self-gratification; she reduced our unique relationship to an episode any random pair of people could duplicate. Thus I lost more than a partner: I lost a witness to that time, someone who could corroborate that it had meant anything.&lt;/p&gt;
&lt;p&gt;This creates a particular structure of trauma: the future is gone and the past is delegitimised. One is left hanging. The moments I had believed were the most beautiful and real now seem to be contested. If she no longer acknowledges them, do they still count as real? Hence my philosophical interrogations: what does it mean to be seen in love? Is love a recognition of a person’s existence, or a valuation of their usefulness, potential, social position? If the latter, it follows that someone can enter my life intensely and yet later revoke my right even to remember it as special. If love can retract recognition in such a manner, what is its essence?&lt;/p&gt;
&lt;p&gt;These questions may sound abstract, but they cut to the bone of modern intimacy. Many relationships are tacitly built not on unconditional affirmation of the other’s existence, but on a silent mechanism of valuation. So long as life is smooth and the relationship yields pleasure, stability and a shared prospect, this mechanism is disguised by romantic language. When conditions change, it shows itself. The abrupt cancellation of a person, to which I was subjected, forces me to confront this unspoken economy.&lt;/p&gt;
&lt;h2 id=&quot;v-my-drift-towards-fatalism-was-an-anaesthetic&quot;&gt;V: My drift towards fatalism was an anaesthetic&lt;/h2&gt;
&lt;p&gt;After this breakdown, I found myself moving from ‘where there’s a will there’s a way’ to a sense that everything happens as if arranged by fate. At first I believed this was a kind of insight, but now I see it as primarily a way to staunch psychological bleeding.&lt;/p&gt;
&lt;p&gt;The reason is simple. To insist on human agency would mean facing relentless counterfactuals: if I had not demanded we live in the same city, if I had not expressed my insecurity so violently, if I had not weaponised affection into moral debt, would we have survived? If she had not altered her commitments, or had not rewritten our relationship after receiving an offer from Zhejiang University, might we have actually had the future we imagined? These questions are infinite. To quiet them, the mind naturally slides into fatalism: everything happened as it must; I was carried by forces beyond my control.&lt;/p&gt;
&lt;p&gt;Fate thus becomes a temporary defence, translating personal failure into structural inevitability. Yet it comes at a price: it disarms agency. If I say it was all preordained, then the lessons of personal responsibility dissolve. This may be soothing, but it is not explanatory. A more accurate understanding is that there were significant structural constraints – distance, study destinations, diverging life paths, differences in personality, an imbalanced attachment pattern – and there were also clear human decisions. Recognising structural limits does not absolve us of the ways we worsened matters. Fatalism may dull the pain, but it cannot offer a truthful account.&lt;/p&gt;
&lt;h2 id=&quot;vi-what-keeps-piercing-me-today-is-not-merely-her-but-the-version-of-myself-she-once-affirmed&quot;&gt;VI: What keeps piercing me today is not merely her, but the version of myself she once affirmed&lt;/h2&gt;
&lt;p&gt;So why, years later, does a glimpse of someone who looks like her still cause such devastation? My conclusion is that it is less about her as an individual and more about the self that was recognised in her eyes. First loves contribute to one’s self-model, and that, more than their priority, is what makes them powerful. To be thus loved, needed, affirmed confers a basic sense: I can be seen, needed, cherished, and included in another’s future. This amounts to an ontological validation. And when the same person who conferred that validation later withdraws it, and in doing so declares me to be of no worth, what is wounded is not attachment alone. It is a deep belief about myself.&lt;/p&gt;
&lt;p&gt;This partly explains why I went on to develop theoretical frameworks on conditional recognition versus seeing someone’s existence for its own sake, on how modern society has colonised love with a market logic, on status-sensitive performativity. On the surface I was exploring social structures; at a deeper level, I was trying to answer a personal question: if someone I believed to be deeply authentic could retract recognition so severely, what then is love? If relationships are contingent on performance, growth, position, future prospects and class anxieties, can a person ever be loved as such? These questions show the fusion of my personal trauma with social critique.&lt;/p&gt;
&lt;p&gt;Yet, even with these analyses, there are two ways to use such theory. One is to reclaim my narrative: to say that her withdrawal says more about her values than about my worth; that she was operating under a valuation framework; that her departure reflects her story rather than my essence. The other way is to elevate my wound into philosophy while leaving her verdict intact – to suggest that her leaving proves something inherently deficient about me. For a long time I leaned towards the latter: I could criticise societal structures, yet I still implicitly accepted that she had looked through me and found me wanting.&lt;/p&gt;
&lt;h2 id=&quot;vii-reclaiming-the-narrative-is-not-about-debasing-her-but-about-demoting-her-judgement-to-a-testimony&quot;&gt;VII: Reclaiming the narrative is not about debasing her, but about demoting her judgement to a testimony&lt;/h2&gt;
&lt;p&gt;For years I remained in a courtroom she built. She issued charges: immaturity, suffocation, uselessness, self-indulgence, substitutability, repulsiveness. I spent my inner life drafting defences: I am not those things, I was in pain, I gave so much, I had my reasons. As long as I remained absorbed in such rebuttals, I implicitly accepted her authority to judge. True sovereignty over my story requires me to leave that courtroom entirely.&lt;/p&gt;
&lt;p&gt;This does not mean declaring her ‘evil’ or ‘cold’ while I am ‘holy’. That would still be a reaction trapped within her framework. Instead, it involves a layered distinction. I can acknowledge that certain of her criticisms of my behaviour were accurate: my high pressure, controlling tendencies, insults and intrusions would suffocate anyone. I can acknowledge that she acted partly out of her own pain and self-protection. But I must also refuse to let these acknowledgements confer upon her the right to pronounce final judgement on my being. She may offer a testimony from her perspective, but she is not the final arbiter.&lt;/p&gt;
&lt;p&gt;This is difficult precisely because she was once my primary certifier. To detach her judgement from my worth feels like a betrayal of my own history. But closeness does not equate to truth; intimacy does not grant judicial authority. She saw many aspects of me, but she saw them through the lenses of her needs, fears, ambitions and defences. Hence her testimony is inevitably shaped. It has value, but it is not a verdict.&lt;/p&gt;
&lt;p&gt;True narrative reclamation thus means downgrading her pronouncement from a decree to evidence. It allows me to say: yes, she was hurt; yes, I caused pain; yes, she changed and withdrew; but no, she does not define my essential worth. She is part of the story, not its whole. The relationship is a chapter, not the totality. Initial loves do not return; the innocence of young adulthood does not return; the first experience of being so confirmed and then so cancelled will not be repeated. Those unique circumstances cannot be replaced. But their unrepeatability does not prevent me from learning more accurate, more sober ways of understanding people and relationships.&lt;/p&gt;
&lt;h2 id=&quot;viii-what-remains-is-not-simply-hurt-but-a-colder-recognition-of-the-age-and-of-myself&quot;&gt;VIII: What remains is not simply hurt, but a colder recognition of the age and of myself&lt;/h2&gt;
&lt;p&gt;To limit this story to personal misfortune would be to misread its significance. Yes, it is my experience; yes, it shattered me; but it also forced me to witness unsettling dimensions of contemporary life.&lt;/p&gt;
&lt;p&gt;I have come to see that modern culture has embedded instrumental reason deep within ostensibly romantic interactions. People do not consciously treat each other as tools; nevertheless, the maintenance of relationships is increasingly subject to calculations of trajectory, position, growth rate, psychological cost and class mobility. Schooling is like this, work is like this, and relationships are rarely exempt. A person may speak tenderly and poetically when all is well, yet when core interests, paths or status differentials are triggered, the underlying logic reveals itself: is the other being regarded as an irreplaceable person, or as a re-evaluated asset? The more sophisticated the language, the more harmful the outcome can be: cruelty can be delivered through ‘rational’ terms.&lt;/p&gt;
&lt;p&gt;Simultaneously, my own conduct was exposed. I had thought of myself as defending love, commitment and community, but under pressure I could quickly adopt the logic of dominance and leverage. I turned affection into a debt, pain into a weapon, communication into a tribunal. In other words, I employed my own forms of harm. She injured me with withdrawal and humiliation; I injured her with control and violation. These harms did not cancel each other out, but they interacted and magnified within our mismatched systems until the relationship disintegrated. By the end, one of us held the narrative high ground, and the other remained stuck.&lt;/p&gt;
&lt;p&gt;Thus I must acknowledge: many relationships collapse because two people enacting incompatible doctrines inflict injuries that feed back into a destructive cycle. Purity on one side and rottenness on the other is the rarer case. It is not always necessary to decide which party is wholly to blame. More often, both parties commit serious mistakes under different imperatives. The challenge is to recognise these mechanisms without sliding into self-exculpation or fatalism.&lt;/p&gt;
&lt;h2 id=&quot;ix-in-conclusion-what-this-story-means-to-me-now&quot;&gt;IX: In conclusion, what this story means to me now&lt;/h2&gt;
&lt;p&gt;If pressed to summarise this relationship as objectively as I can, I would not call it ‘true love betrayed’ nor reduce it to ‘two youngsters mutually torturing one another’. It was a relationship that once was genuine, deep and filled with communal imagination. Under the pressures of distance, diverging life paths, unequal attachments and reversed power dynamics, it evolved into a scene of mutual harm. Eventually, it ended in a high-impact act of de-personifying severance. It neither was a fabrication nor something a mere ‘we are not compatible’ can explain away. It matters to me because it compels me to confront questions I would rather avoid: is love built on unconditional recognition of a person’s existence? Does modern intimacy submit increasingly to instrumental reason? Who do I become when my attachment system is destabilised? And when my principal witness withdraws her confirmation of me, how do I redefine myself?&lt;/p&gt;
&lt;p&gt;That moment in Beijing – the stab like a needle – is now less of a mystery. It happened because somewhere within me she continued to hold the authority to define. Time had passed; that authority had not. As long as that authority is unrevoked, any trace that resonates with our story has the power to reopen old circuits.&lt;/p&gt;
&lt;p&gt;Thus, this essay does not seek to elicit sympathy or to judge. It sets out to reset permissions: I acknowledge the reality of what we had and the authenticity of her earlier love. I acknowledge the real pain and the need for medical intervention that followed. I accept that I behaved in ways that caused harm. I accept that she had reasons, that she too was wounded and that her departure reflected her values. But I no longer accept her as the judge of my essence. She is a witness, not the court; a participant, not the narrator. That chapter was formative, but it will not define the entire text of my life.&lt;/p&gt;
&lt;p&gt;It is unnecessary to declare ‘I have forgotten’, or ‘I will never trust again’. The truth is more nuanced: I was forced to relinquish certain naive beliefs, yet through that loss I have gained a more clear-eyed view. This view is not comfortable, but it returns the narrative to my own hands.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>LLM Inference on Edge Platforms</title>
    <link href="https://www.ailven.cc/posts/ai-infra-260314.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/ai-infra-260314.html</id>
    <published>2026-03-14T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A systems-level analysis of prefill, decode, KV-cache layout, prefix reuse, and the memory-bandwidth economics that govern practical LLM inference on edge hardware.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;Prefill, Decode, and the Economics of KV Cache&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 14 March 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h2 id=&quot;abstract&quot;&gt;Abstract&lt;/h2&gt;
&lt;p&gt;Large Language Model inference is often described in high-level product terms: prompt in, tokens out. That description is sufficient for a user experience discussion, but it is too coarse for systems engineering. Real inference stacks treat generation as a sequence of distinct computational regimes, each with a different bottleneck, resource profile, and optimisation strategy. The most useful decomposition is into &lt;strong&gt;embedding&lt;/strong&gt;, &lt;strong&gt;prefill&lt;/strong&gt;, and &lt;strong&gt;decode&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This article argues that the engineering significance of this decomposition becomes especially clear on edge and on-device platforms. In such environments, the decisive constraints are rarely abstract model quality alone; rather, they are the concrete economics of memory capacity, memory bandwidth, scheduling overhead, and cache persistence. The discussion therefore focuses on four tightly coupled issues: the asymmetry between prefill and decode, the role of the &lt;strong&gt;key-value (KV) cache&lt;/strong&gt; as persistent model state, the practical value and limits of &lt;strong&gt;prefix reuse&lt;/strong&gt;, and the architectural choices required to make these mechanisms viable on bandwidth-constrained hardware.&lt;/p&gt;
&lt;p&gt;The central thesis is that KV cache is not merely a convenience layer bolted onto autoregressive decoding. It is the state-management substrate that makes LLM serving possible at all. Yet the same mechanism that saves computation also shifts the dominant systems problem from matrix arithmetic to data movement. On edge hardware, where bandwidth and thermal headroom are scarce, this shift becomes the defining challenge of inference design.&lt;/p&gt;
&lt;h2 id=&quot;i-introduction-why-stage-decomposition-matters&quot;&gt;I. Introduction: Why Stage Decomposition Matters&lt;/h2&gt;
&lt;p&gt;The Transformer architecture does not explicitly divide inference into three named phases. From the perspective of pure model mathematics, a forward pass is simply a forward pass. Nevertheless, modern inference engines almost universally reason in terms of:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;tokenisation -&amp;gt; embedding -&amp;gt; prefill -&amp;gt; decode&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;This decomposition survives because it corresponds to real differences in workload shape.&lt;/p&gt;
&lt;p&gt;During &lt;strong&gt;embedding&lt;/strong&gt;, the system maps discrete token IDs into dense vectors. During &lt;strong&gt;prefill&lt;/strong&gt;, it processes the entire prompt in parallel and materialises the initial KV cache. During &lt;strong&gt;decode&lt;/strong&gt;, it generates one token at a time while repeatedly consulting previously stored state. The point is not terminological neatness; the point is that the hardware bottleneck moves from stage to stage. A design that is ideal for prompt ingestion may be poor for token-by-token generation, and a layout that maximises raw throughput in the data centre may still fail on a mobile or embedded device because of memory traffic.&lt;/p&gt;
&lt;p&gt;This is why stage decomposition has become a stable abstraction in production systems. It allows engineers to reason separately about:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;first-token latency versus sustained generation throughput&lt;/li&gt;
&lt;li&gt;arithmetic intensity versus memory intensity&lt;/li&gt;
&lt;li&gt;transient compute buffers versus persistent attention state&lt;/li&gt;
&lt;li&gt;request-local work versus shareable prefix state&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;On edge platforms these distinctions are not academic. They determine whether a model is merely runnable in a benchmark or genuinely deployable in a user-facing application.&lt;/p&gt;
&lt;h2 id=&quot;ii-a-systems-cost-model-for-llm-inference&quot;&gt;II. A Systems Cost Model for LLM Inference&lt;/h2&gt;
&lt;p&gt;To discuss inference professionally, it helps to separate three kinds of cost: &lt;strong&gt;weight access&lt;/strong&gt;, &lt;strong&gt;activation/attention compute&lt;/strong&gt;, and &lt;strong&gt;KV-cache traffic&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Suppose a model has:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;code&gt;L&lt;/code&gt; transformer layers&lt;/li&gt;
&lt;li&gt;hidden width &lt;code&gt;H&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;N_kv&lt;/code&gt; KV heads&lt;/li&gt;
&lt;li&gt;head dimension &lt;code&gt;D&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;prompt length &lt;code&gt;T&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;generated length &lt;code&gt;G&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;element size &lt;code&gt;b&lt;/code&gt; bytes&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The KV cache size for a sequence of length &lt;code&gt;T&lt;/code&gt; is roughly:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;KV bytes ~= 2 x L x T x N_kv x D x b&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;The factor of &lt;code&gt;2&lt;/code&gt; reflects keys and values. This expression is intentionally simple, but it captures the engineering point: cache growth is linear in sequence length and layers, and the constant factors are large enough to dominate memory budgets surprisingly quickly.&lt;/p&gt;
&lt;p&gt;Now consider the two main runtime phases:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Prefill&lt;/strong&gt; performs attention over the full prompt and is dominated by large matrix operations with substantial token-level parallelism.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Decode&lt;/strong&gt; processes only one new token at a time, but that token must attend over the entire previously accumulated context.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;In asymptotic notation, prefill has the familiar quadratic dependence on prompt length because the attention matrix grows with the prompt. Decode, with KV cache reuse, avoids recomputing the full history; however, each step still requires reading prior KV state. This means the &lt;em&gt;computational&lt;/em&gt; cost per step is modest relative to prefill, while the &lt;em&gt;memory-access&lt;/em&gt; cost remains tied to context length.&lt;/p&gt;
&lt;p&gt;That distinction is crucial. In other words, decode is cheaper in arithmetic than naive recomputation, but not cheap in bytes moved. The engineering bottleneck therefore shifts from FLOPs to bandwidth.&lt;/p&gt;
&lt;p&gt;This is also the right place to introduce a practical serving distinction:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;TTFT&lt;/strong&gt; (time to first token) is dominated largely by prompt ingestion and prefill&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;steady-state tokens/sec&lt;/strong&gt; is dominated largely by decode efficiency&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;An inference stack that optimises only one of these metrics is incomplete. Production systems must usually trade between them.&lt;/p&gt;
&lt;h2 id=&quot;iii-prefill-as-a-throughput-oriented-compute-phase&quot;&gt;III. Prefill as a Throughput-Oriented Compute Phase&lt;/h2&gt;
&lt;p&gt;Prefill is the stage in which the system absorbs the prompt and converts it into reusable model state. Conceptually, it is still a standard forward pass, but operationally it is the most throughput-friendly part of inference.&lt;/p&gt;
&lt;p&gt;The reason is straightforward. During prefill, the model processes many prompt tokens together. This gives the runtime substantial parallel work:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;large GEMMs in projection and feed-forward layers&lt;/li&gt;
&lt;li&gt;causal attention over the prompt window&lt;/li&gt;
&lt;li&gt;efficient accelerator utilisation due to batchable dense kernels&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;For this reason, prefill is often described as &lt;strong&gt;compute-bound&lt;/strong&gt;. That description is directionally correct, especially on server-class GPUs, but it is worth being more precise. Prefill is compute-favourable because it exhibits higher arithmetic intensity than decode. The machine can do a relatively large amount of arithmetic per byte loaded, and the runtime has enough parallel work to keep tensor cores or vector units occupied.&lt;/p&gt;
&lt;p&gt;Yet even here, edge systems complicate the picture. On smaller accelerators, limited SRAM, reduced on-chip cache, and lower memory bandwidth can make prompt ingestion less cleanly compute-bound than it appears in data-centre discussions. A long prompt may still be expensive not only because the attention matrix grows, but also because the device cannot keep intermediate tensors and weights resident efficiently. Therefore, the practical statement is:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Prefill is the most compute-friendly phase, but not always purely compute-bound on edge hardware.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This nuance matters for implementation. Systems intended for constrained devices often rely on:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;chunked prefill for long prompts&lt;/li&gt;
&lt;li&gt;lower-precision weights and activations&lt;/li&gt;
&lt;li&gt;fused attention kernels that reduce intermediate writes&lt;/li&gt;
&lt;li&gt;admission control that caps prompt length before decode even begins&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The by-product of prefill is equally important: the runtime materialises the KV cache for every processed token. This cache is not a secondary artifact. It is the persistent state that decode will depend on for the remainder of the session.&lt;/p&gt;
&lt;h2 id=&quot;iv-decode-as-a-stateful-bandwidth-dominated-pipeline&quot;&gt;IV. Decode as a Stateful, Bandwidth-Dominated Pipeline&lt;/h2&gt;
&lt;p&gt;Decode is where the user sees tokens appear, but from a systems perspective it is also where inference becomes structurally awkward. The pipeline has little token-level parallelism because each step depends on the previous output. The runtime therefore loses the high-throughput geometry that made prefill efficient.&lt;/p&gt;
&lt;p&gt;At each decode step, the model:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;embeds the newly chosen token&lt;/li&gt;
&lt;li&gt;computes fresh queries, keys, and values for that token&lt;/li&gt;
&lt;li&gt;reads previously stored KV state from all earlier positions&lt;/li&gt;
&lt;li&gt;produces the next-token logits&lt;/li&gt;
&lt;li&gt;appends the new token’s KV entries to the cache&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;The key observation is that only a tiny amount of &lt;em&gt;new&lt;/em&gt; computation is being introduced per step, while a large volume of &lt;em&gt;old&lt;/em&gt; state must still be consulted. That is why decode is typically &lt;strong&gt;memory-bandwidth dominated&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This statement is sometimes simplified into ‘decode is linear rather than quadratic’, but that formulation can be misleading for engineering work. What matters in practice is not just asymptotic complexity, but the ratio between arithmetic and bytes transferred. Decode often has poor arithmetic intensity: the accelerator performs relatively little compute while repeatedly streaming cache lines, global-memory pages, or unified-memory segments containing KV state.&lt;/p&gt;
&lt;p&gt;On edge devices, this situation is even harsher for several reasons:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;memory bandwidth is far lower than on server GPUs&lt;/li&gt;
&lt;li&gt;unified memory may force contention between CPU, GPU, and other subsystems&lt;/li&gt;
&lt;li&gt;thermal throttling reduces sustained performance&lt;/li&gt;
&lt;li&gt;context growth steadily increases the amount of state read per step&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The result is that decode latency often degrades as the conversation grows, even if the model weights remain constant and the amount of new work per token appears small.&lt;/p&gt;
&lt;p&gt;This is why decode should be understood not merely as ‘generation’ but as a &lt;strong&gt;stateful streaming pipeline&lt;/strong&gt;. Its primary challenge is not deep mathematics; it is the repeated movement and management of persistent attention state.&lt;/p&gt;
&lt;h2 id=&quot;v-kv-cache-as-the-persistent-state-layer-of-inference&quot;&gt;V. KV Cache as the Persistent State Layer of Inference&lt;/h2&gt;
&lt;p&gt;In many high-level discussions, KV cache is introduced as an optimisation. That is true in a narrow sense, but the label understates its role. In serving systems, KV cache is better understood as the persistent state layer that bridges prefill and decode.&lt;/p&gt;
&lt;p&gt;For every layer and every processed token, the runtime stores:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;a key vector&lt;/li&gt;
&lt;li&gt;a value vector&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These tensors become the historical memory against which future queries attend. Once this viewpoint is adopted, several engineering issues become clearer.&lt;/p&gt;
&lt;h3 id=&quot;1-capacity-is-a-first-class-resource&quot;&gt;1. Capacity is a first-class resource&lt;/h3&gt;
&lt;p&gt;KV cache competes with model weights, temporary workspaces, runtime bookkeeping, and sometimes the application itself for memory budget. On an edge device, where memory is often shared with the operating system and other workloads, cache capacity is not an abstract scaling concern. It is an operational limit.&lt;/p&gt;
&lt;h3 id=&quot;2-allocation-strategy-matters&quot;&gt;2. Allocation strategy matters&lt;/h3&gt;
&lt;p&gt;If cache is allocated as large contiguous regions per request, fragmentation and reallocation become problematic. Long-running sessions, uneven output lengths, and dynamic batching can quickly produce allocator inefficiency.&lt;/p&gt;
&lt;h3 id=&quot;3-cache-layout-affects-kernel-efficiency&quot;&gt;3. Cache layout affects kernel efficiency&lt;/h3&gt;
&lt;p&gt;Even when two layouts store the same logical information, their physical arrangement influences coalesced memory access, cache locality, and page reuse. A poor layout can convert an already bandwidth-sensitive decode path into a pathological one.&lt;/p&gt;
&lt;h3 id=&quot;4-cache-lifetime-must-be-managed-explicitly&quot;&gt;4. Cache lifetime must be managed explicitly&lt;/h3&gt;
&lt;p&gt;Inference runtimes need policies for session eviction, truncation, reuse, and sharing. Without such policies, a system may achieve good single-request performance while collapsing under multi-session pressure.&lt;/p&gt;
&lt;p&gt;For these reasons, serious inference systems usually adopt block-based or page-like cache organisation rather than naive contiguous tensors.&lt;/p&gt;
&lt;h2 id=&quot;vi-prefix-reuse-block-tables-and-the-difference-between-shared-storage-and-shared-work&quot;&gt;VI. Prefix Reuse, Block Tables, and the Difference Between Shared Storage and Shared Work&lt;/h2&gt;
&lt;p&gt;Many serving workloads contain repeated prompt structure. System instructions, safety scaffolding, tool descriptions, and conversation history often form a relatively stable &lt;strong&gt;prefix&lt;/strong&gt;, while only the newest user turn acts as the &lt;strong&gt;suffix&lt;/strong&gt;. This observation leads naturally to prefix reuse.&lt;/p&gt;
&lt;p&gt;If the prefix is stable, the runtime can precompute and retain the prefix KV cache. Later requests then only need to process the suffix before entering decode. In abstract form:&lt;/p&gt;
&lt;p&gt;&lt;code&gt;prefill(prefix) -&amp;gt; persistent cache&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;prefill(suffix) -&amp;gt; request-local cache&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;&lt;code&gt;decode&lt;/code&gt;&lt;/p&gt;
&lt;p&gt;When the prefix is large and the suffix is small, this eliminates a substantial amount of redundant prefill work. In practice, however, making prefix reuse effective requires more than simply storing a tensor and pointing multiple requests at it.&lt;/p&gt;
&lt;p&gt;Modern runtimes therefore use &lt;strong&gt;block tables&lt;/strong&gt; or similar indirection mechanisms. Instead of forcing each request to own one contiguous KV region, the runtime composes a logical sequence out of blocks. A shared prefix can then be represented once and referenced by many requests, while each request appends its own suffix and generated tokens through separate blocks.&lt;/p&gt;
&lt;p&gt;This yields several benefits:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;copy-on-write semantics become possible&lt;/li&gt;
&lt;li&gt;long prefixes can be shared without physical duplication&lt;/li&gt;
&lt;li&gt;dynamic growth during decode becomes easier to manage&lt;/li&gt;
&lt;li&gt;allocator fragmentation is reduced&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;However, prefix sharing introduces a subtle but extremely important distinction:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;shared storage is not the same as shared compute&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Even if the prefix KV exists only once in memory, many decode kernels still reread that prefix separately for each active request. The storage cost has been amortised, but the bandwidth cost may not have been. This is the hidden limit of naive prefix reuse.&lt;/p&gt;
&lt;p&gt;That distinction is especially important for edge serving. A mobile or embedded device may have enough capacity to hold the shared prefix exactly once, yet still lack the bandwidth to scan it repeatedly for multiple decode streams. This is why prefix reuse is necessary but not sufficient. It solves redundant prefill computation, not automatically redundant decode traffic.&lt;/p&gt;
&lt;h2 id=&quot;vii-why-edge-inference-is-structurally-harder-than-server-inference&quot;&gt;VII. Why Edge Inference Is Structurally Harder Than Server Inference&lt;/h2&gt;
&lt;p&gt;It is tempting to treat edge deployment as a smaller version of server deployment. That intuition is usually wrong. Edge inference is harder not just because the device is weaker, but because the shape of the constraints is different.&lt;/p&gt;
&lt;h3 id=&quot;1-memory-bandwidth-is-disproportionately-scarce&quot;&gt;1. Memory bandwidth is disproportionately scarce&lt;/h3&gt;
&lt;p&gt;Server GPUs are designed to sustain very high-bandwidth tensor workloads. Edge devices are not. Once decode becomes bandwidth-limited, server-oriented assumptions break down quickly.&lt;/p&gt;
&lt;h3 id=&quot;2-memory-is-often-shared-and-externally-contested&quot;&gt;2. Memory is often shared and externally contested&lt;/h3&gt;
&lt;p&gt;An on-device runtime may share memory fabric with the CPU, display pipeline, camera stack, ISP, or other application subsystems. The inference engine is therefore not merely slower; it is less isolated.&lt;/p&gt;
&lt;h3 id=&quot;3-thermal-limits-are-part-of-the-serving-model&quot;&gt;3. Thermal limits are part of the serving model&lt;/h3&gt;
&lt;p&gt;Data-centre accelerators are engineered for sustained throughput. Phones, laptops, and embedded modules are engineered for mixed workloads and power envelopes. A decode loop that looks acceptable for ten seconds may degrade materially over longer sessions.&lt;/p&gt;
&lt;h3 id=&quot;4-weight-placement-and-cache-placement-interact&quot;&gt;4. Weight placement and cache placement interact&lt;/h3&gt;
&lt;p&gt;On smaller devices, it is not enough to ask whether the model fits. One must ask whether weights, KV cache, scratch buffers, and runtime metadata can coexist while still leaving enough bandwidth headroom for generation. This is where quantised weights alone are often insufficient. Compressing the model without addressing cache growth may simply move the bottleneck rather than remove it.&lt;/p&gt;
&lt;h3 id=&quot;5-scheduling-policy-becomes-product-critical&quot;&gt;5. Scheduling policy becomes product-critical&lt;/h3&gt;
&lt;p&gt;In a server environment, one may accept queueing and multiplexing to improve aggregate throughput. On-device, the user usually experiences one foreground interaction at a time, so responsiveness, jitter, and thermal stability matter more than peak batch throughput.&lt;/p&gt;
&lt;p&gt;For these reasons, edge inference should not be understood as a truncated form of cloud inference. It is a distinct systems regime with different optimisation priorities.&lt;/p&gt;
&lt;h2 id=&quot;viii-practical-optimisation-directions-and-their-trade-offs&quot;&gt;VIII. Practical Optimisation Directions and Their Trade-Offs&lt;/h2&gt;
&lt;p&gt;Because the bottlenecks of edge inference are dominated by state size and state movement, the most valuable optimisations are those that either reduce KV footprint, reduce bytes read per step, or improve the usefulness of each memory transfer.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Technique&lt;/th&gt;
&lt;th&gt;Primary benefit&lt;/th&gt;
&lt;th&gt;What it does not solve&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;Weight quantisation&lt;/td&gt;
&lt;td&gt;Reduces model memory and weight bandwidth&lt;/td&gt;
&lt;td&gt;Does not stop KV cache from growing with context&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;KV-cache quantisation&lt;/td&gt;
&lt;td&gt;Reduces cache capacity and decode traffic&lt;/td&gt;
&lt;td&gt;May introduce quality or implementation complexity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Grouped-query / multi-query attention&lt;/td&gt;
&lt;td&gt;Shrinks KV representation by reducing KV-head multiplicity&lt;/td&gt;
&lt;td&gt;Does not remove long-context rereads entirely&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Sliding-window attention&lt;/td&gt;
&lt;td&gt;Caps effective context length and stabilises decode cost&lt;/td&gt;
&lt;td&gt;Sacrifices access to older context unless combined with retrieval/summarisation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Prefix reuse&lt;/td&gt;
&lt;td&gt;Eliminates repeated prefill for shared prompt segments&lt;/td&gt;
&lt;td&gt;Does not automatically eliminate repeated decode scans&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Paged/block allocation&lt;/td&gt;
&lt;td&gt;Improves memory utilisation and sharing&lt;/td&gt;
&lt;td&gt;Cannot by itself create missing bandwidth&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Speculative decoding&lt;/td&gt;
&lt;td&gt;Can raise output throughput in some regimes&lt;/td&gt;
&lt;td&gt;On constrained devices, verifier overhead may erase gains&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;Several broader lessons follow from this table.&lt;/p&gt;
&lt;p&gt;First, there is rarely a single decisive optimisation. Edge inference usually becomes viable through layered reductions across weights, cache size, scheduler overhead, and kernel inefficiency.&lt;/p&gt;
&lt;p&gt;Second, techniques that appear equivalent in theory are not equivalent in implementation. For example, reducing logical context length, reducing physical KV precision, and changing the number of KV heads all affect different parts of the bottleneck stack.&lt;/p&gt;
&lt;p&gt;Third, optimisations should be evaluated against the actual product objective. If the goal is low TTFT for a short-lived assistant query, chunked prefill and prompt reuse may dominate. If the goal is long interactive sessions on-device, cache management and decode bandwidth become much more important.&lt;/p&gt;
&lt;h2 id=&quot;ix-a-better-mental-model-for-future-work&quot;&gt;IX. A Better Mental Model for Future Work&lt;/h2&gt;
&lt;p&gt;The most productive way to think about inference on edge platforms is not ‘How do we run a Transformer on a smaller machine?’ but rather:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;How do we manage persistent model state under strict bandwidth and thermal constraints while preserving useful output quality?&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;That framing shifts attention towards the right engineering questions:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Can prefix state be shared without being reread redundantly?&lt;/li&gt;
&lt;li&gt;Can cache layout be aligned more tightly with decode access patterns?&lt;/li&gt;
&lt;li&gt;Can attention be reorganised so that long stable prefixes behave more like compressed memory than raw token history?&lt;/li&gt;
&lt;li&gt;Can serving policy adapt context retention dynamically based on latency budget, device temperature, or interaction mode?&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;These are not merely kernel-level questions. They cut across model architecture, runtime design, scheduler policy, and product constraints.&lt;/p&gt;
&lt;p&gt;In that sense, the future of efficient edge inference is unlikely to come from a single miracle kernel. It will more likely come from co-design: models with cache-friendly attention structure, runtimes with more intelligent state sharing, and serving policies that explicitly acknowledge that memory bandwidth, not just parameter count, is the scarce currency.&lt;/p&gt;
&lt;h2 id=&quot;x-conclusion&quot;&gt;X. Conclusion&lt;/h2&gt;
&lt;p&gt;The standard decomposition of LLM inference into embedding, prefill, and decode is not a pedagogical convenience. It is a systems abstraction that exposes three different operational regimes.&lt;/p&gt;
&lt;p&gt;Embedding is relatively lightweight and interface-defining. Prefill is the throughput-oriented phase in which prompt tokens are transformed into reusable internal state. Decode is the stateful, low-parallelism phase in which the runtime repeatedly trades memory traffic for avoided recomputation.&lt;/p&gt;
&lt;p&gt;KV cache sits at the centre of this design. It makes autoregressive generation practical, but it also redefines the optimisation problem. Once cache is introduced, efficient inference is no longer just a question of arithmetic throughput. It becomes a question of how much state must be stored, how often that state must be reread, how intelligently it can be shared, and whether the hardware can sustain those transfers under real deployment constraints.&lt;/p&gt;
&lt;p&gt;For edge platforms, this is the decisive engineering reality. The governing resource is often not abstract model size alone, but the full economics of state: capacity, bandwidth, lifetime, locality, and thermal sustainability. Any serious discussion of professional LLM inference infrastructure therefore has to begin, and eventually end, with KV cache.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Performative Learning under Modern Instrumental Rationality</title>
    <link href="https://www.ailven.cc/posts/moment-260110.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260110.html</id>
    <published>2026-01-10T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>This essay analyses performative learning as a structurally induced response to modern regimes of instrumental rationality. It identifies a status-sensitive subtype whose behaviour is shaped by ambition, externalised self-worth, and acute responsiveness to rank differentials.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 10 January 2026&lt;br&gt;&lt;strong&gt;Latest Update Date&lt;/strong&gt;: 15 January 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In contemporary large-scale societies – particularly within institutionalised environments such as schools, universities, and corporate organisations – learning has undergone a subtle yet profound transformation. Increasingly, learning is structured as an &lt;em&gt;act of appearance&lt;/em&gt;: a strategically staged process oriented towards visibility, evaluation, and comparative advantage, rather than around the internal demands of understanding, mastery, or intellectual coherence. This phenomenon, which may be provisionally termed performative learning, is a structural response to modern regimes of instrumental rationality.&lt;/p&gt;
&lt;p&gt;Yet performative learning is not a psychologically homogeneous phenomenon. Field observations across educational and occupational settings suggest that within this broad pattern exists a subgroup whose behavioural and affective dispositions exhibit a particularly sharp alignment with performativity. This group is characterised by a highly refined sensitivity to status differentials, coupled with a concealed yet persistent aspiration towards dominance. Understanding this internal differentiation is essential if performative learning is to be analysed with conceptual seriousness rather than moral caricature.&lt;/p&gt;
&lt;h2 id=&quot;i-learning-as-performance-in-a-society-of-evaluation&quot;&gt;I. Learning as Performance in a Society of Evaluation&lt;/h2&gt;
&lt;p&gt;Modern institutions increasingly operate through metrics, rankings, dashboards, and narrativised trajectories of progress. Learning, under such conditions, becomes legible only insofar as it can be &lt;em&gt;rendered visible&lt;/em&gt; – certified, narrated, compared, and archived. The epistemic question (‘Do I understand?’) is gradually displaced by a performative one (‘How does this learning appear?’).&lt;/p&gt;
&lt;p&gt;This displacement does not require cynicism or bad faith. Many individuals sincerely equate learning with its demonstrability. What matters is the production of recognisable signals – credentials, fluency, rhetorical competence, alignment with dominant discourses – rather than any internal reorganisation of cognition. Learning thus becomes a publicly consumable artefact more than an inward process of formation.&lt;/p&gt;
&lt;p&gt;However, within this environment, some individuals go further. For them, performative learning becomes a &lt;em&gt;vehicle for positional ambition&lt;/em&gt;, beyond any rational adaptation to evaluation systems.&lt;/p&gt;
&lt;h2 id=&quot;ii-status-sensitivity-and-the-vertical-imagination-of-learning&quot;&gt;II. Status Sensitivity and the Vertical Imagination of Learning&lt;/h2&gt;
&lt;p&gt;Among performative learners, a recurrent pattern emerges: a pronounced vertical imagination of social life. Individuals continuously and often tacitly assess their relative position – who is above, who is below, who is rising, who is expendable. Learning, in this schema, serves to locate oneself advantageously within a hierarchy, more than to acquire knowledge.&lt;/p&gt;
&lt;p&gt;This manifests behaviourally in a striking asymmetry. Towards those perceived as superior in competence, authority, or symbolic capital, the individual exhibits warmth, humility, deference, and even admiration. Towards those perceived as inferior, the posture shifts – sometimes subtly, sometimes overtly – towards condescension, detachment, or quiet disdain.&lt;/p&gt;
&lt;p&gt;Crucially, the subject need not experience this as hypocrisy. It reflects a conditional ethics of interaction: respect is indexed to position rather than to personhood. One might say that such individuals practise moral &lt;em&gt;contingency&lt;/em&gt; rather than moral inconsistency.&lt;/p&gt;
&lt;h2 id=&quot;iii-ambition-concealment-and-the-art-of-self-presentation&quot;&gt;III. Ambition, Concealment, and the Art of Self-Presentation&lt;/h2&gt;
&lt;p&gt;A notable feature of this subgroup is the coexistence of strong ambition with careful self-concealment. There is often a deep, sometimes unarticulated desire to occupy a position of superiority – to command, to judge, to be deferred to. This desire is rarely expressed directly. It is sublimated instead into acceptable institutional currencies: excellence, diligence, adaptability, learning capacity.&lt;/p&gt;
&lt;p&gt;Performative learning offers an ideal medium for this sublimation. By &lt;em&gt;excelling&lt;/em&gt; at the appearance of learning – rapid uptake of fashionable concepts, fluent reproduction of dominant vocabularies, strategic demonstration of competence – the individual advances position without overtly declaring ambition. Deference to stronger figures is an investment strategy, not mere politeness. Superiority over weaker figures, when it appears, functions as delayed confirmation of worth.&lt;/p&gt;
&lt;p&gt;The alternation between humility and arrogance is better analysed as a situational modulation of selfhood, finely attuned to hierarchical gradients, than as a personality flaw in isolation.&lt;/p&gt;
&lt;h2 id=&quot;iv-externalised-self-worth-and-the-economy-of-recognition&quot;&gt;IV. Externalised Self-Worth and the Economy of Recognition&lt;/h2&gt;
&lt;p&gt;At a deeper psychological level, this pattern reflects a form of externally anchored self-worth. Value is secured through comparative recognition rather than through internal standards of understanding or craftsmanship. The self becomes intelligible to itself only via the gaze of others – particularly those positioned above.&lt;/p&gt;
&lt;p&gt;Here, performative learning serves a dual function. Upwards, learning performances are addressed to superiors as signals of potential, loyalty, or excellence. Downwards, the same performances establish distance from those deemed less capable, less refined, or less successful.&lt;/p&gt;
&lt;p&gt;This dynamic produces a fragile equilibrium. As long as upward recognition is forthcoming and downward differentiation is possible, the self remains coherent. When either condition fails – when one encounters a superior who is indifferent, or when one’s own position deteriorates – the psychic structure is exposed to instability.&lt;/p&gt;
&lt;h2 id=&quot;v-structural-reinforcement-in-modern-institutions&quot;&gt;V. Structural Reinforcement in Modern Institutions&lt;/h2&gt;
&lt;p&gt;It would be misleading to frame this behavioural constellation as a personal pathology. Contemporary institutions actively reward it.&lt;/p&gt;
&lt;p&gt;Modern schools and workplaces incentivise upward signalling more than lateral cooperation, punish open ambition less than misplaced humility, and tolerate – sometimes implicitly endorsing – condescension towards subordinates while demanding politeness towards superiors.&lt;/p&gt;
&lt;p&gt;Under such conditions, individuals who can precisely calibrate their demeanour to hierarchical context are often labelled as ‘emotionally intelligent’, ‘politically savvy’, or ‘mature’. The moral content of their behaviour – who deserves respect, and why – remains largely unscrutinised.&lt;/p&gt;
&lt;p&gt;Performative learning, in this sense, is one of status-sensitive ambition’s most efficient instruments.&lt;/p&gt;
&lt;h2 id=&quot;vi-the-inner-tension-of-performative-mastery&quot;&gt;VI. The Inner Tension of Performative Mastery&lt;/h2&gt;
&lt;p&gt;Despite its adaptive advantages, this configuration carries latent costs. Because learning is oriented towards appearance, knowledge often remains fragmentary and reversible – retained so long as it serves positional needs, abandoned when it does not. Depth gives way to flexibility; coherence to opportunism.&lt;/p&gt;
&lt;p&gt;Relationally, the individual may find it difficult to sustain genuinely reciprocal bonds. Relationships are silently evaluated for their positional utility, making equality uncomfortable and dependency threatening. Respect given today may be withdrawn tomorrow if the hierarchy shifts.&lt;/p&gt;
&lt;p&gt;Perhaps most critically, the self remains perpetually exposed to the risk of status loss. Since worth is externally adjudicated, failure becomes existential rather than circumstantial. The same sensitivity that enables rapid ascent renders descent psychologically catastrophic.&lt;/p&gt;
&lt;h2 id=&quot;vii-beyond-moral-condemnation&quot;&gt;VII. Beyond Moral Condemnation&lt;/h2&gt;
&lt;p&gt;It is tempting to treat such individuals with moral disdain. Yet doing so would miss the analytic point. What is at stake is a moral economy in which learning, ambition, and recognition are tightly coupled to visibility and hierarchy, and not individual virtue.&lt;/p&gt;
&lt;p&gt;Performative learning, especially in its status-sensitive form, should therefore be understood as a &lt;em&gt;symptom&lt;/em&gt; rather than an anomaly: a human response to systems that conflate knowing with showing, and worth with rank. Its cold rationality mirrors the cold rationality of the institutions that cultivate it.&lt;/p&gt;
&lt;h2 id=&quot;viii-concluding-remarks&quot;&gt;VIII. Concluding Remarks&lt;/h2&gt;
&lt;p&gt;Performative learning under modern instrumental rationality is one of the dominant logics of education and professional development, and not a marginal distortion of them. Within it, the emergence of individuals who combine ambitious aspiration, hierarchical sensitivity, and strategic self-presentation is neither accidental nor rare. These individuals learn in order to be placed – and, if possible, placed above.&lt;/p&gt;
&lt;p&gt;To understand them is not to excuse them. It is to confront, without sentimentality, the conditions under which learning itself becomes a theatre of power.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Cognitive Stratification and Individual Development</title>
    <link href="https://www.ailven.cc/posts/moment-260105.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-260105.html</id>
    <published>2026-01-05T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>A sociological reconstruction of individual development through a four-stage model: institutional screening, linguistic leverage, abstract logic, and holistic self-actualisation – synthesising Bourdieu’s cultural capital with cognitive psychology.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;The Intergenerational Transmission of Cultural Capital: A Sociological Reconstruction of Individual Development&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;As the first article of 2026, I’d like to discuss an interesting yet profound topic – observations on personal growth.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 5 January 2026&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h3 id=&quot;【楔子】-攀登者的阶梯：关于成长与边界的四重观察&quot;&gt;【楔子】 攀登者的阶梯：关于成长与边界的四重观察&lt;/h3&gt;
&lt;p&gt;沉潜于世二十余载，往来于迥异的阶层与环境之间，我逐渐在各色的人生样本中剥离出一种隐秘而残酷的秩序。一个个体的成长高度——其所习得的知识、所掌握的技能、所构筑的精神世界——往往被某种先在的条件划定了边界。这种差异并非杂乱无章，而是呈现出一种层层递进的、近乎宿命般的结构。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;其一，是生存的基准。&lt;/strong&gt;&lt;br&gt;一切的起点，往往始于那个关于“县城青年”的叙事。对于在这片广袤土地上成长的孩子而言，能否在这十二年的基础教育中通过严酷的筛选，不仅决定了他们是否具备进入主流视野的资格，更划定了一条生存的底线。唯有将那些枯燥的基础知识内化为本能，达到足以叩开顶尖学府（如985院校）大门的严谨水准，一个普通人方能证明自己具备了在这个工业化社会创造价值的基本能力。这是入场券，也是大多数人耗尽青春翻越的第一座山丘。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;其二，是视野的突围。&lt;/strong&gt;&lt;br&gt;在这条基准线之上，当生存不再是唯一的焦虑，语言的能力便成为了决定人生半径的关键变量。这里所说的英语，不再是试卷上的分数，而是一把打破信息壁垒的钥匙。在扎实的专业基础之上，若能驾驭这门全球通用的语言，便意味着从局部的内卷走向了广阔的世界。这并非某种文化上的盲从，而是为自己争取到了在全球任何角落体面生活、自由选择的权利——从“被选择”到“拥有选择权”，这是一次维度的跃升。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;其三，是逻辑的重构。&lt;/strong&gt;&lt;br&gt;若要寻求更进一步的超越，数学——尤其是那些现代、抽象且深邃的数学理论（如数论、群论、离散数学）——则构成了通往卓越的阶梯。在这个阶段，数学不再是计算的工具，而是构筑世界的底层逻辑。当一个人具备了这种高阶的抽象思维能力，他便拥有了将专业技能转化为爆发性生产力的引擎。无论是在学术的无人区还是工业的最前沿，这种能力足以让他完成从“执行者”到“开创者”的质变，为社会留下不可磨灭的推动力。&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;其四，是灵魂的自由。&lt;/strong&gt;&lt;br&gt;能够抵达第三层级，或许已是普通人依靠天赋与勤奋所能触及的天花板。然而，在天花板之上，仍有一片属于少数人的星空。他们往往不再背负“县城青年”的沉重标签，早在少年时期便已展露出惊人的从容。&lt;/p&gt;
&lt;p&gt;我曾有一位北大的挚友，便是这片星空的注脚。他在高中时便已跻身NOI亚洲顶尖行列，保送、直博、发表顶级学术成果，这一路对他而言似乎顺理成章。然而，真正令人震撼的并非这些光环，而是他在学术之外的丰盈：他精通竹笛与丹青，钢琴十级，球场上挥洒自如，更在西方哲学与红学研究中有着深厚的造诣。对他而言，卓越不再是谋生的手段，而是生命本真的流淌。这种在科学、艺术与人文之间游刃有余的“全人”姿态，昭示着一种超越了功利的、更为纯粹的自我实现。&lt;/p&gt;
&lt;p&gt;这四个层级，如同一座静默的金字塔，映射着我们这个时代的成长路径与阶层分野。&lt;/p&gt;
&lt;hr&gt;
&lt;h2 id=&quot;abstract&quot;&gt;Abstract&lt;/h2&gt;
&lt;p&gt;This paper postulates a systematic theoretical framework regarding the nexus between skill acquisition, cognitive development, and social mobility in contemporary industrialised societies. Through an analysis of developmental trajectories across disparate socio-economic backgrounds, we propose the ‘Four-Stage Model of Cognitive Stratification’. This model delineates individual development into four distinct tiers: the phase of institutionalised screening via basic education; the phase of linguistic leverage and global perspective; the phase of high-order productivity through abstract logic; and the ultimate phase of distinction via holistic self-actualisation. Drawing upon Pierre Bourdieu’s theories of cultural capital, Gary Becker’s human capital approach, and insights from cognitive psychology, this treatise argues that these stages represent not merely the accumulation of skills, but the complex interplay of social resources, familial endowment, and cognitive capacity. The study reveals the underlying logic of class differentiation, tracing the ascent from the realm of necessity to the realm of freedom.&lt;/p&gt;
&lt;h2 id=&quot;i-introduction-the-isomorphism-of-competence-and-stratification&quot;&gt;I. Introduction: The Isomorphism of Competence and Stratification&lt;/h2&gt;
&lt;p&gt;In a highly specialised and globalised modern society, the trajectory of individual growth is rarely a stochastic process; rather, it exhibits highly structural characteristics. Whilst social mobility remains a theoretical possibility, there exists a profound correlation between the knowledge structures, skill sets, and aesthetic dispositions an individual masters, and their locus within the social hierarchy.&lt;/p&gt;
&lt;p&gt;Traditional scholarship in the sociology of education has frequently fixated on academic credentials as institutionalised validation, often neglecting the substantive cognitive depth and the nature of skills behind these credentials. In reality, the core competencies an individual pursues and masters at various life stages – ranging from standardised foundational knowledge to cross-cultural linguistic proficiency, and from abstract mathematical logic to non-utilitarian artistic cultivation – constitute a rigorous ‘Pyramid of Competence’. This pyramid maps not only the value of an individual’s human capital but also reflects the stock of cultural capital within their family of origin and the logic of social reproduction.&lt;/p&gt;
&lt;p&gt;This paper endeavours to transcend specific cultural contexts or regional singularities. Instead, it categorises individual development into four universal strata, providing an in-depth sociological examination of the implications of each level and their decisive impact on individual destiny.&lt;/p&gt;
&lt;h2 id=&quot;ii-stage-i-the-threshold-of-discipline-and-institutionalised-screening&quot;&gt;II. Stage I: The Threshold of Discipline and Institutionalised Screening&lt;/h2&gt;
&lt;p&gt;For any individual in a modern society, the inception of upward mobility invariably begins with standardised basic education. For those born into environments of relative resource scarcity – such as provincial towns or underdeveloped rural areas – this stage represents the singular aperture through which they may enter the mainstream competitive system.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Standardised Knowledge as a Test of Compliance&lt;/strong&gt;&lt;br&gt;During the K-12 educational phase (primary through secondary education), individuals are required to master standardised knowledge in literacy, numeracy, and general sciences. For those aiming to transcend their class origins through education, the core imperative of this stage is not ‘innovation’, but ‘precision in execution’.&lt;/p&gt;
&lt;p&gt;From the perspective of socialisation, success in this phase – exemplified by admission to elite research universities – primarily assays the individual’s &lt;strong&gt;Executive Function&lt;/strong&gt;. This encompasses impulse control, working memory, and, crucially, the capacity for delayed gratification. The rigorous mastery of often repetitive and rote foundational knowledge serves as a proxy, demonstrating to society that the individual possesses the potential to function as a disciplined component of the industrial workforce or as a junior intellectual labourer.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. The Acquisition of Institutionalised Cultural Capital&lt;/strong&gt;&lt;br&gt;Success at this stage marks the acquisition of what Bourdieu termed ‘Institutionalised Cultural Capital’ – the academic credential. For youth from resource-poor backgrounds, this is often the sole chip they hold in the high-stakes game of social mobility. However, this also represents a ‘glass ceiling’ for the majority. As basic education becomes universal, the scarcity value of foundational knowledge diminishes rapidly. Its marginal utility plummets once the entry ticket to an elite institution is secured. Mere mastery of the curriculum guarantees only a position at the base of the division of labour; it ensures survival and the right to participate in basic social cooperation, but it confers little power to exercise genuine autonomy over one’s life choices.&lt;/p&gt;
&lt;h2 id=&quot;iii-stage-ii-linguistic-capital-and-the-leverage-of-information-arbitrage&quot;&gt;III. Stage II: Linguistic Capital and the Leverage of Information Arbitrage&lt;/h2&gt;
&lt;p&gt;Upon crossing the threshold of basic education and entering higher education or the broader labour market, &lt;strong&gt;universal foreign language proficiency&lt;/strong&gt; (predominantly high-level English in the contemporary context) emerges as the first watershed determining the upper limits of development.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Language as a Meta-Tool and Information Bandwidth&lt;/strong&gt;&lt;br&gt;At this juncture, foreign language proficiency transcends its status as an academic ‘subject’ and metamorphoses into a &lt;strong&gt;Meta-tool&lt;/strong&gt; for information acquisition. From the standpoint of Information Theory, the calibre of one’s linguistic ability dictates the ‘bandwidth’ and ‘signal-to-noise ratio’ of external information reception.&lt;/p&gt;
&lt;p&gt;Given that the frontiers of global science, technology, commercial logic, and academic discourse are primarily encoded in a universal lingua franca, individuals with high-order linguistic competence can plug directly into the global information network, thereby eliminating &lt;strong&gt;Information Asymmetry&lt;/strong&gt;. Conversely, those reliant solely on vernacular sources are often relegated to consuming second-hand information that is translated, filtered, or delayed. Over the long cycle of a career, this latency and distortion accumulate into a significant cognitive chasm.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. Geo-Arbitrage in the Global Labour Market&lt;/strong&gt;&lt;br&gt;Social Network Theory suggests that actors capable of bridging ‘Structural Holes’ garner higher social returns. Superior linguistic capability empowers individuals to transcend geographical and cultural boundaries, granting access to the Global Labour Market.&lt;/p&gt;
&lt;p&gt;This confers upon the individual the power of &lt;strong&gt;‘Geo-arbitrage’&lt;/strong&gt;: the ability to seek optimal resource allocation on a planetary scale – working in high-wage economies or serving top-tier global institutions without being tethered to the economic structure of their birthplace. For individuals from peripheral regions, this signifies a qualitative leap from ‘being selected by the local environment’ to ‘actively selecting one’s environment,’ marking a substantial expansion of personal freedom.&lt;/p&gt;
&lt;h2 id=&quot;iv-stage-iii-abstract-logic-and-the-construction-of-high-order-productivity&quot;&gt;IV. Stage III: Abstract Logic and the Construction of High-Order Productivity&lt;/h2&gt;
&lt;p&gt;If the second stage grants perspective and choice, the third stage determines an individual’s capacity to reach the zenith of professional fields and generate breakthrough value. This is predicated upon &lt;strong&gt;High-Order Mathematical Thinking and Abstract Logical Capability&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Modern Mathematics as the Underlying Language of Reality&lt;/strong&gt;&lt;br&gt;The mathematical capability referred to here is not the arithmetic of basic education, but modern abstract theories such as Discrete Mathematics, Group Theory, Number Theory, and Topology. In terms of cognitive psychology, mastery of these theories implies that the individual’s mind possesses robust &lt;strong&gt;Abstract Modelling capabilities&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;In high-value-added sectors such as Artificial Intelligence, Financial Engineering, Cryptography, and Complex Systems Science, the intricate problems of the real world must be reduced to mathematical models to be solvable. Individuals capable of wielding these modern mathematical tools cease to be mere ‘operators’ of existing technical systems; they become the ‘architects’ of rules and the ‘creators’ of systems.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. The High Marginal Utility of Skills and the Technocratic Apex&lt;/strong&gt;&lt;br&gt;The economic theory of ‘Skill-Biased Technological Change’ (SBTC) posits that modern technological progress favours high-skilled labour. Individuals with profound backgrounds in mathematics and logic can transmute professional skills into highly scalable productivity.&lt;/p&gt;
&lt;p&gt;Whether in pioneering academic research or algorithmic innovation in industry, the scarcity of this capability is acute, commanding the highest premiums in the labour market. Individuals at this level are typically the paragons of the Technocratic class. Through the extreme application of Intellectual Capital, they push the frontiers of societal productivity. This often represents the ceiling of career achievement attainable through sheer personal endeavour and intellectual aptitude.&lt;/p&gt;
&lt;h2 id=&quot;v-stage-iv-embodied-cultural-capital-and-the-distinction-of-self-actualisation&quot;&gt;V. Stage IV: Embodied Cultural Capital and the Distinction of Self-Actualisation&lt;/h2&gt;
&lt;p&gt;Above the preceding three stages lies a stratum that is rarer still, one that is arduous to reach through short-term effort alone. Individuals in this tier possess not only superior cognitive faculties but also profound attainments in the humanities, arts, and philosophy. They are often no longer constrained by the logic of ‘survival’ or ‘livelihood’, pursuing instead what Abraham Maslow termed &lt;strong&gt;Self-Actualisation&lt;/strong&gt; or even &lt;strong&gt;Self-Transcendence&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;1. Polymathy and the Fruits of Holistic Education&lt;/strong&gt;&lt;br&gt;The archetypal characteristic of this stage is Polymathy and the ‘Renaissance Man’ ideal. Whether winning gold medals in international science Olympiads or achieving professional standards in music, painting, or literary critique, these feats demonstrate the outcome of &lt;strong&gt;Holistic Education&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;Such achievements are rarely the result of a single generation’s striving. They require substantial economic capital from the family in the early years and, more critically, a high-cognitive domestic environment. This resource investment bears a high compounding interest over time. For instance, mastering a musical instrument or conducting deep research into classical philosophy requires immense ‘leisure time’ and ‘non-productive investment’. This stands in stark contrast to the first stage, where every waking hour must be dedicated to examination preparation as a survival strategy.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;2. Aesthetics as the Moat of Class Distinction&lt;/strong&gt;&lt;br&gt;In &lt;em&gt;Distinction: A Social Critique of the Judgement of Taste&lt;/em&gt;, Bourdieu argues that aesthetic taste is a core marker of social stratification. Compared to economic capital, &lt;strong&gt;Embodied Cultural Capital&lt;/strong&gt; – knowledge and cultivation internalised into an individual’s disposition, speech, and taste – is far more difficult to imitate or acquire later in life.&lt;/p&gt;
&lt;p&gt;Individuals in the fourth stage do not pursue knowledge for its ‘utility’, but for ‘truth’ and ‘beauty’. This effortless command of the ‘useless’ constitutes the most formidable &lt;strong&gt;moat&lt;/strong&gt; of the elite class. They do not need to prove their survival value through mathematics or languages (though they often possess these skills); they confirm their existence by creating beauty and interpreting the essence of the world. This is a &lt;strong&gt;‘Habitus of Freedom’&lt;/strong&gt; born of a privileged environment.&lt;/p&gt;
&lt;h2 id=&quot;vi-discussion-the-funnel-model-of-stratification-and-intergenerational-dynamics&quot;&gt;VI. Discussion: The ‘Funnel Model’ of Stratification and Intergenerational Dynamics&lt;/h2&gt;
&lt;p&gt;Synthesising the above, we may conceptualise these four stages as a &lt;strong&gt;‘Funnel Model of Cognitive Stratification’&lt;/strong&gt;. This model exposes the harsh realities of social mobility and the significance of intergenerational accumulation.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Path Dependence and Linearity&lt;/strong&gt;:&lt;br&gt; For individuals originating from resource-constrained environments (Stage I start), the life strategy is often linear: they must first conquer basic education (Stage I) to gain entry, then address linguistic deficits (Stage II) to gain options, and finally rely on intellectual talent to assault the professional summit (Stage III). Each leap involves tremendous energy expenditure, leaving few with the surplus energy to touch the realm of Stage IV.&lt;br&gt; Conversely, individuals from elite backgrounds (Stage IV start) often experience non-linear development. They internalise the global perspective of Stage II and the liberal arts literacy of Stage IV during adolescence. Stage I is a default configuration for them, and Stage III is a choice based on interest rather than a necessity for survival.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;The Dynamic Flux of Marginal Utility&lt;/strong&gt;:&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Devaluation of Stage I&lt;/strong&gt;: With the massification of higher education, the signalling function of basic degrees is weakening.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Appreciation of Stage III&lt;/strong&gt;: In the age of Artificial Intelligence, the rewards for fundamental logic and abstract capability are growing exponentially.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;The Irreplaceability of Stage IV&lt;/strong&gt;: As technology substitutes standardised skills (Stage I) and even some high-level technical skills (Stages II/III), the creativity, aesthetic judgement, philosophical speculation, and complex emotional resonance represented by Stage IV will remain the final bastion of human competitiveness.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id=&quot;vii-conclusion&quot;&gt;VII. Conclusion&lt;/h2&gt;
&lt;p&gt;Through a deep dissection of the process of knowledge structure and skill acquisition, we observe that an individual’s developmental height is contingent not only upon personal intellect and industry but is also heavily constrained by their structural position in society and the accumulation of familial cultural capital.&lt;/p&gt;
&lt;p&gt;From the discipline of basic education (Stage I), through the breakout of linguistic capability (Stage II), to the construction of mathematical logic (Stage III), and finally arriving at the freedom of liberal arts (Stage IV), this is a pilgrimage from the &lt;strong&gt;Kingdom of Necessity&lt;/strong&gt; to the &lt;strong&gt;Kingdom of Freedom&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;For the social observer, understanding this model facilitates a more objective view of the life trajectories of different groups. For those who have climbed from the bottom, reaching Stage III is the ultimate victory of personal struggle and commands immense respect. For those exhibiting the traits of Stage IV, we must recognise the cultural sedimentation of generations behind them. The progress of society lies not only in enabling more individuals to escape scarcity through Stages I and II but in fostering educational equity that allows gifted youth from the margins the opportunity to touch the scientific peaks of Stage III and the humanistic radiance of Stage IV, thereby achieving a prosperity of collective intelligence and culture.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>The Counterfeit of the Sublime</title>
    <link href="https://www.ailven.cc/posts/moment-251223.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-251223.html</id>
    <published>2025-12-23T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>Using Yu Xiaoqing as a sociopsychological specimen, this analysis dissects the pathology of Performative Idealism – the sophisticated egoist who camouflages instrumental rationality beneath an aestheticised moral shell.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;On the Pathological Structure and Sociological Metaphor of ‘Performative Idealism’&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Taking the Yu Xiaoqing Archetype as a Point of Departure&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 23 December 2025&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In the television drama series &lt;em&gt;Soldiers and Their Commander&lt;/em&gt; (in Chinese named as 我的团长我的团), the character of Yu Xiaoqing provides a socio-psychological specimen of penetrating insight. He is a ‘Performative Idealist’ rather than a hypocrite in the traditional sense, and he has achieved a high degree of psychological self-consistency. The core characteristic of this personality type is the alienation of idealism into an aesthetic ornament and an extension of power: extreme romance and sublimity in zones free of conflict of interest, an instantaneous switch to extreme instrumental rationality when core interests are triggered, and a rigorous logical system for sanctifying that betrayal.&lt;/p&gt;
&lt;p&gt;This personality extends well beyond the elite class. It resembles a collective symptom widely diffused throughout modern society, particularly among groups gripped by the anxiety of class mobility. This article aims to dissect the deep-seated psychological mechanisms and sociological essence of this ‘sophisticated egoism wrapped in an idealistic shell’ through the mirror of Yu Xiaoqing.&lt;/p&gt;
&lt;h2 id=&quot;i-the-dramaturgical-perspective-the-accumulation-and-display-of-moral-capital&quot;&gt;I. The Dramaturgical Perspective: The Accumulation and Display of Moral Capital&lt;/h2&gt;
&lt;p&gt;From the perspective of Erving Goffman’s ‘Dramaturgical Theory’, the life of a Yu Xiaoqing-style figure is essentially a meticulously choreographed ‘front stage’ performance. For this category of individuals, idealism, patriotism, and humanistic concern serve as critical stage properties required to construct their ‘persona’, rather than as internalised faiths.&lt;/p&gt;
&lt;p&gt;Yu Xiaoqing’s study, his superior weaponry, and his reverence for General Yue Fei constitute the scenography of his performance. In modern society, this corresponds to performative philanthropy on social media, vision statements in the corporate world, or the middle class’s paranoid imitation of ‘high culture’ lifestyles. Pierre Bourdieu pointed out that cultural and moral capital can be converted into power. These individuals are well-versed in this alchemy; by high-profile proselytising of idealism, they accumulate ‘moral capital’, thereby establishing superiority and legitimacy within the group.&lt;/p&gt;
&lt;p&gt;This performance possesses a potent deceptive quality because, in the absence of conflicting interests, they do indeed invest genuine emotion. They are moved by the sense of sublimity they cultivate. However, this emotion is authentic yet narcissistic. What they love is the ‘self who loves the masses’, not the masses. This idealism, based on aesthetics rather than ethics, is destined to be fragile and superficial. It provides the halo but refuses to bear the cost.&lt;/p&gt;
&lt;h2 id=&quot;ii-instrumental-rationality-and-the-reification-of-others-the-cold-logic-behind-the-veil-of-warmth&quot;&gt;II. Instrumental Rationality and the ‘Reification’ of Others: The Cold Logic Behind the Veil of Warmth&lt;/h2&gt;
&lt;p&gt;When this personality type faces a choice involving interests, their psychological mechanism undergoes a violent rupture and reconstruction. This is the ultimate embodiment of ‘Instrumental Reason’ as critiqued by the Frankfurt School.&lt;/p&gt;
&lt;p&gt;When core interests (such as position, wealth, or class security) remain untouched, their attitude towards others is often affectionate, brimming with brotherhood or team spirit. However, once the existence of another becomes an obstacle, or if sacrificing another can yield significant personal gain, the ‘flesh-and-blood human’ is instantly ‘reified’ (Verdinglichung) in their cognition.&lt;/p&gt;
&lt;p&gt;In the drama, this means living soldiers become flags on a sandbox; in modern professional or social relationships, it means erstwhile allies instantly transform into disposable pawns. The terror of this transformation lies in its seamless abruptness. One moment, they are discussing poetry and distant horizons over wine; the next, they are coolly calculating the yield rate of betrayal.&lt;/p&gt;
&lt;p&gt;This stark contrast reveals the desolation at the core of their personality: others are never ends, only means. They lack what Martin Buber termed the ‘I-Thou’ relationship, engaging only in an ‘I-It’ relationship of utility. All idealistic rhetoric is merely sophisticated packaging designed to make this process of utilisation appear less voracious.&lt;/p&gt;
&lt;h2 id=&quot;iii-sartrean-bad-faith-the-legitimisation-narrative-of-betrayal&quot;&gt;III. Sartrean ‘Bad Faith’: The Legitimisation Narrative of Betrayal&lt;/h2&gt;
&lt;p&gt;Why can such individuals maintain a stance of righteousness, or even tragedy, while practising self-interest (such as sacrificing subordinates or acting perfidiously)? This involves the concept of ‘Bad Faith’ (Mauvaise foi) proposed by Jean-Paul Sartre.&lt;/p&gt;
&lt;p&gt;Yu Xiaoqing-style figures cannot endure the self-perception of being base; thus, they must construct a grand narrative to conceal their private desires. When they choose betrayal for personal gain, ‘I want a promotion’ and ‘I am afraid of losing’ become ‘This is for the greater good’, ‘This is for long-term strategy’, ‘This is the price of growth.’&lt;/p&gt;
&lt;p&gt;They excel at escaping moral anxiety through conceptual substitution: ‘personal greed’ becomes ‘collective interest’, ‘cold-blooded calculation’ becomes ‘mature rationality’, ‘exploitation of the weak’ becomes ‘the law of natural selection’.&lt;/p&gt;
&lt;p&gt;Through this psychological defence mechanism, they successfully reshape themselves from perpetrators into ‘burden-bearing martyrs’. They may even weep for the ‘difficult decisions’ they are forced to make, appearing more pained than their victims. This performative suffering is the most sophisticated camouflage of the refined egoist: it dissolves the legitimacy of the victim’s grievance and further reinforces the moral invincibility of the perpetrator.&lt;/p&gt;
&lt;h2 id=&quot;iv-social-pathology-under-the-anxiety-of-class-mobility&quot;&gt;IV. Social Pathology Under the Anxiety of Class Mobility&lt;/h2&gt;
&lt;p&gt;Expanding our horizon to the broader modern society, the proliferation of the Yu Xiaoqing personality is, in reality, a product of the anxiety surrounding class mobility. In a highly competitive and atomised social structure, many who pursue ‘upward mobility’ or a ‘refined life’ are forced, or actively choose, to mutate into this personality type.&lt;/p&gt;
&lt;p&gt;For those yearning to climb the ladder, pure idealism is an expensive luxury, while being an unvarnished villain invites social exclusion. Therefore, the ‘Machiavellian in saint’s clothing’ becomes the optimal evolutionary strategy.&lt;/p&gt;
&lt;p&gt;The first strategy is elite mimicry: they imitate the tone, aesthetics, and values of the upper class (just as Yu imitates Yue Fei), attempting to obtain an entry ticket to the class via the synchronisation of cultural symbols. The second is defensive indifference. Deep down, they lack security and view the world as a zero-sum game. Consequently, all ‘sentiments’ have a stop-loss limit; once personal safety (interests) is touched, the survival instinct immediately commandeers the brain, and empathy for others resets to zero.&lt;/p&gt;
&lt;p&gt;The pervasive existence of this personality in modern society leads to a systemic collapse of trust. People appear to be enthusiastically exchanging visions, while in reality, everyone is clutching a dagger in the shadows, ready to cut the rope at any moment.&lt;/p&gt;
&lt;h2 id=&quot;conclusion-the-triumph-of-nihilism&quot;&gt;Conclusion: The Triumph of Nihilism&lt;/h2&gt;
&lt;p&gt;Yu Xiaoqing-style figures, and the countless ‘micro-Yu Xiaoqings’ in modern society, appear to be the victors of rationality. They enjoy the reputational halo brought by idealism while harvesting the tangible fruits of realism.&lt;/p&gt;
&lt;p&gt;However, from the perspective of existentialist philosophy, this is a state of utter nihilism. They live within layers of lies and masks, ultimately losing the ability to perceive the real world. They deceive not only the audience but eventually themselves. When one becomes accustomed to viewing everyone around them (including comrades, family, and subordinates) as ‘fuel’ for their objectives, they effectively imprison themselves in a labyrinth composed of mirrors – where there is no true ‘Other’, merely infinite projections of a self consumed by desire and fear.&lt;/p&gt;
&lt;p&gt;This ‘sophisticated egoism’ may be tactically invincible, but strategically, it is a devastating blow to the foundation of humanity. It manufactures a world that is magnificent yet frigid, where all sublimity is a synonym for calculation, and behind every tear-filled eye lies a cold, calculating abacus.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>Killing Processes in K8s Pods</title>
    <link href="https://www.ailven.cc/posts/devops-251212.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/devops-251212.html</id>
    <published>2025-12-12T00:00:00Z</published>
    <updated>2026-09-30T00:00:00Z</updated>
    <summary>In Kubernetes pods, processes may appear in ps -ef output but fail to respond to kill commands. This occurs when the target process runs in a different PID namespace. The solution uses nsenter to enter the target namespace and send signals using the container-local PID.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 12 December 2025&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;A &lt;code&gt;kill -9&lt;/code&gt; that answers &lt;code&gt;No such process&lt;/code&gt; for a PID that &lt;code&gt;ps&lt;/code&gt; has just printed is one of the more confusing failures a container will hand you. The process plainly exists and is plainly visible. The signal simply never arrives. Almost every instance of this comes down to a PID namespace boundary between the shell issuing the signal and the process meant to receive it.&lt;/p&gt;
&lt;h2 id=&quot;problem&quot;&gt;Problem&lt;/h2&gt;
&lt;p&gt;Process visible in &lt;code&gt;ps -ef&lt;/code&gt; but cannot be killed:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;$ ps -ef | grep &amp;lt;process&amp;gt;
root  3563929  ...  /bin/bash ./script.sh

$ kill -9 3563929
-bash: kill: (3563929) - No such process
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;root-cause&quot;&gt;Root cause&lt;/h2&gt;
&lt;p&gt;The target runs in a different PID namespace from the shell. Namespaces partition the PID space, so a number that is valid inside one namespace does not resolve to the same process in another — or necessarily to any process at all. The kernel reports &lt;code&gt;No such process&lt;/code&gt; because, from the shell&amp;#39;s own namespace, that number really is unassigned. Nothing is wrong with the process, and nothing is wrong with &lt;code&gt;kill&lt;/code&gt;; the two are simply not in the same numbering scheme.&lt;/p&gt;
&lt;p&gt;Two commands confirm it:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# 1. Check if PID namespaces differ
readlink /proc/&amp;lt;PID&amp;gt;/ns/pid
readlink /proc/$$/ns/pid
# Different output = namespace mismatch

# 2. Find the process&amp;#39;s PID within its own namespace
cat /proc/&amp;lt;PID&amp;gt;/status | grep NSpid
# Output: NSpid: &amp;lt;host_PID&amp;gt; &amp;lt;container_PID&amp;gt;
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;&lt;code&gt;NSpid&lt;/code&gt; lists the process&amp;#39;s PID in each namespace it belongs to, outermost first. The second number is the one a &lt;code&gt;kill&lt;/code&gt; issued from inside the container would accept.&lt;/p&gt;
&lt;h2 id=&quot;solution&quot;&gt;Solution&lt;/h2&gt;
&lt;p&gt;Use &lt;code&gt;nsenter&lt;/code&gt; to enter the target&amp;#39;s namespace and signal it with the container-local PID:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# Get container PID from NSpid (second number)
CONTAINER_PID=$(cat /proc/&amp;lt;HOST_PID&amp;gt;/status | grep NSpid | awk &amp;#39;{print $3}&amp;#39;)

# Enter namespace and kill
nsenter -t &amp;lt;HOST_PID&amp;gt; -p -m kill -9 $CONTAINER_PID
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;For a process whose &lt;code&gt;NSpid&lt;/code&gt; reads &lt;code&gt;3563929 5984&lt;/code&gt;, that becomes:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;nsenter -t 3563929 -p -m kill -9 5984
&lt;/code&gt;&lt;/pre&gt;
&lt;h2 id=&quot;generalising-the-fix&quot;&gt;Generalising the fix&lt;/h2&gt;
&lt;p&gt;Three cases account for most unkillable processes: the process is already gone, it is a zombie, or its namespace differs from yours. The script below tests them in that order and dispatches accordingly.&lt;/p&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Issue&lt;/th&gt;
&lt;th&gt;Diagnosis&lt;/th&gt;
&lt;th&gt;Solution&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td&gt;PID namespace mismatch&lt;/td&gt;
&lt;td&gt;Different namespace IDs&lt;/td&gt;
&lt;td&gt;&lt;code&gt;nsenter -t &amp;lt;PID&amp;gt; -p -m kill &amp;lt;container_PID&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Zombie process&lt;/td&gt;
&lt;td&gt;State = &lt;code&gt;Z&lt;/code&gt; in &lt;code&gt;ps aux&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Kill parent: &lt;code&gt;kill -9 &amp;lt;PPid&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Process already terminated&lt;/td&gt;
&lt;td&gt;&lt;code&gt;/proc/&amp;lt;PID&amp;gt;&lt;/code&gt; missing&lt;/td&gt;
&lt;td&gt;No action needed&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;#!/bin/bash
TARGET_PID=$1

# Check if process exists
[ ! -d &amp;quot;/proc/$TARGET_PID&amp;quot; ] &amp;amp;&amp;amp; echo &amp;quot;Process not found&amp;quot; &amp;amp;&amp;amp; exit 1

# Check if zombie process
STATE=$(grep &amp;quot;^State:&amp;quot; /proc/$TARGET_PID/status | awk &amp;#39;{print $2}&amp;#39;)
if [ &amp;quot;$STATE&amp;quot; = &amp;quot;Z&amp;quot; ]; then
    PPID=$(grep &amp;quot;^PPid:&amp;quot; /proc/$TARGET_PID/status | awk &amp;#39;{print $2}&amp;#39;)
    echo &amp;quot;Zombie process - killing parent: $PPID&amp;quot;
    kill -9 $PPID
    exit 0
fi

# Check PID namespace
PROC_NS=$(readlink /proc/$TARGET_PID/ns/pid)
CURR_NS=$(readlink /proc/$$/ns/pid)

if [ &amp;quot;$PROC_NS&amp;quot; != &amp;quot;$CURR_NS&amp;quot; ]; then
    echo &amp;quot;PID namespace mismatch - using nsenter&amp;quot;
    CONTAINER_PID=$(grep &amp;quot;^NSpid:&amp;quot; /proc/$TARGET_PID/status | awk &amp;#39;{print $3}&amp;#39;)
    nsenter -t $TARGET_PID -p -m kill -9 $CONTAINER_PID
else
    kill -9 $TARGET_PID
fi
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;Where the pod is reachable, two simpler routes exist:&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-bash&quot;&gt;# From within the pod
kubectl exec -it &amp;lt;pod-name&amp;gt; -- kill &amp;lt;PID&amp;gt;

# Terminate the entire pod
kubectl delete pod &amp;lt;pod-name&amp;gt; --force --grace-period=0
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The first is preferable when one process is at fault, since it keeps the rest of the pod alive. The second takes every process in the pod with it and belongs at the end of the list, not the start.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>On Conditional Recognition and Existential Seeing: The Economics of Love</title>
    <link href="https://www.ailven.cc/posts/moment-251203.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/moment-251203.html</id>
    <published>2025-12-03T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>An exploration of how intimate relationships become arenas of social discipline, where love is conditioned on conformity to externally imposed success metrics – and what it means to resist this through existential seeing.</summary>
    <content type="html">&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 3 December 2025&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;In our exploration of life attitudes and intimate relationships, a sharp and pervasive dilemma emerges: those we love deeply and trust most – our partners – often implicitly set the benchmarks of a socially recognised successful life, characterised by achievement, development, foresight, and perpetual progression, as the unspoken terms of the relationship. Their departure, prompted by our perceived failure to meet these expectations, is experienced not merely as an emotional loss but as a profound negation.&lt;/p&gt;
&lt;p&gt;As previously analysed, &lt;strong&gt;the essence of such departure is tantamount to a verdict of ‘social death sentence’.&lt;/strong&gt; Its core message is: ‘Your value as a partner or loved one is deeply contingent upon your alignment with our metrics of social success. Your “failure to meet the standard” renders you unworthy of our continued investment and companionship.’ Behind this verdict, the identity of ‘they’ comes into focus: &lt;strong&gt;they are not holders of truth, but conduits of collective fear.&lt;/strong&gt; Their demands (‘you must have achievement, development, and vision’) are rooted in survival anxiety and status anxiety; their love often manifests as a stark &lt;strong&gt;‘conditional’ mode&lt;/strong&gt; – loving you for your bright prospects, your role as a source of pride, your fit for their imagination of an ‘excellent life’; once these conditions dissolve, the love is withdrawn.&lt;/p&gt;
&lt;p&gt;This preliminary conclusion strikes at the heart of the pain but does not yet reach the philosophical and sociological core of the issue. To truly understand the deep structure of this relational tragedy, we must introduce and examine a pair of key concepts: &lt;strong&gt;‘Conditional Recognition’ and ‘Existential Seeing’.&lt;/strong&gt; The conflict between them constitutes one of the most fundamental tensions in contemporary intimate relationships.&lt;/p&gt;
&lt;hr&gt;
&lt;h4 id=&quot;i-conditional-recognition-the-internalisation-of-social-discipline-and-the-economics-of-love&quot;&gt;&lt;strong&gt;I. Conditional Recognition: The Internalisation of Social Discipline and the Economics of Love&lt;/strong&gt;&lt;/h4&gt;
&lt;p&gt;‘Conditional Recognition’ refers to the approval, acceptance, and love an individual receives being strictly dependent on their conformity to a set of externally imposed, measurable social value standards. Within intimate relationships, it manifests when a partner equates your ‘value of being’ with your ‘functional value’.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Its Essence is the Capillarisation of Social Discipline:&lt;/strong&gt; Society, through systems like family, education, and media, continuously disseminates a script for a ‘successful life’: stable career advancement, prominent social status, expanding wealth and influence. This script is essentially an instrument of discipline that maintains social structure and competitive order. When a partner internalises this script as the sole criterion for choosing and maintaining a relationship, social discipline achieves its most subtle and potent infiltration – it invades the most private sphere of intimacy from the public domain, transforming what should be a sanctuary into another arena of performance review. Demanding that a partner ‘continuously improve and climb higher’ is often unrelated to the individual’s genuine happiness or potential; it is, unconsciously, a demand for the other to become a bulwark against one’s own survival anxiety and a buoy for one’s own social standing.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Relationship as Value Exchange: A Stark ‘Economics of Love’:&lt;/strong&gt; From this perspective, intimate relationships shed their emotional mystique, revealing their rational, calculative nature as &lt;strong&gt;‘value exchange’.&lt;/strong&gt; This enters the realm of the ‘economics of love’ or ‘psycho-economics’. The individual is viewed as an ‘asset’ holding specific capital (e.g., prospects, vision, social reputation), and the relationship as an ‘investment’ expecting future returns (e.g., security, social respect, class stability). When one party judges the other’s capital to be depreciating or failing to appreciate – that is, when the expected return no longer covers the emotional ‘opportunity cost’ – ‘divestment’ (leaving) becomes the choice aligned with their psycho-economic rationality. This is not a moral condemnation but a revelation of a widely operative, unconscious logic. The statement ‘you have not met the standard’ translates to: ‘Your valuation has fallen below my psychological threshold; I can no longer derive sufficient emotional and social profit from this relationship.’&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Manifestation and Consequence: The Withdrawal of Recognition is the Negation of Being:&lt;/strong&gt; The manifestation of ‘Conditional Recognition’ in relationships extends far beyond the decisive moment of departure. It permeates daily life: anxiety over ‘stagnation’, shame about ‘mediocrity’, the alienation of shared life into mutual ‘progress projects’. Its deepest injury lies in making the evaluated individual feel that: &lt;strong&gt;the essence of my ‘I’, my very being, is meaningless and unworthy of love unless it is attached to those quantifiable achievements.&lt;/strong&gt; This withdrawal of recognition is, therefore, a negation at the existential level. It echoes the philosopher Hegel’s insight that recognition is a condition for the existence of self-consciousness. When the closest person grants recognition only conditionally, the individual is plunged into a profound ‘crisis of recognition’.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h4 id=&quot;ii-existential-seeing-resistance-to-discipline-and-ontological-affirmation&quot;&gt;&lt;strong&gt;II. Existential Seeing: Resistance to Discipline and Ontological Affirmation&lt;/strong&gt;&lt;/h4&gt;
&lt;p&gt;In opposition to ‘Conditional Recognition’ stands &lt;strong&gt;‘Existential Seeing’&lt;/strong&gt;. This is not a romantic fantasy but a more fundamental, albeit more demanding, relational stance.&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Its Essence is Ontological Affirmation Stripped of Social Indicators:&lt;/strong&gt; ‘Existential Seeing’ means that I see you, I love you, not for your function, your performance, your position on the social ladder, but &lt;strong&gt;for your &lt;em&gt;‘Dasein’&lt;/em&gt; – your unique existence as a ‘being-there’.&lt;/strong&gt; I see your joys and sorrows, your vulnerability and courage, your unique way of perceiving and engaging with the world – those aspects that cannot be captured in a CV or measured by KPIs. This is a gaze that, stripped of all social exchange value, is almost ‘luxurious’. It affirms: ‘Your existence &lt;em&gt;itself&lt;/em&gt; is meaningful and worthy of companionship.’ This seeing constitutes the most fundamental resistance to the social discipline that instrumentalises human beings.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Relationship as Practice of ‘Existential Resistance’:&lt;/strong&gt; In a society where everyone is caught in a performance race, practising ‘Existential Seeing’ is itself an act of &lt;strong&gt;‘existential resistance’.&lt;/strong&gt; It resists the trend of instrumentalising all relationships, including the most intimate ones. It requires both parties to have the courage to suspend their internal fears – fear of falling behind, fear of others’ judgements, fear of an uncertain future – and, within this suspended space, to encounter the authentic other, stripped of all social armour. This demands significant spiritual strength and independence, for it implies the necessity of co-creating, to some degree, a ‘relational enclave’ not wholly dominated by external value systems.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Manifestation and Dilemma: Scarcity and Misinterpretation:&lt;/strong&gt; In reality, ‘Existential Seeing’ is often dismissed as ‘unrealistic’ or ‘indulgent’. It is easily mistaken for a passive laissez-faire attitude that ignores any need for the other’s growth. However, genuine ‘Existential Seeing’ is nothing of the sort. It may equally encourage growth and exploration, but its &lt;strong&gt;motivation and foundation are entirely different&lt;/strong&gt;: not stemming from fear (of being left behind) but from curiosity and trust in the other’s life potential; not demanding the other reach a pre-set ‘successful’ destination but accompanying the other on their own path towards becoming a more complete self. Its dilemma lies in its extreme scarcity within a world thoroughly colonised by the logic of ‘Conditional Recognition’, and its high demand for the level of conscious awareness in both individuals involved.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;hr&gt;
&lt;h4 id=&quot;conclusion-the-inevitable-rupture-and-the-liberation-of-clarity&quot;&gt;&lt;strong&gt;Conclusion: The Inevitable Rupture and the Liberation of Clarity&lt;/strong&gt;&lt;/h4&gt;
&lt;p&gt;Returning to the initial predicament: when those we love deeply leave us for ‘failing to meet the standard’, what we encounter is the &lt;strong&gt;ultimate crushing of the need for ‘Existential Seeing’ by the regime of ‘Conditional Recognition’&lt;/strong&gt;.&lt;/p&gt;
&lt;p&gt;This is not a failure of private sentiment but a &lt;strong&gt;clash within the intimate arena between two value paradigms, two modes of being.&lt;/strong&gt; The partner demanding constant ascent has essentially enshrined the social fears and performance ethics they have internalised as the supreme law of love. Their departure proclaims that, before this law, your authentic being – which cannot be metricised – has been judged ‘of insufficient value’.&lt;/p&gt;
&lt;p&gt;Recognising this brings not comfort, but a form of brutal clarity and potential liberation. It allows us to see clearly:&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;What was lost was not a home, but a joint venture:&lt;/strong&gt; What we mourn so deeply may never have been a ‘home’ capable of our full existence, but rather a ‘joint venture’ based on social value exchange. The dissolution of the venture is painful, but it should not be confused with the loss of an existential foundation.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Separation is a logical inevitability:&lt;/strong&gt; When one party insists on ‘Conditional Recognition’ while the other craves or has come to need ‘Existential Seeing’, the rupture of the relationship is almost a structural necessity. The two cannot coexist long-term, for the logic of the former will inevitably erode the space for the latter.&lt;/p&gt;
&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;It points to the ultimate question for the self:&lt;/strong&gt; The final interrogation turns inwards: Can I, to some degree, first practise ‘Existential Seeing’ upon myself? Can I affirm the fundamental value of my own being, apart from the clamour of societal judgement? This is no easy task, yet it is the starting point for any genuine freedom and the possibility of building deep relationships.&lt;/p&gt;
&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;Therefore, those who left us for being ‘not accomplished enough’ play, in a broader view, the role of ruthless revealers. Through their actions, they coldly demonstrate the dominant economic laws of love in our time. And our pain and reflection may become the seed of an existential resistance – beginning with the question: ‘Am I willing to be recognised only conditionally?’ and culminating in the exploration: ‘Can I see and embrace the self and the other who, transcending all metrics, is worthy of existence &lt;em&gt;as such&lt;/em&gt;?’ This exploration offers no easy answers, but it shifts us from private regret and blame towards a public, profound reflection on our era, human nature, and the meaning of existence.&lt;/p&gt;
</content>
  </entry>
  <entry>
    <title>In-depth Analysis of the US Stock Market on 20 November 2025</title>
    <link href="https://www.ailven.cc/posts/finance-251121.html" rel="alternate" type="text/html"/>
    <id>https://www.ailven.cc/posts/finance-251121.html</id>
    <published>2025-11-21T00:00:00Z</published>
    <updated>2026-11-12T00:00:00Z</updated>
    <summary>On 20 November 2025, the US stock market experienced a dramatic open-high, close-low reversal triggered by Nvidia earnings, AI investment anxiety, and shifting Fed rate expectations – the S&amp;P 500’s most severe intraday reversal since April.</summary>
    <content type="html">&lt;p&gt;&lt;em&gt;A ‘Perfect Storm’ Triggered by AI&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Publication Date&lt;/strong&gt;: 21 November 2025&lt;br&gt;&lt;strong&gt;Author&lt;/strong&gt;: Ailven LIU 刘翔&lt;br&gt;&lt;strong&gt;All Rights Reserved&lt;/strong&gt;&lt;/p&gt;
&lt;h3 id=&quot;executive-summary&quot;&gt;Executive Summary&lt;/h3&gt;
&lt;p&gt;On 20 November 2025, the US stock market experienced a dramatic ‘open-high, close-low’ reversal. Spurred by a better-than-expected earnings report from Artificial Intelligence (AI) giant Nvidia, the market opened significantly higher, with the Dow Jones Industrial Average surging over 700 points at its peak. However, the optimism was short-lived. The market sentiment shifted abruptly in the afternoon, leading to a sharp sell-off across the board, with technology stocks leading the decline. Both the S&amp;amp;P 500 and the Nasdaq Composite hit their lowest closing levels in over two months, and the S&amp;amp;P 500 underwent its most severe intraday reversal since April. This decline was not triggered by a single factor but was a ‘perfect storm’ resulting from a combination of deep-seated anxiety about the return on AI investments, a sudden drop in expectations for a Federal Reserve rate cut, technical selling, and multiple market uncertainties.&lt;/p&gt;
&lt;h3 id=&quot;key-market-data&quot;&gt;Key Market Data&lt;/h3&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th align=&quot;left&quot;&gt;Index&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;Closing Level&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;Decline (Points)&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;Decline (%)&lt;/th&gt;
&lt;th align=&quot;left&quot;&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;&lt;strong&gt;Dow Jones Industrial Average&lt;/strong&gt;&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;45,752.26&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-386.51&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-0.84%&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Rallied by 717 points at its session high&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;&lt;strong&gt;S&amp;amp;P 500&lt;/strong&gt;&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;6,538.76&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-103.40&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-1.56%&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Lowest close since 10 September&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;&lt;strong&gt;Nasdaq Composite&lt;/strong&gt;&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;22,078.05&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-486.18&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;-2.15%&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Lowest close since 11 September&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td align=&quot;left&quot;&gt;&lt;strong&gt;VIX (Cboe Volatility Index)&lt;/strong&gt;&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;20.14&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;+3.52&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;+21.2%&lt;/td&gt;
&lt;td align=&quot;left&quot;&gt;Highest close since 24 April&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;&lt;/table&gt;
&lt;p&gt;&lt;em&gt;Source: Reuters, NBC News [1][2]&lt;/em&gt;&lt;/p&gt;
&lt;h3 id=&quot;event-panorama-from-euphoria-to-collapse&quot;&gt;Event Panorama: From Euphoria to Collapse&lt;/h3&gt;
&lt;p&gt;The market’s trajectory for the day centred on AI giant Nvidia. After the previous day’s market close, Nvidia reported third-quarter earnings that far exceeded market expectations and provided a strong sales forecast for the fourth quarter. CEO Jensen Huang expressed strong confidence in the future of AI [3]. This news significantly boosted market sentiment, driving US stocks to open sharply higher on 20 November, with the Nasdaq Composite soaring nearly 2.5% in early trading.&lt;/p&gt;
&lt;p&gt;However, the reversal began quietly in the afternoon. Nvidia’s stock, which had been up as much as 5%, took a sharp downturn, ultimately closing with a loss of over 3%. The fall of this market pillar quickly triggered a chain reaction. The entire semiconductor sector plummeted by 4.8%, and technology stocks became the epicentre of the sell-off. The intraday trading range for both the Dow and the Nasdaq exceeded 1,000 points, with the Nasdaq recording its largest intraday swing of 4.9 percentage points since the tariff turmoil in April [1]. The wave of selling spread throughout the market, leading to all three major indices closing in the red.&lt;/p&gt;
&lt;h3 id=&quot;in-depth-causal-analysis-a-confluence-of-headwinds&quot;&gt;In-depth Causal Analysis: A Confluence of Headwinds&lt;/h3&gt;
&lt;h4 id=&quot;1-the-soul-searching-of-ai-investment-profitability-and-valuation-bubbles&quot;&gt;1. The ‘Soul-Searching’ of AI Investment: Profitability and Valuation Bubbles&lt;/h4&gt;
&lt;p&gt;Despite Nvidia’s stellar earnings, the market did not fully embrace the news. Instead, it triggered deep-seated concerns about whether the massive investments in the AI sector could generate corresponding profits. This anxiety was not unfounded. Renowned investor Michael Burry, the protagonist of ‘The Big Short’, posted a series of sharp critiques on social media regarding the business models of the AI industry [4].&lt;/p&gt;
&lt;blockquote&gt;
&lt;p&gt;Burry argued that ‘just because something is used does not mean it is profitable.’ He questioned the existence of extensive ‘give-and-take deals’ among AI companies to support an artificial boom and pointed out that companies like Nvidia were embellishing their financial statements through accounting practices such as extending the depreciation periods of their assets. He even asked directly: ‘OpenAI is the linchpin here. Can anyone name their auditor?’ [4]&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;This perspective resonated with traders. Matt Maley, Chief Market Strategist at Miller Tabak, noted, ‘Can AI truly achieve the level of profitability priced in by the market? That is the key question. Traders are worried about whether today’s AI investments will be profitable in five years, so they’ve decided to take some profits off the table.’ [5] The spread of this sentiment was the direct catalyst that turned tech stocks, led by Nvidia, from heroes to villains.&lt;/p&gt;
&lt;h4 id=&quot;2-plummeting-fed-rate-cut-expectations-strong-jobs-data-backfires&quot;&gt;2. Plummeting Fed Rate Cut Expectations: Strong Jobs Data ‘Backfires’&lt;/h4&gt;
&lt;p&gt;The September non-farm payrolls report, delayed for weeks by the government shutdown, came in surprisingly strong, with 119,000 jobs added, far exceeding the expected 50,000 [1][2]. This ‘stale’ data became another straw that broke the market’s optimistic back. The robust employment figures significantly reduced market expectations for a third interest rate cut by the Federal Reserve in December.&lt;/p&gt;
&lt;p&gt;According to the CME FedWatch Tool, the probability of a December rate cut plummeted from a near-certainty of 97% in mid-October to less than 40% on 20 November [6]. Institutions like Morgan Stanley quickly adjusted their forecasts, stating that the Fed would likely pause its rate cuts in December [2]. This reversal in interest rate expectations dealt a direct blow to growth-oriented technology stocks, which rely on a low-interest-rate environment, by increasing their financing costs and reducing investors’ risk appetite.&lt;/p&gt;
&lt;h4 id=&quot;3-technical-and-structural-factors-deleveraging-and-programmatic-selling&quot;&gt;3. Technical and Structural Factors: Deleveraging and Programmatic Selling&lt;/h4&gt;
&lt;p&gt;The market’s sharp reversal was also closely linked to technical and structural factors. Chris Murphy, co-head of derivative strategy at Susquehanna International Group, pointed out that with two major catalysts – Nvidia’s earnings and Fed rate cut expectations – now in the rearview mirror, investors began to question what could drive a year-end rally. This made the net-long positions of systematic strategies like Commodity Trading Advisors (CTAs) ‘vulnerable’, potentially triggering large-scale programmatic selling once a market correction began [5].&lt;/p&gt;
&lt;p&gt;Furthermore, the market’s ‘breadth’ was poor. Even as the main indices rose in the morning, the number of declining stocks far outpaced advancing ones, indicating that the rally lacked a broad foundation. Additionally, the sharp sell-off in the cryptocurrency market exacerbated risk-off sentiment. Bitcoin’s price fell below $87,000 to a multi-month low, which was seen as a signal of deleveraging among retail investors – a group that had been a significant force behind the market’s earlier rise [2][5].&lt;/p&gt;
&lt;h4 id=&quot;4-the-aftermath-of-the-government-shutdown-uncertainty-in-a-data-black-hole&quot;&gt;4. The Aftermath of the Government Shutdown: Uncertainty in a Data ‘Black Hole’&lt;/h4&gt;
&lt;p&gt;The 43-day US federal government shutdown caused an unprecedented disruption in the release of economic data. Although the government has since reopened, key economic data for October (such as employment and inflation reports) will be combined with the November report and released after the Fed’s December meeting [7]. This means the Fed will have to make its decisions with incomplete information. This ‘data black hole’ has greatly increased market uncertainty, causing any minor disturbance to be amplified and exacerbating market volatility.&lt;/p&gt;
&lt;h3 id=&quot;conclusion&quot;&gt;Conclusion&lt;/h3&gt;
&lt;p&gt;The major reversal in the US stock market on 20 November 2025, was a classic stampede from a peak of optimistic expectations to a grim reality. Nvidia’s earnings report, which should have been a shot in the arm for the market, unexpectedly became a litmus test for investors to scrutinise the returns on AI investments. When deep-seated valuation anxieties converged with a macroeconomic reversal in interest rate expectations, technical programmatic selling, and uncertainty from a data vacuum, a violent market correction became inevitable. This serves as a warning to investors that in a market driven by a single narrative (such as AI), any wavering in the foundation of that narrative can trigger a disproportionately severe and volatile reaction.&lt;/p&gt;
&lt;h3 id=&quot;references&quot;&gt;References&lt;/h3&gt;
&lt;p&gt;[1] &lt;a href=&quot;https://www.reuters.com/business/us-stock-futures-jump-nvidia-results-ease-ai-bubble-concerns-2025-11-20/&quot;&gt;Wall Street indexes end lower after sharp reversal, tech leads drop&lt;/a&gt;&lt;br&gt;[2] &lt;a href=&quot;https://www.nbcnews.com/business/business-news/stock-market-nvidia-walmart-jobs-rcna244960&quot;&gt;Stock market sinks on AI, interest rate worries&lt;/a&gt;&lt;br&gt;[3] &lt;a href=&quot;https://finance.sina.com.cn/jjxw/2025-11-20/doc-infxzepe3899869.shtml&quot;&gt;Nvidia Earnings Call Transcript: Three Major Transformations Drive AI Infrastructure Investment, $500 Billion Revenue Target May Be Surpassed&lt;/a&gt;&lt;br&gt;[4] &lt;a href=&quot;https://www.businessinsider.com/big-short-michael-burry-nvidia-stock-earnings-ai-bubble-microchips-2025-11&quot;&gt;‘Big Short’ investor Michael Burry takes aim at Nvidia after its earnings blowout&lt;/a&gt;&lt;br&gt;[5] &lt;a href=&quot;https://www.moomoo.com/hans/news/post/61844455&quot;&gt;Nvidia’s Blowout Earnings Trigger a Major US Stock Reversal, S&amp;amp;P Marks Worst Day Since April!&lt;/a&gt;&lt;br&gt;[6] &lt;a href=&quot;https://www.cbsnews.com/news/federal-reserve-december-2025-rate-cut-probability-fomc-meeting-economy/&quot;&gt;Is the Federal Reserve likely to cut interest rates in December? Here’s what economists say.&lt;/a&gt;&lt;br&gt;[7] &lt;a href=&quot;https://www.worldjournal.com/wj/story/121172/9151502?zh-cn&quot;&gt;Affected by 43-day Government Shutdown, Department of Labor Delays Release of Employment Data&lt;/a&gt;&lt;/p&gt;
</content>
  </entry>
</feed>
