---
title: "gparse-model-attention — Mova code model"
description: "Mova code model of gparse/src/model.rs: components with their labels, obernants and the model text."
url: "https://groupants.elamur.ai/free/en/mova/gparse/src/model.rs.groupant/attention.groupant/"
---
Elamur: [Mova](https://md.mova.elamur.ai/free/en/) · [Dictionary](https://md.dictionary.elamur.ai/free/en/) · [Research](https://md.research.elamur.ai/free/en/) · **Groupants**

This page as HTML: <https://groupants.elamur.ai/free/en/mova/gparse/src/model.rs.groupant/attention.groupant/>

# gparse-model-attention

The groupant beside gparse/src/model.rs in the Mova repository: what the code computes, what it never does, how to go back from its outputs, its invariants, cost and checks.

## Facts

- **kind:** groupant_fragment
- **status:** partial source bound model
- **source:** [`gparse/src/model.rs`](https://github.com/ElamurAI/mova-free/blob/main/gparse/src/model.rs)
- **source SHA-256:** `a380c81aa11433fd…`
- **covers:** Embedding sum and pre-LN transformer dispatch attention softmax by (fused,train), then biaffine masked arcs. (bytes 8855–12658)
- **part of:** [gparse-model](https://md.groupants.elamur.ai/free/en/mova/gparse/src/model.rs.groupant/)
- **repository folder:** [`gparse/src/model.rs.groupant/attention.groupant`](https://github.com/ElamurAI/mova-free/tree/main/gparse/src/model.rs.groupant/attention.groupant)
- **repository commit:** `48f2e6f74146`

> **Implementation material.** A Mova code model describes this code at one source revision: partial models, finite checks and their stated scope. It is not a reviewed general exposition, and its labels are shown as written, never raised.

## Components

| component | label | where |
|---|---|---|
| **approximant** | partial | [approximant](#approximant) |
| **ast** | partial | `../syntax.json` |
| **cost** | symbolic source derived | [invariant](#invariant) |
| **descriptant** | present | [descriptant](#descriptant) |
| **invariant** | partial | [invariant](#invariant) |
| **tests** | partial bounded checks | [tests](#tests) |

## Obernants

The typed relations around the forward model: the behaviour complement (what the code never does), the reverse reconstruction (every input compatible with an output) and specialised views.

| id | type | label | where |
|---|---|---|---|
| `behavior_complement` | behavior_complement | partial source derived model | [behavior_complement](#behavior-complement) |
| `reverse_reconstruction` | reverse_reconstruction | partial source derived model | [reverse_reconstruction](#reverse-reconstruction) |
| `gradient_dispatch` | gradient_dispatch | partial source derived model | [gradient_dispatch](#gradient-dispatch) |
| `mask_bias` | mask_bias | partial source derived model | [mask_bias](#mask-bias) |
| `residual_state` | residual_state | partial source derived model | [residual_state](#residual-state) |
| `dimension_domain` | dimension_domain | partial source derived model | [dimension_domain](#dimension-domain) |

## Model text


### descriptant

#### Source-bound description

Embedding sum and pre-LN transformer dispatch attention softmax by (fused,train), then biaffine masked arcs.

##### Epoch 2026-10-31

Prospective model was recorded before edits. Added tests use bounded synthetic fixtures; production algorithms unchanged. Coverage is partial, not proof. No trained corpus, live teacher or throughput experiment was executed.

##### Epoch 2026-11-04

The input sum gains the subword mean (Net::subword) for models with ngram>0; the attention path itself is unchanged.

### approximant

#### Forward approximant

Embedding sum and pre-LN transformer dispatch attention softmax by (fused,train), then biaffine masked arcs.

Domain: valid source-specific dimensions and resources stated in invariant; no automatic production substitution. Numerical observations are finite checks, not an end-to-end relative bound. Executable Rust surrogate: unsupported gap; existing exact tests are not surrogates. Future metric-specific caps may be 1/2/5/10%, never above 10%.

### invariant

#### Invariant and cost

Fused training uses p*(g-dot) backward; inference fused path has no Q/K backward. Valid B,T,heads dimensions required.

##### Cost

O(B*T*d+L*(B*T*d²+B*T²*d+B*T*d*f)+B*T*d*a+B*T*a²+B*T²*a); attention probabilities occupy B*H*T², with retained graphs in training. BTd embedding/final-LN work remains at L=0.

Allocation, finite arithmetic, platform dimensions and external failures are not universally certified.

### models

## Typed model catalog

### behavior_complement

Public train fused cannot invoke upstream inference-only softmax.

### reverse_reconstruction

Categorical outputs cannot recover encoder or logits; encoder itself admits embedding-sum parameter symmetries.

### gradient_dispatch

(true,true)=custom differentiable;(true,false)=upstream fused;(false,_)=reference.

### mask_bias

keys-1 multiplied1e9 on attention/head columns; finite negative bias not hard exclusion for arbitrary huge logits.

### residual_state

Pre-LN attention and FF outputs add to h; final LN precedes head projections.

### dimension_domain

heads nonzero,d divisible heads,position/feature indices valid; no centralized config validation.

### tests

#### Checks and limits

Selected parent checks relevant to this fragment (not exhaustive): model::tests::embeddings_get_gradients; model::fused::which_ops_backprop; activation_configuration_roundtrips; CUDA gradient/GELU/synthetic optimizer checks; CPU prediction

Selected commands run twice debug and twice release; exact receipts and source-to-test mapping retained in wave30 gparse external run directory. Tests do not certify allocation failure, all NaN/overflow inputs, trained-model accuracy or corpus timing. Compiled mutants cover selected boundaries; per-source coverage is recorded in the external map, not presumed.