Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Jan 26, 2026

LabelAttentionClassifier was missing validation for head configuration constraints, allowing invalid configurations that cause shape mismatches at runtime when embedding_dim is not divisible by n_head.

Changes

  • Added early validation in LabelAttentionClassifier.__init__:
    • embedding_dim must be divisible by n_head (prevents shape mismatch in c_proj layer)
    • n_head must be divisible by n_kv_head (required for Group Query Attention)
  • Follows existing TextEmbedder validation pattern for consistency
  • Error messages include integer values and remainders for clarity

Example

Invalid configurations now fail fast with clear error messages:

config = TextEmbedderConfig(
    embedding_dim=127,  # Not divisible by 8
    label_attention_config=LabelAttentionConfig(n_head=8, n_kv_head=4, num_classes=5)
)
# Raises: ValueError: embedding_dim (127) must be divisible by n_head (8). 
#         Got head_dim = 15 with remainder 7

Previously this would fail later with cryptic tensor shape errors in c_proj.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits January 26, 2026 17:34
Co-authored-by: meilame-tayebjee <114609737+meilame-tayebjee@users.noreply.github.com>
…essages

Co-authored-by: meilame-tayebjee <114609737+meilame-tayebjee@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jan 26, 2026

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • astral.sh
    • Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI changed the title [WIP] Update cross attention labels text implementation Add validation for LabelAttentionClassifier head configuration Jan 26, 2026
@meilame-tayebjee meilame-tayebjee marked this pull request as ready for review January 27, 2026 09:30
@meilame-tayebjee meilame-tayebjee merged commit d516e6b into 24-add-cross-attention-labels-text Jan 27, 2026
@meilame-tayebjee meilame-tayebjee deleted the copilot/sub-pr-60 branch January 27, 2026 09:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants