Announced 12 Oct 2026 · Sources checked
What did OpenBMB and its collaborators release?
The release has three parts. The paper, “Chaos in the Text: Revealing the Modality Preference in Mixed-Modality Retrievers”, was posted to arXiv on 8 October 2026 by Yubo Sun, Chunyi Peng, Yukun Yan, Zhenghao Liu, Zhipeng Xu, Sen Mei, Linlin Xin, Zheni Zeng and Maosong Sun, from the University of the Chinese Academy of Sciences, Northeastern University, Tsinghua University and Nanjing University. Training and evaluation code went up in the OpenBMB/Trident repository on 10 October.
On 12 October the team released the Trident-Qwen3VL-2B model alongside the Trident-Train and Trident-Eval datasets. The model is fine-tuned from Qwen3-VL-2B-Instruct with a LoRA adapter that is already merged, so no base model or PEFT library is needed. It maps text, images and combined text-image inputs into one vector space, so a single text query can rank all three kinds of document in one list.
OpenBMB is the open-source group also behind the MiniCPM models, including the on-device MiniCPM-V 4.7 released the same week. Trident is a different kind of release: a research fix for a retrieval problem rather than a general chat model.
What is “Chaos in the Text”?
Real document collections are mixed. A company knowledge base or a retrieval-augmented generation (RAG) system might hold plain text pages, scanned charts and slides with captions, and the answer to a question can sit in any of them. The retriever has to rank all of them together.
The authors tested this by taking image benchmarks such as ChartQA and SlideVQA and gradually replacing image documents with text descriptions of the same content, written by Qwen3-VL-235B-A22B. Retrieval quality followed a V shape: strong on an all-image corpus, strong on an all-text corpus, and much worse in between, when both coexist. In their headline example, mean NDCG@10 falls from 0.94 for an image-dominant corpus to 0.65 for a mixed one, recovering to 0.88 when text dominates.
They then added distractors, irrelevant documents sampled from Wikipedia, to the original image corpora. Irrelevant images had a modest effect. The same number of irrelevant text passages caused a sharp decline across Jina-CLIP-v2 and Qwen3-VL-Embedding 2B and 8B. That is the phenomenon they name “Chaos in the Text”: unrelated text crowding out the image that actually holds the answer.
Why do retrievers favour text over images?
To find the cause, the researchers built three semantically identical versions of each corpus, pure text, pure image and fused, and compared similarity scores for the same queries. Even for correct query-document pairs, text versions received systematically higher scores than image versions. For Qwen3-VL-Embedding-2B, mean positive-pair similarity was 0.664 for text against 0.484 for images.
That gap matters when modalities compete. The authors measured how often a text hard negative, an irrelevant but similar-looking passage, outscores the relevant document. When the relevant document was an image, this happened for 87.8% of queries with Qwen3-VL-Embedding-2B and 94.6% with Jina-CLIP-v2. When the relevant document was text, the rates fell to 17.8% and 25.8%.
The paper calls this modality preference and distinguishes it from the well-known modality gap, the geometric separation between image and text embeddings. A uniform gap is harmless when every candidate shares one modality, because the offset cancels out in ranking. In a mixed index, the bias directly decides which documents make the top 10.
How does Trident fix the bias?
Trident has two parts. First, modality-balanced data: for each training document the team generates a detailed text description and a short summary with Qwen3-VL-235B-A22B. The description is the text view, the original image is the image view, and the summary plus the image is the fused view. All three count as correct answers for the same query.
Second, a loss the authors call Multi-Positive View InfoNCE. Training each modality with its own softmax would not help, they show, because adding a constant offset to one modality’s scores leaves its internal ranking unchanged, so scores stay incomparable across modalities. Trident instead puts all three positive views and the in-batch negatives in one shared softmax and averages the positives’ log-probabilities.
Mathematically, the authors decompose this loss into two terms: relevance discrimination, which pushes probability onto the correct document as a whole, and positive-view balance, a KL-divergence term that stops any single view, usually text, from taking most of that probability. The effect is to raise relevant images that score too low and lower irrelevant text that scores too high.
How well does Trident-Qwen3VL-2B perform?
On six visual-document benchmarks, ChartQA, DocVQA, InfoVQA, SlideVQA, ViDoSeek and Dude, the paper reports results on each benchmark’s original image corpus and on a Mix setting that merges mixed-modality corpora from all six. Trident-Qwen3VL-2B averages 73.90 NDCG@10 on Mix, against 40.12 for Qwen3-VL-Embedding-2B, 47.17 for the four-times-larger Qwen3-VL-Embedding-8B and 61.91 for Jina-embeddings-v5-omni-small.
It also improves the original single-modality average over its own baseline, 80.51 against 77.59, though Qwen3-VL-Embedding-8B still leads there at 83.47. A CLIP-based variant, Trident-JinaCLIP, lifts Jina-CLIP-v2’s Mix average from 42.68 to 55.51. An ablation without the shared-softmax loss drops Trident-Qwen3VL-2B’s Mix score to 66.36, which the authors use to argue that both the data and the loss matter.
Usage is through the repository’s Python backend: clone the code, download the weights, and encode queries and documents, each a dictionary with optional text and a list of image paths. Scores are cosine similarities comparable across document types. Readers comparing options can set it against Google’s EmbeddingGemma 2, which also targets on-device multimodal embeddings.
Who should care, and what are the caveats?
Anyone building search over mixed collections, such as scanned reports, slide decks and web pages, should take the diagnosis seriously even before adopting the model: a retriever that looks strong on single-modality benchmarks may quietly bury image evidence once text is added. Testing a retriever on a mixed index, with text distractors, is a cheap check.
The caveats are standard for a fresh paper. All numbers are the authors’ own and have not been independently reproduced. The Mix benchmark is built by the same team, with text views generated by a large Qwen model, which may differ from human-written text in real collections. The paper’s main experiments train on OpenDocVQA’s roughly 41,000 queries, while the repository says the released checkpoint was trained on the union with about 170,000 natural-image queries, so its scores may not match Table 1 exactly.
Licensing is layered: the Trident weights and code are MIT, the Qwen3-VL-2B-Instruct base remains under Apache 2.0, and training images keep their original licences. The model card documents English only, and the paper has not yet been peer-reviewed.
Common questions
What does Trident-Qwen3VL-2B do?
It turns text, images or combined text-image documents into vectors in one shared space, so a text query can rank all of them in a single list with comparable similarity scores.
Is the improvement independently verified?
No. The benchmark results come from the authors’ own paper and evaluation suite, posted on 8 October 2026 and not yet peer-reviewed. The code and data are public, so others can reproduce them.
Can I use it commercially?
The Trident weights and code are MIT-licensed, but the Qwen3-VL-2B-Instruct base model’s Apache 2.0 terms still apply, and the training images keep their original licences.
What to remember
Trident’s main contribution is a warning backed by data: multimodal retrievers can favour text so strongly that relevant images vanish from mixed search results. If your index mixes text and images, test for that bias, and consider an approach like Trident that trains all views on one scale.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.
Our editorial standards
