278

Papers

10/10

Roadblocks Active

0

Connections

Orchard is an open-source framework for training AI agents on software engineering, GUI control, and personal assistant tasks, achieving 67.5% on SWE-bench Verified — a leading benchmark for autonomous code repair — using a combination of supervised fine-tuning on distilled trajectories and reinforcement learning with a technique called Balanced Adaptive Rollout. Critically, it introduces credit-assignment fine-tuning that learns from failed agent trajectories, not just successful ones, which addresses a long-standing waste problem in agentic training data. Because training recipes, models, and the Kubernetes-native environment are released openly, this sets a new reproducible baseline for the open-source community to build on.

██████████ 0.9 agent-tool-use Preprint

ReadSaveConnections

This paper tests what happens when frontier LLMs face medical questions where the correct answer has been deliberately removed: models still commit to a wrong answer 53–82% of the time across two medical benchmarks, rather than saying 'I don't know.' In open-ended evaluation, models gave inappropriate answers on roughly 30% of standard health questions and 78% of adversarial queries written by physicians. Safety-oriented prompting reduces but does not eliminate this behavior, meaning the problem is structural rather than fixable with simple instructions — a serious concern for any clinical deployment.

██████████ 0.9 alignment-safety Preprint

ReadSaveConnections

MemLens reveals a sharp trade-off in multimodal AI memory: models with long context windows handle short conversations well using direct visual grounding, but accuracy degrades significantly as conversation length grows, while memory-augmented agent architectures stay stable across lengths but lose visual detail due to compression. The most striking finding is that removing images from evidence drops frontier models below 2% accuracy on over 80% of visually grounded questions, proving that text summaries cannot substitute for stored visual information. This benchmark gives the field a concrete target for measuring progress on multimodal long-term memory, which current systems largely fail.

██████████ 0.9 long-context Preprint

ReadSaveConnections

MemEye introduces a 742-question benchmark specifically designed to test whether multimodal agents actually use visual evidence — or just get by on text captions and shortcuts. Testing 13 memory methods across 4 vision-language model backbones, it finds that most current architectures fail to preserve fine-grained visual details and cannot track how a scene changes over time. The benchmark's ablation gates — which test whether a question can be answered without images, without memory, or with shortcuts — make it harder to game than prior memory benchmarks, giving a cleaner signal on genuine multimodal memory capability.

██████████ 0.8 multimodal-understanding Preprint

ReadSaveConnections

Video-Zero trains video understanding models to improve themselves without any human annotations, by having a 'Questioner' agent identify temporal evidence in video clips and generate grounded questions, while a 'Solver' agent answers and is rewarded for finding the right time spans. The key insight is that naive self-improvement fails because models learn to exploit static image cues rather than actually tracking what changes over time — forcing temporal localization into the reward fixes this. Evaluated across 13 benchmarks on three model families over four evolution cycles with code publicly released, it offers a credible path to annotation-free video model improvement.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections

ATLAS introduces discrete 'functional tokens' — single vocabulary entries that represent visual operations like zooming or drawing auxiliary lines — which can act both as agentic tool calls and as internal reasoning steps within the same model, without requiring separate architectures for each. The model is trained on 178K curated visual reasoning examples followed by reinforcement learning that rewards both correct answers and valid use of these tokens, with a modified loss function to handle their sparsity. This unified approach avoids the overhead of switching between agentic and non-agentic modes and shows competitive results on challenging visual reasoning benchmarks.

██████████ 0.8 multimodal-understanding Preprint

ReadSaveConnections

When AI systems retrieve information from knowledge graphs to answer questions, they typically cite the specific nodes they used as evidence — but this paper shows that the uncited surrounding graph structure (neighboring nodes traversed but not cited) also substantially affects what answers are produced. Using controlled ablations on a 30-question benchmark, the authors demonstrate that removing cited entities changes answers and reduces accuracy, but accurate answers can also depend on graph context that was never cited at all. This means standard citation-based explainability in graph-retrieval AI systems misrepresents the actual reasoning path, creating a hidden accountability gap.

██████████ 0.8 hallucination-grounding Preprint

ReadSaveConnections

For AI robots that navigate by following language instructions, this paper identifies two distinct bottlenecks: a slow planning layer that maps semantic relationships between objects, and a fast reactive layer that uses bounding boxes for immediate movement decisions. A key finding is 'perception saturation' — improving 3D perception accuracy beyond a certain threshold yields diminishing returns on navigation success, meaning the limiting factor shifts from perception to planning quality. This gives robot AI developers a principled way to allocate engineering effort rather than assuming better perception always translates to better navigation.

██████████ 0.8 embodied-ai Preprint

ReadSaveConnections

CLVR tackles the problem of generating images from complex descriptions by coupling a language reasoning model with an image diffusion model in a closed loop, where each generated image is verified against the intended semantics before being accepted as a training signal. An automated data engine filters training trajectories using two independent judge models, reducing hallucinated or inconsistent image-text pairs in the training data. A new reinforcement learning variant called PPRL addresses instability that arises when training over long sequences of interleaved text and images, which has been a practical barrier to scaling this type of reasoning-driven generation.

██████████ 0.7 hallucination-grounding Preprint

ReadSaveConnections

SceneFunRI is a benchmark testing whether vision-language models can locate objects that are hidden or occluded in a scene when given a task description — for example, finding a door handle behind furniture. The best-performing model (Gemini Flash) achieves only 15.2% on the strictest accuracy metric, revealing a fundamental gap in current VLMs' ability to reason spatially about things they cannot directly see. The benchmark introduces a prompting strategy called Spatial Process of Elimination (SPoE) that helps models reason about where invisible objects must be based on surrounding context, providing a baseline method for future improvement.

██████████ 0.7 multimodal-understanding Preprint

ReadSaveConnections