748 lines
221 KiB
XML
748 lines
221 KiB
XML
<?xml version='1.0' encoding='UTF-8'?>
|
||
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" version="2.0">
|
||
<channel>
|
||
<title>Hugging Face Daily Papers</title>
|
||
<link>https://huggingface.co/papers</link>
|
||
<description>Daily research papers curated by the Hugging Face community.</description>
|
||
<docs>http://www.rssboard.org/rss-specification</docs>
|
||
<generator>python-feedgen</generator>
|
||
<language>en</language>
|
||
<lastBuildDate>Mon, 27 Oct 2025 00:12:18 +0000</lastBuildDate>
|
||
<item>
|
||
<title>Emergent Misalignment via In-Context Learning: Narrow in-context examples can produce broadly misaligned LLMs</title>
|
||
<link>https://arxiv.org/abs/2510.11288</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.11288.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Nikita Afonin, Nikita Andriyanov, Nikhil Bageshpura, Kyle Liu, Kevin Zhu, Sunishchal Dev, Ashwinee Panda, Alexander Panchenko, Oleg Rogov, Elena Tutubalina, Mikhail Seleznyov</p><p><b>Upvotes:</b> 44</p><p><b>Summary:</b> Recent work has shown that narrow finetuning can produce broadly misaligned LLMs, a phenomenon termed emergent misalignment (EM). While concerning, these findings were limited to finetuning and activation steering, leaving out in-context learning (ICL). We therefore ask: does EM emerge in ICL? We find that it does: across three datasets, three frontier models produce broadly misaligned responses at rates between 2% and 17% given 64 narrow in-context examples, and up to 58% with 256 examples. We also examine mechanisms of EM by eliciting step-by-step reasoning (while leaving in-context examples unchanged). Manual analysis of the resulting chain-of-thought shows that 67.5% of misaligned traces explicitly rationalize harmful outputs by adopting a reckless or dangerous ''persona'', echoing prior results on finetuning-induced EM.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.11288</guid>
|
||
<pubDate>Mon, 13 Oct 2025 11:23:56 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>A^2FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning</title>
|
||
<link>https://arxiv.org/abs/2510.12838</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.12838.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Qianben Chen, Jingyi Cao, Jiayu Zhang, Tianrui Qin, Xiaowan Li, King Zhu, Dingfeng Shi, He Zhu, Minghao Liu, Xiaobo Liang, Xin Gui, Ge Zhang, Jian Yang, Yuchen Eleanor Jiang, Wangchunshu Zhou</p><p><b>Upvotes:</b> 22</p><p><b>Summary:</b> Large language models split into two families: reasoning-centric LLMs, which strengthen internal chain-of-thought reasoning but cannot invoke external tools, and agentic LLMs, which learn to interact with environments and leverage tools but often lag in deep reasoning. This divide arises from fundamentally different training objectives, leading to mismatched strengths and inefficiency on simple queries, where both families tend to overthink or over-call tools. In this work, we present Adaptive Agent Foundation Model (A^2FM), a unified framework that follows a route-then-align principle: the model first learns task-aware routing and then aligns mode-specific trajectories under a shared backbone. To address the inefficiency gap, we introduce a third mode-instant-that handles simple queries directly, preventing unnecessary reasoning or tool calls while complementing the agentic and reasoning modes. To jointly enhance accuracy and efficiency, we propose Adaptive Policy Optimization (APO), which enforces adaptive sampling across modes and applies a cost-regularized reward. On the 32B scale, A^2FM achieves 13.4% on BrowseComp, 70.4% on AIME25, and 16.7% on HLE, setting new SOTA among comparable models and performing competitively with frontier LLMs across agentic, reasoning, and general benchmarks. Notably, the adaptive execution achieves a cost of pass of only $0.00487 per correct answer-cutting cost by 45.2% relative to reasoning and 33.5% relative to agentic, thus delivering substantially higher cost efficiency while maintaining comparable accuracy.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.12838</guid>
|
||
<pubDate>Mon, 13 Oct 2025 17:08:25 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Diff-XYZ: A Benchmark for Evaluating Diff Understanding</title>
|
||
<link>https://arxiv.org/abs/2510.12487</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.12487.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Evgeniy Glukhov, Michele Conti, Egor Bogomolov, Yaroslav Golubev, Alexander Bezzubov</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Reliable handling of code diffs is central to agents that edit and refactor repositories at scale. We introduce Diff-XYZ, a compact benchmark for code-diff understanding with three supervised tasks: apply (old code + diff rightarrow new code), anti-apply (new code - diff rightarrow old code), and diff generation (new code - old code rightarrow diff). Instances in the benchmark are triples langle old code, new code, diff rangle drawn from real commits in CommitPackFT, paired with automatic metrics and a clear evaluation protocol. We use the benchmark to do a focused empirical study of the unified diff format and run a cross-format comparison of different diff representations. Our findings reveal that different formats should be used depending on the use case and model size. For example, representing diffs in search-replace format is good for larger models in the diff generation scenario, yet not suited well for diff analysis and smaller models. The Diff-XYZ benchmark is a reusable foundation for assessing and improving diff handling in LLMs that can aid future development of diff formats and models editing code. The dataset is published on HuggingFace Hub: https://huggingface.co/datasets/JetBrains-Research/diff-xyz.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.12487</guid>
|
||
<pubDate>Tue, 14 Oct 2025 13:23:01 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Language Models Model Language</title>
|
||
<link>https://arxiv.org/abs/2510.12766</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.12766.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Łukasz Borchmann</p><p><b>Upvotes:</b> 23</p><p><b>Summary:</b> Linguistic commentary on LLMs, heavily influenced by the theoretical frameworks of de Saussure and Chomsky, is often speculative and unproductive. Critics challenge whether LLMs can legitimately model language, citing the need for "deep structure" or "grounding" to achieve an idealized linguistic "competence." We argue for a radical shift in perspective towards the empiricist principles of Witold Ma\'nczak, a prominent general and historical linguist. He defines language not as a "system of signs" or a "computational system of the brain" but as the totality of all that is said and written. Above all, he identifies frequency of use of particular language elements as language's primary governing principle. Using his framework, we challenge prior critiques of LLMs and provide a constructive guide for designing, evaluating, and interpreting language models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.12766</guid>
|
||
<pubDate>Tue, 14 Oct 2025 17:45:31 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>CiteGuard: Faithful Citation Attribution for LLMs via Retrieval-Augmented Validation</title>
|
||
<link>https://arxiv.org/abs/2510.17853</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17853.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yee Man Choi, Xuehang Guo, Yi R., Fung, Qingyun Wang</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Large Language Models (LLMs) have emerged as promising assistants for scientific writing. However, there have been concerns regarding the quality and reliability of the generated text, one of which is the citation accuracy and faithfulness. While most recent work relies on methods such as LLM-as-a-Judge, the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we reframe citation evaluation as a problem of citation attribution alignment, which is assessing whether LLM-generated citations match those a human author would include for the same text. We propose CiteGuard, a retrieval-aware agent framework designed to provide more faithful grounding for citation validation. CiteGuard improves the prior baseline by 12.3%, and achieves up to 65.4% accuracy on the CiteME benchmark, on par with human-level performance (69.7%). It also enables the identification of alternative but valid citations.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17853</guid>
|
||
<pubDate>Wed, 15 Oct 2025 00:32:26 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>AlphaQuanter: An End-to-End Tool-Orchestrated Agentic Reinforcement Learning Framework for Stock Trading</title>
|
||
<link>https://arxiv.org/abs/2510.14264</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.14264.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zheye Deng, Jiashu Wang</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> While Large Language Model (LLM) agents show promise in automated trading, they still face critical limitations. Prominent multi-agent frameworks often suffer from inefficiency, produce inconsistent signals, and lack the end-to-end optimization required to learn a coherent strategy from market feedback. To address this, we introduce AlphaQuanter, a single-agent framework that uses reinforcement learning (RL) to learn a dynamic policy over a transparent, tool-augmented decision workflow, which empowers a single agent to autonomously orchestrate tools and proactively acquire information on demand, establishing a transparent and auditable reasoning process. Extensive experiments demonstrate that AlphaQuanter achieves state-of-the-art performance on key financial metrics. Moreover, its interpretable reasoning reveals sophisticated strategies, offering novel and valuable insights for human traders. Our code for data acquisition and agent training is publicly available at: https://github.com/AlphaQuanter/AlphaQuanter</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.14264</guid>
|
||
<pubDate>Thu, 16 Oct 2025 03:30:22 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning</title>
|
||
<link>https://arxiv.org/abs/2510.14265</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.14265.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xukai Wang, Xuanbo Liu, Mingrui Chen, Haitian Zhong, Xuanlin Yang, Bohan Zeng, Jinbo Hu, Hao Liang, Junbo Niu, Xuchen Li, Ruitao Wu, Ruichuan An, Yang Shi, Liu Liu, Xu-Yao Zhang, Qiang Liu, Zhouchen Lin, Wentao Zhang, Bin Dong</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> With the advancement of powerful large-scale reasoning models, effectively evaluating the reasoning capabilities of these models has become increasingly important. However, existing benchmarks designed to assess the reasoning abilities of large models tend to be limited in scope and lack the flexibility to adapt their difficulty according to the evolving reasoning capacities of the models. To address this, we propose MorphoBench, a benchmark that incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models. Specifically, we curate the benchmark by selecting and collecting complex reasoning questions from existing benchmarks and sources such as Olympiad-level competitions. Additionally, MorphoBench adaptively modifies the analytical challenge of questions by leveraging key statements generated during the model's reasoning process. Furthermore, it includes questions generated using simulation software, enabling dynamic adjustment of benchmark difficulty with minimal resource consumption. We have gathered over 1,300 test questions and iteratively adjusted the difficulty of MorphoBench based on the reasoning capabilities of models such as o3 and GPT-5. MorphoBench enhances the comprehensiveness and validity of model reasoning evaluation, providing reliable guidance for improving both the reasoning abilities and scientific robustness of large models. The code has been released in https://github.com/OpenDCAI/MorphoBench.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.14265</guid>
|
||
<pubDate>Thu, 16 Oct 2025 03:30:56 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Explore to Evolve: Scaling Evolved Aggregation Logic via Proactive Online Exploration for Deep Research Agents</title>
|
||
<link>https://arxiv.org/abs/2510.14438</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.14438.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Rui Wang, Ce Zhang, Jun-Yu Ma, Jianshu Zhang, Hongru Wang, Yi Chen, Boyang Xue, Tianqing Fang, Zhisong Zhang, Hongming Zhang, Haitao Mi, Dong Yu, Kam-Fai Wong</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> Deep research web agents not only retrieve information from diverse sources such as web environments, files, and multimodal inputs, but more importantly, they need to rigorously analyze and aggregate knowledge for insightful research. However, existing open-source deep research agents predominantly focus on enhancing information-seeking capabilities of web agents to locate specific information, while overlooking the essential need for information aggregation, which would limit their ability to support in-depth research. We propose an Explore to Evolve paradigm to scalably construct verifiable training data for web agents. Begins with proactive online exploration, an agent sources grounded information by exploring the real web. Using the collected evidence, the agent then self-evolves an aggregation program by selecting, composing, and refining operations from 12 high-level logical types to synthesize a verifiable QA pair. This evolution from high-level guidance to concrete operations allowed us to scalably produce WebAggregatorQA, a dataset of 10K samples across 50K websites and 11 domains. Based on an open-source agent framework, SmolAgents, we collect supervised fine-tuning trajectories to develop a series of foundation models, WebAggregator. WebAggregator-8B matches the performance of GPT-4.1, while the 32B variant surpasses GPT-4.1 by more than 10% on GAIA-text and closely approaches Claude-3.7-sonnet. Moreover, given the limited availability of benchmarks that evaluate web agents' information aggregation abilities, we construct a human-annotated evaluation split of WebAggregatorQA as a challenging test set. On this benchmark, Claude-3.7-sonnet only achieves 28%, and GPT-4.1 scores 25.8%. Even when agents manage to retrieve all references, they still struggle on WebAggregatorQA, highlighting the need to strengthen the information aggregation capabilities of web agent foundations.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.14438</guid>
|
||
<pubDate>Thu, 16 Oct 2025 08:37:42 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DeLeaker: Dynamic Inference-Time Reweighting For Semantic Leakage Mitigation in Text-to-Image Models</title>
|
||
<link>https://arxiv.org/abs/2510.15015</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15015.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Mor Ventura, Michael Toker, Or Patashnik, Yonatan Belinkov, Roi Reichart</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> Text-to-Image (T2I) models have advanced rapidly, yet they remain vulnerable to semantic leakage, the unintended transfer of semantically related features between distinct entities. Existing mitigation strategies are often optimization-based or dependent on external inputs. We introduce DeLeaker, a lightweight, optimization-free inference-time approach that mitigates leakage by directly intervening on the model's attention maps. Throughout the diffusion process, DeLeaker dynamically reweights attention maps to suppress excessive cross-entity interactions while strengthening the identity of each entity. To support systematic evaluation, we introduce SLIM (Semantic Leakage in IMages), the first dataset dedicated to semantic leakage, comprising 1,130 human-verified samples spanning diverse scenarios, together with a novel automatic evaluation framework. Experiments demonstrate that DeLeaker consistently outperforms all baselines, even when they are provided with external information, achieving effective leakage mitigation without compromising fidelity or quality. These results underscore the value of attention control and pave the way for more semantically precise T2I models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15015</guid>
|
||
<pubDate>Thu, 16 Oct 2025 17:39:21 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>NANO3D: A Training-Free Approach for Efficient 3D Editing Without Masks</title>
|
||
<link>https://arxiv.org/abs/2510.15019</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15019.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Junliang Ye, Shenghao Xie, Ruowen Zhao, Zhengyi Wang, Hongyu Yan, Wenqiang Zu, Lei Ma, Jun Zhu</p><p><b>Upvotes:</b> 55</p><p><b>Summary:</b> 3D object editing is essential for interactive content creation in gaming, animation, and robotics, yet current approaches remain inefficient, inconsistent, and often fail to preserve unedited regions. Most methods rely on editing multi-view renderings followed by reconstruction, which introduces artifacts and limits practicality. To address these challenges, we propose Nano3D, a training-free framework for precise and coherent 3D object editing without masks. Nano3D integrates FlowEdit into TRELLIS to perform localized edits guided by front-view renderings, and further introduces region-aware merging strategies, Voxel/Slat-Merge, which adaptively preserve structural fidelity by ensuring consistency between edited and unedited areas. Experiments demonstrate that Nano3D achieves superior 3D consistency and visual quality compared with existing methods. Based on this framework, we construct the first large-scale 3D editing datasets Nano3D-Edit-100k, which contains over 100,000 high-quality 3D editing pairs. This work addresses long-standing challenges in both algorithm design and data availability, significantly improving the generality and reliability of 3D editing, and laying the groundwork for the development of feed-forward 3D editing models. Project Page:https://jamesyjl.github.io/Nano3D</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15019</guid>
|
||
<pubDate>Thu, 16 Oct 2025 17:51:50 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Directional Reasoning Injection for Fine-Tuning MLLMs</title>
|
||
<link>https://arxiv.org/abs/2510.15050</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15050.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Chao Huang, Zeliang Zhang, Jiang Liu, Ximeng Sun, Jialian Wu, Xiaodong Yu, Ze Wang, Chenliang Xu, Emad Barsoum, Zicheng Liu</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> Multimodal large language models (MLLMs) are rapidly advancing, yet their reasoning ability often lags behind that of strong text-only counterparts. Existing methods to bridge this gap rely on supervised fine-tuning over large-scale multimodal reasoning data or reinforcement learning, both of which are resource-intensive. A promising alternative is model merging, which interpolates parameters between reasoning-enhanced LLMs and multimodal variants. However, our analysis shows that naive merging is not always a "free lunch": its effectiveness varies drastically across model families, with some (e.g., LLaVA, Idefics) benefiting while others (e.g., Qwen) suffer performance degradation. To address this, we propose Directional Reasoning Injection for Fine-Tuning (DRIFT) MLLMs, a lightweight method that transfers reasoning knowledge in the gradient space, without destabilizing multimodal alignment. DRIFT precomputes a reasoning prior as the parameter-space difference between reasoning and multimodal variants, then uses it to bias gradients during multimodal fine-tuning. This approach preserves the simplicity of standard supervised fine-tuning pipelines while enabling efficient reasoning transfer. Extensive experiments on multimodal reasoning benchmarks, including MathVista and MathVerse, demonstrate that DRIFT consistently improves reasoning performance over naive merging and supervised fine-tuning, while matching or surpassing training-heavy methods at a fraction of the cost.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15050</guid>
|
||
<pubDate>Thu, 16 Oct 2025 18:06:46 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learning</title>
|
||
<link>https://arxiv.org/abs/2510.15110</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15110.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang-Ting Cheng, Yejin Choi, Jan Kautz, Pavlo Molchanov</p><p><b>Upvotes:</b> 15</p><p><b>Summary:</b> Reasoning language models such as OpenAI-o1, DeepSeek-R1, and Qwen achieve strong performance via extended chains of thought but often generate unnecessarily long outputs. Maximizing intelligence per token--accuracy relative to response length--remains an open problem. We revisit reinforcement learning (RL) with the simplest length penalty--truncation--and show that accuracy degradation arises not from the lack of sophisticated penalties but from inadequate RL optimization. We identify three key challenges: (i) large bias in advantage estimation, (ii) entropy collapse, and (iii) sparse reward signal. We address them with Doing Length pEnalty Right (DLER), a training recipe combining batch-wise reward normalization, higher clipping, dynamic sampling, and a simple truncation length penalty. DLER achieves state-of-the-art accuracy--efficiency trade-offs, cutting output length by over 70 percent while surpassing all previous baseline accuracy. It also improves test-time scaling: compared to DeepSeek-R1-7B, DLER-7B generates multiple concise responses in parallel with 28 percent higher accuracy and lower latency. We further introduce Difficulty-Aware DLER, which adaptively tightens truncation on easier questions for additional efficiency gains. We also propose an update-selective merging method that preserves baseline accuracy while retaining the concise reasoning ability of the DLER model, which is useful for scenarios where RL training data is scarce.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15110</guid>
|
||
<pubDate>Thu, 16 Oct 2025 20:05:57 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transition</title>
|
||
<link>https://arxiv.org/abs/2510.15280</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15280.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Fan Liu, Jindong Han, Tengfei Lyu, Weijia Zhang, Zhe-Rui Yang, Lu Dai, Cancheng Liu, Hao Liu</p><p><b>Upvotes:</b> 14</p><p><b>Summary:</b> Foundation models (FMs), such as GPT-4 and AlphaFold, are reshaping the landscape of scientific research. Beyond accelerating tasks such as hypothesis generation, experimental design, and result interpretation, they prompt a more fundamental question: Are FMs merely enhancing existing scientific methodologies, or are they redefining the way science is conducted? In this paper, we argue that FMs are catalyzing a transition toward a new scientific paradigm. We introduce a three-stage framework to describe this evolution: (1) Meta-Scientific Integration, where FMs enhance workflows within traditional paradigms; (2) Hybrid Human-AI Co-Creation, where FMs become active collaborators in problem formulation, reasoning, and discovery; and (3) Autonomous Scientific Discovery, where FMs operate as independent agents capable of generating new scientific knowledge with minimal human intervention. Through this lens, we review current applications and emerging capabilities of FMs across existing scientific paradigms. We further identify risks and future directions for FM-enabled scientific discovery. This position paper aims to support the scientific community in understanding the transformative role of FMs and to foster reflection on the future of scientific discovery. Our project is available at https://github.com/usail-hkust/Awesome-Foundation-Models-for-Scientific-Discovery.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15280</guid>
|
||
<pubDate>Fri, 17 Oct 2025 03:40:26 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Latent Diffusion Model without Variational Autoencoder</title>
|
||
<link>https://arxiv.org/abs/2510.15301</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15301.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Minglei Shi, Haolin Wang, Wenzhao Zheng, Ziyang Yuan, Xiaoshi Wu, Xintao Wang, Pengfei Wan, Jie Zhou, Jiwen Lu</p><p><b>Upvotes:</b> 43</p><p><b>Summary:</b> Recent progress in diffusion-based visual generation has largely relied on latent diffusion models with variational autoencoders (VAEs). While effective for high-fidelity synthesis, this VAE+diffusion paradigm suffers from limited training efficiency, slow inference, and poor transferability to broader vision tasks. These issues stem from a key limitation of VAE latent spaces: the lack of clear semantic separation and strong discriminative structure. Our analysis confirms that these properties are crucial not only for perception and understanding tasks, but also for the stable and efficient training of latent diffusion models. Motivated by this insight, we introduce SVG, a novel latent diffusion model without variational autoencoders, which leverages self-supervised representations for visual generation. SVG constructs a feature space with clear semantic discriminability by leveraging frozen DINO features, while a lightweight residual branch captures fine-grained details for high-fidelity reconstruction. Diffusion models are trained directly on this semantically structured latent space to facilitate more efficient learning. As a result, SVG enables accelerated diffusion training, supports few-step sampling, and improves generative quality. Experimental results further show that SVG preserves the semantic and discriminative capabilities of the underlying self-supervised representations, providing a principled pathway toward task-general, high-quality visual representations.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15301</guid>
|
||
<pubDate>Fri, 17 Oct 2025 04:17:44 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>When to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensembling</title>
|
||
<link>https://arxiv.org/abs/2510.15346</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15346.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Heecheol Yun, Kwangmin Ki, Junghyun Lee, Eunho Yang</p><p><b>Upvotes:</b> 31</p><p><b>Summary:</b> Ensembling Large Language Models (LLMs) has gained attention as a promising approach to surpass the performance of individual models by leveraging their complementary strengths. In particular, aggregating models' next-token probability distributions to select the next token has been shown to be effective in various tasks. However, while successful for short-form answers, its application to long-form generation remains underexplored. In this paper, we show that using existing ensemble methods in long-form generation requires a careful choice of ensembling positions, since the standard practice of ensembling at every token often degrades performance. We identify two key factors for determining these positions: tokenization mismatch across models and consensus in their next-token probability distributions. Based on this, we propose SAFE, (Stable And Fast LLM Ensembling), a framework that selectively ensembles by jointly considering these factors. To further improve stability, we introduce a probability sharpening strategy that consolidates probabilities spread across multiple sub-word tokens representing the same word into a single representative token. Our experiments on diverse benchmarks, including MATH500 and BBH, demonstrate that SAFE outperforms existing methods in both accuracy and efficiency, with gains achieved even when ensembling fewer than 1% of tokens.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15346</guid>
|
||
<pubDate>Fri, 17 Oct 2025 06:18:29 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning</title>
|
||
<link>https://arxiv.org/abs/2510.15444</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15444.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhi Zhou, Yuhao Tan, Zenan Li, Yuan Yao, Lan-Zhe Guo, Yu-Feng Li, Xiaoxing Ma</p><p><b>Upvotes:</b> 137</p><p><b>Summary:</b> Test-time scaling seeks to improve the reasoning performance of large language models (LLMs) by adding computational resources. A prevalent approach within the field is sampling-based test-time scaling methods, which enhance reasoning by generating multiple reasoning paths for a given input during inference. However, despite its practical success, the theoretical foundations remain underexplored. In this paper, we provide the first theoretical framework for analyzing sampling-based test-time scaling methods, grounded in the perspective of confidence estimation. Based on the framework, we analyze two dominant paradigms: self-consistency and perplexity, and reveal key limitations: self-consistency suffers from high estimation error while perplexity exhibits substantial modeling error and possible degradation of the estimation error convergence. To address these limitations, we introduce RPC, a hybrid method that leverages our theoretical insights through two key components: Perplexity Consistency and Reasoning Pruning. Perplexity Consistency combines the strengths of self-consistency and perplexity, boosting the convergence rate of estimation error from linear to exponential while preserving model error. Reasoning Pruning prevents degradation by eliminating low-probability reasoning paths. Both theoretical analysis and empirical results across seven benchmark datasets demonstrate that RPC has a strong potential for reducing reasoning error. Notably, RPC achieves reasoning performance comparable to self-consistency while not only enhancing confidence reliability but also reducing sampling costs by 50%. The code and resources are available at https://wnjxyk.github.io/RPC.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15444</guid>
|
||
<pubDate>Fri, 17 Oct 2025 08:59:30 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Language Models are Injective and Hence Invertible</title>
|
||
<link>https://arxiv.org/abs/2510.15511</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15511.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Giorgos Nikolaou, Tommaso Mencattini, Donato Crisostomi, Andrea Santilli, Yannis Panagakis, Emanuele Rodola'</p><p><b>Upvotes:</b> 48</p><p><b>Summary:</b> Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model's representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous representations are injective and therefore lossless, a property established at initialization and preserved during training. Second, we confirm this result empirically through billions of collision tests on six state-of-the-art language models, and observe no collisions. Third, we operationalize injectivity: we introduce SipIt, the first algorithm that provably and efficiently reconstructs the exact input text from hidden activations, establishing linear-time guarantees and demonstrating exact invertibility in practice. Overall, our work establishes injectivity as a fundamental and exploitable property of language models, with direct implications for transparency, interpretability, and safe deployment.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15511</guid>
|
||
<pubDate>Fri, 17 Oct 2025 10:25:30 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Imaginarium: Vision-guided High-Quality 3D Scene Layout Generation</title>
|
||
<link>https://arxiv.org/abs/2510.15564</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15564.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiaoming Zhu, Xu Huang, Qinghongbing Xie, Zhi Deng, Junsheng Yu, Yirui Guan, Zhongyuan Liu, Lin Zhu, Qijun Zhao, Ligang Liu, Long Zeng</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Generating artistic and coherent 3D scene layouts is crucial in digital content creation. Traditional optimization-based methods are often constrained by cumbersome manual rules, while deep generative models face challenges in producing content with richness and diversity. Furthermore, approaches that utilize large language models frequently lack robustness and fail to accurately capture complex spatial relationships. To address these challenges, this paper presents a novel vision-guided 3D layout generation system. We first construct a high-quality asset library containing 2,037 scene assets and 147 3D scene layouts. Subsequently, we employ an image generation model to expand prompt representations into images, fine-tuning it to align with our asset library. We then develop a robust image parsing module to recover the 3D layout of scenes based on visual semantics and geometric information. Finally, we optimize the scene layout using scene graphs and overall visual semantics to ensure logical coherence and alignment with the images. Extensive user testing demonstrates that our algorithm significantly outperforms existing methods in terms of layout richness and quality. The code and dataset will be available at https://github.com/HiHiAllen/Imaginarium.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15564</guid>
|
||
<pubDate>Fri, 17 Oct 2025 11:48:08 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Build Your Personalized Research Group: A Multiagent Framework for Continual and Interactive Science Automation</title>
|
||
<link>https://arxiv.org/abs/2510.15624</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15624.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ed Li, Junyu Ren, Xintian Pan, Cat Yan, Chuanhao Li, Dirk Bergemann, Zhuoran Yang</p><p><b>Upvotes:</b> 14</p><p><b>Summary:</b> The automation of scientific discovery represents a critical milestone in Artificial Intelligence (AI) research. However, existing agentic systems for science suffer from two fundamental limitations: rigid, pre-programmed workflows that cannot adapt to intermediate findings, and inadequate context management that hinders long-horizon research. We present freephdlabor, an open-source multiagent framework featuring fully dynamic workflows determined by real-time agent reasoning and a \textit{modular architecture} enabling seamless customization -- users can modify, add, or remove agents to address domain-specific requirements. The framework provides comprehensive infrastructure including automatic context compaction, workspace-based communication to prevent information degradation, memory persistence across sessions, and non-blocking human intervention mechanisms. These features collectively transform automated research from isolated, single-run attempts into continual research programs that build systematically on prior explorations and incorporate human feedback. By providing both the architectural principles and practical implementation for building customizable co-scientist systems, this work aims to facilitate broader adoption of automated research across scientific domains, enabling practitioners to deploy interactive multiagent systems that autonomously conduct end-to-end research -- from ideation through experimentation to publication-ready manuscripts.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15624</guid>
|
||
<pubDate>Fri, 17 Oct 2025 13:13:32 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Attention Sinks in Diffusion Language Models</title>
|
||
<link>https://arxiv.org/abs/2510.15731</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15731.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Maximo Eduardo Rulli, Simone Petruzzi, Edoardo Michielon, Fabrizio Silvestri, Simone Scardapane, Alessio Devoto</p><p><b>Upvotes:</b> 44</p><p><b>Summary:</b> Masked Diffusion Language Models (DLMs) have recently emerged as a promising alternative to traditional Autoregressive Models (ARMs). DLMs employ transformer encoders with bidirectional attention, enabling parallel token generation while maintaining competitive performance. Although their efficiency and effectiveness have been extensively studied, the internal mechanisms that govern DLMs remain largely unexplored. In this work, we conduct an empirical analysis of DLM attention patterns, focusing on the attention sinking phenomenon, an effect previously observed in various transformer-based architectures. Our findings reveal that DLMs also exhibit attention sinks, but with distinct characteristics. First, unlike in ARMs, the sink positions in DLMs tend to shift throughout the generation process, displaying a dynamic behaviour. Second, while ARMs are highly sensitive to the removal of attention sinks, DLMs remain robust: masking sinks leads to only a minor degradation in performance. These results provide new insights into the inner workings of diffusion-based language models and highlight fundamental differences in how they allocate and utilize attention compared to autoregressive models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15731</guid>
|
||
<pubDate>Fri, 17 Oct 2025 15:23:58 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Scaling Instruction-Based Video Editing with a High-Quality Synthetic Dataset</title>
|
||
<link>https://arxiv.org/abs/2510.15742</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15742.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Qingyan Bai, Qiuyu Wang, Hao Ouyang, Yue Yu, Hanlin Wang, Wen Wang, Ka Leong Cheng, Shuailei Ma, Yanhong Zeng, Zichen Liu, Yinghao Xu, Yujun Shen, Qifeng Chen</p><p><b>Upvotes:</b> 48</p><p><b>Summary:</b> Instruction-based video editing promises to democratize content creation, yet its progress is severely hampered by the scarcity of large-scale, high-quality training data. We introduce Ditto, a holistic framework designed to tackle this fundamental challenge. At its heart, Ditto features a novel data generation pipeline that fuses the creative diversity of a leading image editor with an in-context video generator, overcoming the limited scope of existing models. To make this process viable, our framework resolves the prohibitive cost-quality trade-off by employing an efficient, distilled model architecture augmented by a temporal enhancer, which simultaneously reduces computational overhead and improves temporal coherence. Finally, to achieve full scalability, this entire pipeline is driven by an intelligent agent that crafts diverse instructions and rigorously filters the output, ensuring quality control at scale. Using this framework, we invested over 12,000 GPU-days to build Ditto-1M, a new dataset of one million high-fidelity video editing examples. We trained our model, Editto, on Ditto-1M with a curriculum learning strategy. The results demonstrate superior instruction-following ability and establish a new state-of-the-art in instruction-based video editing.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15742</guid>
|
||
<pubDate>Fri, 17 Oct 2025 15:31:40 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Chronos-2: From Univariate to Universal Forecasting</title>
|
||
<link>https://arxiv.org/abs/2510.15821</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15821.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Abdul Fatir Ansari, Oleksandr Shchur, Jaris Küken, Andreas Auer, Boran Han, Pedro Mercado, Syama Sundar Rangapuram, Huibin Shen, Lorenzo Stella, Xiyuan Zhang, Mononito Goswami, Shubham Kapoor, Danielle C. Maddix, Pablo Guerron, Tony Hu, Junming Yin, Nick Erickson, Prateek Mutalik Desai, Hao Wang, Huzefa Rangwala, George Karypis, Yuyang Wang, Michael Bohlke-Schneider</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> Pretrained time series models have enabled inference-only forecasting systems that produce accurate predictions without task-specific training. However, existing approaches largely focus on univariate forecasting, limiting their applicability in real-world scenarios where multivariate data and covariates play a crucial role. We present Chronos-2, a pretrained model capable of handling univariate, multivariate, and covariate-informed forecasting tasks in a zero-shot manner. Chronos-2 employs a group attention mechanism that facilitates in-context learning (ICL) through efficient information sharing across multiple time series within a group, which may represent sets of related series, variates of a multivariate series, or targets and covariates in a forecasting task. These general capabilities are achieved through training on synthetic datasets that impose diverse multivariate structures on univariate series. Chronos-2 delivers state-of-the-art performance across three comprehensive benchmarks: fev-bench, GIFT-Eval, and Chronos Benchmark II. On fev-bench, which emphasizes multivariate and covariate-informed forecasting, Chronos-2's universal ICL capabilities lead to substantial improvements over existing models. On tasks involving covariates, it consistently outperforms baselines by a wide margin. Case studies in the energy and retail domains further highlight its practical advantages. The in-context learning capabilities of Chronos-2 establish it as a general-purpose forecasting model that can be used "as is" in real-world forecasting pipelines.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15821</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:00:53 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>VISTA: A Test-Time Self-Improving Video Generation Agent</title>
|
||
<link>https://arxiv.org/abs/2510.15831</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15831.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Do Xuan Long, Xingchen Wan, Hootan Nakhost, Chen-Yu Lee, Tomas Pfister, Sercan Ö. Arık</p><p><b>Upvotes:</b> 18</p><p><b>Summary:</b> Despite rapid advances in text-to-video synthesis, generated video quality remains critically dependent on precise user prompts. Existing test-time optimization methods, successful in other domains, struggle with the multi-faceted nature of video. In this work, we introduce VISTA (Video Iterative Self-improvemenT Agent), a novel multi-agent system that autonomously improves video generation through refining prompts in an iterative loop. VISTA first decomposes a user idea into a structured temporal plan. After generation, the best video is identified through a robust pairwise tournament. This winning video is then critiqued by a trio of specialized agents focusing on visual, audio, and contextual fidelity. Finally, a reasoning agent synthesizes this feedback to introspectively rewrite and enhance the prompt for the next generation cycle. Experiments on single- and multi-scene video generation scenarios show that while prior methods yield inconsistent gains, VISTA consistently improves video quality and alignment with user intent, achieving up to 60% pairwise win rate against state-of-the-art baselines. Human evaluators concur, preferring VISTA outputs in 66.4% of comparisons.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15831</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:12:08 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Paper2Web: Let's Make Your Paper Alive!</title>
|
||
<link>https://arxiv.org/abs/2510.15842</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15842.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuhang Chen, Tianpeng Lv, Siyi Zhang, Yixiang Yin, Yao Wan, Philip S. Yu, Dongping Chen</p><p><b>Upvotes:</b> 24</p><p><b>Summary:</b> Academic project websites can more effectively disseminate research when they clearly present core content and enable intuitive navigation and interaction. However, current approaches such as direct Large Language Model (LLM) generation, templates, or direct HTML conversion struggle to produce layout-aware, interactive sites, and a comprehensive evaluation suite for this task has been lacking. In this paper, we introduce Paper2Web, a benchmark dataset and multi-dimensional evaluation framework for assessing academic webpage generation. It incorporates rule-based metrics like Connectivity, Completeness and human-verified LLM-as-a-Judge (covering interactivity, aesthetics, and informativeness), and PaperQuiz, which measures paper-level knowledge retention. We further present PWAgent, an autonomous pipeline that converts scientific papers into interactive and multimedia-rich academic homepages. The agent iteratively refines both content and layout through MCP tools that enhance emphasis, balance, and presentation quality. Our experiments show that PWAgent consistently outperforms end-to-end baselines like template-based webpages and arXiv/alphaXiv versions by a large margin while maintaining low cost, achieving the Pareto-front in academic webpage generation.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15842</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:35:58 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>BLIP3o-NEXT: Next Frontier of Native Image Generation</title>
|
||
<link>https://arxiv.org/abs/2510.15857</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15857.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiuhai Chen, Le Xue, Zhiyang Xu, Xichen Pan, Shusheng Yang, Can Qin, An Yan, Honglu Zhou, Zeyuan Chen, Lifu Huang, Tianyi Zhou, Junnan Li, Silvio Savarese, Caiming Xiong, Ran Xu</p><p><b>Upvotes:</b> 21</p><p><b>Summary:</b> We present BLIP3o-NEXT, a fully open-source foundation model in the BLIP3 series that advances the next frontier of native image generation. BLIP3o-NEXT unifies text-to-image generation and image editing within a single architecture, demonstrating strong image generation and image editing capabilities. In developing the state-of-the-art native image generation model, we identify four key insights: (1) Most architectural choices yield comparable performance; an architecture can be deemed effective provided it scales efficiently and supports fast inference; (2) The successful application of reinforcement learning can further push the frontier of native image generation; (3) Image editing still remains a challenging task, yet instruction following and the consistency between generated and reference images can be significantly enhanced through post-training and data engine; (4) Data quality and scale continue to be decisive factors that determine the upper bound of model performance. Building upon these insights, BLIP3o-NEXT leverages an Autoregressive + Diffusion architecture in which an autoregressive model first generates discrete image tokens conditioned on multimodal inputs, whose hidden states are then used as conditioning signals for a diffusion model to generate high-fidelity images. This architecture integrates the reasoning strength and instruction following of autoregressive models with the fine-detail rendering ability of diffusion models, achieving a new level of coherence and realism. Extensive evaluations of various text-to-image and image-editing benchmarks show that BLIP3o-NEXT achieves superior performance over existing models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15857</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:50:58 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training</title>
|
||
<link>https://arxiv.org/abs/2510.15859</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15859.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Pengkai Wang, Qi Zuo, Pengwei Liu, Zhijie Sang, Congkai Xie, Hongxia Yang</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> Large Language Models (LLMs) have shown substantial advances through reinforcement learning (RL), particularly in domains where rewards can be programmatically verified, such as mathematics and code. In these areas, models benefit from a well-defined operational base guided by explicit rule-based objectives. However, this progress reveals a significant limitation: in open-ended domains where rewards are ambiguous, subjective, or context-dependent, such as creative writing, scientific reasoning, and notably medical consultation, robust reward functions are lacking, making these areas challenging for current RL strategies. To bridge this gap, we introduce ORBIT, an open-ended rubric-based incremental training framework specifically designed for high-stakes medical dialogue. ORBIT integrates syn- thetic dialogue generation with the dynamic creation of rubrics, employing these rubrics to direct an incremental RL process. In particular, this approach does not depend on external medical knowledge or manual rules, instead utilizing rubric-guided feedback to shape learning. When implemented on the Qwen3-4B-Instruct model, our method can greatly enhance its performance on the HealthBench-Hard benchmark from 7.0 to 27.2 using only 2k samples, thus achieving state-of-the-art results for models of this scale. Our analysis confirms that rubric-driven RL fos-ters consistent performance gains across diverse consultation scenarios, going beyond simple numerical improvements. These findings underscore rubric-based feedback as a scalable strategy for advancing LLMs in intricate, open-ended tasks.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15859</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:51:28 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>LightsOut: Diffusion-based Outpainting for Enhanced Lens Flare Removal</title>
|
||
<link>https://arxiv.org/abs/2510.15868</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15868.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shr-Ruei Tsai, Wei-Cheng Chang, Jie-Ying Lee, Chih-Hai Su, Yu-Lun Liu</p><p><b>Upvotes:</b> 23</p><p><b>Summary:</b> Lens flare significantly degrades image quality, impacting critical computer vision tasks like object detection and autonomous driving. Recent Single Image Flare Removal (SIFR) methods perform poorly when off-frame light sources are incomplete or absent. We propose LightsOut, a diffusion-based outpainting framework tailored to enhance SIFR by reconstructing off-frame light sources. Our method leverages a multitask regression module and LoRA fine-tuned diffusion model to ensure realistic and physically consistent outpainting results. Comprehensive experiments demonstrate LightsOut consistently boosts the performance of existing SIFR methods across challenging scenarios without additional retraining, serving as a universally applicable plug-and-play preprocessing solution. Project page: https://ray-1026.github.io/lightsout/</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15868</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:59:50 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery</title>
|
||
<link>https://arxiv.org/abs/2510.15869</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15869.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jie-Ying Lee, Yi-Ruei Liu, Shr-Ruei Tsai, Wei-Cheng Chang, Chung-Ho Wu, Jiewen Chan, Zhenjun Zhao, Chieh Hubert Lin, Yu-Lun Liu</p><p><b>Upvotes:</b> 40</p><p><b>Summary:</b> Synthesizing large-scale, explorable, and geometrically accurate 3D urban scenes is a challenging yet valuable task in providing immersive and embodied applications. The challenges lie in the lack of large-scale and high-quality real-world 3D scans for training generalizable generative models. In this paper, we take an alternative route to create large-scale 3D scenes by synergizing the readily available satellite imagery that supplies realistic coarse geometry and the open-domain diffusion model for creating high-quality close-up appearances. We propose Skyfall-GS, the first city-block scale 3D scene creation framework without costly 3D annotations, also featuring real-time, immersive 3D exploration. We tailor a curriculum-driven iterative refinement strategy to progressively enhance geometric completeness and photorealistic textures. Extensive experiments demonstrate that Skyfall-GS provides improved cross-view consistent geometry and more realistic textures compared to state-of-the-art approaches. Project page: https://skyfall-gs.jayinnn.dev/</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15869</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:59:51 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM</title>
|
||
<link>https://arxiv.org/abs/2510.15870</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.15870.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, Yuming Lou, Dong Yang, Zhijian Liu, Yukang Chen, Ambrish Dantrey, Ehsan Jahangiri, Sreyan Ghosh, Daguang Xu, Ehsan Hosseini-Asl, Danial Mohseni Taheri, Vidya Murali, Sifei Liu, Jason Lu, Oluwatobi Olabiyi, Frank Wang, Rafael Valle, Bryan Catanzaro, Andrew Tao, Song Han, Jan Kautz, Hongxu Yin, Pavlo Molchanov</p><p><b>Upvotes:</b> 73</p><p><b>Summary:</b> Advancing machine intelligence requires developing the ability to perceive across multiple modalities, much as humans sense the world. We introduce OmniVinci, an initiative to build a strong, open-source, omni-modal LLM. We carefully study the design choices across model architecture and data curation. For model architecture, we present three key innovations: (i) OmniAlignNet for strengthening alignment between vision and audio embeddings in a shared omni-modal latent space; (ii) Temporal Embedding Grouping for capturing relative temporal alignment between vision and audio signals; and (iii) Constrained Rotary Time Embedding for encoding absolute temporal information in omni-modal embeddings. We introduce a curation and synthesis pipeline that generates 24M single-modal and omni-modal conversations. We find that modalities reinforce one another in both perception and reasoning. Our model, OmniVinci, outperforms Qwen2.5-Omni with +19.05 on DailyOmni (cross-modal understanding), +1.7 on MMAR (audio), and +3.9 on Video-MME (vision), while using just 0.2T training tokens - a 6 times reduction compared to Qwen2.5-Omni's 1.2T. We finally demonstrate omni-modal advantages in downstream applications spanning robotics, medical AI, and smart factory.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.15870</guid>
|
||
<pubDate>Fri, 17 Oct 2025 17:59:59 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Embody 3D: A Large-scale Multimodal Motion and Behavior Dataset</title>
|
||
<link>https://arxiv.org/abs/2510.16258</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16258.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Claire McLean, Makenzie Meendering, Tristan Swartz, Orri Gabbay, Alexandra Olsen, Rachel Jacobs, Nicholas Rosen, Philippe de Bree, Tony Garcia, Gadsden Merrill, Jake Sandakly, Julia Buffalini, Neham Jain, Steven Krenn, Moneish Kumar, Dejan Markovic, Evonne Ng, Fabian Prada, Andrew Saba, Siwei Zhang, Vasu Agrawal, Tim Godisart, Alexander Richard, Michael Zollhoefer</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> The Codec Avatars Lab at Meta introduces Embody 3D, a multimodal dataset of 500 individual hours of 3D motion data from 439 participants collected in a multi-camera collection stage, amounting to over 54 million frames of tracked 3D motion. The dataset features a wide range of single-person motion data, including prompted motions, hand gestures, and locomotion; as well as multi-person behavioral and conversational data like discussions, conversations in different emotional states, collaborative activities, and co-living scenarios in an apartment-like space. We provide tracked human motion including hand tracking and body shape, text annotations, and a separate audio track for each participant.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16258</guid>
|
||
<pubDate>Fri, 17 Oct 2025 23:06:36 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>RL makes MLLMs see better than SFT</title>
|
||
<link>https://arxiv.org/abs/2510.16333</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16333.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Junha Song, Sangdoo Yun, Dongyoon Han, Jaegul Choo, Byeongho Heo</p><p><b>Upvotes:</b> 43</p><p><b>Summary:</b> A dominant assumption in Multimodal Language Model (MLLM) research is that its performance is largely inherited from the LLM backbone, given its immense parameter scale and remarkable capabilities. This has created a void in the understanding of the vision encoder, which determines how MLLMs perceive images. The recent shift in MLLM training paradigms, from Supervised Finetuning (SFT) to Reinforcement Learning (RL), magnifies this oversight-namely, the significant lack of analysis on how such training reshapes the vision encoder as well as the MLLM. To address this, we first investigate the impact of training strategies on MLLMs, where RL shows a clear advantage over SFT in strongly vision-related VQA benchmarks. Motivated by this, we conduct a critical yet under-explored analysis of the vision encoder of MLLMs through diverse and in-depth experiments, ranging from ImageNet classification and segmentation to gradient visualization. Our results demonstrate that MLLM's post-training strategy (i.e., SFT or RL) not only leads to distinct outcomes on MLLM downstream tasks, but also fundamentally reshapes MLLM's underlying visual representations. Specifically, the key finding of our study is that RL produces stronger and precisely localized visual representations compared to SFT, boosting the ability of the vision encoder for MLLM. We then reframe our findings into a simple recipe for building strong vision encoders for MLLMs, Preference-Instructed Vision OpTimization (PIVOT). When integrated into MLLMs, a PIVOT-trained vision encoder outperforms even larger and more heavily-trained counterparts, despite requiring less than 1% of the computational cost of standard vision pretraining. This result opens an effective and efficient path for advancing the vision backbones of MLLMs. Project page available at https://june-page.github.io/pivot/</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16333</guid>
|
||
<pubDate>Sat, 18 Oct 2025 03:37:17 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling</title>
|
||
<link>https://arxiv.org/abs/2510.16751</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16751.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Erik Riise, Mehmet Onurcan Kaya, Dim P. Papadopoulos</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> While inference-time scaling through search has revolutionized Large Language Models, translating these gains to image generation has proven difficult. Recent attempts to apply search strategies to continuous diffusion models show limited benefits, with simple random sampling often performing best. We demonstrate that the discrete, sequential nature of visual autoregressive models enables effective search for image generation. We show that beam search substantially improves text-to-image generation, enabling a 2B parameter autoregressive model to outperform a 12B parameter diffusion model across benchmarks. Systematic ablations show that this advantage comes from the discrete token space, which allows early pruning and computational reuse, and our verifier analysis highlights trade-offs between speed and reasoning capability. These findings suggest that model architecture, not just scale, is critical for inference-time optimization in visual generation.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16751</guid>
|
||
<pubDate>Sun, 19 Oct 2025 08:28:06 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>FinSight: Towards Real-World Financial Deep Research</title>
|
||
<link>https://arxiv.org/abs/2510.16844</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16844.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiajie Jin, Yuyao Zhang, Yimeng Xu, Hongjin Qian, Yutao Zhu, Zhicheng Dou</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Generating professional financial reports is a labor-intensive and intellectually demanding process that current AI systems struggle to fully automate. To address this challenge, we introduce FinSight (Financial InSight), a novel multi agent framework for producing high-quality, multimodal financial reports. The foundation of FinSight is the Code Agent with Variable Memory (CAVM) architecture, which unifies external data, designed tools, and agents into a programmable variable space, enabling flexible data collection, analysis and report generation through executable code. To ensure professional-grade visualization, we propose an Iterative Vision-Enhanced Mechanism that progressively refines raw visual outputs into polished financial charts. Furthermore, a two stage Writing Framework expands concise Chain-of-Analysis segments into coherent, citation-aware, and multimodal reports, ensuring both analytical depth and structural consistency. Experiments on various company and industry-level tasks demonstrate that FinSight significantly outperforms all baselines, including leading deep research systems in terms of factual accuracy, analytical depth, and presentation quality, demonstrating a clear path toward generating reports that approach human-expert quality.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16844</guid>
|
||
<pubDate>Sun, 19 Oct 2025 14:05:35 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DeepAnalyze: Agentic Large Language Models for Autonomous Data Science</title>
|
||
<link>https://arxiv.org/abs/2510.16872</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16872.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, Xiaoyong Du</p><p><b>Upvotes:</b> 83</p><p><b>Summary:</b> Autonomous data science, from raw data sources to analyst-grade deep research reports, has been a long-standing challenge, and is now becoming feasible with the emergence of powerful large language models (LLMs). Recent workflow-based data agents have shown promising results on specific data tasks but remain fundamentally limited in achieving fully autonomous data science due to their reliance on predefined workflows. In this paper, we introduce DeepAnalyze-8B, the first agentic LLM designed for autonomous data science, capable of automatically completing the end-toend pipeline from data sources to analyst-grade deep research reports. To tackle high-complexity data science tasks, we propose a curriculum-based agentic training paradigm that emulates the learning trajectory of human data scientists, enabling LLMs to progressively acquire and integrate multiple capabilities in real-world environments. We also introduce a data-grounded trajectory synthesis framework that constructs high-quality training data. Through agentic training, DeepAnalyze learns to perform a broad spectrum of data tasks, ranging from data question answering and specialized analytical tasks to open-ended data research. Experiments demonstrate that, with only 8B parameters, DeepAnalyze outperforms previous workflow-based agents built on most advanced proprietary LLMs. The model, code, and training data of DeepAnalyze are open-sourced, paving the way toward autonomous data science.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16872</guid>
|
||
<pubDate>Sun, 19 Oct 2025 15:13:42 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Chem-R: Learning to Reason as a Chemist</title>
|
||
<link>https://arxiv.org/abs/2510.16880</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16880.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Weida Wang, Benteng Chen, Di Zhang, Wanhao Liu, Shuchen Pu, Ben Gao, Jin Zeng, Lei Bai, Wanli Ouyang, Xiaoyong Wei, Tianshu Yu, Tianfan Fu, Shuzhou Sun, Jiatong Li, Zifu Wang, Yuqiang Li, Shufei Zhang</p><p><b>Upvotes:</b> 50</p><p><b>Summary:</b> Although large language models (LLMs) have significant potential to advance chemical discovery, current LLMs lack core chemical knowledge, produce unreliable reasoning trajectories, and exhibit suboptimal performance across diverse chemical tasks. To address these challenges, we propose Chem-R, a generalizable Chemical Reasoning model designed to emulate the deliberative processes of chemists. Chem-R is trained through a three-phase framework that progressively builds advanced reasoning capabilities, including: 1) Chemical Foundation Training, which establishes core chemical knowledge. 2) Chemical Reasoning Protocol Distillation, incorporating structured, expert-like reasoning traces to guide systematic and reliable problem solving. 3) Multi-task Group Relative Policy Optimization that optimizes the model for balanced performance across diverse molecular- and reaction-level tasks. This structured pipeline enables Chem-R to achieve state-of-the-art performance on comprehensive benchmarks, surpassing leading large language models, including Gemini-2.5-Pro and DeepSeek-R1, by up to 46% on molecular tasks and 66% on reaction tasks. Meanwhile, Chem-R also consistently outperforms the existing chemical foundation models across both molecular and reaction level tasks. These results highlight Chem-R's robust generalization, interpretability, and potential as a foundation for next-generation AI-driven chemical discovery.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16880</guid>
|
||
<pubDate>Sun, 19 Oct 2025 15:27:13 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Uniworld-V2: Reinforce Image Editing with Diffusion Negative-aware Finetuning and MLLM Implicit Feedback</title>
|
||
<link>https://arxiv.org/abs/2510.16888</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16888.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zongjian Li, Zheyuan Liu, Qihui Zhang, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Yang Ye, Wangbo Yu, Yuwei Niu, Li Yuan</p><p><b>Upvotes:</b> 18</p><p><b>Summary:</b> Instruction-based image editing has achieved remarkable progress; however, models solely trained via supervised fine-tuning often overfit to annotated patterns, hindering their ability to explore and generalize beyond training distributions. To this end, we introduce Edit-R1, a novel post-training framework for instruction-based image editing based on policy optimization. Specifically, we utilize Diffusion Negative-aware Finetuning (DiffusionNFT), a likelihood-free policy optimization method consistent with the flow matching forward process, thereby enabling the use of higher-order samplers and more efficient training. Another key challenge here is the absence of a universal reward model, resulting from the diverse nature of editing instructions and tasks. To bridge this gap, we employ a Multimodal Large Language Model (MLLM) as a unified, training-free reward model, leveraging its output logits to provide fine-grained feedback. Furthermore, we carefully design a low-variance group filtering mechanism to reduce MLLM scoring noise and stabilize optimization. UniWorld-V2, trained with this framework, achieves state-of-the-art results on the ImgEdit and GEdit-Bench benchmarks, scoring 4.49 and 7.83, respectively. Crucially, our framework is model-agnostic, delivering substantial performance gains when applied to diverse base models like Qwen-Image-Edit and FLUX-Kontext, demonstrating its wide applicability. Code and models are publicly available at https://github.com/PKU-YuanGroup/UniWorld-V2.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16888</guid>
|
||
<pubDate>Sun, 19 Oct 2025 15:38:06 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variations</title>
|
||
<link>https://arxiv.org/abs/2510.16893</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16893.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Bo-Han Feng, Chien-Feng Liu, Yu-Hsuan Li Liang, Chih-Kai Yang, Szu-Wei Fu, Zhehuai Chen, Ke-Han Lu, Sung-Feng Huang, Chao-Han Huck Yang, Yu-Chiang Frank Wang, Yun-Nung Chen, Hung-yi Lee</p><p><b>Upvotes:</b> 16</p><p><b>Summary:</b> Large audio-language models (LALMs) extend text-based LLMs with auditory understanding, offering new opportunities for multimodal applications. While their perception, reasoning, and task performance have been widely studied, their safety alignment under paralinguistic variation remains underexplored. This work systematically investigates the role of speaker emotion. We construct a dataset of malicious speech instructions expressed across multiple emotions and intensities, and evaluate several state-of-the-art LALMs. Our results reveal substantial safety inconsistencies: different emotions elicit varying levels of unsafe responses, and the effect of intensity is non-monotonic, with medium expressions often posing the greatest risk. These findings highlight an overlooked vulnerability in LALMs and call for alignment strategies explicitly designed to ensure robustness under emotional variation, a prerequisite for trustworthy deployment in real-world settings.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16893</guid>
|
||
<pubDate>Sun, 19 Oct 2025 15:41:25 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>SAKE: Towards Editing Auditory Attribute Knowledge of Large Audio-Language Models</title>
|
||
<link>https://arxiv.org/abs/2510.16917</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.16917.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Chih-Kai Yang, Yen-Ting Piao, Tzu-Wen Hsu, Szu-Wei Fu, Zhehuai Chen, Ke-Han Lu, Sung-Feng Huang, Chao-Han Huck Yang, Yu-Chiang Frank Wang, Yun-Nung Chen, Hung-yi Lee</p><p><b>Upvotes:</b> 18</p><p><b>Summary:</b> Knowledge editing offers an efficient way to update model knowledge without full retraining, but prior work has concentrated almost exclusively on textual or visual modalities. We introduce SAKE, the first benchmark specifically designed for editing auditory attribute knowledge in Large Audio-Language Models (LALMs). Unlike factual updates, SAKE targets several abstract auditory attributes, capturing knowledge types that go beyond conventional textual and visual domains. We benchmark seven editing methods on two LALMs along four dimensions: reliability, generality, audio/text locality, and portability. Results highlight challenges such as preserving intra-attribute knowledge unrelated to the edit, generalizing edits to multimodal reasoning, and maintaining edits under sequential updates. SAKE provides a principled framework to study how knowledge editing extends to the auditory modalities, opening new directions for maintaining and adapting LALMs in more diverse real-world scenarios.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.16917</guid>
|
||
<pubDate>Sun, 19 Oct 2025 16:22:09 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Video Reasoning without Training</title>
|
||
<link>https://arxiv.org/abs/2510.17045</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17045.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Deepak Sridhar, Kartikeya Bhardwaj, Jeya Pradha Jeyaraj, Nuno Vasconcelos, Ankita Nayak, Harris Teague</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Video reasoning using Large Multimodal Models (LMMs) relies on costly reinforcement learning (RL) and verbose chain-of-thought, resulting in substantial computational overhead during both training and inference. Moreover, the mechanisms that control the thinking process in these reasoning models are very limited. In this paper, using entropy of the model's output as a signal, we discover that the high-quality models go through a series of micro-explorations and micro-exploitations which keep the reasoning process grounded (i.e., avoid excessive randomness while the model is exploring or thinking through an answer). We further observe that once this "thinking" process is over, more accurate models demonstrate a better convergence by reducing the entropy significantly via a final exploitation phase (i.e., a more certain convergence towards a solution trajectory). We then use these novel, theoretically-grounded insights to tune the model's behavior directly at inference, without using any RL or supervised fine-tuning. Specifically, during inference, our proposed approach called V-Reason (Video-Reason) adapts the value cache of the LMM via a few optimization steps on a small, trainable controller using an entropy-based objective, i.e., no supervision from any dataset or RL is necessary. This tuning improves the model's micro-exploration and exploitation behavior during inference. Our experiments show that our proposed method achieves significant improvements over the base instruction-tuned models across several video reasoning datasets, narrowing the gap with RL-trained models to within 0.6% average accuracy without any training, while offering massive efficiency benefits: output tokens are reduced by 58.6% compared to the RL model.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17045</guid>
|
||
<pubDate>Sun, 19 Oct 2025 23:17:13 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>FineVision: Open Data Is All You Need</title>
|
||
<link>https://arxiv.org/abs/2510.17269</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17269.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Luis Wiedmann, Orr Zohar, Amir Mahla, Xiaohan Wang, Rui Li, Thibaud Frere, Leandro von Werra, Aritra Roy Gosthipaty, Andrés Marafioti</p><p><b>Upvotes:</b> 53</p><p><b>Summary:</b> The advancement of vision-language models (VLMs) is hampered by a fragmented landscape of inconsistent and contaminated public datasets. We introduce FineVision, a meticulously collected, curated, and unified corpus of 24 million samples - the largest open resource of its kind. We unify more than 200 sources into 185 subsets via a semi-automated, human-in-the-loop pipeline: automation performs bulk ingestion and schema mapping, while reviewers audit mappings and spot-check outputs to verify faithful consumption of annotations, appropriate formatting and diversity, and safety; issues trigger targeted fixes and re-runs. The workflow further applies rigorous de-duplication within and across sources and decontamination against 66 public benchmarks. FineVision also encompasses agentic/GUI tasks with a unified action space; reviewers validate schemas and inspect a sample of trajectories to confirm executable fidelity. Models trained on FineVision consistently outperform those trained on existing open mixtures across a broad evaluation suite, underscoring the benefits of scale, data hygiene, and balanced automation with human oversight. We release the corpus and curation tools to accelerate data-centric VLM research.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17269</guid>
|
||
<pubDate>Mon, 20 Oct 2025 07:54:46 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Towards Mixed-Modal Retrieval for Universal Retrieval-Augmented Generation</title>
|
||
<link>https://arxiv.org/abs/2510.17354</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17354.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Chenghao Zhang, Guanting Dong, Xinyu Yang, Zhicheng Dou</p><p><b>Upvotes:</b> 32</p><p><b>Summary:</b> Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for enhancing large language models (LLMs) by retrieving relevant documents from an external corpus. However, existing RAG systems primarily focus on unimodal text documents, and often fall short in real-world scenarios where both queries and documents may contain mixed modalities (such as text and images). In this paper, we address the challenge of Universal Retrieval-Augmented Generation (URAG), which involves retrieving and reasoning over mixed-modal information to improve vision-language generation. To this end, we propose Nyx, a unified mixed-modal to mixed-modal retriever tailored for URAG scenarios. To mitigate the scarcity of realistic mixed-modal data, we introduce a four-stage automated pipeline for generation and filtering, leveraging web documents to construct NyxQA, a dataset comprising diverse mixed-modal question-answer pairs that better reflect real-world information needs. Building on this high-quality dataset, we adopt a two-stage training framework for Nyx: we first perform pre-training on NyxQA along with a variety of open-source retrieval datasets, followed by supervised fine-tuning using feedback from downstream vision-language models (VLMs) to align retrieval outputs with generative preferences. Experimental results demonstrate that Nyx not only performs competitively on standard text-only RAG benchmarks, but also excels in the more general and realistic URAG setting, significantly improving generation quality in vision-language tasks.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17354</guid>
|
||
<pubDate>Mon, 20 Oct 2025 09:56:43 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Deep Self-Evolving Reasoning</title>
|
||
<link>https://arxiv.org/abs/2510.17498</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17498.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zihan Liu, Shun Zheng, Xumeng Wen, Yang Wang, Jiang Bian, Mao Yang</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Long-form chain-of-thought reasoning has become a cornerstone of advanced reasoning in large language models. While recent verification-refinement frameworks have enabled proprietary models to solve Olympiad-level problems, their effectiveness hinges on strong, reliable verification and correction capabilities, which remain fragile in open-weight, smaller-scale models. This work demonstrates that even with weak verification and refinement capabilities on hard tasks, the reasoning limits of such models can be substantially extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning (DSER). We conceptualize iterative reasoning as a Markov chain, where each step represents a stochastic transition in the solution space. The key insight is that convergence to a correct solution is guaranteed as long as the probability of improvement marginally exceeds that of degradation. By running multiple long-horizon, self-evolving processes in parallel, DSER amplifies these small positive tendencies, enabling the model to asymptotically approach correct answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously unsolvable problems and boosts overall performance, enabling this compact model to surpass the single-turn accuracy of its 600B-parameter teacher through majority voting. Beyond its immediate utility for test-time scaling, the DSER framework serves to diagnose the fundamental limitations of current open-weight reasoners. By clearly delineating their shortcomings in self-verification, refinement, and stability, our findings establish a clear research agenda for developing next-generation models with powerful, intrinsic self-evolving capabilities.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17498</guid>
|
||
<pubDate>Mon, 20 Oct 2025 12:51:42 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Annotation-Efficient Universal Honesty Alignment</title>
|
||
<link>https://arxiv.org/abs/2510.17509</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17509.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shiyu Ni, Keping Bi, Jiafeng Guo, Minghao Tang, Jingtong Wu, Zengxin Han, Xueqi Cheng</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> Honesty alignment-the ability of large language models (LLMs) to recognize their knowledge boundaries and express calibrated confidence-is essential for trustworthy deployment. Existing methods either rely on training-free confidence estimation (e.g., token probabilities, self-consistency) or training-based calibration with correctness annotations. While effective, achieving universal honesty alignment with training-based calibration requires costly, large-scale labeling. To support annotation-efficient training, we introduce Elicitation-Then-Calibration (EliCal), a two-stage framework that first elicits internal confidence using inexpensive self-consistency supervision, then calibrates this confidence with a small set of correctness annotations. To support a large-scale study, we release HonestyBench, a benchmark covering ten free-form QA datasets with 560k training and 70k evaluation instances annotated with correctness and self-consistency signals. Experiments show that EliCal achieves near-optimal alignment with only 1k correctness annotations (0.18% of full supervision) and better alignment performance on unseen MMLU tasks than the calibration-only baseline, offering a scalable solution toward universal honesty alignment in LLMs.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17509</guid>
|
||
<pubDate>Mon, 20 Oct 2025 13:05:22 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>MUG-V 10B: High-efficiency Training Pipeline for Large Video Generation Models</title>
|
||
<link>https://arxiv.org/abs/2510.17519</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17519.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yongshun Zhang, Zhongyi Fan, Yonghang Zhang, Zhangzikang Li, Weifeng Chen, Zhongwei Feng, Chaoyue Wang, Peng Hou, Anxiang Zeng</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> In recent years, large-scale generative models for visual content (e.g., images, videos, and 3D objects/scenes) have made remarkable progress. However, training large-scale video generation models remains particularly challenging and resource-intensive due to cross-modal text-video alignment, the long sequences involved, and the complex spatiotemporal dependencies. To address these challenges, we present a training framework that optimizes four pillars: (i) data processing, (ii) model architecture, (iii) training strategy, and (iv) infrastructure for large-scale video generation models. These optimizations delivered significant efficiency gains and performance improvements across all stages of data preprocessing, video compression, parameter scaling, curriculum-based pretraining, and alignment-focused post-training. Our resulting model, MUG-V 10B, matches recent state-of-the-art video generators overall and, on e-commerce-oriented video generation tasks, surpasses leading open-source baselines in human evaluations. More importantly, we open-source the complete stack, including model weights, Megatron-Core-based large-scale training code, and inference pipelines for video generation and enhancement. To our knowledge, this is the first public release of large-scale video generation training code that exploits Megatron-Core to achieve high training efficiency and near-linear multi-node scaling, details are available in https://github.com/Shopee-MUG/MUG-V{our webpage}.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17519</guid>
|
||
<pubDate>Mon, 20 Oct 2025 13:20:37 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>From Charts to Code: A Hierarchical Benchmark for Multimodal Models</title>
|
||
<link>https://arxiv.org/abs/2510.17932</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17932.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu, Yifei Tao, Dongxing Mao, Yang Wan, Jingru Tan, Min Zeng, Min Li, Alex Jinpeng Wang</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> We introduce Chart2Code, a new benchmark for evaluating the chart understanding and code generation capabilities of large multimodal models (LMMs). Chart2Code is explicitly designed from a user-driven perspective, capturing diverse real-world scenarios and progressively increasing task difficulty. It consists of three levels: Level 1 (Chart Reproduction) reproduces charts from a reference figure and user query; Level 2 (Chart Editing) involves complex modifications such as changing chart types or adding elements; and Level 3 (Long-Table to Chart Generation) requires models to transform long, information-dense tables into faithful charts following user instructions. To our knowledge, this is the first hierarchical benchmark that reflects practical chart2code usage while systematically scaling task complexity. In total, Chart2Code contains 2,023 tasks across 22 chart types, paired with multi-level evaluation metrics that assess both code correctness and the visual fidelity of rendered charts. We benchmark 25 state-of-the-art (SoTA) LMMs, including both proprietary and the latest open-source models such as GPT-5, Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Experimental results demonstrate that even the SoTA model GPT-5 averages only 0.57 on code-based evaluation and 0.22 on chart-quality assessment across the editing tasks, underscoring the difficulty of Chart2Code. We anticipate this benchmark will drive advances in multimodal reasoning and foster the development of more robust and general-purpose LMMs. Our code and data are available on Chart2Code.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17932</guid>
|
||
<pubDate>Mon, 20 Oct 2025 15:11:56 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>PICABench: How Far Are We from Physically Realistic Image Editing?</title>
|
||
<link>https://arxiv.org/abs/2510.17681</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17681.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuandong Pu, Le Zhuo, Songhao Han, Jinbo Xing, Kaiwen Zhu, Shuo Cao, Bin Fu, Si Liu, Hongsheng Li, Yu Qiao, Wenlong Zhang, Xi Chen, Yihao Liu</p><p><b>Upvotes:</b> 59</p><p><b>Summary:</b> Image editing has achieved remarkable progress recently. Modern editing models could already follow complex instructions to manipulate the original content. However, beyond completing the editing instructions, the accompanying physical effects are the key to the generation realism. For example, removing an object should also remove its shadow, reflections, and interactions with nearby objects. Unfortunately, existing models and benchmarks mainly focus on instruction completion but overlook these physical effects. So, at this moment, how far are we from physically realistic image editing? To answer this, we introduce PICABench, which systematically evaluates physical realism across eight sub-dimension (spanning optics, mechanics, and state transitions) for most of the common editing operations (add, remove, attribute change, etc). We further propose the PICAEval, a reliable evaluation protocol that uses VLM-as-a-judge with per-case, region-level human annotations and questions. Beyond benchmarking, we also explore effective solutions by learning physics from videos and construct a training dataset PICA-100K. After evaluating most of the mainstream models, we observe that physical realism remains a challenging problem with large rooms to explore. We hope that our benchmark and proposed solutions can serve as a foundation for future work moving from naive content editing toward physically consistent realism.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17681</guid>
|
||
<pubDate>Mon, 20 Oct 2025 15:53:57 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver</title>
|
||
<link>https://arxiv.org/abs/2510.17699</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17699.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Aleksandr Oganov, Ilya Bykov, Eva Neudachina, Mishan Aliev, Alexander Tolmachev, Alexander Sidorov, Aleksandr Zuev, Andrey Okhotin, Denis Rakitin, Aibek Alanov</p><p><b>Upvotes:</b> 20</p><p><b>Summary:</b> While diffusion models achieve state-of-the-art generation quality, they still suffer from computationally expensive sampling. Recent works address this issue with gradient-based optimization methods that distill a few-step ODE diffusion solver from the full sampling process, reducing the number of function evaluations from dozens to just a few. However, these approaches often rely on intricate training techniques and do not explicitly focus on preserving fine-grained details. In this paper, we introduce the Generalized Solver: a simple parameterization of the ODE sampler that does not require additional training tricks and improves quality over existing approaches. We further combine the original distillation loss with adversarial training, which mitigates artifacts and enhances detail fidelity. We call the resulting method the Generalized Adversarial Solver and demonstrate its superior performance compared to existing solver training methods under similar resource constraints. Code is available at https://github.com/3145tttt/GAS.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17699</guid>
|
||
<pubDate>Mon, 20 Oct 2025 16:14:38 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>QueST: Incentivizing LLMs to Generate Difficult Problems</title>
|
||
<link>https://arxiv.org/abs/2510.17715</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17715.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Hanxu Hu, Xingxing Zhang, Jannis Vamvas, Rico Sennrich, Furu Wei</p><p><b>Upvotes:</b> 30</p><p><b>Summary:</b> Large Language Models have achieved strong performance on reasoning tasks, solving competition-level coding and math problems. However, their scalability is limited by human-labeled datasets and the lack of large-scale, challenging coding problem training data. Existing competitive coding datasets contain only thousands to tens of thousands of problems. Previous synthetic data generation methods rely on either augmenting existing instruction datasets or selecting challenging problems from human-labeled data. In this paper, we propose QueST, a novel framework which combines difficulty-aware graph sampling and difficulty-aware rejection fine-tuning that directly optimizes specialized generators to create challenging coding problems. Our trained generators demonstrate superior capability compared to even GPT-4o at creating challenging problems that benefit downstream performance. We leverage QueST to generate large-scale synthetic coding problems, which we then use to distill from strong teacher models with long chain-of-thought or to conduct reinforcement learning for smaller models, proving effective in both scenarios. Our distillation experiments demonstrate significant performance gains. Specifically, after fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we surpass the performance of the original Qwen3-8B on LiveCodeBench. With an additional 112K examples (i.e., 28K human-written problems paired with multiple synthetic solutions), our 8B model matches the performance of the much larger DeepSeek-R1-671B. These findings indicate that generating complex problems via QueST offers an effective and scalable approach to advancing the frontiers of competitive coding and reasoning for large language models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17715</guid>
|
||
<pubDate>Mon, 20 Oct 2025 16:29:53 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>MT-Video-Bench: A Holistic Video Understanding Benchmark for Evaluating Multimodal LLMs in Multi-Turn Dialogues</title>
|
||
<link>https://arxiv.org/abs/2510.17722</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17722.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yaning Pan, Zekun Wang, Qianqian Xie, Yongqian Wen, Yuanxing Zhang, Guohui Zhang, Haoxuan Hu, Zhiyu Pan, Yibing Huang, Zhidong Gan, Yonghong Lin, An Ping, Tianhao Peng, Jiaheng Liu</p><p><b>Upvotes:</b> 16</p><p><b>Summary:</b> The recent development of Multimodal Large Language Models (MLLMs) has significantly advanced AI's ability to understand visual modalities. However, existing evaluation benchmarks remain limited to single-turn question answering, overlooking the complexity of multi-turn dialogues in real-world scenarios. To bridge this gap, we introduce MT-Video-Bench, a holistic video understanding benchmark for evaluating MLLMs in multi-turn dialogues. Specifically, our MT-Video-Bench mainly assesses six core competencies that focus on perceptivity and interactivity, encompassing 987 meticulously curated multi-turn dialogues from diverse domains. These capabilities are rigorously aligned with real-world applications, such as interactive sports analysis and multi-turn video-based intelligent tutoring. With MT-Video-Bench, we extensively evaluate various state-of-the-art open-source and closed-source MLLMs, revealing their significant performance discrepancies and limitations in handling multi-turn video dialogues. The benchmark will be publicly available to foster future research.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17722</guid>
|
||
<pubDate>Mon, 20 Oct 2025 16:38:40 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Executable Knowledge Graphs for Replicating AI Research</title>
|
||
<link>https://arxiv.org/abs/2510.17795</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17795.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yujie Luo, Zhuoyun Yu, Xuehai Wang, Yuqi Zhu, Ningyu Zhang, Lanning Wei, Lun Du, Da Zheng, Huajun Chen</p><p><b>Upvotes:</b> 12</p><p><b>Summary:</b> Replicating AI research is a crucial yet challenging task for large language model (LLM) agents. Existing approaches often struggle to generate executable code, primarily due to insufficient background knowledge and the limitations of retrieval-augmented generation (RAG) methods, which fail to capture latent technical details hidden in referenced papers. Furthermore, previous approaches tend to overlook valuable implementation-level code signals and lack structured knowledge representations that support multi-granular retrieval and reuse. To overcome these challenges, we propose Executable Knowledge Graphs (xKG), a modular and pluggable knowledge base that automatically integrates technical insights, code snippets, and domain-specific knowledge extracted from scientific literature. When integrated into three agent frameworks with two different LLMs, xKG shows substantial performance gains (10.9% with o3-mini) on PaperBench, demonstrating its effectiveness as a general and extensible solution for automated AI research replication. Code will released at https://github.com/zjunlp/xKG.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17795</guid>
|
||
<pubDate>Mon, 20 Oct 2025 17:53:23 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Glyph: Scaling Context Windows via Visual-Text Compression</title>
|
||
<link>https://arxiv.org/abs/2510.17800</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17800.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiale Cheng, Yusen Liu, Xinyu Zhang, Yulin Fei, Wenyi Hong, Ruiliang Lyu, Weihan Wang, Zhe Su, Xiaotao Gu, Xiao Liu, Yushi Bai, Jie Tang, Hongning Wang, Minlie Huang</p><p><b>Upvotes:</b> 57</p><p><b>Summary:</b> Large language models (LLMs) increasingly rely on long-context modeling for tasks such as document understanding, code analysis, and multi-step reasoning. However, scaling context windows to the million-token level brings prohibitive computational and memory costs, limiting the practicality of long-context LLMs. In this work, we take a different perspective-visual context scaling-to tackle this challenge. Instead of extending token-based sequences, we propose Glyph, a framework that renders long texts into images and processes them with vision-language models (VLMs). This approach substantially compresses textual input while preserving semantic information, and we further design an LLM-driven genetic search to identify optimal visual rendering configurations for balancing accuracy and compression. Through extensive experiments, we demonstrate that our method achieves 3-4x token compression while maintaining accuracy comparable to leading LLMs such as Qwen3-8B on various long-context benchmarks. This compression also leads to around 4x faster prefilling and decoding, and approximately 2x faster SFT training. Furthermore, under extreme compression, a 128K-context VLM could scale to handle 1M-token-level text tasks. In addition, the rendered text data benefits real-world multimodal tasks, such as document understanding. Our code and model are released at https://github.com/thu-coai/Glyph.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17800</guid>
|
||
<pubDate>Mon, 20 Oct 2025 17:58:56 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ConsistEdit: Highly Consistent and Precise Training-free Visual Editing</title>
|
||
<link>https://arxiv.org/abs/2510.17803</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17803.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zixin Yin, Ling-Hao Chen, Lionel Ni, Xili Dai</p><p><b>Upvotes:</b> 12</p><p><b>Summary:</b> Recent advances in training-free attention control methods have enabled flexible and efficient text-guided editing capabilities for existing generation models. However, current approaches struggle to simultaneously deliver strong editing strength while preserving consistency with the source. This limitation becomes particularly critical in multi-round and video editing, where visual errors can accumulate over time. Moreover, most existing methods enforce global consistency, which limits their ability to modify individual attributes such as texture while preserving others, thereby hindering fine-grained editing. Recently, the architectural shift from U-Net to MM-DiT has brought significant improvements in generative performance and introduced a novel mechanism for integrating text and vision modalities. These advancements pave the way for overcoming challenges that previous methods failed to resolve. Through an in-depth analysis of MM-DiT, we identify three key insights into its attention mechanisms. Building on these, we propose ConsistEdit, a novel attention control method specifically tailored for MM-DiT. ConsistEdit incorporates vision-only attention control, mask-guided pre-attention fusion, and differentiated manipulation of the query, key, and value tokens to produce consistent, prompt-aligned edits. Extensive experiments demonstrate that ConsistEdit achieves state-of-the-art performance across a wide range of image and video editing tasks, including both structure-consistent and structure-inconsistent scenarios. Unlike prior methods, it is the first approach to perform editing across all inference steps and attention layers without handcraft, significantly enhancing reliability and consistency, which enables robust multi-round and multi-region editing. Furthermore, it supports progressive adjustment of structural consistency, enabling finer control.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17803</guid>
|
||
<pubDate>Mon, 20 Oct 2025 17:59:52 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>AION-1: Omnimodal Foundation Model for Astronomical Sciences</title>
|
||
<link>https://arxiv.org/abs/2510.17960</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.17960.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Liam Parker, Francois Lanusse, Jeff Shen, Ollie Liu, Tom Hehir, Leopoldo Sarra, Lucas Meyer, Micah Bowles, Sebastian Wagner-Carena, Helen Qu, Siavash Golkar, Alberto Bietti, Hatim Bourfoune, Nathan Casserau, Pierre Cornette, Keiya Hirashima, Geraud Krawezik, Ruben Ohana, Nicholas Lourie, Michael McCabe, Rudy Morel, Payel Mukhopadhyay, Mariel Pettee, Bruno Regaldo-Saint Blancard, Kyunghyun Cho, Miles Cranmer, Shirley Ho</p><p><b>Upvotes:</b> 26</p><p><b>Summary:</b> While foundation models have shown promise across a variety of fields, astronomy still lacks a unified framework for joint modeling across its highly diverse data modalities. In this paper, we present AION-1, a family of large-scale multimodal foundation models for astronomy. AION-1 integrates heterogeneous imaging, spectroscopic, and scalar data using a two-stage architecture: modality-specific tokenization followed by transformer-based masked modeling of cross-modal token sequences. The model is pretrained on five large-scale surveys: Legacy Survey, Hyper Suprime-Cam (HSC), Sloan Digital Sky Survey (SDSS), Dark Energy Spectroscopic Instrument (DESI), and Gaia. These span more than 200 million observations of stars, galaxies, and quasars. With a single frozen encoder, AION-1 achieves strong results on a broad suite of downstream tasks, including galaxy and stellar property estimation, galaxy morphology classification, similarity-based retrieval, galaxy image segmentation, and spectral super-resolution. We release AION-1 model variants ranging from 300 M to 3.1 B parameters. Beyond astronomy, AION-1 provides a scalable blueprint for multimodal scientific foundation models that can seamlessly integrate noisy, instrument-specific observations. All code, tokenizers, pretrained weights, and a lightweight evaluation suite are released under an open-source license.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.17960</guid>
|
||
<pubDate>Mon, 20 Oct 2025 18:00:00 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Is Multilingual LLM Watermarking Truly Multilingual? A Simple Back-Translation Solution</title>
|
||
<link>https://arxiv.org/abs/2510.18019</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18019.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Asim Mohamed, Martin Gubri</p><p><b>Upvotes:</b> 16</p><p><b>Summary:</b> Multilingual watermarking aims to make large language model (LLM) outputs traceable across languages, yet current methods still fall short. Despite claims of cross-lingual robustness, they are evaluated only on high-resource languages. We show that existing multilingual watermarking methods are not truly multilingual: they fail to remain robust under translation attacks in medium- and low-resource languages. We trace this failure to semantic clustering, which fails when the tokenizer vocabulary contains too few full-word tokens for a given language. To address this, we introduce STEAM, a back-translation-based detection method that restores watermark strength lost through translation. STEAM is compatible with any watermarking method, robust across different tokenizers and languages, non-invasive, and easily extendable to new languages. With average gains of +0.19 AUC and +40%p TPR@1% on 17 languages, STEAM provides a simple and robust path toward fairer watermarking across diverse languages.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18019</guid>
|
||
<pubDate>Mon, 20 Oct 2025 18:51:20 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Efficient Long-context Language Model Training by Core Attention Disaggregation</title>
|
||
<link>https://arxiv.org/abs/2510.18121</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18121.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yonghao Zhuang, Junda Chen, Bo Pang, Yi Gu, Yibo Zhu, Yimin Jiang, Ion Stoica, Eric Xing, Hao Zhang</p><p><b>Upvotes:</b> 108</p><p><b>Summary:</b> We present core attention disaggregation (CAD), a technique that improves long-context large language model training by decoupling the core attention computation, softmax(QK^T)V, from the rest of the model and executing it on a separate pool of devices. In existing systems, core attention is colocated with other layers; at long context lengths, its quadratic compute growth compared to the near-linear growth of other components causes load imbalance and stragglers across data and pipeline parallel groups. CAD is enabled by two observations. First, core attention is stateless: it has no trainable parameters and only minimal transient data, so balancing reduces to scheduling compute-bound tasks. Second, it is composable: modern attention kernels retain high efficiency when processing fused batches of token-level shards with arbitrary lengths. CAD partitions core attention into token-level tasks and dispatches them to dedicated attention servers, which dynamically rebatch tasks to equalize compute without sacrificing kernel efficiency. We implement CAD in a system called DistCA, which uses a ping-pong execution scheme to fully overlap communication with computation and in-place execution on attention servers to reduce memory use. On 512 H200 GPUs and context lengths up to 512k tokens, DistCA improves end-to-end training throughput by up to 1.35x, eliminates data and pipeline parallel stragglers, and achieves near-perfect compute and memory balance.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18121</guid>
|
||
<pubDate>Mon, 20 Oct 2025 21:40:51 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>World-in-World: World Models in a Closed-Loop World</title>
|
||
<link>https://arxiv.org/abs/2510.18135</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18135.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiahan Zhang, Muqing Jiang, Nanru Dai, Taiming Lu, Arda Uzunoglu, Shunchi Zhang, Yana Wei, Jiahao Wang, Vishal M. Patel, Paul Pu Liang, Daniel Khashabi, Cheng Peng, Rama Chellappa, Tianmin Shu, Alan Yuille, Yilun Du, Jieneng Chen</p><p><b>Upvotes:</b> 80</p><p><b>Summary:</b> Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive perception for decision making. Progress on this question has been limited by fragmented evaluation: most existing benchmarks adopt open-loop protocols that emphasize visual quality in isolation, leaving the core issue of embodied utility unresolved, i.e., do WMs actually help agents succeed at embodied tasks? To address this gap, we introduce World-in-World, the first open platform that benchmarks WMs in a closed-loop world that mirrors real agent-environment interactions. World-in-World provides a unified online planning strategy and a standardized action API, enabling heterogeneous WMs for decision making. We curate four closed-loop environments that rigorously evaluate diverse WMs, prioritize task success as the primary metric, and move beyond the common focus on visual quality; we also present the first data scaling law for world models in embodied settings. Our study uncovers three surprises: (1) visual quality alone does not guarantee task success, controllability matters more; (2) scaling post-training with action-observation data is more effective than upgrading the pretrained video generators; and (3) allocating more inference-time compute allows WMs to substantially improve closed-loop performance.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18135</guid>
|
||
<pubDate>Mon, 20 Oct 2025 22:09:15 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DeepSeek-OCR: Contexts Optical Compression</title>
|
||
<link>https://arxiv.org/abs/2510.18234</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18234.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Haoran Wei, Yaofeng Sun, Yukun Li</p><p><b>Upvotes:</b> 55</p><p><b>Summary:</b> We present DeepSeek-OCR as an initial investigation into the feasibility of compressing long contexts via optical 2D mapping. DeepSeek-OCR consists of two components: DeepEncoder and DeepSeek3B-MoE-A570M as the decoder. Specifically, DeepEncoder serves as the core engine, designed to maintain low activations under high-resolution input while achieving high compression ratios to ensure an optimal and manageable number of vision tokens. Experiments show that when the number of text tokens is within 10 times that of vision tokens (i.e., a compression ratio < 10x), the model can achieve decoding (OCR) precision of 97%. Even at a compression ratio of 20x, the OCR accuracy still remains at about 60%. This shows considerable promise for research areas such as historical long-context compression and memory forgetting mechanisms in LLMs. Beyond this, DeepSeek-OCR also demonstrates high practical value. On OmniDocBench, it surpasses GOT-OCR2.0 (256 tokens/page) using only 100 vision tokens, and outperforms MinerU2.0 (6000+ tokens per page on average) while utilizing fewer than 800 vision tokens. In production, DeepSeek-OCR can generate training data for LLMs/VLMs at a scale of 200k+ pages per day (a single A100-40G). Codes and model weights are publicly accessible at http://github.com/deepseek-ai/DeepSeek-OCR.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18234</guid>
|
||
<pubDate>Tue, 21 Oct 2025 02:41:44 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs</title>
|
||
<link>https://arxiv.org/abs/2510.18245</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18245.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Song Bian, Tao Yu, Shivaram Venkataraman, Youngsuk Park</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Scaling the number of parameters and the size of training data has proven to be an effective strategy for improving large language model (LLM) performance. Yet, as these models grow increasingly powerful and widely deployed, the cost of inference has become a pressing concern. Despite its importance, the trade-off between model accuracy and inference efficiency remains underexplored. In this work, we examine how key architectural factors, hidden size, the allocation of parameters between MLP and attention (mlp-to-attention ratio), and grouped-query attention (GQA), influence both inference cost and accuracy. We introduce a conditional scaling law that augments the Chinchilla framework with architectural information, along with a search framework for identifying architectures that are simultaneously inference-efficient and accurate. To validate our approach, we train more than 200 models spanning 80M to 3B parameters and 8B to 100B training tokens, and fit the proposed conditional scaling law. Our results show that the conditional scaling law reliably predicts optimal architectural choices and that the resulting models outperform existing open-source baselines. Under the same training budget, optimized architectures achieve up to 2.1% higher accuracy and 42% greater inference throughput compared to LLaMA-3.2.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18245</guid>
|
||
<pubDate>Tue, 21 Oct 2025 03:08:48 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ssToken: Self-modulated and Semantic-aware Token Selection for LLM Fine-tuning</title>
|
||
<link>https://arxiv.org/abs/2510.18250</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18250.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiaohan Qin, Xiaoxing Wang, Ning Liao, Cancheng Zhang, Xiangdong Zhang, Mingquan Feng, Jingzhi Wang, Junchi Yan</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> Data quality plays a critical role in enhancing supervised fine-tuning (SFT) for large language models (LLMs), and token-level data selection has emerged as a promising direction for its fine-grained nature. Despite their strong empirical performance, existing token-level selection methods share two key limitations: (1) requiring training or accessing an additional reference model, and (2) relying solely on loss information for token selection, which cannot well preserve semantically important tokens that are not favored by loss-based metrics. To address these challenges, we propose ssToken, a Self-modulated and Semantic-aware Token Selection approach. ssToken leverages readily accessible history models to compute the per-token loss difference with the current model, which serves as a self-modulated signal that enables the model to adaptively select tokens along its optimization trajectory, rather than relying on excess loss from an offline-trained reference model as in prior works. We further introduce a semantic-aware, attention-based token importance estimation metric, orthogonal to loss-based selection and providing complementary semantic information for more effective filtering. Extensive experiments across different model families and scales demonstrate that both self-modulated selection and semantic-aware selection alone outperform full-data fine-tuning, while their integration--ssToken--achieves synergistic gains and further surpasses prior token-level selection methods, delivering performance improvements while maintaining training efficiency.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18250</guid>
|
||
<pubDate>Tue, 21 Oct 2025 03:21:04 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>OmniNWM: Omniscient Driving Navigation World Models</title>
|
||
<link>https://arxiv.org/abs/2510.18313</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18313.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Bohan Li, Zhuang Ma, Dalong Du, Baorui Peng, Zhujin Liang, Zhenqiang Liu, Chao Ma, Yueming Jin, Hao Zhao, Wenjun Zeng, Xin Jin</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise action control, and a lack of reward awareness. In this paper, we introduce OmniNWM, an omniscient panoramic navigation world model that addresses all three dimensions within a unified framework. For state, OmniNWM jointly generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy. A flexible forcing strategy enables high-quality long-horizon auto-regressive generation. For action, we introduce a normalized panoramic Plucker ray-map representation that encodes input trajectories into pixel-level signals, enabling highly precise and generalizable control over panoramic video generation. Regarding reward, we move beyond learning reward functions with external image-based models: instead, we leverage the generated 3D occupancy to directly define rule-based dense rewards for driving compliance and safety. Extensive experiments demonstrate that OmniNWM achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, while providing a reliable closed-loop evaluation framework through occupancy-grounded rewards. Project page is available at https://github.com/Arlo0o/OmniNWM.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18313</guid>
|
||
<pubDate>Tue, 21 Oct 2025 05:49:01 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Extracting alignment data in open models</title>
|
||
<link>https://arxiv.org/abs/2510.18554</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18554.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Federico Barbero, Xiangming Gu, Christopher A. Choquette-Choo, Chawin Sitawarin, Matthew Jagielski, Itay Yona, Petar Veličković, Ilia Shumailov, Jamie Hayes</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> In this work, we show that it is possible to extract significant amounts of alignment training data from a post-trained model -- useful to steer the model to improve certain capabilities such as long-context reasoning, safety, instruction following, and maths. While the majority of related work on memorisation has focused on measuring success of training data extraction through string matching, we argue that embedding models are better suited for our specific goals. Distances measured through a high quality embedding model can identify semantic similarities between strings that a different metric such as edit distance will struggle to capture. In fact, in our investigation, approximate string matching would have severely undercounted (by a conservative estimate of 10times) the amount of data that can be extracted due to trivial artifacts that deflate the metric. Interestingly, we find that models readily regurgitate training data that was used in post-training phases such as SFT or RL. We show that this data can be then used to train a base model, recovering a meaningful amount of the original performance. We believe our work exposes a possibly overlooked risk towards extracting alignment data. Finally, our work opens up an interesting discussion on the downstream effects of distillation practices: since models seem to be regurgitating aspects of their training set, distillation can therefore be thought of as indirectly training on the model's original dataset.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18554</guid>
|
||
<pubDate>Tue, 21 Oct 2025 12:06:00 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>BAPO: Stabilizing Off-Policy Reinforcement Learning for LLMs via Balanced Policy Optimization with Adaptive Clipping</title>
|
||
<link>https://arxiv.org/abs/2510.18927</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18927.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhiheng Xi, Xin Guo, Yang Nan, Enyu Zhou, Junrui Shen, Wenxiang Chen, Jiaqi Liu, Jixuan Huang, Zhihao Zhang, Honglin Guo, Xun Deng, Zhikai Lei, Miao Zheng, Guoteng Wang, Shuo Zhang, Peng Sun, Rui Zheng, Hang Yan, Tao Gui, Qi Zhang, Xuanjing Huang</p><p><b>Upvotes:</b> 77</p><p><b>Summary:</b> Reinforcement learning (RL) has recently become the core paradigm for aligning and strengthening large language models (LLMs). Yet, applying RL in off-policy settings--where stale data from past policies are used for training--improves sample efficiency, but remains challenging: policy entropy declines sharply, optimization often becomes unstable and may even collapse. Through theoretical and empirical analysis, we identify two key insights: (i) an imbalance in optimization, where negative-advantage samples dominate the policy gradient, suppressing useful behaviors and risking gradient explosions; and (ii) the derived Entropy-Clip Rule, which reveals that the fixed clipping mechanism in PPO-like objectives systematically blocks entropy-increasing updates, thereby driving the policy toward over-exploitation at the expense of exploration. Building on these insights, we propose BAlanced Policy Optimization with Adaptive Clipping (BAPO), a simple yet effective method that dynamically adjusts clipping bounds to adaptively re-balance positive and negative contributions, preserve entropy, and stabilize RL optimization. Across diverse off-policy scenarios--including sample replay and partial rollout--BAPO achieves fast, stable, and data-efficient training. On AIME 2024 and AIME 2025 benchmarks, our 7B BAPO model surpasses open-source counterparts such as SkyWork-OR1-7B, while our 32B BAPO model not only achieves state-of-the-art results among models of the same scale but also outperforms leading proprietary systems like o3-mini and Gemini-2.5-Flash-Thinking.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18927</guid>
|
||
<pubDate>Tue, 21 Oct 2025 12:55:04 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views</title>
|
||
<link>https://arxiv.org/abs/2510.18632</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18632.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhangquan Chen, Manyuan Zhang, Xinlei Yu, Xufang Luo, Mingze Sun, Zihao Pan, Yan Feng, Peng Pei, Xunliang Cai, Ruqi Huang</p><p><b>Upvotes:</b> 20</p><p><b>Summary:</b> Though recent advances in vision-language models (VLMs) have achieved remarkable progress across a wide range of multimodal tasks, understanding 3D spatial relationships from limited views remains a significant challenge. Previous reasoning methods typically rely on pure text (e.g., topological cognitive maps) or on 2D visual cues. However, their limited representational capacity hinders performance in specific tasks that require 3D spatial imagination. To address this limitation, we propose 3DThinker, a framework that can effectively exploits the rich geometric information embedded within images while reasoning, like humans do. Our framework is the first to enable 3D mentaling during reasoning without any 3D prior input, and it does not rely on explicitly labeled 3D data for training. Specifically, our training consists of two stages. First, we perform supervised training to align the 3D latent generated by VLM while reasoning with that of a 3D foundation model (e.g., VGGT). Then, we optimize the entire reasoning trajectory solely based on outcome signals, thereby refining the underlying 3D mentaling. Extensive experiments across multiple benchmarks show that 3DThinker consistently outperforms strong baselines and offers a new perspective toward unifying 3D representations into multimodal reasoning. Our code will be available at https://github.com/zhangquanchen/3DThinker.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18632</guid>
|
||
<pubDate>Tue, 21 Oct 2025 13:36:58 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>MoGA: Mixture-of-Groups Attention for End-to-End Long Video Generation</title>
|
||
<link>https://arxiv.org/abs/2510.18692</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18692.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Weinan Jia, Yuning Lu, Mengqi Huang, Hualiang Wang, Binyuan Huang, Nan Chen, Mu Liu, Jidong Jiang, Zhendong Mao</p><p><b>Upvotes:</b> 38</p><p><b>Summary:</b> Long video generation with Diffusion Transformers (DiTs) is bottlenecked by the quadratic scaling of full attention with sequence length. Since attention is highly redundant, outputs are dominated by a small subset of query-key pairs. Existing sparse methods rely on blockwise coarse estimation, whose accuracy-efficiency trade-offs are constrained by block size. This paper introduces Mixture-of-Groups Attention (MoGA), an efficient sparse attention that uses a lightweight, learnable token router to precisely match tokens without blockwise estimation. Through semantic-aware routing, MoGA enables effective long-range interactions. As a kernel-free method, MoGA integrates seamlessly with modern attention stacks, including FlashAttention and sequence parallelism. Building on MoGA, we develop an efficient long video generation model that end-to-end produces minute-level, multi-shot, 480p videos at 24 fps, with a context length of approximately 580k. Comprehensive experiments on various video generation tasks validate the effectiveness of our approach.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18692</guid>
|
||
<pubDate>Tue, 21 Oct 2025 14:50:42 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation</title>
|
||
<link>https://arxiv.org/abs/2510.18701</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18701.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yibin Wang, Zhimin Li, Yuhang Zang, Jiazi Bu, Yujie Zhou, Yi Xin, Junjun He, Chunyu Wang, Qinglin Lu, Cheng Jin, Jiaqi Wang</p><p><b>Upvotes:</b> 64</p><p><b>Summary:</b> Recent progress in text-to-image (T2I) generation underscores the importance of reliable benchmarks in evaluating how accurately generated images reflect the semantics of their textual prompt. However, (1) existing benchmarks lack the diversity of prompt scenarios and multilingual support, both essential for real-world applicability; (2) they offer only coarse evaluations across primary dimensions, covering a narrow range of sub-dimensions, and fall short in fine-grained sub-dimension assessment. To address these limitations, we introduce UniGenBench++, a unified semantic assessment benchmark for T2I generation. Specifically, it comprises 600 prompts organized hierarchically to ensure both coverage and efficiency: (1) spans across diverse real-world scenarios, i.e., 5 main prompt themes and 20 subthemes; (2) comprehensively probes T2I models' semantic consistency over 10 primary and 27 sub evaluation criteria, with each prompt assessing multiple testpoints. To rigorously assess model robustness to variations in language and prompt length, we provide both English and Chinese versions of each prompt in short and long forms. Leveraging the general world knowledge and fine-grained image understanding capabilities of a closed-source Multi-modal Large Language Model (MLLM), i.e., Gemini-2.5-Pro, an effective pipeline is developed for reliable benchmark construction and streamlined model assessment. Moreover, to further facilitate community use, we train a robust evaluation model that enables offline assessment of T2I model outputs. Through comprehensive benchmarking of both open- and closed-sourced T2I models, we systematically reveal their strengths and weaknesses across various aspects.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18701</guid>
|
||
<pubDate>Tue, 21 Oct 2025 14:56:46 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>IF-VidCap: Can Video Caption Models Follow Instructions?</title>
|
||
<link>https://arxiv.org/abs/2510.18726</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18726.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shihao Li, Yuanxing Zhang, Jiangtao Wu, Zhide Lei, Yiwen He, Runzhe Wen, Chenxi Liao, Chengkang Jiang, An Ping, Shuo Gao, Suhan Wang, Zhaozhou Bian, Zijun Zhou, Jingyi Xie, Jiayi Zhou, Jing Wang, Yifan Yao, Weihao Xie, Yingshui Tan, Yanghai Wang, Qianqian Xie, Zhaoxiang Zhang, Jiaheng Liu</p><p><b>Upvotes:</b> 24</p><p><b>Summary:</b> Although Multimodal Large Language Models (MLLMs) have demonstrated proficiency in video captioning, practical applications require captions that follow specific user instructions rather than generating exhaustive, unconstrained descriptions. Current benchmarks, however, primarily assess descriptive comprehensiveness while largely overlooking instruction-following capabilities. To address this gap, we introduce IF-VidCap, a new benchmark for evaluating controllable video captioning, which contains 1,400 high-quality samples. Distinct from existing video captioning or general instruction-following benchmarks, IF-VidCap incorporates a systematic framework that assesses captions on two dimensions: format correctness and content correctness. Our comprehensive evaluation of over 20 prominent models reveals a nuanced landscape: despite the continued dominance of proprietary models, the performance gap is closing, with top-tier open-source solutions now achieving near-parity. Furthermore, we find that models specialized for dense captioning underperform general-purpose MLLMs on complex instructions, indicating that future work should simultaneously advance both descriptive richness and instruction-following fidelity.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18726</guid>
|
||
<pubDate>Tue, 21 Oct 2025 15:25:08 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>UltraGen: High-Resolution Video Generation with Hierarchical Attention</title>
|
||
<link>https://arxiv.org/abs/2510.18775</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18775.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Teng Hu, Jiangning Zhang, Zihan Su, Ran Yi</p><p><b>Upvotes:</b> 15</p><p><b>Summary:</b> Recent advances in video generation have made it possible to produce visually compelling videos, with wide-ranging applications in content creation, entertainment, and virtual reality. However, most existing diffusion transformer based video generation models are limited to low-resolution outputs (<=720P) due to the quadratic computational complexity of the attention mechanism with respect to the output width and height. This computational bottleneck makes native high-resolution video generation (1080P/2K/4K) impractical for both training and inference. To address this challenge, we present UltraGen, a novel video generation framework that enables i) efficient and ii) end-to-end native high-resolution video synthesis. Specifically, UltraGen features a hierarchical dual-branch attention architecture based on global-local attention decomposition, which decouples full attention into a local attention branch for high-fidelity regional content and a global attention branch for overall semantic consistency. We further propose a spatially compressed global modeling strategy to efficiently learn global dependencies, and a hierarchical cross-window local attention mechanism to reduce computational costs while enhancing information flow across different local windows. Extensive experiments demonstrate that UltraGen can effectively scale pre-trained low-resolution video models to 1080P and even 4K resolution for the first time, outperforming existing state-of-the-art methods and super-resolution based two-stage pipelines in both qualitative and quantitative evaluations.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18775</guid>
|
||
<pubDate>Tue, 21 Oct 2025 16:23:21 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ProCLIP: Progressive Vision-Language Alignment via LLM-based Embedder</title>
|
||
<link>https://arxiv.org/abs/2510.18795</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18795.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiaoxing Hu, Kaicheng Yang, Ziyong Feng, Qi Ming, Zonghao Guo, Xiang An, Ziyong Feng, Junchi Yan, Xue Yang</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> The original CLIP text encoder is limited by a maximum input length of 77 tokens, which hampers its ability to effectively process long texts and perform fine-grained semantic understanding. In addition, the CLIP text encoder lacks support for multilingual inputs. All these limitations significantly restrict its applicability across a broader range of tasks. Recent studies have attempted to replace the CLIP text encoder with an LLM-based embedder to enhance its ability in processing long texts, multilingual understanding, and fine-grained semantic comprehension. However, because the representation spaces of LLMs and the vision-language space of CLIP are pretrained independently without alignment priors, direct alignment using contrastive learning can disrupt the intrinsic vision-language alignment in the CLIP image encoder, leading to an underutilization of the knowledge acquired during pre-training. To address this challenge, we propose ProCLIP, a curriculum learning-based progressive vision-language alignment framework to effectively align the CLIP image encoder with an LLM-based embedder. Specifically, ProCLIP first distills knowledge from CLIP's text encoder into the LLM-based embedder to leverage CLIP's rich pretrained knowledge while establishing initial alignment between the LLM embedder and CLIP image encoder. Subsequently, ProCLIP further aligns the CLIP image encoder with the LLM-based embedder through image-text contrastive tuning, employing self-distillation regularization to avoid overfitting. To achieve a more effective alignment, instance semantic alignment loss and embedding structure alignment loss are employed during representation inheritance and contrastive tuning. The Code is available at https://github.com/VisionXLab/ProCLIP</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18795</guid>
|
||
<pubDate>Tue, 21 Oct 2025 16:48:49 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Search Self-play: Pushing the Frontier of Agent Capability without Supervision</title>
|
||
<link>https://arxiv.org/abs/2510.18821</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18821.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Hongliang Lu, Yuhang Wen, Pengyu Cheng, Ruijin Ding, Haotian Xu, Jiaqi Guo, Chutian Wang, Haonan Chen, Xiaoxi Jiang, Guanjun Jiang</p><p><b>Upvotes:</b> 14</p><p><b>Summary:</b> Reinforcement learning with verifiable rewards (RLVR) has become the mainstream technique for training LLM agents. However, RLVR highly depends on well-crafted task queries and corresponding ground-truth answers to provide accurate rewards, which requires massive human efforts and hinders the RL scaling processes, especially under agentic scenarios. Although a few recent works explore task synthesis methods, the difficulty of generated agentic tasks can hardly be controlled to provide effective RL training advantages. To achieve agentic RLVR with higher scalability, we explore self-play training for deep search agents, in which the learning LLM utilizes multi-turn search engine calling and acts simultaneously as both a task proposer and a problem solver. The task proposer aims to generate deep search queries with well-defined ground-truth answers and increasing task difficulty. The problem solver tries to handle the generated search queries and output the correct answer predictions. To ensure that each generated search query has accurate ground truth, we collect all the searching results from the proposer's trajectory as external knowledge, then conduct retrieval-augmentation generation (RAG) to test whether the proposed query can be correctly answered with all necessary search documents provided. In this search self-play (SSP) game, the proposer and the solver co-evolve their agent capabilities through both competition and cooperation. With substantial experimental results, we find that SSP can significantly improve search agents' performance uniformly on various benchmarks without any supervision under both from-scratch and continuous RL training setups. The code is at https://github.com/Alibaba-Quark/SSP.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18821</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:19:35 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Towards Faithful and Controllable Personalization via Critique-Post-Edit Reinforcement Learning</title>
|
||
<link>https://arxiv.org/abs/2510.18849</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18849.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Chenghao Zhu, Meiling Tao, Tiannan Wang, Dongyi Ding, Yuchen Eleanor Jiang, Wangchunshu Zhou</p><p><b>Upvotes:</b> 19</p><p><b>Summary:</b> Faithfully personalizing large language models (LLMs) to align with individual user preferences is a critical but challenging task. While supervised fine-tuning (SFT) quickly reaches a performance plateau, standard reinforcement learning from human feedback (RLHF) also struggles with the nuances of personalization. Scalar-based reward models are prone to reward hacking which leads to verbose and superficially personalized responses. To address these limitations, we propose Critique-Post-Edit, a robust reinforcement learning framework that enables more faithful and controllable personalization. Our framework integrates two key components: (1) a Personalized Generative Reward Model (GRM) that provides multi-dimensional scores and textual critiques to resist reward hacking, and (2) a Critique-Post-Edit mechanism where the policy model revises its own outputs based on these critiques for more targeted and efficient learning. Under a rigorous length-controlled evaluation, our method substantially outperforms standard PPO on personalization benchmarks. Personalized Qwen2.5-7B achieves an average 11\% win-rate improvement, and personalized Qwen2.5-14B model surpasses the performance of GPT-4.1. These results demonstrate a practical path to faithful, efficient, and controllable personalization.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18849</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:40:03 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Model</title>
|
||
<link>https://arxiv.org/abs/2510.18855</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18855.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ling Team, Anqi Shen, Baihui Li, Bin Hu, Bin Jing, Cai Chen, Chao Huang, Chao Zhang, Chaokun Yang, Cheng Lin, Chengyao Wen, Congqi Li, Deng Zhao, Dingbo Yuan, Donghai You, Fagui Mao, Fanzhuang Meng, Feng Xu, Guojie Li, Guowei Wang, Hao Dai, Haonan Zheng, Hong Liu, Jia Guo, Jiaming Liu, Jian Liu, Jianhao Fu, Jiannan Shi, Jianwen Wang, Jianxin Lai, Jin Yang, Jun Mei, Jun Zhou, Junbo Zhao, Junping Zhao, Kuan Xu, Le Su, Lei Chen, Li Tang, Liang Jiang, Liangcheng Fu, Lianhao Xu, Linfeng Shi, Lisha Liao, Longfei Zheng, Meng Li, Mingchun Chen, Qi Zuo, Qiang Cheng, Qianggang Cao, Qitao Shi, Quanrui Guo, Senlin Zhu, Shaofei Wang, Shaomian Zheng, Shuaicheng Li, Shuwei Gu, Siba Chen, Tao Wu, Tao Zhang, Tianyu Zhang, Tianyu Zhou, Tiwei Bie, Tongkai Yang, Wang Hong, Wang Ren, Weihua Chen, Wenbo Yu, Wengang Zheng, Xiangchun Wang, Xiaodong Yan, Xiaopei Wan, Xin Zhao, Xinyu Kong, Xinyu Tang, Xudong Han, Xudong Wang, Xuemin Yang, Xueyu Hu, Yalin Zhang, Yan Sun, Yicheng Shan, Yilong Wang, Yingying Xu, Yongkang Liu, Yongzhen Guo, Yuanyuan Wang, Yuchen Yan, Yuefan Wang, Yuhong Guo, Zehuan Li, Zhankai Xu, Zhe Li, Zhenduo Zhang, Zhengke Gui, Zhenxuan Pan, Zhenyu Huang, Zhenzhong Lan, Zhiqiang Ding, Zhiqiang Zhang, Zhixun Li, Zhizhen Liu, Zihao Wang, Zujie Wen</p><p><b>Upvotes:</b> 57</p><p><b>Summary:</b> We present Ring-1T, the first open-source, state-of-the-art thinking model with a trillion-scale parameter. It features 1 trillion total parameters and activates approximately 50 billion per token. Training such models at a trillion-parameter scale introduces unprecedented challenges, including train-inference misalignment, inefficiencies in rollout processing, and bottlenecks in the RL system. To address these, we pioneer three interconnected innovations: (1) IcePop stabilizes RL training via token-level discrepancy masking and clipping, resolving instability from training-inference mismatches; (2) C3PO++ improves resource utilization for long rollouts under a token budget by dynamically partitioning them, thereby obtaining high time efficiency; and (3) ASystem, a high-performance RL framework designed to overcome the systemic bottlenecks that impede trillion-parameter model training. Ring-1T delivers breakthrough results across critical benchmarks: 93.4 on AIME-2025, 86.72 on HMMT-2025, 2088 on CodeForces, and 55.94 on ARC-AGI-v1. Notably, it attains a silver medal-level result on the IMO-2025, underscoring its exceptional reasoning capabilities. By releasing the complete 1T parameter MoE model to the community, we provide the research community with direct access to cutting-edge reasoning capabilities. This contribution marks a significant milestone in democratizing large-scale reasoning intelligence and establishes a new baseline for open-source model performance.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18855</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:46:14 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>LightMem: Lightweight and Efficient Memory-Augmented Generation</title>
|
||
<link>https://arxiv.org/abs/2510.18866</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18866.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jizhan Fang, Xinle Deng, Haoming Xu, Ziyan Jiang, Yuqi Tang, Ziwen Xu, Shumin Deng, Yunzhi Yao, Mengru Wang, Shuofei Qiao, Huajun Chen, Ningyu Zhang</p><p><b>Upvotes:</b> 101</p><p><b>Summary:</b> Despite their remarkable capabilities, Large Language Models (LLMs) struggle to effectively leverage historical interaction information in dynamic and complex environments. Memory systems enable LLMs to move beyond stateless interactions by introducing persistent information storage, retrieval, and utilization mechanisms. However, existing memory systems often introduce substantial time and computational overhead. To this end, we introduce a new memory system called LightMem, which strikes a balance between the performance and efficiency of memory systems. Inspired by the Atkinson-Shiffrin model of human memory, LightMem organizes memory into three complementary stages. First, cognition-inspired sensory memory rapidly filters irrelevant information through lightweight compression and groups information according to their topics. Next, topic-aware short-term memory consolidates these topic-based groups, organizing and summarizing content for more structured access. Finally, long-term memory with sleep-time update employs an offline procedure that decouples consolidation from online inference. Experiments on LongMemEval with GPT and Qwen backbones show that LightMem outperforms strong baselines in accuracy (up to 10.9% gains) while reducing token usage by up to 117x, API calls by up to 159x, and runtime by over 12x. The code is available at https://github.com/zjunlp/LightMem.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18866</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:58:17 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DSI-Bench: A Benchmark for Dynamic Spatial Intelligence</title>
|
||
<link>https://arxiv.org/abs/2510.18873</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18873.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, Minjie Hong, Zhou Zhao</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Reasoning about dynamic spatial relationships is essential, as both observers and objects often move simultaneously. Although vision-language models (VLMs) and visual expertise models excel in 2D tasks and static scenarios, their ability to fully understand dynamic 3D scenarios remains limited. We introduce Dynamic Spatial Intelligence and propose DSI-Bench, a benchmark with nearly 1,000 dynamic videos and over 1,700 manually annotated questions covering nine decoupled motion patterns of observers and objects. Spatially and temporally symmetric designs reduce biases and enable systematic evaluation of models' reasoning about self-motion and object motion. Our evaluation of 14 VLMs and expert models reveals key limitations: models often conflate observer and object motion, exhibit semantic biases, and fail to accurately infer relative relationships in dynamic scenarios. Our DSI-Bench provides valuable findings and insights about the future development of general and expertise models with dynamic spatial intelligence.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18873</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:59:36 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ProfBench: Multi-Domain Rubrics requiring Professional Knowledge to Answer and Judge</title>
|
||
<link>https://arxiv.org/abs/2510.18941</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18941.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhilin Wang, Jaehun Jung, Ximing Lu, Shizhe Diao, Ellie Evans, Jiaqi Zeng, Pavlo Molchanov, Yejin Choi, Jan Kautz, Yi Dong</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Evaluating progress in large language models (LLMs) is often constrained by the challenge of verifying responses, limiting assessments to tasks like mathematics, programming, and short-form question-answering. However, many real-world applications require evaluating LLMs in processing professional documents, synthesizing information, and generating comprehensive reports in response to user queries. We introduce ProfBench: a set of over 7000 response-criterion pairs as evaluated by human-experts with professional knowledge across Physics PhD, Chemistry PhD, Finance MBA and Consulting MBA. We build robust and affordable LLM-Judges to evaluate ProfBench rubrics, by mitigating self-enhancement bias and reducing the cost of evaluation by 2-3 orders of magnitude, to make it fair and accessible to the broader community. Our findings reveal that ProfBench poses significant challenges even for state-of-the-art LLMs, with top-performing models like GPT-5-high achieving only 65.9\% overall performance. Furthermore, we identify notable performance disparities between proprietary and open-weight models and provide insights into the role that extended thinking plays in addressing complex, professional-domain tasks. Data: https://huggingface.co/datasets/nvidia/ProfBench and Code: https://github.com/NVlabs/ProfBench</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18941</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:59:44 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Grasp Any Region: Towards Precise, Contextual Pixel Understanding for Multimodal LLMs</title>
|
||
<link>https://arxiv.org/abs/2510.18876</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.18876.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Haochen Wang, Yuhao Wang, Tao Zhang, Yikang Zhou, Yanwei Li, Jiacong Wang, Ye Tian, Jiahao Meng, Zilong Huang, Guangcan Mai, Anran Wang, Yunhai Tong, Zhuochen Wang, Xiangtai Li, Zhaoxiang Zhang</p><p><b>Upvotes:</b> 34</p><p><b>Summary:</b> While Multimodal Large Language Models (MLLMs) excel at holistic understanding, they struggle in capturing the dense world with complex scenes, requiring fine-grained analysis of intricate details and object inter-relationships. Region-level MLLMs have been a promising step. However, previous attempts are generally optimized to understand given regions in isolation, neglecting crucial global contexts. To address this, we introduce Grasp Any Region (GAR) for comprehen- sive region-level visual understanding. Empowered by an effective RoI-aligned feature replay technique, GAR supports (1) precise perception by leveraging necessary global contexts, and (2) modeling interactions between multiple prompts. Together, it then naturally achieves (3) advanced compositional reasoning to answer specific free-form questions about any region, shifting the paradigm from passive description to active dialogue. Moreover, we construct GAR-Bench, which not only provides a more accurate evaluation of single-region comprehension, but also, more importantly, measures interactions and complex reasoning across multiple regions. Extensive experiments have demonstrated that GAR-1B not only maintains the state-of-the-art captioning capabilities, e.g., outperforming DAM-3B +4.5 on DLC-Bench, but also excels at modeling relationships between multiple prompts with advanced comprehension capabilities, even surpassing InternVL3-78B on GAR-Bench-VQA. More importantly, our zero-shot GAR-8B even outperforms in-domain VideoRefer-7B on VideoRefer-BenchQ, indicating its strong capabilities can be easily transferred to videos.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.18876</guid>
|
||
<pubDate>Tue, 21 Oct 2025 17:59:59 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Are they lovers or friends? Evaluating LLMs' Social Reasoning in English and Korean Dialogues</title>
|
||
<link>https://arxiv.org/abs/2510.19028</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19028.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Eunsu Kim, Junyeong Park, Juhyun Oh, Kiwoong Park, Seyoung Song, A. Seza Dogruoz, Najoung Kim, Alice Oh</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> As large language models (LLMs) are increasingly used in human-AI interactions, their social reasoning capabilities in interpersonal contexts are critical. We introduce SCRIPTS, a 1k-dialogue dataset in English and Korean, sourced from movie scripts. The task involves evaluating models' social reasoning capability to infer the interpersonal relationships (e.g., friends, sisters, lovers) between speakers in each dialogue. Each dialogue is annotated with probabilistic relational labels (Highly Likely, Less Likely, Unlikely) by native (or equivalent) Korean and English speakers from Korea and the U.S. Evaluating nine models on our task, current proprietary LLMs achieve around 75-80% on the English dataset, whereas their performance on Korean drops to 58-69%. More strikingly, models select Unlikely relationships in 10-25% of their responses. Furthermore, we find that thinking models and chain-of-thought prompting, effective for general reasoning, provide minimal benefits for social reasoning and occasionally amplify social biases. Our findings reveal significant limitations in current LLMs' social reasoning capabilities, highlighting the need for efforts to develop socially-aware language models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19028</guid>
|
||
<pubDate>Tue, 21 Oct 2025 19:12:47 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Steering Autoregressive Music Generation with Recursive Feature Machines</title>
|
||
<link>https://arxiv.org/abs/2510.19127</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19127.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Daniel Zhao, Daniel Beaglehole, Taylor Berg-Kirkpatrick, Julian McAuley, Zachary Novack</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Controllable music generation remains a significant challenge, with existing methods often requiring model retraining or introducing audible artifacts. We introduce MusicRFM, a framework that adapts Recursive Feature Machines (RFMs) to enable fine-grained, interpretable control over frozen, pre-trained music models by directly steering their internal activations. RFMs analyze a model's internal gradients to produce interpretable "concept directions", or specific axes in the activation space that correspond to musical attributes like notes or chords. We first train lightweight RFM probes to discover these directions within MusicGen's hidden states; then, during inference, we inject them back into the model to guide the generation process in real-time without per-step optimization. We present advanced mechanisms for this control, including dynamic, time-varying schedules and methods for the simultaneous enforcement of multiple musical properties. Our method successfully navigates the trade-off between control and generation quality: we can increase the accuracy of generating a target musical note from 0.23 to 0.82, while text prompt adherence remains within approximately 0.02 of the unsteered baseline, demonstrating effective control with minimal impact on prompt fidelity. We release code to encourage further exploration on RFMs in the music domain.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19127</guid>
|
||
<pubDate>Tue, 21 Oct 2025 23:23:14 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Loopholing Discrete Diffusion: Deterministic Bypass of the Sampling Wall</title>
|
||
<link>https://arxiv.org/abs/2510.19304</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19304.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Mingyu Jo, Jaesik Yoon, Justin Deschenaux, Caglar Gulcehre, Sungjin Ahn</p><p><b>Upvotes:</b> 22</p><p><b>Summary:</b> Discrete diffusion models offer a promising alternative to autoregressive generation through parallel decoding, but they suffer from a sampling wall: once categorical sampling occurs, rich distributional information collapses into one-hot vectors and cannot be propagated across steps, forcing subsequent steps to operate with limited information. To mitigate this problem, we introduce Loopholing, a novel and simple mechanism that preserves this information via a deterministic latent pathway, leading to Loopholing Discrete Diffusion Models (LDDMs). Trained efficiently with a self-conditioning strategy, LDDMs achieve substantial gains-reducing generative perplexity by up to 61% over prior baselines, closing (and in some cases surpassing) the gap with autoregressive models, and producing more coherent text. Applied to reasoning tasks, LDDMs also improve performance on arithmetic benchmarks such as Countdown and Game of 24. These results also indicate that loopholing mitigates idle steps and oscillations, providing a scalable path toward high-quality non-autoregressive text generation.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19304</guid>
|
||
<pubDate>Wed, 22 Oct 2025 07:08:47 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Unified Reinforcement and Imitation Learning for Vision-Language Models</title>
|
||
<link>https://arxiv.org/abs/2510.19307</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19307.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Byung-Kwan Lee, Ryo Hachiuma, Yong Man Ro, Yu-Chiang Frank Wang, Yueh-Hua Wu</p><p><b>Upvotes:</b> 22</p><p><b>Summary:</b> Vision-Language Models (VLMs) have achieved remarkable progress, yet their large scale often renders them impractical for resource-constrained environments. This paper introduces Unified Reinforcement and Imitation Learning (RIL), a novel and efficient training algorithm designed to create powerful, lightweight VLMs. RIL distinctively combines the strengths of reinforcement learning with adversarial imitation learning. This enables smaller student VLMs not only to mimic the sophisticated text generation of large teacher models but also to systematically improve their generative capabilities through reinforcement signals. Key to our imitation framework is an LLM-based discriminator that adeptly distinguishes between student and teacher outputs, complemented by guidance from multiple large teacher VLMs to ensure diverse learning. This unified learning strategy, leveraging both reinforcement and imitation, empowers student models to achieve significant performance gains, making them competitive with leading closed-source VLMs. Extensive experiments on diverse vision-language benchmarks demonstrate that RIL significantly narrows the performance gap with state-of-the-art open- and closed-source VLMs and, in several instances, surpasses them.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19307</guid>
|
||
<pubDate>Wed, 22 Oct 2025 07:12:14 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>KORE: Enhancing Knowledge Injection for Large Multimodal Models via Knowledge-Oriented Augmentations and Constraints</title>
|
||
<link>https://arxiv.org/abs/2510.19316</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19316.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, Qing Li</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Large Multimodal Models encode extensive factual knowledge in their pre-trained weights. However, its knowledge remains static and limited, unable to keep pace with real-world developments, which hinders continuous knowledge acquisition. Effective knowledge injection thus becomes critical, involving two goals: knowledge adaptation (injecting new knowledge) and knowledge retention (preserving old knowledge). Existing methods often struggle to learn new knowledge and suffer from catastrophic forgetting. To address this, we propose KORE, a synergistic method of KnOwledge-oRientEd augmentations and constraints for injecting new knowledge into large multimodal models while preserving old knowledge. Unlike general text or image data augmentation, KORE automatically converts individual knowledge items into structured and comprehensive knowledge to ensure that the model accurately learns new knowledge, enabling accurate adaptation. Meanwhile, KORE stores previous knowledge in the covariance matrix of LMM's linear layer activations and initializes the adapter by projecting the original weights into the matrix's null space, defining a fine-tuning direction that minimizes interference with previous knowledge, enabling powerful retention. Extensive experiments on various LMMs, including LLaVA-v1.5-7B, LLaVA-v1.5-13B, and Qwen2.5-VL-7B, show that KORE achieves superior new knowledge injection performance and effectively mitigates catastrophic forgetting.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19316</guid>
|
||
<pubDate>Wed, 22 Oct 2025 07:26:55 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents</title>
|
||
<link>https://arxiv.org/abs/2510.19336</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19336.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kai Shi, Jun Yang, Ni Yang, Binqiang Pan, Qingsong Xie, Chao Zhang, Zhenyu Yang, Tianhuang Su, Haonan Lu</p><p><b>Upvotes:</b> 16</p><p><b>Summary:</b> Mobile Phone Agents (MPAs) have emerged as a promising research direction due to their broad applicability across diverse scenarios. While Multimodal Large Language Models (MLLMs) serve as the foundation for MPAs, their effectiveness in handling multiple mobile phone tasks simultaneously remains limited. Although multitask supervised fine-tuning (SFT) is widely adopted for multitask learning, existing approaches struggle to determine optimal training data compositions for peak performance. To address this challenge, we propose DaMo (Data Mixture Optimizer) - a novel solution employing a trainable network that predicts optimal data mixtures by forecasting downstream task performance for any given dataset ratio. To support comprehensive evaluation, we introduce PhoneAgentBench, the first specialized benchmark to evaluate MLLMs on multimodal mobile phone tasks, comprising 1235 QA pairs spanning diverse real-world industrial mobile application scenarios. Demonstrating strong predictive capability (R^2=0.81) in small-scale pilot experiments, DaMo efficiently extrapolates optimal data mixing configurations. Our results show DaMo achieves a 3.38% performance improvement on PhoneAgentBench compared to alternative methods. Furthermore, extensive experiments across established benchmarks including BFCL-v3, MME-Reasoning, MME-Perception, and OCRBench reveal DaMo's superior generalization, outperforming other approaches by 2.57% in terms of average score. When used solely for MLLM optimization on the BFCL-v3 task, DaMo improves the metrics by 12.47% than other methods. Notably, DaMo maintains robust scalability, preserving its effectiveness when applied to other model architectures. The code and dataset are available at https://github.com/OPPO-Mente-Lab/DaMo.git</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19336</guid>
|
||
<pubDate>Wed, 22 Oct 2025 07:57:59 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Every Attention Matters: An Efficient Hybrid Architecture for Long-Context Reasoning</title>
|
||
<link>https://arxiv.org/abs/2510.19338</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19338.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ling Team, Bin Han, Caizhi Tang, Chen Liang, Donghao Zhang, Fan Yuan, Feng Zhu, Jie Gao, Jingyu Hu, Longfei Li, Meng Li, Mingyang Zhang, Peijie Jiang, Peng Jiao, Qian Zhao, Qingyuan Yang, Wenbo Shen, Xinxing Yang, Yalin Zhang, Yankun Ren, Yao Zhao, Yibo Cao, Yixuan Sun, Yue Zhang, Yuchen Fang, Zibin Lin, Zixuan Cheng, Jun Zhou</p><p><b>Upvotes:</b> 92</p><p><b>Summary:</b> In this technical report, we present the Ring-linear model series, specifically including Ring-mini-linear-2.0 and Ring-flash-linear-2.0. Ring-mini-linear-2.0 comprises 16B parameters and 957M activations, while Ring-flash-linear-2.0 contains 104B parameters and 6.1B activations. Both models adopt a hybrid architecture that effectively integrates linear attention and softmax attention, significantly reducing I/O and computational overhead in long-context inference scenarios. Compared to a 32 billion parameter dense model, this series reduces inference cost to 1/10, and compared to the original Ring series, the cost is also reduced by over 50%. Furthermore, through systematic exploration of the ratio between different attention mechanisms in the hybrid architecture, we have identified the currently optimal model structure. Additionally, by leveraging our self-developed high-performance FP8 operator library-linghe, overall training efficiency has been improved by 50%. Benefiting from the high alignment between the training and inference engine operators, the models can undergo long-term, stable, and highly efficient optimization during the reinforcement learning phase, consistently maintaining SOTA performance across multiple challenging complex reasoning benchmarks.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19338</guid>
|
||
<pubDate>Wed, 22 Oct 2025 07:59:38 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>LoongRL:Reinforcement Learning for Advanced Reasoning over Long Contexts</title>
|
||
<link>https://arxiv.org/abs/2510.19363</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19363.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Siyuan Wang, Gaokai Zhang, Li Lyna Zhang, Ning Shang, Fan Yang, Dongyao Chen, Mao Yang</p><p><b>Upvotes:</b> 54</p><p><b>Summary:</b> Reasoning over long contexts is essential for large language models. While reinforcement learning (RL) enhances short-context reasoning by inducing "Aha" moments in chain-of-thought, the advanced thinking patterns required for long-context reasoning remain largely unexplored, and high-difficulty RL data are scarce. In this paper, we introduce LoongRL, a data-driven RL method for advanced long-context reasoning. Central to LoongRL is KeyChain, a synthesis approach that transforms short multi-hop QA into high-difficulty long-context tasks by inserting UUID chains that hide the true question among large collections of distracting documents. Solving these tasks requires the model to trace the correct chain step-by-step, identify the true question, retrieve relevant facts and reason over them to answer correctly. RL training on KeyChain data induces an emergent plan-retrieve-reason-recheck reasoning pattern that generalizes far beyond training length. Models trained at 16K effectively solve 128K tasks without prohibitive full-length RL rollout costs. On Qwen2.5-7B and 14B, LoongRL substantially improves long-context multi-hop QA accuracy by +23.5% and +21.1% absolute gains. The resulting LoongRL-14B reaches a score of 74.2, rivaling much larger frontier models such as o3-mini (74.5) and DeepSeek-R1 (74.9). It also improves long-context retrieval, passes all 128K needle-in-a-haystack stress tests, and preserves short-context reasoning capabilities.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19363</guid>
|
||
<pubDate>Wed, 22 Oct 2025 08:35:28 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ColorAgent: Building A Robust, Personalized, and Interactive OS Agent</title>
|
||
<link>https://arxiv.org/abs/2510.19386</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19386.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ning Li, Qiqiang Lin, Zheng Wu, Xiaoyun Mo, Weiming Zhang, Yin Zhao, Xiangmou Qu, Jiamu Zhou, Jun Wang, Congmin Zheng, Yuanyi Song, Hongjiang Chen, Heyuan Huang, Jihong Wang, Jiaxin Yin, Jingwei Yu, Junwei Liao, Qiuying Peng, Xingyu Lou, Jun Wang, Weiwen Liu, Zhuosheng Zhang, Weinan Zhang</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> With the advancements in hardware, software, and large language model technologies, the interaction between humans and operating systems has evolved from the command-line interface to the rapidly emerging AI agent interactions. Building an operating system (OS) agent capable of executing user instructions and faithfully following user desires is becoming a reality. In this technical report, we present ColorAgent, an OS agent designed to engage in long-horizon, robust interactions with the environment while also enabling personalized and proactive user interaction. To enable long-horizon interactions with the environment, we enhance the model's capabilities through step-wise reinforcement learning and self-evolving training, while also developing a tailored multi-agent framework that ensures generality, consistency, and robustness. In terms of user interaction, we explore personalized user intent recognition and proactive engagement, positioning the OS agent not merely as an automation tool but as a warm, collaborative partner. We evaluate ColorAgent on the AndroidWorld and AndroidLab benchmarks, achieving success rates of 77.2% and 50.7%, respectively, establishing a new state of the art. Nonetheless, we note that current benchmarks are insufficient for a comprehensive evaluation of OS agents and propose further exploring directions in future work, particularly in the areas of evaluation paradigms, agent collaboration, and security. Our code is available at https://github.com/MadeAgents/mobile-use.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19386</guid>
|
||
<pubDate>Wed, 22 Oct 2025 09:02:48 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>GigaBrain-0: A World Model-Powered Vision-Language-Action Model</title>
|
||
<link>https://arxiv.org/abs/2510.19430</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19430.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> GigaBrain Team, Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Haoyun Li, Jie Li, Jiagang Zhu, Lv Feng, Peng Li, Qiuping Deng, Runqi Ouyang, Wenkang Qin, Xinze Chen, Xiaofeng Wang, Yang Wang, Yifan Li, Yilong Li, Yiran Ding, Yuan Xu, Yun Ye, Yukun Zhou, Zhehao Dong, Zhenan Wang, Zhichao Liu, Zheng Zhu</p><p><b>Upvotes:</b> 37</p><p><b>Summary:</b> Training Vision-Language-Action (VLA) models for generalist robots typically requires large-scale real-world robot data, which is expensive and time-consuming to collect. The inefficiency of physical data collection severely limits the scalability, and generalization capacity of current VLA systems. To address this challenge, we introduce GigaBrain-0, a novel VLA foundation model empowered by world model-generated data (e.g., video generation, real2real transfer, human transfer, view transfer, sim2real transfer data). By leveraging world models to generate diverse data at scale, GigaBrain-0 significantly reduces reliance on real robot data while improving cross-task generalization. Our approach further improves policy robustness through RGBD input modeling and embodied Chain-of-Thought (CoT) supervision, enabling the model to reason about spatial geometry, object states, and long-horizon dependencies during task execution. This leads to substantial gains in real-world performance on dexterous, long-horizon, and mobile manipulation tasks. Extensive experiments demonstrate that GigaBrain-0 achieves superior generalization across variations in appearances (e.g., textures, colors), object placements, and camera viewpoints. Additionally, we present GigaBrain-0-Small, an optimized lightweight variant designed to run efficiently on devices such as the NVIDIA Jetson AGX Orin.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19430</guid>
|
||
<pubDate>Wed, 22 Oct 2025 09:57:13 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>VideoAgentTrek: Computer Use Pretraining from Unlabeled Videos</title>
|
||
<link>https://arxiv.org/abs/2510.19488</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19488.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Dunjie Lu, Yiheng Xu, Junli Wang, Haoyuan Wu, Xinyuan Wang, Zekun Wang, Junlin Yang, Hongjin Su, Jixuan Chen, Junda Chen, Yuchen Mao, Jingren Zhou, Junyang Lin, Binyuan Hui, Tao Yu</p><p><b>Upvotes:</b> 18</p><p><b>Summary:</b> Training computer-use agents requires massive amounts of GUI interaction data, but manually annotating action trajectories at scale is prohibitively expensive. We present VideoAgentTrek, a scalable pipeline that automatically mines training data from publicly available screen-recorded videos at web scale, eliminating the need for manual annotation. Our approach addresses a key challenge: raw videos contain implicit demonstrations but lack explicit action labels. To solve this, we develop Video2Action, an inverse dynamics module (IDM) with two components: (1) a video grounding model that detects and localizes GUI actions with precise temporal boundaries and context, and (2) an action-content recognizer that extracts structured parameters like click coordinates and typed text with high fidelity. Applied to 39,000 YouTube tutorial videos, our pipeline generates 1.52 million interaction steps automatically. We leverage this data through continued pretraining followed by supervised fine-tuning. On OSWorld-Verified, our approach improves task success rates from 9.3% (SFT-only baseline) to 15.8%, a 70% relative improvement. On AgentNetBench, step accuracy increases from 64.1% to 69.3%. Our results demonstrate that passive internet videos can be transformed into high-quality supervision for computer-use agents, providing a scalable alternative to expensive manual annotation.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19488</guid>
|
||
<pubDate>Wed, 22 Oct 2025 11:25:48 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Decomposed Attention Fusion in MLLMs for Training-Free Video Reasoning Segmentation</title>
|
||
<link>https://arxiv.org/abs/2510.19592</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19592.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Su Ho Han, Jeongseok Hyun, Pilhyeon Lee, Minho Shim, Dongyoon Wee, Seon Joo Kim</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Multimodal large language models (MLLMs) demonstrate strong video understanding by attending to visual tokens relevant to textual queries. To directly adapt this for localization in a training-free manner, we cast video reasoning segmentation as a video QA task and extract attention maps via rollout mechanism. However, raw attention maps are noisy and poorly aligned with object regions. We propose Decomposed Attention Fusion (DecAF), which refines these maps through two mechanisms: (1) contrastive object-background fusion and (2) complementary video-frame fusion. This method suppresses irrelevant activations and enhances object-focused cues, enabling direct conversion of attention maps into coarse segmentation masks. In addition, we introduce attention-guided SAM2 prompting for obtaining fine-grained masks. Unlike existing methods that jointly train MLLMs with SAM, our method operates entirely without retraining. DecAF outperforms training-free methods and achieves performance comparable to training-based methods on both referring and reasoning VOS benchmarks. The code will be available at https://github.com/HYUNJS/DecAF.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19592</guid>
|
||
<pubDate>Wed, 22 Oct 2025 13:42:59 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Human-Agent Collaborative Paper-to-Page Crafting for Under $0.1</title>
|
||
<link>https://arxiv.org/abs/2510.19600</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19600.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Qianli Ma, Siyu Wang, Yilin Chen, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang</p><p><b>Upvotes:</b> 63</p><p><b>Summary:</b> In the quest for scientific progress, communicating research is as vital as the discovery itself. Yet, researchers are often sidetracked by the manual, repetitive chore of building project webpages to make their dense papers accessible. While automation has tackled static slides and posters, the dynamic, interactive nature of webpages has remained an unaddressed challenge. To bridge this gap, we reframe the problem, arguing that the solution lies not in a single command, but in a collaborative, hierarchical process. We introduce AutoPage, a novel multi-agent system that embodies this philosophy. AutoPage deconstructs paper-to-page creation into a coarse-to-fine pipeline from narrative planning to multimodal content generation and interactive rendering. To combat AI hallucination, dedicated "Checker" agents verify each step against the source paper, while optional human checkpoints ensure the final product aligns perfectly with the author's vision, transforming the system from a mere tool into a powerful collaborative assistant. To rigorously validate our approach, we also construct PageBench, the first benchmark for this new task. Experiments show AutoPage not only generates high-quality, visually appealing pages but does so with remarkable efficiency in under 15 minutes for less than \0.1. Code and dataset will be released at https://mqleet.github.io/AutoPage_ProjectPage/{Webpage}$.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19600</guid>
|
||
<pubDate>Wed, 22 Oct 2025 13:53:57 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>HSCodeComp: A Realistic and Expert-level Benchmark for Deep Search Agents in Hierarchical Rule Application</title>
|
||
<link>https://arxiv.org/abs/2510.19631</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19631.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yiqian Yang, Tian Lan, Qianghuai Jia, Li Zhu, Hui Jiang, Hang Zhu, Longyue Wang, Weihua Luo, Kaifu Zhang</p><p><b>Upvotes:</b> 24</p><p><b>Summary:</b> Effective deep search agents must not only access open-domain and domain-specific knowledge but also apply complex rules-such as legal clauses, medical manuals and tariff rules. These rules often feature vague boundaries and implicit logic relationships, making precise application challenging for agents. However, this critical capability is largely overlooked by current agent benchmarks. To fill this gap, we introduce HSCodeComp, the first realistic, expert-level e-commerce benchmark designed to evaluate deep search agents in hierarchical rule application. In this task, the deep reasoning process of agents is guided by these rules to predict 10-digit Harmonized System Code (HSCode) of products with noisy but realistic descriptions. These codes, established by the World Customs Organization, are vital for global supply chain efficiency. Built from real-world data collected from large-scale e-commerce platforms, our proposed HSCodeComp comprises 632 product entries spanning diverse product categories, with these HSCodes annotated by several human experts. Extensive experimental results on several state-of-the-art LLMs, open-source, and closed-source agents reveal a huge performance gap: best agent achieves only 46.8% 10-digit accuracy, far below human experts at 95.0%. Besides, detailed analysis demonstrates the challenges of hierarchical rule application, and test-time scaling fails to improve performance further.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19631</guid>
|
||
<pubDate>Wed, 22 Oct 2025 14:28:33 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders</title>
|
||
<link>https://arxiv.org/abs/2510.19779</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19779.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuezhou Hu, Jiaxin Guo, Xinyu Feng, Tuo Zhao</p><p><b>Upvotes:</b> 55</p><p><b>Summary:</b> Speculative Decoding (SD) accelerates large language model inference by employing a small draft model to generate predictions, which are then verified by a larger target model. The effectiveness of SD hinges on the alignment between these models, which is typically enhanced by Knowledge Distillation (KD). However, conventional KD methods aim to minimize the KL divergence between the draft and target models across all tokens, a goal that is misaligned with the true objective of SD, which is to maximize token acceptance rate. Therefore, draft models often struggle to fully assimilate the target model's knowledge due to capacity constraints, leading to suboptimal performance. To address this challenge, we propose AdaSPEC, a novel method that incorporates selective token filtering into the KD process. AdaSPEC utilizes a reference model to identify and filter out difficult-to-fit tokens, enabling the distillation of a draft model that better aligns with the target model on simpler tokens. This approach improves the overall token acceptance rate without compromising generation quality. We evaluate AdaSPEC across diverse tasks, including arithmetic reasoning, instruction-following, coding, and summarization, using model configurations of 31M/1.4B and 350M/2.7B parameters. Our results demonstrate that AdaSPEC consistently outperforms the state-of-the-art DistillSpec method, achieving higher acceptance rates across all tasks (up to 15\%). The code is publicly available at https://github.com/yuezhouhu/adaspec.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19779</guid>
|
||
<pubDate>Wed, 22 Oct 2025 17:13:00 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editing</title>
|
||
<link>https://arxiv.org/abs/2510.19808</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19808.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yusu Qian, Eli Bocek-Rivele, Liangchen Song, Jialing Tong, Yinfei Yang, Jiasen Lu, Wenze Hu, Zhe Gan</p><p><b>Upvotes:</b> 20</p><p><b>Summary:</b> Recent advances in multimodal models have demonstrated remarkable text-guided image editing capabilities, with systems like GPT-4o and Nano-Banana setting new benchmarks. However, the research community's progress remains constrained by the absence of large-scale, high-quality, and openly accessible datasets built from real images. We introduce Pico-Banana-400K, a comprehensive 400K-image dataset for instruction-based image editing. Our dataset is constructed by leveraging Nano-Banana to generate diverse edit pairs from real photographs in the OpenImages collection. What distinguishes Pico-Banana-400K from previous synthetic datasets is our systematic approach to quality and diversity. We employ a fine-grained image editing taxonomy to ensure comprehensive coverage of edit types while maintaining precise content preservation and instruction faithfulness through MLLM-based quality scoring and careful curation. Beyond single turn editing, Pico-Banana-400K enables research into complex editing scenarios. The dataset includes three specialized subsets: (1) a 72K-example multi-turn collection for studying sequential editing, reasoning, and planning across consecutive modifications; (2) a 56K-example preference subset for alignment research and reward model training; and (3) paired long-short editing instructions for developing instruction rewriting and summarization capabilities. By providing this large-scale, high-quality, and task-rich resource, Pico-Banana-400K establishes a robust foundation for training and benchmarking the next generation of text-guided image editing models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19808</guid>
|
||
<pubDate>Wed, 22 Oct 2025 17:43:15 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>olmOCR 2: Unit Test Rewards for Document OCR</title>
|
||
<link>https://arxiv.org/abs/2510.19817</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19817.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jake Poznanski, Luca Soldaini, Kyle Lo</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> We present olmOCR 2, the latest in our family of powerful OCR systems for converting digitized print documents, like PDFs, into clean, naturally ordered plain text. olmOCR 2 is powered by olmOCR-2-7B-1025, a specialized, 7B vision language model (VLM) trained using reinforcement learning with verifiable rewards (RLVR), where our rewards are a diverse set of binary unit tests. To scale unit test creation, we develop a pipeline for generating synthetic documents with diverse and challenging layouts, known ground-truth HTML source code, and extracted test cases. We show that RL training on these test cases results in state-of-the-art performance on olmOCR-Bench, our English-language OCR benchmark, with the largest improvements in math formula conversion, table parsing, and multi-column layouts compared to previous versions. We release our model, data and code under permissive open licenses.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19817</guid>
|
||
<pubDate>Wed, 22 Oct 2025 17:53:02 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets</title>
|
||
<link>https://arxiv.org/abs/2510.19944</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.19944.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiashi Feng, Xiu Li, Jing Lin, Jiahang Liu, Gaohong Liu, Weiqiang Lou, Su Ma, Guang Shi, Qinlong Wang, Jun Wang, Zhongcong Xu, Xuanyu Yi, Zihao Yu, Jianfeng Zhang, Yifan Zhu, Rui Chen, Jinxin Chi, Zixian Du, Li Han, Lixin Huang, Kaihua Jiang, Yuhan Li, Guan Luo, Shuguang Wang, Qianyi Wu, Fan Yang, Junyang Zhang, Xuanmeng Zhang</p><p><b>Upvotes:</b> 12</p><p><b>Summary:</b> Developing embodied AI agents requires scalable training environments that balance content diversity with physics accuracy. World simulators provide such environments but face distinct limitations: video-based methods generate diverse content but lack real-time physics feedback for interactive learning, while physics-based engines provide accurate dynamics but face scalability limitations from costly manual asset creation. We present Seed3D 1.0, a foundation model that generates simulation-ready 3D assets from single images, addressing the scalability challenge while maintaining physics rigor. Unlike existing 3D generation models, our system produces assets with accurate geometry, well-aligned textures, and realistic physically-based materials. These assets can be directly integrated into physics engines with minimal configuration, enabling deployment in robotic manipulation and simulation training. Beyond individual objects, the system scales to complete scene generation through assembling objects into coherent environments. By enabling scalable simulation-ready content creation, Seed3D 1.0 provides a foundation for advancing physics-based world simulators. Seed3D 1.0 is now available on https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?modelId=doubao-seed3d-1-0-250928&tab=Gen3D</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.19944</guid>
|
||
<pubDate>Wed, 22 Oct 2025 18:16:32 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seeking</title>
|
||
<link>https://arxiv.org/abs/2510.20168</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20168.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tian Lan, Bin Zhu, Qianghuai Jia, Junyang Ren, Haijun Li, Longyue Wang, Zhao Xu, Weihua Luo, Kaifu Zhang</p><p><b>Upvotes:</b> 23</p><p><b>Summary:</b> Current search agents fundamentally lack the ability to simultaneously perform deep reasoning over multi-hop retrieval and wide-scale information collection-a critical deficiency for real-world applications like comprehensive market analysis and business development. To bridge this gap, we introduce DeepWideSearch, the first benchmark explicitly designed to evaluate agents to integrate depth and width in information seeking. In DeepWideSearch, agents must process a large volume of data, each requiring deep reasoning over multi-hop retrieval paths. Specifically, we propose two methods to converse established datasets, resulting in a curated collection of 220 questions spanning 15 diverse domains. Extensive experiments demonstrate that even state-of-the-art agents achieve only 2.39% average success rate on DeepWideSearch, highlighting the substantial challenge of integrating depth and width search in information-seeking tasks. Furthermore, our error analysis reveals four failure modes: lack of reflection, overreliance on internal knowledge, insufficient retrieval, and context overflow-exposing key limitations in current agent architectures. We publicly release DeepWideSearch to catalyze future research on more capable and robust information-seeking agents.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20168</guid>
|
||
<pubDate>Thu, 23 Oct 2025 03:28:45 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values</title>
|
||
<link>https://arxiv.org/abs/2510.20187</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20187.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Dian Yu, Yulai Zhao, Kishan Panaganti, Linfeng Song, Haitao Mi, Dong Yu</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> We propose Reinforcement Learning with Explicit Human Values (RLEV), a method that aligns Large Language Model (LLM) optimization directly with quantifiable human value signals. While Reinforcement Learning with Verifiable Rewards (RLVR) effectively trains models in objective domains using binary correctness rewards, it overlooks that not all tasks are equally significant. RLEV extends this framework by incorporating human-defined value signals directly into the reward function. Using exam-style data with explicit ground-truth value labels, RLEV consistently outperforms correctness-only baselines across multiple RL algorithms and model scales. Crucially, RLEV policies not only improve value-weighted accuracy but also learn a value-sensitive termination policy: concise for low-value prompts, thorough for high-value ones. We demonstrate this behavior stems from value-weighted gradient amplification on end-of-sequence tokens. Ablation studies confirm the gain is causally linked to value alignment. RLEV remains robust under noisy value signals, such as difficulty-based labels, demonstrating that optimizing for an explicit utility function offers a practical path to aligning LLMs with human priorities.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20187</guid>
|
||
<pubDate>Thu, 23 Oct 2025 04:15:22 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ImpossibleBench: Measuring LLMs' Propensity of Exploiting Test Cases</title>
|
||
<link>https://arxiv.org/abs/2510.20270</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20270.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ziqian Zhong, Aditi Raghunathan, Nicholas Carlini</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> The tendency to find and exploit "shortcuts" to complete tasks poses significant risks for reliable assessment and deployment of large language models (LLMs). For example, an LLM agent with access to unit tests may delete failing tests rather than fix the underlying bug. Such behavior undermines both the validity of benchmark results and the reliability of real-world LLM coding assistant deployments. To quantify, study, and mitigate such behavior, we introduce ImpossibleBench, a benchmark framework that systematically measures LLM agents' propensity to exploit test cases. ImpossibleBench creates "impossible" variants of tasks from existing benchmarks like LiveCodeBench and SWE-bench by introducing direct conflicts between the natural-language specification and the unit tests. We measure an agent's "cheating rate" as its pass rate on these impossible tasks, where any pass necessarily implies a specification-violating shortcut. As a practical framework, ImpossibleBench is not just an evaluation but a versatile tool. We demonstrate its utility for: (1) studying model behaviors, revealing more fine-grained details of cheating behaviors from simple test modification to complex operator overloading; (2) context engineering, showing how prompt, test access and feedback loop affect cheating rates; and (3) developing monitoring tools, providing a testbed with verified deceptive solutions. We hope ImpossibleBench serves as a useful framework for building more robust and reliable LLM systems. Our implementation can be found at https://github.com/safety-research/impossiblebench.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20270</guid>
|
||
<pubDate>Thu, 23 Oct 2025 06:58:32 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidence</title>
|
||
<link>https://arxiv.org/abs/2510.20470</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20470.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kun Ouyang, Yuanxin Liu, Linli Yao, Yishuo Cai, Hao Zhou, Jie Zhou, Fandong Meng, Xu Sun</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> Video reasoning, which requires multi-step deduction across frames, remains a major challenge for multimodal large language models (MLLMs). While reinforcement learning (RL)-based methods enhance reasoning capabilities, they often rely on text-only chains that yield ungrounded or hallucinated conclusions. Conversely, frame-retrieval approaches introduce visual grounding but still struggle with inaccurate evidence localization. To address these challenges, we present Conan, a framework for evidence-grounded multi-step video reasoning. Conan identifies contextual and evidence frames, reasons over cross-frame clues, and adaptively decides when to conclude or explore further. To achieve this, we (1) construct Conan-91K, a large-scale dataset of automatically generated reasoning traces that includes frame identification, evidence reasoning, and action decision, and (2) design a multi-stage progressive cold-start strategy combined with an Identification-Reasoning-Action (AIR) RLVR training framework to jointly enhance multi-step visual reasoning. Extensive experiments on six multi-step reasoning benchmarks demonstrate that Conan surpasses the baseline Qwen2.5-VL-7B-Instruct by an average of over 10% in accuracy, achieving state-of-the-art performance. Furthermore, Conan generalizes effectively to long-video understanding tasks, validating its strong scalability and robustness.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20470</guid>
|
||
<pubDate>Thu, 23 Oct 2025 12:11:46 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Open-o3 Video: Grounded Video Reasoning with Explicit Spatio-Temporal Evidence</title>
|
||
<link>https://arxiv.org/abs/2510.20579</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20579.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jiahao Meng, Xiangtai Li, Haochen Wang, Yue Tan, Tao Zhang, Lingdong Kong, Yunhai Tong, Anran Wang, Zhiyang Teng, Yujing Wang, Zhuochen Wang</p><p><b>Upvotes:</b> 44</p><p><b>Summary:</b> Most video reasoning models only generate textual reasoning traces without indicating when and where key evidence appears. Recent models such as OpenAI-o3 have sparked wide interest in evidence-centered reasoning for images, yet extending this ability to videos is more challenging, as it requires joint temporal tracking and spatial localization across dynamic scenes. We introduce Open-o3 Video, a non-agent framework that integrates explicit spatio-temporal evidence into video reasoning, and carefully collect training data and design training strategies to address the aforementioned challenges. The model highlights key timestamps, objects, and bounding boxes alongside its answers, allowing reasoning to be grounded in concrete visual observations. To enable this functionality, we first curate and build two high-quality datasets, STGR-CoT-30k for SFT and STGR-RL-36k for RL, with carefully constructed temporal and spatial annotations, since most existing datasets offer either temporal spans for videos or spatial boxes on images, lacking unified spatio-temporal supervision and reasoning traces. Then, we adopt a cold-start reinforcement learning strategy with multiple specially designed rewards that jointly encourage answer accuracy, temporal alignment, and spatial precision. On V-STAR benchmark, Open-o3 Video achieves state-of-the-art performance, raising mAM by 14.4% and mLGM by 24.2% on the Qwen2.5-VL baseline. Consistent improvements are also observed on a broad range of video understanding benchmarks, including VideoMME, WorldSense, VideoMMMU, and TVGBench. Beyond accuracy, the reasoning traces produced by Open-o3 Video also provide valuable signals for test-time scaling, enabling confidence-aware verification and improving answer reliability.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20579</guid>
|
||
<pubDate>Thu, 23 Oct 2025 14:05:56 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>From Masks to Worlds: A Hitchhiker's Guide to World Models</title>
|
||
<link>https://arxiv.org/abs/2510.20668</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20668.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jinbin Bai, Yu Lei, Hecong Wu, Yuchen Zhu, Shufan Li, Yi Xin, Xiangtai Li, Molei Tao, Aditya Grover, Ming-Hsuan Yang</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> This is not a typical survey of world models; it is a guide for those who want to build worlds. We do not aim to catalog every paper that has ever mentioned a ``world model". Instead, we follow one clear road: from early masked models that unified representation learning across modalities, to unified architectures that share a single paradigm, then to interactive generative models that close the action-perception loop, and finally to memory-augmented systems that sustain consistent worlds over time. We bypass loosely related branches to focus on the core: the generative heart, the interactive loop, and the memory system. We show that this is the most promising path towards true world models.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20668</guid>
|
||
<pubDate>Thu, 23 Oct 2025 15:46:44 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>Thought Communication in Multiagent Collaboration</title>
|
||
<link>https://arxiv.org/abs/2510.20733</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20733.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yujia Zheng, Zhuokai Zhao, Zijian Li, Yaqi Xie, Mingze Gao, Lizhu Zhang, Kun Zhang</p><p><b>Upvotes:</b> 12</p><p><b>Summary:</b> Natural language has long enabled human cooperation, but its lossy, ambiguous, and indirect nature limits the potential of collective intelligence. While machines are not subject to these constraints, most LLM-based multi-agent systems still rely solely on natural language, exchanging tokens or their embeddings. To go beyond language, we introduce a new paradigm, thought communication, which enables agents to interact directly mind-to-mind, akin to telepathy. To uncover these latent thoughts in a principled way, we formalize the process as a general latent variable model, where agent states are generated by an unknown function of underlying thoughts. We prove that, in a nonparametric setting without auxiliary information, both shared and private latent thoughts between any pair of agents can be identified. Moreover, the global structure of thought sharing, including which agents share which thoughts and how these relationships are structured, can also be recovered with theoretical guarantees. Guided by the established theory, we develop a framework that extracts latent thoughts from all agents prior to communication and assigns each agent the relevant thoughts, along with their sharing patterns. This paradigm naturally extends beyond LLMs to all modalities, as most observational data arise from hidden generative processes. Experiments on both synthetic and real-world benchmarks validate the theory and demonstrate the collaborative advantages of thought communication. We hope this work illuminates the potential of leveraging the hidden world, as many challenges remain unsolvable through surface-level observation alone, regardless of compute or data scale.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20733</guid>
|
||
<pubDate>Thu, 23 Oct 2025 16:48:02 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion</title>
|
||
<link>https://arxiv.org/abs/2510.20766</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20766.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Noam Issachar, Guy Yariv, Sagie Benaim, Yossi Adi, Dani Lischinski, Raanan Fattal</p><p><b>Upvotes:</b> 26</p><p><b>Summary:</b> Diffusion Transformer models can generate images with remarkable fidelity and detail, yet training them at ultra-high resolutions remains extremely costly due to the self-attention mechanism's quadratic scaling with the number of image tokens. In this paper, we introduce Dynamic Position Extrapolation (DyPE), a novel, training-free method that enables pre-trained diffusion transformers to synthesize images at resolutions far beyond their training data, with no additional sampling cost. DyPE takes advantage of the spectral progression inherent to the diffusion process, where low-frequency structures converge early, while high-frequencies take more steps to resolve. Specifically, DyPE dynamically adjusts the model's positional encoding at each diffusion step, matching their frequency spectrum with the current stage of the generative process. This approach allows us to generate images at resolutions that exceed the training resolution dramatically, e.g., 16 million pixels using FLUX. On multiple benchmarks, DyPE consistently improves performance and achieves state-of-the-art fidelity in ultra-high-resolution image generation, with gains becoming even more pronounced at higher resolutions. Project page is available at https://noamissachar.github.io/DyPE/.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20766</guid>
|
||
<pubDate>Thu, 23 Oct 2025 17:42:14 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>AlphaFlow: Understanding and Improving MeanFlow Models</title>
|
||
<link>https://arxiv.org/abs/2510.20771</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20771.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Qu, Ivan Skorokhodov</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> MeanFlow has recently emerged as a powerful framework for few-step generative modeling trained from scratch, but its success is not yet fully understood. In this work, we show that the MeanFlow objective naturally decomposes into two parts: trajectory flow matching and trajectory consistency. Through gradient analysis, we find that these terms are strongly negatively correlated, causing optimization conflict and slow convergence. Motivated by these insights, we introduce alpha-Flow, a broad family of objectives that unifies trajectory flow matching, Shortcut Model, and MeanFlow under one formulation. By adopting a curriculum strategy that smoothly anneals from trajectory flow matching to MeanFlow, alpha-Flow disentangles the conflicting objectives, and achieves better convergence. When trained from scratch on class-conditional ImageNet-1K 256x256 with vanilla DiT backbones, alpha-Flow consistently outperforms MeanFlow across scales and settings. Our largest alpha-Flow-XL/2+ model achieves new state-of-the-art results using vanilla DiT backbones, with FID scores of 2.58 (1-NFE) and 2.15 (2-NFE).</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20771</guid>
|
||
<pubDate>Thu, 23 Oct 2025 17:45:06 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>ARGenSeg: Image Segmentation with Autoregressive Image Generation Model</title>
|
||
<link>https://arxiv.org/abs/2510.20803</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20803.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiaolong Wang, Lixiang Ru, Ziyuan Huang, Kaixiang Ji, Dandan Zheng, Jingdong Chen, Jun Zhou</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> We propose a novel AutoRegressive Generation-based paradigm for image Segmentation (ARGenSeg), achieving multimodal understanding and pixel-level perception within a unified framework. Prior works integrating image segmentation into multimodal large language models (MLLMs) typically employ either boundary points representation or dedicated segmentation heads. These methods rely on discrete representations or semantic prompts fed into task-specific decoders, which limits the ability of the MLLM to capture fine-grained visual details. To address these challenges, we introduce a segmentation framework for MLLM based on image generation, which naturally produces dense masks for target objects. We leverage MLLM to output visual tokens and detokenize them into images using an universal VQ-VAE, making the segmentation fully dependent on the pixel-level understanding of the MLLM. To reduce inference latency, we employ a next-scale-prediction strategy to generate required visual tokens in parallel. Extensive experiments demonstrate that our method surpasses prior state-of-the-art approaches on multiple segmentation datasets with a remarkable boost in inference speed, while maintaining strong understanding capabilities.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20803</guid>
|
||
<pubDate>Thu, 23 Oct 2025 17:58:26 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>LayerComposer: Interactive Personalized T2I via Spatially-Aware Layered Canvas</title>
|
||
<link>https://arxiv.org/abs/2510.20820</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20820.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Guocheng Gordon Qian, Ruihang Zhang, Tsai-Shien Chen, Yusuf Dalva, Anujraaj Argo Goyal, Willi Menapace, Ivan Skorokhodov, Meng Dong, Arpit Sahni, Daniil Ostashev, Ju Hu, Sergey Tulyakov, Kuan-Chieh Jackson Wang</p><p><b>Upvotes:</b> 7</p><p><b>Summary:</b> Despite their impressive visual fidelity, existing personalized generative models lack interactive control over spatial composition and scale poorly to multiple subjects. To address these limitations, we present LayerComposer, an interactive framework for personalized, multi-subject text-to-image generation. Our approach introduces two main contributions: (1) a layered canvas, a novel representation in which each subject is placed on a distinct layer, enabling occlusion-free composition; and (2) a locking mechanism that preserves selected layers with high fidelity while allowing the remaining layers to adapt flexibly to the surrounding context. Similar to professional image-editing software, the proposed layered canvas allows users to place, resize, or lock input subjects through intuitive layer manipulation. Our versatile locking mechanism requires no architectural changes, relying instead on inherent positional embeddings combined with a new complementary data sampling strategy. Extensive experiments demonstrate that LayerComposer achieves superior spatial control and identity preservation compared to the state-of-the-art methods in multi-subject personalized image generation.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20820</guid>
|
||
<pubDate>Thu, 23 Oct 2025 17:59:55 +0000</pubDate>
|
||
</item>
|
||
<item>
|
||
<title>HoloCine: Holistic Generation of Cinematic Multi-Shot Long Video Narratives</title>
|
||
<link>https://arxiv.org/abs/2510.20822</link>
|
||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2510.20822.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yihao Meng, Hao Ouyang, Yue Yu, Qiuyu Wang, Wen Wang, Ka Leong Cheng, Hanlin Wang, Yixuan Li, Cheng Chen, Yanhong Zeng, Yujun Shen, Huamin Qu</p><p><b>Upvotes:</b> 32</p><p><b>Summary:</b> State-of-the-art text-to-video models excel at generating isolated clips but fall short of creating the coherent, multi-shot narratives, which are the essence of storytelling. We bridge this "narrative gap" with HoloCine, a model that generates entire scenes holistically to ensure global consistency from the first shot to the last. Our architecture achieves precise directorial control through a Window Cross-Attention mechanism that localizes text prompts to specific shots, while a Sparse Inter-Shot Self-Attention pattern (dense within shots but sparse between them) ensures the efficiency required for minute-scale generation. Beyond setting a new state-of-the-art in narrative coherence, HoloCine develops remarkable emergent abilities: a persistent memory for characters and scenes, and an intuitive grasp of cinematic techniques. Our work marks a pivotal shift from clip synthesis towards automated filmmaking, making end-to-end cinematic creation a tangible future. Our code is available at: https://holo-cine.github.io/.</p></description>
|
||
<guid isPermaLink="false">https://arxiv.org/abs/2510.20822</guid>
|
||
<pubDate>Thu, 23 Oct 2025 17:59:59 +0000</pubDate>
|
||
</item>
|
||
</channel>
|
||
</rss>
|