bot: update RSS feed
This commit is contained in:
github-actions[bot]
210
feed.xml
210
feed.xml
@@ -7,132 +7,160 @@
|
|||||||
<docs>http://www.rssboard.org/rss-specification</docs>
|
<docs>http://www.rssboard.org/rss-specification</docs>
|
||||||
<generator>python-feedgen</generator>
|
<generator>python-feedgen</generator>
|
||||||
<language>en</language>
|
<language>en</language>
|
||||||
<lastBuildDate>Tue, 26 Aug 2025 00:02:14 +0000</lastBuildDate>
|
<lastBuildDate>Wed, 27 Aug 2025 00:02:35 +0000</lastBuildDate>
|
||||||
<item>
|
<item>
|
||||||
<title>Selective Contrastive Learning for Weakly Supervised Affordance Grounding</title>
|
<title>MV-RAG: Retrieval Augmented Multiview Diffusion</title>
|
||||||
<link>https://arxiv.org/abs/2508.07877</link>
|
<link>https://arxiv.org/abs/2508.16577</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.07877.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> WonJun Moon, Hyun Seok Seong, Jae-Pil Heo</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> Facilitating an entity's interaction with objects requires accurately identifying parts that afford specific actions. Weakly supervised affordance grounding (WSAG) seeks to imitate human learning from third-person demonstrations, where humans intuitively grasp functional parts without needing pixel-level annotations. To achieve this, grounding is typically learned using a shared classifier across images from different perspectives, along with distillation strategies incorporating part discovery process. However, since affordance-relevant parts are not always easily distinguishable, models primarily rely on classification, often focusing on common class-specific patterns that are unrelated to affordance. To address this limitation, we move beyond isolated part-level learning by introducing selective prototypical and pixel contrastive objectives that adaptively learn affordance-relevant cues at both the part and object levels, depending on the granularity of the available information. Initially, we find the action-associated objects in both egocentric (object-focused) and exocentric (third-person example) images by leveraging CLIP. Then, by cross-referencing the discovered objects of complementary views, we excavate the precise part-level affordance clues in each perspective. By consistently learning to distinguish affordance-relevant regions from affordance-irrelevant background context, our approach effectively shifts activation from irrelevant areas toward meaningful affordance cues. Experimental results demonstrate the effectiveness of our method. Codes are available at github.com/hynnsk/SelectiveCL.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16577.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yosef Dayani, Omer Benishu, Sagie Benaim</p><p><b>Upvotes:</b> 28</p><p><b>Summary:</b> Text-to-3D generation approaches have advanced significantly by leveraging pretrained 2D diffusion priors, producing high-quality and 3D-consistent outputs. However, they often fail to produce out-of-domain (OOD) or rare concepts, yielding inconsistent or inaccurate results. To this end, we propose MV-RAG, a novel text-to-3D pipeline that first retrieves relevant 2D images from a large in-the-wild 2D database and then conditions a multiview diffusion model on these images to synthesize consistent and accurate multiview outputs. Training such a retrieval-conditioned model is achieved via a novel hybrid strategy bridging structured multiview data and diverse 2D image collections. This involves training on multiview data using augmented conditioning views that simulate retrieval variance for view-specific reconstruction, alongside training on sets of retrieved real-world 2D images using a distinctive held-out view prediction objective: the model predicts the held-out view from the other views to infer 3D consistency from 2D data. To facilitate a rigorous OOD evaluation, we introduce a new collection of challenging OOD prompts. Experiments against state-of-the-art text-to-3D, image-to-3D, and personalization baselines show that our approach significantly improves 3D consistency, photorealism, and text adherence for OOD/rare concepts, while maintaining competitive performance on standard benchmarks.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.07877</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.16577</guid>
|
||||||
<pubDate>Mon, 11 Aug 2025 11:49:37 +0000</pubDate>
|
<pubDate>Fri, 22 Aug 2025 17:59:40 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Tasks</title>
|
<title>Beyond Memorization: Extending Reasoning Depth with Recurrence, Memory and Test-Time Compute Scaling</title>
|
||||||
<link>https://arxiv.org/abs/2508.08240</link>
|
<link>https://arxiv.org/abs/2508.16745</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.08240.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kaijun Wang, Liqin Lu, Mingyu Liu, Jianuo Jiang, Zeju Li, Bolin Zhang, Wancai Zheng, Xinyi Yu, Hao Chen, Chunhua Shen</p><p><b>Upvotes:</b> 34</p><p><b>Summary:</b> Language-guided long-horizon mobile manipulation has long been a grand challenge in embodied semantic reasoning, generalizable manipulation, and adaptive locomotion. Three fundamental limitations hinder progress: First, although large language models have improved spatial reasoning and task planning through semantic priors, existing implementations remain confined to tabletop scenarios, failing to address the constrained perception and limited actuation ranges of mobile platforms. Second, current manipulation strategies exhibit insufficient generalization when confronted with the diverse object configurations encountered in open-world environments. Third, while crucial for practical deployment, the dual requirement of maintaining high platform maneuverability alongside precise end-effector control in unstructured settings remains understudied. In this work, we present ODYSSEY, a unified mobile manipulation framework for agile quadruped robots equipped with manipulators, which seamlessly integrates high-level task planning with low-level whole-body control. To address the challenge of egocentric perception in language-conditioned tasks, we introduce a hierarchical planner powered by a vision-language model, enabling long-horizon instruction decomposition and precise action execution. At the control level, our novel whole-body policy achieves robust coordination across challenging terrains. We further present the first benchmark for long-horizon mobile manipulation, evaluating diverse indoor and outdoor scenarios. Through successful sim-to-real transfer, we demonstrate the system's generalization and robustness in real-world deployments, underscoring the practicality of legged manipulators in unstructured environments. Our work advances the feasibility of generalized robotic assistants capable of complex, dynamic tasks. Our project page: https://kaijwang.github.io/odyssey.github.io/</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16745.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ivan Rodkin, Daniil Orel, Konstantin Smirnov, Arman Bolatov, Bilal Elbouardi, Besher Hassan, Yuri Kuratov, Aydar Bulatov, Preslav Nakov, Timothy Baldwin, Artem Shelmanov, Mikhail Burtsev</p><p><b>Upvotes:</b> 15</p><p><b>Summary:</b> Reasoning is a core capability of large language models, yet understanding how they learn and perform multi-step reasoning remains an open problem. In this study, we explore how different architectures and training methods affect model multi-step reasoning capabilities within a cellular automata framework. By training on state sequences generated with random Boolean functions for random initial conditions to exclude memorization, we demonstrate that most neural architectures learn to abstract the underlying rules. While models achieve high accuracy in next-state prediction, their performance declines sharply if multi-step reasoning is required. We confirm that increasing model depth plays a crucial role for sequential computations. We demonstrate that an extension of the effective model depth with recurrence, memory, and test-time compute scaling substantially enhances reasoning capabilities.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.08240</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.16745</guid>
|
||||||
<pubDate>Mon, 11 Aug 2025 17:54:31 +0000</pubDate>
|
<pubDate>Fri, 22 Aug 2025 18:57:08 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts</title>
|
<title>TaDiCodec: Text-aware Diffusion Speech Tokenizer for Speech Language Modeling</title>
|
||||||
<link>https://arxiv.org/abs/2508.10390</link>
|
<link>https://arxiv.org/abs/2508.16790</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.10390.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Chiyu Zhang, Lu Zhou, Xiaogang Xu, Jiafei Wu, Liming Fang, Zhe Liu</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Evaluating jailbreak attacks is challenging when prompts are not overtly harmful or fail to induce harmful outputs. Unfortunately, many existing red-teaming datasets contain such unsuitable prompts. To evaluate attacks accurately, these datasets need to be assessed and cleaned for maliciousness. However, existing malicious content detection methods rely on either manual annotation, which is labor-intensive, or large language models (LLMs), which have inconsistent accuracy in harmful types. To balance accuracy and efficiency, we propose a hybrid evaluation framework named MDH (Malicious content Detection based on LLMs with Human assistance) that combines LLM-based annotation with minimal human oversight, and apply it to dataset cleaning and detection of jailbroken responses. Furthermore, we find that well-crafted developer messages can significantly boost jailbreak success, leading us to propose two new strategies: D-Attack, which leverages context simulation, and DH-CoT, which incorporates hijacked chains of thought. The Codes, datasets, judgements, and detection results will be released in github repository: https://github.com/AlienZhang1996/DH-CoT.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16790.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuancheng Wang, Dekun Chen, Xueyao Zhang, Junan Zhang, Jiaqi Li, Zhizheng Wu</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Speech tokenizers serve as foundational components for speech language models, yet current designs exhibit several limitations, including: 1) dependence on multi-layer residual vector quantization structures or high frame rates, 2) reliance on auxiliary pre-trained models for semantic distillation, and 3) requirements for complex two-stage training processes. In this work, we introduce the Text-aware Diffusion Transformer Speech Codec (TaDiCodec), a novel approach designed to overcome these challenges. TaDiCodec employs end-to-end optimization for quantization and reconstruction through a diffusion autoencoder, while integrating text guidance into the diffusion decoder to enhance reconstruction quality and achieve optimal compression. TaDiCodec achieves an extremely low frame rate of 6.25 Hz and a corresponding bitrate of 0.0875 kbps with a single-layer codebook for 24 kHz speech, while maintaining superior performance on critical speech generation evaluation metrics such as Word Error Rate (WER), speaker similarity (SIM), and speech quality (UTMOS). Notably, TaDiCodec employs a single-stage, end-to-end training paradigm, and obviating the need for auxiliary pre-trained models. We also validate the compatibility of TaDiCodec in language model based zero-shot text-to-speech with both autoregressive modeling and masked generative modeling, demonstrating its effectiveness and efficiency for speech language modeling, as well as a significantly small reconstruction-generation gap. We will open source our code and model checkpoints. Audio samples are are available at https:/tadicodec.github.io/. We release code and model checkpoints at https:/github.com/HeCheng0625/Diffusion-Speech-Tokenizer.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.10390</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.16790</guid>
|
||||||
<pubDate>Thu, 14 Aug 2025 06:46:56 +0000</pubDate>
|
<pubDate>Fri, 22 Aug 2025 20:45:03 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>EgoTwin: Dreaming Body and View in First Person</title>
|
<title>If We May De-Presuppose: Robustly Verifying Claims through Presupposition-Free Question Decomposition</title>
|
||||||
<link>https://arxiv.org/abs/2508.13013</link>
|
<link>https://arxiv.org/abs/2508.16838</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.13013.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jingqiao Xiu, Fangzhou Hong, Yicong Li, Mengze Li, Wentao Wang, Sirui Han, Liang Pan, Ziwei Liu</p><p><b>Upvotes:</b> 15</p><p><b>Summary:</b> While exocentric video synthesis has achieved great progress, egocentric video generation remains largely underexplored, which requires modeling first-person view content along with camera motion patterns induced by the wearer's body movements. To bridge this gap, we introduce a novel task of joint egocentric video and human motion generation, characterized by two key challenges: 1) Viewpoint Alignment: the camera trajectory in the generated video must accurately align with the head trajectory derived from human motion; 2) Causal Interplay: the synthesized human motion must causally align with the observed visual dynamics across adjacent video frames. To address these challenges, we propose EgoTwin, a joint video-motion generation framework built on the diffusion transformer architecture. Specifically, EgoTwin introduces a head-centric motion representation that anchors the human motion to the head joint and incorporates a cybernetics-inspired interaction mechanism that explicitly captures the causal interplay between video and motion within attention operations. For comprehensive evaluation, we curate a large-scale real-world dataset of synchronized text-video-motion triplets and design novel metrics to assess video-motion consistency. Extensive experiments demonstrate the effectiveness of the EgoTwin framework.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16838.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Shubhashis Roy Dipta, Francis Ferraro</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Prior work has shown that presupposition in generated questions can introduce unverified assumptions, leading to inconsistencies in claim verification. Additionally, prompt sensitivity remains a significant challenge for large language models (LLMs), resulting in performance variance as high as 3-6%. While recent advancements have reduced this gap, our study demonstrates that prompt sensitivity remains a persistent issue. To address this, we propose a structured and robust claim verification framework that reasons through presupposition-free, decomposed questions. Extensive experiments across multiple prompts, datasets, and LLMs reveal that even state-of-the-art models remain susceptible to prompt variance and presupposition. Our method consistently mitigates these issues, achieving up to a 2-5% improvement.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.13013</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.16838</guid>
|
||||||
<pubDate>Mon, 18 Aug 2025 15:33:09 +0000</pubDate>
|
<pubDate>Fri, 22 Aug 2025 23:34:24 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Learnable SMPLify: A Neural Solution for Optimization-Free Human Pose Inverse Kinematics</title>
|
<title>Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning</title>
|
||||||
<link>https://arxiv.org/abs/2508.13562</link>
|
<link>https://arxiv.org/abs/2508.16949</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.13562.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuchen Yang, Linfeng Dong, Wei Wang, Zhihang Zhong, Xiao Sun</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> In 3D human pose and shape estimation, SMPLify remains a robust baseline that solves inverse kinematics (IK) through iterative optimization. However, its high computational cost limits its practicality. Recent advances across domains have shown that replacing iterative optimization with data-driven neural networks can achieve significant runtime improvements without sacrificing accuracy. Motivated by this trend, we propose Learnable SMPLify, a neural framework that replaces the iterative fitting process in SMPLify with a single-pass regression model. The design of our framework targets two core challenges in neural IK: data construction and generalization. To enable effective training, we propose a temporal sampling strategy that constructs initialization-target pairs from sequential frames. To improve generalization across diverse motions and unseen poses, we propose a human-centric normalization scheme and residual learning to narrow the solution space. Learnable SMPLify supports both sequential inference and plug-in post-processing to refine existing image-based estimators. Extensive experiments demonstrate that our method establishes itself as a practical and simple baseline: it achieves nearly 200x faster runtime compared to SMPLify, generalizes well to unseen 3DPW and RICH, and operates in a model-agnostic manner when used as a plug-in tool on LucidAction. The code is available at https://github.com/Charrrrrlie/Learnable-SMPLify.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16949.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yang Zhou, Sunzhu Li, Shunyu Liu, Wenkai Fang, Jiale Zhao, Jingwen Yang, Jianwei Lv, Kongcheng Zhang, Yihe Zhou, Hengtong Lu, Wei Chen, Yan Xie, Mingli Song</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> Recent advances in Large Language Models (LLMs) have underscored the potential of Reinforcement Learning (RL) to facilitate the emergence of reasoning capabilities. Despite the encouraging results, a fundamental dilemma persists as RL improvement relies on learning from high-quality samples, yet the exploration for such samples remains bounded by the inherent limitations of LLMs. This, in effect, creates an undesirable cycle in which what cannot be explored cannot be learned. In this work, we propose Rubric-Scaffolded Reinforcement Learning (RuscaRL), a novel instructional scaffolding framework designed to break the exploration bottleneck for general LLM reasoning. Specifically, RuscaRL introduces checklist-style rubrics as (1) explicit scaffolding for exploration during rollout generation, where different rubrics are provided as external guidance within task instructions to steer diverse high-quality responses. This guidance is gradually decayed over time, encouraging the model to internalize the underlying reasoning patterns; (2) verifiable rewards for exploitation during model training, where we can obtain robust LLM-as-a-Judge scores using rubrics as references, enabling effective RL on general reasoning tasks. Extensive experiments demonstrate the superiority of the proposed RuscaRL across various benchmarks, effectively expanding reasoning boundaries under the best-of-N evaluation. Notably, RuscaRL significantly boosts Qwen-2.5-7B-Instruct from 23.6 to 50.3 on HealthBench-500, surpassing GPT-4.1. Furthermore, our fine-tuned variant on Qwen3-30B-A3B-Instruct achieves 61.1 on HealthBench-500, outperforming leading LLMs including OpenAI-o3.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.13562</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.16949</guid>
|
||||||
<pubDate>Tue, 19 Aug 2025 06:53:57 +0000</pubDate>
|
<pubDate>Sat, 23 Aug 2025 08:47:31 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>CRISP: Persistent Concept Unlearning via Sparse Autoencoders</title>
|
<title>REGEN: Real-Time Photorealism Enhancement in Games via a Dual-Stage Generative Network Framework</title>
|
||||||
<link>https://arxiv.org/abs/2508.13650</link>
|
<link>https://arxiv.org/abs/2508.17061</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.13650.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tomer Ashuach, Dana Arad, Aaron Mueller, Martin Tutek, Yonatan Belinkov</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> As large language models (LLMs) are increasingly deployed in real-world applications, the need to selectively remove unwanted knowledge while preserving model utility has become paramount. Recent work has explored sparse autoencoders (SAEs) to perform precise interventions on monosemantic features. However, most SAE-based methods operate at inference time, which does not create persistent changes in the model's parameters. Such interventions can be bypassed or reversed by malicious actors with parameter access. We introduce CRISP, a parameter-efficient method for persistent concept unlearning using SAEs. CRISP automatically identifies salient SAE features across multiple layers and suppresses their activations. We experiment with two LLMs and show that our method outperforms prior approaches on safety-critical unlearning tasks from the WMDP benchmark, successfully removing harmful knowledge while preserving general and in-domain capabilities. Feature-level analysis reveals that CRISP achieves semantically coherent separation between target and benign concepts, allowing precise suppression of the target features.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17061.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Stefanos Pasios, Nikos Nikolaidis</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Photorealism is an important aspect of modern video games since it can shape the player experience and simultaneously impact the immersion, narrative engagement, and visual fidelity. Although recent hardware technological breakthroughs, along with state-of-the-art rendering technologies, have significantly improved the visual realism of video games, achieving true photorealism in dynamic environments at real-time frame rates still remains a major challenge due to the tradeoff between visual quality and performance. In this short paper, we present a novel approach for enhancing the photorealism of rendered game frames using generative adversarial networks. To this end, we propose Real-time photorealism Enhancement in Games via a dual-stage gEnerative Network framework (REGEN), which employs a robust unpaired image-to-image translation model to produce semantically consistent photorealistic frames that transform the problem into a simpler paired image-to-image translation task. This enables training with a lightweight method that can achieve real-time inference time without compromising visual quality. We demonstrate the effectiveness of our framework on Grand Theft Auto V, showing that the approach achieves visual results comparable to the ones produced by the robust unpaired Im2Im method while improving inference speed by 32.14 times. Our findings also indicate that the results outperform the photorealism-enhanced frames produced by directly training a lightweight unpaired Im2Im translation method to translate the video game frames towards the visual characteristics of real-world images. Code, pre-trained models, and demos for this work are available at: https://github.com/stefanos50/REGEN.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.13650</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17061</guid>
|
||||||
<pubDate>Tue, 19 Aug 2025 09:01:22 +0000</pubDate>
|
<pubDate>Sat, 23 Aug 2025 15:28:05 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Sketch3DVE: Sketch-based 3D-Aware Scene Video Editing</title>
|
<title>PosterGen: Aesthetic-Aware Paper-to-Poster Generation via Multi-Agent LLMs</title>
|
||||||
<link>https://arxiv.org/abs/2508.13797</link>
|
<link>https://arxiv.org/abs/2508.17188</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.13797.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Feng-Lin Liu, Shi-Yang Li, Yan-Pei Cao, Hongbo Fu, Lin Gao</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Recent video editing methods achieve attractive results in style transfer or appearance modification. However, editing the structural content of 3D scenes in videos remains challenging, particularly when dealing with significant viewpoint changes, such as large camera rotations or zooms. Key challenges include generating novel view content that remains consistent with the original video, preserving unedited regions, and translating sparse 2D inputs into realistic 3D video outputs. To address these issues, we propose Sketch3DVE, a sketch-based 3D-aware video editing method to enable detailed local manipulation of videos with significant viewpoint changes. To solve the challenge posed by sparse inputs, we employ image editing methods to generate edited results for the first frame, which are then propagated to the remaining frames of the video. We utilize sketching as an interaction tool for precise geometry control, while other mask-based image editing methods are also supported. To handle viewpoint changes, we perform a detailed analysis and manipulation of the 3D information in the video. Specifically, we utilize a dense stereo method to estimate a point cloud and the camera parameters of the input video. We then propose a point cloud editing approach that uses depth maps to represent the 3D geometry of newly edited components, aligning them effectively with the original 3D scene. To seamlessly merge the newly edited content with the original video while preserving the features of unedited regions, we introduce a 3D-aware mask propagation strategy and employ a video diffusion model to produce realistic edited videos. Extensive experiments demonstrate the superiority of Sketch3DVE in video editing. Homepage and code: http://http://geometrylearning.com/Sketch3DVE/</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17188.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zhilin Zhang, Xiang Zhang, Jiaqi Wei, Yiwei Xu, Chenyu You</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Multi-agent systems built upon large language models (LLMs) have demonstrated remarkable capabilities in tackling complex compositional tasks. In this work, we apply this paradigm to the paper-to-poster generation problem, a practical yet time-consuming process faced by researchers preparing for conferences. While recent approaches have attempted to automate this task, most neglect core design and aesthetic principles, resulting in posters that require substantial manual refinement. To address these design limitations, we propose PosterGen, a multi-agent framework that mirrors the workflow of professional poster designers. It consists of four collaborative specialized agents: (1) Parser and Curator agents extract content from the paper and organize storyboard; (2) Layout agent maps the content into a coherent spatial layout; (3) Stylist agents apply visual design elements such as color and typography; and (4) Renderer composes the final poster. Together, these agents produce posters that are both semantically grounded and visually appealing. To evaluate design quality, we introduce a vision-language model (VLM)-based rubric that measures layout balance, readability, and aesthetic coherence. Experimental results show that PosterGen consistently matches in content fidelity, and significantly outperforms existing methods in visual designs, generating posters that are presentation-ready with minimal human refinements.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.13797</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17188</guid>
|
||||||
<pubDate>Tue, 19 Aug 2025 12:57:31 +0000</pubDate>
|
<pubDate>Sun, 24 Aug 2025 02:25:45 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR</title>
|
<title>MEENA (PersianMMMU): Multimodal-Multilingual Educational Exams for N-level Assessment</title>
|
||||||
<link>https://arxiv.org/abs/2508.14029</link>
|
<link>https://arxiv.org/abs/2508.17290</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.14029.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiao Liang, Zhongzhi Li, Yeyun Gong, Yelong Shen, Ying Nian Wu, Zhijiang Guo, Weizhu Chen</p><p><b>Upvotes:</b> 27</p><p><b>Summary:</b> Reinforcement Learning with Verifiable Rewards (RLVR) has recently emerged as a key paradigm for post-training Large Language Models (LLMs), particularly for complex reasoning tasks. However, vanilla RLVR training has been shown to improve Pass@1 performance at the expense of policy entropy, leading to reduced generation diversity and limiting the Pass@k performance, which typically represents the upper bound of LLM reasoning capability. In this paper, we systematically analyze the policy's generation diversity from the perspective of training problems and find that augmenting and updating training problems helps mitigate entropy collapse during training. Based on these observations, we propose an online Self-play with Variational problem Synthesis (SvS) strategy for RLVR training, which uses the policy's correct solutions to synthesize variational problems while ensuring their reference answers remain identical to the originals. This self-improving strategy effectively maintains policy entropy during training and substantially improves Pass@k compared with standard RLVR, sustaining prolonged improvements and achieving absolute gains of 18.3% and 22.8% in Pass@32 performance on the competition-level AIME24 and AIME25 benchmarks. Experiments on 12 reasoning benchmarks across varying model sizes from 3B to 32B consistently demonstrate the generalizability and robustness of SvS.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17290.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Omid Ghahroodi, Arshia Hemmat, Marzia Nouri, Seyed Mohammad Hadi Hosseini, Doratossadat Dastgheib, Mohammad Vali Sanian, Alireza Sahebi, Reihaneh Zohrabi, Mohammad Hossein Rohban, Ehsaneddin Asgari, Mahdieh Soleymani Baghshah</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Recent advancements in large vision-language models (VLMs) have primarily focused on English, with limited attention given to other languages. To address this gap, we introduce MEENA (also known as PersianMMMU), the first dataset designed to evaluate Persian VLMs across scientific, reasoning, and human-level understanding tasks. Our dataset comprises approximately 7,500 Persian and 3,000 English questions, covering a wide range of topics such as reasoning, mathematics, physics, diagrams, charts, and Persian art and literature. Key features of MEENA include: (1) diverse subject coverage spanning various educational levels, from primary to upper secondary school, (2) rich metadata, including difficulty levels and descriptive answers, (3) original Persian data that preserves cultural nuances, (4) a bilingual structure to assess cross-linguistic performance, and (5) a series of diverse experiments assessing various capabilities, including overall performance, the model's ability to attend to images, and its tendency to generate hallucinations. We hope this benchmark contributes to enhancing VLM capabilities beyond English.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.14029</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17290</guid>
|
||||||
<pubDate>Tue, 19 Aug 2025 17:42:45 +0000</pubDate>
|
<pubDate>Sun, 24 Aug 2025 10:32:37 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Distilled-3DGS:Distilled 3D Gaussian Splatting</title>
|
<title>Explain Before You Answer: A Survey on Compositional Visual Reasoning</title>
|
||||||
<link>https://arxiv.org/abs/2508.14037</link>
|
<link>https://arxiv.org/abs/2508.17298</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.14037.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Lintao Xiang, Xinkai Chen, Jianhuang Lai, Guangcong Wang</p><p><b>Upvotes:</b> 4</p><p><b>Summary:</b> 3D Gaussian Splatting (3DGS) has exhibited remarkable efficacy in novel view synthesis (NVS). However, it suffers from a significant drawback: achieving high-fidelity rendering typically necessitates a large number of 3D Gaussians, resulting in substantial memory consumption and storage requirements. To address this challenge, we propose the first knowledge distillation framework for 3DGS, featuring various teacher models, including vanilla 3DGS, noise-augmented variants, and dropout-regularized versions. The outputs of these teachers are aggregated to guide the optimization of a lightweight student model. To distill the hidden geometric structure, we propose a structural similarity loss to boost the consistency of spatial geometric distributions between the student and teacher model. Through comprehensive quantitative and qualitative evaluations across diverse datasets, the proposed Distilled-3DGS, a simple yet effective framework without bells and whistles, achieves promising rendering results in both rendering quality and storage efficiency compared to state-of-the-art methods. Project page: https://distilled3dgs.github.io . Code: https://github.com/lt-xiang/Distilled-3DGS .</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17298.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Fucai Ke, Joy Hsu, Zhixi Cai, Zixian Ma, Xin Zheng, Xindi Wu, Sukai Huang, Weiqing Wang, Pari Delir Haghighi, Gholamreza Haffari, Ranjay Krishna, Jiajun Wu, Hamid Rezatofighi</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Compositional visual reasoning has emerged as a key research frontier in multimodal AI, aiming to endow machines with the human-like ability to decompose visual scenes, ground intermediate concepts, and perform multi-step logical inference. While early surveys focus on monolithic vision-language models or general multimodal reasoning, a dedicated synthesis of the rapidly expanding compositional visual reasoning literature is still missing. We fill this gap with a comprehensive survey spanning 2023 to 2025 that systematically reviews 260+ papers from top venues (CVPR, ICCV, NeurIPS, ICML, ACL, etc.). We first formalize core definitions and describe why compositional approaches offer advantages in cognitive alignment, semantic fidelity, robustness, interpretability, and data efficiency. Next, we trace a five-stage paradigm shift: from prompt-enhanced language-centric pipelines, through tool-enhanced LLMs and tool-enhanced VLMs, to recently minted chain-of-thought reasoning and unified agentic VLMs, highlighting their architectural designs, strengths, and limitations. We then catalog 60+ benchmarks and corresponding metrics that probe compositional visual reasoning along dimensions such as grounding accuracy, chain-of-thought faithfulness, and high-resolution perception. Drawing on these analyses, we distill key insights, identify open challenges (e.g., limitations of LLM-based reasoning, hallucination, a bias toward deductive reasoning, scalable supervision, tool integration, and benchmark limitations), and outline future directions, including world-model integration, human-AI collaborative reasoning, and richer evaluation protocols. By offering a unified taxonomy, historical roadmap, and critical outlook, this survey aims to serve as a foundational reference and inspire the next generation of compositional visual reasoning research.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.14037</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17298</guid>
|
||||||
<pubDate>Tue, 19 Aug 2025 17:59:26 +0000</pubDate>
|
<pubDate>Sun, 24 Aug 2025 11:01:51 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>CARFT: Boosting LLM Reasoning via Contrastive Learning with Annotated Chain-of-Thought-based Reinforced Fine-Tuning</title>
|
<title>Semantic Diffusion Posterior Sampling for Cardiac Ultrasound Dehazing</title>
|
||||||
<link>https://arxiv.org/abs/2508.15868</link>
|
<link>https://arxiv.org/abs/2508.17326</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.15868.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Wenqiao Zhu, Ji Liu, Rongjuncheng Zhang, Haipang Wu, Yulun Zhang</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Reasoning capability plays a significantly critical role in the the broad applications of Large Language Models (LLMs). To enhance the reasoning performance of LLMs, diverse Reinforcement Learning (RL)-based fine-tuning approaches have been proposed to address the limited generalization capability of LLMs trained solely via Supervised Fine-Tuning (SFT). Despite their effectiveness, two major limitations hinder the advancement of LLMs. First, vanilla RL-based approaches ignore annotated Chain-of-Thought (CoT) and incorporate unstable reasoning path sampling, which typically results in model collapse, unstable training process, and suboptimal performance. Second, existing SFT approaches generally overemphasize the annotated CoT, potentially leading to performance degradation due to insufficient exploitation of potential CoT. In this paper, we propose a Contrastive learning with annotated CoT-based Reinforced Fine-Tuning approach, i.e., , to enhance the reasoning performance of LLMs while addressing the aforementioned limitations. Specifically, we propose learning a representation for each CoT. Based on this representation, we design novel contrastive signals to guide the fine-tuning process. Our approach not only fully exploits the available annotated CoT but also stabilizes the fine-tuning procedure by incorporating an additional unsupervised learning signal. We conduct comprehensive experiments and in-depth analysis with three baseline approaches, two foundation models, and two datasets to demonstrate significant advantages of in terms of robustness, performance (up to 10.15\%), and efficiency (up to 30.62\%). Code is available at https://github.com/WNQzhu/CARFT.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17326.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tristan S. W. Stevens, Oisín Nolan, Ruud J. G. van Sloun</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Echocardiography plays a central role in cardiac imaging, offering dynamic views of the heart that are essential for diagnosis and monitoring. However, image quality can be significantly degraded by haze arising from multipath reverberations, particularly in difficult-to-image patients. In this work, we propose a semantic-guided, diffusion-based dehazing algorithm developed for the MICCAI Dehazing Echocardiography Challenge (DehazingEcho2025). Our method integrates a pixel-wise noise model, derived from semantic segmentation of hazy inputs into a diffusion posterior sampling framework guided by a generative prior trained on clean ultrasound data. Quantitative evaluation on the challenge dataset demonstrates strong performance across contrast and fidelity metrics. Code for the submitted algorithm is available at https://github.com/tristan-deep/semantic-diffusion-echo-dehazing.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.15868</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17326</guid>
|
||||||
<pubDate>Thu, 21 Aug 2025 00:20:47 +0000</pubDate>
|
<pubDate>Sun, 24 Aug 2025 12:20:18 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>TPLA: Tensor Parallel Latent Attention for Efficient Disaggregated Prefill \& Decode Inference</title>
|
<title>T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation</title>
|
||||||
<link>https://arxiv.org/abs/2508.15881</link>
|
<link>https://arxiv.org/abs/2508.17472</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.15881.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xiaojuan Tang, Fanxu Meng, Pingzhi Tang, Yuxuan Wang, Di Yin, Xing Sun, Muhan Zhang</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Multi-Head Latent Attention (MLA), introduced in DeepSeek-V2, compresses key-value states into a low-rank latent vector, caching only this vector to reduce memory. In tensor parallelism (TP), however, attention heads are computed across multiple devices, and each device must load the full cache, eroding the advantage of MLA over Grouped Query Attention (GQA). We propose Tensor-Parallel Latent Attention (TPLA): a scheme that partitions both the latent representation and each head's input dimension across devices, performs attention independently per shard, and then combines results with an all-reduce. TPLA preserves the benefits of a compressed KV cache while unlocking TP efficiency. Unlike Grouped Latent Attention (GLA), every head in TPLA still leverages the full latent representation, maintaining stronger representational capacity. TPLA is drop-in compatible with models pre-trained using MLA: it supports MLA-style prefilling and enables efficient tensor-parallel decoding without retraining. Applying simple orthogonal transforms -- e.g., the Hadamard transform or PCA -- before TP slicing further mitigates cross-shard interference, yielding minimal accuracy degradation. By reducing the per-device KV cache for DeepSeek-V3 and Kimi-K2, we achieve 1.79x and 1.93x speedups, respectively, at a 32K-token context length while maintaining performance on commonsense and LongBench benchmarks. TPLA can be implemented with FlashAttention-3, enabling practical end-to-end acceleration.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17472.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Kaiyue Sun, Rongyao Fang, Chengqi Duan, Xian Liu, Xihui Liu</p><p><b>Upvotes:</b> 20</p><p><b>Summary:</b> We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.15881</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17472</guid>
|
||||||
<pubDate>Thu, 21 Aug 2025 15:25:40 +0000</pubDate>
|
<pubDate>Sun, 24 Aug 2025 17:59:38 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoning</title>
|
<title>UQ: Assessing Language Models on Unsolved Questions</title>
|
||||||
<link>https://arxiv.org/abs/2508.15746</link>
|
<link>https://arxiv.org/abs/2508.17580</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.15746.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Qiaoyu Zheng, Yuze Sun, Chaoyi Wu, Weike Zhao, Pengcheng Qiu, Yongguo Yu, Kun Sun, Yanfeng Wang, Ya Zhang, Weidi Xie</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Accurate diagnosis with medical large language models is hindered by knowledge gaps and hallucinations. Retrieval and tool-augmented methods help, but their impact is limited by weak use of external knowledge and poor feedback-reasoning traceability. To address these challenges, We introduce Deep-DxSearch, an agentic RAG system trained end-to-end with reinforcement learning (RL) that enables steer tracebale retrieval-augmented reasoning for medical diagnosis. In Deep-DxSearch, we first construct a large-scale medical retrieval corpus comprising patient records and reliable medical knowledge sources to support retrieval-aware reasoning across diagnostic scenarios. More crutially, we frame the LLM as the core agent and the retrieval corpus as its environment, using tailored rewards on format, retrieval, reasoning structure, and diagnostic accuracy, thereby evolving the agentic RAG policy from large-scale data through RL. Experiments demonstrate that our end-to-end agentic RL training framework consistently outperforms prompt-engineering and training-free RAG approaches across multiple data centers. After training, Deep-DxSearch achieves substantial gains in diagnostic accuracy, surpassing strong diagnostic baselines such as GPT-4o, DeepSeek-R1, and other medical-specific frameworks for both common and rare disease diagnosis under in-distribution and out-of-distribution settings. Moreover, ablation studies on reward design and retrieval corpus components confirm their critical roles, underscoring the uniqueness and effectiveness of our approach compared with traditional implementations. Finally, case studies and interpretability analyses highlight improvements in Deep-DxSearch's diagnostic policy, providing deeper insight into its performance gains and supporting clinicians in delivering more reliable and precise preliminary diagnoses. See https://github.com/MAGIC-AI4Med/Deep-DxSearch.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17580.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Fan Nie, Ken Ziyu Liu, Zihao Wang, Rui Sun, Wei Liu, Weijia Shi, Huaxiu Yao, Linjun Zhang, Andrew Y. Ng, James Zou, Sanmi Koyejo, Yejin Choi, Percy Liang, Niklas Muennighoff</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> Benchmarks shape progress in AI research. A useful benchmark should be both difficult and realistic: questions should challenge frontier models while also reflecting real-world usage. Yet, current paradigms face a difficulty-realism tension: exam-style benchmarks are often made artificially difficult with limited real-world value, while benchmarks based on real user interaction often skew toward easy, high-frequency problems. In this work, we explore a radically different paradigm: assessing models on unsolved questions. Rather than a static benchmark scored once, we curate unsolved questions and evaluate models asynchronously over time with validator-assisted screening and community verification. We introduce UQ, a testbed of 500 challenging, diverse questions sourced from Stack Exchange, spanning topics from CS theory and math to sci-fi and history, probing capabilities including reasoning, factuality, and browsing. UQ is difficult and realistic by construction: unsolved questions are often hard and naturally arise when humans seek answers, thus solving them yields direct real-world value. Our contributions are threefold: (1) UQ-Dataset and its collection pipeline combining rule-based filters, LLM judges, and human review to ensure question quality (e.g., well-defined and difficult); (2) UQ-Validators, compound validation strategies that leverage the generator-validator gap to provide evaluation signals and pre-screen candidate solutions for human review; and (3) UQ-Platform, an open platform where experts collectively verify questions and solutions. The top model passes UQ-validation on only 15% of questions, and preliminary human verification has already identified correct answers among those that passed. UQ charts a path for evaluating frontier models on real-world, open-ended challenges, where success pushes the frontier of human knowledge. We release UQ at https://uq.stanford.edu.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.15746</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17580</guid>
|
||||||
<pubDate>Thu, 21 Aug 2025 17:42:47 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 01:07:59 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>InMind: Evaluating LLMs in Capturing and Applying Individual Human Reasoning Styles</title>
|
<title>MeshSplat: Generalizable Sparse-View Surface Reconstruction via Gaussian Splatting</title>
|
||||||
<link>https://arxiv.org/abs/2508.16072</link>
|
<link>https://arxiv.org/abs/2508.17811</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16072.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zizhen Li, Chuanhao Li, Yibin Wang, Qi Chen, Diping Song, Yukang Feng, Jianwen Sun, Jiaxin Ai, Fanrui Zhang, Mingzhu Sun, Kaipeng Zhang</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> LLMs have shown strong performance on human-centric reasoning tasks. While previous evaluations have explored whether LLMs can infer intentions or detect deception, they often overlook the individualized reasoning styles that influence how people interpret and act in social contexts. Social deduction games (SDGs) provide a natural testbed for evaluating individualized reasoning styles, where different players may adopt diverse but contextually valid reasoning strategies under identical conditions. To address this, we introduce InMind, a cognitively grounded evaluation framework designed to assess whether LLMs can capture and apply personalized reasoning styles in SDGs. InMind enhances structured gameplay data with round-level strategy traces and post-game reflections, collected under both Observer and Participant modes. It supports four cognitively motivated tasks that jointly evaluate both static alignment and dynamic adaptation. As a case study, we apply InMind to the game Avalon, evaluating 11 state-of-the-art LLMs. General-purpose LLMs, even GPT-4o frequently rely on lexical cues, struggling to anchor reflections in temporal gameplay or adapt to evolving strategies. In contrast, reasoning-enhanced LLMs like DeepSeek-R1 exhibit early signs of style-sensitive reasoning. These findings reveal key limitations in current LLMs' capacity for individualized, adaptive reasoning, and position InMind as a step toward cognitively aligned human-AI interaction.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17811.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Hanzhi Chang, Ruijie Zhu, Wenjie Chang, Mulin Yu, Yanzhe Liang, Jiahao Lu, Zhuoyuan Li, Tianzhu Zhang</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> Surface reconstruction has been widely studied in computer vision and graphics. However, existing surface reconstruction works struggle to recover accurate scene geometry when the input views are extremely sparse. To address this issue, we propose MeshSplat, a generalizable sparse-view surface reconstruction framework via Gaussian Splatting. Our key idea is to leverage 2DGS as a bridge, which connects novel view synthesis to learned geometric priors and then transfers these priors to achieve surface reconstruction. Specifically, we incorporate a feed-forward network to predict per-view pixel-aligned 2DGS, which enables the network to synthesize novel view images and thus eliminates the need for direct 3D ground-truth supervision. To improve the accuracy of 2DGS position and orientation prediction, we propose a Weighted Chamfer Distance Loss to regularize the depth maps, especially in overlapping areas of input views, and also a normal prediction network to align the orientation of 2DGS with normal vectors predicted by a monocular normal estimator. Extensive experiments validate the effectiveness of our proposed improvement, demonstrating that our method achieves state-of-the-art performance in generalizable sparse-view mesh reconstruction tasks. Project Page: https://hanzhichang.github.io/meshsplat_web</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16072</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17811</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 04:04:00 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 09:04:20 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>AgentFly: Fine-tuning LLM Agents without Fine-tuning LLMs</title>
|
<title>Limitations of Normalization in Attention Mechanism</title>
|
||||||
<link>https://arxiv.org/abs/2508.16153</link>
|
<link>https://arxiv.org/abs/2508.17821</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16153.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, Jun Wang</p><p><b>Upvotes:</b> 53</p><p><b>Summary:</b> In this paper, we introduce a novel learning paradigm for adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning the underlying LLMs. Existing approaches are often either rigid, relying on static, handcrafted reflection workflows, or computationally intensive, requiring gradient updates of LLM model parameters. In contrast, our method enables low-cost continual adaptation via memory-based online reinforcement learning. We formalise this as a Memory-augmented Markov Decision Process (M-MDP), equipped with a neural case-selection policy to guide action decisions. Past experiences are stored in an episodic memory, either differentiable or non-parametric. The policy is continually updated based on environmental feedback through a memory rewriting mechanism, whereas policy improvement is achieved through efficient memory reading (retrieval). We instantiate our agent model in the deep research setting, namely AgentFly, which attains top-1 on GAIA validation (87.88% Pass@3) and 79.40% on the test set. It reaches 66.6% F1 and 80.4% PM on the DeepResearcher dataset, outperforming the state-of-the-art training-based method, while case-based memory adds 4.7% to 9.6% absolute points on out-of-distribution tasks. Our approach offers a scalable and efficient pathway for developing generalist LLM agents capable of continuous, real-time learning without gradient updates, advancing machine learning towards open-ended skill acquisition and deep research scenarios. The code is available at https://github.com/Agent-on-the-Fly/AgentFly.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17821.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova, Radu State</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> This paper investigates the limitations of the normalization in attention mechanisms. We begin with a theoretical framework that enables the identification of the model's selective ability and the geometric separation involved in token selection. Our analysis includes explicit bounds on distances and separation criteria for token vectors under softmax scaling. Through experiments with pre-trained GPT-2 model, we empirically validate our theoretical results and analyze key behaviors of the attention mechanism. Notably, we demonstrate that as the number of selected tokens increases, the model's ability to distinguish informative tokens declines, often converging toward a uniform selection pattern. We also show that gradient sensitivity under softmax normalization presents challenges during training, especially at low temperature settings. These findings advance current understanding of softmax-based attention mechanism and motivate the need for more robust normalization and selection strategies in future attention architectures.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16153</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17821</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 07:25:30 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 09:25:05 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>AgentScope 1.0: A Developer-Centric Framework for Building Agentic Applications</title>
|
<title>German4All - A Dataset and Model for Readability-Controlled Paraphrasing in German</title>
|
||||||
<link>https://arxiv.org/abs/2508.16279</link>
|
<link>https://arxiv.org/abs/2508.17973</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16279.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Dawei Gao, Zitao Li, Yuexiang Xie, Weirui Kuang, Liuyi Yao, Bingchen Qian, Zhijian Ma, Yue Cui, Haohao Luo, Shen Li, Lu Yi, Yi Yu, Shiqi He, Zhiling Luo, Wenmeng Zhou, Zhicheng Zhang, Xuguang He, Ziqian Chen, Weikai Liao, Farruh Isakulovich Kushnazarov, Yaliang Li, Bolin Ding, Jingren Zhou</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Driven by rapid advancements of Large Language Models (LLMs), agents are empowered to combine intrinsic knowledge with dynamic tool use, greatly enhancing their capacity to address real-world tasks. In line with such an evolution, AgentScope introduces major improvements in a new version (1.0), towards comprehensively supporting flexible and efficient tool-based agent-environment interactions for building agentic applications. Specifically, we abstract foundational components essential for agentic applications and provide unified interfaces and extensible modules, enabling developers to easily leverage the latest progress, such as new models and MCPs. Furthermore, we ground agent behaviors in the ReAct paradigm and offer advanced agent-level infrastructure based on a systematic asynchronous design, which enriches both human-agent and agent-agent interaction patterns while improving execution efficiency. Building on this foundation, we integrate several built-in agents tailored to specific practical scenarios. AgentScope also includes robust engineering support for developer-friendly experiences. We provide a scalable evaluation module with a visual studio interface, making the development of long-trajectory agentic applications more manageable and easier to trace. In addition, AgentScope offers a runtime sandbox to ensure safe agent execution and facilitates rapid deployment in production environments. With these enhancements, AgentScope provides a practical foundation for building scalable, adaptive, and effective agentic applications.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.17973.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Miriam Anschütz, Thanh Mai Pham, Eslam Nasrallah, Maximilian Müller, Cristian-George Craciun, Georg Groh</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> The ability to paraphrase texts across different complexity levels is essential for creating accessible texts that can be tailored toward diverse reader groups. Thus, we introduce German4All, the first large-scale German dataset of aligned readability-controlled, paragraph-level paraphrases. It spans five readability levels and comprises over 25,000 samples. The dataset is automatically synthesized using GPT-4 and rigorously evaluated through both human and LLM-based judgments. Using German4All, we train an open-source, readability-controlled paraphrasing model that achieves state-of-the-art performance in German text simplification, enabling more nuanced and reader-specific adaptations. We opensource both the dataset and the model to encourage further research on multi-level paraphrasing</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16279</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.17973</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 10:35:56 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 12:40:32 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Do What? Teaching Vision-Language-Action Models to Reject the Impossible</title>
|
<title>Visual-CoG: Stage-Aware Reinforcement Learning with Chain of Guidance for Text-to-Image Generation</title>
|
||||||
<link>https://arxiv.org/abs/2508.16292</link>
|
<link>https://arxiv.org/abs/2508.18032</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16292.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Wen-Han Hsieh, Elvis Hsieh, Dantong Niu, Trevor Darrell, Roei Herzig, David M. Chan</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Recently, Vision-Language-Action (VLA) models have demonstrated strong performance on a range of robotic tasks. These models rely on multimodal inputs, with language instructions playing a crucial role -- not only in predicting actions, but also in robustly interpreting user intent, even when the requests are impossible to fulfill. In this work, we investigate how VLAs can recognize, interpret, and respond to false-premise instructions: natural language commands that reference objects or conditions absent from the environment. We propose Instruct-Verify-and-Act (IVA), a unified framework that (i) detects when an instruction cannot be executed due to a false premise, (ii) engages in language-based clarification or correction, and (iii) grounds plausible alternatives in perception and action. Towards this end, we construct a large-scale instruction tuning setup with structured language prompts and train a VLA model capable of handling both accurate and erroneous requests. Our approach leverages a contextually augmented, semi-synthetic dataset containing paired positive and false-premise instructions, enabling robust detection and natural language correction. Our experiments show that IVA improves false premise detection accuracy by 97.56% over baselines, while increasing successful responses in false-premise scenarios by 50.78%.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18032.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yaqi Li, Peng Chen, Mingyang Han, Bu Pi, Haoxiang Shi, Runzhou Zhao, Yang Yao, Xuan Zhang, Jun Song</p><p><b>Upvotes:</b> 34</p><p><b>Summary:</b> Despite the promising progress of recent autoregressive models in text-to-image (T2I) generation, their ability to handle multi-attribute and ambiguous prompts remains limited. To address these limitations, existing works have applied chain-of-thought (CoT) to enable stage-aware visual synthesis and employed reinforcement learning (RL) to improve reasoning capabilities. However, most models provide reward signals only at the end of the generation stage. This monolithic final-only guidance makes it difficult to identify which stages contribute positively to the final outcome and may lead to suboptimal policies. To tackle this issue, we propose a Visual-Chain of Guidance (Visual-CoG) paradigm consisting of three stages: semantic reasoning, process refining, and outcome evaluation, with stage-aware rewards providing immediate guidance throughout the image generation pipeline. We further construct a visual cognition benchmark, VisCog-Bench, which comprises four subtasks to evaluate the effectiveness of semantic reasoning. Comprehensive evaluations on GenEval, T2I-CompBench, and the proposed VisCog-Bench show improvements of 15%, 5%, and 19%, respectively, demonstrating the superior performance of the proposed Visual-CoG. We will release all the resources soon.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16292</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18032</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 10:54:33 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 13:53:02 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>RotaTouille: Rotation Equivariant Deep Learning for Contours</title>
|
<title>Neither Valid nor Reliable? Investigating the Use of LLMs as Judges</title>
|
||||||
<link>https://arxiv.org/abs/2508.16359</link>
|
<link>https://arxiv.org/abs/2508.18076</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16359.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Odin Hoff Gardaa, Nello Blaser</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> Contours or closed planar curves are common in many domains. For example, they appear as object boundaries in computer vision, isolines in meteorology, and the orbits of rotating machinery. In many cases when learning from contour data, planar rotations of the input will result in correspondingly rotated outputs. It is therefore desirable that deep learning models be rotationally equivariant. In addition, contours are typically represented as an ordered sequence of edge points, where the choice of starting point is arbitrary. It is therefore also desirable for deep learning methods to be equivariant under cyclic shifts. We present RotaTouille, a deep learning framework for learning from contour data that achieves both rotation and cyclic shift equivariance through complex-valued circular convolution. We further introduce and characterize equivariant non-linearities, coarsening layers, and global pooling layers to obtain invariant representations for downstream tasks. Finally, we demonstrate the effectiveness of RotaTouille through experiments in shape classification, reconstruction, and contour regression.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18076.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Khaoula Chehbouni, Mohammed Haddou, Jackie Chi Kit Cheung, Golnoosh Farnadi</p><p><b>Upvotes:</b> 0</p><p><b>Summary:</b> Evaluating natural language generation (NLG) systems remains a core challenge of natural language processing (NLP), further complicated by the rise of large language models (LLMs) that aims to be general-purpose. Recently, large language models as judges (LLJs) have emerged as a promising alternative to traditional metrics, but their validity remains underexplored. This position paper argues that the current enthusiasm around LLJs may be premature, as their adoption has outpaced rigorous scrutiny of their reliability and validity as evaluators. Drawing on measurement theory from the social sciences, we identify and critically assess four core assumptions underlying the use of LLJs: their ability to act as proxies for human judgment, their capabilities as evaluators, their scalability, and their cost-effectiveness. We examine how each of these assumptions may be challenged by the inherent limitations of LLMs, LLJs, or current practices in NLG evaluation. To ground our analysis, we explore three applications of LLJs: text summarization, data annotation, and safety alignment. Finally, we highlight the need for more responsible evaluation practices in LLJs evaluation, to ensure that their growing role in the field supports, rather than undermines, progress in NLG.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16359</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18076</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 13:05:55 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 14:43:10 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>AetherCode: Evaluating LLMs' Ability to Win In Premier Programming Competitions</title>
|
<title>SpotEdit: Evaluating Visually-Guided Image Editing Methods</title>
|
||||||
<link>https://arxiv.org/abs/2508.16402</link>
|
<link>https://arxiv.org/abs/2508.18159</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.16402.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zihan Wang, Jiaze Chen, Zhicheng Liu, Markus Mak, Yidi Du, Geonsik Moon, Luoqi Xu, Aaron Tua, Kunshuo Peng, Jiayi Lu, Mingfei Xia, Boqian Zou, Chenyang Ran, Guang Tian, Shoutai Zhu, Yeheng Duan, Zhenghui Kang, Zhenxing Lin, Shangshu Li, Qiang Luo, Qingshen Long, Zhiyong Chen, Yihan Xiao, Yurong Wu, Daoguang Zan, Yuyi Fu, Mingxuan Wang, Ming Ding</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Competitive programming has emerged as a critical benchmark for evaluating the reasoning and coding capabilities of Large Language Models (LLMs). Despite impressive progress on existing benchmarks, we argue that current evaluations overstate model proficiency, masking a substantial gap between LLMs and elite human programmers. This gap arises from two key limitations: insufficient difficulty and scope of benchmark problems, and evaluation bias from low-quality test cases. To address these shortcomings, we present AetherCode, a new benchmark that draws problems from premier programming competitions such as IOI and ICPC, offering broader coverage and higher difficulty. AetherCode further incorporates comprehensive, expert-validated test suites built through a hybrid of automated generation and human curation, ensuring rigorous and reliable assessment. By combining challenging problem design with robust evaluation, AetherCode provides a more faithful measure of LLM capabilities and sets a new standard for future research in code reasoning.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18159.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Sara Ghazanfari, Wei-An Lin, Haitong Tian, Ersin Yumer</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> Visually-guided image editing, where edits are conditioned on both visual cues and textual prompts, has emerged as a powerful paradigm for fine-grained, controllable content generation. Although recent generative models have shown remarkable capabilities, existing evaluations remain simple and insufficiently representative of real-world editing challenges. We present SpotEdit, a comprehensive benchmark designed to systematically assess visually-guided image editing methods across diverse diffusion, autoregressive, and hybrid generative models, uncovering substantial performance disparities. To address a critical yet underexplored challenge, our benchmark includes a dedicated component on hallucination, highlighting how leading models, such as GPT-4o, often hallucinate the existence of a visual cue and erroneously perform the editing task. Our code and benchmark are publicly released at https://github.com/SaraGhazanfari/SpotEdit.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2508.16402</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18159</guid>
|
||||||
<pubDate>Fri, 22 Aug 2025 14:04:55 +0000</pubDate>
|
<pubDate>Mon, 25 Aug 2025 16:08:57 +0000</pubDate>
|
||||||
|
</item>
|
||||||
|
<item>
|
||||||
|
<title>ST-Raptor: LLM-Powered Semi-Structured Table Question Answering</title>
|
||||||
|
<link>https://arxiv.org/abs/2508.18190</link>
|
||||||
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18190.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zirui Tang, Boyu Niu, Xuanhe Zhou, Boxiu Li, Wei Zhou, Jiannan Wang, Guoliang Li, Xinyi Zhang, Fan Wu</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> Semi-structured tables, widely used in real-world applications (e.g., financial reports, medical records, transactional orders), often involve flexible and complex layouts (e.g., hierarchical headers and merged cells). These tables generally rely on human analysts to interpret table layouts and answer relevant natural language questions, which is costly and inefficient. To automate the procedure, existing methods face significant challenges. First, methods like NL2SQL require converting semi-structured tables into structured ones, which often causes substantial information loss. Second, methods like NL2Code and multi-modal LLM QA struggle to understand the complex layouts of semi-structured tables and cannot accurately answer corresponding questions. To this end, we propose ST-Raptor, a tree-based framework for semi-structured table question answering using large language models. First, we introduce the Hierarchical Orthogonal Tree (HO-Tree), a structural model that captures complex semi-structured table layouts, along with an effective algorithm for constructing the tree. Second, we define a set of basic tree operations to guide LLMs in executing common QA tasks. Given a user question, ST-Raptor decomposes it into simpler sub-questions, generates corresponding tree operation pipelines, and conducts operation-table alignment for accurate pipeline execution. Third, we incorporate a two-stage verification mechanism: forward validation checks the correctness of execution steps, while backward validation evaluates answer reliability by reconstructing queries from predicted answers. To benchmark the performance, we present SSTQA, a dataset of 764 questions over 102 real-world semi-structured tables. Experiments show that ST-Raptor outperforms nine baselines by up to 20% in answer accuracy. The code is available at https://github.com/weAIDB/ST-Raptor.</p></description>
|
||||||
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18190</guid>
|
||||||
|
<pubDate>Mon, 25 Aug 2025 16:48:51 +0000</pubDate>
|
||||||
|
</item>
|
||||||
|
<item>
|
||||||
|
<title>Hermes 4 Technical Report</title>
|
||||||
|
<link>https://arxiv.org/abs/2508.18255</link>
|
||||||
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18255.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ryan Teknium, Roger Jin, Jai Suphavadeeprasit, Dakota Mahan, Jeffrey Quesnelle, Joe Li, Chen Guang, Shannon Sands, Karan Malhotra</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> We present Hermes 4, a family of hybrid reasoning models that combine structured, multi-turn reasoning with broad instruction-following ability. We describe the challenges encountered during data curation, synthesis, training, and evaluation, and outline the solutions employed to address these challenges at scale. We comprehensively evaluate across mathematical reasoning, coding, knowledge, comprehension, and alignment benchmarks, and we report both quantitative performance and qualitative behavioral analysis. To support open research, all model weights are published publicly at https://huggingface.co/collections/NousResearch/hermes-4-collection-68a731bfd452e20816725728</p></description>
|
||||||
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18255</guid>
|
||||||
|
<pubDate>Mon, 25 Aug 2025 17:45:06 +0000</pubDate>
|
||||||
|
</item>
|
||||||
|
<item>
|
||||||
|
<title>MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs</title>
|
||||||
|
<link>https://arxiv.org/abs/2508.18264</link>
|
||||||
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18264.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Sixun Dong, Juhua Hu, Mian Zhang, Ming Yin, Yanjie Fu, Qi Qian</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> Vision-Language Models (VLMs) demonstrate impressive performance in understanding visual content with language instruction by converting visual input to vision tokens. However, redundancy in vision tokens results in the degenerated inference efficiency of VLMs. While many algorithms have been proposed to reduce the number of vision tokens, most of them apply only unimodal information (i.e., vision/text) for pruning and ignore the inherent multimodal property of vision-language tasks. Moreover, it lacks a generic criterion that can be applied to different modalities. To mitigate this limitation, in this work, we propose to leverage both vision and text tokens to select informative vision tokens by the criterion of coverage. We first formulate the subset selection problem as a maximum coverage problem. Afterward, a subset of vision tokens is optimized to cover the text tokens and the original set of vision tokens, simultaneously. Finally, a VLM agent can be adopted to further improve the quality of text tokens for guiding vision pruning. The proposed method MMTok is extensively evaluated on benchmark datasets with different VLMs. The comparison illustrates that vision and text information are complementary, and combining multimodal information can surpass the unimodal baseline with a clear margin. Moreover, under the maximum coverage criterion on the POPE dataset, our method achieves a 1.87x speedup while maintaining 98.7% of the original performance on LLaVA-NeXT-13B. Furthermore, with only four vision tokens, it still preserves 87.7% of the original performance on LLaVA-1.5-7B. These results highlight the effectiveness of coverage in token selection.</p></description>
|
||||||
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18264</guid>
|
||||||
|
<pubDate>Mon, 25 Aug 2025 17:57:49 +0000</pubDate>
|
||||||
|
</item>
|
||||||
|
<item>
|
||||||
|
<title>InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency</title>
|
||||||
|
<link>https://arxiv.org/abs/2508.18265</link>
|
||||||
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2508.18265.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Bowen Zhou, Weijie Su, Kai Chen, Yu Qiao, Wenhai Wang, Gen Luo</p><p><b>Upvotes:</b> 112</p><p><b>Summary:</b> We introduce InternVL 3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the Cascade Reinforcement Learning (Cascade RL) framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a Visual Resolution Router (ViR) that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled Vision-Language Deployment (DvD) strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0\% gain in overall reasoning performance and a 4.05times inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks -- narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.</p></description>
|
||||||
|
<guid isPermaLink="false">https://arxiv.org/abs/2508.18265</guid>
|
||||||
|
<pubDate>Mon, 25 Aug 2025 17:58:17 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
</channel>
|
</channel>
|
||||||
</rss>
|
</rss>
|
||||||
|
|||||||
Reference in New Issue
Block a user