bot: update RSS feed
This commit is contained in:
github-actions[bot]
204
feed.xml
204
feed.xml
@@ -7,160 +7,118 @@
|
|||||||
<docs>http://www.rssboard.org/rss-specification</docs>
|
<docs>http://www.rssboard.org/rss-specification</docs>
|
||||||
<generator>python-feedgen</generator>
|
<generator>python-feedgen</generator>
|
||||||
<language>en</language>
|
<language>en</language>
|
||||||
<lastBuildDate>Thu, 26 Jun 2025 00:02:39 +0000</lastBuildDate>
|
<lastBuildDate>Fri, 27 Jun 2025 00:02:40 +0000</lastBuildDate>
|
||||||
<item>
|
<item>
|
||||||
<title>Lost in the Mix: Evaluating LLM Understanding of Code-Switched Text</title>
|
<title>DualTHOR: A Dual-Arm Humanoid Simulation Platform for Contingency-Aware Planning</title>
|
||||||
<link>https://arxiv.org/abs/2506.14012</link>
|
<link>https://arxiv.org/abs/2506.16012</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.14012.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Amr Mohamed, Yang Zhang, Michalis Vazirgiannis, Guokan Shang</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Code-switching (CSW) is the act of alternating between two or more languages within a single discourse. This phenomenon is widespread in multilingual communities, and increasingly prevalent in online content, where users naturally mix languages in everyday communication. As a result, Large Language Models (LLMs), now central to content processing and generation, are frequently exposed to code-switched inputs. Given their widespread use, it is crucial to understand how LLMs process and reason about such mixed-language text. This paper presents a systematic evaluation of LLM comprehension under code-switching by generating CSW variants of established reasoning and comprehension benchmarks. While degradation is evident when foreign tokens disrupt English textx2013even under linguistic constraintsx2013embedding English into other languages often improves comprehension. Though prompting yields mixed results, fine-tuning offers a more stable path to degradation mitigation.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16012.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Boyu Li, Siyuan He, Hang Xu, Haoqi Yuan, Yu Zang, Liwei Hu, Junpeng Yue, Zhenxiong Jiang, Pengbo Hu, Börje F. Karlsson, Yehui Tang, Zongqing Lu</p><p><b>Upvotes:</b> 18</p><p><b>Summary:</b> Developing embodied agents capable of performing complex interactive tasks in real-world scenarios remains a fundamental challenge in embodied AI. Although recent advances in simulation platforms have greatly enhanced task diversity to train embodied Vision Language Models (VLMs), most platforms rely on simplified robot morphologies and bypass the stochastic nature of low-level execution, which limits their transferability to real-world robots. To address these issues, we present a physics-based simulation platform DualTHOR for complex dual-arm humanoid robots, built upon an extended version of AI2-THOR. Our simulator includes real-world robot assets, a task suite for dual-arm collaboration, and inverse kinematics solvers for humanoid robots. We also introduce a contingency mechanism that incorporates potential failures through physics-based low-level execution, bridging the gap to real-world scenarios. Our simulator enables a more comprehensive evaluation of the robustness and generalization of VLMs in household environments. Extensive evaluations reveal that current VLMs struggle with dual-arm coordination and exhibit limited robustness in realistic environments with contingencies, highlighting the importance of using our simulator to develop more capable VLMs for embodied tasks. The code is available at https://github.com/ds199895/DualTHOR.git.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.14012</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.16012</guid>
|
||||||
<pubDate>Mon, 16 Jun 2025 21:19:27 +0000</pubDate>
|
<pubDate>Thu, 19 Jun 2025 04:13:36 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Intelligent Operation and Maintenance and Prediction Model Optimization for Improving Wind Power Generation Efficiency</title>
|
<title>RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation</title>
|
||||||
<link>https://arxiv.org/abs/2506.16095</link>
|
<link>https://arxiv.org/abs/2506.18088</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16095.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Xun Liu, Xiaobin Wu, Jiaqi He, Rajan Das Gupta</p><p><b>Upvotes:</b> 4</p><p><b>Summary:</b> This study explores the effectiveness of predictive maintenance models and the optimization of intelligent Operation and Maintenance (O&M) systems in improving wind power generation efficiency. Through qualitative research, structured interviews were conducted with five wind farm engineers and maintenance managers, each with extensive experience in turbine operations. Using thematic analysis, the study revealed that while predictive maintenance models effectively reduce downtime by identifying major faults, they often struggle with detecting smaller, gradual failures. Key challenges identified include false positives, sensor malfunctions, and difficulties in integrating new models with older turbine systems. Advanced technologies such as digital twins, SCADA systems, and condition monitoring have significantly enhanced turbine maintenance practices. However, these technologies still require improvements, particularly in AI refinement and real-time data integration. The findings emphasize the need for continuous development to fully optimize wind turbine performance and support the broader adoption of renewable energy.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18088.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Qiwei Liang, Zixuan Li, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, Kailun Su, Tianling Xu, Guodong Liu, Mengkang Hu, Huan-ang Gao, Kaixuan Wang, Zhixuan Liang, Yusen Qin, Xiaokang Yang, Ping Luo, Yao Mu</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> Simulation-based data synthesis has emerged as a powerful paradigm for enhancing real-world robotic manipulation. However, existing synthetic datasets remain insufficient for robust bimanual manipulation due to two challenges: (1) the lack of an efficient, scalable data generation method for novel tasks, and (2) oversimplified simulation environments that fail to capture real-world complexity. We present RoboTwin 2.0, a scalable simulation framework that enables automated, large-scale generation of diverse and realistic data, along with unified evaluation protocols for dual-arm manipulation. We first construct RoboTwin-OD, a large-scale object library comprising 731 instances across 147 categories, each annotated with semantic and manipulation-relevant labels. Building on this foundation, we develop an expert data synthesis pipeline that combines multimodal large language models (MLLMs) with simulation-in-the-loop refinement to generate task-level execution code automatically. To improve sim-to-real transfer, RoboTwin 2.0 incorporates structured domain randomization along five axes: clutter, lighting, background, tabletop height and language instructions, thereby enhancing data diversity and policy robustness. We instantiate this framework across 50 dual-arm tasks spanning five robot embodiments, and pre-collect over 100,000 domain-randomized expert trajectories. Empirical results show a 10.9% gain in code generation success and improved generalization to novel real-world scenarios. A VLA model fine-tuned on our dataset achieves a 367% relative improvement (42.0% vs. 9.0%) on unseen scene real-world tasks, while zero-shot models trained solely on our synthetic data achieve a 228% relative gain, highlighting strong generalization without real-world supervision. We release the data generator, benchmark, dataset, and code to support scalable research in robust bimanual manipulation.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.16095</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.18088</guid>
|
||||||
<pubDate>Thu, 19 Jun 2025 07:31:03 +0000</pubDate>
|
<pubDate>Sun, 22 Jun 2025 16:26:53 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning</title>
|
<title>ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation</title>
|
||||||
<link>https://arxiv.org/abs/2506.16141</link>
|
<link>https://arxiv.org/abs/2506.18095</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.16141.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yi Chen, Yuying Ge, Rui Wang, Yixiao Ge, Junhao Cheng, Ying Shan, Xihui Liu</p><p><b>Upvotes:</b> 24</p><p><b>Summary:</b> Recent reinforcement learning approaches, such as outcome-supervised GRPO, have advanced Chain-of-Thought reasoning in large language models (LLMs), yet their adaptation to multimodal LLMs (MLLMs) is unexplored. To address the lack of rigorous evaluation for MLLM post-training methods, we introduce SEED-Bench-R1, a benchmark with complex real-world videos requiring balanced perception and reasoning. It offers a large training set and evaluates generalization across three escalating challenges: in-distribution, cross-environment, and cross-environment-task scenarios. Using SEED-Bench-R1, we find that standard GRPO, while improving answer accuracy, often reduces logical coherence between reasoning steps and answers, with only a 57.9% consistency rate. This stems from reward signals focusing solely on final answers, encouraging shortcuts, and strict KL penalties limiting exploration.To address this, we propose GRPO-CARE, a consistency-aware RL framework optimizing both answer correctness and reasoning coherence without explicit supervision. GRPO-CARE introduces a two-tiered reward: (1) a base reward for answer correctness, and (2) an adaptive consistency bonus, computed by comparing the model's reasoning-to-answer likelihood (via a slowly-evolving reference model) against group peers.This dual mechanism amplifies rewards for reasoning paths that are both correct and logically consistent. Replacing KL penalties with this adaptive bonus, GRPO-CARE outperforms standard GRPO on SEED-Bench-R1, achieving a 6.7% performance gain on the hardest evaluation level and a 24.5% improvement in consistency. It also shows strong transferability, improving model performance across diverse video understanding benchmarks. Our work contributes a systematically designed benchmark and a generalizable post-training framework, advancing the development of more interpretable and robust MLLMs.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18095.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Junying Chen, Zhenyang Cai, Pengcheng Chen, Shunian Chen, Ke Ji, Xidong Wang, Yunjin Yang, Benyou Wang</p><p><b>Upvotes:</b> 53</p><p><b>Summary:</b> Recent advances in multimodal generative models have unlocked photorealistic, instruction-aligned image generation, yet leading systems like GPT-4o-Image remain proprietary and inaccessible. To democratize these capabilities, we present ShareGPT-4o-Image, the first dataset comprising 45K text-to-image and 46K text-and-image-to-image data, all synthesized using GPT-4o's image generation capabilities for distilling its advanced image generation abilities. Leveraging this dataset, we develop Janus-4o, a multimodal large language model capable of both text-to-image and text-and-image-to-image generation. Janus-4o not only significantly improves text-to-image generation over its predecessor, Janus-Pro, but also newly supports text-and-image-to-image generation. Notably, it achieves impressive performance in text-and-image-to-image generation from scratch, using only 91K synthetic samples and 6 hours of training on an 8 A800-GPU machine. We hope the release of ShareGPT-4o-Image and Janus-4o will foster open research in photorealistic, instruction-aligned image generation.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.16141</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.18095</guid>
|
||||||
<pubDate>Thu, 19 Jun 2025 08:49:13 +0000</pubDate>
|
<pubDate>Sun, 22 Jun 2025 16:51:09 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent</title>
|
<title>Use Property-Based Testing to Bridge LLM Code Generation and Validation</title>
|
||||||
<link>https://arxiv.org/abs/2506.17612</link>
|
<link>https://arxiv.org/abs/2506.18315</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.17612.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yunlong Lin, Zixu Lin, Kunjie Lin, Jinbin Bai, Panwang Pan, Chenxin Li, Haoyu Chen, Zhongdao Wang, Xinghao Ding, Wenbo Li, Shuicheng Yan</p><p><b>Upvotes:</b> 17</p><p><b>Summary:</b> Photo retouching has become integral to contemporary visual storytelling, enabling users to capture aesthetics and express creativity. While professional tools such as Adobe Lightroom offer powerful capabilities, they demand substantial expertise and manual effort. In contrast, existing AI-based solutions provide automation but often suffer from limited adjustability and poor generalization, failing to meet diverse and personalized editing needs. To bridge this gap, we introduce JarvisArt, a multi-modal large language model (MLLM)-driven agent that understands user intent, mimics the reasoning process of professional artists, and intelligently coordinates over 200 retouching tools within Lightroom. JarvisArt undergoes a two-stage training process: an initial Chain-of-Thought supervised fine-tuning to establish basic reasoning and tool-use skills, followed by Group Relative Policy Optimization for Retouching (GRPO-R) to further enhance its decision-making and tool proficiency. We also propose the Agent-to-Lightroom Protocol to facilitate seamless integration with Lightroom. To evaluate performance, we develop MMArt-Bench, a novel benchmark constructed from real-world user edits. JarvisArt demonstrates user-friendly interaction, superior generalization, and fine-grained control over both global and local adjustments, paving a new avenue for intelligent photo retouching. Notably, it outperforms GPT-4o with a 60% improvement in average pixel-level metrics on MMArt-Bench for content fidelity, while maintaining comparable instruction-following capabilities. Project Page: https://jarvisart.vercel.app/.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18315.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Lehan He, Zeren Chen, Zhe Zhang, Jing Shao, Xiang Gao, Lu Sheng</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Large Language Models (LLMs) excel at code generation, but ensuring their outputs to be functionally correct, especially in complex programming tasks, is a persistent challenge. While traditional Test-Driven Development (TDD) offers a path for code refinement, its efficacy with LLMs is often undermined by the scarcity of high-quality test cases or the pitfalls of automated test generation, including biased tests or inaccurate output predictions that can misdirect the correction process. This paper introduces Property-Generated Solver, a novel framework that leverages Property-Based Testing (PBT) to validate high-level program properties or invariants, instead of relying on specific input-output examples. These properties are often simpler to define and verify than directly predicting exhaustive test oracles, breaking the "cycle of self-deception" where tests might share flaws with the code they are meant to validate. Property-Generated Solver employs two collaborative LLM-based agents: a Generator dedicated to code generation and iterative refinement, and a Tester that manages the PBT life-cycle and formulate semantically rich feedback from property violations. The resulting comprehensive and actionable feedback then guides the Generator in its refinement efforts. By establishing PBT as the core validation engine within this iterative, closed-loop paradigm, Property-Generated Solver provides a robust mechanism for steering LLMs towards more correct and generalizable code. Extensive experimental results on multiple code generation benchmarks demonstrate that Property-Generated Solver achieves substantial pass@1 improvements, ranging from 23.1% to 37.3% relative gains over established TDD methods.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.17612</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.18315</guid>
|
||||||
<pubDate>Sat, 21 Jun 2025 06:36:00 +0000</pubDate>
|
<pubDate>Mon, 23 Jun 2025 06:01:12 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models</title>
|
<title>The Debugging Decay Index: Rethinking Debugging Strategies for Code LLMs</title>
|
||||||
<link>https://arxiv.org/abs/2506.18945</link>
|
<link>https://arxiv.org/abs/2506.18403</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18945.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zihan Wang, Rui Pan, Jiarui Yao, Robert Csordas, Linjie Li, Lu Yin, Jiajun Wu, Tong Zhang, Manling Li, Shiwei Liu</p><p><b>Upvotes:</b> 10</p><p><b>Summary:</b> We propose Chain-of-Experts (CoE), a new Mixture-of-Experts (MoE) architecture that introduces sequential expert communication within each layer. Unlike traditional MoE models, where experts operate independently in parallel, CoE processes tokens iteratively across a chain of experts inside a layer. To support dynamic expert selection across iterations, CoE employs a dedicated router at each iteration step within a layer. This design allows tokens to re-evaluate and select different experts during each iteration, rather than being statically assigned. As a result, CoE introduces a flexible routing mechanism that increases the diversity of expert combinations and enriches the model's representational capacity. CoE demonstrates improved performance under fixed compute: on math reasoning tasks, it reduces validation loss from 1.20 to 1.12 compared to a standard MoE. Beyond performance, CoE offers a new scaling axis: depth through expert iteration, which complements conventional width/depth scaling. For example, using 2x iterations matches the performance of 3x expert selections (in width), while reducing memory usage by 17.6-42% relative to other scaling strategies. Our analysis reveals that CoE's benefits stem from its iterative residual structure and enhanced expert specialization empowered by iterative routing, which together unlock more expressive representations. Code is available at https://github.com/ZihanWang314/coe.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18403.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Muntasir Adnan, Carlos C. N. Kuhn</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> The effectiveness of AI debugging follows a predictable exponential decay pattern; most models lose 60-80% of their debugging capability within just 2-3 attempts, despite iterative debugging being a critical capability for practical code generation systems. We introduce the Debugging Decay Index (DDI), a mathematical framework that quantifies when debugging becomes ineffective and predicts intervention points. Our strategic fresh start approach shifts from exploitation to exploration at strategic points in the debugging process, demonstrating that well-timed interventions can rescue the effectiveness of debugging. DDI reveals a fundamental limitation in current AI debugging and provides the first quantitative framework for optimising iterative code generation strategies.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.18945</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.18403</guid>
|
||||||
<pubDate>Mon, 23 Jun 2025 02:15:43 +0000</pubDate>
|
<pubDate>Mon, 23 Jun 2025 08:40:45 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications</title>
|
<title>Is There a Case for Conversation Optimized Tokenizers in Large Language Models?</title>
|
||||||
<link>https://arxiv.org/abs/2506.18951</link>
|
<link>https://arxiv.org/abs/2506.18674</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18951.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jinyang Li, Xiaolong Li, Ge Qu, Per Jacobsson, Bowen Qin, Binyuan Hui, Shuzheng Si, Nan Huo, Xiaohan Xu, Yue Zhang, Ziwei Tang, Yuanshuai Li, Florensia Widjaja, Xintong Zhu, Feige Zhou, Yongfeng Huang, Yannis Papakonstantinou, Fatma Ozcan, Chenhao Ma, Reynold Cheng</p><p><b>Upvotes:</b> 14</p><p><b>Summary:</b> Resolution of complex SQL issues persists as a significant bottleneck in real-world database applications. Current Large Language Models (LLMs), while adept at text-to-SQL translation, have not been rigorously evaluated on the more challenging task of debugging SQL issues. To address this gap, we introduce BIRD-CRITIC, a new SQL issue debugging benchmark comprising 530 PostgreSQL tasks (BIRD-CRITIC-PG) and 570 multi-dialect tasks (BIRD-CRITIC-Multi), distilled from authentic user issues and replayed within new environments to facilitate rigorous evaluation. Baseline evaluations underscore the task's complexity, with the leading reasoning model O3-Mini achieving only 38.87% success rate on BIRD-CRITIC-PG and 33.33% on BIRD-CRITIC-Multi. Meanwhile, advancing open-source models for database tasks is crucial for empowering local development while safeguarding data privacy. Therefore, we present Six-Gym (Sql-fIX-Gym), a training environment for elevating open-source model capabilities for SQL issue debugging. This environment leverages SQL-Rewind strategy, which automatically generates executable issue-solution datasets by reverse-engineering issues from verified SQLs. However, popular trajectory-based fine-tuning methods do not explore substantial supervisory signals. We further propose f-Plan Boosting, which extracts high-level debugging plans from SQL solutions, enabling teacher LLMs to produce 73.7% more successful trajectories for training. We integrate these components into an open-source agent, Bird-Fixer. Based on Qwen-2.5-Coder-14B, Bird-Fixer achieves 38.11% success rate on BIRD-CRITIC-PG and 29.65% on BIRD-CRITIC-Multi, surpassing leading proprietary models such as Claude-3.7-Sonnet and GPT-4.1, marking a significant step toward democratizing sophisticated SQL-debugging capabilities. The leaderboard and source code are available: https://bird-critic.github.io/</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18674.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Raquel Ferrando, Javier Conde, Gonzalo Martínez, Pedro Reviriego</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> The computational and energy costs of Large Language Models (LLMs) have increased exponentially driven by the growing model sizes and the massive adoption of LLMs by hundreds of millions of users. The unit cost of an LLM is the computation of a token. Therefore, the tokenizer plays an important role in the efficiency of a model, and they are carefully optimized to minimize the number of tokens for the text in their training corpus. One of the most popular applications of LLMs are chatbots that interact with users. A key observation is that, for those chatbots, what is important is the performance of the tokenizer in the user text input and the chatbot responses. Those are most likely different from the text in the training corpus. So, a question that immediately arises is whether there is a potential benefit in optimizing tokenizers for chatbot conversations. In this paper, this idea is explored for different tokenizers by using a publicly available corpus of chatbot conversations to redesign their vocabularies and evaluate their performance in this domain. The results show that conversation-optimized tokenizers consistently reduce the number of tokens in chatbot dialogues, which can lead to meaningful energy savings, in the range of 5% to 10% while having minimal or even slightly positive impact on tokenization efficiency for the original training corpus.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.18951</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.18674</guid>
|
||||||
<pubDate>Mon, 23 Jun 2025 09:41:37 +0000</pubDate>
|
<pubDate>Mon, 23 Jun 2025 14:18:46 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Matrix-Game: Interactive World Foundation Model</title>
|
<title>Inverse-and-Edit: Effective and Fast Image Editing by Cycle Consistency Models</title>
|
||||||
<link>https://arxiv.org/abs/2506.18701</link>
|
<link>https://arxiv.org/abs/2506.19103</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18701.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yifan Zhang, Chunli Peng, Boyang Wang, Puyi Wang, Qingcheng Zhu, Fei Kang, Biao Jiang, Zedong Gao, Eric Li, Yang Liu, Yahui Zhou</p><p><b>Upvotes:</b> 38</p><p><b>Summary:</b> We introduce Matrix-Game, an interactive world foundation model for controllable game world generation. Matrix-Game is trained using a two-stage pipeline that first performs large-scale unlabeled pretraining for environment understanding, followed by action-labeled training for interactive video generation. To support this, we curate Matrix-Game-MC, a comprehensive Minecraft dataset comprising over 2,700 hours of unlabeled gameplay video clips and over 1,000 hours of high-quality labeled clips with fine-grained keyboard and mouse action annotations. Our model adopts a controllable image-to-world generation paradigm, conditioned on a reference image, motion context, and user actions. With over 17 billion parameters, Matrix-Game enables precise control over character actions and camera movements, while maintaining high visual quality and temporal coherence. To evaluate performance, we develop GameWorld Score, a unified benchmark measuring visual quality, temporal quality, action controllability, and physical rule understanding for Minecraft world generation. Extensive experiments show that Matrix-Game consistently outperforms prior open-source Minecraft world models (including Oasis and MineWorld) across all metrics, with particularly strong gains in controllability and physical consistency. Double-blind human evaluations further confirm the superiority of Matrix-Game, highlighting its ability to generate perceptually realistic and precisely controllable videos across diverse game scenarios. To facilitate future research on interactive image-to-world generation, we will open-source the Matrix-Game model weights and the GameWorld Score benchmark at https://github.com/SkyworkAI/Matrix-Game.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19103.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ilia Beletskii, Andrey Kuznetsov, Aibek Alanov</p><p><b>Upvotes:</b> 35</p><p><b>Summary:</b> Recent advances in image editing with diffusion models have achieved impressive results, offering fine-grained control over the generation process. However, these methods are computationally intensive because of their iterative nature. While distilled diffusion models enable faster inference, their editing capabilities remain limited, primarily because of poor inversion quality. High-fidelity inversion and reconstruction are essential for precise image editing, as they preserve the structural and semantic integrity of the source image. In this work, we propose a novel framework that enhances image inversion using consistency models, enabling high-quality editing in just four steps. Our method introduces a cycle-consistency optimization strategy that significantly improves reconstruction accuracy and enables a controllable trade-off between editability and content preservation. We achieve state-of-the-art performance across various image editing tasks and datasets, demonstrating that our method matches or surpasses full-step diffusion models while being substantially more efficient. The code of our method is available on GitHub at https://github.com/ControlGenAI/Inverse-and-Edit.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.18701</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.19103</guid>
|
||||||
<pubDate>Mon, 23 Jun 2025 14:40:49 +0000</pubDate>
|
<pubDate>Mon, 23 Jun 2025 20:34:43 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>USAD: Universal Speech and Audio Representation via Distillation</title>
|
<title>Thought Anchors: Which LLM Reasoning Steps Matter?</title>
|
||||||
<link>https://arxiv.org/abs/2506.18843</link>
|
<link>https://arxiv.org/abs/2506.19143</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.18843.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Heng-Jui Chang, Saurabhchand Bhati, James Glass, Alexander H. Liu</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Self-supervised learning (SSL) has revolutionized audio representations, yet models often remain domain-specific, focusing on either speech or non-speech tasks. In this work, we present Universal Speech and Audio Distillation (USAD), a unified approach to audio representation learning that integrates diverse audio types - speech, sound, and music - into a single model. USAD employs efficient layer-to-layer distillation from domain-specific SSL models to train a student on a comprehensive audio dataset. USAD offers competitive performance across various benchmarks and datasets, including frame and instance-level speech processing tasks, audio tagging, and sound classification, achieving near state-of-the-art results with a single encoder on SUPERB and HEAR benchmarks.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19143.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Paul C. Bogdan, Uzay Macar, Neel Nanda, Arthur Conmy</p><p><b>Upvotes:</b> 1</p><p><b>Summary:</b> Reasoning large language models have recently achieved state-of-the-art performance in many fields. However, their long-form chain-of-thought reasoning creates interpretability challenges as each generated token depends on all previous ones, making the computation harder to decompose. We argue that analyzing reasoning traces at the sentence level is a promising approach to understanding reasoning processes. We present three complementary attribution methods: (1) a black-box method measuring each sentence's counterfactual importance by comparing final answers across 100 rollouts conditioned on the model generating that sentence or one with a different meaning; (2) a white-box method of aggregating attention patterns between pairs of sentences, which identified ``broadcasting'' sentences that receive disproportionate attention from all future sentences via ``receiver'' attention heads; (3) a causal attribution method measuring logical connections between sentences by suppressing attention toward one sentence and measuring the effect on each future sentence's tokens. Each method provides evidence for the existence of thought anchors, reasoning steps that have outsized importance and that disproportionately influence the subsequent reasoning process. These thought anchors are typically planning or backtracking sentences. We provide an open-source tool (www.thought-anchors.com) for visualizing the outputs of our methods, and present a case study showing converging patterns across methods that map how a model performs multi-step reasoning. The consistency across methods demonstrates the potential of sentence-level analysis for a deeper understanding of reasoning models.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.18843</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.19143</guid>
|
||||||
<pubDate>Mon, 23 Jun 2025 17:02:00 +0000</pubDate>
|
<pubDate>Mon, 23 Jun 2025 21:28:45 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in LLMs</title>
|
<title>MATE: LLM-Powered Multi-Agent Translation Environment for Accessibility Applications</title>
|
||||||
<link>https://arxiv.org/abs/2506.19290</link>
|
<link>https://arxiv.org/abs/2506.19502</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19290.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Liang Zeng, Yongcong Li, Yuzhen Xiao, Changshi Li, Chris Yuhao Liu, Rui Yan, Tianwen Wei, Jujie He, Xuchen Song, Yang Liu, Yahui Zhou</p><p><b>Upvotes:</b> 27</p><p><b>Summary:</b> Software engineering (SWE) has recently emerged as a crucial testbed for next-generation LLM agents, demanding inherent capabilities in two critical dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds) and long-context dependency resolution (e.g., >32k tokens). However, the data curation process in SWE remains notoriously time-consuming, as it heavily relies on manual annotation for code file filtering and the setup of dedicated runtime environments to execute and validate unit tests. Consequently, most existing datasets are limited to only a few thousand GitHub-sourced instances. To this end, we propose an incremental, automated data-curation pipeline that systematically scales both the volume and diversity of SWE datasets. Our dataset comprises 10,169 real-world Python task instances from 2,531 distinct GitHub repositories, each accompanied by a task specified in natural language and a dedicated runtime-environment image for automated unit-test validation. We have carefully curated over 8,000 successfully runtime-validated training trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE model on these trajectories, we uncover a striking data scaling phenomenon: the trained model's performance for software engineering capabilities in LLMs continues to improve as the data size increases, showing no signs of saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on the SWE-bench Verified benchmark without using verifiers or multiple rollouts, establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based LLMs built on the OpenHands agent framework. Furthermore, with the incorporation of test-time scaling techniques, the performance further improves to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter models. We release the Skywork-SWE-32B model checkpoint to accelerate future research.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19502.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Aleksandr Algazinov, Matt Laing, Paul Laban</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Accessibility remains a critical concern in today's society, as many technologies are not developed to support the full range of user needs. Existing multi-agent systems (MAS) often cannot provide comprehensive assistance for users in need due to the lack of customization stemming from closed-source designs. Consequently, individuals with disabilities frequently encounter significant barriers when attempting to interact with digital environments. We introduce MATE, a multimodal accessibility MAS, which performs the modality conversions based on the user's needs. The system is useful for assisting people with disabilities by ensuring that data will be converted to an understandable format. For instance, if the user cannot see well and receives an image, the system converts this image to its audio description. MATE can be applied to a wide range of domains, industries, and areas, such as healthcare, and can become a useful assistant for various groups of users. The system supports multiple types of models, ranging from LLM API calling to using custom machine learning (ML) classifiers. This flexibility ensures that the system can be adapted to various needs and is compatible with a wide variety of hardware. Since the system is expected to run locally, it ensures the privacy and security of sensitive information. In addition, the framework can be effectively integrated with institutional technologies (e.g., digital healthcare service) for real-time user assistance. Furthermore, we introduce ModCon-Task-Identifier, a model that is capable of extracting the precise modality conversion task from the user input. Numerous experiments show that ModCon-Task-Identifier consistently outperforms other LLMs and statistical models on our custom data. Our code and data are publicly available at https://github.com/AlgazinovAleksandr/Multi-Agent-MATE.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19290</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.19502</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 03:53:36 +0000</pubDate>
|
<pubDate>Tue, 24 Jun 2025 10:40:23 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Mem4Nav: Boosting Vision-and-Language Navigation in Urban Environments with a Hierarchical Spatial-Cognition Long-Short Memory System</title>
|
<title>Outlier-Safe Pre-Training for Robust 4-Bit Quantization of Large Language Models</title>
|
||||||
<link>https://arxiv.org/abs/2506.19433</link>
|
<link>https://arxiv.org/abs/2506.19697</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19433.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Lixuan He, Haoyu Dong, Zhenxing Chen, Yangcheng Yu, Jie Feng, Yong Li</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Vision-and-Language Navigation (VLN) in large-scale urban environments requires embodied agents to ground linguistic instructions in complex scenes and recall relevant experiences over extended time horizons. Prior modular pipelines offer interpretability but lack unified memory, while end-to-end (M)LLM agents excel at fusing vision and language yet remain constrained by fixed context windows and implicit spatial reasoning. We introduce Mem4Nav, a hierarchical spatial-cognition long-short memory system that can augment any VLN backbone. Mem4Nav fuses a sparse octree for fine-grained voxel indexing with a semantic topology graph for high-level landmark connectivity, storing both in trainable memory tokens embedded via a reversible Transformer. Long-term memory (LTM) compresses and retains historical observations at both octree and graph nodes, while short-term memory (STM) caches recent multimodal entries in relative coordinates for real-time obstacle avoidance and local planning. At each step, STM retrieval sharply prunes dynamic context, and, when deeper history is needed, LTM tokens are decoded losslessly to reconstruct past embeddings. Evaluated on Touchdown and Map2Seq across three backbones (modular, state-of-the-art VLN with prompt-based LLM, and state-of-the-art VLN with strided-attention MLLM), Mem4Nav yields 7-13 pp gains in Task Completion, sufficient SPD reduction, and >10 pp nDTW improvement. Ablations confirm the indispensability of both the hierarchical map and dual memory modules. Our codes are open-sourced via https://github.com/tsinghua-fib-lab/Mem4Nav.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19697.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jungwoo Park, Taewhoo Lee, Chanwoong Yoon, Hyeon Hwang, Jaewoo Kang</p><p><b>Upvotes:</b> 34</p><p><b>Summary:</b> Extreme activation outliers in Large Language Models (LLMs) critically degrade quantization performance, hindering efficient on-device deployment. While channel-wise operations and adaptive gradient scaling are recognized causes, practical mitigation remains challenging. We introduce Outlier-Safe Pre-Training (OSP), a practical guideline that proactively prevents outlier formation rather than relying on post-hoc mitigation. OSP combines three key innovations: (1) the Muon optimizer, eliminating privileged bases while maintaining training efficiency; (2) Single-Scale RMSNorm, preventing channel-wise amplification; and (3) a learnable embedding projection, redistributing activation magnitudes originating from embedding matrices. We validate OSP by training a 1.4B-parameter model on 1 trillion tokens, which is the first production-scale LLM trained without such outliers. Under aggressive 4-bit quantization, our OSP model achieves a 35.7 average score across 10 benchmarks (compared to 26.5 for an Adam-trained model), with only a 2% training overhead. Remarkably, OSP models exhibit near-zero excess kurtosis (0.04) compared to extreme values (1818.56) in standard models, fundamentally altering LLM quantization behavior. Our work demonstrates that outliers are not inherent to LLMs but are consequences of training strategies, paving the way for more efficient LLM deployment. The source code and pretrained checkpoints are available at https://github.com/dmis-lab/Outlier-Safe-Pre-Training.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19433</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.19697</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 09:00:43 +0000</pubDate>
|
<pubDate>Tue, 24 Jun 2025 15:03:57 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Can Large Language Models Capture Human Annotator Disagreements?</title>
|
<title>Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content</title>
|
||||||
<link>https://arxiv.org/abs/2506.19467</link>
|
<link>https://arxiv.org/abs/2506.20331</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19467.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Jingwei Ni, Yu Fan, Vilém Zouhar, Donya Rooein, Alexander Hoyle, Mrinmaya Sachan, Markus Leippold, Dirk Hovy, Elliott Ash</p><p><b>Upvotes:</b> 14</p><p><b>Summary:</b> Human annotation variation (i.e., annotation disagreements) is common in NLP and often reflects important information such as task subjectivity and sample ambiguity. While Large Language Models (LLMs) are increasingly used for automatic annotation to reduce human effort, their evaluation often focuses on predicting the majority-voted "ground truth" labels. It is still unclear, however, whether these models also capture informative human annotation variation. Our work addresses this gap by extensively evaluating LLMs' ability to predict annotation disagreements without access to repeated human labels. Our results show that LLMs struggle with modeling disagreements, which can be overlooked by majority label-based evaluations. Notably, while RLVR-style (Reinforcement learning with verifiable rewards) reasoning generally boosts LLM performance, it degrades performance in disagreement prediction. Our findings highlight the critical need for evaluating and improving LLM annotators in disagreement modeling. Code and data at https://github.com/EdisonNi-hku/Disagreement_Prediction.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20331.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Rian Touchent, Nathan Godey, Eric de la Clergerie</p><p><b>Upvotes:</b> 3</p><p><b>Summary:</b> We introduce Biomed-Enriched, a biomedical text dataset constructed from PubMed via a two-stage annotation process. In the first stage, a large language model annotates 400K paragraphs from PubMed scientific articles, assigning scores for their type (review, study, clinical case, other), domain (clinical, biomedical, other), and educational quality. The educational quality score (rated 1 to 5) estimates how useful a paragraph is for college-level learning. These annotations are then used to fine-tune a small language model, which propagates the labels across the full PMC-OA corpus. The resulting metadata allows us to extract refined subsets, including 2M clinical case paragraphs with over 450K high-quality ones from articles with commercial-use licenses, and to construct several variants via quality filtering and domain upsampling. Clinical text is typically difficult to access due to privacy constraints, as hospital records cannot be publicly shared. Hence, our dataset provides an alternative large-scale, openly available collection of clinical cases from PubMed, making it a valuable resource for biomedical and clinical NLP. Preliminary continual-pretraining experiments with OLMo2 suggest these curated subsets enable targeted improvements, with clinical upsampling boosting performance by ~5% on MMLU ProfMed and educational quality filtering improving MedQA and MedMCQA by ~1%. Combinations of these techniques led to faster convergence, reaching same performance with a third of training tokens, indicating potential for more efficient and effective biomedical pretraining strategies.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19467</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20331</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 09:49:26 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 11:30:25 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Guidance in the Frequency Domain Enables High-Fidelity Sampling at Low CFG Scales</title>
|
<title>HiWave: Training-Free High-Resolution Image Generation via Wavelet-Based Diffusion Sampling</title>
|
||||||
<link>https://arxiv.org/abs/2506.19713</link>
|
<link>https://arxiv.org/abs/2506.20452</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19713.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Seyedmorteza Sadat, Tobias Vontobel, Farnood Salehi, Romann M. Weber</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> Classifier-free guidance (CFG) has become an essential component of modern conditional diffusion models. Although highly effective in practice, the underlying mechanisms by which CFG enhances quality, detail, and prompt alignment are not fully understood. We present a novel perspective on CFG by analyzing its effects in the frequency domain, showing that low and high frequencies have distinct impacts on generation quality. Specifically, low-frequency guidance governs global structure and condition alignment, while high-frequency guidance mainly enhances visual fidelity. However, applying a uniform scale across all frequencies -- as is done in standard CFG -- leads to oversaturation and reduced diversity at high scales and degraded visual quality at low scales. Based on these insights, we propose frequency-decoupled guidance (FDG), an effective approach that decomposes CFG into low- and high-frequency components and applies separate guidance strengths to each component. FDG improves image quality at low guidance scales and avoids the drawbacks of high CFG scales by design. Through extensive experiments across multiple datasets and models, we demonstrate that FDG consistently enhances sample fidelity while preserving diversity, leading to improved FID and recall compared to CFG, establishing our method as a plug-and-play alternative to standard classifier-free guidance.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20452.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Tobias Vontobel, Seyedmorteza Sadat, Farnood Salehi, Romann M. Weber</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Diffusion models have emerged as the leading approach for image synthesis, demonstrating exceptional photorealism and diversity. However, training diffusion models at high resolutions remains computationally prohibitive, and existing zero-shot generation techniques for synthesizing images beyond training resolutions often produce artifacts, including object duplication and spatial incoherence. In this paper, we introduce HiWave, a training-free, zero-shot approach that substantially enhances visual fidelity and structural coherence in ultra-high-resolution image synthesis using pretrained diffusion models. Our method employs a two-stage pipeline: generating a base image from the pretrained model followed by a patch-wise DDIM inversion step and a novel wavelet-based detail enhancer module. Specifically, we first utilize inversion methods to derive initial noise vectors that preserve global coherence from the base image. Subsequently, during sampling, our wavelet-domain detail enhancer retains low-frequency components from the base image to ensure structural consistency, while selectively guiding high-frequency components to enrich fine details and textures. Extensive evaluations using Stable Diffusion XL demonstrate that HiWave effectively mitigates common visual artifacts seen in prior methods, achieving superior perceptual quality. A user study confirmed HiWave's performance, where it was preferred over the state-of-the-art alternative in more than 80% of comparisons, highlighting its effectiveness for high-quality, ultra-high-resolution image synthesis without requiring retraining or architectural modifications.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19713</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20452</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 15:19:42 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 13:58:37 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>SRFT: A Single-Stage Method with Supervised and Reinforcement Fine-Tuning for Reasoning</title>
|
<title>GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching</title>
|
||||||
<link>https://arxiv.org/abs/2506.19767</link>
|
<link>https://arxiv.org/abs/2506.20480</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19767.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuqian Fu, Tinghong Chen, Jiajun Chai, Xihuai Wang, Songjun Tu, Guojun Yin, Wei Lin, Qichao Zhang, Yuanheng Zhu, Dongbin Zhao</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Large language models (LLMs) have achieved remarkable progress in reasoning tasks, yet the optimal integration of Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) remains a fundamental challenge. Through comprehensive analysis of token distributions, learning dynamics, and integration mechanisms from entropy-based perspectives, we reveal key differences between these paradigms: SFT induces coarse-grained global changes to LLM policy distributions, while RL performs fine-grained selective optimizations, with entropy serving as a critical indicator of training effectiveness. Building on these observations, we propose Supervised Reinforcement Fine-Tuning (SRFT), a single-stage method that unifies both fine-tuning paradigms through entropy-aware weighting mechanisms. Our approach simultaneously applies SFT and RL to directly optimize the LLM using demonstrations and self-exploration rollouts rather than through two-stage sequential methods. Extensive experiments show that SRFT achieves 59.1% average accuracy, outperforming zero-RL methods by 9.0% on five mathematical reasoning benchmarks and 10.9% on three out-of-distribution benchmarks.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20480.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Guinan Su, Li Shen, Lu Yin, Shiwei Liu, Yanwu Yang, Jonas Geiping</p><p><b>Upvotes:</b> 3</p><p><b>Summary:</b> Large language models (LLMs) have shown remarkable capabilities in language understanding and generation. However, such impressive capability typically comes with a substantial model size, which presents significant challenges in deployment and inference. While structured pruning of model parameters offers a promising way to reduce computational costs at deployment time, current methods primarily focus on single model pruning. In this work, we develop a novel strategy to compress models by strategically combining or merging layers from finetuned model variants, which preserves the original model's abilities by aggregating capabilities accentuated in different finetunes. We pose the optimal tailoring of these LLMs as a zero-order optimization problem, adopting a search space that supports three different operations: (1) Layer removal, (2) Layer selection from different candidate models, and (3) Layer merging. Our experiments demonstrate that this approach leads to competitive model pruning, for example, for the Llama2-13B model families, our compressed models maintain approximately 97.3\% of the original performance while removing sim25% of parameters, significantly outperforming previous state-of-the-art methods. The code is available at https://github.com/Guinan-Su/auto-merge-llm.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19767</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20480</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 16:31:37 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 14:24:59 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study</title>
|
<title>ReCode: Updating Code API Knowledge with Reinforcement Learning</title>
|
||||||
<link>https://arxiv.org/abs/2506.19794</link>
|
<link>https://arxiv.org/abs/2506.20495</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19794.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuqi Zhu, Yi Zhong, Jintian Zhang, Ziheng Zhang, Shuofei Qiao, Yujie Luo, Lun Du, Da Zheng, Huajun Chen, Ningyu Zhang</p><p><b>Upvotes:</b> 8</p><p><b>Summary:</b> Large Language Models (LLMs) hold promise in automating data analysis tasks, yet open-source models face significant limitations in these kinds of reasoning-intensive scenarios. In this work, we investigate strategies to enhance the data analysis capabilities of open-source LLMs. By curating a seed dataset of diverse, realistic scenarios, we evaluate models across three dimensions: data understanding, code generation, and strategic planning. Our analysis reveals three key findings: (1) Strategic planning quality serves as the primary determinant of model performance; (2) Interaction design and task complexity significantly influence reasoning capabilities; (3) Data quality demonstrates a greater impact than diversity in achieving optimal performance. We leverage these insights to develop a data synthesis methodology, demonstrating significant improvements in open-source LLMs' analytical reasoning capabilities.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20495.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Haoze Wu, Yunzhi Yao, Wenhao Yu, Huajun Chen, Ningyu Zhang</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Large Language Models (LLMs) exhibit remarkable code generation capabilities but falter when adapting to frequent updates in external library APIs. This critical limitation, stemming from reliance on outdated API knowledge from their training data, even with access to current documentation, impedes reliable code generation in dynamic environments. To tackle this issue, we propose ReCode (rule-based Reinforcement learning for Code Update), a novel framework that mimics human programmer adaptation to API changes. Specifically, we construct a dataset of approximately 2,000 data entries to train the LLMs to perform version migration based on updated information. Then, we introduce a modified string similarity metric for code evaluation as the reward for reinforcement learning. Our experiments demonstrate that ReCode substantially boosts LLMs' code generation performance in dynamic API scenarios, especially on the unseen CodeUpdateArena task. Crucially, compared to supervised fine-tuning, ReCode has less impact on LLMs' general code generation abilities. We apply ReCode on various LLMs and reinforcement learning algorithms (GRPO and DAPO), all achieving consistent improvements. Notably, after training, Qwen2.5-Coder-7B outperforms that of the 32B parameter code instruction-tuned model and the reasoning model with the same architecture. Code is available at https://github.com/zjunlp/ReCode.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19794</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20495</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 17:04:23 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 14:41:13 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality</title>
|
<title>OctoThinker: Mid-training Incentivizes Reinforcement Learning Scaling</title>
|
||||||
<link>https://arxiv.org/abs/2506.19807</link>
|
<link>https://arxiv.org/abs/2506.20512</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19807.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Baochang Ren, Shuofei Qiao, Wenhao Yu, Huajun Chen, Ningyu Zhang</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Large Language Models (LLMs), particularly slow-thinking models, often exhibit severe hallucination, outputting incorrect content due to an inability to accurately recognize knowledge boundaries during reasoning. While Reinforcement Learning (RL) can enhance complex reasoning abilities, its outcome-oriented reward mechanism often lacks factual supervision over the thinking process, further exacerbating the hallucination problem. To address the high hallucination in slow-thinking models, we propose Knowledge-enhanced RL, KnowRL. KnowRL guides models to perform fact-based slow thinking by integrating a factuality reward, based on knowledge verification, into the RL training process, helping them recognize their knowledge boundaries. KnowRL guides models to perform fact-based slow thinking by integrating a factuality reward, based on knowledge verification, into the RL training process, helping them recognize their knowledge boundaries. This targeted factual input during RL training enables the model to learn and internalize fact-based reasoning strategies. By directly rewarding adherence to facts within the reasoning steps, KnowRL fosters a more reliable thinking process. Experimental results on three hallucination evaluation datasets and two reasoning evaluation datasets demonstrate that KnowRL effectively mitigates hallucinations in slow-thinking models while maintaining their original strong reasoning capabilities. Our code is available at https://github.com/zjunlp/KnowRL.</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20512.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zengzhi Wang, Fan Zhou, Xuefeng Li, Pengfei Liu</p><p><b>Upvotes:</b> 21</p><p><b>Summary:</b> Different base language model families, such as Llama and Qwen, exhibit divergent behaviors during post-training with reinforcement learning (RL), especially on reasoning-intensive tasks. What makes a base language model suitable for reinforcement learning? Gaining deeper insight into this question is essential for developing RL-scalable foundation models of the next generation. In this work, we investigate how mid-training strategies shape RL dynamics, focusing on two representative model families: Qwen and Llama. Our study reveals that (1) high-quality mathematical corpora, such as MegaMath-Web-Pro, significantly improve both base model and RL performance, while existing alternatives (e.g., FineMath-4plus) fail to do so; (2) further adding QA-style data, particularly long chain-of-thought (CoT) reasoning examples, enhances RL outcomes, and instruction data further unlocks this effect; (3) while long-CoT improves reasoning depth, it can also induce verbosity of model responses and unstability of RL training, underscoring the importance of data formatting; (4) scaling mid-training consistently leads to stronger downstream RL performance. Building on these insights, we introduce a two-stage mid-training strategy, Stable-then-Decay, in which base models are first trained on 200B tokens with a constant learning rate, followed by 20B tokens across three CoT-focused branches with learning rate decay. This yields OctoThinker, a family of models demonstrating strong RL compatibility and closing the performance gap with more RL-friendly model families, i.e., Qwen. We hope our work will help shape pre-training strategies for foundation models in the RL era. To support further research, we release our open-source models along with a curated math reasoning-intensive corpus of over 70 billion tokens (i.e., MegaMath-Web-Pro-Max).</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19807</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20512</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 17:17:17 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 14:58:13 +0000</pubDate>
|
||||||
</item>
|
</item>
|
||||||
<item>
|
<item>
|
||||||
<title>Scaling Speculative Decoding with Lookahead Reasoning</title>
|
<title>When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs</title>
|
||||||
<link>https://arxiv.org/abs/2506.19830</link>
|
<link>https://arxiv.org/abs/2506.20544</link>
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19830.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yichao Fu, Rui Ge, Zelei Shao, Zhijie Deng, Hao Zhang</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Reasoning models excel by generating long chain-of-thoughts, but decoding the resulting thousands of tokens is slow. Token-level speculative decoding (SD) helps, but its benefit is capped, because the chance that an entire gamma-token guess is correct falls exponentially as gamma grows. This means allocating more compute for longer token drafts faces an algorithmic ceiling -- making the speedup modest and hardware-agnostic. We raise this ceiling with Lookahead Reasoning, which exploits a second, step-level layer of parallelism. Our key insight is that reasoning models generate step-by-step, and each step needs only to be semantically correct, not exact token matching. In Lookahead Reasoning, a lightweight draft model proposes several future steps; the target model expands each proposal in one batched pass, and a verifier keeps semantically correct steps while letting the target regenerate any that fail. Token-level SD still operates within each reasoning step, so the two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x while preserving answer quality, and its speedup scales better with additional GPU throughput. Our code is available at https://github.com/hao-ai-lab/LookaheadReasoning</p></description>
|
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.20544.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Ammar Khairi, Daniel D'souza, Ye Shen, Julia Kreutzer, Sara Hooker</p><p><b>Upvotes:</b> 6</p><p><b>Summary:</b> Recent advancements in large language models (LLMs) have shifted focus toward scaling inference-time compute, improving performance without retraining the model. A common approach is to sample multiple outputs in parallel, and select one of these as the final output. However, work to date has focused on English and a handful of domains such as math and code. In contrast, we are most interested in techniques that generalize across open-ended tasks, formally verifiable tasks, and across languages. In this work, we study how to robustly scale inference-time compute for open-ended generative tasks in a multilingual, multi-task setting. Our findings show that both sampling strategy based on temperature variation and selection strategy must be adapted to account for diverse domains and varied language settings. We evaluate existing selection methods, revealing that strategies effective in English often fail to generalize across languages. We propose novel sampling and selection strategies specifically adapted for multilingual and multi-task inference scenarios, and show they yield notable gains across languages and tasks. In particular, our combined sampling and selection methods lead to an average +6.8 jump in win-rates for our 8B models on m-ArenaHard-v2.0 prompts, against proprietary models such as Gemini. At larger scale, Command-A (111B model) equipped with our methods, shows +9.0 improvement in win-rates on the same benchmark with just five samples against single-sample decoding, a substantial increase at minimal cost. Our results underscore the need for language- and task-aware approaches to inference-time compute, aiming to democratize performance improvements in underrepresented languages.</p></description>
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19830</guid>
|
<guid isPermaLink="false">https://arxiv.org/abs/2506.20544</guid>
|
||||||
<pubDate>Tue, 24 Jun 2025 17:48:10 +0000</pubDate>
|
<pubDate>Wed, 25 Jun 2025 15:37:53 +0000</pubDate>
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>SimpleGVR: A Simple Baseline for Latent-Cascaded Video Super-Resolution</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19838</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19838.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Liangbin Xie, Yu Li, Shian Du, Menghan Xia, Xintao Wang, Fanghua Yu, Ziyan Chen, Pengfei Wan, Jiantao Zhou, Chao Dong</p><p><b>Upvotes:</b> 9</p><p><b>Summary:</b> Latent diffusion models have emerged as a leading paradigm for efficient video generation. However, as user expectations shift toward higher-resolution outputs, relying solely on latent computation becomes inadequate. A promising approach involves decoupling the process into two stages: semantic content generation and detail synthesis. The former employs a computationally intensive base model at lower resolutions, while the latter leverages a lightweight cascaded video super-resolution (VSR) model to achieve high-resolution output. In this work, we focus on studying key design principles for latter cascaded VSR models, which are underexplored currently. First, we propose two degradation strategies to generate training pairs that better mimic the output characteristics of the base model, ensuring alignment between the VSR model and its upstream generator. Second, we provide critical insights into VSR model behavior through systematic analysis of (1) timestep sampling strategies, (2) noise augmentation effects on low-resolution (LR) inputs. These findings directly inform our architectural and training innovations. Finally, we introduce interleaving temporal unit and sparse local attention to achieve efficient training and inference, drastically reducing computational overhead. Extensive experiments demonstrate the superiority of our framework over existing methods, with ablation studies confirming the efficacy of each design choice. Our work establishes a simple yet effective baseline for cascaded video super-resolution generation, offering practical insights to guide future advancements in efficient cascaded synthesis systems.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19838</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:57:26 +0000</pubDate>
|
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>Improving Progressive Generation with Decomposable Flow Matching</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19839</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19839.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Moayed Haji-Ali, Willi Menapace, Ivan Skorokhodov, Arpit Sahni, Sergey Tulyakov, Vicente Ordonez, Aliaksandr Siarohin</p><p><b>Upvotes:</b> 5</p><p><b>Summary:</b> Generating high-dimensional visual modalities is a computationally intensive task. A common solution is progressive generation, where the outputs are synthesized in a coarse-to-fine spectral autoregressive manner. While diffusion models benefit from the coarse-to-fine nature of denoising, explicit multi-stage architectures are rarely adopted. These architectures have increased the complexity of the overall approach, introducing the need for a custom diffusion formulation, decomposition-dependent stage transitions, add-hoc samplers, or a model cascade. Our contribution, Decomposable Flow Matching (DFM), is a simple and effective framework for the progressive generation of visual media. DFM applies Flow Matching independently at each level of a user-defined multi-scale representation (such as Laplacian pyramid). As shown by our experiments, our approach improves visual quality for both images and videos, featuring superior results compared to prior multistage frameworks. On Imagenet-1k 512px, DFM achieves 35.2% improvements in FDD scores over the base architecture and 26.4% over the best-performing baseline, under the same training compute. When applied to finetuning of large models, such as FLUX, DFM shows faster convergence speed to the training distribution. Crucially, all these advantages are achieved with a single model, architectural simplicity, and minimal modifications to existing training pipelines.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19839</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:58:02 +0000</pubDate>
|
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>Orthogonal Finetuning Made Scalable</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19847</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19847.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zeju Qiu, Weiyang Liu, Adrian Weller, Bernhard Schölkopf</p><p><b>Upvotes:</b> 2</p><p><b>Summary:</b> Orthogonal finetuning (OFT) offers highly parameter-efficient adaptation while preventing catastrophic forgetting, but its high runtime and memory demands limit practical deployment. We identify the core computational bottleneck in OFT as its weight-centric implementation, which relies on costly matrix-matrix multiplications with cubic complexity. To overcome this, we propose OFTv2, an input-centric reformulation that instead uses matrix-vector multiplications (i.e., matrix-free computation), reducing the computational cost to quadratic. We further introduce the Cayley-Neumann parameterization, an efficient orthogonal parameterization that approximates the matrix inversion in Cayley transform via a truncated Neumann series. These modifications allow OFTv2 to achieve up to 10x faster training and 3x lower GPU memory usage without compromising performance. In addition, we extend OFTv2 to support finetuning quantized foundation models and show that it outperforms the popular QLoRA in training stability, efficiency, and memory usage.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19847</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:59:49 +0000</pubDate>
|
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>ScaleCap: Inference-Time Scalable Image Captioning via Dual-Modality Debiasing</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19848</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19848.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Long Xing, Qidong Huang, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Jinsong Li, Shuangrui Ding, Weiming Zhang, Nenghai Yu, Jiaqi Wang, Feng Wu, Dahua Lin</p><p><b>Upvotes:</b> 23</p><p><b>Summary:</b> This paper presents ScaleCap, an inference-time scalable image captioning strategy that generates comprehensive and detailed image captions. The key challenges of high-quality image captioning lie in the inherent biases of LVLMs: multimodal bias resulting in imbalanced descriptive granularity, offering detailed accounts of some elements while merely skimming over others; linguistic bias leading to hallucinated descriptions of non-existent objects. To address these issues, we propose a scalable debiased captioning strategy, which continuously enriches and calibrates the caption with increased inference budget. Specifically, we propose two novel components: heuristic question answering and contrastive sentence rating. The former generates content-specific questions based on the image and answers them to progressively inject relevant information into the caption. The latter employs sentence-level offline contrastive decoding to effectively identify and eliminate hallucinations caused by linguistic biases. With increased inference cost, more heuristic questions are raised by ScaleCap to progressively capture additional visual details, generating captions that are more accurate, balanced, and informative. Extensive modality alignment experiments demonstrate the effectiveness of ScaleCap. Annotating 450K images with ScaleCap and using them for LVLM pretraining leads to consistent performance gains across 11 widely used benchmarks. Furthermore, ScaleCap showcases superb richness and fidelity of generated captions with two additional tasks: replacing images with captions in VQA task, and reconstructing images from captions to assess semantic coverage. Code is available at https://github.com/Cooperx521/ScaleCap.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19848</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:59:55 +0000</pubDate>
|
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>Unified Vision-Language-Action Model</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19850</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19850.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Yuqi Wang, Xinghang Li, Wenxuan Wang, Junbo Zhang, Yingyan Li, Yuntao Chen, Xinlong Wang, Zhaoxiang Zhang</p><p><b>Upvotes:</b> 11</p><p><b>Summary:</b> Vision-language-action models (VLAs) have garnered significant attention for their potential in advancing robotic manipulation. However, previous approaches predominantly rely on the general comprehension capabilities of vision-language models (VLMs) to generate action signals, often overlooking the rich temporal and causal structure embedded in visual observations. In this paper, we present UniVLA, a unified and native multimodal VLA model that autoregressively models vision, language, and action signals as discrete token sequences. This formulation enables flexible multimodal tasks learning, particularly from large-scale video data. By incorporating world modeling during post-training, UniVLA captures causal dynamics from videos, facilitating effective transfer to downstream policy learning--especially for long-horizon tasks. Our approach sets new state-of-the-art results across several widely used simulation benchmarks, including CALVIN, LIBERO, and Simplenv-Bridge, significantly surpassing previous methods. For example, UniVLA achieves 95.5% average success rate on LIBERO benchmark, surpassing pi0-FAST's 85.5%. We further demonstrate its broad applicability on real-world ALOHA manipulation and autonomous driving.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19850</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:59:57 +0000</pubDate>
|
|
||||||
</item>
|
|
||||||
<item>
|
|
||||||
<title>AnimaX: Animating the Inanimate in 3D with Joint Video-Pose Diffusion Models</title>
|
|
||||||
<link>https://arxiv.org/abs/2506.19851</link>
|
|
||||||
<description><p><img src="https://cdn-thumbnails.huggingface.co/social-thumbnails/papers/2506.19851.png" alt="Paper thumbnail" style="max-width: 300px; height: auto;" /></p><p><b>Authors:</b> Zehuan Huang, Haoran Feng, Yangtian Sun, Yuanchen Guo, Yanpei Cao, Lu Sheng</p><p><b>Upvotes:</b> 41</p><p><b>Summary:</b> We present AnimaX, a feed-forward 3D animation framework that bridges the motion priors of video diffusion models with the controllable structure of skeleton-based animation. Traditional motion synthesis methods are either restricted to fixed skeletal topologies or require costly optimization in high-dimensional deformation spaces. In contrast, AnimaX effectively transfers video-based motion knowledge to the 3D domain, supporting diverse articulated meshes with arbitrary skeletons. Our method represents 3D motion as multi-view, multi-frame 2D pose maps, and enables joint video-pose diffusion conditioned on template renderings and a textual motion prompt. We introduce shared positional encodings and modality-aware embeddings to ensure spatial-temporal alignment between video and pose sequences, effectively transferring video priors to motion generation task. The resulting multi-view pose sequences are triangulated into 3D joint positions and converted into mesh animation via inverse kinematics. Trained on a newly curated dataset of 160,000 rigged sequences, AnimaX achieves state-of-the-art results on VBench in generalization, motion fidelity, and efficiency, offering a scalable solution for category-agnostic 3D animation. Project page: https://anima-x.github.io/{https://anima-x.github.io/}.</p></description>
|
|
||||||
<guid isPermaLink="false">https://arxiv.org/abs/2506.19851</guid>
|
|
||||||
<pubDate>Tue, 24 Jun 2025 17:59:58 +0000</pubDate>
|
|
||||||
</item>
|
</item>
|
||||||
</channel>
|
</channel>
|
||||||
</rss>
|
</rss>
|
||||||
|
|||||||
Reference in New Issue
Block a user