Cappy: 小さなスコアラーで大規模マルチタスク言語モデルを凌駕・強化

<span class="byline-author">Posted by Yun Zhu and Lijuan Liu, Software Engineers, Google Research</span>

<img src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFNlqVAnwoYdZ97LvC4-ipR6FeOc4o9udsTUtNBBWl5Y4XHclcrz3kTCibizteSBc_xsVLh-pyRiCCNfIzTDHEs7VsJcUMCk0EjUxzvKITKCncdx1y7u9JXGkXM6TyoZY5RhUt2l_up-Us0yIV-0-EUvHsjOlFNSSNgNHlpwK1PAliqcj4gSoLsYXhIi18/s320/Cappy%20hero.jpg" style="display: none;" />

<p>

Large language model (LLM) advancements have led to a new paradigm that unifies various natural language processing (NLP) tasks within an instruction-following framework. This paradigm is exemplified by recent multi-task LLMs, such as <a href="https://arxiv.org/abs/2110.08207">T0</a>, <a href="https://arxiv.org/abs/2210.11416">FLAN</a>, and <a href="https://arxiv.org/abs/2212.12017">OPT-IML</a>. First, multi-task data is gathered with each task following a task-specific template, where each labeled example is converted into an instruction (e.g., <em>"</em>Put the concepts together to form a sentence: ski, mountain, skier<em>”</em>) paired with a corresponding response (e.g., <em>"</em>Skier skis down the mountain<em>"</em>). These instruction-response pairs are used to train the LLM, resulting in a conditional generation model that takes an instruction as input and generates a response. Moreover, multi-task LLMs have exhibited remarkable task-wise generalization capabilities as they can address unseen tasks by understanding and solving brand-new instructions.

</p>

<a name='more'></a>

<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto;"><tbody><tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMcacnhPA68XiEskvhExF4SGFh4997UZzwvhYfXt-ReGXtzfGTamLB3LZoYSh8WWuf1dmlBnNAUecAMhrBTOMVF6vxsw3BqY8Ld5xPgSdZY_cywScxxxQ5e6uwhawA5VYDEj6VtSyOTNGZtjdLXieeFV5OLiDk3bnB-xaz4MIbvUO-7RPadk8iQDv3206V/s640/Cappy%20instruction-following.gif" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="177" data-original-width="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhMcacnhPA68XiEskvhExF4SGFh4997UZzwvhYfXt-ReGXtzfGTamLB3LZoYSh8WWuf1dmlBnNAUecAMhrBTOMVF6vxsw3BqY8Ld5xPgSdZY_cywScxxxQ5e6uwhawA5VYDEj6VtSyOTNGZtjdLXieeFV5OLiDk3bnB-xaz4MIbvUO-7RPadk8iQDv3206V/s16000/Cappy%20instruction-following.gif" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">The demonstration of the instruction-following pre-training of multi-task LLMs, e.g., FLAN. Pre-training tasks under this paradigm improves the performance for unseen tasks.</td></tr></tbody></table>

<p>

Due to the complexity of understanding and solving various tasks solely using instructions, the size of multi-task LLMs typically spans from several billion parameters to hundreds of billions (e.g., <a href="https://arxiv.org/abs/2210.11416">FLAN-11B</a>, <a href="https://arxiv.org/abs/2110.08207">T0-11B</a> and <a href="https://arxiv.org/abs/2212.12017">OPT-IML-175B</a>). As a result, operating such sizable models poses significant challenges because they demand considerable computational power and impose substantial requirements on the memory capacities of GPUs and TPUs, making their training and inference expensive and inefficient. Extensive storage is required to maintain a unique LLM copy for each downstream task. Moreover, the most powerful multi-task LLMs (e.g., FLAN-PaLM-540B) are closed-sourced, making them impossible to be adapted. However, in practical applications, harnessing a single multi-task LLM to manage all conceivable tasks in a zero-shot manner remains difficult, particularly when dealing with complex tasks, personalized tasks and those that cannot be succinctly defined using instructions. On the other hand, the size of downstream training data is usually insufficient to train a model well without incorporating rich prior knowledge. Hence, it is long desired to adapt LLMs with downstream supervision while bypassing storage, memory, and access issues.

</p>

<p>

Certain <em>parameter-efficient tuning</em> strategies, including <a href="https://aclanthology.org/2021.acl-long.353.pdf">prompt tuning</a> and <a href="https://openreview.net/pdf?id=nZeVKeeFYf9">adapters</a>, substantially diminish storage requirements, but they still perform back-propagation through LLM parameters during the tuning process, thereby keeping their memory demands high. Additionally, some <em><a href="https://arxiv.org/pdf/2301.00234.pdf">in-context learning</a></em> techniques circumvent parameter tuning by integrating a limited number of supervised examples