Re-Reading提升大型语言模型推理能力_数学推理语言模型资源-CSDN文库

自然语言处理

需积分: 1 121 浏览量 2024-09-20 21:34:04 上传评论收藏 2.42MB PDF 举报

资源推荐

资源详情

资源评论

Re-Reading Improves Reasoning in Large Language Models

Xiaohan Xu

, Chongyang Tao

, Tao Shen

, Can Xu

Hongbo Xu

, Guodong Long

, Jian-guang Lou

Institute of Information Engineering, CAS, {xuxiaohan,hbxu}@iie.ac.cn

Microsoft Corporation, {chotao,caxu,jlou}@microsoft.com

AAII, School of CS, FEIT, UTS, {tao.shen,guodong.long}@uts.edu.au

Abstract

To enhance the reasoning capabilities of off-

the-shelf Large Language Models (LLMs),

we introduce a simple, yet general and effec-

tive prompting method, RE2, i.e., Re-Reading

the question as input. Unlike most thought-

eliciting prompting methods, such as Chain-of-

Thought (CoT), which aim to elicit the reason-

ing process in the output, RE2 shifts the fo-

cus to the input by processing questions twice,

thereby enhancing the understanding process.

Consequently, RE2 demonstrates strong gen-

erality and compatibility with most thought-

eliciting prompting methods, including CoT.

Crucially, RE2 facilitates a "bidirectional" en-

coding in unidirectional decoder-only LLMs

because the ﬁrst pass could provide global in-

formation for the second pass. We begin with

a preliminary empirical study as the founda-

tion of RE2, illustrating its potential to enable

"bidirectional" attention mechanisms. We then

evaluate RE2 on extensive reasoning bench-

marks across 14 datasets, spanning 112 exper-

iments, to validate its effectiveness and gener-

ality. Our ﬁndings indicate that, with the ex-

ception of a few scenarios on vanilla ChatGPT,

RE2 consistently enhances the reasoning per-

formance of LLMs through a simple re-reading

strategy. Further analyses reveal RE2’s adapt-

ability, showing how it can be effectively inte-

grated with different LLMs, thought-eliciting

prompting, and ensemble strategies.

1 Introduction

In the ever-evolving landscape of artiﬁcial in-

telligence, large language models (LLMs) have

emerged as a cornerstone of natural language un-

derstanding and generation (Brown et al., 2020;

Touvron et al., 2023a,a; OpenAI, 2023). As these

LLMs have grown in capability, a pivotal challenge

has come to the forefront: imbuing them with the

This work was done during internship at Microsoft.

Our code is available at

https://github.com/Tebmer/

Rereading-LLM-Reasoning/

Input

CoT

Input

CoT+RE2

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis

balls. Each can has 3 tennis balls. How many tennis balls

does he have now?

A: Let’s think step by step.

Q: Roger has 5 tennis balls. He buys 2 more cans of tennis

balls. Each can has 3 tennis balls. How many tennis balls

does he have now?

Read the question again: Roger has 5 tennis balls. He buys 2

more cans of tennis balls. Each can has 3 tennis balls. How

many tennis balls does he have now?

A: Let’s think step by step.

Figure 1: Example inputs of CoT prompting versus

CoT prompting with RE2. RE2 is a simple prompting

method that repeats the question as input. Typically,

tokens in the question, such as "tennis balls", cannot

see subsequent tokens in the original setup for LLMs

(the top ﬁgure). In contrast, LLMs with RE2 allows

"tennis balls" in the second pass to see the entire ques-

tion containing "How many ...", achieving an effect of a

"bidirectional" understanding (the bottom ﬁgure).

ability to reason effectively. The capacity to engage

in sound reasoning is a hallmark of human intelli-

gence, enabling us to infer, deduce, and solve prob-

lems. In LLMs, this skill is paramount for enhanc-

ing their practical utility. Despite their remarkable

capabilities, LLMs often struggle with nuanced rea-

soning (Blair-Stanek et al., 2023; Arkoudas, 2023),

prompting researchers to explore innovative strate-

gies to bolster their reasoning prowess (Wei et al.,

2022b; Gao et al., 2023; Besta et al., 2023).

Existing research on reasoning has predomi-

nantly concentrated on designing diverse thought-

eliciting prompting strategies to elicit reasoning

processes in the output phase, such as Chain-

of-Thought (CoT) (Wei et al., 2022b), Program-

Aided Language Model (PAL) (Gao et al., 2023),

etc. (Yao et al., 2023a; Besta et al., 2023; Wang

et al., 2023a). In contrast, scant attention has been

arXiv:2309.06275v2 [cs.CL] 29 Feb 2024

Second

Pass

First Pass Second Pass

Query

Key

Second Pass → (First Pass + Second Pass)

Figure 2: Illustration of the attention distribution of

each token in the second pass attending to the entire

input in LLaMA-2 using the GSM8K dataset. A darker

cell indicates higher attention. The region within the

dashed upper triangle demonstrates that every token in

the second pass has obvious attention to its subsequent

tokens in the ﬁrst pass. This suggests that re-reading

in LLMs is promising for achieving a "bidirectional"

understanding of the question.

paid to the understanding of the input phase. In fact,

comprehension is the ﬁrst step before solving the

problem, which is crucially important. However,

in the era of generative AI, most LLMs adopt the

decoder-only LLMs with unidirectional attention,

like GPT-3 (Brown et al., 2020) and LLaMA (Tou-

vron et al., 2023b). Compared with encoder-based

language models that feature bidirectional atten-

tion (Devlin et al., 2019), the unidirectional atten-

tion limits every token’s visibility to only previ-

ous tokens when encoding a question, potentially

impairing the global understanding of the ques-

tion (Du et al., 2022) (the top ﬁgure in Figure 1).

Fortunately, many cognitive science studies have re-

vealed that humans tend to re-read questions during

learning and problem-solving to enhance the com-

prehension process (Dowhower, 1987, 1989; Ozek

and Civelek, 2006). Motivated by this, we also

conduct a preliminary empirical study for LLaMA-

2 (Touvron et al., 2023b) by repeating the ques-

tion two times using the GSM8K dataset (Cobbe

et al., 2021). Figure 2 shows that LLaMA-2 with

re-reading is promising to achieve a "bidirectional"

understanding of the question, and further improve

the reasoning performance.

Based on the observation and inspired by the hu-

man strategy of re-reading, we present a simple yet

effective and general reasoning prompting strategy,

RE2, i.e., Re-Reading the question as input (see

the illustration in Figure 1). Similar to the human

problem-solving process, where the primary task is

to comprehend the problem, we place our emphasis

on designing the prompting strategy on the input

phase. Thus, RE2 is orthogonal to and compatible

with most thought-eliciting prompting methods in

the output phase, such as CoT, PAL, etc. Moreover,

instead of processing the input in only a single

pass, the repetition of questions enables LLMs to

allocate more computational resources to input en-

coding, akin to "horizontally" increasing the depth

of neural networks. Therefore, LLMs with RE2

is promising to have a deeper understanding of

the question and improve reasoning performance.

More interestingly, LLMs with RE2 show potential

for a "bidirectional" understanding of questions in

the context of unidirectional LLMs. This is be-

cause every token in the second pass could also

attend to its subsequent tokens in the ﬁrst pass (see

illustration in Figure 1 and Figure 2).

To validate the efﬁcacy and generality of RE2,

we conducted extensive experiments spanning arith-

metic, commonsense, and symbolic reasoning tasks

across 14 datasets and 112 experiments. The

results show that, with the exception of certain

scenarios on vanilla ChatGPT, our RE2 with a

simple re-reading strategy consistently enhances

the reasoning performance of LLMs. RE2 ex-

hibits versatility across various LLMs, such as

Text-Davinci-003, ChatGPT, LLaMA-2-13B, and

LLaMA-2-70B, spanning both instruction ﬁne-

tuning (IFT) and non-IFT models. We also ex-

plore RE2 in task settings of zero-shot and few-

shot, thought-eliciting prompting methods, and the

self-consistency setting, highlighting its generality.

2 Methodology

2.1 Vanilla Chain-of-Thought for Reasoning

We begin with a uniﬁed formulation to leverage

LLMs with CoT prompting to solve reasoning tasks.

In formal, given an input

and a target

, a LLM

p with CoT prompting can be formulated as

y ∼

z∼ p(z|C

)

p(y|C

, z) · p(z|C

where C

= c

(cot)

(x). (1)

In this formulation,

denotes the prompted

input.

(cot)

(·)

represents the template with CoT

prompting instructions, such as ‘let’s think step by

step’.

stands for a latent variable of rationale, and

denotes a sampled rationale in natural language.

Consequently, the LLMs can break down complex

tasks into more manageable reasoning steps, treat-

ing each step as a component of the overall solution

chain. We employ CoT as a baseline to solve rea-

soning tasks without compromising its generality.

In addition to CoT, our proposed simple RE2 can

serve as a "plug & play" module adaptable to most

other prompting methods (§2.3).

2.2 Re-Reading (RE2) Improves Reasoning

Drawing inspiration from the human strategy of

re-reading, we introduce this strategy for LLM rea-

soning, dubbed RE2, to enhance understanding in

the input phase. With RE2, the prompting process

in Eq. 1 can be readily rephrased as:

y ∼

z∼ p(z|C

)

p(y|C

, z) · p(z|C

where C

= c

(cot)

(re2(x)). (2)

In this formulation,

re2(·)

is the re-reading opera-

tion of the input. We don’t seek complex adjust-

ments for LLMs but aim for a general implementa-

tion of re2(x) that is as simple as follows:

RE2 Prompting

Q: {Input Query}

Read the question again: {Input Query}

# Thought-eliciting prompt (e.g.,“Let’s

think step by step") #

where ‘{Input Query}‘ is a placeholder for the in-

put query,

. The left part of this prompting could

incorporate other thought-eliciting prompts. In-

tuitively, RE2 offers two advantages to enhance

the understanding process: (1) it allocates more

computational resources to the input, and (2) it

facilitates a "bidirectional" understanding of the

question, where the ﬁrst pass provides global infor-

mation for the second pass.

2.3 Generality of RE2

Due to RE2’s simplicity and emphasis on the input

phase, it can be seamlessly integrated with a wide

range of LLMs and algorithms, including few-shot

settings, self-consistency, various thought-eliciting

prompting strategies, and more. We offer insights

into the integration of RE2 with other thought-

eliciting prompting strategies as an illustration.

Compared with those thought-eliciting prompt-

ing strategies that focus on the output phase, RE2

shifts the emphasis towards understanding the in-

put. Therefore, RE2 exhibits signiﬁcant compati-

bility with them, acting as a “plug & play” module.

This synergy has the potential to further enhance

the reasoning abilities of LLMs. With a speciﬁc

thought-eliciting prompting,

, designed to elicit

thoughts from the LLMs, Eq. (3) is rewritten as:

y ∼

z∼ p(z|C

)

p(y|C

, z) · p(z|C

where C

= c

(τ )

(re2(x)). (3)

Here,

denotes various thought-eliciting prompt-

ings beyond CoT, such as Plan-and-Solve (Wang

et al., 2023a), and Program-Aided Prompt (Gao

et al., 2023), etc. We also conducted lots of experi-

ments to validate the generality of RE2 in §3.4.

3 Experiments

3.1 Benchmarks

We assess RE2 prompting across three key cat-

egories of reasoning benchmarks. Details of all

datasets are shown in Appendix A

Arithmetic Reasoning We consider the fol-

lowing seven arithmetic reasoning benchmarks:

the GSM8K benchmark of math word prob-

lems (Cobbe et al., 2021), the SVAMP dataset of

math word problems with varying structures (Patel

et al., 2021), the ASDiv dataset of diverse math

word problems (Miao et al., 2020), the AQuA

dataset of algebraic word problems (Ling et al.,

2017), the AddSub (Hosseini et al., 2014) of math

word problems on addition and subtraction for third

to ﬁfth grader, MultiArith (Roy and Roth, 2015)

dataset of math problems with multiple steps, and

the SingelEQ (Roy et al., 2015) dataset of elemen-

tary math word problems with single operation.

Commonsense and Symbolic Reasoning For

commonsense reasoning, we use CSQA (Talmor

et al., 2019), StrategyQA (Geva et al., 2021), and

the ARC (Clark et al., 2018). CSQA dataset con-

sists of questions that necessitate various common-

sense knowledge. The StrategyQA dataset com-

prises questions that demand multi-step reasoning.

The ARC dataset (denoted as ARC-t) is divided

into two sets: a Challenge Set (denoted as ARC-

c), containing questions that both retrieval-based

and word co-occurrence algorithms answered in-

correctly, and an Easy Set (denoted as ARC-e). We

evaluate two symbolic reasoning tasks: date under-

standing (Suzgun et al., 2023a) and Coinﬂip (Wei

et al., 2022b). Date understanding is a subset of

BigBench datasets (Suzgun et al., 2023a), which

have posed challenges for previous ﬁne-tuning ef-

forts. Coinﬂip is a dataset of questions on whether

LLMs Methods GSM SVAMP ASDIV AQUA MultiArith SingleEQ AddSub

davinci-003

Vanilla 19.48 67.60 69.00 28.74 31.33 86.22 89.87

Vanilla+RE2 24.79 ↑ 5.31 70.90 ↑ 3.30 71.20 ↑ 2.20 30.31 ↑ 1.57 42.33 ↑ 11.00 87.20 ↑ 0.98 92.15 ↑ 2.28

CoT 58.98 78.30 77.60 40.55 89.33 92.32 91.39

CoT+RE2 61.64 ↑ 2.68 81.00 ↑ 2.70 78.60 ↑ 1.00 44.49 ↑ 3.94 93.33 ↑ 4.00 93.31 ↑ 0.99 91.65 ↑ 0.26

ChatGPT

Vanilla 77.79 81.50

∗

87.00

∗

63.39

∗

97.83

∗

95.28

∗

92.41

∗

Vanilla+RE2 79.45 ↑ 1.66 84.20 ↑ 2.70 88.40 ↑ 1.40 58.27 ↓ 5.12 96.67 ↓ 1.16 94.49 ↓ 0.79 91.65 ↓ 0.76

CoT 78.77 78.70 85.60 55.91 95.50 93.70 88.61

CoT+RE2 80.59 ↑ 1.82 80.00 ↑ 1.30 86.00 ↑ 0.40 59.06 ↑ 3.15 96.50 ↑ 1.00 95.28 ↑ 1.58 89.87 ↑ 1.26

Table 1: Results on arithmetic reasoning benchmarks.

∗

denotes that Vanilla is even superior to CoT prompting.

LLMs Methods

Commonsense Symbolic

CSQA StrategyQA ARC-e ARC-c ARC-t Date Coin

davinci-003

Vanilla 74.20

∗

59.74 84.81 72.01 80.58 40.92 49.80

Vanilla+RE2 76.99 ↑ 2.79 59.91 ↑ 0.17 88.22 ↑ 3.41 75.68 ↑ 3.67 84.07 ↑ 3.49 42.01 ↑ 1.09 52.40 ↑ 2.60

CoT 71.66 67.55 85.69 73.21 81.57 46.07 95.60

CoT+RE2 73.05 ↑ 1.39 66.24 ↓ 1.31 87.84 ↑ 2.15 76.02 ↑ 2.81 83.94 ↑ 2.37 52.57 ↑ 6.50 99.60 ↑ 4.00

ChatGPT

Vanilla 76.66

∗

62.36 94.32

∗

85.41

∗

91.37

∗

47.43

∗

52.00

Vanilla+RE2 78.38 ↑ 1.72 66.99 ↑ 4.63 93.81 ↓ 0.51 83.19 ↓ 2.22 90.30 ↓ 1.07 47.97 ↑ 0.54 57.20 ↑ 5.20

CoT 69.94 67.82 93.35 83.53 90.11 43.63 88.80

CoT+RE2 71.66 ↑ 1.72 69.34 ↑ 1.52 93.14 ↓ 0.21 84.47 ↑ 0.94 90.27 ↑ 0.16 47.15 ↑ 3.52 95.20 ↑ 6.40

Table 2: Results on commonsense and symbolic reasoning benchmarks.

∗

denotes that Vanilla is even superior to

CoT prompting.

a coin is still heads up after it is ﬂipped or not based

on steps given in the questions.

3.2 Language Models and Implementations

Baseline Prompting. In our implementation, we

rigorously evaluate the performance of our RE2

model on two baseline prompting methods: Vanilla

and CoT. The Vanilla approach aligns with the

standard prompting method outlined in (Wei et al.,

2022b; Kojima et al., 2022), wherein no speciﬁc

prompts are employed to elicit thoughts from

LLMs. Conversely, the CoT method guides the

model through a step-by-step thought process.

RE2 Prompting. We incorporate our RE2 strat-

egy into these baseline methods to assess its im-

pact, denoted as Vanilla+RE2 and CoT+RE2. To

avoid the impact of randomness introduced by the

demonstrations in a few-shot setting, we mainly

assess our method in a zero-shot setting, following

(Chen et al., 2023; Wang et al., 2023a; Du et al.,

2023). Additionally, for different tasks, we design

answer-format instructions in prompts to regulate

the structure of the ﬁnal answer, facilitating precise

answer extraction. Detailed information regard-

ing the baseline prompting, RE2 prompting, and

answer-format instructions can be found in the pa-

per’s Appendix B.

Implementations. Our decoding strategy uses

greedy decoding with a temperature setting of

0, thus leading to deterministic outputs. For

these experiments, we employ two powerful back-

bones: ChatGPT (gpt-3.5-turbo-0613) (OpenAI,

2022) and davinci-003 (text-davinci-003)

, across

all prompting methods, including Vanilla, CoT,

Vanilla+RE2, and CoT+RE2.

3.3 Evaluation Results

Table 1 presents the results on arithmetic reasoning

datasets, and Table 2 on commonsense reasoning

and symbolic reasoning. In almost all scenarios,

LLMs with RE2 achieve consistent improvements

across both LLMs (davinci-003 and ChatGPT) and

prompting methods (Vanilla and CoT). Speciﬁcally,

davinci-003 with Vanilla+RE2 shows average im-

provements of 3.81, 2.51, and 1.85 in arithmetic,

commonsense, and symbolic tasks, respectively.

With CoT, davinci-003 generates intermediate rea-

soning steps, signiﬁcantly enhancing the reasoning

performance of LLMs. By applying RE2, davinci-

003 with CoT+RE2 demonstrates further improve-

ment, with average gains of 2.22, 1.23, and 5.25

in the same categories, respectively. These results

indicate that RE2 can beneﬁt LLMs in directly gen-

erating answers and improve the performance of

CoT leading to correct answers.

When applied to ChatGPT, RE2 exhibits consis-

https://platform.openai.com/docs/models/gpt-3-5

1 2 3 4 5

Times of reading

Accuracy

77.79

78.77

79.45

80.59

79.15

80.89

78.77

80.29

77.56

80.29

1 2 3 4 5

Times of reading

19.48

58.98

24.79

61.64

18.8

60.12

18.35

58.83

17.06

57.47

ChatGPT Vanilla ChatGPT CoT davinci-003 Vanilla davinci-003 CoT

Figure 3: Evaluation results of the times of reading on

GSM benchmark.

tent improvement on most datasets, except for a

slight drop in performance on a few datasets, e.g.,

AQUA and MultiArith, when using Vanilla+RE2.

This exception could be due to ChatGPT’s expo-

sure to these datasets with CoT outputs during in-

struction ﬁne-tuning (IFT) (Chen et al., 2023). On

such datasets, ChatGPT with Vanilla still produces

CoT-like output (see examples in Appendix E) and

even outperforms ChatGPT with CoT (as indicated

by the

∗

results in Tables 1 and 2). Chen et al.

(2023) obtained similar experimental results and

suggested that this occurs because ChatGPT may

have been exposed to these task datasets contain-

ing CoT explanations without explicit prompting.

Therefore, additional explicit instructions, like CoT

or RE2, might disrupt this learned pattern in Chat-

GPT, possibly leading to decreased performance.

Nonetheless, on some datasets like SVAMP, AS-

DIV, CSQA, and Date, RE2 still manages to im-

prove the baseline Vanilla prompting. Moreover, in

datasets where CoT prompting normally surpasses

Vanilla prompting, such as GSM, StrategyQA, and

Coin, RE2 signiﬁcantly enhances Vanilla prompt-

ing (

↑

4.63 on StrategyQA and

↑

5.20 on the Coin

dataset). Overall, our RE2 method still achieves

improvements in 71% of the experiments on Chat-

GPT. More examples from the experiment results

can be found in Appendix E.

3.4 Discussions

Times of Question Reading We delve deeper

into the impact of the times of question re-reading

on reasoning performance. Figure 3 illustrates how

the performance of two distinct LLMs evolves con-

cerning various times of question re-reading. An

overarching pattern emerges across all models: per-

formance improves until the number of re-reads

reaches 2 or 3, after which it begins to decline with

LLMs Methods GSM

ChatGPT

PS 75.59

PS+RE2 76.27

PAL 75.59

PAL + RE2 79.38

davinci-003

PS 55.65

PS+RE2 58.68

PAL 68.61

PAL + RE2 70.20

Table 3: Evaluation results of some thought-eliciting

promptings beyond CoT with RE2.

further increases in question re-reading times. The

potential reasons for inferior performance when

reading the question multiple times are two-fold:

i) overly repeating questions may act as demon-

strations to encourage LLMs to repeat the ques-

tion rather than generate the answer, and ii) re-

peating the question signiﬁcantly increase the in-

consistency of the LLMs between our inference

and pretraining/alignment (intuitively in the learn-

ing corpora, we usually repeat a question twice to

emphasize the key part, rather not more). It’s note-

worthy that reading the question two times tends

to be optimal for accommodating most scenarios

in our experiments, which is why we refer to this

practice as “re-reading” in our paper.

Compatibility with Thought-Eliciting Prompt

Strategies Compared to previous methods at-

tempting to elicit thoughts in the output from

LLMs, our RE2 emphasizes the understanding of

the input. Therefore, we are intrigued to explore

whether RE2 is effective with various thought-

eliciting prompting strategies other than CoT. To

investigate this, we apply RE2 to two other recently

introduced prompting methods, namely, Plan-and-

Solve (PS) (Wang et al., 2023a) and Program-

Aided Language models (PAL) (Gao et al., 2023).

The former model devises a plan to divide the en-

tire task into smaller subtasks, and then carries out

the subtasks according to the plan, while the latter

generates programs as the intermediate reasoning

steps. We directly apply our RE2 to these two

methods by making a simple alteration to the in-

put by repeating the question. Table 3 presents

the evaluation ﬁndings on the GSM benchmark.

Our observations reveal a consistent trend, akin to

what was observed with chain-of-thought prompt-

ing. These results suggest that the effectiveness

of our RE2 mechanism generally extends across

various prompting methodologies.

剩余24页未读，继续阅读

评论收藏

内容反馈

豪AI冰

粉丝: 73
资源: 44

Re-Reading提升大型语言模型推理能力

Tony Buzan - The Photo-Reading Whole Mind System [Speed-Reading Course. Learn It. Do It. Use It. Wow!!]

cambridge-english-key-sample-paper-1-reading-and-writing v2.pdf

swift-ReadingList允许用户跟踪和归类他们阅读书籍的iOS应用

KET样卷 cambridge-english-key-for-schools-sample-paper-1-reading-and-writing v2

unit15 Destinations---Reading

开源项目-enocom-gopher-reading-list.zip

unit-3-life-in-the-future-reading-语言点.ppt

Unit1-Art---Reading(公开课).pptx

选修8-Unit2-Cloning-Reading.ppt

唇语识别系统使用机器视觉技术源码lip-reading-deeplearning-master

必修五-unit-3-reading-语法点.ppt

Bionic-reading浏览器插件

t-l-51729-little-red-riding-hood-traditional-tales-differentiated-reading-comprehension-activity.pdf

申请免费下载i-Reading电子书表格.docx

hexo-theme-reading,河西博客主题：.zip

Secondary School Admission Test - Reading Comprehension题库.docx

engine-reading.c

nlp-reading-group, 在亚利桑那州大学，NLP阅读小组.zip

Bitmap-Reading.rar_reading

PS2-keyboard-reading-test.rar_reading

use-reading-time:react一个React钩子，显示可以读取多少时间的内容

Unit5Themeparks-Warmingupandpre-reading必修4[课件].ppt

思维导图UNIT1-READING-1.pdf

人教版高中英语必修4-Unit5--Reading公开课课件(共42张PPT)PPT课件.pptx

Chapter4. The Body Shop -Reading.pdf

filename-reading.rar_reading_文件名

前端项目-reading-time.zip

YOLOv8-deepsort 实现智能车辆目标检测+车辆跟踪+车辆计数

YOLOv8网络结构图，自制visio文件，yolov8.vsds，需要的自取，在原有的基础上直接改就行了

最新资源