王思远__A Multi-Agent Communication Framework for Question-Worthy
需积分: 0 200 浏览量
2022-08-03
14:34:08
上传
评论
收藏 980KB PDF 举报
www.postersession.com
A Multi-Agent Communication Framework for
Question-Worthy Phrase Extraction and Question Generation
Siyuan Wang
1
, Zhongyu Wei
1
, Zhihao Fan
1
, Yang Liu
2
, Xuanjing Huang
3
1 School of Data Science, Fudan University, China
2 LAIX Inc., China
3 School of Computer Science, Fudan University, China
Contributions:
Propose to generate multiple questions given input
sentence without ground-truth answers.
Extract question-worthy phrases from the input
sentence and generate questions based on such
information.
Develop a multi-agents communication framework to
learn the two tasks simultaneously.
We propose to extract question-worthy phrases and use such information for better question generation.
We introduce a multi-agent communication framework to learn tasks of phrase extraction and question generation simultaneously.
Our framework is able to generate multiple questions given input sentence without any ground-truth answers.
Stanford Question Answering Dataset
(SQuAD)
Answers, extractive from sentences,
are treated as target question-
worthy phrases.
We have totally 61623 sentences
corresponding to 90682 questions.
More than 30% sentences have
multiple questions.
Given a sentence, our research aims to identify
question-worthy phrases first and generate questions
with the assistance of these phrases.
Local Agent: it applies pointer network boundary
model to extract question-worthy phrases.
Generation Agent: it is based on seq-to-seq model with
attention mechanism, takes both sentence and an
phrase as input, to generate a question.
MP Extraction Agent: it employs Match-LSTM and
Pointer Network, taking question information from the
generation agent.
Distribution of number of questions per sentence
Comparison of Extraction Models
Metrics: EM(ExactMatch), F1 score, Average numbers of phrases.
Comparative Models
𝓔
𝐍𝐄𝐑
: extract name entities as question-worthy phrases
𝓔
𝐋𝐨𝐜𝐚𝐥
: the local extraction agent
𝓔
𝐌𝐏
: the extraction agent in message passing module
Model EM F1 Avg.# of phrases
𝓔
𝐍𝐄𝐑
13.12% 17.33 0.86
𝓔
𝐋𝐨𝐜𝐚𝐥
24.27% 38.63 1.43
𝓔
𝐌𝐏
35.77% 46.71 1.38
Results of different extraction models. (underline: significance test, 𝑝 < 0.01)
Comparison of Generation Models
Metrics: BLEU 1-4, METEOR, ROUGE
L
.
Comparative Models
𝐍𝐐𝐆
𝐑𝐮𝐥𝐞
: a rule-based model applying an overgenerate-and-rank
approach
𝐍𝐐𝐆
𝐏𝐮𝐫𝐞
: a pure version of QG using Seq2Seq model with attention
𝐍𝐐𝐆
𝐍𝐄𝐑
: take phrases from
𝓔
𝐍𝐄𝐑
as assistance to generate questions
𝐍𝐐𝐆
𝐋𝐨𝐜𝐚𝐥
: question generation using phrases from
𝓔
𝐋𝐨𝐜𝐚𝐥
𝐍𝐐𝐆
𝐌𝐏
: the generation agent in MP module, using phrases from
𝓔
𝐌𝐏
𝐍𝐐𝐆
𝐀𝐧𝐬𝐰𝐞𝐫
: answer-aware, use the ground truth of answers to
generate questions, upper bound
Model BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR
𝐑𝐎𝐔𝐆𝐄
𝑳
𝐍𝐐𝐆
𝐑𝐮𝐥𝐞
38.15 21.03 14.15 9.98 13.38 29.00
𝐍𝐐𝐆
𝐏𝐮𝐫𝐞
43.83 23.80 14.46 9.05 14.63 36.50
𝐍𝐐𝐆
𝐍𝐄𝐑
44.00 23.79 14.52 9.22 14.89 36.32
𝐍𝐐𝐆
𝐋𝐨𝐜𝐚𝐥
44.36 24.58 15.23 9.76 15.15 37.00
𝐍𝐐𝐆
𝐌𝐏
45.70* 25.87* 16.33* 10.56* 15.76* 38.09*
𝐍𝐐𝐆
𝐀𝐧𝐬𝐰𝐞𝐫
47.49 27.81 17.9 11.81 16.84 40.23
Results of different generation models. (underline: significance test, 𝑝 < 0.01; *: 𝑝 < 0.05)
AAAI 2019
Introduction
Framework
Multi-Agent Communication Framework
Dataset
Question Number
Sentence Quantity
1 41,356
2 14,499
3 3,921
4 1,198
≥ 5
649
in total 61,623
Experiments
Conclusion
Fudan SDS NLP Group
Case
Study
评论0