重新构想人工智能与人类协作，用于生成自由文本解释.docx资源-CSDN文库

版权申诉

41 浏览量 2024-04-17 12:48:07 上传评论收藏 1.51MB DOCX 举报

### 重新构想人工智能与人类协作，用于生成自由文本解释 #### 概述随着自然语言处理技术的不断发展，大型语言模型（如T5、T0Filter等）在生成流畅文本方面的能力越来越强。这些模型能否准确地为分类决策提供解释呢？本文探讨了如何利用少量的人类编写示例来生成自由文本解释，并通过实验发现高质量的提示能够显著提升生成文本的质量。此外，在与人工编写的解释进行对比时，GPT-3等模型生成的解释往往更受偏好。 #### 关键知识点详解 1. **自由文本解释的重要性**： - 随着自然语言理解任务的复杂性增加，自然语言处理领域开始重视使用自由形式的自然语言来解释模型的决策过程。 - 自由文本解释有助于提高模型的透明度和可解释性，使得用户更容易理解和接受模型的决策。 2. **高质量提示的重要性**： - 作者发现，编写更高质量的提示可以显著提高模型生成文本的质量。 - 这表明对于生成自由文本解释而言，良好的提示设计至关重要。 3. **GPT-3与人工编写的解释对比**： - 实验结果显示，在多数情况下，参与者更倾向于选择GPT-3生成的解释而非人工编写的解释。 - 这可能是因为GPT-3等模型能够生成更加流畅、自然且具有逻辑性的解释。 4. **人类参与的重要性**： - 尽管自动模型在生成解释方面表现出色，但研究还指出，它们在提供新颖信息和支持标签等方面仍有改进空间。 - 因此，结合人类监督的过滤机制对于提高解释的质量至关重要。 5. **接受性判断的作用**： - 为了进一步优化解释的质量，研究者们开发了一个结合GPT-3与人类监督过滤器的管道。 - 该过滤器能够根据人类的二元接受性判断对GPT-3生成的解释进行筛选。 6. **接受性与解释属性的相关性**： - 研究显示，尽管接受性判断具有一定的主观性，但它与解释的各种细粒度属性之间存在部分相关性。 - 例如，一个被广泛接受的解释通常具备准确性、语法正确性和充分性等特点。 7. **未来展望**： - 结合人类与AI的合作模式将继续是提高自由文本解释质量和可信度的关键方向。 - 未来的研究可以进一步探索如何优化模型以生成更加丰富、精确且易于理解的解释，同时保持其自动化和高效性。 #### 总结本文通过实验证明了高质量提示对于生成自由文本解释的重要性，并展示了GPT-3等模型在生成自然语言解释方面的潜力。同时，研究强调了人类参与在提升解释质量方面的重要作用，特别是在确保解释的新颖性和支持性方面。通过引入接受性判断作为过滤机制，可以进一步提高模型生成解释的质量。这一研究成果为自然语言处理领域的可解释性和人机协作提供了有价值的参考。

资源推荐

资源详情

资源评论

Filter Model

Reframing Human-AI Collaboration for

Generating Free-Text Explanations

Sarah Wiegreffe

♣

Jack Hessel

†

Swabha Swayamdipta

†

Mark Riedl

♣

Yejin Choi

令†

♣

School of Interactive Computing, Georgia Institute of Technology

†

Allen

Institute

for

Artificial

Intelligence

令

Paul G. Allen School of Computer Science and Engineering, University of Washington

saw@gatech.edu,

riedl@cc.gatech.edu,

{jackh,swabhas,yejinc}@allenai.org

Abstract

Large language models are increasingly capa-

ble of generating fluent-appearing text with rel-

atively little task-specific supervision. But can

these models accurately explain classification

decisions? We consider the task of generating

free-text explanations using human-written ex-

amples in a few-shot manner.

We find that

(1) authoring higher quality prompts results

in higher quality generations; and (2) surpris-

ingly, in a head-to-head comparison, crowd-

workers often prefer explanations generated by

GPT-3 to crowdsourced explanations in exist-

“Usually a hamburger with friends

indicates a good time.”

Select-

Explanation-

Level

Acceptability

Predictions

Crowdworker

Acceptability

ing datasets. Our human studies also show,

however, that while models often produce fac-

tual, grammatical, and sufficient explanations,

they have room to improve along axes such

greedy

stochastic

Labels

as providing novel information and supporting

the label. We create a pipeline that combines

GPT-3 with a supervised filter that incorpo-

rates binary acceptability judgments from hu-

mans in the loop. Despite the intrinsic subjec-

tivity of acceptability judgments, we demon-

strate that acceptability is partially correlated

with various fine-grained attributes of explana-

tions. Our approach is able to consistently fil-

ter GPT-3-generated explanations deemed ac-

ceptable by humans.

1 Introduction

As natural language understanding tasks have be-

come increasingly complex, the field of explainable

NLP has embraced explanations written in free-

form natural language. In contrast to extractive

explanations that highlight tokens in the input, free-

text explanations provide a natural interface be-

tween machine computation and human end-users

(Hendricks et al., 2016; Camburu et al., 2018). The

dominant paradigm for producing free-text expla-

nations is via direct supervision, i.e., training an

autoregressive, generative language model to pre-

dict human-authored explanations directly (Kim

et al., 2018; Park et al., 2018; Ehsan et al., 2018;

Narang et al., 2020; Wiegreffe et al., 2021, i.a.).

“When eating a hamburger with friends, what is

one trying to do?” have fun. Explanation: …

Instance + Explanation Prompt

Figure 1: Illustration of our overgeneration + filtra-

tion pipeline for producing human acceptable explana-

tions for CommonsenseQA and SNLI (see examples in

Table 1). Authors of this work write explanations to

prompt GPT-3, generating 5 explanations per instance.

An acceptability filter, trained on human binary accept-

ability judgments, determines which of these generated

explanations are plausible. Evaluation is performed at

both the explanation and the instance level.

However, collecting high-quality written expla-

nations to serve as supervision is difficult and ex-

pensive. More than 70% of existing free-text ex-

planation datasets are crowdsourced (Wiegreffe

and Marasovic´, 2021), and even the most metic-

ulous crowdsourcing efforts frequently fail to elicit

logically consistent and grammatical explanations

(Narang et al., 2020). Furthermore, a lack of

standardized crowdsourcing design has resulted in

highly varied datasets, which are hard to compare

or combine (Tan, 2021).

632

Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics:

Human Language Technologies, pages 632 - 658

GPT-3 DaVinci

top score

Overgenerating Explanation Filtering Acceptable

Evaluation

Candidates

Explanations

633

Recent progress in prompting large language

models (LLMs) provides a potentially promising

alternate to large-scale crowdsourcing. The in-

context learning paradigm, wherein powerful lan-

guage models are prompted in a few-shot manner

with just a few examples, has proven surprisingly

effective across a range of NLP tasks (Radford

et al., 2019; Brown et al., 2020; Shin et al., 2020;

Schick and Schütze, 2021a, i.a.). In this work

we ask: can LLMs also generate reliable expla-

nations? In human subjects studies, we find that

GPT-3 (Brown et al., 2020) can be readily made

to generate explanations via prompting, and sur-

prisingly, humans often prefer GPT-3 generated

explanations to crowdsourced explanations in ex-

isting datasets (§2).

Two additional human subjects studies, however,

demonstrate that GPT-3-generated explanations

still have significant room for improvement along

axes such as providing new information (i.e., avoid-

ing repetition) and supporting the label; human

subjects found less than half of greedily-decoded

GPT-3-generated explanations to be acceptable

with 100% agreement. To improve upon this, we

re-frame the role of crowd annotators: instead of

asking them to write explanations as in prior work,

we (1) repeatedly query GPT-3 to generate multi-

ple candidate explanations for each input instance,

and (2) ask crowdworkers to rate the acceptability

of each candidate generation. After showing that

GPT-3 can usually generate an explanation that hu-

mans unanimously find acceptable within as few as

five queries (§3), we use a small number of these

binary crowdworker judgments to supervise an ac-

ceptability filtering model, which can be applied

to select high quality candidates among GPT-3’s

outputs (Figure 1; §4).

Despite intrinsic subjectivity in acceptability rat-

ings, our supervised model improves upon the

already-competitive few-shot paradigm by consis-

tently selecting (human-identified) high quality ex-

planations better than strong baselines. Human

evaluations reveal that the filtration model not only

improves acceptability, but also other axes like sup-

porting the label and providing novel information.

In summary, our main findings are:

few-shot prompting with GPT-3 produces sur-

prisingly competitive explanations, providing

a promising alternative to crowd-authored free-

text explanation corpora;

ii.

binary human labeling can instead be leveraged

to train a filter that selects high-quality machine-

generated explanations; and

iii.

in areas where GPT-3 struggles, including infor-

mation content, supporting the label, and overall

acceptability, our proposed overgenerate-and-

filter pipeline improves generated explanations.

We publicly release our code and data.

2 GPT-3 is Competitive with

Crowdsourced Explanation Datasets

We investigate three research questions:

Are GPT-3-generated explanations preferable to

crowdsourced ones in existing datasets? (§2.1)

Can improving prompt quality improve GPT-3-

generated explanations? (§2.2)

Along what fine-grained dimensions are GPT-3-

generated explanations preferred, and do these

correlate with overall acceptability? (§2.3)

Explanation tasks and datasets. We consider

two English tasks: CommonsenseQA and natural

language inference (NLI), shown in Table 1. Com-

monsenseQA (Talmor et al., 2019) is a multiple

choice task posed over commonsense questions.

Crowdsourced free-text explanations for instances

in CommonsenseQA are provided in the CoS-E

v1.11 (Rajani et al., 2019) and ECQA (Aggarwal

et al., 2021) datasets.

ECQA explanations are coun-

terfactual, i.e., annotators were instructed to ex-

plain not only the correct answer choice but also

why the others are incorrect.

ECQA was released to

address the quality issues of CoS-E (Narang et al.,

2020); for completeness, we experiment with both.

Our second task is NLI, which involves inferring

whether a given hypothesis sentence entails, contra-

dicts, or is neutral towards a premise. This task is

instantiated with the SNLI dataset (Bowman et al.,

2015) and crowdsourced explanations from e-SNLI

(Camburu et al., 2018). For each task, we report

results on a fixed, randomly-sampled 250-instance

test set not observed during prompt design.

Few-shot prompting for explanations. We use

GPT-3 Davinci

(Brown et al., 2020), an autore-

gressive language model with ~175B parameters

trained on a large dataset of text scraped from the

internet. We prompt the model with several (ques-

tion, gold answer, explanation) triplets followed by

an unexplained question-gold answer instance for

https://github.com/allenai/few_shot_explanations/

Non-instruct version, available in Summer 2021.

634

SNLI (Bowman et al., 2015)

Premise: Dark-haired man wearing a watch and oven

mitt about to cook some meat in the kitchen.

Hypothesis: A man is cooking something to eat.

Label: entailment

e-SNLI (Camburu et al., 2018): Meat is cooked in a

kitchen, and is a food that you eat. Using an oven mitt

implies you’re about to cook with hot utensils.

GPT-3: Cooking is usually done to prepare food to eat.

CommonsenseQA (Talmor et al., 2019)

Question: What is the result of applying for job?

Answer Choices: anxiety and fear, increased workload,

praise, less sleep, or being employed

Correct Choice: being employed

CoS-E (Rajani et al., 2019): being employed applying

for job

ECQA (Aggarwal et al., 2021): Applying for a job is

followed by attending interview which results in being

employed.

Applying for a job may not result in the other

options.

GPT-3: Applying for a job can result in being em-

ployed, which is a positive outcome.

Table 1: Task-specific instances, along with their

crowdsourced explanations from the respective

datasets, shown alongside explanations generated

greedily by GPT-3. In our experiments, the SNLI GPT-

3 explanation was preferred over its corresponding

e-SNLI explanation by 2/3 annotators. For Common-

senseQA, 3/3 preferred the GPT-3 explanation to the

CoS-E explanation, and 2/3 to the ECQA one.

which we expect the model to generate an explana-

tion.

We use a total of 115 randomly sampled train

instances to create our prompts; each prompt con-

sists of 8-24 randomly selected examples from this

set. For each instance, we generate a single expla-

nation with greedy decoding. More details about

prompt construction are in Appendix A; example

prompts are given in Tables 2 and 11.

Crowdsourcing evaluation. Given that existing

automatic metrics often do not correlate well with

human judgements of explanation quality (Clinciu

et al., 2021; Kayser et al., 2021),

we conduct human

We condition on the gold label as a methodological con-

trol to ensure reliable human evaluation. In pilot studies, we

found it hard to avoid bias against explanations when we

Let’s explain classification decisions.

A young boy wearing a tank-top is climbing a tree.

question: A boy was showing off for a girl.

true, false, or neither? neither

why? A boy might climb a tree to show off for a girl,

but he also might do it for fun or for other reasons.

###

A person on a horse jumps over a broken down airplane.

question: A person is outdoors, on a horse.

true, false, or neither? true

why? Horse riding is an activity almost always done

outdoors. Additionally, a plane is a large object and is

most likely to be found outdoors.

###

There is a red truck behind the horses.

question: The horses are becoming suspicious of my

apples.

true, false, or neither? false

why? The presence of a red truck does not imply there

are apples, nor does it imply the horses are suspicious.

###

A dog carries an object in the snow.

question: A dog is asleep in its dog house.

true, false, or neither? false

why?

Table 2: Example of a prompt with 3 training examples

for SNLI: presented are the premise/hypothesis pairs,

the gold labels, and the explanations (written by us)

that act as input to GPT-3 (in practice, we use 8-24 ex-

amples per prompt). The text generated by the model

acts as the free-text explanation. In this case, the model

greedily auto-completes (given 12 examples): “A dog

cannot carry something while asleep”.

studies for evaluation.

We ensure each experiment

has a substantial number of distinct crowdworkers

to mitigate individual annotator bias (Table 17).

We present workers with a dataset instance, its

gold label, and two explanations for the instance

generated under different conditions (“head-to-

head”). We then ask them to make a preferential

selection, collecting 3 annotations per data point.

We report inter-annotator agreement (IAA) using

Krippendorff’s α (Krippendorff, 2011). We find

low-to-moderate agreement across studies, indicat-

ing the subjective nature of the task; see Tables 3,

4, and 5. Appendix B contains further details on

quality control.

disagreed with the predicted label. Prior work (Kayser et al.,

2021; Marasovic´ et al., 2021) has removed this confounder

by only considering explanations for correctly-predicted in-

stances, which may overestimate explanation quality. Our

method allows us to report results on a truly representative

sample. We experiment with incorrect vs. correct predictions

in Appendix C, finding that GPT-3 can competitively explain

gold labels for instances it predicted incorrectly.

Our studies were conducted on the Amazon Mechanical

Turk platform, at the Allen Institute for AI. We selected work-

ers located in Australia, Canada, New Zealand, the UK, or the

US, with a past HIT approval rate of >98% and >5000 HITs

approved, and compensated them at a rate of $15/hour. Each

worker completed qualifying exams on explanation evaluation

and the NLI task.

635

Preferred Explanation (%)

Dataset Crowd Tie GPT-3 α

CoS-

20.3

34.9

44.8

0.5

ECQA

52.7

12.9

34.4

0.2

e-SNLI

63.6

11.6

24.8

0.3

Table 3: Head-to-head human evaluation for 250 ex-

planations generated by GPT-3 vs. written by crowd-

workers in the datasets, along with Krippendorff’s α

for IAA. Results are shown as % preferences. We

prompted GPT-3 with crowd explanations from the cor-

responding datasets.

2.1

Are GPT-3 explanations preferred over

crowdsourced ones?

We perform a head-to-head comparison of expla-

nations generated by GPT-3 with greedy decod-

ing vs. gold human-written explanations in the

original datasets. The crowdsourced explanations

serve as a reasonable upper bound for what a su-

pervised explanation generation model trained on

them could produce. Table 1 contains examples of

GPT-3-preferred explanations.

Results are shown in Table 3. GPT-3 greedily-

decoded explanations are frequently preferred or

comparable to crowdsourced explanations in CoS-

E, which is not too surprising given the dataset has

many ungrammatical explanations (Narang et al.,

2020).

And, while ECQA and e-SNLI explanations

are strongly preferred to GPT-3, there are still a non-

trivial number of cases where GPT-3 explanations

are competitive (47.3% and 36.4%, respectively).

2.2

Can improving prompt quality improve

GPT-3-generated explanations?

Preferred GPT-3 Explanation (%)

Dataset Crowd Prompts Tie Our Prompts α

CoS-

6.9

13.5

79.6 0.2

ECQA

15.9

9.5

74.7 0.3

SNLI

30.8

26.8

42.4 0.5

Table 4: Head-to-head human evaluation for 250 ex-

planations generated by GPT-3 prompted with either

author-written explanations or crowdsourced explana-

tions from the associated datasets, along with Krippen-

dorff’s α for IAA.

Given that low-quality training instances may re-

sult in low-quality predictions (especially in a few

shot setting),

we ask: can we improve GPT-3 gen-

For example, GPT-3 reproduces known data artifacts in

the CoS-E corpus when prompted with explanations from it,

such as the recurring phrase “rivers flow trough (sp) valleys”.

Preferred Explanation (%)

Dataset

Crowd

Tie

GPT-

3 α

CoS-

7.2

13.9

78.9 0.5

ECQA

44.5

9.7

45.7 0.4

e-SNLI

49.6

9.7

40.7 0.2

Table 5: Head-to-head human evaluation for 250 ex-

planations generated by GPT-3 vs. written by crowd-

workers in the datasets, along with Krippendorff’s α

for IAA. GPT-3 explanations were prompted with ex-

planations written by the authors.

erations simply by conditioning on higher-quality

instances? To this end, we replace the 115 crowd-

sourced explanations from the original datasets for

prompting GPT-3 with explanations carefully writ-

ten by the authors of this paper (see Table 12 for

examples). Our prompts are used to generate a dif-

ferent set of GPT-3 explanations on the same test

data.

We perform a head-to-head human evaluation

of the GPT-3 generations conditioned on the ex-

planations we wrote vs. those conditioned on the

crowdsourced explanations. Results in Table 4

show that, for all three corpora, generations condi-

tioned on our explanations outperform generations

conditioned on crowdsourced ones, illustrating the

importance of good-quality prompts for GPT-3.

We repeat the experiment of §2.1, but with our

prompts instead of dataset prompts. With this

change, GPT-3 generations are even more com-

petitive (Table 5). For all three datasets, more than

half the time, few-shot prompting results in an ex-

planation at least as good as a human-written ex-

planation. For subsequent experiments, we prompt

GPT-3 with the author-written explanations.

2.3

What types of explanations does GPT-3

generate?

Pairwise evaluations can only offer perspective on

the relative quality of generated explanations. Are

crowd annotators simply comparing explanations

on surface-level features like grammaticality?

To understand finer-grained characteristics of ex-

planations, we design a second human study to col-

lect absolute Likert-scale judgments across seven

axes of quality (with each explanation judged by

3 annotators). The first three axes capture surface-

level features: generality, grammaticality, and fac-

tuality. The next three capture richer aspects of

explanation quality: whether new information is

introduced

requirement

for

non-vacuous

explana-

636

0.5

-0.5

-1

0.5

-1

0.5

-0.5

-1

General Generally True Grammatical Introduced Yes Too Much Acceptable

Specific

Generally False Ungrammatical None Introduced

No Not Enough Not Acceptable

Generality Factuality Grammar New Info Supports Label Amount Info* Acceptability

Figure 2: Absolute evaluation for GPT-3 and crowdsourced explanations for CommonsenseQA via CoS-E (top)

and ECQA (middle) datasets, and for NLI via the e-SNLI dataset (bottom). The distribution of mean scores of 3

annotators for each instance in the test set is plotted. All attributes besides Factuality and Amount Info are binary.

*Amount Info is the only attribute for which a value of 0 is preferred to a value of 1. See Table 18 for more details.

tions), whether explanations support the gold label,

and whether the amount of information given is

sufficient. Finally, we ask for an overall judgement

of quality: is the explanation acceptable? We ex-

plain our design process in Appendix B.3. Results

on the crowdsourced and GPT-3 explanations for

both tasks are given in Figure 2.

For both tasks, GPT-3 explanations do well in

all 3 surface-level categories, with statistically sig-

nificantly greater ratings in generality and gram-

maticality (and factuality for CommonsenseQA)

compared to crowdsourced explanations, and dis-

tributional means close to 1. In these categories,

there is little room for improvement.

On the other hand, GPT-3 explanations do not

contain as much new information as ECQA and

e-SNLI explanations, indicating substantial room

for improvement (mean=0.1 for both tasks com-

pared to 0.6 for ECQA and 0.2 for SNLI; these

differences are statistically significant at p ≤ 0.01).

GPT-3

explanations

are

substantially

support-

ive of the label vs. CoS-E, but not as supportive

as ECQA or e-SNLI (all statistically significant

at p

≤

0.1).

Indeed, the mean rating of GPT-3

senseQA and −0.1 for NLI, demonstrating room

for improvement. These axes are crucial to ensur-

ing explanations are not vacuous and are on-topic.

Finally, GPT-3 explanations are judged as accept-

able at higher rates than CoS-E or ECQA explana-

tions,

but

not

e-SNLI

explanations.

Mean

scores

0.5 (CommonsenseQA) and 0.0 (NLI) indicate that

GPT-3 explanations have

room to

improve overall.

Correlation between acceptability and other at-

tributes To understand what factors are impor-

tant for the overall “acceptability" judgement, we

compute Spearman correlation (ρ; Spearman, 1987)

between acceptability and all other attributes (Ta-

ble 6). Each is positively correlated with accept-

ability, though with varying degrees of magnitude.

Acceptability is least correlated with “new infor-

mation," and most correlated with grammaticality,

generality, and the explanation’s support for the

label. Overall, the results indicate that human pref-

erence for explanations is not fully explained by

any one attribute, and is not limited to surface-level

features.

explanations for label support is 0.5 for Common-

We do not evaluate explanations counterfactually, which

Krippendorff’s α is 0.48 for CommonsenseQA annota- may explain why ECQA explanations are often labeled as

tions and 0.31 for SNLI; see Table 15. having “too much information”. See Appendix B.1.

Enough

Sometimes/

Partially True

GPT-3

CoS-E

GPT-3

ECQA

GPT-3

e-SNLI

剩余26页未读，继续阅读

评论收藏

内容反馈

版权申诉

百态老人

粉丝: 1w+
资源: 2万+

重新构想人工智能与人类协作，用于生成自由文本解释.docx

人工智能完整教程：技术体系介绍，项目开发指导.docx

清华大学中国人工智能学会：2019人工智能发展报告.docx

ChatGPT技术用于智能文本生成的实现方法.docx

生成式人工智能.docx

机器学习实验报告-朴素贝叶斯学习和分类文本.docx

RED二次开发-Node-RED与AI技术结合.docxRED二次开发-Node-RED与HTTP协议集成.docx

GitHub协作最佳实践与案例分析.docx

敏捷开发角色解释.docx

MATLAB使用文本文件.docx

地震响应分析：多自由度系统地震响应.docx地震响应分析：多自由度系统地震响应all.docx地震响应分析：多自由度系统地震响应-（11）.动力时程分析方法.docx地震响应分析：多自由度系统地

招商信诺2020年校园招聘真题及解析.docx.docx

协作队伍安全生产责任书.docx

地震动模拟：地震动的随机模拟.docx地震动模拟：地震动的随机模拟all.docx地震动模拟：地震动的随机模拟-（10）.地震动的时频域分析.docx地震动模拟：地震动的随机模拟-（11）.地震

附件3：第六届中国研究生人工智能创新大赛华为专项赛题详情.docx

2019年全国职业院校技能大赛人工智能技术与应用赛项申报方案.docx

CryEngine引擎开发：AI敌人行为-（1）.CryEngine引擎基础.docxCryEngine引擎开发：AI敌人行为-（2）.AI系统概览.docx

1秒生成几万字.docx

材料特性仿真：高分子材料的断裂仿真.docx材料特性仿真：高分子材料的断裂仿真all.docx材料特性仿真：高分子材料的断裂仿真v1.docx材料特性仿真：高分子材料的断裂仿真-（10）.高分子

测试与验证仿真：功能测试.docx测试与验证仿真：功能测试all.docx测试与验证仿真：功能测试v1.docx测试与验证仿真：功能测试-1.功能测试基础理论.docx测试与验证仿真：功能测试

你常见的“区块链”英文缩写和名词解释.docx

2019-2020年度人工智能-人工智能单选题.docx

CryEngine引擎开发：角色控制与AI-（5）.输入处理与控制.docxCryEngine引擎开发：角色控制与AI-（6）.AI基础概念与架构.docx

遂宁市2019年全市专业技术人员继续教育公需科目 人工智能与健康试题02.docx

AVEVA系统平台报表设计与生成教程.Tex.header.docx

测试与验证仿真：测试向量生成.docx测试与验证仿真：测试向量生成all.docx测试与验证仿真：测试向量生成-（10）.测试向量的有效性评估.docx测试与验证仿真：测试向量生成-（11）.测

CryEngine引擎开发：AI行为树与状态机-（10）.AI状态机的设计与优化.docxCryEngine引擎开发：AI行为树与状态机-（11）.AI行为树与状态机的调试技巧.docx

matlab中如何读取文本文件.docx

冲击动力学基础：数值模拟方法.docx冲击动力学基础：数值模拟方法all.docx冲击动力学基础：数值模拟方法-1.冲击动力学基础理论.docx冲击动力学基础：数值模拟方法-10.断裂和损伤的数

ecshop生成清晰缩略图.docx

docx4j及其依赖包

最新资源

遂宁市2019年全市专业技术人员继续教育公需科目人工智能与健康试题02.docx