【免费】AIGC.pdf资源-CSDN文库

需积分: 0 113 浏览量 2023-05-14 01:32:30 上传评论收藏 2.61MB PDF 举报

"AIGC.pdf" AIGC（Artificial Intelligence Generated Content，人工智能生成内容）是指通过人工智能模型生成数字内容，例如图像、音乐和自然语言，以提高内容创作效率和质量。本文将对AIGC的历史、基本组件、最近的发展进行综述，并从单模态和多模态交互两个角度对AIGC进行介绍。一、AIGC的历史 AIGC的历史可以追溯到 Generative Adversarial Networks（GAN）的出现，GAN是由 Ian Goodfellow 等人在 2014 年提出的。GAN 的出现标志着生成式人工智能的开端。之后，出现了许多其他的生成式模型，例如 Variational Autoencoder（VAE）和 Generative Adversarial Networks（GAN）。这些模型的出现使得生成式人工智能得到了快速发展。近年来，大规模模型的出现使得AIGC的发展取得了突破性进展。这些模型可以从大量数据中学习，并生成高质量的内容。例如，Transformer 模型的出现使得自然语言处理取得了突破性进展，并推动了AIGC的发展。二、AIGC的基本组件 AIGC的基本组件包括： 1. intent extraction：AIGC 的核心组件是 intent extraction，即从人类指令中提取意图信息。 2. Knowledge Retrieval：AIGC 模型需要从大量数据中检索相关信息，以便生成高质量的内容。 3. Generation：AIGC 模型通过 intent extraction 和知识检索生成内容。三、AIGC的单模态交互 AIGC 的单模态交互是指在一个模态（例如文本或图像）下的生成任务。例如： 1. 文本生成：AIGC 模型可以生成文本，例如聊天机器人、自动写作等。 2. 图像生成：AIGC 模型可以生成图像，例如图像合成、图像编辑等。四、AIGC的多模态交互 AIGC 的多模态交互是指在多个模态（例如文本、图像和音乐）下的生成任务。例如： 1. 文本到图像生成：AIGC 模型可以根据文本指令生成图像。 2. 图像到文本生成：AIGC 模型可以根据图像生成文本描述。五、AIGC的应用 AIGC 的应用非常广泛，例如： 1. 自动写作：AIGC 模型可以生成高质量的文本，例如新闻、博客等。 2. 图像合成：AIGC 模型可以生成高质量的图像，例如产品设计、电影特效等。 3. 聊天机器人：AIGC 模型可以生成文本，例如客服机器人、智能客服等。 AIGC 是一个具有广泛应用前景的技术领域。今后，AIGC 将继续发展和完善，推动人类社会的进步。

资源推荐

资源详情

资源评论

111

A Comprehensive Survey of AI-Generated Content (AIGC):

A History of Generative AI from GAN to ChatGPT

YIHAN CAO

∗

, Lehigh University & Carnegie Mellon University, USA

SIYU LI, Lehigh University, USA

YIXIN LIU, Lehigh University, USA

ZHILING YAN, Lehigh University, USA

YUTONG DAI, Lehigh University, USA

PHILIP S. YU, University of Illinois at Chicago, USA

LICHAO SUN, Lehigh University, USA

Recently, ChatGPT, along with DALL-E-2 [

] and Codex [

],has been gaining signicant attention from society.

As a result, many individuals have become interested in related resources and are seeking to uncover the

background and secrets behind its impressive performance. In fact, ChatGPT and other Generative AI (GAI)

techniques belong to the category of Articial Intelligence Generated Content (AIGC), which involves the

creation of digital content, such as images, music, and natural language, through AI models. The goal of

AIGC is to make the content creation process more ecient and accessible, allowing for the production of

high-quality content at a faster pace. AIGC is achieved by extracting and understanding intent information

from instructions provided by human, and generating the content according to its knowledge and the intent

information. In recent years, large-scale models have become increasingly important in AIGC as they provide

better intent extraction and thus, improved generation results. With the growth of data and the size of the

models, the distribution that the model can learn becomes more comprehensive and closer to reality, leading

to more realistic and high-quality content generation. This survey provides a comprehensive review on the

history of generative models, and basic components, recent advances in AIGC from unimodal interaction and

multimodal interaction. From the perspective of unimodality, we introduce the generation tasks and relative

models of text and image. From the perspective of multimodality, we introduce the cross-application between

the modalities mentioned above. Finally, we discuss the existing open problems and future challenges in AIGC.

CCS Concepts:

• Computer systems organization → Embedded systems

; Redundancy; Robotics;

• Net-

works → Network reliability.

Additional Key Words and Phrases: datasets, neural networks, gaze detection, text tagging

ACM Reference Format:

Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu, and Lichao Sun. 2018. A Comprehensive

Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT. J. ACM 37, 4,

Article 111 (August 2018), 44 pages. https://doi.org/XXXXXXX.XXXXXXX

∗

Incoming Ph.D. student at Lehigh University.

Authors’ addresses: Yihan Cao, yihanc@andrew.cmu.edu, Lehigh University & Carnegie Mellon University, Pittsburgh, PA,

USA; Siyu Li, applicantlisiyu@hotmail.com, Lehigh University, Bethlehem, PA, USA; Yixin Liu, lis221@lehigh.edu, Lehigh

University, Bethlehem, PA, USA; Zhiling Yan, zhilingyan724@outlook.com, Lehigh University, Bethlehem, PA, USA; Yutong

Dai, lis221@lehigh.edu, Lehigh University, Bethlehem, PA, USA; Philip S. Yu, University of Illinois at Chicago, Chicago,

Illinois, USA, psyu@uic.edu; Lichao Sun, lis221@lehigh.edu, Lehigh University, Bethlehem, PA, USA.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee

provided that copies are not made or distributed for prot or commercial advantage and that copies bear this notice and

the full citation on the rst page. Copyrights for components of this work owned by others than ACM must be honored.

Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires

prior specic permission and/or a fee. Request permissions from permissions@acm.org.

0004-5411/2018/8-ART111 $15.00

https://doi.org/XXXXXXX.XXXXXXX

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

arXiv:2303.04226v1 [cs.AI] 7 Mar 2023

111:2 Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu, and Lichao Sun

1 INTRODUCTION

In recent years, Articial Intelligence Generated Content (AIGC) has gained much attention beyond

the computer science community, where the whole society begins to be interested in the various

content generation products built by large tech companies [

], such as ChatGPT [

] and DALL-E-

2 [

]. AIGC refers to content that is generated using advanced Generative AI (GAI) techniques, as

opposed to being created by human authors, which can automate the creation of large amounts

of content in a short amount of time. For example, ChatGPT is a language model developed by

OpenAI for building conversational AI systems, which can eciently understand and respond to

human language inputs in a meaningful way. In addition, DALL-E-2 is another state-of-the-art GAI

model also developed by OpenAI, which is capable of creating unique and high-quality images

from textual descriptions in a few minutes, such as "an astronaut riding a horse ina photorealistic

style" as shown in Figure 1. As the remarkable achievements in AIGC, many people believe it will

be the new era of AI and make signicant impacts on the whole world.

Instructio n 2:

Teddy bears working on

new AI research on the

moon in the 1980s.

DALL·E 2

Instructio n 1:

An astronaut riding a

horse in a photorealistic

style.

Figure 1 Figure 2

Fig. 1. Examples of AIGC in image generation. Text instructions are given to OpenAI DALL-E-2 model, and it

generates two images according to the instructions.

Technically, AIGC refers to, given human instructions which could help teach and guide the model

to complete the task, utilizing GAI algorithms to generate content that satises the instruction.

This generation process usually consists of two steps: extracting intent information from human

instructions and generating content according to the extracted intentions. However, the paradigm

of GAI models containing the above two steps is not entirely novel, as demonstrated by previous

studies [

]. The core advancements in recent AIGC compared to prior works are the result of

training more sophisticated generative models on larger datasets, using larger foundation model

architectures, and having access to extensive computational resources. For example, the main

framework of GPT-3 maintains the same as GPT-2, but the pre-training data size grows from

WebText [

](38GB) to CommonCrawl [

](570GB after ltering), and the foundation model size

grows from 1.5B to 175B. Therefore, GPT-3 has better generalization ability than GPT-2 on various

tasks, such as human intent extraction.

In addition to the benets brought by the increase in data volume and computational power,

researchers are also exploring ways to integrate new technologies with GAI algorithms. For example,

ChatGPT utilizes reinforcement learning from human feedback (RLHF) [

–

] to determine the

most appropriate response for a given instruction, thus improving model’s reliability and accuracy

over time. This approach allows ChatGPT to better understand human preferences in long dialogues.

Meanwhile, in computer vision, stable diusion [

], proposed by Stability.AI in 2022, has also

shown great success in image generation. Unlike prior methods, generative diusion models

can help generate high-resolution images by controlling the trade-o between exploration and

exploitation, resulting in a harmonious combination of diversity in the generated images and

similarity to the training data.

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

A Comprehensive Survey of AI-Generated Content (AIGC):

A History of Generative AI from GAN to ChatGPT 111:3

Unimodal

Multimodal

Data

Generative AI Models

Pre-train

Data

Pre-train

Please write a

story about a cat.

Once upon a time,

there was a cat

named

Jessy….

Prompt

Decode

Describe this

picture.

Draw a picture

of a cat.

Instruction 𝐼

Result 𝑅

Instruction 𝐼

Write a song

about a cat.

Prompt

Result 𝑅

This is a cat.

Fig. 2. Overview of AIGC. Generally, GAI models can be categorized into two types: unimodal models and

multimodal models. Unimodal models receive instructions from the same modality as the generated content

modality, whereas multimodal models accept cross-modal instructions and produce results of dierent

modalities.

By combining these advancements, models have made signicant progress in AIGC tasks and

have been adopted in various industries, including art [

], advertising [

], and education [

]. In

the near future, AIGC will continue to be a signicant area of research in machine learning. It is

therefore crucial to conduct an extensive review of past research and identify the open problems in

this eld. This survey is the rst one that focuses on the core technologies and applications in the

eld of AIGC.

1.1 Major Contributions

This is the rst comprehensive survey of AIGC that summarizes GAI in the aspects of techniques

and applications. Previous surveys have focused on GAI from various angles, including natural

language generation [

], image generation[

], generation in multimodal machine learning [

However, these prior works only focus on a specic part of AIGC. In this survey, we rst provide

a review of foundation techniques commonly used in AIGC. Then, we further oer a thorough

summary of advanced GAI algorithms, both in terms of unimodal generation and multimodal

generation, as shown in Figure 2. In addition, we examine the applications and potential challenges

of AIGC. Finally, we highlight the open problems and future directions in this eld. In summary,

the main contributions of this paper are as follows:

•

To our best knowledge, we are the rst to provide a formal denition and a thorough survey

for AIGC and AI-enhanced generation process.

•

We review the history, foundation techniques of AIGC and conduct a comprehensive analysis

of recent advances in GAI tasks and models from the perspective of unimodal generation

and multimodal generation.

• We discuss the main challenges facing AIGC and future research trends confronting AIGC.

1.2 Organization

The rest of the survey is organized as follows. Section 2 reviews the history of AIGC mainly from

the view of vision and language modalities. Section 3 introduces the basic components that are

widely used in nowadays GAI model training. Section 4 summarizes recent advances of GAI models,

among which, Section 4.1 reviews the advances from unimodal perspective and Section 4.2 reviews

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

111:4 Yihan Cao, Siyu Li, Yixin Liu, Zhiling Yan, Yutong Dai, Philip S. Yu, and Lichao Sun

the advances from the perspective of multimodal generation. Among multimodal generation, we

introduce vision language models, text audio models, text graph models and text code models.

Section 5 and Section 6 introduce the applications of GAI models in AIGC and some other important

research that are related to this area. Furthermore, Sections 7, 8 reveal the risk, open problems and

future directions of AIGC technologies. Finally, we conclude our research in 9.

2 HISTORY OF GENERATIVE AI

Generative models have a long history in articial intelligence, dating back to the 1950s with the

development of Hidden Markov Models (HMMs) [

] and Gaussian Mixture Models (GMMs) [

These models generated sequential data such as speech and time series. However, it wasn’t until

the advent of deep learning that generative models saw signicant improvements in performance.

In early years of deep generative models, dierent areas do not have much overlap in general.

In natural language processing (NLP), a traditional method to generate sentences is to learn

word distribution using N-gram language modeling [

] and then search for the best sequence.

However, this method cannot eectively adapt to long sentences. To solve this problem, recurrent

neural networks (RNNs) [

] were later introduced for language modeling tasks , allowing for

modeling relatively long dependency. This was followed by the development of Long Short-Term

Memory (LSTM) [

] and Gated Recurrent Unit (GRU) [

], which leveraged gating mechanism to

control memory during training. These methods are capable of attending to around 200 tokens in a

sample [26], which marks a signicant improvement compared to N-gram language models.

Meanwhile, in computer vision (CV), before the advent of deep learning-based methods, tra-

ditional image generation algorithms used techniques such as texture synthesis [

] and texture

mapping [

]. These algorithms were based on hand-designed features, and were limited in their abil-

ity to generate complex and diverse images. In 2014, Generative Adversarial Networks (GANs) [

]

was rst proposed, which was a signicant milestone in this area, due to its impressive results

in various applications. Variational Autoencoders (VAEs) [

] and other methods like diusion

generative models [

] have also been developed for more ne-grained control over the image

generation process and the ability to generate high-quality images.

The advancement of generative models in various domains has followed dierent paths, but

eventually, the intersection emerged: the transformer architecture [

]. Introduced by Vaswani et

al. for NLP tasks in 2017, Transformer has later been applied in CV and then become the dominant

backbone for many generative models in various domains [

]. In the eld of NLP, many

prominent large language models, e.g., BERT and GPT, adopt the transformer architecture as

their primary building block, oering advantages over previous building blocks, i.e., LSTM and

GRU. In CV, Vision Transformer (ViT) [

] and Swin Transformer [

] later takes this concept

even further by combining the transformer architecture with visual components, allowing it to

be applied to image based downstreams. Except for the improvement that transformer brought

to individual modalities, this intersection also enabled models from dierent domains to be fused

together for multimodal tasks. One such example of multimodal models is CLIP [

]. CLIP is a

joint vision-language model that combines the transformer architecture with visual components,

allowing it to be trained on a massive amount of text and image data. Since it combines visual

and language knowledge during pre-training, it can also be used as image encoders in multimodal

prompting for generation. In all, the emergence of transformer based models revolutionized AI

generation and led to the possibility of large-scale training.

In recent years, researchers have also begun to introduce new techniques based on these models.

For instance, in NLP, instead of ne-tuning, people sometimes prefer few-shot prompting [

which refers to including a few examples selected from the dataset in the prompt, to help the

model better understand task requirements. And in visual language, researchers often combine

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

A Comprehensive Survey of AI-Generated Content (AIGC):

A History of Generative AI from GAN to ChatGPT 111:5

2014 20202018

2016

NLP

N-Gram

NLP

LSTM/GRU

NLP

Transformer

NLP

StyleGAN

BigBiGAN

DDPM

ViT

MoCo

BiGAN

RevNet

Show-Tell

DALL-E

BLIP2

DALL-E 2

VisualBERT

ViLBERT

UNITER

Unimodal- CV & NLP

Multimodal – Vision Language

CLIP

ALBEF

BLIP

VQ-GAN

GAN

VAE

Flow

ELMO

BERT

GPT-2

StyleNet

StackGAN

CAVP

DMGAN

VQ-VA E

Sparrow

chatGPT

GPT-3

OPT

BART

Fig. 3. The history of Generative AI in CV, NLP and VL.

modality-specic models with self-supervised contrastive learning objectives to provide more

robust representations.

In the future, as AIGC becomes increasingly important, more and more technologies shall be

introduced, empowering this area with vitality.

3 FOUNDATIONS FOR AIGC

In this section, we introduce foundation models that are commonly used in AIGC.

3.1 Foundation Model

3.1.1 Transformer. Transformer is the backbone architecture for many state-of-the-art models,

such as GPT-3 [

], DALL-E-2 [

], Codex [

], and Gopher [

]. It was rst proposed to solve the

limitations of traditional models such as RNNs in handling variable-length sequences and context-

awareness. Transformer architecture is mainly based on a self-attention mechanism that allows

the model to attend to dierent parts in a input sequence. Transformer consists of an encoder and

a decoder. The encoder takes in the input sequence and generates hidden representations, while

the decoder takes in the hidden representation and generates output sequence. Each layer of the

encoder and decoder consists of a multi-head attention and a feed-forward neural network. The

multi-head attention is the core component of Transformer, which learns to assign dierent weights

to tokens according their relevance. This information routing method allows the model to be better

at handling long term dependency, hence, improving the performance in a wide range of NLP tasks.

Another advantage of transformer is that its architecture makes it highly parallelizable, and allows

data to trump inductive biases [

]. This property makes transformer well-suited for large-scale

pre-training, enabling transformer based models to become adaptable to dierent downstream

tasks.

3.1.2 Pre-trained Language Models. Since the introduction of the Transformer architecture, it has

become the dominant choice in natural language processing due to its parallelism and learning

capabilities. Generally, these transformer based pre-trained language models can be commonly

classied into two types based on their training tasks: autoregressive language modeling and

masked language modeling [

]. Given a sentence, which is composed of several tokens, the

objective of masked language modeling, e.g., BERT [

] and RoBERTa [

], refers to predicting the

probability of a masked token given context information. The most notable example of masked

language modeling is BERT [

], which includes masked language modeling and next sentence

J. ACM, Vol. 37, No. 4, Article 111. Publication date: August 2018.

剩余43页未读，继续阅读

评论收藏

内容反馈

OWYANGLIN

粉丝: 0
资源: 1

AIGC.pdf

【13页】人工智能发生了方向性改变，重点关注AIGC.pdf

【28页】网达软件：大视频时代运营商AIGC龙头.pdf

人工智能专题研究：AIGC投资框架.pdf

2023从算力、算法、数据及应用看AIGC.pdf

腾讯研究院AIGC发展趋势报告2023.pdf

一篇文章搞清楚AIGC.pdf

AIGC：拐点时刻.pdf

AIGC发展研究.pdf

AIGC赋能企业管理.pdf

玩转AIGC与应用部署.pdf

ETFS All Commodities AIGC.pdf

AIGC for Various Data Modalities 适用于各种数据模式的 AIGC.pdf

内容生成的新时代——3D AIGC.pdf

2024年互联网投资方向：强个股、价格力、AIGC.pdf

阿里云：从零开始玩转AIGC.pdf

2023年 【8页】视觉中国 下游素材需求有望复苏，与百度战略合作加速AIGC落地.pdf

2023第二季度AIGC人才供需报告.pdf

计算机行业双周报2023年第3期总第43期：全面注册制催化金融IT行情 ChatGPT进一步打开AIGC商业化想象.pdf

2023-05-26_中银证券_微博-SW（09898）：广告投放仍谨慎，探索社交媒体平台AIGC.pdf

中国AIGC产业全景报告暨AIGC+50榜单.pdf

清华大学：从营销AIGC化到AIGC营销化.pdf

AIGC专栏（一）AIGC概述_watermark.pdf

AIGC+办公专题报告.pdf

奇点将至，探他山之石 ——从算力、算法、数据和应用看AIGC.pdf

2023中国AIGC文生图产业自皮书.pdf

【QuestMobile】2024上半年AIGC APP流量与场景研究报告.pdf

AIGC+教育行业报告（2024）PDF（57页）.pdf

最新资源

2023年【8页】视觉中国下游素材需求有望复苏，与百度战略合作加速AIGC落地.pdf