ChatGPThasMasteredthePrinciplesofEconomics:NowWhat？资源-CSDN文库

自然语言处理

需积分: 1 44 浏览量 2023-04-20 07:39:05 上传评论收藏 591KB PDF 举报

资源推荐

资源详情

资源评论

ChatGPT has Mastered the Principles of Economics: Now

What?

Wayne Geerling

, G. Dirk Mateer

, Jadrian Wooten

and Nikhil Damodaran

Abstract

ChatGPT is a natural language processing model that can generate

conversational style responses to user inputs. It also performs extremely

well on a popular standardized assessment used in principles of

economics courses around the country. ChatGPT ranked in the 91st

percentile for Microeconomics and the 99th percentile for

Macroeconomics when compared to students who take the TUCE exam at

the end of their principles course. The emergence of artificial intelligence

presents a significant challenge to traditional assessment methods in

higher education. It is not possible to evaluate students’ intellectual ability

through artificial intelligence, as it only reflects their ability to access

information. An important implication of this finding is that educators will

likely need to redesign their curriculum in at least one of the following

three ways: reintroduce proctored, in-person assessments, augment

learning with chatbots, and/or increase the prevalence of experiential

learning projects that artificial intelligence struggles to replicate well.

JEL Codes: A22, I21

Keywords: TUCE, ChatGPT, assessment, artificial intelligence, academic integrity

Corresponding author: Professor of Instruction, Department of Economics, Room 2.102 B, University of

Texas at Austin. Email: wayne.geerling@austin.utexas.edu

Professor of Instruction, Department of Economics, Room 3.102 B, University of Texas at Austin. Email:

dmateer@utexas.edu

Collegiate Associate Professor of Economics, Department of Economics, Pamplin 3034, Virginia

Polytechnic Institute and State University. Email: jadrian@vt.edu

Lecturer, Department of Economics, Cabin 7, T4 Academic Block, OP Jindal Global University. Email:

ndamodaran@jgu.edu.in

Electronic copy available at: https://ssrn.com/abstract=4356034

I. Introduction

On November 30, 2022, OpenAI launched ChatGPT (Generative Pre-trained

Transformer), a chatbot that quickly gained attention for—among other things—its

potential to disrupt traditional assessment methods. ChatGPT allows the user to enter a

prompt and receive a unique, detailed response on a wide range of knowledge

domains. The chatbot has already demonstrated its ability to pass a medical licensing

exam (Gilson et al. 2023), the law school admissions test (Choi et al. 2023), and a

common assessment used in introductory physics courses (West, 2023). Can

ChatGPT’s responses surpass human knowledge among undergraduate students

studying economics?

To evaluate this, we use the Test of Understanding in College Economics (TUCE),

published by the National Council on Economic Education (NCEE) and in use across

the United States for more than 50 years. It is one of the most widely used assessment

tools for basic economic knowledge and consists of two versions: one covering

microeconomic concepts and one covering macroeconomic concepts. Each version of

the test has 30 multiple-choice questions with four answer choices each. Both versions

include three questions covering international economics, but the questions are unique

to each version. The TUCE is a norm-referenced measure that can be used to compare

students’ knowledge levels across a wide range of abilities. A score of around 50% is

desirable for research purposes, as it provides appropriate levels of item discrimination

and test reliability. A score of less than 50% does not necessarily indicate a failing level

of knowledge in a course, as instructors may prioritize different concepts from those

tested in the TUCE. By using the TUCE as both a pre- and post-test assessment,

educators can measure perceived learning during the semester (Smith and Wagner,

2018).

ChatGPT operates using algorithms that process data, allowing it to string words

together in response to a prompt. Unlike humans, ChatGPT has access to vast troves of

information available on the internet and uses large language modeling to recognize

patterns in the words in each prompt to mimic human writing when dispensing

knowledge.

While ChatGPT is a powerful tool, its abilities are limited to the pool of

information it has been trained on. ChatGPT creates responses to user prompts using a

transformer-based neural network architecture based on the training data to generate

contextually appropriate and coherent responses. ChatGPT doesn’t actually “know”

anything, but instead generates responses based on probabilities assigned to each

word in the vocabulary, which are calculated through a process of iterative training on a

large corpus of text. In this paper, we assess ChatGPT’s performance on the

microeconomics and macroeconomics versions of the TUCE and compare it to the

results of college students.

While not the focus of this paper, we would be remiss if we did not mention the potential

mischief ChatGPT will cause in the short term. To understand the potential impact of

ChatGPT on academic integrity, it is important to acknowledge that cheating is not a

The current version of ChatGPT was modeled on data from the internet and other sources, up until the end of

2021.

Electronic copy available at: https://ssrn.com/abstract=4356034

new issue, and ChatGPT is simply the latest tool that can be used for a variety of

purposes, ethical considerations aside. Instructors have always been asked to consider

how to effectively assess student learning, but now must contend with ChatGPT’s

capabilities. Instructors are the guardians of the learning environment and chatbots are

“smarter” than most students. The first step instructors should take is to minimize

cheating. Emphasizing proctored assessments is a good way to ensure that students

earn scores that reflect their understanding while reducing academic integrity issues.

Second, and much more importantly in the long run, instructors should develop ways to

utilize chatbots to augment learning. The second step is evolving as we write and is

briefly touched upon in a later section. Economic educators must embrace this

technological disruptor in order to better prepare our students for the jobs of the future.

One way to accomplish both goals is to emphasize experiential learning and authentic

learning projects in which students create their own content. When learning is

meaningful and relevant, students have less incentive to cheat.

In the following sections, we briefly review the literature on the role of chatbots in

education and then compare ChatGPT’s performance on the TUCE with the results

achieved by college students after completing a semester of their principles course. We

conclude by offering some practical advice on identifying alternative assessments that

complement ChatGPT as a learning tool.

II. The Role of Chatbots in Education

Chatbots are a technology application that promote interpersonal communication and

learning. They provide information and knowledge through interactive methods and

easy-to-operate interfaces (Hwang & Chang, 2021). With the exponential growth in the

mobile device market over the past decade, the popularity of chatbots is being driven by

their ability to provide an interactive medium through which to learn, one not constrained

by time and place (Zhou et al., 2020). A Georgia Tech computer science professor

made headlines in 2016 for using artificial intelligence to build a virtual teaching

assistant (Goel & Polepeddi, 2018). The chatbot known as “Jill” received very positive

student evaluations, and students only seemed to suspect something was amiss when

their teaching assistant responded quickly at all hours of the day.

Interaction with technologies, either by natural language or speech, is possible because

as technology develops, users become more used to interacting with digital entities.

Chatbots are now used across a wide range of domains, including marketing, customer

service, technical support, education and training (Smutny & Schreiberova, 2020).

Personal digital assistants like Siri (Apple), Alexa (Amazon), Cortana (Microsoft) and

Google Assistant (Google) lie at the forefront of technology in voice recognition and

‘artificial intelligence’ and have effectively replaced much of the day-to-day tasks once

performed by assistants or secretaries (Smutny & Schreiberova, 2020). The use of

digital technologies is now expected by the current generation of young people who

were born into an era of the internet and smartphones (Selwyn, 2021).

https://theconversation.com/motivation-is-a-key-factor-in-whether-students-cheat-155274

Electronic copy available at: https://ssrn.com/abstract=4356034

剩余14页未读，继续阅读

评论收藏

内容反馈

前端筱悦

粉丝: 352
资源: 622

ChatGPT has Mastered the Principles of Economics: Now What？

最新资源

ChatGPT has Mastered the Principles of Economics: Now What？

re1Students should have mastered the following

hash table spell checking

The.Complete.Guide.for.CPP.Examination.Preparation.2nd.Edition.1498705227

A Practical Guide to Designing with Data.pdf

ShaderX 1.-.Vertex.and.Pixel.Shader.Tips.and.Tricks

Mastering Internet of Things-Packt Publishing(2018).epub

User Experience Re-Mastered: Your Guide to Getting the Right Design

Practical Predictive Analytics

Mastering PostCSS for Web Design

3D Animation for the Raw Beginner Using Autodesk Maya, 2nd Edition.pdf

Learning.Internet.of.Things

ERP数据库设计方法、规范、技巧.docx

Automate.the.Boring.Stuff.with.Python.Practical.Programming

Sybex - Mastering Visual Basic .NET (VBL).pdf

数据结构作业Hash表

leetcode文本分词-RICKFENG-DSA:Leetcode/学习笔记

Beginning ASP.NET 4 in VB 2010

Beginning ASP.NET 4 in C# 2010

stable-diffusion部署需要的包

大规模语言模型：从理论到实践

21个免费无限制免登录chatgpt资源， OpenAI GPT-4\3.5 模型的智能对话链接

人工智能大模型介绍.pptx

ChatGPT智能AI机器人微信小程序源码-带部署教程

llama3-中文微调训练集，让llama3更懂中文

diabetes糖尿病数据集

LM Studio windows版本安装

transformer代码

线性代数-同济大学第七版

最新资源