使用GitHub Copilot生成的代码性能评估.docx

Measuring the Performance of Code

Produced with GitHub Copilot

by

Daniel Erhabor

A thesis

presented to the University of Waterloo

in fulfillment of the

thesis requirement for the degree of

Master of Mathematics

I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis,

including any required final revisions, as accepted by my examiners.

I understand that my thesis may be made electronically available to the public.

Abstract

GitHub Copilot is an artificially intelligent programming assistant used by many devel-

opers. While a few studies have evaluated the security risks of using Copilot, there has not

been any study to show if it aids developers in producing code with better performance.

We evaluate the performance of code produced when developers use GitHub Copilot ver-

sus when they do not. To this end, we conducted a user study with 32 participants where

each participant solved two C++ programming problems, one with Copilot and the other

without it and measured the running time of the participants’ solutions on our test data.

Our results suggest that using Copilot can produce code with a significantly slower running

Acknowledgements

I want to thank my supervisors, Meiyappan Nagappan, Samer Al-Kiswany, my collab-

orator on this work, Sreeharsha Udayashankar, the participants for participating in the

study, members of WASL and SWAG research groups, and other people who advised on

things related to this work.

Dedication

I dedicate this thesis to my friends and family.