Java Core Assessment Framework
Suren Gishyan and Albert Sahakyan
Abstract—At a time when educational resources for masterin g Java are increasingly abundant [1][2][3], when Java is
considered one of the most popular programming languages [4] and when there are 20,000 [5] open jobs in the US (and
growing) that require familiarity with Java programming, the need for a well-researched framework dening how to assess
Java skill is greater than ever. This paper proposes such a framework with the aim of enabling direct measurement of
modern Java (Java 8 SE) development skill and helping the tec hn ic al recruitment industry go beyond resumes in assessing
this specic area of software development.
F
1Introduction
T
he status quo in today’s technical recruiting pro-
cess is to use resume as a proxy for skill, which
leads to biased, inecient recruiting and evaluation
practices. Some forward-looking companies are us-
ing automated assessment tools to create their own
tests, which they send to candidates as the rst step
of the interview process. However, there are two ma-
jor problems with that approach. Firstly, the internal
teams creating these assessments are not experts in
test design, which may lead to creation of tests that
are not EEOC-compliant, or focus too much on some
skills that might not be as important for the rol e while
neglecting others that could be critical. Secondly, after
internal teams spend a substantial amount of time
creating these assessments, many of these assessments
are eventually p osted online on sites like Glassdoor and
Stack Overow, making the validity of the test results
questionable.
The Java SCA (Speciali ze d Coding Assessment) frame-
work described in this paper can be used to create stan-
dardized tests to measure Java programming skills and
knowledge. The tests are based on research that ensur es
they are highly c onsistent and EEOC-compliant. In ad-
dition, basing the test on a framework allows the SCA to
be scaled with a large p ool of questions that adhere to
the same guidelines. On each test administration, ques-
tions can be randomly selected from the pool, reducing
the risk that a test-taker gains an unfair advantage by
memorizing the questions in advance.
The Coding Score obtained from the test is a singular
measure meant to give recruiters, hiring managers and
educators – as well as the test-takers themselves – a
quick view of the test-taker’s skills. It measures each
test taker’s code-writing and debugging skills as well
as knowledge of Java. The test should be administered
in a proctored env ironment, and the solutions should
be automatically checked for plagiarism.
In the next sec tion we will discuss which topics are cov-
ered and will draw guidelines for creating tasks based
on the framework. After war ds, we will dive deeper
into the reasoning of creating the framework and later
discuss the possibl e outcomes and scoring of the test.
More details on how Coding Score is calculated can
be found in the paper for General Coding Assessment
Framework [6].
2TheFramework
The maximum allowed completion time for the test
is 80 minutes. Each test has 4 implementation tasks
and 16 multiple choice quiz questions; in total, ther e
are 20 tasks. Each test contains an implementation
task (requires writing at least 25-40 lines of c ode,
but the description clearly explains what needs to be
done), a bugxtask(checkingdebuggingabilities),two
recovery tasks (meant to assess a deeper understanding
of the Java language by enforcing limitations on code
changes), and 16 quiz tasks grouped into 4 categories
by topics, with each topic containing 4 tasks.
2.1 Topics
The test will cover the following 10 topics of Java 8 SE.
1) Java Basics
Platform independence, main meth od and its argu-
ments, command line, import statement, primitive
剩余9页未读，继续阅读
资源评论
