Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Problems of Student Average Score
Different Scaling
Interval Property: does linear growth in scores imply
linear growth in knowledge?
Singular Dimension
Ignores large information in item response data
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Problems of Student Average Score
Different Scaling
Interval Property: does linear growth in scores imply
linear growth in knowledge?
Singular Dimension
Ignores large information in item response data
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Problems of Student Average Score
Different Scaling
Interval Property: does linear growth in scores imply
linear growth in knowledge?
Singular Dimension
Ignores large information in item response data
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Problems of Student Average Score
Different Scaling
Interval Property: does linear growth in scores imply
linear growth in knowledge?
Singular Dimension
Ignores large information in item response data
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Motivation 1: Item Discrepancy
Suppose James and Jiguang both score 80% in a 50-question
math exam. Do James and Jiguang have the same math
abiliity?
What if we know James answered most of the difficult
questions correctly, whereas Jiguang got most of the easy
questions correctly?
Assign more scores to more difficult questions?
Different feedbacks?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Motivation 1: Item Discrepancy
Suppose James and Jiguang both score 80% in a 50-question
math exam. Do James and Jiguang have the same math
abiliity?
What if we know James answered most of the difficult
questions correctly, whereas Jiguang got most of the easy
questions correctly?
Assign more scores to more difficult questions?
Different feedbacks?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Motivation 1: Item Discrepancy
Suppose James and Jiguang both score 80% in a 50-question
math exam. Do James and Jiguang have the same math
abiliity?
What if we know James answered most of the difficult
questions correctly, whereas Jiguang got most of the easy
questions correctly?
Assign more scores to more difficult questions?
Different feedbacks?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Motivation 1: Item Discrepancy
Suppose James and Jiguang both score 80% in a 50-question
math exam. Do James and Jiguang have the same math
abiliity?
What if we know James answered most of the difficult
questions correctly, whereas Jiguang got most of the easy
questions correctly?
Assign more scores to more difficult questions?
Different feedbacks?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Theory Intuition
Question: What latent variables are affecting the probability of
student i answering a question j correctly?
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
U
ij
: whether student i can answer question j correctly
Figure: IRT Graphical Model
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Theory Intuition
Question: What latent variables are affecting the probability of
student i answering a question j correctly?
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
U
ij
: whether student i can answer question j correctly
Figure: IRT Graphical Model
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Theory Intuition
Question: What latent variables are affecting the probability of
student i answering a question j correctly?
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
U
ij
: whether student i can answer question j correctly
Figure: IRT Graphical Model
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Theory Intuition
Question: What latent variables are affecting the probability of
student i answering a question j correctly?
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
U
ij
: whether student i can answer question j correctly
Figure: IRT Graphical Model
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Theory Intuition
Question: What latent variables are affecting the probability of
student i answering a question j correctly?
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
U
ij
: whether student i can answer question j correctly
Figure: IRT Graphical Model
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Standard IRT Model
Birnbaum (1968) proposed a Two-Parameter Logistic IRT
Model:
P(U
ij
= 1|θ
i
, d
j
, α
j
) =
e
α
j
(θ
i
−d
j
)
1 + e
α
j
(θ
i
−d
j
)
U
ij
: whether student i can answer question j correctly.
Student i’s latent ability: θ
i
∈
R
Difficulty of item j: d
j
∈
R
Discrimination power of item j: α
j
∈
R
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Data Overview
We used the item response dataset provided by Massachusetts
Department of Elementary and Secondary Education (DESE),
which can link roughly 70,000 students over grades 6, 7 and 8.
Randomly sample 4,000 students for training, and
additional 15,000 students for testing.
Consider only binary items.
Table: Number of Math and English Items in Each Grade
Grade Math Items English Items
6 24 18
7 34 14
8 40 -
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Item Response Data Overview
We used the item response dataset provided by Massachusetts
Department of Elementary and Secondary Education (DESE),
which can link roughly 70,000 students over grades 6, 7 and 8.
Randomly sample 4,000 students for training, and
additional 15,000 students for testing.
Consider only binary items.
Table: Number of Math and English Items in Each Grade
Grade Math Items English Items
6 24 18
7 34 14
8 40 -
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experimentation Setup
Given students math item response data in grade 7, how do we
predict students’ math test performance in grade 8?
Baseline Approach: rank students by students’ 7th grade
average score.
Our Approach: rank students by 7th grade average
probabilities output by the IRT Model.
If our approach is more predictive of a student’s math
performance in future, then it suggests there exists
valuable information contained in the item response data.
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experimentation Setup
Given students math item response data in grade 7, how do we
predict students’ math test performance in grade 8?
Baseline Approach: rank students by students’ 7th grade
average score.
Our Approach: rank students by 7th grade average
probabilities output by the IRT Model.
If our approach is more predictive of a student’s math
performance in future, then it suggests there exists
valuable information contained in the item response data.
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experimentation Setup
Given students math item response data in grade 7, how do we
predict students’ math test performance in grade 8?
Baseline Approach: rank students by students’ 7th grade
average score.
Our Approach: rank students by 7th grade average
probabilities output by the IRT Model.
If our approach is more predictive of a student’s math
performance in future, then it suggests there exists
valuable information contained in the item response data.
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 1: IRT Result
We compare the rank-order correlation with 8th grade average
score using 7th grade average score (baseline) and IRT 7th
grade prediction.
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 1: IRT Result
We compare the rank-order correlation with 8th grade average
score using 7th grade average score (baseline) and IRT 7th
grade prediction.
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
What if Single Latent Factor is not Enough?
For many math items, solving them requires more than one
type of latent abilities.
Figure: A Geometric Problem also Testing Algebra
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Multidimensional Item Response Theory
We can easily extend IRT model to multidimensional. Suppose
now we have m = 2 factors: Geometry and Algebra.
P(U
ij
= 1|Θ
i
, d
j
, α
j
) =
e
α
T
j
(Θ
i
−d
j
)
1 + e
α
T
j
(Θ
i
−d
j
)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Multidimensional Item Response Theory
We can easily extend IRT model to multidimensional. Suppose
now we have m = 2 factors: Geometry and Algebra.
P(U
ij
= 1|Θ
i
, d
j
, α
j
) =
e
α
T
j
(Θ
i
−d
j
)
1 + e
α
T
j
(Θ
i
−d
j
)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 2: MIRT Result
We compare the rank-order correlation with 8th grade average
score using 7th grade average score (baseline), and IRT and
MIRT 7th grade prediction.
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT2 0.87877 (0.87477, 0.88256)
MIRT3 0.88120 (0.87727, 0.88491)
MIRT4 0.87522 (0.87108, 0.87908)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 2: MIRT Result
We compare the rank-order correlation with 8th grade average
score using 7th grade average score (baseline), and IRT and
MIRT 7th grade prediction.
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT2 0.87877 (0.87477, 0.88256)
MIRT3 0.88120 (0.87727, 0.88491)
MIRT4 0.87522 (0.87108, 0.87908)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Math Knowledge is Non-Compensatory
Is math test Item compensatory (i.e. can students’ strength in
one math area compensate their weak spot in another one)?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Math Knowledge is Non-Compensatory
Is math test Item compensatory (i.e. can students’ strength in
one math area compensate their weak spot in another one)?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
MIRT V.S. Partially Compensatory MIRT Model
(PMIRT)
Observe the compensatory nature of the MIRT equation:
P(U
ij
= 1|Θ
i
, d
j
, α
j
) =
e
α
T
j
(Θ
i
−d
j
)
1 + e
α
T
j
(Θ
i
−d
j
)
(1)
=
exp(
P
m
α
jm
θ
im
+ const)
1 + exp
(
P
m
α
jm
θ
im
+ const)
(2)
PMIRT Equation:
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
MIRT V.S. Partially Compensatory MIRT Model
(PMIRT)
Observe the compensatory nature of the MIRT equation:
P(U
ij
= 1|Θ
i
, d
j
, α
j
) =
e
α
T
j
(Θ
i
−d
j
)
1 + e
α
T
j
(Θ
i
−d
j
)
(1)
=
exp(
P
m
α
jm
θ
im
+ const)
1 + exp
(
P
m
α
jm
θ
im
+ const)
(2)
PMIRT Equation:
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 3: PMIRT Result
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT2 0.87877 (0.87477, 0.88256)
MIRT3 0.88120 (0.87727, 0.88491)
MIRT4 0.87522 (0.87108, 0.87908)
PMIRT2 0.88143 (0.87744, 0.88520)
PMIRT3 0.88206 (0.87807, 0.88581)
PMIRT4 0.88203 (0.87806, 0.88578)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Hierarchical Nature of Mathematics
”Mathematics, as it’s generally taught in American school
systems, can be a saintly mother of a subject. It climbs
logically and majestically from addition through subtraction,
multiplication, and division. Then it sweeps up toward the
heavens of mathematical beauty. But math can also be a
wicked stepmother. She is utterly unforgiving if you happen to
miss any step of the logical sequence—and missing a step is
easy to do. ”
- Barbara Oakley, How to Excel at Math and Science
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 4: Hierarchical PMIRT Result
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Correlation 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT2 0.87877 (0.87477, 0.88256)
MIRT3 0.88120 (0.87727, 0.88491)
MIRT4 0.87522 (0.87108, 0.87908)
PMIRT2 0.88143 (0.87744, 0.88520)
PMIRT3 0.88206 (0.87807, 0.88581)
PMIRT4 0.88203 (0.87806, 0.88578)
treemodel3 0.88124 (0.87724, 0.88498)
treemodel4 0.88195 (0.87796, 0.88570)
binarytree3 0.88129 (0.87729, 0.88504)
binarytree7 0.88190 (0.87789, 0.88565)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Adding English Items to PMIRT Model
Math items are
allowed to load on
English factor.
English items are not
allowed to load on
Math factor.
We still only use this
model to generate
probabilities of
answering 7th grade
math items correctly.
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 5: Power of English Items
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Corr 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT3 0.88120 (0.87727, 0.88491)
PMIRT3 0.88206 (0.87807, 0.88581)
treemodel4 0.88195 (0.87796, 0.88570)
treemodel3(efixed) 0.88526 (0.88139, 0.88891)
treemodel4(efixed) 0.88512 (0.88125, 0.88876)
treemodel5(efixed) 0.88515 (0.88128, 0.88880)
PMIRT4(efixed) 0.88523 (0.88136, 0.88887)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Adding Math Items from the Past: A Simple
Longitudinal Model
Recall we haven’t taken advantage of the 6th grade math item
response data yet. To include previous items, consider a linear
growth in each dimension of latent ability over years:
Let student i’s ability m at time t be θ
imt
Let k
mt
and b
mt
be the slope and intercept repectively.
θ
im(t+1)
= k
mt
θ
imt
+ b
mt
+
imt
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Adding Math Items from the Past: A Simple
Longitudinal Model
Recall we haven’t taken advantage of the 6th grade math item
response data yet. To include previous items, consider a linear
growth in each dimension of latent ability over years:
Let student i’s ability m at time t be θ
imt
Let k
mt
and b
mt
be the slope and intercept repectively.
θ
im(t+1)
= k
mt
θ
imt
+ b
mt
+
imt
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Adding Math Items from the Past: A Simple
Longitudinal Model
Recall we haven’t taken advantage of the 6th grade math item
response data yet. To include previous items, consider a linear
growth in each dimension of latent ability over years:
Let student i’s ability m at time t be θ
imt
Let k
mt
and b
mt
be the slope and intercept repectively.
θ
im(t+1)
= k
mt
θ
imt
+ b
mt
+
imt
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Experiment 6: Previous Math Items Improve
Prediction Power
Table: Rank-order Correlation Comparison in the Test Set
Model Rank-Order Corr 95%CI
Baseline 0.87672 (0.87266, 0.88058)
IRT 0.88040 (0.87637, 0.88416)
MIRT3 0.88120 (0.87727, 0.88491)
PMIRT3 0.88206 (0.87807, 0.88581)
treemodel4 0.88195 (0.87796, 0.88570)
treemodel3(efixed) 0.88526 (0.88139, 0.88891)
lpmirt2 0.89376 (0.89017, 0.89715)
lpmirt3 0.89387 (0.89027, 0.89725)
lpmirt4 0.89363 (0.89004, 0.89700)
ltreemodel3 0.89369 (0.89010, 0.89709)
ltreemodell4 0.89361 (0.89001, 0.89699)
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Next Steps
Requesting more data from the Texas Education Research
Center.
Predict students’ long-term outcomes such as employment
status.
Modeling: experimenting with various priors, finding more
efficient way to determine latent structure.
Application: how can we propose better education policy
based on our findings?
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
Knowledge Cluster Details
Compute the average
of each cluster to pro-
vide better targeting
remedial assistance?
Data: 2018 grade
10 Mathematics test
in Massachusetts
G: Geometry
A: Algebra
P: Probability
N: Number of
Quantities
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
A Simple Hierarchical PMIRT Model
For each test item j:
Generate a Beta(s, t)
variable for each factor:
c
1
, c
2
, and c
3
Define c
0
j
=
Q
ji
c
i
c
0
1
= c
1
c
0
2
= c
1
· c
2
c
0
3
= c
3
Probability equation is
unchanged :
P(U
ij
= 1|Θ
i
, d
i
, α
i
, c
j
) =
m
Y
l=1
(
e
α
jl
(θ
il
−d
jl
)
1 + e
α
jl
(θ
il
−d
jl
)
)
c
0
jl
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
A Simple Hierarchical PMIRT Model
For each test item j:
Generate a Beta(s, t)
variable for each factor:
c
1
, c
2
, and c
3
Define c
0
j
=
Q
ji
c
i
c
0
1
= c
1
c
0
2
= c
1
· c
2
c
0
3
= c
3
Probability equation is
unchanged :
P(U
ij
= 1|Θ
i
, d
i
, α
i
, c
j
) =
m
Y
l=1
(
e
α
jl
(θ
il
−d
jl
)
1 + e
α
jl
(θ
il
−d
jl
)
)
c
0
jl
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
A Simple Hierarchical PMIRT Model
For each test item j:
Generate a Beta(s, t)
variable for each factor:
c
1
, c
2
, and c
3
Define c
0
j
=
Q
ji
c
i
c
0
1
= c
1
c
0
2
= c
1
· c
2
c
0
3
= c
3
Probability equation is
unchanged :
P(U
ij
= 1|Θ
i
, d
i
, α
i
, c
j
) =
m
Y
l=1
(
e
α
jl
(θ
il
−d
jl
)
1 + e
α
jl
(θ
il
−d
jl
)
)
c
0
jl
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
A Simple Hierarchical PMIRT Model
For each test item j:
Generate a Beta(s, t)
variable for each factor:
c
1
, c
2
, and c
3
Define c
0
j
=
Q
ji
c
i
c
0
1
= c
1
c
0
2
= c
1
· c
2
c
0
3
= c
3
Probability equation is
unchanged :
P(U
ij
= 1|Θ
i
, d
i
, α
i
, c
j
) =
m
Y
l=1
(
e
α
jl
(θ
il
−d
jl
)
1 + e
α
jl
(θ
il
−d
jl
)
)
c
0
jl
Better
Understand
Student
Learning
Jiguang Li
Motivations
Item Response
Theory
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 5
Experiment 6
Conclusions
Appendix
A Simple Hierarchical PMIRT Model
For each test item j:
Generate a Beta(s, t)
variable for each factor:
c
1
, c
2
, and c
3
Define c
0
j
=
Q
ji
c
i
c
0
1
= c
1
c
0
2
= c
1
· c
2
c
0
3
= c
3
Probability equation is
unchanged :
P(U
ij
= 1|Θ
i
, d
i
, α
i
, c
j
) =
m
Y
l=1
(
e
α
jl
(θ
il
−d
jl
)
1 + e
α
jl
(θ
il
−d
jl
)
)
c
0
jl
