KENTUCKY
EARLY
MATHEMATICS
TESTING PROGRAM
2007
ANNUAL REPORT
Submitted
to the
Kentucky
Council on Postsecondary Education
and
the
Kentucky
Board of Education
From
the Directors
We have finished a rewarding and
productive seventh year as directors of the Kentucky Early Mathematics Testing
Program (KEMTP). We are extraordinarily pleased and gratified by the support
and cooperation we have received from so many talented people in the
mathematics community and the larger educational community throughout the
state. We would like to thank these people for all their support and hard work
in behalf of the KEMTP. We would especially like to thank the high school
mathematics teachers who took the time and effort to give the KEMTP test to
their students and to reinforce the message that the testing sends.
The KEMTP is using its resources
to add an entirely new dimension to its online testing. The Statewide
Mathematics Placement Testing Group, employing the human, financial
and technological resources of the KEMTP, has developed a secure,
online mathematics placement testing program that can be used free of charge by any participating
educational institution in the state. We would like to thank members of this
group for their enthusiastic participation in this exciting project. The
project is discussed in detail in this report.
We are indebted to so many
individuals who have strongly supported our efforts that it is impossible to
thank them all individually. A few, however, deserve our special thanks. Among
them are Paul Eakin and Ken Kubota, leaders of the UK group that develops and
supports online testing, Sue Cain from EKU, Diane Williams from NKU, and Linda
Mayhew and Linda Howard from ECTC for their leadership of the placement testing
project, our CPE liaison and ardent advocate Dianne Bazell, CPE vice president
Jim Applegate for his unwavering support of the placement testing project, our
legislative champion Rep. Jon Draud (the author and sole sponsor of the
legislation creating the KEMTP), and our talented assistant Mary Golden.
We would also like to thank
Northern Kentucky University and the University of Kentucky for their strong
institutional support of our work and the Council on Postsecondary Education
for funding the program.
Dan Curtin Steve
Newman
Northern Kentucky University Northern
Kentucky University
Department of Mathematics Department
of Mathematics
Highland Heights, KY 41099 Highland
Heights, KY 41099
(859) 572-6348 (859)
572-5332
July, 2007
Table of Contents
I. OVERVIEW
.. 4
II. STATEWIDE MATHEMATICS PLACEMENT TESTING
PROJECT
. 5
1.
A Brief History of the Project
.
..... 5
2. Secure Testing Process
.
..
. 5
3. Rationale for Placement Testing
...
5
4. Potential Uses for Placement Testing System
...
. 6
5. Related Statewide Initiatives
..
...
6
6. Important Information for High School Personnel
.. 7
III. KEMTP TESTS AND TESTING
OPTIONS
7
IV. TOOL BOX REVISITED
..
.. 8
V. PREDICTIVE VALUE OF THE
TEST
.
...
.... 8
VI. STATEWIDE TESING RESULTS..
10
1.
Feedback to Students and Teachers
. 10
2. Statewide Summary of Scores
.
. 11
3. Participating
High Schools and Counties
. 13
I. Overview
The Kentucky Early Mathematics Testing Program (KEMTP) was created when House Bill 178, written and sponsored by Rep. Jon Draud, was unanimously passed by the 2000 General Assembly. The program offers a voluntary, online mathematics test intended primarily for high school sophomores and juniors in Kentucky. The test provides students an assessment of their mathematical preparation for college early enough so that they can improve their mathematical preparation while still in high school and thereby avoid placement into remedial courses in college and increase their chances of success in college-level mathematics and science courses. The program is funded by the Kentucky Council on Postsecondary Education and is a partnership between Northern Kentucky University (NKU), which administers the program, and the University of Kentucky (UK), which develops and maintains the website. The program began testing in the spring of 2001 and has been testing every semester since then.
The bill requires that the public university that conducts the testing program shall submit an annual report to the Kentucky Board of Education and the Council on Postsecondary Education regarding its activities. This report is therefore submitted to these organizations by program directors Dan Curtin and Steve Newman from Northern Kentucky University, the public university selected to conduct the testing program.
Its a little difficult for us to believe that the KEMTP has completed its seventh year, with 6,032 Kentucky high school students from 41 high schools taking the KEMTP test last year.
In this report, we discuss the remarkably rapid development of a secure, statewide online placement testing system using the human, financial and technological resources of the KEMTP. The technology for the system was developed at the University of Kentucky (UK). A developmental mathematics test and the college algebra placement test were developed by a statewide placement testing group. These tests have been put online and have been used for placement since last spring by Eastern Kentucky University (EKU), Northern Kentucky University (NKU) and UK. Elizabethtown Community and Technical College (ECTC) began similar testing last summer. Morehead State University and Thomas More College will use the system in the fall. We hope other postsecondary institutions and adult education centers will join them in taking advantage of this system, offered free of charge to institutions and individuals.
We give some interesting data in a recent national report entitled The Toolbox Revisited showing that the highest level of mathematics taken in high school is the strongest indicator of eventual completion of a bachelors degree. We also give some interesting data collected at the University of Kentucky and Northern Kentucky University showing the effectiveness of the KEMTP test in predicting final grades in college mathematics courses.
We conclude with a statewide summary
of KEMTP testing data from last year, including the overall statewide average,
the statewide results by question, and the participating high schools and
counties.
II. Statewide
Mathematics Placement Testing Project
1. A Brief History of the Placement Testing Project
The KEMTP Advisory Board began preliminary discussions of online
placement testing at its March meeting at Centre College in 2006. (The KEMTP
Advisory Board consists of college and high school mathematics teachers as well
as representatives from the CPE and the Kentucky Department of Education). The
central issue discussed at this meeting was whether a secure online test was
needed to guarantee that the test taker was indeed the person he or she claimed
to be. The consensus was that an online placement test must be secure in this
sense so that accurate placement would be assured.
The Statewide Mathematics Placement Testing Group was formed following
this meeting. The group consists of mathematics faculty from all eight state
universities, including six department chairs; nine community and technical
colleges; and two private colleges as well as representatives from the Council
on Postsecondary Education (CPE) and adult education. The group developed,
discussed, modified and approved a developmental mathematics placement test and
a college algebra placement test. It was easy to reach consensus on these two
tests since they are based on remarkably similar institutional placement tests
that have been given for many years.
Ken Kubota and his colleagues in the mathematical sciences computing
group at UK designed the secure online system that supports the statewide
placement tests. This system is part of a much more comprehensive UK system
that supports the KEMTP, the mathematics and Spanish homework systems at UK,
and many other academic projects. The UK system is extraordinarily complex and
versatile, and yet remarkably easy to use. Kentucky is indeed fortunate to have
such superb system available to support its educational efforts.
2. The
Secure Testing Process
Placement tests are given in a computer lab. A testing coordinator with
an appropriate UK account sets up each testing session with the specified test,
session name, date, time and password. A proctor, who does not need a UK
account, sets up the computers using the session name and password, and checks
student identification. Students enter their names, last four digits of their
social security number, a testing number, and take the test. The format for the
test is identical to the KEMTP tests. When they finish the test, they submit
their answers, view their results, and are advised as to the recommended
mathematics course they should take. Alternatively, the testing coordinator can
download a spreadsheet of their results and an adviser can use the results on
the spreadsheet to advise them at a later date.
The system is constructed so that students could be tested at a high
school or college near their homes and the results could then be used for
placement purposes at the postsecondary institution of their choice.
3. Rationale for Placement Testing
Colleges offer a variety of developmental mathematics courses for
under-prepared students, generally consisting of prealgebra, elementary
algebra, and intermediate algebra. If students ACT math score is below 18 (19
beginning in 2009), they require remedial assistance. A developmental placement
test, consisting entirely of prealgebra and algebra questions, is more finely
grained than the much broader ACT and can provide more targeted placement
information, and thus more accurate placement, than using the ACT alone.
All colleges in Kentucky offer a college algebra course. This course is
a prerequisite for a bachelors degree in wide variety of college majors,
including majors in any business discipline. If students get an ACT math score
of 22 or more, then they are guaranteed placement in a college algebra course
at any public college or university in the state because of a statewide
placement policy enacted by the CPE in 2004. Students with lower scores than
required by the institution they attend can take a college algebra placement
test to see whether they have the algebra skills required to succeed in college
algebra. A calculus placement test serves the same purpose for a calculus
course.
4. Potential
Uses for Placement Testing System
There are a great many potential uses for the placement testing system.
High school seniors could be given college placement tests at their high
schools and the results could be shared electronically with the Kentucky
colleges of the students choice.
Students would not have to make a special trip to a college campus. Any
Kentucky college that students chose to attend would honor the placement test
results. Similar testing could be conducted for adult education students at any
one of the 120 adult education centers in the state.
Every high school and adult education center in the state is already on
the list of schools on the UK site. This would make implementation of remote
placement testing just as easy placement testing on a college campus.
Individual school districts, public and private schools, and adult education centers could design their own tests, put them online on the UK system, and use the secure placement testing portion of the UK site to administer these tests online. We hope that some school districts, public and private schools, and adult education centers will take advantage of this innovative testing method made possible by the UK system.
5. Related Statewide Initiatives
In the last year alone, there have been extraordinary efforts by leaders in education, business and government to improve education in general and mathematics education in particular. The Developmental Education Task Force and the Science Technology Engineering and Mathematics (STEM) Task Force have made sweeping recommendations for change. In response, the Council on Postsecondary Education (CPE) has proposed raised the bar for college readiness in mathematics from 18 to19 on the ACT mathematics test and from 18 to 21 on the ACT reading test beginning in 2009. If this proposal is adopted, then institutions will be required to give placement tests to students who do not meet college readiness levels in mathematics, English and reading.
The Developmental Education Task Force recommended that common placement tests be given statewide because such tests would permit institutions to share testing expertise and testing data, and thereby make more accurate placement decisions that will result in greater student success. The placement testing group has already shared testing expertise in the process of constructing a developmental mathematics placement test and a college algebra placement test. Construction of a calculus placement test is underway. The online testing system is ideally suited for sharing testing data among Kentuckys postsecondary institutions and adult education providers.
6. Important Information for High School Personnel
About Placement Testing
High school administrators, counselors and especially high school
mathematics teachers should be aware of college placement policies in
mathematics in order to advise their students. High school personnel should be
aware of the Postsecondary Placement Policy from the Council on Postsecondary
Education given in this newsletter. They should also be aware of the different
placement levels in mathematics and what courses at these levels are required
for different college majors. For example, students intending to major in any
business discipline at any public university in Kentucky must take a
mathematics course at the level of college algebra or above. These students,
even if they are not required to take a developmental algebra course to
graduate, often must take such a course in order to take college algebra and
thus earn a degree in a business discipline. Likewise, students intending to
major in a STEM discipline must take calculus. These students must often take
college algebra, precalculus, or both, in college even if they essentially took
these courses in high school.
The lesson to be learned is that students must prepare appropriately in their high school mathematics courses for the rigors of postsecondary education. Appropriate preparation allows students the option of pursuing a wide range of college majors without the need for re-taking high school courses in college. Appropriate preparation also provides students a smoother transition to postsecondary education and increases the chances that they will earn a college degree.
III. KEMTP Tests and Testing Options
There are a variety of KEMTP tests and testing options available to
teachers and students.
Ψ
Teachers can choose either an
Algebra I competency test for students who have not taken Algebra II or the
more advanced KEMTP test for students who have taken Algebra II and geometry.
There are online practice tests that students can use to prepare for each of
these tests.
Ψ
Teachers can give the test online in
the school computer laboratory to all students in a given class during one
class period.
Ψ
Teachers can ask their students to
take the test outside of class, either at home or at school during a convenient
time. It is easy for teachers to check their online roll books to determine
whether a student took the test. Some teachers offer extra credit for taking
the test and additional extra credit for doing well on the test.
Ψ
Teachers can give the test in the
traditional fashion. The KEMTP supplies all necessary bubble sheets and copies
of the test. Some schools use the traditional testing method to give the test
to large groups of students simultaneously.
KEMTP tests should prove to be a valuable resource in helping students prepare for the ACT assessment to be given to all public high school juniors in March of 2008. The KEMTP is carefully aligned with the ACT and is offered in the same multiple-choice format. Students planning to take college placement tests using the online system will find that these online placement tests are offered through the same website as the KEMTP in exactly the same format.
IV. Toolbox Revisited
Answers in the Tool Box, written by Clifford Adelman of
the U. S. Department of Education, was an influential study published in 1999
showing that high school curriculum intensity and quality had the strongest
influence on eventual attainment of a bachelors degree, stronger than
socioeconomic status, parents education, high school grades, and scores on
standardized tests. Within the curriculum intensity and quality category, the highest level of mathematics taken in
high school has the strongest relationship with ultimate completion of a
bachelors degree in any college major. This study is a major factor behind
the current emphasis on providing all students with a rigorous curriculum. In
particular, it was a major factor behind the decision by the Kentucky Board of
Education and the Kentucky Department of Education to make Algebra II a
graduation requirement.
The Toolbox Revisited is a follow up study published
in 2006 that confirms and reinforces the conclusions of the earlier study. The
earlier study followed a cohort that graduated from high school in 1982 and the
later study followed a cohort that graduated in 1992. Table 1 compares the
highest level of mathematics taken in high school with eventual attainment of a
bachelors degree for both these cohorts.
TABLE 1
Toolbox Data from 1982 and
1992
|
Level
of HS Mathematics Attained |
Percentage
Reaching this Level (1982) |
Percentage
Bachelors at this Level (1982) |
Percentage
Reaching this Level (1992) |
Percentage Bachelors
at this Level (1992) |
|
Calculus |
5.2 |
82.1 |
9.7 |
83.3 |
|
Precalculus |
4.8 |
75.9 |
10.8 |
74.6 |
|
Trigonometry |
9.3 |
64.7 |
12.1 |
60.0 |
|
Algebra II |
24.6 |
46.4 |
30.0 |
39.3 |
|
Geometry |
16.3 |
31.0 |
14.2 |
16.7 |
|
Algebra I |
21.8 |
13.4 |
16.5 |
7.0 |
|
Pre-algebra |
18.0 |
5.4 |
6.7 |
3.9 |
We hope this information will be useful to school administrators, counselors and teachers in their efforts to persuade qualified students to take more mathematics classes beyond the level of Algebra II, even if these students are not planning on a mathematics intensive major in college.
V. Predictive Value of Test
The University of Kentucky (UK) has been gathering data over the last several years that show the remarkable effectiveness of the KEMTP test in predicting the final grades in its college algebra, elementary calculus and calculus courses. Northern Kentucky University (NKU) collected similar data in its college algebra course with similar results. Altogether, 4,025 students have been involved.
The data show clearly how important it is for students to learn and to retain the essential algebra and geometry assessed on the KEMTP test. We hope this will encourage more teachers and school administrators to give the test to their students, and will encourage students to take a mathematics course every year of their high school career, especially their senior year, in order to prepare for the rigors of college mathematics.
The process for collecting data is straightforward. Students are given 12 or 15 randomly selected questions from the KEMTP test on the first or second day of class and their scores are compared with their final grades. Table 2 shows the combined results over two semesters of college algebra at UK. Note that of the 51 students who scored over 70% on the KEMTP test, 43% received a final grade of A, 65% received an A or B, and 78% received an A, B or C. The other rows are read in a similar fashion. Table 3 shows the combined results over two semesters of college algebra at NKU.
Spring 2003&Spring 2004 College Algebra at UK: 1119 Students
KEMTP Test Score
|
Percent
(A) |
Percent
(A,B) |
Percent
(A,B,C) |
Number
|
|
Greater than 70% |
43 |
65 |
78 |
51 |
|
Between 50% and 70% |
25 |
50 |
70 |
279 |
|
Between 30% and 49% |
15 |
36 |
56 |
411 |
|
Less than 30% |
6 |
20 |
42 |
378 |
TABLE 3
Fall 2003&Spring 2004 College Algebra at NKU: 327 Students
KEMTP Test Score
|
Percent
(A) |
Percent
(A,B) |
Percent
(A,B,C) |
Number
|
|
Greater than 70% |
33 |
59 |
74 |
27 |
|
Between 50% and 70% |
19 |
42 |
56 |
89 |
|
Between 30% and 49% |
6 |
14 |
29 |
133 |
|
Less than 30% |
3 |
10 |
27 |
78 |
Elementary calculus at UK is required of all business and economics majors. Calculus at UK is required of all mathematics, physics, chemistry, computer science and engineering majors. The KEMTP test was given to 2423 elementary calculus students over four semesters and to 156 calculus students in the spring of 2002. The results are shown in Table 4 and Table 5.
TABLE 4
Elementary Calculus at UK: 2423 Students over Four Semesters
KEMTP Test Score
|
Percent
(A) |
Percent
(A,B) |
Percent
(A,B,C) |
Number
|
|
Greater than 70% |
33 |
50 |
65 |
765 |
|
Between 50% and 70% |
12 |
26 |
43 |
1004 |
|
Less than 50% |
6 |
14 |
30 |
654 |
KEMTP Test Score
|
Percent
(A) |
Percent
(A,B) |
Percent
(A,B,C) |
Number
|
|
Greater than 80% |
23 |
54 |
82 |
56 |
|
Between 65% and 79% |
14 |
34 |
52 |
65 |
|
Less than 65% |
0 |
9 |
20 |
35 |
VI. Statewide Testing Results
1. Feedback to
Students and Teachers.
Feedback to students comes in the form of a letter, delivered either online immediately after test results are submitted if the test is taken online or by the testing program in hard copy form if the test is taken on paper. In either case, students are given their score on the test and an assessment of their mathematical preparation for college based on their score. This assessment is broken into four categories as described in Table 6.
TABLE 6
Advice Given Students Based on KEMTP Test Score
|
KEMTP
Test Score |
Advice
Given |
|
68% to 100% (17-25 correct answers) |
Well prepared for entry-level college mathematics
course |
|
52%-64% (13-16 correct answers) |
Possible need for a remedial course in
college |
|
36% to 48% (9-12 correct answers) |
Probable need for one, possibly two, remedial
courses |
|
0% to 32% (0-8 correct answers) |
Probable need for at least two remedial
courses |
In the same letter, students are urged to improve their mathematical preparation for college by working harder in their current mathematics course and by taking additional mathematics courses while still in high school, especially during their senior year. They are also given the mathematical topic involved in each question and whether they answered this question correctly. This diagnostic aspect of the test helps students determine their mathematical strengths and weaknesses.
The online tests, both the KEMTP test and the KEMTP Junior
test, give students additional information about which of the standards in the
Kentucky Core Content for Assessment in high school mathematics are covered by
each of the questions.
2. Statewide
Summary of Scores.
There were 3,601 Kentucky high school students who took the KEMTP test. This test is appropriate for those students who have completed Algebra I, geometry, and Algebra II. On the KEMTP website, this test is called the advanced exam. The number and percentage of students of students who scored in each of the four categories is shown in Table 7.
TABLE 7
|
Fall 2006 Spring 2007 |
|
|
|
KEMTP Test Score |
Number |
Percentage |
|
68% to 100% (17-25 correct answers) |
793 |
22 |
|
52% to 64% (13-16 correct answers) |
519 |
14.4 |
|
36% to 48% (9-12 correct answers) |
771 |
21.4 |
|
0% to 32% (0-8 correct answers) |
1518 |
42 |
There were 2,461 Kentucky high school students who took the Algebra I competency test, affectionately called KEMTP Junior. On the KEMTP website, this test is called the basic exam. The number and percentage of students of students who scored in each of the four categories is shown in Table 8.
TABLE 8
|
Fall 2006 Spring 2007 |
|
|
|
KEMTP Junior Test Score |
Number |
Percentage |
|
68% to 100% (17-25 correct answers) |
898 |
36.4 |
|
52% to 64% (13-16 correct answers) |
425 |
17.2 |
|
36% to 48% (9-12 correct answers) |
439 |
17.8 |
|
0% to 32% (0-8 correct answers) |
699 |
28.4 |
The 3,601 students who took the KEMTP test had an average score was 11.0. For each of the 25 questions on the test, the number and percentage of these students who answered that question correctly is given in Table 9.
KEMTP Test
Number of Students: 3601
State Average: 11.0
Question
|
Number of Students with Correct Answer |
Percent of Students with Correct Answer |
|
1.
Find slope of a line |
1654 |
45.9 |
|
2.
Solve linear inequality |
1478 |
41.0 |
|
3.
Find angle in isosceles triangle |
1543 |
42.8 |
|
4.
Factor a binomial expression |
1918 |
53.3 |
|
5.
Factor a quadratic expression |
1664 |
46.2 |
|
6.
Find volume of a sphere |
864 |
24.0 |
|
7.
Find equation of a parabola |
1798 |
49.9 |
|
8.
Subtract rational expressions |
1166 |
32.4 |
|
9.
Apply laws of exponents |
1441 |
40.0 |
|
10. Solve system of
linear equations |
2161 |
60.0 |
|
11. Add quadratic
polynomials |
1438 |
39.9 |
|
12. Evaluate a function |
2029 |
56.3 |
|
13. Find angle in a
triangle |
2136 |
59.3 |
|
14. Formulate linear
expression |
1666 |
46.3 |
|
15. Solve linear
equation |
2048 |
56.9 |
|
16. Simplify
rational expression |
1423 |
39.5 |
|
17. Solve
proportionality problem |
2049 |
56.9 |
|
18. Solve geometric word problem |
1756 |
48.8 |
|
19. Apply
Pythagorean theorem |
1509 |
41.9 |
|
20. Solve quadratic
equation |
1477 |
41.0 |
|
21. Solve
trigonometry problem |
910 |
25.3 |
|
22. Translate a
graph |
1206 |
33.5 |
|
23. Solve
probability problem |
1281 |
35.6 |
|
24. Solve similar
triangles problem |
1035 |
28.7 |
|
25. Find slope of a
line |
1863 |
51.7 |
3. Participating
High Schools and Counties.
The 41 Kentucky high schools that participated in the
KEMTP, together with the counties in which these schools are located, are
listed in Table 10.
TABLE 10
Participating Schools and Counties
|
|
School Name |
County |
|
1 |
Assumption High School |
Jefferson |
|
2 |
Breathitt County High School |
Breathitt |
|
3 |
Central Hardin High School |
Hardin |
|
4 |
Conner High School |
Boone |
|
5 |
Cumberland County |
Cumberland |
|
6 |
Dixie Heights High School |
Kenton |
|
7 |
Edmonson County High School |
Edmonson |
|
8 |
Fern Creek Traditional High School |
Jefferson |
|
9 |
Grayson County |
Grayson |
|
10 |
Hancock County High School |
Hancock |
|
11 |
Henry Clay High School |
Fayette |
|
12 |
Holy Cross High School |
Kenton |
|
13 |
Jackson City High School |
Breathitt |
|
14 |
Lafayette High School |
Fayette |
|
15 |
Lexington Christian Academy High School |
Fayette |
|
16 |
Louisville Male Traditional High School |
Jefferson |
|
17 |
Ludlow High School |
Kenton |
|
18 |
Mercer County High School |
Mercer |
|
19 |
Mercy Academy High School |
Jefferson |
|
20 |
Metcalf County High School |
Metcalf |
|
21 |
Monroe County High School |
Monroe |
|
22 |
Montgomery County High School |
Montgomery |
|
23 |
Muhlenburg North High School |
Muhlenburg |
|
24 |
Newport Central Catholic High School |
Campbell |
|
25 |
Oldham County High School |
Oldham |
|
26 |
Owen County High School |
Owen |
|
27 |
Owensboro Catholic High School |
Daviess |
|
28 |
Pendleton County High School |
Pendleton |
|
29 |
Prestonsburg County High School |
Prestonsburg |
|
30 |
Rowan County High School |
Rowan |
|
31 |
Ryle High School |
Boone |
|
32 |
Sacred Heart Academy High School |
Jefferson |
|
33 |
Saint Henry High School |
Kenton |
|
34 |
Scott High School |
Kenton |
|
35 |
Seneca High School |
Jefferson |
|
36 |
Sheldon Clark High School |
Martin |
|
37 |
Somerset High School |
Pulaski |
|
38 |
Southwestern Pulaski County High School |
Pulaski |
|
39 |
Spencer County High School |
Spencer |
|
40 |
Walton Verona High School |
Boone |
|
41 |
Western Hills Academy High School |
Franklin |