December 1, 2003

Introduction. This proposal details an assessment mechanism for the undergraduate degree programs in mathematics at The University of Toledo. The Department of Mathematics currently offers B.S. and B.A. degree programs with specializations in pure mathematics, applied mathematics, actuarial science, statistics, mathematics and computer science, and in addition a B.A. program in mathematics for adolescent and young adult (AYA) educators. Currently, the department has about 130 majors that populate the pure, applied, statistics, and education specializations. Because these four specializations contain almost all of the department’s students, the current proposal will only involve assessment of these programs.

The objective of this plan is to provide the department with data relating to the degree of understanding of mathematics gained by our majors during their course of study. The plan is designed in accordance with the principles set forth by the Mathematical Association of America’s Committee on the Undergraduate Program in Mathematics (CUPM). The CUPM guidelines articulated in 1995 describes assessment procedures as consisting of four elements: (1) setting the learning goals and objectives, (2) determination of areas and methods of assessment, (3) establishing methods for gathering assessment data, and (4) using assessment results to improve the mathematics major.

In the department’s view, the most important element, and one that measures the value of all preceding steps, is whether assessment data is used to improve education in the major as prescribed by the programs’ educational goals. It is for this reason that this plan focuses the area of assessment on measuring the value of course-work in training students to become proficient in their disciplines. In order to ensure that assessment plays a central role in curricular discussions, the plan includes a two-tiered mechanism for data gathering. At the first level, data will be presented in accessible database format and will measure in certain key courses the degree of success students have in satisfying the program goals. This data will measure the performance of each major in achieving the educational goals associated with a course and will be derived by establishing a time-dependent correspondence between the grade in the course and the level of achievement. Such a measure will allow us to track student performance through the length of our program and pinpoint subjects where preparation can be improved. Assessment data gathered in this way will be supplemented with student portfolios that will consist of the final exams in the key courses from which performance measures are taken. The portfolios will allow us to obtain detailed longitudinal information concerning deficiencies indicated by the performance data.

Some words of explanation may be useful to address the question as to why in mathematics assessment of course outcomes is sufficient to determine the overall quality of the undergraduate major. Although capstone courses do exist in mathematics, and ETS does provide a Major Field Test in mathematics, many departments with recognized assessment procedures, such as those at the Colorado School of Mines and North Dakota State University, center their assessment activities on course outcomes. The reason for this is that in mathematics the curriculum of the undergraduate major does not have a wide variety of areas in which the student needs expertise and which the student needs to integrate into a general appreciation of the major. On the contrary, undergraduate training in the mathematics prepares students for a small number of terminal courses that contain the principles and techniques that undergraduate majors can be expected to know. It is also true that this body of knowledge is quite stable in time in that it is only infrequently updated with new information or points of view. Thus, in assessing the success of our undergraduate program it is critically important to focus on the success of our students in these terminal courses, and correspondingly to examine our success in preparing students for this course-work.

Student Outcomes.

Mission: The learning goals and objectives elaborated here follow naturally from the department’s mission as resources for education and research in the mathematical sciences. The mission is delineated in the following statement:

The mission of the Department of Mathematics is to transmit mathematical ideas through teaching and teaching related activities; contribute to the advancement of mathematics through quality research; utilize the departments’ resources to aid the University and local communities in the applications of mathematics and statistics; and to serve as a resource of mathematical knowledge and pedagogy for the University and the local community.

Educational Objectives: In this spirit, the course work in mathematics associated with the degree programs in Pure Mathematics, Applied Mathematics, Statistics, and Mathematics for AYA Education, should prepare students, at the completion of their degree program, to satisfy the following general expectations.

(1) Students should learn single and multi-variable calculus that is reflected by the ability to solve detailed problems, and knowledge of how calculus is applied in other areas of mathematics such as differential or integral equations, differential geometry, probability, and numerical analysis.

(2) Students should learn the principles of linear algebra, in particular, the theory of linear equations, and vector spaces. The students’ knowledge of the subject should include an appreciation of the role that linear algebra plays in calculus and in the student’s area of specialization.

(3) Students should learn the formal structure of mathematics. This includes the role that axioms and definitions play in describing mathematical objects. Students should develop the ability to construct mathematical hypotheses, and the ability to prove mathematical statements.

(4) Students should have the capacity to read and comprehend mathematical literature appropriate to their specialization.

In addition, in each specialization and upon completion of the program, students will be expected to achieve the following goals:

(5) Pure Mathematics: Students in pure mathematics will be expected to grasp the basic principles of real analysis, topology, and algebra. In topology and analysis, this means that students will be familiar with the structure of real numbers, metric spaces, and the theory of functions of one variable. In algebra, this implies an understanding of the principles of group theory, and the structure of rings and fields.

(6) Applied Mathematics: Students in applied mathematics will be expected to be familiar with the mathematical structures most commonly used in solving practical problems. This includes the uses of linear algebra in solving discrete or combinatorial applied problems, and the use of differential equations in modeling. Students should have the skills required to implement algorithmic solutions to applied problems.

(7) Statistics: Students of statistics will be expected to learn the foundations and applications of probability for both discrete and continuous random variables and vectors. They will also be expected to grasp the reasoning behind, applications of, and properties of basic statistical methods. This includes methods for point and interval estimation and testing statistical hypotheses such as likelihood-based methods and methods related to the set of basic statistical distributions.

(8) Mathematics for AYA Mathematics Education: Students in AYA mathematics will be expected to learn geometry and of the historical development of mathematics. In geometry, students should be familiar with the effect of axioms on various geometries, express geometric properties in analytical terms, explore and conjecture about geometric properties using technology, and prove or disprove such conjectures. In the history of mathematics, students should be able to site major milestones in the development of mathematics, be aware of the persons involved in such milestones, and describe the historical climate that lead to such milestones.

Types of Assessment. The following approach to assessing the success of our students in achieving the above stated goals relies on the fact that the knowledge to be acquired is developed in a sequence of courses, each a prerequisite course to the following course and each often reexamining the same material in a deeper level of abstraction. Thus, for each of the goals stated above there is a point in the program, depending on the specialization, where success in a specific pivotal course indicates that the knowledge and skills associated with the goal have been acquired. The method to be introduced involves two tiers of data collection. The first tier, that will allow us to track students through our program and to gain general information as to their success in learning and our success at educating, will be to set up a relation for each section of each pivotal course between the grades in these courses and the program goals associated with that course. This relationship will be established jointly by the instructor and the Math Majors Committee and will be based on the content of the section’s final exam (which are typically cumulative; we will require that they be cumulative for the courses in question). The advantage of this format is that it will quickly generate a large volume of readily analyzable data. To refine our understanding of this data, the department will establish a portfolio archive that will consist of the students’ final exams in the pivotal courses for each of our majors. To implement this method, the department has identified a number of pivotal courses where the grades and final exams will serve as an indicator of the extent to which a student has met one or more of the above goals. The tables below list the pivotal required courses for each program and corresponding expectations that the grade in the course and final exam would measure.

Methods for gathering assessment data: The relationship between the grades received in the above courses and the degree to which the students have satisfied the above mentioned goals will be established by the following procedure. First, for each of the courses above, the department’s Undergraduate Curriculum Committee will formulate a description of how the course material relates realizing the associate goal. Then each semester, based upon this outline, instructors of sections of these courses will prepare a recommendation indicating the grade levels that correspond to the following three categories: 1) the student exceeds expectations, 2) meets expectations, or 3) fails to meet expectations. This recommendation will be submitted to the Math Majors Committee together with a copy of the final exam in the course at the beginning of the following semester. The committee will review these recommendations and certify their suitability. If there is disagreement between the instructor and the committee on the correspondence between grades and attainment of goals, the committee will consider carefully the instructor’s explanations, but, ultimately, the committee has authority to fix the correspondence. Once the correspondence has been established the data will be collected using an SQL query to the Data Warehouse. In addition, the committee will add to the student portfolio archive copies of the students’ final exams in each of the pivotal courses in the students’ specializations. The final exams will provide detailed information so that topics that caused failure to meet expectations can be identified. They also will provide assessment personnel with data with which to reference to the expectation levels. It should be mentioned that in upper division undergraduate mathematics courses there is considerable uniformity over time and between institutions concerning the essential material that must be included in these courses. Further, instructors have universally used final exams to determine whether students have learned these basic techniques and concepts. Consequently, due to these historical facts, final exams in upper division math courses provide uniform detailed information about the level of learning.

Using assessment data: Each fall, after compiling assessment information from the previous spring, the Math Majors Committee will review the assessment data collected during the previous year and probe it for trends that might warrant more extensive investigation that would involve examination of student portfolios. The committee will also segregate students who show a pattern of weakness in the program, and determine if their difficulties could be addressed through program development or revision.

Timeline for Assessment.

December 2003. Portfolio archive of final exams in pivotal course established for majors.

January 2004. Undergraduate Curriculum Committee drafts guidelines relating to learning goals to pivotal course content.

February 2004. Guidelines distributed to instructors of fall semester courses and instructors make recommendations to Math Majors Committee concerning relationship between grade and achievement level.

March 2004. Math Majors Committee reviews instructors’ recommendations and establishes correspondences; data for fall semester is collected.

May 2004. Instructors add final exams to student portfolios, and submit grades and achievement level correspondence recommendations to Math Majors Committee

September 2004. Math Majors Committee sets spring semester grade and achievement level correspondence and collects data for spring. Committee reviews assessment data from 2003-2004 AY, submits report with recommendation to chair.

December 2004. Instructors add final exams to student portfolios, and submit grades and achievement level correspondence recommendations to Math Majors Committee.

January 2005. Undergraduate Curriculum Committee reviews guidelines relating to learning goals to pivotal course content in the light of assessment data collected over the previous year and recommends changes, if necessary

January 2005. Math Majors Committee reviews instructor recommendations and collects assessment data for fall semester.

May 2005. Instructors add final exams to student portfolios, and submit grades and achievement level correspondence recommendations to Math Majors Committee using revised guidelines.

September 2005. Math Majors Committee sets spring grades to achievement level correspondence and collects assessment data. Committee reviews assessment data for 2004-2005 AY, and reports on trends to chair. Committee also evaluates the effectiveness of department’s assessment procedure, submits recommendations to chair and department.

Specific Program/Curricular Changes. The plan outlined above for the assessment of the department’s majors has not been implemented and, as a result, we cannot report any changes. The department’s earlier assessment plan, that was consistently implemented, was directed at the effectiveness of our service courses and involved the embedding of similar questions in the final of all lower division calculus courses. The data accumulated from student responses to the embedded questions have been extensively discussed, and may have been useful in pointing instructors to particular topics where students seem to be having difficulty. However, some elements of the current proposal have been in use over the past year. A study based upon student grades examining how our majors performed in calculus relative to their performance in MATH 3190, “Introduction to Mathematical Analysis”, has convinced the department that special attention needs to be paid to students seeking further training in mathematics past calculus during their calculus studies. As a result of this study, a new calculus sequence has been proposed that will help students think critically about calculus, and, hopefully, prepare them for further studies in mathematics or other areas where deeper understanding of calculus is required. It is hoped that the further analysis of student success that will occur as a result of this assessment plan will point out future programmatic changes that will help students in mathematics and other disciplines complete their programs of study in a timely manner and allow us to attract more majors to mathematics.

Specific Changes To Planning and Reallocation. The assessment vehicle that has been proposed has as its primary focus the assessment of student learning in courses related to mathematics undergraduate majors, and, as such, its impact on the department’s programs will largely be in the area of curricular revision and reform. However, it is conceivable that information gathered in this effort may lead to change in program emphasis and could also lead to a reallocation of department resources. However, the department’s major programs are fairly streamed lined and concentrate on the basic area of the mathematical sciences, so it is at the moment hard to see how further efficiencies could be achieved through program planning or reallocation.

Assessment Liaisons. The Math Majors Committee with assistance from the chair and associate chair would be responsible for implementing the above plan. The Math Majors Committee chair is elected annually; the current chair is William Thomas. The current department chair and associate chair are Geoffrey Martin and Don White.

Conclusion. The value of an effective assessment procedure arises not only from the useful data that it generates, but also from the fact that the process itself leads to an evaluation of the assessed programs. Since undergraduate instruction in mathematics is entirely centered on course work, and since this course work is cumulative, we feel that focusing of assessment methods on course sequences is important to our majors and will not only provide valuable time-ordered data on the progress of our majors, but the process itself will also create opportunities to relate what we are teaching in our courses to our stated learning goals. In this way, the data that is collected becomes immediately relevant to the discussion of teaching methods and course content, and makes assessment an integral part of curriculum revision and development. In some sense, the present plan aims to improve the impact of assessment on program development as compared to the effect that the earlier assessment procedure had on our service courses. While the method of embedded questions produced useful information about specific difficulties that students had with particular topics, it lacked broad measures of student success that could serve as dashboard indicators of how well our program was functioning. The two-tiered system that we plan to implement will produce information at both levels and hopefully function more effectively in guiding program development.

Pure Mathematics
Course Identifier	Course Name	Goal Measured
MATH 3860/MATH 3820	Intro to Differential Equations	(1)
MATH 3190	Intro to Mathematical Analysis	(3)
MATH 4300	Linear Algebra I	(2)
MATH 4820	Intro to Real Analysis	(4) (5)
MATH 4330	Abstract Algebra I	(4) (5)

Applied Mathematics
Course Identifier	Course Name	Goal Measured
MATH 3860/MATH 3820	Intro to Differential Equations	(1)
MATH 3190	Intro to Mathematical Analysis	(3)
MATH 4820	Intro to Real Analysis	(4) (6)
MATH 4300/ MATH4350	Linear Algebra I/Applied Linear Algebra	(2)
MATH 4720	Methods of Numerical Analysis II	(4) (6)

Statistics
Course Identifier	Course Name	Goal Measured
MATH 3860/MATH 3820	Intro to Differential Equations	(1)
MATH 3190	Intro to Mathematical Analysis	(3)
MATH 4300/ MATH4350	Linear Algebra I/Applied Linear Algebra	(2)
MATH 4680	Intro to the Theory of Probability	(4) (7)
MATH 4690	Intro to Mathematical Statistics	(4) (7)

	AYA Mathematics
Course Identifier		Course Name		Goal Measured
	MATH 3860/MATH 3820		Intro to Differential Equations		(1)
	MATH 3190		Intro to Mathematical Analysis		(3)
	MATH 4300/ MATH4350		Linear Algebra I/Applied Linear Algebra		(2)
	MATH 3450		Modern Geometry II		(4) (8)
	MATH 3510		History of Mathematics		(8)