Commission on Learning Resources and Instructional Technology

Grant Proposals

1998-1999

 

 

Measuring Their Minds:

 Assessing four years of student learning information competencies

Final Report

by Gabriela Sonntag, California State University San Marcos

 

                        Project Description

 

At CSUSM our goal has been to infuse information literacy into the courses through library-use assignments.  This grant allowed Renee Curry, Professor of Literature and Writing and myself, to test a multi-pronged assessment of both the General Education class assignments and the students' information competencies. 

 

Faculty/Course Assignment Assessment

 

To begin assessment we collected the syllabus and the assignments from all the lower-division GE courses since the program began in Fall 1995.  Just over 100 syllabi were collected and reviewed.  A depository of these syllabi and assignments was to have been established in the Library with the intention of adding to it each year for on-going assessment.  However this would duplicate the efforts of the General Education Office where we have access to these syllabi.

 

To facilitate our review of the assignments we developed a checklist that outlines the CSU information competencies.  After our review we emailed the various teaching teams with information on our project and a copy of the checklist with our initial evaluations.  We requested to meet with them to discuss it further.  Subsequently we scheduled meetings with all available GE faculty.  In all we met with 7 groups of GE faculty representing the following areas: GEW, GEO, GEL, GESS, GEH.  The idea was to help us better understand their objectives for the assignment and also help clarify skills and competencies attained. 

 

The checklist was very helpful in our discussions as faculty were able to better their understanding of the competencies and also begin to use a common language for describing these information skills.  In our interviews we heard many comments like " I do that, I just never called it that."  Many professors were very enthusiastic about continuing similar efforts including suggestions for meetings of the larger GE faculty to discuss competencies across disciplines.  Included in the interview were questions to discover faculty satisfaction with the outcomes.  In fact we were able to see that faculty attitudes towards Information Competency and toward library instruction in general were very positive.  This also gave us the opportunity to discuss how the library's Information Literacy Program could work collaboratively with the professor to further student's competencies.  These interviews helped to strengthen Librarian/Faculty collaboration in the GE.  This is a major success for the project.

 

We evaluated all assignments received based on the checklist highlighting all skills and learning outcomes.  Faculty were asked for permission to post these assignments on the web and also to serve as contacts for interested faculty.  Most were happy to give this permission and the webpages can be found at: http://library.csusm.edu/departments/ilp/ge/area-comp.html.  A website for the project has been created and will be maintained as part of the on-going assessment and as a resource for all interested in information competency instruction.  It can be found at: http://library.csusm.edu/departments/ilp/ilp_projects/measuring_minds_project.asp

 

Student Information Competency Assessment

 

With much assistance from members of the General Education Committee, enrollment Services and others, we were able to get a list of the 30 students who began as First-Year students in 1995 and who were to graduate in May 1999.  We contacted the students by email with follow-up phone calls.  We conducted three focus groups in May 1999 for a total of 14 students.  The focus group allowed us to get feedback on the GE program as a whole and to develop a better understanding of their attitudes toward the ILP components of these courses. 

 

Initially we had thought to develop an assessment webpage similar to the one developed by Kathleen Dunn, CSU Pomona for Freshmen, using the template created by Paul Adalian Cal Poly San Luis Obispo.  However we opted for a paper and pencil test as a pilot.  Using this test we measured the core competencies of these students.  The questions were divided according to the competencies outlined by the CSU. Questions 1-3 measured the student's ability to articulate a research problem.  Questions 4-10 measured their competence in making determinations about the type of sources needed to complete their research.  Questions11-15 looked to their ability to use electronic databases. The competency related to making selections from and synthesizing information was measured with questions 16-19 and their ability to evaluate information was part of questions 20-22.  Questions 23 and 24 measured understanding of intellectual property and copyright. Finally question 25 looked to their recognition of information competencies as long-term, adaptable cross-disciplinary skills.  Initial analysis shows us that more work should be done to develop a better instrument.  We also recognize that paper and pencil tests fall far short of what is satisfactory to measure information competencies. 

 

Because of the complexity of information competencies in a real-world situation another aspect of the test for the students included a hands-on problem-solving activity.  Five different problems were developed such that students were given a set time to solve a problem designed to show their use of competencies and information skills in a real-life situation.  Not just the "right" answer but the process they used for this test formed part of our analysis.  As they proceeded to solve the problem testers watched and took note of their progress.  Notes include which screens students went to, the order in which they progressed as well as their final results.  

Deliverables

Our goal was to design an assessment system that can be easily adapted by other CSU campuses in their information literacy efforts.  To this end we deliver:

·       A checklist for analyzing library use assignments for information literacy skills (skills)

·       A webpage of all assignments that can be used as models by other CSU faculty. Faculty designing these will also serve as contacts for other campuses.

·       A questionnaire for conducting focus groups with graduating students (attitudinal measures)

·       A set of real-world problems that can be used to test students (measure skill attainment)