Towards an automated introductory chemistry open lab

Daniel C. Tofan
Chemistry Department
Eastern Kentucky University

Overview:

1. The EKU model
2. Some statistics
3. Automating laboratory management

Part 1: The EKU Model

Eastern Kentucky University is offering two introductory courses for non majors: CHE 101 (Introductory Chemistry I) and CHE 105 (Chemistry for Health Sciences). CHE 101 is also our general education course, taken by students from more than 40 different majors each year. The laboratory portion of both CHE 101 and CHE 105 is run as a single open lab. By open lab we mean that there are no scheduled sections, instead the lab is open several (typical three) days a week between specific hours. Students can attend anytime the lab is open, and since they do not have a stable lab instructor as is the case with scheduled sections, they are required to take a number of steps to ensure that they are well prepared for the laboratory experience. Specifically, they need to read a document (available on the lab website) that introduces the material and gives a detailed procedure, they need to watch videos that show them the experiment being performed by a faculty member, and they need to take a pre-lab quiz to test their knowledge of concepts involved in the experiment.

Once a student is ready to perform the experiment, they come to the open lab during open hours (during the current semester the lab is open Tuesday through Thursday 9-6, but in the past it used to be open for more hours), where an instructor will check him/her in, the student will perform the experiment, and then check out. The main reason we are running an open lab is because we need to accommodate a large number of students (approximately 400 each semester in average) in a small laboratory with 22 workstations. It would take a lab twice as big, or twice as many hours to serve this number of students in a traditional laboratory setting. The experiments are designed so that the time spent in lab by the average student is about 75 minutes per experiment. If we had scheduled section, we would have to reserve two full hours for each section, plus the gap between sections. With the open lab, the turnaround is much shorter, since a student may check in and start working as soon as a station becomes available.

There are other advantages to the open lab:

• it offers flexibility in scheduling, for both students and instructors; when polled about the kind of schedule they would prefer for this lab, students chose flexibility over having the same instructor every time
• students have the feeling of being more independent, and self preparation is greatly encouraged through the set of pre-laboratory activities that we have in place
• it is a great resource saver for the department; not only do we need less personnel to staff the laboratory, but also it is easier to reuse the same equipment and it is much less demanding in terms of space, student drawers etc.
• this laboratory serves as a “launching ramp” for instructors coming in with less teaching experience; after spending time in the open lab they become more skilled and ready to take on teaching more advanced freshman and upper class labs

There are, of course, disadvantages to using an open lab format:

• students do not have the interaction with a permanent instructor that knows them and can provided more personalized help
• team work is generally precluded, as the open lab is designed to promote individual work; it would be quite difficult for pairs of students to show up together consistently given that they do have the flexibility of attending whenever they want
• grading tends to be less consistent than with scheduled sections, as the same grader will grade papers from different students every time; this leads to quite a few complaints about grading
• it becomes the job of the laboratory coordinator to handle complaints and mediate conflicts
• less reliable instructors are sometimes assigned to this laboratory, due to its nature
• students do tend to come to lab unprepared, regardless of how many pre-lab activities we make them do before lab; this results in students spending more time in lab than necessary and frustration on instructor’s part
• the biggest problem of all is student procrastination

Indeed, the latter is our single most important problem: even though the lab is open three days a week, the largest number of students will wait until Thursday (the last day) to attend lab. This creates long lines outside the lab (in general, the lab is running to capacity on Thursdays and students will stand in line even up to two hours waiting for a station to become available).

Part 2: Some statistics

Over the last 5 years or so, various statistics have been compiled using data we collected about student attendance. What was recorded was when students were coming to lab, how long they took for each experiment, what grades were recorded in pre-labs and on the lab reports, etc. Following is a series of graphs that show what kind of information we obtained.

For most of the last 5 years, the lab was open Tuesdays, Wednesdays and Thursdays. The hours varied, but in general it was 10-6 on TW and 10-8 on R. Figure 1 shows the percentage of students that attended each of the three days the lab was open between Spring 1999 and Spring 2004, regardless of the particular semester.

Figure 1: Student attendance distribution per day of week (percentage of total)

Consistently, Tuesday is our less successful day, with only 18.0% of the total number of students attending in average, while Thursday has always been the busiest with 50.6%. This demonstrates student procrastination: putting off the lab until the last day. When we looked at the minima and maxima (calculated as the average for a given semester), we noticed that the maximum average number of students in a semester that attended Tuesday was lower than the minimum average number that we have for Wednesday, and the maximum for Wednesday was yet lower than the minimum for Thursday (Figure 2). In other words, we never had (in looking at semesterly averages) more students in a Tuesday than we had on a Wednesday, and we never had more Wednesday than we had on a Thursday. So the preference for Thursday is not a seasonal one, or determined by some other semester-dependent reason; it must be procrastination. The reasons for procrastination is yet unknown – we suspect that students less prepared will put off lab as long as possible, only to attend at the last minute when there are more people in the lab and possibly the lack of preparedness goes unobserved and unpunished.

Figure 2: Minimum and maximum student attendance per day of week (percentage of total) between Spring 1999 and Spring 2004

This claim can somewhat be supported by looking at how the lab grades varied by day of the week. Figure 3 shows that between Spring 1999 and Fall 2002, there was an overall decline in the grade average from Tuesday to Thursday.

Figure 3: Distribution of lab grades per day of the week (averages calculated between Spring 1999 and Fall 2002)

Student procrastination had serious consequences: there were long lines on Thursdays, which led to the fact that many students were not able to get into the lab by closing time and thus more complaints to the coordinator, and the instructors’ time was used inefficiently, as they were very busy Thursdays and getting bored on Tuesdays. One such extreme situation occurred in Fall 2001, when in one particular week we had 15 students all day Tuesday, 68 on Wednesday and 129 on Thursday. There were 364 students enrolled that semester (not all attended, however), so possibly around 100 students missed that lab because they did not attend early in the week and could not get in on Thursday.

The student procrastination problem has been around for as long as the open lab was (since the early 70’s). We cannot really explain it. We tried to work around it by opening one extra day (in Fall 2004 we opened for 6 hours on Friday in addition to the 27 “regular” hours) and, in another instance, by offering extra points for early attendance (in Spring 2005 we offered 1 point to students who attended Tuesday and 0.5 points to students who attended Wednesday). The results of these two approaches are shown in figures 4 and 5.

Figure 4: Student attendance distribution in Fall 2004 after opening 6 hours on Fridays

Figure 5: Student attendance distribution in Spring 2005 after giving extra points for early attendance

Opening Friday did nothing but take away even more students from the Tuesday numbers (reduced to 13.2%) and add a substantial Friday column (24.6%). This confirmed our prediction that if given the chance, students will put off lab even more and really go at the very last moment. The second approach had the desired result: an almost even distribution of the numbers of students by the three available days. Thursday still wins with 39.8%, but the difference from Tuesday was only 10%. Still, we are wondering if giving extra points to the students to determine them to attend lab early in the week is the right thing to do, instead of teaching them a sense of responsibility and conveying the idea that they should not be rewarded for not procrastinating. So in Fall 2005 we are no longer giving extra points for early attendance. So far it does not look too good in what attendance is concerned, but we will wait until the end of the semester to look at the data.

What is of more concern to the department, however, is how efficient our laboratory is. We defined the operating efficiency of the open lab as the ration between the amount of time the lab is occupied by students and the amount of time the lab is available. During a regular week of 27 open hours, the number of minutes available to students at all 22 stations is 27 x 60 x 22 = 35640. We then added together the number of minutes spent in lab by all students at all occupied stations during a semester, and were able to calculate the operating efficiency per semester as a percentage, representing how much the lab was occupied. Figure 6 shows how our operating efficiency varied between Spring 1999 and Spring 2005.

Figure 6: Laboratory operating efficiency between Spring 1999 and Spring 2005

The open lab efficiency varied between a low of 34.2% in Spring 1999 and a high of 66.7% in Fall 2003. In general, it can be noted that Fall semesters are more efficient than Spring semesters. Since the number of hours the lab was open during this time was generally the same, the reason for this trend is because we always have more students in the Fall than in the Spring. Fall of 2003 saw an unusually high number of students registered for this lab (522). So it seemed natural that in order to achieve a high efficiency we would need a large number of students that are evenly distributed throughout the week in what regards their attendance. We plotted the percent efficiency versus the number of students attending regularly and obtained the graph in Figure 7. By finding a linear fit and extrapolating to 100%, we find that in order to operate at (theoretical) maximum efficiency we would need about 510 students in the lab per week, evenly distributed throughout the three days and with minimum turnaround time.

Figure 7: Correlation between percent occupancy and number of students attending regularly; extrapolation to 100% efficiency shown

The extrapolation in Figure 7 assumes an average time spent in lab of 70 minutes per student. The situation just described is not realistic, so we looked at how the operating efficiency varied with an empirical quantity that measures the “amount of procrastination” of students. This was defined as the difference between the percentage of students attending Thursday and those attending Tuesday. What was obtained is shown in Figure 8.

Figure 8: Correlation between percent occupancy and "student procrastination" (measured as the difference between Thursday and Tuesday attendance as percentage average)

Note that in two extreme cases, one where the difference between Thursday and Tuesday was about 11% (low procrastination) and one when the difference was about 45% (high procrastination), the operating efficiency was essentially the same: 47%. This was very surprising, and it taught us something else: we keep the lab open too many hours. In other words, if we get the same percent occupancy when students attend evenly during the week and again when there is a huge difference between Tuesday and Thursday, it means that we should reduce the open hours. Based on this analysis, we calculated that we need 24 open hours in Fall 2005 with 470 students registered and an average time of 75 minutes per student per experiment in order to achieve 90% efficiency. We actually scheduled 27 hours (T-R 9-6) to be on the safe side, and will look at the numbers again at the end of the semester. The 27 hours should give us about 80% efficiency if 95% of the students registered attend regularly (the latter percentage is a bit optimistic).

Part 3: Automating Laboratory Management

Statistics such as the ones shown were possible through tedious data collection over the past few years. In the past we had student workers help with the check-in and check-out process, which was quite time consuming because we collected a lot of information (time of check in, time of check out, the station number assigned to the student, what unknowns they were given, number of technique points given by the instructor. All this data was written by hand on the lab reports, and then we had to rely on student workers again to enter all the times and numbers in Excel. Collecting this data was very painful and resulted in ugly spreadsheets (Figure 9).

Figure 9: Sample spreadsheet showing manual data collection (attendance and prelab scores)

In Fall 2005 the author of this paper became the Laboratory Coordinator for CHE 101/105 and began an overhaul of the lab management process. The goal was to introduce computer technology to collect data. Three areas were identified as targets for automation:

• pre-lab activities
• lab management
• lab work and grading

At the present time (and for the past three years) pre-lab quizzes are delivered online through Blackboard. Students log in and take the quiz, generally being required to obtain a certain passing score (they have multiple attempts). Currently there is no way for a software program to check Blackboard scores automatically (at least under the current Blackboard license for EKU), so instructors needed to manually check each student’s grade upon check-in to make sure they passed the pre-lab. Other disadvantages of Blackboard are:

• our current question pools are small (20-30 questions per pool), which results in the same questions repeating when students take multiple attempts; this make it easy to get the correct answer after a few trials, but it does not necessarily mean that the student actually worked to determine the correct answer
• Blackboard does not provide information about the number of attempts a student needed to answer a question
• it is not possible to create truly parameterized questions

We tried to overcome some of these issues by writing software that will generate question pools containing hundreds of questions that are essentially variations of the same question with parameters. Here is an example of a parameterized question with three different versions:

• Given a 5.0 g sample of vinegar having a mass percentage of acetic acid equal to 9.1 %, calculate what volume of NaOH solution of molarity 0.82 M will be necessary to titrate this vinegar sample.
• Given a 0.384 g sample of vinegar having a mass percentage of acetic acid equal to 9.0 %, calculate what volume of NaOH solution of molarity 0.027 M will be necessary to titrate this vinegar sample.
• Given a 5.318 g sample of vinegar having a mass percentage of acetic acid equal to 8.00 %, calculate what volume of NaOH solution of molarity 0.5070 M will be necessary to titrate this vinegar sample.

Notice that the text of the problem is identical in all three cases, only numbers changed (we can make units change as well if we want). The program will generate many possible answers, in this particular case 12 for each question, to reduce the possibility of guessing to a minimum. Here are the sets of 12 possible answers generated for the three questions above:

Notice that all answers are provided in scientific notation, as one of the goals of this laboratory is to teach students how to consistently use correct significant figures and scientific notation. The correct answer is in red for each question. The incorrect answers that do not only differ in the number of significant digits are also generated by the program using specific rules, based on assumptions that we make regarding how a student might solve this problem (for example by dividing the molar mass of a compound by the mass of the sample to find moles, as we observed that many students do).

Our lab management is done through a piece of software called LabWhiz^® Student Manager^®, which helps us keep track of every student who is working in the laboratory at any given time. Some of the features of this software:
- automatically records the time of check-in and check-out
- lets instructors assign a workstation based on a graphical interface
- provides an overall picture of lab occupancy
- allows recording events that occur in lab, moving students to other stations, removing students from lab, assigning technique points

Following is a sequence of screenshots form the actual program, which better illustrate how it works.

At any given moment, the current instructor in the lab will know how many workstations are available and is ready to check in a new student as long as that number is not zero:

When a student comes to check in, the instructor picks the student’s name from the roster for the student’s lecture section (designated using the instructor’s initial), checks the student’s Blackboard pre-lab score, verifies that the student has put safety glasses on (the check-in process is done inside the lab) and can assign unknowns to the student if the experiment requires so:

The next step is to assign a station number to the student. The instructor clicks on the “Assign workstation” button and is presented with a graphical representation of the numbered stations; those available are shown in blue:

At any given time, the instructor can switch to the lab occupancy screen, which will show who is currently working in the lab, at what station, since what time, and what unknowns they have (if any):

During lab, the instructor has a number of options available for each student. He/she can record events taking place during lab or can remove a student from the lab for a serious infringement of safety policy or procedures:

When a student finishes his/her experiment, the instructor will check the student out and will assign a number of techniques points (4 in the past, now only 2) by checking a series of items predefined in the software:

All the data recorded in the lab (from the moment of check-in to the moment of check-out) is recorded in a database and is available to the lab coordinator at any time.

The following steps are planned towards improving the open lab and automating some of its features, categorized by the three areas identified above:

A. Pre-lab activities

• the lab manual will be written as a multimedia file, including color images, video, hypertext
• procedure videos will be shot in higher resolution, better quality (possibly DVD)
• new software will be written to administer pre-lab test to replace Blackboard; high degrees of question parameterization will be possible
• all pre-lab results will be available in a database searchable by the coordinator
• a dedicated computer lab will be used for pre-lab activities

B. Lab management

• we plan to have a central server that will communicate with the lab management software through the network and will be able to offer students statistics about each week’s experiment, to help them plan their time better in order to attend lab in optimal conditions
• we plan to offer a self check-in station to eliminate the need for an instructor to spend time for that activity; while some of the details need to be worked out (such as how to prevent a student wearing inappropriate clothing to check him/herself in), we anticipate that the station will work something like this:
  • student enters their ID number in a computer (or scans their ID card), system checks pre-lab score and allows the student in the lab if they passed the prelab
  • if allowed in lab, the software assigns a workstation, unknowns (if any), prints report sheet with all student information (name, section etc.)
• the instructor will carry a handheld computer that will allow him/her to freely move in the lab and supervise all students without being tied to the check-in station for too long; the handheld will communicate wirelessly with a server and will allow the instructor to make notes about the students’ technique and to check a student out right at their workstation, while making sure that the station is clean and ready for the next student

C. Lab work and grading

• the laboratory will be outfitted with computers at every station; computers will be used for data collection during lab (using probes) and to automatically grade numerical answers entered by the students
• the students will automatically receive grades based on the accuracy of their results and input from the instructor (open answer questions will still be graded manually)
• a report will be printed for each student upon check-out using a printer installed in the lab; students will thus leave knowing how their results were graded and will have a “receipt” of lab attendance and performance
• all results will be saved in a database and available to the coordinator for further processing and educational research

Currently, work is in progress on the first feature listed under section B above. Here is a summary of the kind of information that will be offered to the students hopefully this semester via the web:

Welcome to the CHE 101/105 Lab Planner
Current day/time: Wednesday, 12:26 PM
Number of stations currently available: 11
Average time needed to complete this experiment based on recent data: 67 minutes
Compared to last week’s lab, this week’s experiment is in average 14 minutes shorter
Enter desired time to attend lab: 3:30 PM (student input)
Projected available stations at that time based on recent attendance patterns: 3 or 4

We also have started testing a computer based system in one of our general chemistry labs for majors. We have 18 computers (one per student) and we are sing Vernier probes for data collection. We are running a pilot project this semester to test the setup. Below are a few pictures of our computerized wet lab:

The computers are Dell XS-280 models, chosen for their very small footprint (mounted behind the 19” flat screen monitors). The computers are all networked and students are able to save the data files and email the data to themselves for after lab processing. We use foldable, spill-proof keyboards that are ideal for a lab setting. When not in use, these keyboards are folded and stored on the shelf above the workbench. The keyboard and the mouse are the only computer parts sitting on the bench while students are working. The CPU and monitor are mounted on the top shelf, as seen in the pictures.

I invite anybody interested in our setup to contact me for further details.

Acknowledgements

Susan Godbey, former lab coordinator (data collection 1999-2002)
Linda Parsons, former lab coordinator (data collection 2002-2004)

Go back to the CONFCHEM Fall 2005 Newsletter.