You are on page 1of 19

GradeCam Software Evaluation

Carly Hill
ED TECH 505-4172 12/9/2011

Contents
Learning Reflection ........................................................................................................................ 3 Executive Summary ........................................................................................................................ 4 Purpose of the Evaluation ............................................................................................................... 5 Background Information ................................................................................................................. 6 Description of Evaluation Design ................................................................................................... 8 Results ........................................................................................................................................... 10 Discussion of Results .................................................................................................................... 12 Conclusions and Recommendations ............................................................................................. 13 Appendix 1: EPD .......................................................................................................................... 15 Appendix 2: Timeline ................................................................................................................... 16 Appendix 3: GradeCam Survey .................................................................................................... 18

Learning Reflection
Before taking this class, I put evaluation and research projects into the same category thinking that the two were one in the same. After completing my own evaluation and this course, I have learned that they are two very different things and that each can be useful to a teacher in the classroom. By completing this evaluation project, I have learned more about a tool I use in my own classroom as well as how I can evaluate other programs that I use in my own classroom which links with AECT Standard 3.3. I also know that I can use this process to help my peers by being able to give them pros and cons of using a program within their own classroom. I feel confident that I could do this process now on my own and deliver the results to the administrators within my building and other colleagues on various programs and this links with Standard 5 of the AECT Standards. I can also take the principles of the evaluation and apply them to my own teaching. One thing that I know I will immediately implement in my classroom is evaluation process of long-range planning. I already do this to some extent, but learning the evaluation process and the steps have shown me an ordered way of going about analyzing my teaching and how I can make it better. I think that this piece of the evaluation is just like the reflection that many teachers already do, but looking at it as part of the ordered process of a goal-oriented evaluation, I can improve my teaching by looking at my students results on the end of course tests and compare that with how I taught the material and look to see where I can make improvements. Completing this process relates to the AECT Standard 1 as I can now analyze more effectively how my assessments compare to the objectives of my course as well as determining if I am using the correct assessment in my class to decide on how I want my students to show me what they have learned. I also learned about how to create my own survey which I have never done before. I found that SurveyMonkey was such a great and easy to use tool that I can implement in my own classroom to have students do an anonymous mid-course evaluations of my class and of my teaching style. I like this rather than having them complete it in class because they might feel like they cannot say what they really feel if I am in the room with them. This way they can respond honestly to what they think about the class and how they would want to fix it or what they like. I can also use it for other purposes within my government class where students can use it to create surveys on different topics we discuss in class. This new tool that I learned how to use connects to AECT Standard 2.

Executive Summary
This evaluation was conducted on the GradeCam software program and how it is used by teachers at Timberline High School. This particular program was chosen to be evaluated because it can be used easily by many teachers as a way to quickly check student understanding on concepts that appear on quizzes or tests a teacher might give. It is also a less expensive option for the school than the Scantron machine as it does not require the purchase of one big, immobile machine and specialized answer sheets. GradeCam has the capability to be moved easily from room to room and also comes with templates of answers sheets that teachers can easily copy as many times necessary. The software is easily installed on most computers and can be done without much hassle. The evaluator was curious to see how many teachers used this program, what content area was it most utilized in, and if more advanced features were used to get more complex data about the objective assessment they gave to their students. A survey was created by the evaluator using the SurveyMonkey website and a link was distributed via email to all teachers at the high school. In total, 37 teachers responded and only 10 teachers use GradeCam currently in their classrooms with the majority of use being in the Science, Social Studies, and the English departments. Eight teachers reported using the feature to allow students to grade their own work and nine teachers responded that they use the Item Analysis function to get more insight into which concepts their students are struggling with. The survey also asked for teachers personal reasons behind why or why not they allowed their students to grade their own work and how the item analysis function impacted their teaching. Many teachers responded that they did not really allow students to grade their own work using the Student Window function, but many did respond that they used the item analysis to directly inform their teaching.

Purpose of the Evaluation


What was the purpose of doing this particular evaluation? The purpose of completing this evaluation was to see how teachers utilized the GradeCam program in their classrooms. There are fifteen document cameras and licenses that Timberline High School has access to and the evaluator wished to see how many teachers used this instead of the Scantron machine as it is less expensive due to the fact that GradeCam does not require the purchase of specialized answer sheets. GradeCam can also quickly check student understanding of the material and they can use that feedback to inform their teaching What were the central questions to be answered through the evaluation? In order to look at how teachers are using GradeCam in their classrooms, the evaluation looked at three specific questions: 1. Are teachers using Grade Cam as a formative assessment tool to guide instruction? 2. Is Grade Cam used more often by teachers in different content areas? 3. Do teachers allow students to grade their own work through the "student window" or do teachers primarily scan the work themselves? The first question was chosen to look at how teachers used the item analysis function that GradeCam has built into the software and how teachers used that information to guide their instruction. Quizzes or tests can be graded quickly and the item analysis can be used to get instant feedback while students are still in class immediately after they turn in the work or possibly addressed the next day. For the second question, the evaluator wanted to see if different content areas were more likely to use GradeCam over others. For example, teachers in the content areas such as Math or Science might be more likely to give their students objective assessments that can be graded using GradeCam software whereas English teachers might not find it as useful as they are more likely to used subjective assessments. Lastly, the GradeCam has the ability to allow students to grade their own work and teachers were asked if they allowed students to grade their own work or if they primarily did this themselves. This question was asked to see if teachers who used GradeCam were more willing to let students take charge of this or if they continued to want to be the ones scanning the work. Who would be most impacted by the results of the evaluation? Results from this evaluation will affect those teachers who already use GradeCam as well as those who do not utilize the program. For those who already use it, they can look at the results of this evaluation (particularly the teachers personal responses) and see how individual teachers are using it their classrooms and they might be persuaded to change how they are using it in their classroom. Teachers who do not use GradeCam, but might use Scantron could see the results of this evaluation and see the pros of using it to complete quick, objective grading in their own classroom rather than going down to the copy room to use the only Scantron machine in the high school. They will also be made aware of different functions that GradeCam offers like item 5

analysis or other functions like being able to compare students of the same class that are in different sections and being able to see how students are performing on questions based on state standards.

Background Information
Rationale and Goals GradeCam is a software program that was purchased by the Boise School District and distributed to teachers at Timberline High School to allow teachers to quickly grade any multiple choice quiz or test in order to provide feedback to students in a timely manner. By using a document camera that is connected to a teachers computer with a USB cable, students or teachers can scan the answer sheet and the assignment is graded right on the spot. Data that is collected by the GradeCam software can be used to do an item analysis of questions on the test so that teachers can see what areas students are struggling with so that those areas can be revisited quickly in class. GradeCam comes equipped with a feature where questions given by the teacher on a multiple choice exam can be linked to state standards so teachers can measure how well their students are doing on those items. These results can be used by a teacher to re-teach content that could appear on a state standardized test that students might not be understanding. Teachers in the classroom can also share their results on their tests with other colleagues so that those who teach the same class can aggregate their scores together to see how their students are doing as an entire group. GradeCam also has a feature that allows students to grade their own work one at a time so that they can get instant feedback and it frees up the teacher to answer questions that students might have about what they missed. Once all of the students work is graded, GradeCam can then import results that are collected in the softwares grade book to any computer-based grade book system that a school may use. This allows the results of the assignment to be quickly added into a students score so that they can be updated on how they are doing in class. Here is a picture of what is included in the basic kit:

Past Products Before the purchase of the GradeCam, teachers primarily used the Scantron hardware and answer sheets to perform the same tasks that they could complete with the GradeCam. The Scantron machine at Timberline High School is located in the main copy room and teachers must use the special answer sheets that can only be purchased through the company that produces the hardware. These forms cost $59.95 for a package of 500 forms. A Scantron machine that is in use at the high school costs $1465.00. The machine does many of the same things that GradeCam does, but there is only one in the school and cannot be moved out of the copy room so teachers must bring their tests to the machine for them to be grade. Here is a picture of the Scantron and a copy of an answer sheet being scanned:

Product Development GradeCam was developed by the Porter family primarily to address the need of cutting down on teacher grading. Tami Porter, who at the time was a middle school teacher, was inspired by what she saw at the grocery store where she witnessed a store clerk scanning items for purchase. With this idea, she went to her teenage son Robert who at the time was working as a software developer and he came up with the image recognition software that would become GradeCam. The small company located in Livermore, California employs Tami, Robert, and Tamis husband Rick. Their mission is provide teachers and schools with low cost equipment that still performs at a high quality level. The company is continuing to improve their product and since the first GradeCam was released, the company has tried to update their products software in a timely and efficient manner. There is a customer support feature on their website that has forums where people post common issues to get help as well as a link to contact customer support with any issues a user might have. Characteristics of the GradeCam As mentioned above, the GradeCam can be attached to any computer using a USB cord and is easy to install on any computer. By following simple steps, teachers can quickly put this software on their computer and use it with ease once it is installed. The document camera that comes with the GradeCam is portable and is very light to carry from room to room if teachers have to share it. Rather than having to go down to a central copy room, these cameras can be 7

passed around if necessary and used as long as the software is installed. Another feature of the GradeCam is that schools do not have to own the GradeCam camera that comes with the based kit in the picture above because the GradeCam software is compatible with many document cameras that a school might have already purchased. If a school needs to purchase a camera, the GradeCam software and document camera used at Timberline High School costs $349.20. This includes a user license and one document camera. Another feature of GradeCam is that it comes with answer sheets that are built into the program. Rather than having to pay for Scantron sheets, GradeCam has customizable answer sheets that can be printed off and copied depending on how many the teacher needs for each class.

Description of Evaluation Design


The evaluation of the GradeCam software that I conducted was based upon the goal-based model. This model requires the use of stated objectives or goals that an evaluator is looking for that rely on questions stated before the process begins. Using those questions, objectives are also formed that direct the evaluation. With this model in mind, I established three questions and objectives that I measured through a qualitative survey that I made through the website Survey Monkey. The questions that I looked at in my evaluation were: 1. Are teachers using Grade Cam as a formative assessment tool to guide instruction? 2. Is Grade Cam used more often by teachers in different content areas? 3. Do teachers allow students to grade their own work through the "student window" or do teachers primarily scan the work themselves? With these questions in mind, I used them to direct the following objectives that I used to craft my survey and what my evaluation would look like: 1. Grade Cam has a reports section that allows teachers do item analysis on any formative assessment that they are giving in class. This can provide them instant feedback on what they need to reteach. 2. Grade Cam can only be used for summative assessment grading. It can be utilized in many content areas though for a quick check of understanding basic terms. 3. Through the student window, students can scan their own assessments and get instant feedback on how they did and what they missed so that they can address their weak areas. After formulating the questions and objectives, I used these to create a survey that I emailed to the 67 teachers that I work with at the high school. I made sure to include questions that addressed each of the objectives that I wanted to look at. In order to find out if teachers used GradeCam as formative assessment tool, I asked teachers Do you use the immediate feedback you receive from the Item Analysis function in Grade Cam to inform your teaching? I also added a follow up question, If yes, how does it affect the topic that you are teacher? With the 8

first question, I wanted to see if teachers use the item-analysis feature for instant feedback. As soon as tests/quizzes have been scanned, an item analysis can be done on the assignment instantly so teachers can get that feedback and do with it what they feel best for their students. My main motivation for the follow up question was so that teachers could go more in-depth than just a simple yes answer and so I could see the ways that teachers used those results in their class. For my second objective, I wanted to see two things; how many teachers use GradeCam and what content areas the program was most used in. First, I asked teachers if they used GradeCam in their classroom and if they did not, I did not have them continue with the survey as they would not know how to answer the questions that followed. Next, I asked teachers what content area they taught in so I could see where the most concentration of the use of GradeCam exists. I was curious see if teachers in one content area or another were more likely to use GradeCam in their classroom. In order to analyze my last objective, I asked teachers Do you allow your students to grade their own work using the GradeCam student view feature? With this question, I wanted to see how many teachers take advantage of these feature so that they can be free to answer questions and students can get exactly what they got on their quiz/test and know exactly which questions they missed. I also had another follow up question on this that asked if they responded no why that was the case. My main motivation for this question was to see what teachers responses were as to why they did not use this function.

Results
The following results were taken from the GradeCam survey made using SurveyMonkey.com and distributed to 67 teachers at Timberline High School. A total of 37 teachers responded to my survey representing 55% of the staff. For the full survey, please see Appendix 3. 1. Do you use a Grade Cam in your classroom? If the answer is no, please submit your survey. Yes 10 Total Respondents: 37 2. What content area do you teach in? No 27

Other Elective Course Music/Drama Physical Education/Health Social Studies English Science Math 0 1 2 3 4 5 6 7

3. Do you allow your students to grade their own work using the Grade Cam student view feature? Yes 8 4. If no, why not? -DON'T USE IT. -unfamiliar with the feature -Don't have one 10 No 8

-I just use it as a document projector. -I am not familiar with Grade Cam -sometimes for formative assessments, but I have found that for tests there is too much activity around the gradecam for me to be sure that they don't cheat and to ensure confidentiality. -I couldn't get it to install properly on my computer, even with the help of the tech experts 5. Do you use the immediate feedback you receive from the Item Analysis function in Grade Cam to inform your teaching? Yes 9 6. If yes, how does it affect the topic you are teaching? -NOT AT ALL. DON'T USE IT. -Of course, the immediate feature allows reteaching, remediation, etc. -Provides an immediate check for understanding -it is better than scantron and allows me to reteach objectives that the students missed. this is a great tool. -It helps when I go back through the test with the students to identify where they had problems and to influence how I teach that topic the next year. It is really nice to assess where the kids went wrong and discuss with them when and where we covered things. -I analyze the areas that I need to reteach and do so either that day or the next. At times I adjust my grades due to analysis. I also give non-graded gradecam assessments strictly for teaching information. -We discuss why they other questions were chosen over the correct one. Can correct misconceptrions -It lets me know what types of problems the students need more practice with. I sometimes have assignments based on which problems they missed on gradecam quizzes. -Allows me to review important concepts that were missed, and if 50% miss, then I reteach that concept and retest No 5

11

-It directs the review of the test and allows me an opportunity to revisit topics that students did

Discussion of Results
There are 67 teachers at Timberline High School and 37 of those individuals answered my survey with only 10 reporting that they used GradeCam in their classroom. This means that only 18% of the staff has or currently uses GradeCam in their class. This shows that it is not a very widely used or known tool among the staff. It could possibly be that many are just used to using the Scantron machine that is centrally located in the staff copy room and are hesitant to change what has worked for them thus far. As mentioned previously, if teachers were to change to using GradeCam, the school would no longer have to purchase the costly answer sheets and teachers would be able to have their own GradeCam in their classroom. Grade Cam was most widely used in the Social Studies, Science, and English department with Math following at a close second. As all of these classes lend themselves to objective assessment, it shows that these teachers would find this tool useful in their classroom. The number of teachers who have their students grade their own work was split in half with eight in each category. Teachers who answered the follow-up response question about why they answered no, identified that they either did not know the function existed or that they found that it was too difficult to negotiate students while they were all grading their own tests. Eight teachers do seem to find it useful showing that this feature is used by some. One teacher even mentioned using it as a document cam for displaying other documents to their class which could also be another useful feature. A majority of teachers (9) responded that they did use the Item Analysis function in GradeCam, while five responded that they did not. The follow-up question asked teachers why they used it and most of them responded that it did impact their teaching soon after the results were looked at. Some also mentioned that it helped clear up any misconceptions that students might have and another teacher stated that it helped facilitate whole class discussions about the material. Two teachers also mentioned that it formed what their end of unit test might look like or what they would review as a result of what the item analysis results were. Without GradeCam, these teachers might not be able to get the instant results that they do get from having their own scanning ability in their rooms and students would definitely not get the instant feedback that GradeCam can give them if they are scanning their own work.

12

Conclusions and Recommendations


Immediate Conclusions GradeCam is not widely used or known about to the staff at Timberline High School Many teachers that use GradeCam, do not allow their students to grade their own work or are not aware of the feature. Mostly all the teachers that do use the item analysis function use it to inform their teaching by changing it how they teach it or revisiting the material in some way. Teachers in PE/Health and Drama do not make much use of GradeCam and it is most used in the core content subjects of Social Studies, Math, English, and Science.

Long-Range Planning In order to expand the use of GradeCam in the high school and in different content areas it is important that teachers are aware of the features of the program and trained in how it could be used in their classroom. Teachers that are comfortable with using the software should be the individuals that lead these workshop opportunities. The administration should also be made aware that this could replace the Scantron machine and if necessary, more GradeCams could be purchased at a much cheaper rate than continuing to use the Scantron. They should be given information so that they can be aware of how helpful this software can be as teachers have it in their own classrooms and they are also mobile and can be moved from different rooms easily. It would be helpful if teachers already using the software gave testimonials to the administration as to how this tool helped them in their classroom either face to face or via email. A training opportunity could also be offered to teachers who are already using this software in the classroom to help make them aware of the more advanced features that they may or may not know about using. Evaluation Insights If this evaluation were to be done over, the survey would be revamped so that people who do not use GradeCam would not be allowed to continue completing it as that did effect results for some of the answers. The two response questions would be changed so that everyone had to give a reason as to why or why not they allow students to grade their own work and they use the item analysis function. There would also be a response survey question added that asked teachers in different content areas why they think this tool is useful to their particular subject so that it could be assessed why different content areas lend themselves more easily to this kind of tool. It would also be constructive to send this survey out to the other high schools in the district to see what district-wide use of GradeCam is like so that it could be determined whether Timberline High School is normal or abnormal in their use the tool.

13

There would be a face to face interview with a couple of the teachers so that there would be more personal insight into how and why they use the GradeCam in their classroom. An interview with teachers who solely uses Scantron would also be used so that it could be assessed why they prefer that program more. There would be an objective included that included a comparison to GradeCam and Scantron so that the evaluation could look at the positives and negatives of these two programs side by side and staff could be asked these questions as well.

14

Appendix 1: EPD
Evaluation Question Program Objective/Goals Activities to Observe Data Source Population/Sampl Data Collection e Design Data Analysis

Are teachers using Grade Cam as a formative assessment tool to guide instruction?

Is Grade Cam used more often by teachers in different content areas?

Do teachers allow students to grade their own work through the "student window" or do teachers primarily scan the work themselves?

Grade Cam has a reports section that allows teachers do item analysis on any formative assessment that they are giving in class. This can provide them instant feedback on what they need to reteach. Grade Cam can only be used for summative assesment grading. It can be utilized in many content areas though for a quick check of understanding basic terms. Through the student window, students can scan their own assessments and get instant feedback on how they did and what they missed so that they can address their weak areas.

Staff Survey Responses

Teachers at Survey created by Timberline High evaluator School

Descriptive Survey created by Analysis with evaluator charts

Staff Survey Responses

Teachers at Survey created by Timberline High evaluator School

Descriptive Analysis with charts and Survey created by testimonials from evaluator interviews

Staff Survey

Teachers at Survey created by Timberline High evaluator School

Descriptive Analysis with charts and Survey created by testimonials from evaluator interviews

15

Appendix 2: Timeline
Date October 13th Day Thursday Task Begin brainstorming questions for the survey Complete survey questions and think about oneon-one survey items Begin crafting online survey using Survey Monkey or Qualitrics Put finishing touches on the survey and have it proofread Finalize list of oneon-one questions Go around Timberline and find out who uses Grade Cams in their classrooms and build a list of willing participants for the one-on-one survey Distribute by email the link for the online survey Close survey and begin tabulating results Conduct one-onone interviews Review data from both surveys Format final report document and set up table on contents Work on Executive Summary section

October 14th

Friday

October 17th

Monday

October 18th19th

Tuesday and Wednesday

October 20th October 21st

Thursday Friday

October 24th

Monday

October 28th

Friday

October 31stNovember 4th November 7th9th November 10th

Monday-Friday MondayWednesday Thrusday

November 11th

Friday

16

November 14th

Monday

Work on Purpose of Evaluation section Work on Background Information section Work on Description of Evaluation Design Work on Results section Work on Discussion of Results section Work on Conclusions & Recommendations section Put together the Appendices Complete the learning reflection Have someone proofread my final report Read over the final report one last time and then submit when ready

November 15th

Tuesday

November 18th19th November 20th November 23rd24th November 25th

WednesdayThursday Friday Monday-Tuesday

Wednesday

November 28th November 30th December 1st

Monday Tuesday Wednesday

December 5th

Monday

17

Appendix 3: GradeCam Survey


1. Do you use a GradeCam in your classroom? If the answer is no, please submit your survey. Thank you for your time! Yes No 2. What content area do you teach in? Math Science English Social Studies Physical Education/Health Music/Drama Other elective course 3. Do you allow your students to grade their own work using the GradeCam student view feature? Yes No 4. If no, why not?

18

5. Do you use the immediate feedback you receive from the Item Analysis function in GradeCam to inform your teaching? Yes No 6. If yes, how does it affect the topic that you are teaching?

19