Pack Hacks for Faculty: Quantitative Literacy on Discussion Boards

Welcome back to Pack Hacks for Faculty. Each month, a member of the NC State faculty will provide quick tips, advice and other insight to facilitate your teaching, research, scholarship or engagement activities. If you are interested in making a submission for a future Pack Hacks for Faculty, please review our submission guidelines and contact provost-communications@ncsu.edu if you have questions.

This month, Justin Post, teaching associate professor and director of online programs in the Department of Statistics, gives advice on using discussion boards (like Moodle) to facilitate engagement on quantitative data. 

Talking Quantitative Data Through Discussion Boards

Justin Post

Quantitative methods and analysis have become pretty ubiquitous across disciplines in recent years.  All fields from the sciences to the humanities have some aspects that revolve around analyzing data, discussing conclusions from studies that use data, and generally using data to make or backup an argument.  The skill of thinking critically about all aspects of a study involving data is foundational to our quantitative literacy competency.  

How can we help our students to think critically about data?  Specifically, how can we help students to consider the implications of the method of data collection and the appropriateness of the analysis on the validity of the conclusions made?

Many of our classes are already packed full with material.  This article describes a minimally invasive way to infuse these critical quantitative literacy skills into your course using an online discussion forum!

Worry Questions

The quality of data used in quantitative studies varies greatly.  What is meant by this is that data are often used to predict future outcomes or make conclusions about some larger population, but the data is not always representative of that population.  Data may have biases, leave portions of the population out, include measurement error, or not include relevant variables that would make relationships clear. These aspects of a study should always be considered when judging the relevance of any conclusions made.

Aside from data quality, the appropriateness of any statistical methodologies should also be critiqued.  Were data representing categories but coded numerically summarized with a mean? Are graphs clear or misleading?  Was the model used checked for violations of assumptions?

Lastly, one should always consider whether claims made based on data are sensible, especially in light of the above considerations about data quality and analysis method used.  Have the authors of the study implied conclusions that can’t be supported from the data? Has the media taken a well designed and analyzed study and improperly made their own conclusions, say using correlation to imply causation?  Are there other relevant things to consider that make the conclusions carry less weight?

To help students think about these aspects and form quantitatively sound reasoning, we can give them questions to consider about any study.  The following questions are adapted from Gal (2002) and a student can be imagined running through this type of list and asking how the questions are relevant for the current situation or task.  

1. Where did the data come from? What kind of study was it? Is this kind of study reasonable in this context?  
2. Was a sample used? How were participants selected?  Did the sample include people/units that are representative of the population?  Overall, could this sample reasonably lead to valid inferences about the target population?
3. Are the reported statistics appropriate for this kind of data? For example, was an average used to summarize ordinal data.  Could outliers cause this summary statistic to misrepresent the true picture?
4. Is a given graph drawn appropriately, or does it distort trends in the data?  
5. Overall, are the claims made here sensible and supported by the data? For example, is correlation confused with causation?  Is a practically meaningless difference made to loom large?  
6. Should additional information or procedures be made available to enable me to evaluate  the sensibility of these arguments? For example, did the writer “conveniently forget” to specify the base of a reported percent-of-change, or the actual sample size?
7. Are there alternative interpretations for the meaning of the findings or different explanations for what caused them? Are there additional implications that are not mentioned? 
8. How reliable or accurate were the instruments or measures (tests, questionnaires, interviews) used to generate the reported data?  
9. How was this probabilistic statement derived? Are the authors making poor assumptions that make the statement invalid? 

Moodle ForumNG

Discussion boards are widely used across the internet.  The basic idea is that someone posts a comment or question and others can respond.  Discussion forums can be easily added to a moodle site and provide a great way to spark a discussion around quantitative literacy.  

An effective way to use discussion boards is to break students up into small groups of 3 to 5.  The instructor provides a prompt or question for the students to consider. The out of class nature gives students time to think and form a relevant argument.  Once these initial posts are done, the other members of the group then need to respond by supporting or critiquing it. This activity can promote critical discussion between students.  Collaboration can be promoted by requiring a final post upon which the group agrees.

If you are new to groups and groupings in moodle, see here for a quick tutorial.  

If you are new to discussion forums in moodle, see here for a quick tutorial.

Writing a good discussion prompt can be challenging!  A good prompt should provide clear expectations to the students as well as promote discussion of quantitative data and interpretation.  In the prompt the instructor can give the relevant quantitative context from an article to allow students’ to thoroughly engage the worry questions or other directed questions.  

An example forum prompt is given here.

Instructor Feedback

One issue with using discussion boards is grading the efforts of students.  What makes a strong post? It is really useful to use a well-defined rubric for what you are looking for in posts and providing this to students gives a transparent method for grading.  This can remove subjectivity and help to provide clear expectations of what is a quality post.  

Given here is an example rubric that could be modified for your purposes.

Summary

• Discuss some important questions our students should ask when faced with a quantitative study.
• Implement a critical thinking discussion between students about quantitative aspects of a study by utilizing a moodle forum (ForumMG).
• Give clear and concise feedback using a curated discussion rubric.

Note: This post was created using work done by Nina Bailey and Allison W. McCulloch from UNC Charlotte.

Reference: 
Gal, I (2002) Adults’ statistical literacy: Meanings, Components, Responsibilities. International statistical review. 70(1), 1-25

Justin Post is a teaching associate professor and director of online programs in the Department of Statistics in the College of Sciences. He can be reached at justin_post@ncsu.edu.