Welcome back to Pack Hacks for Faculty. Each month, a member of the NC State faculty will provide quick tips, advice and other insight to facilitate your teaching, research, scholarship or engagement activities. If you are interested in making a submission for a future Pack Hacks for Faculty, please review our submission guidelines and contact email@example.com if you have questions.
This month, Justin Post, teaching associate professor and director of online programs in the Department of Statistics, gives advice on using discussion boards (like Moodle) to facilitate engagement on quantitative data.
Talking Quantitative Data Through Discussion Boards
Quantitative methods and analysis have become pretty ubiquitous across disciplines in recent years. All fields from the sciences to the humanities have some aspects that revolve around analyzing data, discussing conclusions from studies that use data, and generally using data to make or backup an argument. The skill of thinking critically about all aspects of a study involving data is foundational to our quantitative literacy competency.
How can we help our students to think critically about data? Specifically, how can we help students to consider the implications of the method of data collection and the appropriateness of the analysis on the validity of the conclusions made?
Many of our classes are already packed full with material. This article describes a minimally invasive way to infuse these critical quantitative literacy skills into your course using an online discussion forum!
The quality of data used in quantitative studies varies greatly. What is meant by this is that data are often used to predict future outcomes or make conclusions about some larger population, but the data is not always representative of that population. Data may have biases, leave portions of the population out, include measurement error, or not include relevant variables that would make relationships clear. These aspects of a study should always be considered when judging the relevance of any conclusions made.
Aside from data quality, the appropriateness of any statistical methodologies should also be critiqued. Were data representing categories but coded numerically summarized with a mean? Are graphs clear or misleading? Was the model used checked for violations of assumptions?
Lastly, one should always consider whether claims made based on data are sensible, especially in light of the above considerations about data quality and analysis method used. Have the authors of the study implied conclusions that can’t be supported from the data? Has the media taken a well designed and analyzed study and improperly made their own conclusions, say using correlation to imply causation? Are there other relevant things to consider that make the conclusions carry less weight?
To help students think about these aspects and form quantitatively sound reasoning, we can give them questions to consider about any study. The following questions are adapted from Gal (2002) and a student can be imagined running through this type of list and asking how the questions are relevant for the current situation or task.
- Where did the data come from? What kind of study was it? Is this kind of study reasonable in this context?
- Was a sample used? How were participants selected? Did the sample include people/units that are representative of the population? Overall, could this sample reasonably lead to valid inferences about the target population?
- Are the reported statistics appropriate for this kind of data? For example, was an average used to summarize ordinal data. Could outliers cause this summary statistic to misrepresent the true picture?
- Is a given graph drawn appropriately, or does it distort trends in the data?
- Overall, are the claims made here sensible and supported by the data? For example, is correlation confused with causation? Is a practically meaningless difference made to loom large?
- Should additional information or procedures be made available to enable me to evaluate the sensibility of these arguments? For example, did the writer “conveniently forget” to specify the base of a reported percent-of-change, or the actual sample size?
- Are there alternative interpretations for the meaning of the findings or different explanations for what caused them? Are there additional implications that are not mentioned?
- How reliable or accurate were the instruments or measures (tests, questionnaires, interviews) used to generate the reported data?
- How was this probabilistic statement derived? Are the authors making poor assumptions that make the statement invalid?
Discussion boards are widely used across the internet. The basic idea is that someone posts a comment or question and others can respond. Discussion forums can be easily added to a moodle site and provide a great way to spark a discussion around quantitative literacy.
An effective way to use discussion boards is to break students up into small groups of 3 to 5. The instructor provides a prompt or question for the students to consider. The out of class nature gives students time to think and form a relevant argument. Once these initial posts are done, the other members of the group then need to respond by supporting or critiquing it. This activity can promote critical discussion between students. Collaboration can be promoted by requiring a final post upon which the group agrees.
If you are new to groups and groupings in moodle, see here for a quick tutorial.
If you are new to discussion forums in moodle, see here for a quick tutorial.
Writing a good discussion prompt can be challenging! A good prompt should provide clear expectations to the students as well as promote discussion of quantitative data and interpretation. In the prompt the instructor can give the relevant quantitative context from an article to allow students’ to thoroughly engage the worry questions or other directed questions.
An example forum prompt is given here.
One issue with using discussion boards is grading the efforts of students. What makes a strong post? It is really useful to use a well-defined rubric for what you are looking for in posts and providing this to students gives a transparent method for grading. This can remove subjectivity and help to provide clear expectations of what is a quality post.
Given here is an example rubric that could be modified for your purposes.
|Grade||Posting Rubric||Response Rubric||Points|
|Exemplary (A)||Post is based on logical reasoning grounded in quantitatively sound ideas. Draws from multiple sources. Presents questions or comments that open the discussion for debate.||Challenged a classmates' thinking, asked insightful and thought provoking questions to a classmate, AND responded to questions or comments that were made by a classmate’s post.||4|
|Above Average (B)||Post is based on logical reasoning rooted|
in quantitatively sound ideas. Few opinions exist in post. Draws from credible sources and/or demonstrates sound reasoning.
|Agreed or Disagreed with a classmate with justification but did not challenge the ideas presented.|
Asked insightful questions but did not respond to questions asked by classmates or engage in a back and forth.
|Average (C)||Post demonstrated some quantitatively sound|
connections; perhaps a mixture of reason and opinion. Justifications were not detailed.
|Response attends to quantitatively sound ideas|
generally with perhaps a mixture of reason and opinion. Justifications were not detailed.
|Fair (D)||Post lacked connections to quantitatively sound ideas and/or relied on opinions without justification.||Response lacks focus or clarity. Response heavily depends on opinion. Response does not refer to quantitatively sound ideas and/or credible sources.||1|
|Unsatisfactory (F)||Did not post.||Did not respond.||0|
- Discuss some important questions our students should ask when faced with a quantitative study.
- Implement a critical thinking discussion between students about quantitative aspects of a study by utilizing a moodle forum (ForumMG).
- Give clear and concise feedback using a curated discussion rubric.
Note: This post was created using work done by Nina Bailey and Allison W. McCulloch from UNC Charlotte.
Gal, I (2002) Adults’ statistical literacy: Meanings, Components, Responsibilities. International statistical review. 70(1), 1-25
Justin Post is a teaching associate professor and director of online programs in the Department of Statistics in the College of Sciences. He can be reached at firstname.lastname@example.org.