The Method Section as Conceptual Epicenter

View Segments Segment :

  • Citations
  • Add to My List
  • Embed
  • Help
  • Citations
  • Add to My List
  • Embed
  • Help
Successfully saved clip
Find all your clips in My Lists
Failed to save clip
  • Transcript
  • Transcript

    Auto-Scroll: ONOFF 
    • 00:01

      PETER SMAGORINSKY: Hi, my name's Peter Smagorinsky.I teach in the College of Educationat the University of Georgia.I'm going to talk about an articlethat I've published in the journal, Written Communication,a proud SAGE journal.It was published in 2008 and is called,"The Method Section as ConceptualEpicenter in Constructing Social Science Research Reports."

    • 00:23

      PETER SMAGORINSKY [continued]: And there it is.And so the title's a little bit probably jargony.And so what I want to do is explain why I wrote thisand what I wrote about.So Chris Haas, who's the editor of Written Communication,issued a call for a special themed

    • 00:45

      PETER SMAGORINSKY [continued]: issue on research method.And I'm on the editorial board of Written Communication.And so usually an editorial boardgets special urging to think about contributingto such issues.And so I had been thinking about a problemfor a very long time-- which becamethe topic of this article-- whichis why method sections in most of the articles that I review

    • 01:09

      PETER SMAGORINSKY [continued]: are so bad.And I'd just like to backtrack and explainwhy I've read so many articles with bad method sectionsbecause most of what gets published is very shaped up,and that's what most people are familiar with coming across,which is finished, polished, reviewed, and edited products.But almost all of my reading is in the earlier stages

    • 01:32

      PETER SMAGORINSKY [continued]: of composition.I used to be a journal editor.Michael Smith and I co-edited Researchand the Teaching of English for seven years, 1996 to 2003.And we would get oh, 100, 125 articles a year,of which we would publish maybe a dozen, 10 to 12 articles.

    • 01:55

      PETER SMAGORINSKY [continued]: So 90% of what was submitted wasn't publishableor didn't get published and there were very consistentreasons for that.And then since then, I've gone onto be a very intensive reviewer for other journals.My experiences as a journal editor

    • 02:15

      PETER SMAGORINSKY [continued]: led me to understand the greatest asset to a journalis a reliable reviewer.And I also know that there aren't a lot of them.People turn them down, people accept themand take nine months to turn in a paragraph.I don't wish to suggest that everyone's like this.

    • 02:37

      PETER SMAGORINSKY [continued]: I see a lot of great reviewers, but journal editorsconsistently say they need more better reviewers.And so instead of reading books and articles,I read manuscripts submitted to journals.In 2014, the year that just ended,I just finished my annual review.So I can tell you I reviewed 49 articles for journals.

    • 02:59

      PETER SMAGORINSKY [continued]: I've reviewed as many as 61 or 2.So typically in a year I'll review45 to 60 articles for journals.And I keep seeing the same mistakes over and over.And I often recommend in my reviewthat people read this article.Because it helps them with one of the fundamental problems,

    • 03:19

      PETER SMAGORINSKY [continued]: which is that they've written a poor method section.So for reasons I can't quite fathom,the method section has gotten short shriftin how people learn how to write a social science researcharticle.People are very good at describing their datacollection.I went and I conducted 30 interviews.

    • 03:40

      PETER SMAGORINSKY [continued]: They were this kind of interview blah, blah, blah.And then they'll get to the analysis.And it's really the analysis of where the greatest shortcomingsoccur.And I should backtrack and say, mostlywhat I'm referring to is qualitative research.That's the kind of work that I do, broadly speaking.People who do statistical studies

    • 04:05

      PETER SMAGORINSKY [continued]: don't have the same sorts of challenges,I think, in writing a method section.Because if you say I ran an analysis of varianceon the data, that pretty much means the same thing no matterwho's running the ANOVA and what the data set is.The test is more or less the same every time.

    • 04:27

      PETER SMAGORINSKY [continued]: But if you say, I did a constant comparison analysis.I read through the data.I developed identify patterns.I reduced the patterns.And that's all they say.In other words, they don't say what the patterns were.They don't say what the codes were.

    • 04:48

      PETER SMAGORINSKY [continued]: And because they don't say what they were,they don't say what motivated the codes.In other words, you just can't say this is thisand this is that, it needs to have a theoretical consistencyto it.David Berliner came to campus a couple years agoand he gave a talk on research matters.

    • 05:09

      PETER SMAGORINSKY [continued]: And one of the things he said-- I justwanted to give standing ovation to-- he said,if you can't tell me how you analyzed your data,all you're doing is telling anecdotes.And I thought that was a very concise wayof stating the problem that I consistentlysee in qualitative research submitted social science

    • 05:29

      PETER SMAGORINSKY [continued]: journals.Because people can't tell me how they analyzed the data,I don't believe what they say in the findingsbecause it's not related to any kind of analytic methods.So what this article was an effort to dowas to explain that as a problem.And to give some examples of research

    • 05:50

      PETER SMAGORINSKY [continued]: in which a researcher does code in such a waythat it's aligned with the theoretical perspectivethat they have argued at the very beginning,frames the research.So this idea of alignment across the major sections

    • 06:11

      PETER SMAGORINSKY [continued]: of the paper, in my view, follows from how wellyou write the method section.Because often you'll see a theoretical frameworkwhich itself may or may not be coherent.There's often mix and match of theories without resolvingdiscrepancies.A method section that then doesn'toperationalize that theory in an analytic method.

    • 06:36

      PETER SMAGORINSKY [continued]: Rather, they just say, I read the data several timesand identified themes.And then you go to the findings and there are themes laid out.But if I don't know how they were generated,I don't believe the research.And there are other problems that follow for this.And I'm going to actually stray a little bit from this paper,

    • 06:56

      PETER SMAGORINSKY [continued]: because I just keep seeing the same problem in manuscriptsthat I review.In the findings report, social science research articlesare arguments.So an argument-- if you go back to Stephen Toulimn--an argument includes claims, it includes data or examples,

    • 07:21

      PETER SMAGORINSKY [continued]: and that's as far as most people taketheir presentation of findings.I saw a theme here.My evidence is that this person I interviewedsaid this, this, and this.If you can't explain-- and this iswhat an argumentative warrant is--if you can't explain how the example serves as evidence

    • 07:46

      PETER SMAGORINSKY [continued]: for the claim, you're not making a very powerful argument.That warrant has to be grounded in your own analysis of data.So if you don't explain it in the methods section,you can't possibly articulate it in terms of warranting evidencein the finding section.

    • 08:07

      PETER SMAGORINSKY [continued]: And so what people tend to do is say-- so they'llgive their claim, they'll give an example from the data,and then they'll have kind of a pseudo warrant.And I've referred to this as outsourcing the warrant.And by that I mean, instead of explainingwhy the example is evidentiary in this study,

    • 08:30

      PETER SMAGORINSKY [continued]: they'll say Smith and Jones found blah, blah, blah.So they'll take a claim from another study,import it, and say that that explanation that'sderived from a different study of something elseexplains why my example fits my warrant, rather

    • 08:51

      PETER SMAGORINSKY [continued]: than my own analytic system.And what I try to do when I review and Icome across this problem is I say,try writing the findings and the discussionwithout any references because then you'reforced back into your own analysisto make sense of your own data.And you're not just saying that what your claim is true

    • 09:12

      PETER SMAGORINSKY [continued]: because so and so said something similar in a different pieceof research.So a consequence of what I believeto be the need to explain a method section in detailis that people run up against page maximums provided

    • 09:34

      PETER SMAGORINSKY [continued]: by journals.And I think that this is somethingspecific to print journals.I'm the faculty advisor here at Georgiato the Journal of Language and Literacy Education--and I'm glad I got that plug-in-- and we'rea strictly online journal.We don't have a page max.We publish twice a year and our issuesare now running over 300 pages.We could have 3,000 page issues because we

    • 09:54

      PETER SMAGORINSKY [continued]: have unlimited space.But if you're a print journal, your publisher oftengives you a limit to the number of pagesyou're allowed to produce for the year.And as a result, editors will sayarticles can't be more than 25 pages,or they can't be more than 35 pages.So that really forces you to be very concise about something.

    • 10:16

      PETER SMAGORINSKY [continued]: It appears that that something is often the method section.I've argued against editors, and I'llcontinue to do so, that for qualitative researchto be explained well, it needs to havemore pages than journals tend to give them.And I think the quality gets underminedwhen the authors aren't allowed to explain their work

    • 10:39

      PETER SMAGORINSKY [continued]: in persuasive detail.That leads to us a related problem,which is the section of a paper of this sort.Usually in a qualitative study these days, you'reobligated to describe the context of the investigation,often in a separate section.The problem with the context is you could write 1,000 pages

    • 10:59

      PETER SMAGORINSKY [continued]: and it wouldn't be sufficient.So you have to be very selective in whatcomprises the context of the investigation.Often to the point where you're giving somethinginsufficient treatment.So there are just so many decisionsso you have to make as an author of a qualitative social scienceresearch report.

    • 11:20

      PETER SMAGORINSKY [continued]: That aren't your own decisions.In other words, they're constrainedby the length maximum you're provided by your editorsand/or the journal association.You're constrained by, of course, whatthe editors themselves want.Some may be very method oriented,others may not really care that much about it.

    • 11:40

      PETER SMAGORINSKY [continued]: And, of course, you're responsive to reviewers like mewho are going to hammer you if you don't explain your researchmethod.So I'd like to just summarise kindof this article and the points I've been trying to make here.Especially if you're a novice, but Ithink this applies to more experienced researchers

    • 11:60

      PETER SMAGORINSKY [continued]: as well.If you're going to write a social science researchreport that's data driven, a primary considerationis the alignment across the four sections of the paper.So you're obligated usually to begin with either a theoreticalor a topical overview.In other words, the theory might be,

    • 12:21

      PETER SMAGORINSKY [continued]: I'm using sociocultural theory based on the work of Vygotsky.That's a theoretical section.But you might also introduce it in terms of a topic,bullying as a crisis today.So one or both of those frameworks is in place.What I find is that the alignment across the method

    • 12:43

      PETER SMAGORINSKY [continued]: section and the findings and the conclusion isn't always there.And if you tell me that this is what your framework is,I'm going to look for that in the method section.And again, I'm going to look for it in the finding sectionbecause you can't argue from a data analysiswithout referring back to your analytic method.And that's where I think the argumentative warranting comes

    • 13:06

      PETER SMAGORINSKY [continued]: in.And then your conclusion.What I often find in conclusions is instead of people arguingmodestly-- which is really all you can do for most studies--from their own data, they go and tryto change the world in the concluding section.And I scold authors on that point.

    • 13:26

      PETER SMAGORINSKY [continued]: And say, just stick to what you studied.What does this study show, not howyou want to change the world.So there you go.I hope by engaging with this video you come outof a slightly better writer, or at least a more successfulwriter in terms of publication.And best of luck.

The Method Section as Conceptual Epicenter

View Segments Segment :


Professor Peter Smagorinsky describes a recurring problem that he sees in the qualitative research articles he reviews: the lack of an adequate method section.

SAGE Video Cases
The Method Section as Conceptual Epicenter

Professor Peter Smagorinsky describes a recurring problem that he sees in the qualitative research articles he reviews: the lack of an adequate method section.

Back to Top