A Linear Approximation of the Unequal Variance Likelihood Ratio Transformation and a Linear Ballistic Accumulator Model Extension

View Segments Segment :

  • Citations
  • Add to My List
  • Embed
  • Link
  • Help
  • Citations
  • Add to My List
  • Embed
  • Link
  • Help
Successfully saved clip
Find all your clips in My Lists
Failed to save clip
  • Transcript
  • Transcript

    Auto-Scroll: ONOFF 
    • 00:00

      [MUSIC PLAYING]

    • 00:09

      ADAM OSTH: So first, I'd like to acknowledgethe work of my collaborators.There's Andrew Heathcote right herein the audience from the Universityof Tasmania and Simon Dennis, who, unfortunately, is not herefrom the University of Newcastle.So my talk is going to be focused on sequential samplingmodels, which are decision making modelsthat can account for both response time and choice

    • 00:29

      ADAM OSTH [continued]: probabilities.The particular one I'm going to be talking aboutis the drift diffusion model, whichdescribes a kind of gradual, noisy accumulation of evidenceuntil terminations at one of the response boundaries.The boundary that gets reached is the choice.And the time that gets lapsed is the response time.

    • 00:49

      ADAM OSTH [continued]: And one of the critical parametersof the model, especially the one that'sgoing to be the focus of the talkis the drift rate, which describesthe overall rate of accumulation of the evidence.And increases in the drift rate both increasethe number of correct responses and quickens the responsetimes.Now, one of the big introductionsfrom the seminal Ratcliff '78 paper

    • 01:12

      ADAM OSTH [continued]: was the introduction of cross trial variabilityin drift rates.And the basic idea is that drift rates aren't justfixed for a given condition, but are sampledfrom a normal distribution.And so all conditions would essentiallyhave their own distributions.What gets missed in a lot of descriptions of the diffusionmodel is that this is basically inspired by signal detection

    • 01:34

      ADAM OSTH [continued]: theory, where you have, basically,overlapping distributions of evidence for both targetsand foils that are sampled on every trial.And this marriage of the two approachesessentially solved the problems with each of them.Signal detection theory had difficultyapplying to response times.And diffusion models had difficulty accountingfor the relative speeds of correct an error responses.

    • 01:54

      ADAM OSTH [continued]: So given this unification, it's somewhat ironicthat over the next 20 or so yearsdevelopment of both single detection models and diffusionmodels occurred, to some degree, in parallel with each other.So for instance, in signal detection theory,one of the advances was the rejection of the equal variancesignal detection model.And this was based on the increasing awareness

    • 02:17

      ADAM OSTH [continued]: of the fact that zROC slopes are less than one.They were technically found before then,but there wasn't really a lot of attention on that fact.And one of the other ones was the usageof likelihood ratio transformationsto capture the mirror effect.This was something that was popularizedby Glanzer and colleagues due to the finding of the mirroreffect.Whereas in diffusion models, there

    • 02:38

      ADAM OSTH [continued]: were some advances such as the introductionof variability in the starting point of accumulation, whichwas critical for completing the account of the relative speedsof correct an error responses.And then there was also the introduction of variabilityin non-decision times.So some of this changed with the work of Starns and Ratcliff.

    • 02:59

      ADAM OSTH [continued]: So anybody that's familiar with signal detection theoryknows that the measurement of the relative variabilityof the target distribution is usually accomplishedby constructing an ROC.So essentially, you need some biased manipulationor collection of confidence ratingsin order to measure this.Starns and Ratcliff argue that given

    • 03:19

      ADAM OSTH [continued]: that you can identify the variance parametersin the diffusion model just with a response time distribution,you can essentially measure these various parametersa different way, just using yes/no tasks with no biasmanipulation.And so that's essentially what theydid was they fit nine recognition memory datasetsand allowed the variability to go free across targetsand lures.And what they found was that the response time distribution

    • 03:41

      ADAM OSTH [continued]: supported the same conclusions as ROC studies, whichis that you have greater variability for targetsthan foils.So this gave kind of additional supportfor the unequal variance model and, essentially, almostlike a rejection of the equal variance signal detectionmodel in the case of the diffusion model.So what the present work is going to be focused onis the application of these likelihood ratio models, which

    • 04:01

      ADAM OSTH [continued]: haven't really been explored a great deal, I should say,in diffusion models-- or at least for the applicationsI'm going to be talking about.So in a likelihood ratio model-- solet's say, for instance, you have a simple signal detectionmodel, where you have memory strength for threedifferent classes of items.You've got lures.You have-- can I get the mouse up here?OK.So you have lures.

    • 04:22

      ADAM OSTH [continued]: You have weak targets.And you have strong targets.Now, what you can do essentially hereis you can run these through a transformation.So let's say each of these distributionsI'll just denote as x.So I take samples from any of these distributions.And what I can do is I can take the relative oddsthat this sample came from an old distributionrelative to a new distribution and then take the log.

    • 04:45

      ADAM OSTH [continued]: And what I get out of it are a series of log likelihood ratiodistributions for targets and lures in boththe weak and strong conditions.Now, this might not make a lot of sense initially.But let me go through it.The psychological idea behind thisis that items are not just taken based on their strength alone,but some meta-memory process is involved, which

    • 05:07

      ADAM OSTH [continued]: is not technically specified.So some meta-memory process is involvedthat compares items to their expected memory strengths--so some expectation of how strong or weak the item shouldbe based on the condition.Now, if you look at these distributions--so in the likelihood ratio, after the transformation,you can see that the strong targets are ahead

    • 05:27

      ADAM OSTH [continued]: of the weak targets.That's something you already had to begin with.So that's not really that special.The critical thing is what happens to the lures.If you look at the strong lures, theyare further down on the decision axis than the weak lures.And the reason why is by virtue of this numerator.Now, when you take the lures, the luresare the same technically in the weak and strong conditions.But the difference is that in a strong condition,

    • 05:48

      ADAM OSTH [continued]: they're being held to a higher expectation in this numeratorterm.They're being compared to the strong distribution.And that pushes them further down on the decision axis.Now, what does this do?This gives you the mirror effect.The mirror effect was popularized by Glanzer.And the idea here is that many manipulations-- not necessarilyall, but many manipulations-- produce opposite effects

    • 06:10

      ADAM OSTH [continued]: on hit and false alarm rates.Some of the more popular manipulationsare word frequency.So low frequency words have higherhit rates and lower false alarm ratesthan high frequency words.But you can find this for several others as well.So another one I'm going to be focusing on todayis study time and repetitions when manipulated across listshave the same effect.And what you can see here-- so I'mplotting hits and false alarm rates

    • 06:30

      ADAM OSTH [continued]: for the weak and strong conditionsaccording to these distributions.You can see why it's kind of called the mirror effect,because when you look down here at the false alarm rates,the progression looks like a mirror image of whathappens with the hit rates.So that's why it gets the name.But there's another prediction thatalso emerges from this as well.So one thing you can also see in these strong distributions also

    • 06:50

      ADAM OSTH [continued]: have higher variance than the weak distributions.This can be tested with zROCs.And this prediction has actually been confirmed.Conditions of higher performance tendto have higher variance than conditionswith weaker performance.So if you compare new items from the weak condition to new itemsfrom the strong condition, if you compare those false alarmrates, you tend to get zROC slopes that

    • 07:12

      ADAM OSTH [continued]: are different from one and reflect higher variabilityin a strong condition then a weak condition.There was a paper published recently by Glanzer, Hilford,and Maloney that looked at this for a wide rangeof manipulations and confirmed this predictionof the variance effect.Now, the other thing I'd like to mention about likelihood ratiomodels that make them cool is that nearly all process

    • 07:34

      ADAM OSTH [continued]: models of recognition memory utilize this transformation.So some of the more popular ones, such as for instance REM,and [INAUDIBLE] and SLiM, and also our modelthat was published just this yearin Psych Review, which is just a little bit of shamelessself-advertising.But most of the retrieval models of recognition memoryutilize this transformation.Now, the current approach is basicallyto just use this likelihood ratio transformation

    • 07:56

      ADAM OSTH [continued]: with the diffusion model.Now, the really cool thing about this when you apply it to datais it's extremely simple.There's only a few parameters youneed to estimate to get basically the full mirroreffect.Basically, you have a deep parameterfor the weak condition, as well asa deep parameter for the strong conditionand another parameter, following the Starns and Ratcliffe work,

    • 08:19

      ADAM OSTH [continued]: showing that there's unequal variance.We're having another parameter thatrepresents the ratio of standard deviations.And then what it gives you after you've run this transformation,you have mu and sigma parameters for the weak and strongconditions.So essentially, 8 parameters that it gives youthat you can just plug into the diffusion modelto make these predictions.

    • 08:39

      ADAM OSTH [continued]: And what it gives you is it's compelledto predict these two patterns-- the mirror effectand the variances effect.Now, what we're doing is we're doing a model comparison study,where we're comparing this to a model that'svery similar to the model that Starns and Ratcliff advocated,which is an unconstrained signal detection model.So in this case, we are using mu and sigma parameters-- now,these are corresponding to the drift

    • 09:00

      ADAM OSTH [continued]: rate distributions-- for targets and luresacross the weak and strong conditions.And so this is six parameters instead of three.So it's a more complex model.Now, the thing I want to mention about the model selectionis it's really not guaranteed to favor the likelihood ratiovariance for a couple reasons.

    • 09:20

      ADAM OSTH [continued]: One is that Starns and Ratcliff actuallyallowed variance to vary across the weak and strong conditionsand across different frequency classes and so forth,and they found no change in variance.So it really only varied across targetsand lures and didn't change across anyof the other conditions.So that's already something that mightbe counting against the likelihood ratiomodel, given that it predicts this increase in variance.

    • 09:42

      ADAM OSTH [continued]: And then the other thing that could count against itis that it's strongly compelled to predict the mirror effect.And what that means is that any extent to which the mirroreffect might be an averaging artifactor if there are any subjects which do not show this effect,those are going to be more poorly fit by the likelihoodratio model, whereas the unconstrained model hasthe wiggle room to be able to accommodate these subjects.

    • 10:03

      ADAM OSTH [continued]: So we used a total of six data setsacross two different papers.The first was by Rae, Heathcote, et al.And this used a word frequency manipulationto capture the mirror effect.But it also included a speed accuracy manipulation.And then there were five datasetsfrom Starns, Ratcliff, and White that were relatively vanilla.

    • 10:24

      ADAM OSTH [continued]: There was no bias manipulations or threshold manipulations.It was just a repetition manipulationthat induced the mirror effect.And there were five total datasets there.We estimated the parameters of both models usinghierarchical basin analyses, using the differentialevolution algorithm.I don't have full time to give a proper treatment

    • 10:47

      ADAM OSTH [continued]: of hierarchical base.But the basic idea is you get estimates of both groupand subject level parameters simultaneously.We didn't use quantile summaries.We fit these to all the response timeswith the exception of some very sensible exclusions.Before I go into the fits, I just kind of wantto show you something I think is kind of cool, whichis the ratio of target to lures standard deviation estimates.

    • 11:08

      ADAM OSTH [continued]: So what you can see here, these are posterior distributionson these parameters depicted as violin plots.So basically, the areas that they're greater,that means there's higher likelihoodthat that's where the parameter is.One indicates equal variance between targets and lures.And any values over one indicate greater variabilityof the target distribution.And what you can see here-- so in black

    • 11:29

      ADAM OSTH [continued]: is the unconstrained model.And in red is the likelihood ratio model.Both models converge on the same conclusions,that there's greater variability for targets than for lures.So it looks like we're pretty safe.It doesn't seem to depend upon the choice of model here.But the other thing that's kind of interestingis that most estimates are considerably higherthan the values that are estimated from the ROCliterature.They're much higher than 1.25.

    • 11:49

      ADAM OSTH [continued]: 0.25 So that's just kind of a cool thing.Moving on to the model fit.I don't have time to show all the fits.But I'm just going to show the fits relatively quickly.So we have the choice probabilities on the left.And then we have the correct response times and errorresponse times on the right.Just showing some quick quantile summaries for the 0.1, 0.5,and 0.9 quantiles.So you can see the mirror effect over here on the left.

    • 12:11

      ADAM OSTH [continued]: Now, first up, we have the unconstrained model.So what you're seeing here is the full posterior predictivedistributions for all the different points.The unconstrained model is fitting very well.The only misfit in this particular datasetis its missing on the leading edgeof the correct responses a bit.But aside from that, it's fitting the data extremelywell, which is what we would expect.

    • 12:32

      ADAM OSTH [continued]: It has free parameters for a lot of these different conditions.The likelihood ratio model's fitting worse.It still fits the response times quite well.Very similar to the unconstrained model.You can see that there are some misfit on the choiceprobabilities.We expect this.It's more constrained than the unconstrained model, go figure.So we would expect it to fit to some degree worse.The question is whether or not it performs better

    • 12:52

      ADAM OSTH [continued]: in the model selection.So what we used to select the modelsis we used the Deviance Information Criterion, or DIC.This has three different values that are relevant.First up is D, which is the Deviance, whichis the badness of fit.So lower values are better.We have PD, which is the effective number of parameters.This is a little different than parameter countsis like in AIC and BIC.

    • 13:14

      ADAM OSTH [continued]: This gives an estimate of the total contributiona parameter makes.So it's not just the number of parameters,but the overall kind of simplicity or tightnessof the model's predictions.And then finally, we have DSC, which is the overall modelselection score.Lower values are better.So I have all six datasets listed here horizontally.

    • 13:34

      ADAM OSTH [continued]: On the left is the unconstrained model.And on the right is the likelihood ratio model.First up is the deviance estimates.So not surprising, we get better estimates,we get lower deviance for the unconstrained model.Again, it's fitting better.When you look at the effective number of parameters,the likelihood ratio model, as predicted,is way simpler than the unconstrained model.

    • 13:55

      ADAM OSTH [continued]: It has much lower estimates of PD.And then, finally, we have DIC.And the cool thing here is that across each and every datasetwe get a very big advantage for the likelihood ratio modelover the unconstrained model.Now, what does this mean?What this means is that we essentiallyget a selection of this likelihood ratiomodel over the unequal variance model.

    • 14:16

      ADAM OSTH [continued]: And it suggests that this is a simpler account of the responsetime data.Most subjects seem to be showing the mirror effect.The response times are also seen to be concordantwith this transformation.The other key take home message hereis that the success of the likelihood ratiomodels in this particular datasetshow some promise in extending process models to the domain

    • 14:37

      ADAM OSTH [continued]: of response times.This is something I've just recently gotten into.We fit our new model that just came out to some response timedata recently.And the data is actually fitting extremely well.So hopefully, in the next year or twoI'll be able to come back and give a presentation on that.And yeah, thank you.[APPLAUSE]

A Linear Approximation of the Unequal Variance Likelihood Ratio Transformation and a Linear Ballistic Accumulator Model Extension

View Segments Segment :

Abstract

Dr. Adam Osth presents his research into the drift diffusion model of decision making. He explains how the inclusion of new parameters and signal detection theory has led to better models.

SAGE Video Forum
A Linear Approximation of the Unequal Variance Likelihood Ratio Transformation and a Linear Ballistic Accumulator Model Extension

Dr. Adam Osth presents his research into the drift diffusion model of decision making. He explains how the inclusion of new parameters and signal detection theory has led to better models.

Copy and paste the following HTML into your website

Back to Top