[MUSIC PLAYING]

JOE TWYMAN: My name is Joe Twyman.I'm head of political and social researchfor Europe, the Middle East, and Africa here at YouGov.I also lecture in survey research methodsat the University of Cambridge and the University of Essex.I'm visiting research fellow at the University of Manchester.I'm visiting professor at the University of Sheffield.YouGov has offices all over the worldand over 600 employees in 22 different countries.

JOE TWYMAN [continued]: We specialize in opinion research--all different types of opinion research.But we're most famous for our political research.My hope is that by doing this video I'llbe able to draw on my 15 years of experiencein political research but also opinion researchmore generally to give an idea of the kind of thingswe do to make our work accurate and representative.

JOE TWYMAN [continued]: YouGov is famous for its online research, in other words,using the internet to conduct research that's representativeof different groups, for instance, entire adultpopulations of specific countriesor specific groups within those countries or people whowork for a particular company or peoplewho use a particular product.And we were really in Britain the first company

JOE TWYMAN [continued]: to pioneer this service method.The way it works is that we recruit aroundabout half a million people to our panel, the YouGov panel.And we know an awful lot about those people.We have literally millions of data pointsacross all those people.And what that allows us to do is draw populations

JOE TWYMAN [continued]: that are representative of the wider population.In other words, we draw a sample thatis representative of the country as a wholerather than, for instance, our panel itselfor the internet population.In order to do that, we record a whole hostof sociodemographic information about our respondentswhen they first register.

JOE TWYMAN [continued]: And then we continue to add to that data as they go along.And so we collect, for instance, their age, their gender,their region, which parliamentary constituencythey live in, how many children they have, the type of workthey do.And from that, we build up this detailed pictureof that individual respondent.And then when we want them to take part in a survey,

JOE TWYMAN [continued]: we send them an email.And we say, you have a survey waiting for you.And we only contact those people we want to take part.And we only contact them once.And crucially, only those people who are contactedare allowed to take part.And they are only allowed to take part once.And so what that means is that wecan contact the right number of old people,the right number of young people,

JOE TWYMAN [continued]: the right number of men, the right number of women,to be representative of the country as a whole, not justthe internet population, or indeed,the people on our panel.YouGov's hugely proud of the relationshipthat it has with its panelists.We spend a lot of time, and indeed, a lot of money

JOE TWYMAN [continued]: collecting data, but also involving ourselvesin a two-way conversation with panelists.And so we, for instance, have a websitewhere people can go and look at the resultsthat we've published so they can see how their results have beenor have been used and where they are being seen.And people can take part in lots of different types of surveys.We run fun surveys on the site, for instance,

JOE TWYMAN [continued]: where people can see how their results comparedto other people who've answered the survey.And we spend a lot of money on panel recruitment.We spend hundreds of thousands of pounds a yearrecruiting people to the panel to make sure that we don't justget people who are particularly interested in somethingor who have an axe to grind.Instead, we want "normal" people to sign up to the service.

JOE TWYMAN [continued]: And so we spend a lot of time at that.And we spend a lot of time on panel care.And so people who have questions,who have issues, or just want to chat,can write in to our panel care teamand talk about the kind of questionsthat they're interested in.And they'll write back and be able to help themas far as they can.

JOE TWYMAN [continued]: In terms of the analysis that we do,it's hugely varied depending on the type of work that we do.And the type of work we do is determined by our clients.We're entirely client led.And so if someone is interested in a particular formof research, particular form of analysis,then that's something we'll pursue.In terms of software, we are particularlyfond of things like R and SPSS, these standard software

JOE TWYMAN [continued]: for the industry.But in terms of the kind of analysis that we do,it's everything from multivariate analysis,complicated statistical modeling, to basic tables,the kind of thing that you would see in a newspaper or on TV.

JOE TWYMAN [continued]: Because their accuracy is a good one, because of course,it's very rare that a research agency, any research agency,has the opportunity to compare their predictionsagainst actual outcomes.Now, of course, a key opportunity to do thatis in elections, particularly general elections.But they, particularly in this country,don't come around very often.And only about 5% of our work is political in nature at all.

JOE TWYMAN [continued]: The remaining 95% nobody ever hears about.It's pensions work.It's pet food work.It's shampoo analysis.It's all the work conducted with commercial clientsthat you never see, because people use it for internal use.And so because they're using it for their internal use,they don't want to publish those results.They want to give them an added competitive advantage

JOE TWYMAN [continued]: over other people in their industry,or other products, or other companies.Now, when we do get that opportunityto compare against actual outcomes, we take it.And so that can be elections or it can bethings like reality TV shows.In the past, we've done X-Factor going all the way backto Pop Idol in 2001, the first time that we accuratelypredicted a winner in a reality TV contest.

JOE TWYMAN [continued]: Now, most of the time, we get things right.But it's fair to say that we, along with allthe other agencies, don't always get it right.And in 2015, for instance, we weren't as closeas we'd hoped to be with the British general election, whichwas disappointing, particularly coming offthe back of the European elections in 2014

JOE TWYMAN [continued]: where we've been extremely close.We then came back later in 2015 with an accurate predictionof the Labour Party leadership election.But it just goes to show that, yes, while we don't alwaysget everything right, we certainly don't alwaysget it wrong either.

JOE TWYMAN [continued]: As everyone who's involved with politics knows,there are ups and downs, perhaps even peaksand troughs you might say.And yes, it was hugely disappointingwhen we weren't as close as we'd hoped to be in 2015.And so yes, that was a disappointment.But from that, we were able to identify the thingsthat we have got right.

JOE TWYMAN [continued]: For instance, the big story of the entire last Parliamenthas been about the rise of UKIP and howit was going to be possible to predict how much theywould get in the ballot box.We got that pretty much spot on.The same with the Lib Dems, the Greens.It was simply that balance between Conservative and Labourwhere we weren't right.And so what did we do?We did exactly what we do after every election,

JOE TWYMAN [continued]: after every prediction.We go back and we assess where we went right,and where we went wrong-- where our strengths liesand where our weaknesses emerged.And it gave us an opportunity, fortunately,to go back over millions of data points.We conducted surveys every single dayduring the last Parliament.And so we had a lot of data to analyze,

JOE TWYMAN [continued]: which allowed us to make data-driven conclusionsabout where we thought we were going to go in the future--where we needed to just our processes,where we needed to tweak things, wherewe needed to improve things.And so the strength of our method is that we can go back,and we can look at people.And we can look at their individual journeysacross the Parliament to then make judgments about wherewe can improve next time.

JOE TWYMAN [continued]: Well, it depends on what stage you're talking about.Any poll is simply a snapshot of opinion at that time.And so you won't run a poll at the moment,for instance, to predict what the result is goingto be at the next election.So much can happen.A week is a long time in politics.And there are many weeks remaining.And so lots of things can happen there.Instead, it's a snapshot of where we are at the moment.

JOE TWYMAN [continued]: Now, when you get very, very close to an election,yes, you hope that your poll that you're taking at the timewill effectively model how peopleare going to vote when they get to the ballot box.But of course, things can things can change.In the Scottish referendum, we saw evenon the day people were changing their mind.And we could see this because we were tracking data

JOE TWYMAN [continued]: at an individual level.And so we could see people who had said that theywere going to vote, yes.They were going to vote to leave the Union.They said two, three, four days out, I'm going to vote yes.I'm going to vote yes.I'm going to vote yes.But on the day, they came back to us and said, I voted no.And so things can change.And so you're always thinking about what

JOE TWYMAN [continued]: that prediction actually means.And even if you can get the share of the vote right,then getting that to translate in the British system in termsof number of seats in Parliament is extraordinarily difficult.And some of the greatest academic minds in this country,the finest political scientists the world has to offer,have done a pretty poor job of predicting that in the past.

JOE TWYMAN [continued]: And so it's very difficult whoever takes that on, becauseof the vagaries and idiosyncrasiesof our electoral system.The question of whether polls influence votes or notis quite a difficult one to test.If you ask people you influence by things,quite often they say, no, even whenit comes to very obvious things like advertising.

JOE TWYMAN [continued]: People say, oh, no, no.I never pay attention to advertising,which of course, raises the question, well,why do people bother paying money for advertising?The answer is we know that it does influence things.But it's very difficult to test.With polling, my sense is that what it does is informs people.It informs people of the consequences of their action.I don't think a lot of people will change their mind based

JOE TWYMAN [continued]: on polling.I don't believe that people think,all right, well, given what's happening,I'm going to change my mind.Instead, it will inform some of the consequences.Let's take the Scottish referendum as an example.We did a poll two weeks out from the referendumshowing that it was neck and neck between yes and nowithin the margin of error.

JOE TWYMAN [continued]: This was presented by our clients at the newspapersas yes was ahead.But actually, it was in the margin of error.But that created a lot of noise and a lot of interest.Now, what happened as a result of that?Well, firstly, the political class went crazy.And suddenly, lots of politicians, everyonefrom David Cameron to Gordon Brown,

JOE TWYMAN [continued]: started making concessions.They started saying, this is what we're going to do?Trust us.Don't worry about it.We're going to do all this nice stuff for you.Now, was it the poll that influenced them?And did that then influence voters?Or were voters influenced by the poll,and that just linked into it?It's very difficult to say.But what the poll did say to peoplewas the situation has changed.

JOE TWYMAN [continued]: The situation has changed, and it's fluid.And it's worth pointing out that a number of other polls,not just ours, but from other companies,subsequently backed up that finding thatwas very close between the two.What that showed, all that polling showed,was that there were consequences to your action.If you wanted to stay part of the Union,you had to go out and vote because the polls made that

JOE TWYMAN [continued]: clear that it was close that if you chose not to,but you still wanted to be part of the Union,then you have to live with the consequences.And similarly, if you wanted to leave, you had to get out.And you had to vote.And so it demonstrated that therewas a difference to be made.It provided that information to the entire democratic process.

JOE TWYMAN [continued]: And what happened?An 85% turnout-- virtually unheardof in the modern political era.It informed people there were consequences,and they made actions and made decisions based upon that.And I think that's good for democracy.The Scottish referendum in 2014 was a vote

JOE TWYMAN [continued]: among the Scottish people, people living in Scotland,about whether they wanted to staypart of the British Union, part of the United Kingdom, or not.Voters have a choice.They can either vote yes to leave the Union.Or they could vote no to stay part of Great Britain.Now, for a long time, the no campaign was well ahead.

JOE TWYMAN [continued]: And things didn't really move.But in the last couple of months in August and in September,the situation changed dramaticallyas the campaign started to have an effect,and the gap closed dramatically.What then happened, and you see thisin a lot of referendums of this type,there was then a backlash against that moveaway from change and back towards the status quo.

JOE TWYMAN [continued]: And in the end, the country voted to stay part of Britainby a majority of 55 to 45.People vote for a whole host of different reasons.The decision that people make varies from person to person.

JOE TWYMAN [continued]: And a lot of people, idealists youmight say, like to think that there are people out there whovery carefully download, or indeed order,a copy of each party's manifesto and read each one from coverto cover, and then make a judgement based on the partypolicies.It doesn't really work like that.People, the average person on the street,and the 50% of people less engagedthan the average person on the street,

JOE TWYMAN [continued]: doesn't pay much attention to politics at all.Instead, they're interested, if they'reinterested at all, in these sort of grand narratives,these stories that we tell ourselvesand each other about the different politiciansand the different political parties.Who do I trust?Who can be effective?Who can be right for me?Who can do a good job?

JOE TWYMAN [continued]: These are all questions that we ask ourselves.And actually, it doesn't come down to specifics on policy,because everyone, for instance, wants the economyto get better.Everyone wants their children to be well educated.Everyone wants a health system that provides effective healthfor them and their families.And so you look to the different partiesto develop those narratives.

JOE TWYMAN [continued]: And what's very important in developing those narratives,in a lot of cases, is the leaders.And so the leaders and how they behaveand the sort of things they say, determinewhether people think that the party can be effective,can be trusted.And so David Cameron, for instance, at the last electiondid not do particularly well.But at the same time, he didn't do nearly as badly

JOE TWYMAN [continued]: as Ed Miliband, who was seen as weak and a bit weird.Or Nick Clegg, the leader of the Lib Dems, whowas seen as untrustworthy, because he'dgone back on his promises on tuition fees.Then you have Nicola Sturgeon, whowas this new and exciting person, whoreally most people had never heard of priorto the election itself.Her and her party, the Scottish National Party,

JOE TWYMAN [continued]: dominated the agenda in Scotland ahead of the general electionand then dominated the results on election night,winning so many seats, dominating the voting.Now, why was that?Well, she presented change.She presented something different,something away from the established orderof Westminster politics that had become

JOE TWYMAN [continued]: unpopular across Britain, but particularly northof the border.And prior to the election, you had the Conservatives, Labour,and Lib Dems all supporting being part of the Union,which had got 55% of the vote in the referendum.Then you had the SNP being the only anti-Union approach.And they had got 45% of the vote.

JOE TWYMAN [continued]: So they got all of that was the other splitthe stuff between them.And again, because of the idiosyncrasiesof our electoral system, they were able to reap the benefitsand now have 40-odd seats in Parliament to play with.I question whether it was right to say that SurveyMonkey

JOE TWYMAN [continued]: were accurate with their predictionfor the general election.They have no track record in British politicsand emerged with this one-off poll just aheadof the election.They didn't get much pick up at the time.Now, it's true that the gap between Conservative and Labourwas better in the SurveyMonkey poll.And so the overall story for Conservative and Labour was

JOE TWYMAN [continued]: perhaps clearer to the truth.They were four points out on Conservatives, the same as us.They were three points out for Labour,which was the same as us.And their prediction for the Greenswas double what the Greens actually got.And they were also much higher on the SNPthan they actually were.And not so good on UKIP either.And so, yes, they got some bits right.

JOE TWYMAN [continued]: But they got a lot wrong as well.Now, what that demonstrates to me is that political polling,whether it's in Britain or Americaor elsewhere, like a lot of other polling,is very difficult. And it's becoming more difficult.I think that online research will play an important rolein the future, because online research has always

JOE TWYMAN [continued]: had to adapt to the constantly changing circumstances in whichit exists.By definition, not everyone is on the internet.Yes, nearly 90% of people in Britain are.But you always have to make account for the factthat those 10% aren't and think, OK,how do we make sure that people who don't have the internetare sufficiently represented?

JOE TWYMAN [continued]: Now, telephone research is also now startingto face the same problems.Back in the '70s, but particularlythe '80s and the '90s when telephone researchwas at its high point, landlines were virtuallyubiquitous and mobile phones unknown.But as time has moved on, the situationhas become more difficult. And now, landlinesare becoming increasingly uncommon, particularly

JOE TWYMAN [continued]: for younger people.And so young respondents in a surveyare more difficult to contact by phone.You have to include a degree of mobile phonecoverage in your surveys.To do that with random dialing is illegal in the US.It's very difficult to do in the UK because of caller ID.And so if people get an unknown number,they just send it straight to voicemail.

JOE TWYMAN [continued]: And so the point is that contact rates, response ratesfor surveys of any method, are lower and lower and loweras years go on.And so that means the industry as a wholewhether it's political research or whether it's researchinto pet foods, pensions, and shampoohas to find ways to get people engaged, to get peopletaking part in those surveys, but also

JOE TWYMAN [continued]: to make sure that people who are taking partare truly representative.Response rates for telephone research are below 5%.And that's 5% among the entire population.Among certain groups within the population,they are significantly lower than that.And so you have to ask yourself, well,the 95% of people who aren't taking part in surveys--

JOE TWYMAN [continued]: are they different from the 5% who are?Are the people who are on online panels,whether it's a YouGov panel wherewe spend hundreds of thousands pounds a yearrecruiting people.Or whether it's an open access panelwhere people can just sign up because they have an axeto grind.Are those people the same or different from others?

JOE TWYMAN [continued]: The next five years, in fact, the next 10 years,next 15 years, however long, nobody knows.YouGov's emphasis will be on big data.Now, a lot of people are talking about big data.And it means a lot of different thingsto a lot of different people, actually, in the same waythat internet research meant a lot of different thingsto a lot of different people back in the early 2000s.

JOE TWYMAN [continued]: But for us, big data means bringing togetherall these different data points that we haveand really utilizing the power that bringsto strengthen our offering.In other words, we're going to lookat how we can become more accurate by bringing moreinformation to bear, by combiningthese variables to get a full picture, a fuller picture,

JOE TWYMAN [continued]: of our respondents.And so, for instance, we want to make surethat we have people who are not overly engaged in our sample.In other words, we want properly normal people--the type of people you see on the street.And yes, we have a lot of those already.We want to make sure that we've got as many as we possibly can.

JOE TWYMAN [continued]: And then we want to understand those people in greater detail.And so not just which party they wouldvote for in the case of a political survey,but who they voted for previously,what they think of the economy, what they thinkof different issues, and how those figures havechanged over time.That's going to be hugely important.And we're going to use that to become more accurate by making

JOE TWYMAN [continued]: sure that we've really got a full picture of where we areand where we are going.For us, it's about two things.The first of these is it's about cash.And we pay people to take part in surveys.Now, a few people will say, oh, well, hang on.

JOE TWYMAN [continued]: By incentivizing people, we are in some waychanging their behavior.But you know what?If you don't incentivize people, why should they take part?No one nowadays has this massive incentiveto take part unless it's that they have an axe to grindor they have a point to make.We're not interested in those people,at least we're not just interested in those people.We're interested in normal people.

JOE TWYMAN [continued]: And so we incentivize people to take partso they have something to do.But it's not just about that.We try to make the whole process engaging.We try to make the whole process interesting, fun even,for them to go through.We payout as a company over 12 million poundsin incentives each year.And so there's a lot of money going out of the door

JOE TWYMAN [continued]: to people.I would like to say the 12 million pounds is going to lotsof people and not just one.But we find that that's a good way to keep people interested.And then getting them included in the first placeis part of our recruitment process.As I say, we spend hundreds of thousands a yearrecruiting people, going out to find people,not just taking people who come to us because they're

JOE TWYMAN [continued]: particularly angry or particularly exercisedabout a particular subject.At YouGov in London, our global headquarters,we have over 200 people working here.And we now have over 600 people worldwideacross 22 different countries.In the London office, it's a whole mix of different jobs.

JOE TWYMAN [continued]: We have a lot of data analysts.So these are people who will sit with pages and pages of codein front of them tapping away on their keyboards,doing the hardcore analysis.We have sales people-- people who know research, but areable to then hand it over to the real data experts,for instance.And so they'll be the first point of contact

JOE TWYMAN [continued]: with different companies and different research buyers.We have a lot of admin people whose job it is to,for instance, respond to contacts from the panelists.We have operations people who develop the tablesand the charts that go out to clients.And then we have a lot of research executives.

JOE TWYMAN [continued]: Now, these will be different specialists in different areas.And so we'll have experts in consumer research, mediaresearch, financial research, and in my team,political research.And these are the people who craft the questionsand look at the analysis to draw out the stories from the datato provide insight for our clients.

JOE TWYMAN [continued]: Political polling is hugely importantto the democratic process.If you think about the press, it holds the governmentto account by holding them up to scrutiny.And what political polling does is it provides data.It provides a feedback to government outside elections.It makes them clear what's good, what's bad, what's popular,

JOE TWYMAN [continued]: what's unpopular.Now, whether they choose to follow that or notis up to them.But it provides them with informationon how they're doing.And it also provides them on informationabout where the public is unhappyand things that they could do to improve the situation.And this can be hugely important.In this country, we're very luckyin that we have a lot of informationand a lot of it is accurate and a lot of it is true,

JOE TWYMAN [continued]: whether that's from the press or whether that'sfrom political polling.In other countries, they're not so fortunate.And when you go to a country like Iraq,for instance, where I worked for two and a half years,you see a country that has an absence of information.And you can see how damaging that is notjust to the democratic process, but to society as a whole.

JOE TWYMAN [continued]: And so I think that this entire companyand this entire industry is hugely important.And I think that for political students,it provides an excellent opportunityto enter the world of politics.And so that doesn't mean standing upat the ballot box waving fingers at the opposition.It doesn't necessarily mean running for office.

JOE TWYMAN [continued]: What it means is understanding politicsin the true sense, understanding the machineryof public opinion-- what's good, what's bad, what's popular,what's unpopular, and what needs to be done.And that can be hugely beneficial for studentsas it can be hugely beneficial for society.[MUSIC PLAYING]