Icarus Studios News

Thomas Chesney on Terra Nova recently mentioned a short paper entitled Trust in Second Life. In the paper, Duffy states he's interested in Virtual Worlds as a social research tool, especially with respect to economy. However, he is concerned about the lack of controls and culpability in such an environment. To illustrate his concerns, he chooses to take part in a randomly picked experiment taking place in Second Life. He tells us that, during the experiment, he lied to the automation about his age and felt no real responsibility to finish the demographic survey. He opted out after the main experiment was finished.

The study in which Duffy took part used Berg's trust experiment, which is meant to be used in an investment setting and in cooperation with other participants, yet nothing here qualified as a reasonable investment and the author suspects he was alone (although we can't know the true monitoring situation). At the end of the article, Duffy admits he wrote the paper after visiting only one online experiment, and that he himself assumes the challenges he's postulated (lack of controls and culpability) will be difficult to overcome in any virtual situation. The paper's preparation, oddly enough, doesn't seem to be very empirical.

I have to wonder: how do these concerns differ from those in any other research situation? The biases and concerns Duffy talks about occur as a byproduct of most decision making events. The truthfulness of demographic information is always suspect. Even more to the point, a single report of events is rarely considered, by itself, proof of anything. This doesn't seem to bother historians; they cross-reference available records until they think they have a more accurate picture of events. Behavioral and social scientists argue endlessly over what constitutes good research and what doesn't. This is why there are always competing instructional models, and why it's often said that statistics lie. One segment is never enough. The process of comparing, of averaging results and knowing what actions or responses to look for in what situation: that's what makes the research, not the ability to hand out and recover fifty survey sheets.

So what question is the paper really posing?

As I understood the paper, the real question was if virtual worlds offered a legitimate platform with which to conduct social economy experiments. I'm no economist, but I would say yes. Considering that economy is based on systems of choice, and games are similarly nothing but systems of choice, a multi-player game is certainly just as valid (if no different) than a real world economy, especially for purposes of academic study.

The real difficulty would be in choosing the right virtual world for your study. Virtual worlds are conceptual tools. I could create an environment where I experimented with generating landscapes from fractals, and this might still be a virtual world, but unsuitable for an economic experiment. A virtual world like EVE Online, however, is fully worth economic research and, indeed, has people doing just that. Here again, I'm not an economist, but it occurs to me that being an economist without a game is like being an astronomer without an observatory.

Yes, it is more likely that, if I’m eighty, online I can lie about being younger, but what is the probability I would? What are the chances that an eighty year old, when asked, would fudge their age more than ten years if at all (still making them a senior, demographically)? What are the chances five would do it? Twenty? What are the chances that each of these users would not only lie about their age but all their other preferences, in order to hide unusual results?

Duffy does note that research in Virtual Worlds has a clear advantage over a panel at a physical location, namely that you can reach a wider, more diverse group of people, but this isn't something for which you need a virtual world, specifically. In fact, the overkill that a virtual world would present for a simplistic survey might itself mar the study. The error that many people seem to make when dealing with virtual world theory is treating virtual space like any other gathering place, effective to the same extent as any other sounding board or forum. This can be true, but it's not what makes a digital environment a “virtual world” in the whole sense.

What gives a Virtual World greater power than a related, “lesser” digital environment is the emergence and hold of a specific set of behaviors on the end user—in other words, a depth of culture that can't completely exist in any other situation. This becomes more about matching your methods than anything else. The lack of environmental factors matching to the study influenced Duffy's reaction in a specific way. As much as we like to discuss how virtual money can be equivalent to real money, the author is not a currency farmer (for instance), and by his own admission held little stake in the reward. The overkill of using a virtual world as the study environment possibly skewed the results of the experiment, unless it was an altered version of the trust experiment. This is not only because participants might have lied or quit early, but because some of us walk into a virtual experiment expecting too much precisely because it is in a virtual world, as in the case of this paper.

The study Duffy took part in might have been a phenomenal success, yet from Duffy's point of view, it was serving to illustrate his concerns simply because it took place in Second Life. He was concerned with the possibilities of reporting false information in virtual arenas, which is valid, but the success of the choice being made in this particular experiment had very little to do with whether he lied about his age or not.

The thoughts that occur to me as I read discussions like these are about how, as behavioral scientists, we have to teach ourselves how and when different types of people in certain situations are likely to lie, and when those lies are likely to affect the environment in which they're made. One could argue that this technique makes up the bulk of many behavioral and social scientist's training.

One does not need be a professional to learn these techniques. Psychology undergrads are taught how to score psychological tests and in learning how to judge them, learn how to effectively cheat them. This is the stage that disillusions many would-be social scientists: when they realize that the truth of the research is in the eye of the beholder, and that similarly, this is somewhat the point.

So is this whole debate over the accuracy of online surveys unnecessary? Can a successful lie only be made by a person with a thorough understanding of the culture against which they craft the lie? If this is so, doesn't the lie then become the best indicator of the true community reality?

-Miri Funderburk, Virtual Worlds Team