Saturday, May 24, 2008

Solving Second Life® User Interface and Churn Dilemmas

This is my first post from Windows Live Writer and it's an important post, so I am throwing caution to the wind.

For some time I have been thinking about the Second Life user interface and the loosely linked (perhaps?) churn problem. By thinking, I mean I haven't been in my usual "lean forward" mode, but rather in a more casual lean back, read, stare off into space and ultimately get distracted by something shiny mode. This is a good mode to consider seemingly simple but actually subtlety complex issues, and it allows my mind to wander off into open spaces and into more systems thinking. Lean back thinking is a luxury that we are rarely afforded in the current market of rush to publish micro-bursts of information and ideas. You also have to call your mind back occasionally and ask "What have you done for me lately?" and hope you aren't met with a blank stare.

So much for the frightening imagery of how I think, what's important is today my mind brought back an "aha!" (no, not that Aha).

I've been trained to go ahead and give you my answer right now, here before you go away leaving me with a tl:dr and a pat on the head. So I will, but you won't like it and some of you won't even read past my proposal except to skitter down to the comments and declare your outrage. Nonetheless, here goes:

The Second Life interface and attendant new resident churn problems can be solved using Artificial Intelligence and bots.

Yes, that's what I meant. Two of the most recently hated and feared parts of the Second Life experience can, in fact, improve not only the interface but reduce the new resident churn rate.

Despite the din of outcry across the Second Life resident blogosphere about the client, I have yet to run across a well-formed proposal for redesign. But, what my wandering mind did find was some correlation among sundry bits, explained in the following paragraphs.

At least once a week I mill about looking for the latest discussions, rants and otherwise about the Second Life interface. Honestly, I was looking for a "fix" - a miracle client release - but none were found. Instead I found Nicholaz Edition fanatics, the Linden Lab Viewer roadmap complete with the ability to skin your own (or rather, skin at your own risk), and reams of discourse some of which Digado summarized in his post about Second Life's New User Experience.

But for me, the most thought-provoking discussion arose on Dusan Writer's post about killing sacred cows. From Dusan:

….well, here’s the thing, I ended up wondering if some of the things I take for granted are errors of conception.

Call them the Sacred Cows of Second Life. And maybe it’s time for me to kill one or two of them. For myself in any case.

And the first is that the Second Life interface sucks. Which will lead to the next sacred cow, which is that the newbie orientation experience sucks. And the sacred cow isn’t necessarily that these things aren’t true, but that we should actually do something about it.

Dusan highlights his personal experience and frustrations with the complexity of the interface and ends up asking an important question. (emphasis mine)

All of which is to say that the interface is NOT user friendly, at least for someone like myself who’s unfamiliar with half the terms and look, it took me an hour to learn how to talk when I first got to SL, and it must be even harder now - do I click that little chat balloon or the other button and what’s the difference between “Communicate” and “Talk” or whatever the buttons say, all you really want to do is ask someone where the fun is.

But here’s the question - so what?

...

So the SL viewer seemed insanely difficult to me - so many buttons, so much to learn, and this on TOP of learning where to go, how to talk, who to talk to, and what the culture was all about, the norms, the attitudes, the list of things to see or places to go.

The answers to Dusan's inquiries change regularly, such is the dynamic nature of Second Life. This leads me to believe that the user interface will never be static, but rather will and must organically evolve with the Platform. It's important to note that this is a Platform that includes the technology, features, services as well as people, places and things to do.

Furthermore, the tasks and questions that Dusan is asking are all answerable (at least at the first order) using structured knowledge-based information. However, it's much easier to just ask someone how to do something than to have a priori knowledge that such information exists, find it, then read and understand it.

Those ideas were some of the trinkets that my wandering mind gathered up and placed into the leather bag of cognitive gathering.

Still more seemingly unrelated tidbits included the recent Linden Lab blog post regarding the removal of popular places and the implications of traffic. Many have speculated that these changes were implemented to discourage the use of compute intensive camping bots, that artificially raised the traffic results for parcels within the world. The argument is that removing the incentive of camping bots thereby removes the behavior. Note that is does not, however disallow the use of bots, it merely discourages the use of camping bots for purposes of gaming traffic measures.

Finally, two bits of video crystallized for me what I believe to be the only real solution to the evolution of a user interface against an evolving platform. The first was a video of over a year ago of Hamlet Au's conversation with Social Autopoiesis, a fully formed interactive non-player avatar (a bot). You need to see this full screen, and the YouTube version is blurry, but you can get a Windows Media version here (wait for WMP to launch). NOTE: Includes a few not safe for work words/language.

The second and most compelling video was this one from the research group at Rensselaer Polytechnic Institute in Troy, New York highlighting the work they are doing in Artificial Intelligence (AI) . The researchers are using Second Life because it's a controllable environment, which is both conducive to AI testing and development. One might assume that the Second Life Orientation Islands are a similarly controlled environment, with the advantage of a common set of goals and objectives.

Are you still with me? Are you still in lean back mode, or are you now leaning forward? Before I throw out a Q.E.D., let's review the key points:

The user interface is perceived to be complex and "broken" because the learning curve is steep for even basic tasks including finding people, places and things to do. However, the Platform (technology, services, features, people, places and things) is still evolving rapidly and a moving target is difficult for even the best Human-Computer Interaction (HCI) designers to hit. Furthermore, the diversity of residents and attendant needs in Second Life complicates the use cases for the interface development beyond that which is productive. Customization (skinning) like that which is offered by World of Warcraft interface will allow users to adapt the features that they need, assuming they escape the initial vortex of Orientation Island.

Bots have not been eliminated from the Second Life landscape, and the use of bots for fraudulent reasons is on the Linden Lab radar. Residents and researchers alike have demonstrated the most basic and fundamental uses of chat bots and AI in Second Life. The application of chat bots and simple AI for the first hour in Second Life is perfectly aligned with what new residents need. Some way to ask and be told the answers to basic questions is a logical way to present knowledge based information and possibly more important, is a way for the Linden Lab team to gather data on what tasks are most difficult to grasp, or what information is unclear, and what gets asks most often. Prokofy outlined a few of these most basic knowledge needs in "The First Hour in Second Life". These simple new resident questions are not outside the ability of Pandorabots (maybe you'd like to try your own to test me?), but real value can be derived from early AI such as that presented by Rensselaer.

We have assumed we must "fix" the interface, I am suggesting that is an exercise in futility given the above. You don't "fix" it, you teach it and evolve both the teaching and the interface using actual feedback based on experiential application. So allow me to repeat:

The Second Life interface and attendant new resident churn problems can be solved using Artificial Intelligence and bots.

These bots can be called upon at anytime to answer questions about how to do something in Second Life. They can be stationed at common areas and Help Island, and most importantly they shape the first hour of a resident's existence.

So what does the new Orientation experience the eyes of Grace look like? It goes something like this:

1) Pick your name, your starting "look" AND a serene place of your choosing: such as tropical, forest, desert, city park.

2) You rez and are greeted by your personal artificially intelligent tour guide, the lovely Graceimator.

newb

Graceimator 1.0: "Hello, avatar_name. I am here to help you get started learning about Second Life. We are going to walk about this (serene_place) and learn a few basic tricks at learning stations. Along the way, you can ask me anything, and I will answer the best I can. Do you have any questions before we walk over to the first learning station? Talk to me by typing in the white space at the bottom of your screen, then hit Enter or press the "Say" button at the end of the white space."

Avimator 1.5: "Yes, will I ever get out of this Orientation and meet some people or am I stuck with you until I get frustrated and leave?"

Graceimator 1.0: "You will leave Orientation a happy new resident."

[Insert your personal view on amazing new resident indoctrination here]

One last note, the picture above was taken at the serene but unfinished "Organic" build in Second Life. Unfortunately I didn't see M Linden hanging out there.

20 comments:

Anonymous,  2:19 PM  

This is exactly what BraveNewBot is doing: They demonstrated their receptionist bot for SL on their booth at the VirtualWorlds conference in NY early april
http://www.bravenewbot.com

Grace McDunnough 5:26 PM  

@Anon That is fantastic; it is on the video at least. I tried to get to MASA Group island but I think it's private. Is there any chance I might get a tour?

isadorafiddlesticks 12:40 AM  

other virtual worlds such as there.com have "guides", and i think it's unfortunate that SL is the last to adopt using AI to guide new users in SL.

If I were SL, I'd make a Torley-bot. That man really puts SL in a great light..:D

another post that gets me thinking and reflecting. thanks.

Anonymous,  12:49 AM  

I suspect much of the first hour problem could be solved with an email. Many new residents have been flying computers for a long time. What they need is a cheat sheet that explains moving, teleporting, chatting and searching. They do not need a HUD or even a greeterbot. They need a page or so of text.

When someone I know tells me they're signing up I arrange to be online when they rez, friend them, yank them out of orientation and spend 20 minutes chatting. End of tale. My retention rate thus far is 100%.

It's true the interface is not as broken as we often think. It's also true that old-fashioned text could solve at least some of the orientation problem.

Anonymous,  5:35 AM  

@Grace mcdunnough: We're working on enabling live demos on our island. I'll keep you in touch to be among the first invited. Our offer will storm SL very very soon.

LDinSTL 3:28 PM  

"You don't "fix" it, you teach it and evolve both the teaching and the interface using actual feedback based on experiential application."

Exactly. Bots are something to work toward, but in the meantime...maybe Torley could help.

I'm told that Tech Support folks have an acronym for when users call them up with inane questions -- RTFM (starts with Read the, and ends with Manual, profanity in between). :-)

But with SL not coming in a box, there IS no manual to ignore. Some like to plunge in and learn in a trial by fire, some like to read up on things first, some like to watch a video. None of these options pop up when you enter via the standard SL route.

What about adding a short (visually prominent and attractive) note or visual pointing to your choice of the above? And I mean before you even are prompted (what a concept) to download the SL client. A 3 minute Torley video could clear up a lot, and urging new residents to bookmark a link back to watch it (and a few choice others) repeatedly could go a long way.

As someone who taught a relatively complex subject (chemistry) for many years, I know that there is no way to learn something as complicated as SL in one pass. That's OK--it would be really boring and simplistic if it were that easy. The trick is letting people know what wonders lie ahead and what the journey will involve. It's like traveling to a new city or country. It takes awhile to figure out how things connect, and even longer to remember!

I had been "in world" quite some time before I knew that the wonderful world of Totley Tutorials was just a click away. No one should be so deprived. I'm betting Torley could make the world's best "your first hour in SL" machinima, and the blockbuster sequels. Don't you think?

P.S. And what's with the lack of a pointer to an online beginner's guide WITH VISUALS for the readers in the crowd...and Grace, I apologize for this turning into more of a blog post than a comment. I've been thinking on this a lot too--LOL

Cheers, Chimera Cosmos

Anonymous,  7:28 PM  

SL is about people not 'Bots'

When I rezzed at Orientation Island thank god there was a human mentor to meet and greet and even cloth me. If I had been presented with a robot I probably wouldn't be here now.

Lets keep it real..not AI

Dusan Writer 7:31 AM  

Thanks Grace for such a thoughtful post. I like the bot idea but also feel that if we can get users to invest in their avatars before they even enter the world, they'll put up with a bit more frustration. I've wondered whether orientation island is a Sacred Cow - why not kill it entirely, set up a community resource center like the Ivory Tower of Primitives that we can all use.

In any case, posted extensively about this in response on my blog.

http://dusanwriter.com/?p=508

Grace McDunnough 7:44 AM  

@Anon#3
I fully agree that "SL is about people not 'Bots'" but a one-to-one human to human model for new resident orientation doesn't scale very well in the first hour. If you go through the most recent incarnation of Orientation Island, you will find that the human element is anything but helpful since your attention is on pushing buttons and the "tests" you have to pass. Help Island, however, is quite different and I think there's a whole post that could be written about that experience.

Dale Innis 9:29 AM  

A good post and a good idea; unfortunately I'm pretty certain that the solution you're recommending is one that we don't currently know how to do, and aren't likely to figure out how to do in the next, say, ten years.

People have been writing Elizas and chatbots and AIML-based bots and stuff like that for decades. None of them work very well. None of the ones that the creators can give fancy demos of work very well when faced with talking to someone besides the creators, in a real live context-rich environment.

None can actually answer the kind of free-form questions that newbies ask any better than a keyword-search over the support database could, and (and this is an important point) because they look like people they raise users' expectations high, and when the answers that come back are silly and irrelevant, people get much more annoyed than they would at getting the same answer from a search page.

Smart, articulate, computer-savvy newborns don't need mentor-bots. Less smart, less articulate, computer-phobic newborns will ask the kinds of questions that challenge even human mentors to parse and respond to, and no mentor-bot's going to be able to help with those this decade.

(I'm taking a somewhat devil's-advocate position here; while I do think the problem of even mentorbot-quality AI is still far beyond us, I'd love to be proven wrong!)

Dale Innis 9:45 AM  

P.S. Forgot to mention that the Hamlet video illustrates my point exactly: the bot there is your typical AIML style "AI", that gives surprisingly articulate and intelligent-seeming answers to some questions because the creators have explicitly checked for those questions and given it a word-for-word reply, and gives meaningly and often grammatically incorrect answers to pretty much *all other questions*.

"Where in California Idiot?" hahahahahaha... :)

Prokofy 10:13 AM  

Bad idea. Bots frustrate people enormously. Only makers of bots, and people thinking of them abstractly think this is fun. Haven't you ever been on hold with the Virgin Mobile or Verizon or Amtrak bots that now answer phones and "sound realistic". You will find yourself shouting OPERATOR! OPERATOR! within a few minutes of the idiot thing not helping or understanding.

To be sure, in a text-driven environment, mistakes may not be as likely, but they will still hinge on the need for something that frustrates, baffles, and even scares people: the mean blue drop-down screen menu. That thing is clunky, laggy, and confusing. If there was some *other* way to handle this -- and please don't suggest a HUD -- perhaps it might make sense. But...there isn't.

I've polled newbies extensively and have thousands of answers. All of them overwhelming reply to the question "What is needed to improve the new user experience?" the following:

Jobs for newbies

Nobody ever thinks to ask that question because they default to the "sacred cow" of Second Life which is "create or die" or "buy or die" instead of asking how you make a country, and how you have entry level jobs. Because the Lindens and the elites won't think of this, people with casinos, banks, and camping think it up for you. There is a heavily organized sex industry and its related club and mall industires where people are organizing the newbie experience just fine, and retaining just fine -- they just aren't retaining the culturally sophisticated and intellectuals that you might wish to retain.

Anyone who has handled the newbie firehose comes away instantly with the realization that you cannot serve and retain everyone. That's because many of the people logging on are 15 year old teenage males on their mom's cell or 14 year old girls using their mom's credit card -- or the emotional and intellectual equivalent, with a mere 3-4-5 years in addition. They will not be learning; you will not be doing anything except wasting BUNCHES of time on them, and it's not worth it. You offer what you can and strive for the best percentage you can, and leave the rest for some other platform or entertainment company.

The interface is wonky because the Lindens have not changed from within. When they are ready to change from within and want customers, then the externals will change. There isn't really much you can change about building on an XYZ grid. The things that would make life easier like the ability to grab objects without edits and confusing XYZ grids may be technically not feasible. Rearranging the interface, other than some obvious things like putting the search button on top, will not help retention.

That's because most people coming to SL need engagement -- human, not bot engagement. They aren't able to entertain themselves, and they want someone else to set up venues, introductions, and very lite skill transferrals tailed to what they are interested in.

An email with the basic start-up info is a good idea -- but most people don't read. They ask, and they want someone to answer. A bot might shave off some of the churn, but at root, it cannot solve the human problem: the need for attention and engagement.

Salome 11:29 AM  

In this case, I honestly think that SL is behind the times and should take a page from *gaspshockgroan* the online gaming community.

When you start a game like, say World of Warcraft, you do not have all your tools thrust at you at once. You learn to use the most basic tools and then are slowly introduced to new tools. In the world of gaming these are handled as rewards and leveling. A virtual environment like SL could easily use a similar method.

When you create a new identity, you could rez with an interface greeting that asks if you'd like to complete any of a few basic tutorials with an estimated teach time for each:

Movement and Flying - 3 minutes
Basic SLttiquete - 3 minutes
Avatar Settings - 5 minutes
Using the Map and Teleporting - 5 minutes

and so on.

At each stage, there could be a HUD that highlights only the tools used for that particular section. And there could be rewards for completing the different levels. Such rewards could be sponsored by in-world business and change from quarter to quarter.

I've always thought person-to-person mentors were a horrible idea. While well-intentioned I've known several people who lobby to put themselves in mentoring positions who have no business in such a role.

There needs to be some thinking out of the box and there needs to be a push to show this is a priority of the Lindens. But I see no evidence of that whatsoever.

Grace McDunnough 7:42 PM  

@DaleInnis
I think I addressed your comment about searching here:
The answers to Dusan's inquiries change regularly, such is the dynamic nature of Second Life. This leads me to believe that the user interface will never be static, but rather will and must organically evolve with the Platform. It's important to note that this is a Platform that includes the technology, features, services as well as people, places and things to do.

Furthermore, the tasks and questions that Dusan is asking are all answerable (at least at the first order) using structured knowledge-based information. However, it's much easier to just ask someone how to do something than to have a priori knowledge that such information exists, find it, then read and understand it.

I guess the "someone" I am referring to is arguably *not* someone, but some thing. I have some research that indicates that people prefer to interact using natural language over keyword searches, even when the results are not as robust, but alas I can't locate the link to the blasted thing. I'll pass it along when I find it.

@IDinSTL I like your idea about the short video tutorials before the client download. At least people would know then where to find them again to refer to once they get in world and get confused and overwhelmed.

@Prokofy I have not spent enough time with true new residents for over a year, and you have far more first hand knowledge than most people given your business. You gave me an idea about another approach - which is a virtual tour guide of sorts - to address the entertainment point you made. I *was* going to say a HUD with built in places like the one LL did for WinterFest but I shouldn't think out loud in comments, so I'm going to give it a little more thought.

Dale Innis 7:19 PM  

This is a great discussion. :)

@Grace:

I'm not sure how the fact that the answers vary over time (as I certainly agree that they do) mean that a bot will do a better job answering them than a search would? The two issues seem entirely separate. Having a bot draw on a dynamic and evolving knowledge base seems at least as hard as having a keyword search draw on one (and in both cases just keeping the knowledge base up to date is also a hard problem).

The questions that Dusan's asking, in the way he's asking them, aren't really relevant to the question of improving the newbie experience. My claim (of whose correctness I'm now even more convinced having looked at the bravenewbot and RPI demos) is that the questions that newborns will actually ask, in the way that the newborns will actually ask them, are far beyond our capacity to program a bot to usefully answer.

If we could get people to encode their questions in the appropriate formal language we *might* then be able to answer them automatically and usefully using structured knowledge-based information (although frankly I think even that problem is harder than you're suggesting).

But conducting useful natural-language interactions with real live newborns with real live human speech patterns and spelling skills? I think we're years and years away from that, and so far I see no evidence I'm wrong... :)

Grace McDunnough 8:06 PM  

@DaleInnis Questions in response to structured exercises (the newb 101) in a controlled environment (orientation) are largely predictable, and the answers are as well suited do a knowledge store. I think we actually said the same thing although now you seem to disagree with that point, so maybe we weren't after all. I am not suggesting random questions such that Hamlet presented to a simple Pandorabot. Game developers use this most basic form of AI in this regard often - do they not?

How did you get access to the BraveNewBot demonstration? I thought they were limiting access; must be your VIP status.

Digado 5:47 AM  

Interesting article, an well thought trough (I liked the scenario), but I find it hard to imagine 'the paperclip approach' (MS Office) would help new people find what they want in Second Life. It's not fixing the interface at all, its just providing a tutorial to understand an uninviting and bad interface. As a user generated solutions its interesting, I expect a whole lot better/more from LL to build something truly accessible.

If bots are to be used I think its more likely to see something like static 'NPC' style such as in MMORPG's - introducing a logical, perhaps even fun experience of getting to know the drills of an interface that should still a lot better.

Dale Innis 11:02 PM  

@Grace : I just went to BraveNewBot dot com and watched whatever the most recent thing there was. If they have some wonderfuller thing that isn't available yet, I haven't seen it. :)

Perhaps our main disagreement is about to what extent Orientation Island is in fact a controlled environment in which people are doing structured exercises. Given that everyone who comes in new to SL has their own set of ideas about what they're doing there and what it's all about, and that one of the things that the environment contains is other newborns :) I don't think it's likely to qualify for either of those categories.

But okay, let me try the other side. If we believe there are maybe two dozen things that the majority of newborns are likely to want to know about, we could have a team write down the top ten ways of referring to each of those two dozen things, and write up a paragraph of good information about each of the things (and update them weekly or as needed).

Then we could hook up the mentor-bots to deliver the appropriate paragraph upon recognizing someone referring to one of the things.

That might or might not actually be useful; it would be fun to see an empirical test. Anyone attempting to have an actual conversation with one of these 'bots would find out instantly that it's not possible, but maybe that's okay.

Do you think we have enough technology in natural language understanding and AI to do significantly more than that very simple system? That's where my skepticism is focused; I'm pretty sure we don't...

dandellion Kimban 5:54 PM  

Bots sounds nice, but they are not a solution. As you probably know, there are mentors all over the Orientation and Help Islands. And they have real humans behind them, which makes them more communicative than bots. First, mentors are usually bored so they are willing to talk more, and then, they are more intelligent and will react much better to questions made by noobs. If you've ever heard somebody trying to ask you someting in bad english about something so new as SL can be you know what level of decyphering is needed sometimes on OI.

Grace McDunnough 7:04 PM  

@dandellion I don't know the last time you went through the registration process, but I did recently, six different times before I wrote this post and only once was there a non-newb human there to greet me at Orientation. A single human, not armies of roaming mentors, eager to help. Sadly, at that particular juncture although there were only about 15 of us on OI, that human was able to answer questions from one person at a time. Actually, that is an assumption since they never once answered me. To make matters worse, the HUD was not working and we were all moving from station to station saying "hello" and "help" to see if we could at least pass the chat station. So no, a human was not helpful nor was that even the best solution at that point. A bot may have at least picked up on the consistent error messages from the HUD and alerted a human to the problem that was affecting far more than just one person. Human to human doesn't scale very well - ask any experienced customer service manager.

I agree that the UI needs work, what I don't agree is that will improve the first hour experience nor do I think they can fix it right now because frankly, LL does not have enough data on 1) the first hour and 2) the hour right after that and 3) 40+ hours after that. Bots, at east in the first hour help gather that routine data about what is confusing, what gets asked, etc. For the remainder, some serious ethnographic research is required.

Apture

DISCLAIMER

This blog contains my personal observations about evolving social spaces and virtual worlds. Statements and material posted here do not reflect the position of my employer. Included within are my experiences in the Second Life world. Second Life® and Linden Lab® are registered trademarks of Linden Research, Inc. No infringement is intended.

  © Blogger templates The Professional Template by Ourblogtemplates.com 2008

Back to TOP