Usability testing with children, part 1

/70

Over the past few months, I've been working with Anthony Lewis and Andrea Vaughan on a web app for children. We're not ready to explain what it is or does (you'll understand when you see it), but it's a really exciting project and I can't wait to talk about when it's public. I've also been lucky enough to work with one of my favourite designers, Laura Kalbag.

We wanted to get the prototype in front of kids as soon as possible to make sure it's something they'd want to use, and to get some feedback on what we'd got so far, so we've been approaching teachers to let us try it with some of their students. We're going to do this across the country to get a good sample. This was our first session.

The setup

On Wednesday, we did 2 sessions at the same school in Bath, Somerset, one with a big year 2 group (29 children, aged 6) and another with a small year 5 group (12 children, aged 9).

Usability testing tips from twitter

I asked on twitter for some suggestions on running usability sessions with kids, and I got some amazing responses, so I've made a list of them.

I've done workshops in a classroom before, but only in secondary schools, and I've never done any serious usability testing. I'm not even sure an expert would really count this as usability testing, we were just doing what felt right.

Ideally we'd do individual testing or in pairs, but we didn't want to take up too much of the teacher's time. Most schools don't allow this sort of one-to-one testing without staff supervision, and that can be expensive for them, time consuming for us when we're at the very early stages of testing, and potentially quite scary for the child. We were just looking for general feedback on the idea, and what parts of the app the children would be most and least interested in using.

Year 2 (29 children aged 6)

The classroom had 4 shared desks and was kitted out with netbooks called Fizz Book. There was 1 for every 2 or 3 students, and they ran Windows 7 with IE9. The netbooks were touchscreen and could flip round and turn into tablets, although all of the students were using it laptop-style with the keyboard and trackpad. I'm not sure if this is because they were told to or they chose to. The resolution of the screens seemed quite low.

The teacher mentioned the students were all very confident with technology, and one of the children was very excitedly telling us that he uses an Android device and a dongle for internet access.

Year 5 (12 children aged 9)

This session was in a computer suite. The machines were running Windows XP and IE8. These were workstation computers with keyboards and mice, and there were enough for 1 each.

A quick quiz

We started off each session by asking what technology the kids use. We wanted to make sure the kids would be able to use the app at home, and what type of devices they used. This also helped break the ice a bit and get the children talking about what "toys" they had. Here were the results we got. Bear in mind this is just one school, and we're not looking for accuracy, just a vague idea. We asked the kids to put their hands up rather than fill out a form because of time and reading levels.

Year 2

All but 2 children put their hands up to say they had internet access at home. Most had their own computer, although half shared it with siblings. All use Youtube, and some admitted they had Facebook accounts (I got the impression that their parents had set them up and closely monitored them though).

Their favourite sites were Moshi Monsters, Transformers and Club Penguin. There is a certain level of social importance to having a high status on Moshi Monsters and Club Penguin. Some of the children were saying "I'm a super moshi!" or "I'm a ninja penguin", and I think this is the equivalent of adults having a high klout score, or follower count or whatever.

When asked how many had their own phone, just under half of the children put their hands up.

Pretty much all of the children had a games console, some more than one. All 29 had a Nintendo DS.

Year 5

11 out of 12 of the children had internet access at home, and a third of the group had their own phone. 7 had a portable device like an iPod touch or Nintendo DS and 10 had a games console.

Their favourite websites were Club Penguin, Manga High and Moshi Monsters, and they all contributed to a school blog. None of them had heard of Neopets, which was a Moshi Monster equivilent when I was their age.

Conclusions from survey

When we asked "Do you have internet access at home", I wonder if the "no" responses were from people who didn't have a computer connected to the internet, and whether they took into account mobile internet access. I think the question "Do you have internet access at home" would have had different results from "Can you visit this website at home?", and the person asking needs to consider what they're trying to understand by asking. (Do they want to know if children are able to access the internet at home, are allowed to go on the internet, or do they want to know how many have high speed broadband?). It's better to ask the parents to get accurate results.

Since more of the children owned internet enabled devices like games consoles and phones than their very own computer, anyone building a web app for children would do well to test it in mobile browsers and the browsers in games consoles.

As we asked more questions about technology and what devices the children had at home, I became worried that asking these questions in a big group made those who didn't put their hands up feel embarrassed. Maybe some of the children put their hands up anyway so they wouldn't feel ostracised for not having the latest gadget. I think a less public survey would be better.

The demo

Based on what I've heard from people who run usability tests, what often happens is the tester is sat at a terminal and asked to use the site on their own having never seen it before, given a set of tasks to perform, and talks aloud while using it. This is a good way of finding out if the design makes sense and the site is easy to use without someone having to explain how it works.

We had a big group, so this wasn't possible. When this site is made public, we anticipate teachers will give a demo of the site to their students anyway, so it made sense to run the session this way. We decided to first give a demo of the site and explain what it does, then sit down with a group of children at a computer and see if they could perform some of the main tasks.

Getting feedback

After the demo, we asked the children to split up into groups and open the site on their computers.

One of the things I learnt to do in a usability test is to tell people "you can say what you like about this site, I didn't build it" even if that's not true. Otherwise some people might not say if they don't like something.

Here, we told the children that we'd built it. I'd feel uncomfortable about lying to the children (it's ethically unsound), and it didn't seem to affect the types of responses we got. The group were of the age that they were happy to tell us exactly what they thought.

It's important to make the teacher aware that negative feedback is ok so they don't tell the child off for "being mean" about the visitor's work!

It's also vital to explain at the start, like in any usability test, "we're not testing you, we're testing the website. There are no wrong answers."

The fold

The first thing I noticed was that there wasn't a lot of horizontal space on their tiny netbooks. A lot of this was taken up by status bars, so the screen area was relatively wide, but not very tall. I'd thought a lot about how the site would adapt horizontally, but I hadn't done that much testing on how it would adapt vertically.

In testing, the fold was a big issue. The way the page was designed made it look like it stopped shorter than it did, and they weren't really scrolling at all on a key page.

After the sessions, I mentioned this to Cennydd Bowles who has done a lot of usability studies, and he suggested that Object Permanence might be relevant.

Object permanence is the understanding that objects continue to exist even when they cannot be seen, heard, or touched.

Object Permanence on Wikipedia

On further research, it looks like this would only be an issue with very young children (aged 2 and younger), but I thought I'd mention it here anyway in case anyone is designing for children that age.

Labels

One of the links in the app is to a child's dashboard. It's labelled "dashboard". Anthony asked the class if they knew what a dashboard was. Very few hands went up. "It's the thing on the front of a car". Ok, bad choice of word.

Next to the word "dashboard" is a small 2 in a circle, like the little badge you get on an email icon to show how many unread messages there are. It's basically meant to indicate that here. "Do you know what that number in a circle means?" A few more hands. "Does it show that there are 2 dashboards?" The solution didn't seem at all obvious, so we need to rethink this.

Blocks of text

We created a few different personas. There's one for a teacher (Mrs Brown), two for students (Sam and Ryan) and another for an older student (Zapher). Each persona has an account that has a different view. In the prototype, you're looking at Sam's account. "This is where Sam can see the stars he's collected". However, I'd not been very consistent and in some parts of the app, I'd used the persona "Ryan", so there were bits about Ryan where they should have been about Sam.

While demoing the app, a few children noticed this, even though it was in the middle of a bunch of text. "Who's Ryan?"

I was annoyed that I'd missed this because it was quite difficult to explain how this was a mistake and it should have said Sam. But actually it was a really useful way of seeing how engaged they were with the app. If they hadn't noticed the mistake, maybe it meant they weren't reading the text or trying to understand it.

Asking the children for design solutions

There was a part of the prototype that we knew had design problems. Very basically, a child could be presented with an option of whether or not they want to be anonymous. The much younger children didn't understand the word "anonymous" (all the older children did from using their school blog and reading messages from "anon".), so a text box explaining it wasn't going to be enough.

I explained the difficulty we'd had designing it to the girls I was sitting with. "Wouldn't it be bad if you wanted your name to be secret, but the way we'd designed it was confusing, and people could see your name when you didn't want them to. How would you make it look if you were to design this screen and you wanted to make it really, really clear whether people could see that this was you or not?"

Immediately one of the girls said "Before I click the button, I would have a picture of my face next to it, and if I wanted it to be a secret, my face would not be there or look like a shadow". I thought that was a better solution than we had come up with, and I was so impressed with the maturity of the response. It's definitely worth asking some "how do you think this should this work?" questions.

Language and tone of voice

I've been researching how other kids' sites address the user. There are basically 2 main options: "Your account" and "My account". I was against using "My" after reading a couple of articles against it, like Super Unsuck It! Bye-Bye, My and Whose "My" Is It Anyway?

Additionally, kids in this age group are still slightly egocentric, meaning they have trouble seeing things from other perspectives. As a result, words like “me,” “my,” and “mine” are confusing. A six-year-old boy said, during a user research activity, “Who’s talking here?” When probed for more detail, he said, “This says ‘my user name.’ Whose name? Who is this?” He didn’t understand that the “me” on the site was referring to him.

Designing Web Registration Processes for Kids

Anthony suggested we try it out and see for ourselves whether it's a bad idea. There was every chance we'd be proved wrong, but it's a prototype and that's what they're for.

It wasn't an issue at all in this round of testing. It's something we'll keep looking at, but if you're not sure about something and you have the opportunity to test it out, it's worth trying out.

A few other things that were interesting

  • We referred to the web app as a system, but they called it a game, even though there's not really a game element in it. Just clicking on things was enough to be considered a game.
  • The app had a placeholder video, it was just a blank box with a play symbol in it. Immediately the infant class asked "Can we watch the video?" "It's not really a video, it's just showing that there will be one there. Do you know what it's about?" "No, but we want to watch it!"

If you're thinking of doing this

  • Don't expect to be allowed to run sessions alone with students. Schools feel really uncomfortable about doing this and like to have a member of staff present. If it's really important though, you should get a CRB check.
  • I've always liked to bring stickers to the workshops I do with kids, and they're very popular, even with secondary school aged children. We brought some stickers with us and gave them out to all the students. I would advise anyone planning on running a session with kids to do this, and mention at the start that they'll get one as a thank you for taking part. It makes the world of difference. (Bringing sweets is well intentioned but sometimes frowned upon by schools because of allergies, healthy eating policies, and the way sugar makes kids hyper.)
  • It helped that there were 4 of us so we could talk to the children at the same time in small groups. 3 children in each group felt like a good size. More than that was difficult. Bring helpers!
  • Prepare a proper lesson plan. Schools are more likely to let you take up their teaching time if you can show how the session will run, and if you can make it educational in some way. This might be something simple like a discussion about the design process, or a workshop where the kids design their own apps on paper.
  • If you haven't already, read through the advice people gave me on twitter.