Hover to the Future

About a year ago, shortly after I began my tenure at Amazon and bi-monthly trips to the South Bay Area, I stumbled upon yet another Kickstarter. My Kickstarter portfolio has included books, drink-cooling devices, card games, video games, a design school, and in one case geeky baking sprinkles. All successfully funded, though many late deliveries.

This project, however, was much loftier. The video tugged at the heartstrings of my generation, raised from an early age on Back to the Future – early enough that we were forgiving of the sins of Back to the Future II. Specifically, this Kickstarter from a company called Hendo dangled the tantalizing possibility of real, honest-to-God hoverboard technology.

The reward levels were heavily stratified – several thousand dollars for a dev kit, attendance at the launch party, or ownership of your very own board. And yet, for a measly $100, you could theoretically buy yourself 5 minutes on a board in their lab sometime after March 2015.

As I age, I find myself collecting experiences far more than physical goods (with the exception of Pokemon, but I shall not discuss that weakness here.) And that’s the problem with many Kickstarters – for your average backer, it’s just a bunch of stuff. This was different. And with my new job, I knew I’d likely be flying down anyway. The project seemed less likely to succeed than much of my Kickstarter portfolio, but with the likely free trip I felt it worth the risk, and I pledged – not expecting it to come to fruition.

In early 2015, I received the “stuff” part of my backing – a T-shirt, a sticker. I suspected that would be where it ended.

And then – on April 1st, of all days, I received a personal mail from an administrator at Hendo.

“Action Required!! First, we wanted to thank you again for being one of our supporters.  We are looking forward to having you here with us to ride our Hendo Hoverboard! … Our calendar is filling up quick so we would like to hear back from you by 4/6 on a desired time slot and a month that you would like to come down to the Arx Pax Headquarters.”

Admittedly, I was a bit shocked – and a bit skeptical. April 1st? Really? Still, I replied within a half hour with some proposed dates. I’d had Kickstarters with simple card games come in a year late. How could it be possible that a HOVERBOARD was being delivered on time?

And yet. Hours later, I was confirmed for a timeslot in June. I ended up having to move it once – but on the afternoon of June 11th, I found myself driving at the end of a business trip to a nondescript business park in the South Bay, surrounded by other nondescript scientific offices.

Since scheduling my appointment, I’d gone back to the Kickstarter page to verify what, in fact, I had signed up for. My pledge level entitled me to “up to 5 minutes on a hoverboard.” I naturally assumed it’d be a small group of folks, and wondered who the fellow backers would be who’d be accompanying me. I wondered if the whole thing could honestly be for real.

When I buzzed into the lobby, the receptionist greeted me by name, had me sign some liability forms and an NDA regarding any accidental viewing of the trade secrets behind the hover technology (which I didn’t end up accidentally viewing anyway).

Soon after, a friendly gentleman in a black shirt and jeans from the Arx Pax business development team came out to greet me and to take me on the tour. He explained that I was the only person on the tour that day. I was shocked and delighted, to say the least. The tour was quick, since the team isn’t too large and there’s a lot they can’t show without tipping their hats – but imagine my delight to meet a woman engineer on the engineering team. She joined us shortly thereafter in the demo environment.

Through large metal double doors with very prominent warnings about powerful electromagnets and harm to electronics was the demo environment – a modified loading dock with high ceilings and unfinished beams. The entry hallway boasted a museum of sorts hung high on the wall – previous prototypes, inasmuch as they didn’t show any proprietary technology. The number of form factors and shapes they’d explored was fascinating. I was shown to a table with safety glasses and helmets as they finished charging the board.

The demo area itself is a miniature skater’s half-pipe built out of copper plating, and a large flat copper area for the untrained starter boarders. They’d been clear from the beginning – no beginners on the half pipe! Which was fine by me – I spent 18 months legally handicapped after a catastrophic knee injury, and I’m no Tony Hawk. And it turns out even Tony Hawk is no Tony Hawk when it came to starting on the pipe – the team showed me a video of Tony training, and explained it took a few hours before he could really get going. I should also mention that I have zero skateboarding experience and shattered my kneecap in 2010, so I’m about as far from Tony Hawk as you can get.

I strapped on a pink helmet and safety goggles. With little fanfare, the board was brought out from the secret lab door and placed upon the floor. It resembled, more than anything, Griff Tannen’s hoverboard – big, black, and beefy. They explained that the board required a weightless startup, and so one engineer on either side knelt down, held the device, and one somehow powered it on. Blue lights glowed beneath, and a surprisingly loud sound emanated from the board as the magnetic… whatevers… groaned to life. And just like that, the board was hovering. I seem to remember one of the engineers passing something briefly beneath it, like a magician proving a woman was hovering onstage. I bent over just to see it at level with my own eyes = the board rested on nothing, an inch off the ground

Then, I realized that they weren’t planning on showing off their own skills yet. They were holding the board for me. “Are you ready?” the lead engineer asked. Up until that point, my ability to hover was like Schroedinger’s box. I could be great. I could be dreadful. Faceplanting on a sheet of metal would hurt in more ways than one.

But opportunity is not a lengthy visitor. And thus I walked over, took a breath, and stepped onto my first hoverboard.

Boarding the hoverboard after it's powered on - without it drifting away - is a small challenge

Boarding the hoverboard after it’s powered on – without it drifting away – is a small challenge

In that first wobbly crossing of the flat copper floor, I learned that friction is your friend. Without the force feedback to tell your brain the effect your weight shifts have on the board, it’s far too easy to overcorrect (and in reality you generally have to shift your weight in an unintuitive direction.) Rather than use one’s foot to push off, the thickness of the board means it’s easier to have someone else give you a push for starter momentum. And once you’re moving, the need for balance is constant.

On a slow pass across the floor. It helped to keep my knees bent.

On a slow pass across the floor. It helped to keep my knees bent and loose.

Surprisingly, my mid-foot strike Skechers sneakers (which I adore) made things a bit more challenging, as it made me more sensitive to wobbles in the board. You can see me rocking back and forth on the board in the video. If I were to do it again, I might wear my “barefoot” running shoes or even dance shoes that give my heel and toe equal purchase.

We did a first short round and took a break to watch some videos of other hoverboard users before coming back for one longer session. The batteries on the board don’t last long yet, but I got to use the full charge since I was a solo visitor. (Unexpected luxury!) In the second round, we practiced stopping and spinning the board (verrrrry slooooowly).

It’s very hard to steer without friction to play with, and more than once I ended up hitting the lip of the halfpipe, stopping the board. Interestingly, some movement is enabled by exploiting inconsistencies in the board (it favors one side, so shifting weight towards one side results in a slight spin.) But I am proud to say that I never fell off in any sort of spectacular fashion (though I did, of course, have to step off to regain balance on multiple occasions), and as far as I’m concerned that makes the whole thing a success.


Look Ma, no friction!

I was also delighted to meet the team and hear their stories first-hand. I love that this small team of people (and it is a small team!) made this a reality. Many had to accept their job blindly due to the secretive work, and I empathized with that, having accepted my Amazon offer for an undisclosed project. Great opportunity often hides behind great risk.

Once the board ran out of batteries beneath my feet, we wrapped by taking a look at a future application of the Hendo technology. I’m not sure it’s public knowledge, so I’ll refrain from describing it here. But I was inspired by the possibility it represented. There’s more than half-pipes in their future if all goes well. After a few final posed pictures, we parted ways.

In just a few days, Hendo will be unveiling their next-generation hoverboard, funded by the Kickstarter campaign earnings. I’m sure the version I rode will soon be mounted on the wall in their ad-hoc hall of fame. I hope the noise is a bit under control, and hopefully battery life is increased. But it will likely never become commonplace – unless sidewalks become conductive surfaces, in which case…

But as far as Kickstarter experiences go, being one of the first few hundred human beings to ride a hoverboard is going to be hard to beat. My family was REALLY into the Back to the Future trilogy. The trilogy releases were a Big Deal. Went to Universal Studios when the (terrible) BTTF ride opened and waited 3 hours in line. But never did I think while watching that I’d be one of the first to ride a functioning hoverboard.

And the egalitarian nature of the experience was interesting to me, too – that pledge level was open to everyone, and the price was reasonable as experiential rewards go. Admittedly, $100 is a lot to many people, to say nothing of travel costs if needed. But in 1989 when these movies were made, cutting-edge technology wasn’t financed by the public. I suspect only venture capitalists and the wealthy had access to inspiring experiences and technologies in their true infancy. As it turns out, our version of 2015 is slightly more egalitarian.

Sure, our pledges only got us 5 minutes on the board (7 in my case – bonus time!) … but isn’t skydiving a pretty short experience too? And the team made me feel incredibly special and welcome despite my proportionally small contribution. To hear scientists talk about their work in person was a delight.

I left thankful for the opportunity and inspired about the future. But with the oft-cited Back to the Future future almost in the past, it’s time to look forward and dream even bigger. And by that I mean building warp drives. Who wants to launch THAT Kickstarter?


A Day at Chambers Bay: The US Open

Yesterday, my husband and I spent a very long day at Chambers Bay, WA for the first day of championship play of the US Open.

These tickets were my Christmas gift for my golf fan husband. Such a rare opportunity couldn’t be missed since he plays golf regularly in his hometown of Tacoma, mere minutes from the action. Of course, by the time the idea occurred to me in September, the only championship round tickets available were Thursday – so I snapped up a pair and the waiting game began.

Neither of us had ever attended a PGA tour event before, and of course this is one of the largest of them all. We were spared the first test – parking at the large community lots – thanks to the kindness of one of my husband’s childhood friends, who let us park at his home and gave us a lift to the course.

The course has a beautiful hillside view of the Puget Sound, and feels very expansive – because it is. 30,000 people a day, with only 18,000 grandstand seats – meaning over ten thousand people wandering the paths at any time.

We got to see all the big names – Mickelson, Mcilroy (nice birdie!), Watson, Furyk, Els (rough go), Spieth, Garcia (awkward bunker shots into a par save), Woods (birdie!), etc. etc. – along with some impressive amateurs and some unexpected leaders, like Martin with his birdie on 16.

Some observations from the experience:

Grab the day’s schedule and map at the security station. We used this CONSTANTLY. Would have been permanently lost otherwise.

Our favorite view was from the small bleachers at the 16th hole. We started there by random chance, finding it not the most difficult hole shot for shot, but it had a really interesting green with weird hills and irregular bunkers. Lots of players hit par, but they were either hard pars from bunker saves, or missed putt pars from the cup’s very weird lip. We saw a triple bogey, but we also saw Tiger hit a birdie (more rare than a unicorn lately). The small 16th bleacher is more steeply raked, meaning less obstruction from other spectators – and the height gives it a view of the entire hole, from tee to cup. PLUS it’s on the water, so: water breeze! But your mileage will vary. I’ve heard recommendations for the 15th hole too. If you’re taking the bleacher route you may want to bring a blanket or cushion, as they’re small and unforgiving.

Wear hiking shoes or walking sandals. The course is incredibly sandy. My flexible sneakers weren’t the best choice. We didn’t walk as much as some folks and still put in over 7 miles during our time there. And save energy – depending on what gate you came in, you may have a LONG uphill climb to leave the course. My clothes literally looked like I’d come from safari afterwards, so be prepared to track dust everywhere. It might not be as bad today due to rain, but by Sunday it’ll be blowing sand again.

Avoid the 17th Bottleneck. There’s a choke point near the spectator crossing for the 18th hole, and thousands of people get tied up there. If you can avoid it, do.

Be prepared, but don’t overthink it. Don’t panic about bag size down to the inch. The purse I brought in was more 8 x 6 x 2, not 6 x 6 x 6. That said, they won’t let large-looking opaque bags in. (Basically, NFL stadium rules.) Bags had to go through an X-ray machine. Lots of people, my husband included, were forgetting to pick their items up after going through the security gates. Other things you may want to bring – empty transparent collapsable water bottle, which can be filled at the first aid stations, and some wet wipes or hand sanitizer (it’s dusty and the restroom situation is far from ideal.)

This course doesn’t lend itself well to following players. It’s big, it’s long, and it’s hilly. Much of the walk is literally on sand, so imagine walking 7-10 miles down a lumpy beach. Certainly hundreds of people DID follow Tiger Woods and his trio, but they were jostling for a few spots along the rope, and from many of those spots you could EITHER see the player OR their ball. We got much more enjoyment from sitting at a single hole and watching many different players tackle that challenge.

Bring a device with the official app. The “new scoreboards” are a nice idea, but they were glitching all day. For 3 hours, ALL course scoreboards were showing stats for the 18th hole, meaning the app, paper schedule or the standard board were the only ways to know who was playing. The board on the 16th hole crashed to the desktop a few times, hilariously. Also, the board at the 16th hole is behind the train tracks – if a large, slow train comes in you won’t be able to see it.

Keep your phone on battery saver or airplane mode when not using it. Signal is sporadic and that’ll drain your battery. If your phone dies there’s a charging station in the main spectator village.

Get food before venturing deep into the course. The best variety is in the spectator village. By the time you get out to 16 the offerings are largely chips and PBJ sandwiches, and the satellite locations had a hard time keeping stock replenished as the carts can’t get through the thickest crowds.

Protect yourself. As always, WEAR A HAT, SUNSCREEN AND SUNGLASSES. Even with clouds, the ambient light will get you. There is NO SHADE AT CHAMBERS BAY. None. Zero. Unless you’re in the merch tent or a port a potty (and you won’t want to stay there for long – the upkeep was dire.) You will also want a light jacket for the evening breezes.

Get a return ticket when leaving grandstands. Your return ticket is good for 45 minutes. If no one is saving your seat you’ll probably end up in a new spot, but it’s a really nice system for letting you get a drink, stretch, or bathroom break.

Our recommendation – choose one hole, camp out till just before the second half tee times, grab lunch, and either return to your original spot or move yourself. We were glad we went and I actually found the experience of watching a single hole more interesting than watching on television. Groups were fairly tightly spaced (though will be less so post-cut.)

Happy spectating!

For more thoughts, this article is good (but don’t take their advice about cleats – they are forbidden!)

User Testing: Unlocking Feedback with Post-Test Surveys

For the first time in a while, I’m designing and running my own usability studies. Carnegie Mellon cross-trains its HCI students in both research and design, so it’s a bit like riding a bike – it comes back eventually.

Post-university, I used to run studies and focus groups when I was a producer in the video game industry, because we rarely employed dedicated UX folks – but that didn’t diminish the importance of getting feedback about work in progress. At Microsoft, however, the field of user experience is split into two specialties – Design and Research – so I usually had the luxury of partnering with a dedicated researcher when seeking to observe the use of our products.

Amazon also splits the discipline, but the ratio of designers to researchers is far more extreme. With so much data to draw from, many observations are made directly from usage patterns. My project team is (relatively) small, with no dedicated researcher, and as a v1 product we have no existing data to analyze.

Thus it’s time to step up to the plate again – and as I designed my study, it included an interview portion, an interactive portion, and a post-test survey. I couldn’t help but question the inclusion of the survey. I’ve done it for most prior tests – including an early Disney Friends playtest with 8 year olds, which yielded critical insights, amazingly. (And also revealed that some kids felt their favorite Disney character was Shrek. Open ended question FTW.)

And yet in a way it seems silly – why not just ask the participant these questions, when they’re sitting right in front of you?

As I watch my users go through this process, I am reminded of the great utility of post-test surveys, strange as it may seem to watch someone in front of you filling out a sheet of paper.

Post-test surveys are particularly well-suited to get at specific types of feedback:

Less biased feedback: None of us can deny the bias that occurs when a moderator, perceived to be involved with the project, asks you how you felt about a product. Humans often aim to please, and what we’re told to our faces may not match reality. Surveys partially mitigate this bias. I have consistently found that participants become more open about shortcomings in experiences when telling them to a piece of paper, rather than a human being.

Quantitative feedback: We often want some sort of numerical value assigned to satisfaction so we can compare across users. However, it is awkward to ask participants to rate a series of experience dimensions numerically, and verbally it is easy to lose track. Not to mention that many times participants will want to go back and change prior answers once they get the “hang” of whatever rating system you’ve chosen (1-5, agree/disagree, etc.). Putting this request in survey form gives more consistent, less biased results.

Feedback from bashful participants: Yesterday, I was impressed at the thoughtful detail I got in a post-test survey from a participant who was particularly quiet, despite stated enthusiasm for the product. The survey gave her an opportunity to express herself in a way more natural to her. And as mentioned before, she opened up on both extremes – negatives and positives.

So you want to add a quick survey to your study? A few tips and cautions when designing your post-test survey:

Lead with the negative.
I remember this advice very clearly – given by my late professor Randy Pausch in his Building Virtual Worlds class. There were many surveys in that class, both after user testing our worlds and on our fellow students. Randy cautioned to always lead with asking for negatives – otherwise, the positive experiences recollected will dim the recollection of the negative. And if we’re being honest, the negative feedback is more often the more critical of the two. By leading with the negative, you’re sending a subtle subliminal signal that it’s OK to be critical.

As a result, I always start each post-test survey with 2 questions, in this order:
1. What were the two worst things about the experience?
2. What were the two best things about the experience?

Be very clear about your quantitative scale, and check for error before ending the session.
It happens – no matter how clearly you think you’ve laid out your 1-5 scale, someone’s going to flip it accidentally. While I’d recommend against heavily scrutinizing survey results in front of a participant, choose a couple of barometer values to peek at. If they don’t align with the reaction you’ve gotten in the test, inquire.

Keep open answer questions to a minimum.
Beyond the 2 questions above, I try to limit myself to only a few open questions. In many cases, open questions can be asked in an interview format, which also allows you the opportunity to drill down into interesting answers and gauge your participant’s unspoken emotional response to the question. This also generally means you’ll have a video clip of said answer, which speaks louder than printed words when convincing key stakeholders.

For open answers, provide lines that hint at the length of feedback sought.
Everyone’s handwriting differs, and those with small handwriting might become overwhelmed by the sight of a blank space. A single line indicates a sentence or two at most, and may actually free your participants to respond.

Gauge purchase intent if it’s a retail product.
It never hurts to ask whether a participant would not just use a product, but purchase it for themselves. It often changes things significantly. Similarly, asking what price a participant would pay for an experience can be particularly eye-opening.

If it’s not a direct-purchase product, gauge how likely they’d be to tell the people in charge of purchasing about any preference they have. This is also important in case your screener captured folks genuinely not in your target market.

Don’t repeat your questions.
Did you get the information you’re asking for in your screener? Often you can get most demographic info you need from there – make sure to keep those responses so you can cross-reference them.

Leave the participant some space to complete the survey.
It’s understandably awkward to sit and watch a participant complete a survey – they will likely feel rushed and may change answers. Even if it’s not necessary, I like to leave the room for a bit – and if not possible, I will busy myself with other tasks, like resetting equipment, while keeping an ear out for the inevitable pencils-down moment.

Administer the survey as quickly as possible, ideally BEFORE any followup interviews.
You might inadvertently bias participants by drilling down too deep prior to the survey. If your test is very long (2+ hours), consider breaking the survey into several smaller chunks to get feedback closer to the moment of experience. Don’t wait until they leave the lab.

Post-test surveys are a much different beast than their cousin, the SurveyMonkey style of asynchronous question-and-answer series (often delivered to a much larger audience). Post-test surveys require a lighter touch and a bit of work setting the mood to create a canvas for constructive feedback that doesn’t have to be delivered directly to your face. It’s cheap, fast, and a very valuable tool in your tool belt.

Consider this the first in a series of a few posts on research for designers, assuming I’m not too busy editing videos and hating the sound of my own voice to write further. Until then, happy surveying!