Data + app design and making mobile stuff: a conversation

I had the opportunity to take a phone call with Errol Samuelson, president of Realtor.com and talk about their new iPad app. We covered some of my favorite topics: impact of data on design decisions, how people use mobile stuff and so on.

Even if you aren’t part of my real estate audience I think you’ll find some useful tidbits in here on how an organization develops and iterates using data and design processes.

[vimeo clip_id=”22915522″ portrait=”0″ frameborder=”0″ width=”500″ height=”400″]

The audio is about 25 minutes long and the image doesn’t change. So you can let it run in the background or read the transcript below.

A conversation between Gahlord Dewald and Errol Samuelson re: the Realtor.com iPad app, data, user behavior, mobile, and other exciting things.

Gahlord Dewald: Were there any kind of data driven decisions that you guys made when you were putting together this app? In terms of what kind of information and feedback were you gathering on your own site? Like some of the features like the mapping and all of that. Or was it mostly a collection of good ideas and thought you’d just give it a whack.

Errol Samuelson: There was a bit of both. We have a design team that’s got good experience doing mobile. We had the experience of doing the three phone apps already.

Some of it came from direct user feedback from folks that had used some of those apps. Some of it was good ideas.

We do have a usability lab in our San Francisco Bay area office. So what we do is we bring people in and give them our apps. And we’ll be watching them through the two-way mirror and record their actions.

“We did look at a lot of the usage statistics on our iPhone app. To take a look at what were some of the features people were clicking on. That helped us decide what information to surface in the tabs.”

A lot of the good ideas we had, we brought folks in to test them to see how they were used. For example, this idea where you want to show a list of property and the map at the same time was a result of that testing.

One of the things we found was that you want to have the list and the map simultaneously but sometimes you want more map real estate. So this idea that with one quick tap have the list contract like an accordion and disappear was a result of that testing.

What we found was that folks were interested in sometimes having the list and sometimes showing as much of the map as possible on the iPad screen. The iPad with the bigger surface area just plays so nicely with seeing the map.

We did look at a lot of the usage statistics on our iPhone app. To take a look at what were some of the features people were clicking on. That helped us decide what information to surface in the tabs. And in what order on the listing detail page. So there was some experiential stuff there.

So it was a combination of the three:

1. What were people doing on the phone app
2. Some good ideas
3. User testing

Gahlord Dewald: You showed off that it has a lot of built in feedback functionality as well. On the conference call you made a point of saying you’re not just relying on getting stars and ratings in the app store.

You have a whole variety of different feedback mechanisms built into the app itself.

Errol Samuelson: Part of this stems from the feedback we get on the website today. We have something called Opinion Lab on the website. Every week we get a summarized report that goes to all the executives in the company–all the product folks get it.

If there are items on the Opinion Lab that look interesting you can click and dig into the specific written feedback from users on the site. We found that to be incredibly helpful.

We look at the Opinion Lab surveys in two ways. One is sort of top pain points, top like points. But then we also look at the trending.

For example, the speed of the site may have been just fine four years ago. But if you see complaints about site speed starting to increase then either you’ve made a change to the site that has perhaps impacted that. Or peoples’ requirements are changing. A lot of times something that was just fine two or three years ago is maybe no longer fine today.

“A lot of time improvements can be simple tweaks. Small changes can have a big impact.”

We found that feedback so incredibly useful on the website. We said to ourselves, how do we get that same level of feedback versus star ratings or folks who occasionally add their comments to the app store. Our sense was that we could get on the one hand more granular, more detailed information. But on the other hand also encourage more people to comment and give us feedback.

If you’re like me sometimes you’ll be on your iPad and get prompted to give a rating on the app store. And if I’m busy I’ll say no, not right now. No thank you.

“If you ask the question and then ignore the answer then you’re not really fulfilling that implicit promise.”

But putting the feedback button on the top right hand of the screen, the idea was to make it always available so we can get more feedback. And yes, to your point the goal is to have rapid iterations and constantly improve the app.

A lot of time improvements can be simple tweaks. Small changes can have a big impact.

Gahlord Dewald: That’s one of the things I noticed. In your feedback form there were a lot of different opportunities for people to give various kinds of “voice of customer” data. By asking for that kind of feedback, it’s almost an implicit promise that “Hey we’re going to make changes.”

It sort of puts your feet to the fire. If you collect all that feedback then you’re going to have to make your app better. Because you’re sort of promising it in a way.

Errol Samuelson: That’s right. If you ask the question and then ignore the answer then you’re not really fulfilling that implicit promise. But I think it’s a good kind of pressure. So we’re ok with it.

Gahlord Dewald: Yeah I think that’s great. As a webhead and marketing data analytics kind of guy I was really pleased to see that built in. Partially because the kind of feedback that app developers get through the iTunes store isn’t always relevant to the task that they’re trying to improve.

For the question of “Is this a good app or not?” well the app has 75 different features one of them was bad so I’m going to leave a good review or one of them was good so I’m going to leave a good review. It’s hard to get granular with that.

Errol Samuelson: I’d much rather find out “I like these two things and I hated that one thing.” So if I’m reduced to a binary decision or even a star rating it’s difficult to get that kind of feedback. I’m with you.

Gahlord Dewald: Exactly. Especially when you’re dealing with something like an iPad which has a lot of screen real estate relative to an iPhone. But there’s only so many features you can show a user at once and knowing which ones they like the most helps make that decision a little easier.

I have another question, the app looks great for finding a house. That’s obviously its primary audience. Is there anything in there for people trying to sell a house?

Errol Samuelson: I think there’s a few things. If I’m looking to sell my home what I’m most interested in are things like

• “How should I price it?”
• “Is now a good time to sell?”
• “What are market conditions like? Long time or short time to sell?”
• “What’s my competition?”

Those are the kind of questions I’m going to typically ask a home seller.

For example, if I go into a particular area and I ask myself “How should I price my house?” The Area Scout feature gives you that real time tracking where I can see the average list price and square footage. So that gives me a general feel for the market. If I’m first thinking about selling my house this is probably a good starting point.

Then I’m going to get more specific. I’m going to say “Show me homes that are similar to mine–bedrooms, bathrooms and so on in the same general neighborhood as mine.” One of the challenges you’ll face is just because two homes are close to each other doesn’t mean that you’re going to price them similarly.

I’d much rather find out “I like these two things and I hated that one thing.” So if I’m reduced to a binary decision or even a star rating it’s difficult to get that kind of feedback.

For example, in LA if you’re in one part of Los Angeles you’re in the Beverly Hills school district. Which is considered to be a very sought after, very desirable school district. They have a lot of good funding, the teachers are good, the schools are good, the test scores are good and everything else. So it’s much more desirable than being in the LA Unified school district.

I can look at two homes, one on either side of the street. One is in Beverly Hills school district and one is in LA Unified school district. They can be identical houses and one is going to be noticeably more expensive than the other.

That’s where you can enter your criteria for the similar home into the Realtor.com iPad app and then you can use the area highlighter features. So in my LA example I would exclude everything on the right half of the street. Another example is in Palo Alto, depends on which side of the freeway you’re on.

If I put those together I can do a search and then get very specific and say “Here’s my competition, what’s currently for sale.”

The third thing I would do is look at those homes and see how long they’ve been on the market. Have they been on the market 15 days, 30 days, 180 days? If I’m seeing really elevated days on market one of the things I know is that you get the most views and lift on a listing when it first goes on the market.

“It’s not a little button that jumps out and says “Home sellers click here!” but by doing the same search, but from a home buyers perspective, you can very quickly learn what’s happening in the market.”

So if I saw my competition and their price and then if I saw that they were on the market for a very long time. My personal strategy would be to price materially lower than the competition to get the attention and get the hit quickly. Because I know otherwise it’s going to take a long time on the market.

Quite frankly in some cases if you price it at a lower level you can get enough interest you can actually get a kind of bidding war and do really well.

If I’m a home seller those are all the kinds of things I can deduce just by looking at the data on the iPad app.

I guess it’s a little different. It’s not a little button that jumps out and says “Home sellers click here!” but by doing the same search, but from a home buyers perspective you can very quickly learn what’s happening in the market.

I guess the final thing that’s of interest is that I might be inclined to pull up the listings in a neighborhood and just go through them. I’d see which real estate companies and agents seem to specialize in that market. I could very quickly tell who really has a presence in my market. It would mean they have a list of home buyers that they’re already working with.

So I would also use it to find out which agents and brokers are more active in the area.

Gahlord Dewald: What about some of the features to help Realtors? Given that people are getting it from Realtor.com we’re going to expect something there.

Errol Samuelson: There are a few. One is perhaps not obvious but is really important.

Because the data is accurate and fresh, and we’re doing 15 minute updates. The moment a listing is off the market we pull it off. If there’s a price change we reflect the price change.

This goes a long way to make sure that people who are inquiring about a property are adequately self-qualified. If I’m calling on a property I know what the actual list price is. I’m also not getting upset because the property is no longer on the market. I call you up and you say the property hasn’t been on the market for a month then you have that unpleasant conversation with the consumer.

It’s actually a surprisingly big issue with websites and apps out there. I don’t think it’s actually reported on enough. There’s a lot of really bad data out there. Surprisingly, double digit percentages on some sites.

Gahlord Dewald: From a search engine optimizations standpoint, people hanging on to old data is helpful for them because they generated this content and it’s out there and maybe they’ve got some links to it or whatever.

So pulling it down is difficult for some people to do. Even though it’s obviously the right thing to do–for all the reasons you just mentioned. So you don’t have to have uncomfortable conversations with people. Why show stuff that isn’t there anymore?

That is an important issue and you’re right it doesn’t get talked about enough. I think part of the reason is that there are people who perceive value in keeping it there when it’s out of date.

And there’s the technical challenges. You guys must gone through some technical challenges to poll all of that data every fifteen minutes without melting down your server.

Errol Samuelson: It is incredibly expensive. You have to have lots of servers, lots of bandwidth, lots of capacity.

I’ll give you another reason why I suspect people leave the data up there. And it’s kind of an interesting story.

We power the real estate search experience at MSN. About a year ago our contract with Microsoft was up for renewal. So we were sitting in Redmond. And they were referring to another real estate site which shall remain nameless. But it was one of the bigger ones.

And MSN said “These other guys have more listings than you do.” So there was this perception that better quality was more comprehensive because the listing count was higher.

We sat down with MSN and went through things listing by listing. We compared it to what was actually in the MLS and discovered that “Oh this one’s not on the market, that one expired, that one sold, this one has the wrong price.”

“I think mobile apps is where people will discover if the Emperor is wearing any clothes.”

But I think some sites want to inflate their numbers for listings count. It’s uncomfortable to come back and say I’ve only got 200 matches when on Realtor.com there’s 400 in the market. So I think that may be another reason.

Here’s where it gets interesting. I think mobile apps is where people will discover if the Emperor is wearing any clothes.

So now it’s not where people are sitting at a screen unable to verify the information. I’m now potentially in the neighborhood, I’m in front of a house and there’s a nice big sold sticker on the sign or it says price reduced. And I look at my mobile app. The question is “does my mobile app still show the sold property on the market or flag it as a price reduction?”

I think mobile apps are going to drive the need for websites to become more accurate if they aren’t accurate today. I think that’s a dynamic that is going to be pretty interesting.

Gahlord Dewald: You’re right. People are in specific locations looking at property and if the app that they’re using isn’t reflecting the real world that they see in front of them they’ll probably have questions. I think that will be an interesting thing to watch. I think that’s going to be a really interesting thing to watch.

I’ve been watching this argument between data quality for years. Some sites have the highest quality data and some of them don’t.

It seems to me that the consumer hasn’t cared that much. They’ve cared, but they haven’t cared enough to just not go to the sites that don’t have MLS-driven data.

Errol Samuelson: I’m not sure that they know. I’m not sure they can tell, unfortunately. We try to educate them.

Gahlord Dewald: I think that’s exactly the point, there’s a steep learning curve.

In some ways it reminds me a little bit of the middle years of computing before Apple switched over to the Intel processor. You’d have Apple fans trying to explain that their computers were faster when they had a slower clock speed. And others would be saying “Well it’s got a smaller number it must be slower.”

Trying to understand those differences for consumers, I think, has been a challenge.

Maybe you’re right. When you’re sitting in front of a sold house using your mobile and the app you’re using doesn’t show it as sold obviously this would have an impact on your trust in the app.

Errol Samuelson: I believe it has the potential to kill your brand if you have bad data. We’ve made a big investment in this. It’ll be interesting.

Gahlord Dewald: Right, that’s your business hypothesis. I think it’s a good one. I’m definitely interested to see how it shakes out.

One thing I was thinking about as you were talking about the mobile experience vs the website experience: it’s common these days for the thought leader geeks to be talking about designing for the mobile first.

For example, get the mobile device experience nailed down and then iterate out to your website from there. There’s a part of me that looks at the Realtor.com iPad app and you’ve got all kinds of features that aren’t on your main site. Some of which would be very difficult to apply to Realtor.com

What do you see going forward? A primary shift to mobile device? Or what’s going on there?

Errol Samuelson: One of the luxuries of designing for the mobile device is that you get to start from scratch. There’s no legacy software. You get to start with a white board and say “What would I like to do.” That’s quite liberating.

I think at times there are things you decide to do on the mobile device that you’re absolutely going to port over to the website. We’re doing this.

But I think the bigger thing is that it’s a different medium. It’s like the early days of television–what people did is they ported radio shows and brought them on to TV.

Then people realized that TV was a different medium. So you can actually design for the medium. I think that’s where the power is going to be around mobile.

That’s what we’re trying to do. This is not a port or a “mobile version.” It drives me crazy when people talk about the “mobile version.” Like it’s a sort of variant.

“I expect to see more divergence of feature sets instead of convergence.”

I think it’s a whole different animal. It’s sort of Marshal MacLuhanesque. Where the message is the medium. I think we have to start and say “What is special and unique about mobile?” And then say “How do I make this app leverage those special things?”

Something really amusing happened to me the other day. Sitting in front of the PC in the office. And I could have easily typed in “realtor.com” into my web browser. Instead I reached over to my iPad and turned it on. There was something on the mobile app that I couldn’t do on the PC–the GPS features.

I genuinely believe that, as things progress, there’s going to be some commonality between desktop and mobile devices–they’ll both have listings and stuff like that. But I think, if you’re doing good design the mobile apps will continue to extend, morph and grow along a dimension which is tailored to what they are.

I expect to see more divergence of feature sets instead of convergence. I think the best features will be customized for the mobile device.

One of the things to think about, for example is if you have a PC in the home we share it. My wife uses it, I use it. But the mobile device, my phone, is uniquely mine. No one really uses it except me.

So all of the historical data, the usage data are really about me. They’re personal. And I think that let’s us customize more than we would ever dare on a web device because you don’t know if the last user is the current user.

The thing we thought was stunning is that on the website, we get significantly more email than phone calls. On the mobile phone app it was the exact opposite.

It makes sense. You’ve got a phone in your hand. Why not just press call. Are you going to stop and then pull up a keypad and start typing a message?

Gahlord Dewald: Right, it’s certainly easier to talk on the phone when you’ve got an iPhone than it is to type an email on it.

Errol Samuelson: That’s right. So what’s funny about that, when you think about agent behavior–the mobile devices are going to demand a different kind of behavior.

So for the last ten years we’ve been banging on people as an industry to say “answer your emails, get back to people with x number of minutes or hours.” Almost to the point where people have forgotten about the phone.

We ran a test last year right after we launched the iPhone app. We ran a test because people were actually phoning more. “What happens when they phone?”

We ran an 800 number on a bunch of the listings, with the brokers’ permission to do this. The brokers said we could listen to the first 15 seconds of every call. So we did that.

What we discovered is that 70% of the time, if the call went to voicemail, the person just hung up.

So now all of a sudden, if I’m getting a lot of phone calls–and not every agent is able to answer every phone call every waking moment–then you better have a strategy to handle those calls and not let them go to voice mail.

There’s this almost unintended consequence. The mobile device drives different behavior. The different behavior then drives a new business need. I think the cascade of what’s going to happen is very interesting.

I guess it’s a long winded way of saying to your earlier question, I think the best designers might migrate features back and forth–but they’re going to design for the medium or the device. And hopefully design stuff differently.

Gahlord Dewald: It just makes sense. Given that mobile devices are carrying a lot more sensors around with them. Your regular computer can tell the website owner only a few things. But your iPhone or Android or any mobile device is going to be give you things like location and phone number and that sort of stuff.

Errol Samuelson: And if you look at the new devices, the phones and stuff that’s coming in the future it’s more and more sensor capabilities

They all have built in cameras and built in video cameras. Why not have the ability to take photographs when I’m touring a home right? Augment the site with my own photos. Why just give a five star rating?

I’m going through the home search process right now, and I’ll go to two or three open houses a day and they’ll all start blurring together for me. If I could, in the house if there was something I wanted to remember, snap a picture.

Now that’s not something we do today. But as an example that’s something you would not ever design for on the computer version.

Gahlord Dewald: Yeah, it’s just not there because there’s no camera on your computer and you’re not going to lug your computer around with you.

Thanks so much Errol for taking the time to talk about these issues and about your new iPad app. I wish you the best of luck with it.

I hope I’ll catch up with you at one of the conferences this summer.

Errol Samuelson: Are you going to the NAR midyear conference in May?

Gahlord Dewald: I might be there at the end of the week. If you’re going to be there let’s grab coffee or something.

Errol Samuelson: Yeah that’d be great.

Gahlord Dewald: Awesome! Have a good time!

Errol Samuelson: Alright, take care, bye.