(link to podcast)

0:00:15.1 Chris Ronan: Hello, everyone. Welcome to Inside Geocaching HQ. It is our podcast where we chat about people, projects, et cetera. That are happening at Geocaching HQ in Seattle. I am Chris Ronan. I am one of the 90 or so lackeys who works here at HQ. And today I will be chatting with Annika Ledbetter, who is a member of HQ’s product team, where she is the Senior Engagement Manager. We will be hearing more about what that means. Annika was my teammate when I first arrived here at HQ almost 12 years ago. At that time, she was on the community volunteer support team. She has since worked on the marketing team for a number of years before arriving to her current role. She worked a lot on a recent big project to add tools for hiding a traditional cache to the geocaching mobile app. And that is what we’ll be talking about today. In addition to hearing about the other interesting stuff that she does and has done. So here is me and Annika talking about lots of interesting stuff.

0:01:28.6 Chris Ronan: So Annika is here. And before we got started, before you walked into the room, I was looking at Facebook actually. And I saw that, I don’t know if it was a memory or what, but it was a picture of you and me and Heather…

0:01:37.7 Annika Ledbetter: Oh, no way.

0:01:46.8 Chris Ronan: From my first day at HQ.

0:01:48.2 Annika Ledbetter: What?

0:01:49.6 Chris Ronan: We went out and found, went caching in Fremont. I think there was probably a lunch. Cindy was probably… Cathy, they were probably there too. That was kind of the team back then on the, on the CVS team. And that was where you… Was that where you started at HQ was the community volunteer support team?

0:02:09.5 Annika Ledbetter: No, I started on the what’s now the CE team, the…

0:02:14.2 Chris Ronan: Community engagement.

0:02:18.9 Annika Ledbetter: Thank you. I was like, community entertainment team.

0:02:22.8 Chris Ronan: We have that all. There’s so many times here over the years where I’ll forget what a, or teams like, as you know, teams here at HQ, they’ll have sometimes their own like Team One or, and that’s changed a lot over the years too.

0:02:36.7 Annika Ledbetter: It’s changed a lot.

0:02:37.9 Chris Ronan: And sometimes I don’t even know like, okay, who’s that again? And how did they come up with that name? But yeah, CE I have to remind, or CVS even, because…

0:02:48.5 Annika Ledbetter: Community volunteer support.

0:02:49.8 Chris Ronan: Community volunteer support and if you’re in the US and you know that there’s a large pharmacy chain of that name. So that throws people off too. So yes, you started on the community engagement team.

0:03:02.9 Annika Ledbetter: That’s right. Yeah, I worked there for about a year. And then I moved over to the community volunteer support team. Also long history with that team. And then about another year in, I started working in kind of the localization realm, working a little bit more with engineering, and then moved over to marketing. And then how long is it ago now?

0:03:32.2 Chris Ronan: It feels like you were on the marketing team for a little bit.

0:03:35.0 Annika Ledbetter: A while. That’s where… Yeah, I was the longest at the marketing team. And then it’s kind of just a natural transition to the product team. I just started working more and more with product and moved over to that team, I think about two years ago. But as I’m getting older, time goes faster. And what feels like two years sometimes has been three years and I didn’t notice. So I think it’s two or three years.

0:04:01.7 Chris Ronan: And you’re senior engagement manager.

0:04:03.3 Annika Ledbetter: That’s right.

0:04:04.9 Chris Ronan: And what does that mean? What all does that entail for your day-to-day?

0:04:10.1 Annika Ledbetter: For my day-to-day, it means… So I think my role is more and more going into a classic product management role. Before that, I was working a lot in experimentation, running AB tests. But now it’s more the classic product management role that is basically, I don’t know how to best describe it succinctly, but you’re a bit of a sponge. You take in everything from community problems, needs to business problems, needs. And then you try to, in collaboration with design and development and other teams, you try to solve these problems and help guide a solution for these problems. I would say, yeah, you just kind of, yeah, take it in as a sponge, then you squeeze out, hopefully, an okay solution.

0:04:59.1 Chris Ronan: You talked about AB testing a little bit too. For people that aren’t aware, I think that might be something just to explain briefly what that is too, because that’s been something that I see. You and I don’t always work on a day-to-day together, but I will see you mention results of an AB test in one of our company channels. And that tends to inform which way we’ll go with sometimes what can seem to be minutia. And I don’t mean to in any way be talking down to something, but more just like, wow, an AB test about something like that, but it must be important. Otherwise, there wouldn’t be an AB test for it. So could you talk about what AB testing is just a little bit? We don’t have to spend a whole lot of time, but people might be interested.

0:05:48.6 Annika Ledbetter: Yeah, for sure. AB testing in a nutshell is that you provide two different experiences, or three or four. You can do ABCD tests if you want to. But you have people have different experiences in the app, different feature sets that they see, and then you track success metrics that could be, are they finding more caches? Do they go out more often? Things like that that we care about. We want people to go geocaching, obviously. And so then we look at those two groups of people that have a different experience. And then based on the results, we decide, yes, we want to roll out this feature. Yes, we want to invest more into this feature. Or no, this is not the right solution for the problem that we have tried to solve. So back to the drawing board, we do something else. So it does inform. And why often it’s smaller in scope is because if you throw too many changes at somebody at once, it’s going to be really difficult to understand what worked, what didn’t work. And so… But over time, the hope is you start with small changes, but as you learn more and more about what impacts what, you can make bigger and bigger bets. And so that’s kind of what we’re working towards too.

0:07:10.1 Chris Ronan: Yeah. And sometimes it’ll be the button is here versus the button is a little bit over here. And I’m thinking, come on, what’s the big deal? The button, the button. And then you’ll come back with the results and say, “Oh, well we had… We saw a much bigger…” It’s kind of fascinating really sometimes to see what the differences can be.

0:07:28.9 Annika Ledbetter: And I think what I think is best is it’s kind of, we all have assumptions and we all have strong feelings about things, but data doesn’t lie. And there has been, I mean, I’ve been humbled, but it feels good to be humbled by numbers.

0:07:50.8 Chris Ronan: Yeah, no, totally. There’s times when I see something and I’m like, “Come on, why even do a test on this? It’s obviously going to be…”

0:07:58.1 Annika Ledbetter: Yeah, good.

0:07:58.8 Chris Ronan: And then it turns out to be the other thing. And I’m like, “Okay, that doesn’t make any sense.” But like you said, numbers don’t lie.

0:08:03.6 Annika Ledbetter: No, they don’t. And yeah, we had something… I was so sure would have a positive impact actually had a negative impact where, where we were sure, okay, maybe it’s neutral. I would have been okay with that. But no, it did damage. And it was kind of, okay, thank God.

0:08:21.8 Chris Ronan: And it’s stuck with you because you obviously are still carrying that.

0:08:24.9 Annika Ledbetter: I know. Sleepless nights. No, but, thankfully we tested it, so. And I… That’s kind of, I feel very strongly about continuing that path because of those experiences, you know, we don’t want to put things out there that are not beneficial in the long term.

0:08:43.8 Chris Ronan: Sure. So speaking about that and how it kind of dovetails into our primary topic for today, which is the tools for hiding a traditional cache in the Geocaching app. So I guess we could start with how was it decided that this was something that would be pursued and was potentially of value to the community?

0:09:07.2 Annika Ledbetter: Yeah, so in my job we work with two types of data, that’s qualitative and quantitative data. And so there were in both of those, there were really strong signs that we should make hiding available in the app. On the quantitative data side, we saw that there were fewer and fewer new hiders coming into the game. And we all, I think, can understand that having caches available and new caches available for people to find is somewhat of the core of the game. And so we were, this was a something that we felt strongly we need to solve as a company and for the community. And then also when we did interviews with new hiders to understand why, you know, we are seeing a reduction of them, they often mentioned they were expecting it in the app. Why is it not in the app? They would hide a lot more often if it was in the app. And so that kind of gave us the last push. I think why, honestly, I would say the question to me is more why haven’t we brought it into the app earlier? And I think it was always a very daunting, huge project that felt really hard to, with, you know, we have so many different things we can work on at all times or want to work on at all times, so many competing priorities. And so, yeah, so it became a thing that was kind of pushed up, pushed off. So we did it a little differently, and that was by organizing it around a small team of dedicated people here at the office that worked together and was able to really, really focus on this one project. And that helped us get it, you know, put together and out there in a reasonable amount of time and hopefully also in the most effective way.

0:11:03.6 Chris Ronan: And so for people who aren’t aware, just kind of a quick rundown of what, so you go to the Hides tab in the Geocaching app.

0:11:08.0 Annika Ledbetter: Yes.

0:11:12.4 Chris Ronan: And off from there.

0:11:13.7 Annika Ledbetter: Yeah, there’s a, there’s a plus on top. And from there you can either choose that if you don’t have everything figured out quite yet, you don’t have the container ready and things, you can also just collect coordinates and save those. You have to be at the location for the coordinates to be collected. You can’t just, right, like, pen the map and drop a pin. And that’s on purpose because we want to make sure you are on location and get the most accurate coordinates you can versus just dropping a map pin.

0:11:51.7 Chris Ronan: And we should say the coordinate collector is something that had come out not too long before. So if maybe you haven’t hidden a cache in a while, you might not be entirely familiar with that feature, which is a great feature. I’ve, you and I were talking earlier and we’ve both used it recently. And so even if you’re not planning to use the hide a cache feature in the app, even just being able to use the coordinate collector is really nice. I was somewhere last weekend and, yeah, used the coordinate collector and it shows you the degree of accuracy for the coordinates you’ve collected. And that’s kind of a separate thing. I can go back and pull that up on the website or on the app later, which is really good.

0:12:36.5 Annika Ledbetter: One thing that I really like with coordinate collector, and then it integrates with the hiding, so when you start a cache listing in the app, you can either pull from coordinates that you have already saved or you can use it right then and there if you’re on location. One thing I really like is that you can then use, once you’ve saved the listing as a draft, you can then navigate to the coordinates that you have collected and kind of verify, right? You can see, “Oh, does that actually get me to where I took them?” And then if that’s not the case, you might want to go back, collect a couple more, try to average them. We have some instructions in the Help Center on how to average coordinates if you’re curious about that. But I think this is just a really cool way to verify that you collected them at the right spot or not.

0:13:30.1 Chris Ronan: So I’m just going through the release notes from the hide a cache, or hide a traditional cache in the app feature. So you can name your geocache. You can, as you said, enter coordinates or collect or tap the bookmark icon to collect coordinates. You can set your D/T rating, add cache details, et cetera, all the way up to submitting a cache for review. You can only do this for the traditional cache.

0:13:42.0 Annika Ledbetter: That’s right.

0:13:54.0 Chris Ronan: So what was the thinking behind starting with just the traditional cache?

0:13:59.2 Annika Ledbetter: The vast, vast majority of caches hidden by first-time, but all users really, by all players really, is a traditional cache. So we wanted to start there because we’re gonna be able to match about 90% of all the use cases out there. So it made sense to start there. It’s the least complex cache type to tackle. So that’s where we started and we wanted to see and learn what’s there to learn, where are blockers, what can we improve first before we consider other cache types.

0:14:38.1 Chris Ronan: So when you were going through the process of this, I guess, what were some of the obstacles or some of the things that, some of the concerns? I know, for instance, that you would be running this sort of thing by the reviewer community to ask them what they think about it. So what are some of the things that you tried to address or some things that maybe need to be addressed going down the road?

0:15:02.1 Annika Ledbetter: Yeah, so, right, I mentioned we had seen the concern of fewer new hiders coming into the game, and that’s the problem to solve. And what the feature is doing well is bringing in new hiders. And with that, it’s bringing in inexperienced players that have never hidden before. Some of them haven’t played the game very much. And so that brings some issues with it, right? They will not have a good understanding of the guidelines. They might not understand proximity issues. They might not understand some of the most, how to pick the best container or, there’s a lot that goes into a good cache hide. And if you lack that experience, there’s a steep learning curve. And that does create frustration not only for new hiders that come in really excited and bright-eyed and bushy-tailed, and this can be frustrating to have to adjust, and then also obviously for reviewers.

0:16:16.3 Annika Ledbetter: It makes it more laborious for them to respond to these.

0:16:22.0 Chris Ronan: Yeah, I still remember one of the first caches I tried to hide. I didn’t, and this was a long time ago. It wasn’t like this was last week. This was many, many years ago. But I tried, I didn’t know you couldn’t hide virtual caches. This was after they had been made a legacy cache type and well, well, well before any of the virtual rewards. But I had found them myself, and so I just, and the reviewer very nicely told me, “No, you can’t do that.” And certainly as the years went on, I’ve learned an awful lot about cache hiding, but I still try to remember that version of myself who was very earnest and wanted to do something nice, but just wasn’t aware that there really are this, the game really does, as you correctly said, have a bit of a steep learning curve. And I don’t know, sometimes I feel like new players are unfairly blamed for some of that stuff. And it’s just a reality though of our game that sometimes it takes a while to figure out all those little nuances.

0:17:30.7 Annika Ledbetter: Yes, and that’s what makes the game so beautiful, right? There’s so many layers to it. There’s so much to it. And then that’s also what makes it difficult to get your foot in the door a little bit as a new player, really excited and interested to be part of it, but not quite knowing how to do it right.

0:17:51.4 Chris Ronan: You said that, and this has been, I’m looking here, the release notes were in late March, so it’s been almost two months. You said that it is bringing in, or we’re seeing more.

0:18:03.7 Annika Ledbetter: Yes.

0:18:04.2 Chris Ronan: What are some of the other positives that you’re seeing either from the feedback or from data that you’re reviewing to determine the success or whatever of this project?

0:18:15.0 Annika Ledbetter: Yeah, I mean, again, we’re seeing more hides and we see more new hiders trying their hand at hiding, which is really great. The feedback, I think, mostly has been positive. And there was a few saying, “Finally,” so we’re like, “Yes, we agree, finally.” Yeah, so that’s, I think these are all really, we’re proud of it too, the way we approached it and with this really tight-knit group of people being really passionate about it and working really closely together. I think it was a really rewarding process and yeah.

0:18:56.8 Chris Ronan: And then what are some of the things that come to mind that you could see being improved? I don’t know how specific you can get, but it’s early days, clearly. But are there some things or just in general, some thoughts that you have about where this feature could go over time?

0:19:16.6 Annika Ledbetter: Right now, we will not be able to dedicate that much focus and time to it in the very near future. But I hope that we will be able to address some of the most critical issues. There’s some around communicating with the reviewers that we see that there could be improvements being made. There’s some about just educating new hiders. There’s the problem with communicating proximity issues more clearly and better. So these are all things that we know of and we know as problems that we would like to address at some point, but yeah, I don’t have a timeline for or firm promise on resolution on those.

0:20:05.2 Chris Ronan: How do you evaluate this over time? It sounds like you’re looking at statistics of various sorts, and then are there qualitative feedback mechanisms as well that you’re able to look at too?

0:20:20.4 Annika Ledbetter: So mostly right now it’s quantitative data from, you know, looking at how many are being submitted, how many have been published, and that ratio is really interesting to us. And then qualitative, we are planning to do some interviews with hiders to get more insights, especially those that attempt and fail, and to see and look deeper into issues that they’re encountering.

0:20:45.6 Chris Ronan: So if you have not tried this new feature, open the Geocaching app, and it is in the little bottom, the hides tab, correct?

0:20:57.5 Annika Ledbetter: Yeah, the hides tab. You can access it from the bottom nav in the app, or you can go to your profile. There’s another access point to your hides and…

0:21:06.1 Chris Ronan: Ah, yes, so I should have opened it while we were talking. But yes, the hides, the little star there at the bottom. And then my profile, and yep, there it is.

0:21:16.9 Annika Ledbetter: Yeah. One thing we forgot to, you can also edit caches in the app. So this might be something that people are not as aware of too. We did with it, we are allowing editing, so you can do the whole process within the app.

0:21:30.5 Chris Ronan: Very cool. And you were saying that you are also working, you have a cache of your own, so you’ll be using this feature.

0:21:39.1 Annika Ledbetter: Yes, I did. Actually, I did collect coordinates yesterday. Some of them were a little, had low accuracy, so I have to go back. I want to go back another day and collect a couple more so that I can provide the experience people deserve.

0:21:52.5 Chris Ronan: And I’ve used the coordinate collector many times, including just a few days ago, but I’m not hiding a traditional, I’m hiding a multi. So, but maybe someday.

0:22:01.6 Annika Ledbetter: Maybe someday. Traditionals are cool.

0:22:05.1 Chris Ronan: They are. That was very convincing. Now I need to go hide a traditional cache too.

0:22:08.6 Annika Ledbetter: You do. You do.

0:22:12.1 Chris Ronan: Okay. That was Annika Ledbetter, the Senior Engagement Manager on HQ’s product team. If you haven’t used the Geocaching app to hide a traditional cache, maybe this will inspire you to give it a try. And if there’s a topic you would like to hear about on our podcast, please, please get in touch. Our email address is podcast@geocaching.com. And in the meantime, from me and Annika and all of the lackeys at Geocaching HQ, happy caching.