CHURN FM
CHURN FM is the podcast for subscription economy pros. Every Wednesday we hear how the world’s fastest growing companies are tackling churn and using retention & engagement to fuel their growth.
CHURN FM
E247 | Mastering Mobile Retention: Insights from SoundCloud and VSCO with Andy Carvell from Phiture
Today on the show we have Andy Carvell, CEO and Co-Founder of Phiture.
In this episode, Andy shares the framework Phiture uses to increase mobile app retention, drawing from his extensive work with companies like SoundCloud and VSCO.
We discussed the future of mobile app interactions, including the potential of voice interfaces, AI, and wearable technology, and how these innovations fit into future growth strategies for mobile.
We wrapped up by diving into the importance of having a solid data tracking plan and Andy shared Phiture's systematic approach to help companies optimize their user engagement and retention strategies.
Mentioned Resources
Churn FM is sponsored by Vitally, the all-in-one Customer Success Platform.
[00:00:00] Andy Carvell: Retention is a slow moving metric. It's an output metric. And there are a lot of different inputs to retention all the way through the funnel. And it's, you know, you can run a lot of initiatives which have, you know, little micro improvements on retention, but it can be, take quite a long time also to see the results of that because you've got to wait. And if you want to look at month one retention, you've got to wait at least one month to actually see the results. And even then it's probably only going to increase a little bit from various different initiatives. There are not many initiatives that will move the needle multiple percentage points on retention.
[00:00:39] Andrew Michael: This is Churn.FM, the podcast for subscription economy pros. Each week we hear how the world's fastest growing companies are tackling churn and using retention to fuel their growth.
[00:00:51] VO: How do you build a habit-forming product? We crossed over that magic threshold to negative churn. You need to invest in customer success. It always comes down to retention and engagement. Completely bootstrapped, profitable and growing.
[00:01:05] Andrew Michael: Strategies, tactics and ideas brought together to help your business thrive in the subscription economy. I'm your host, Andrew Michael, and here's today's episode. Hey, Andy, welcome to the show.
[00:01:18] Andy Carvell: Andrew, thanks very much for having me on. It's a pleasure to be here.
[00:01:21] Andrew Michael: It's great to have you. For the listeners, Andy is a partner and co-founder of Phiture, a growth agency specialized in helping apply the mobile growth stack that he developed for mobile businesses. Prior to Phiture, Andy was a senior PM and retention team lead at SoundCloud. So my first question for you, Andy, is what does the future of mobile app growth look like?
[00:01:43] Andy Carvell: That's a really great question, because mobile, I think, is really a lot more about, like the user than it is about any specific hardware or software platform. Right. I think a lot of people, when they think about mobile these days, they're thinking about smartphones, they're thinking about app stores and things like that. You know, but ultimately I think mobile is really just about being with the user all the way through their day. And so depending on how far you want to look into that future, you know, I can certainly imagine a situation where, you know, the mobile phone as we know it today and the smartphone kind of disappears.
[00:02:18] Andy Carvell: Now we're already sort of seeing, we've already seen earlier durations of that, things like Google Glass and, you know, like the, I think Snapchat also tried some glasses thing, you know, wear, and wearable technology, things like that. And I think that, you know, these are still very much kind of like mobile devices. Also the Apple Watch, for example, is another example of a mobile device that's with you all day. It's running more or less the same OS as the phone. And, you know, I can definitely foresee, situation not too far in the future where we're interacting with apps or with brands through mobile technology, but without a physical device in front of us.
[00:02:56] Andy Carvell: It's either in the metaverse, it's AR, maybe there's, if we go a bit further ahead, it's maybe retinal implants or things like that. I think the device will be subsumed into the person, but mobile really will still be there and it's really about that, being able to be with the customer or be with the user all the way through their day at every point. And, you know, the level of interaction that people have already, even with, just with smartphones, you know, how many times you're using your smartphone every day. But these like kind of micro engagements and all kinds of apps and, you know, with all kinds of brands, I think that's just going to get even more seamless.
[00:03:35] Andrew Michael: It's crazy to think what experiences might be like as sort of this evolves over time, because it's definitely moving in that direction. And so today, I think when we think about a digital experience, it's really from a UI perspective and what it looks like on the screen and how users interact with it. And when we think about things we'll do for churn and retention, a large portion of those really revolve around, how can we encourage engagement within this visual interface?
[00:04:02] Andrew Michael: But starting to think a little bit like how does that change over time as things maybe become more, using more of our senses and not only our visual aspect to be able to communicate with apps. I think it's definitely going to be interesting. I don't know. Is this something that you think about at all at Phiture?
[00:04:19] Andy Carvell: Absolutely, we're thinking about it. Yeah. I mean, I remember I was asked quite a few years ago, I think 2017, I was asked to come into a company called Otto Group. They're like a big conglomerate of different, kind of, retail, digital retail businesses in Germany. They wanted a kind of an inspiring future of mobile type talk to kind of kick off their strategy day. I was very honored to be asked to go and do that. I remember even then talking about voice interfaces.
[00:04:50] Andy Carvell: Back then, this was around the same time that Siri was just getting a bit more adoption and had really been built into iPhones and the Amazon Echo and these kind of devices, which at the time, there was a lot of buzz around and maybe a little bit overhyped because they weren't particularly intelligent. Now what we see is like, you know, like ChatGPT has a voice interface and it's just like, it's crazy what you can do with that.
[00:05:16] Andrew Michael: Next level. Yeah.
[00:05:17] Andy Carvell: Yeah. All through your voice. I remember like back then, the sort of, you know, the way I was thinking about it, and I think this is still going to be a challenge for brands is like, if you're interacting with a voice interface, that's essentially, you know, this sort of like voice from the cloud that's going and doing things for you. It's an AI agent maybe, and it's, you know, maybe it's able to make purchases for you or make bookings or, you know, go and gather information and collate it and process it. You know, you're not necessarily interacting with the brands anymore directly. You've got an agent in between that's going and doing that. You're interacting essentially with a faceless voice.
[00:05:56] Andy Carvell: So I think there's a challenge there potentially for brands in the future. How do they stay relevant and keep that direct customer relationship when there's going to be a lot more, kind of intermediary potential agents and things like that happening.
[00:06:10] Andrew Michael: There'll be no such thing as push notifications. Like nobody wants, like something buzzing in the air or like being pinged unexpectedly as well.
[00:06:17] Andy Carvell: Exactly. Yeah. So yeah, we definitely think about that kind of stuff as well. I tend to–
[00:06:22] Andrew Michael: Nice.
[00:06:22] Andy Carvell: I kind of refer to what, what's happening in terms of the technology shift right now is that we're entering the brave new digital world.
[00:06:30] Andrew Michael: Yeah. No, I'm sure we could geek out on this stuff for hours. Like I know you also have a newsletter that you started recently when we chatted and mentioned, obviously talking a little bit in the future and what's happening with AI, well, definitely we can leave that in the show notes for the listeners as well.
[00:06:45] Andy Carvell: Sure.
[00:06:45] Andrew Michael: But I first came across your work actually in one of my early startups, which was a mobile app and was the mobile growth stack. I think that sort of was like, became a synonymous piece at that point in time in terms of growth and how to grow mobile apps. And I'm keen to understand like a little bit, obviously now you've founded the agency off the back of that sort of, what has been like the realization, like let's say from the time that you first launched it to today, like how much of that stack has changed and what do you think still like really holds true and is fundamental to growing mobile apps?
[00:07:20] Andy Carvell: Yeah, really great question, Andrew. So I guess like for a bit of context for listeners who maybe haven't seen the mobile growth stack before. It came about really just from a frustration that I had when I joined SoundCloud back in 2011, realizing that actually, even though they brought me on to grow their mobile apps, there were not many people in the company that really understood mobile very well. I was finding it quite hard to get buy-in for growth initiatives, which I thought were pretty obvious, like things like building relationships with Apple, getting… featuring on the app stores and things like this. Just to give one example.
[00:07:57] Andy Carvell: And I realized that I had to do an awful lot of time kind of like explaining and building cases for the initiatives that I wanted to run, which I think looking back is fair enough and was really great learning for me. But at the time I just wanted to get going. I just wanted to do more and grow these mobile apps for SoundCloud. And at some point out of that kind of frustration of having the same conversations over and over again, I'm like, damn it, I'm just going to put all this down on one slide, right? I'm going to draw a picture.
[00:08:27] Andy Carvell: And then I'm going to show it to, like the senior leadership team and be like, this is what growing a mobile app is all about. These are the key levers. These are the kind of initiatives or topics that we might want to kind of include in a mobile growth strategy. And then these are the ones I want to focus on this quarter, you know, and sort of just be able to point at a page, right? So I got it all into one page. And I, you know, that really worked, that really unlocked a lot of understanding within the company and it unlocked a lot of resources for me to be able to go and do great growth and retention work at SoundCloud.
[00:08:59] Andy Carvell: Kind of off the back of that, I decided to publish it as a blog post. And yeah, it actually really resonated with the industry. So to explain to your listeners kind of, what the mobile growth tech is, it's essentially a one page cheat sheet for mobile growth practitioners, mobile marketers, anybody who wants to grow a mobile app or game. And it's kind of split into different layers, different sections, acquisition, engagement and retention, monetization being the key funnel stages or the key business objectives of pretty much any digital business. And then it's underpinned by an analytics layer, analytics and insights, reflecting the fact that any good growth strategy should be based on data and informed by data and metrics.
[00:09:43] Andy Carvell: And so that basic structure has remained the same through, I think, four or five iterations of the stack. I think what, to my dismay, what has changed is that it's got more complicated. Every time that I've revised the stack, I want to simplify and abstract a little more. I always struggle a little bit with what level of abstraction is useful and what's the right level of abstraction that enables people to take that stack and actually apply it in their business and where am I getting too granular? Yeah. It's got more complicated. I've added a tech layer, but all of the kind of enabling tech that might want to be in there.
[00:10:20] Andy Carvell: In the latest version, I added, you know, some new stuff around predictive modeling, strategic management, stuff like this. So it got more holistic, but also, you know, even more busy.
[00:10:31] Andrew Michael: Busy? Yeah. No, I think the first iteration that I definitely saw, like, I have you to think is obviously for this now personally, if they say this, but it really was sort of like this blueprint. And especially like at that time we're building a team where like, the team wasn't really familiar with even the concept of growth and like what that meant and we're trying to, I was trying to at least like educate the team as a whole. And like having that as a one-page, it was like, the easiest thing in the world was like literally here's the blueprints.
[00:10:57] Andrew Michael: Like these are the different areas that we need to be focusing on. Let's start, like getting good at these aspects and you could then work with everyone on the team to understand, okay, where their skillset lay and what interests they show. And we actually like, in that company, we ended up acquiring over a hundred thousand, like users for the platform through like, following that process, which it was definitely like, that was probably the central piece and part of, like us having an understanding at that point. So very, very impactful.
[00:11:24] Andy Carvell: Wow.
[00:11:25] Andrew Michael: Thank you.
[00:11:27] Andy Carvell: That's amazing to hear.
[00:11:28] Andrew Michael: Yeah. And so obviously like you've taken this now and you've applied this to many, many different companies as well over the time. And today, I think it'd be interesting, like maybe if we could pick a company that you work closely with to dive into sort of this from end to end and how you're typically working with the company. I don't know. Is, anyone that comes to mind, do you think it'd be interesting for us to chat about today?
[00:11:51] Andy Carvell: Sure. Yeah. I think maybe VSCO would be a good example, like VSCO, their consumer app. We work mostly almost, pretty much entirely with consumer apps, consumer brands, typically on the larger side that are going to have already got some organic growth and traction in the industry. We're typically not working with, like pre-product market fit companies, but bigger companies and VSCO I think would be a great example. They're kind of a pretty well known consumer brand. They have a whole bunch of like, really cool video editing and photo editing capabilities within the app. There's also kind of a social discovery aspect for, like discovering other people's content. I think it's probably a reasonable case study. You know, I can explain.
[00:12:35] Andy Carvell: Some of just how we generally approach like looking at like retention engagement and subscriber, subscriber dynamics of subscription churn and use VSCO maybe as an example at some key points as to sort of specific things that we did that drove impact.
[00:12:51] Andrew Michael: That'd be great. Yeah. And I think VSCO, I was familiar with it as well. I previously had it on the phone. I ended up churning, but I think that we can understand a little bit about it. But yeah, I mean, I, at that point I had like, probably like 10 different video editing and photo editing products on my phone, as well as testing different things out. Nice. So, I mean, obviously, like you say, they came to a little bit of a later stage. Like where was the business at the time when they came to you and what does maybe the challenge that they had?
[00:13:19] Andy Carvell: Yeah. So typically I'd say, they're pretty typical in that case in that, you know, they're already very successful, already internationalized to some extent, so they had users from, you know, all areas of the world. And typically when companies come to feature to look for our help, it's usually because they've seen growth, kind of, plateau a little bit. Maybe they've started to see a little bit more increased competition in the landscape, in the category. You know, maybe they've got competitors that are springing up, kind of copycatting them and kind of eating away at their market share a little bit. And I'd say that's pretty typical. They want to get a bit more systematic and data-driven about various aspects of their growth.
[00:13:59] Andy Carvell: And that was pretty much like how VSCO came to us. They're already a very successful app, really sort of biting at the heels of some of the more traditional video and photo editing apps out there. It's a pretty competitive category at the best of times, but VSCO's got a really strong brand and a fairly unique proposition in there. They were doing perfectly well. They had a subscription product for a while, so they were generating a decent chunk of subscription revenue. But they kind of had the feeling that things were not growing as fast as they could be.
[00:14:33] Andy Carvell: There were probably some optimizations, some things that they could do if they got a bit more systematic and a bit more data-driven about their growth and looked maybe, got some help from a company that can help them be a little bit more holistic about that. And also maybe help them to leverage, I think they'd recently licensed a new platform. I think that they were using Braze for their customer engagement platform.
[00:14:54] Andy Carvell: And they were looking for some expert help, like a team that could really help them to get the most value out of that and really tie the pieces together from what they were seeing in their analytics to actually interacting with those users in terms of segmentation and specific interaction campaigns that could then help to actually move some of those metrics that they would then see back in an amplitude kind of analytics setup.
[00:15:22] Andy Carvell: And so yeah, that's pretty typical, basically, that we would have. We would be brought into… to help kind of join the dots with data and help to make the first party data that a brand or an app is working with to help them make that data work more for them and to drive more impact.
[00:15:39] Andrew Michael: Nice. So who are you going to call when your growth starts to stall? It's like Phiture.
[00:15:46] Andy Carvell: I like that. I might steal that.
[00:15:48] Andrew Michael: I like that moment, Inflexion because I think there's definitely like a lot of companies that hit this point in time where they see quite a bit of success and at some point, like the growth starts to stall. They need to start getting a little bit more systematic in their approach to growth. And I'm interested in like at this stage, like what are some of the first things that you looked at for VSCO? Like what did the process look like of evaluating sort of where the perhaps low-hanging fruits were or where you could get started with them?
[00:16:15] Andrew Michael: Because obviously, as you said, they're quite an established company as well. And it's nice to hear from an external perspective as well, because sometimes you have your own internal biases and your blind is on to areas. So what did the process look like for you to sort of understand where the opportunities lay for them?
[00:16:30] Andy Carvell: Yeah. So we always start off with, you know, with an audit. We don't always call it an audit because some, I think, that can sometimes scare clients or client teams like this, you know, some external people are going to come in and audit our work, you know, like it's the scary thing, you know, nobody wants to be audited. It’s what tax people do, right? So I think we need to sort of think about how we brand that sometimes when we talk about it. But however you want to talk about it, before you can start moving metrics and moving things in the right direction, it's really important to look at the data, to look at what's going on, what activities are in place at the moment, what's working, what's not.
[00:17:08] Andy Carvell: So whether you call that an audit or it's like kicking the tires or just doing your homework, also really important for us if we're going to be actually building new interactions with their users, it's really important to understand the brand and the product deeply, to really get under the skin and really understand what makes their product special, why do their users love it, understand their users, their user personas, who's using the product, why are they using the product, all of that stuff.
[00:17:36] Andy Carvell: It's kind of a mixture of qualitative and quantitative. We kind of want to really get into that kind of soft stuff of really understanding the brand identity. Super important if we're going to then start communicating with their users, but also to understand on a quantitative level what's going on. So of course, that means things like doing funnel analysis, looking at all of their key funnels, their onboarding funnel, their subscription funnel, how many people are going from a free user to, into a trial, and then trial to paid, paid retention, paid renewal, all of those kind of things. Looking at frequency of use, what is a typical user, how often is a typical user using VSCO, how often are their power users using it.
[00:18:19] Andy Carvell: And then getting deeper into that, a bit more granular looking at things like feature adoption. So, what features do they have in the app, what's available on the free tier, what's available on the premium tier, and in those two tiers, what is the adoption of those features like? And we go a little bit deeper and look at ways that we might try to influence more people to start a trial or to convert from a trial to a premium, that feature adoption question becomes super relevant because it's like, we also want to understand which of the features that are highly adopted during the trial are most correlated with people actually converting into paid.
[00:18:59] Andy Carvell: More fundamentally, it's just really understanding what are the, probably two or three features in an app which are delivering the 80% of the value. A lot of apps, particularly when they're more mature, the product team's been rolling out more and more features and doing feature launches for a few years. Apps can get quite busy. But what we typically see when we do these, kind of, audits is that there's maybe only one or two features which users are really leaning into and using regularly. But there might be some really cool stuff that they're just not seeing, either because it's not exposed in the UI or you're not onboarding users into those features properly.
[00:19:35] Andy Carvell: You know, sometimes they just never even discover them or maybe that they're just not actually relevant for certain audiences. And they should, they've maybe actually built a lot of stuff that's… that they don't need to build because actually their core value proposition is maybe a bit simpler.
[00:19:51] Andrew Michael: Let's double click into this. Cause I think it definitely is an interesting area of exploration as well in terms of like, what features really do drive value and ultimately like, why do people come in? And as you said, like a lot of times as more mature companies come to being as, like, product team has just shipped a whole lot of product and then people ultimately don't really understand what's driving value for the company or for the business and what does your analysis look like from this perspective? So what are some of the things like types of analysis that you're doing to determine the key drivers of value when it comes to features?
[00:20:23] Andy Carvell: Yeah, great questions. So I mean, I know you've done also like the reforge courses, for example. Also, I'm also a huge fan of Brian Balfour and the Reforge stuff. I think that, you know, I did, I remember doing the retention engagement course a long time ago, reforge. And I think there, that really kind of, like turned me on to doing correlation analysis and really looking at like, you know, what actions are correlated with, like people being into, like a longer term cohort of X or Y. So things which are correlated with longer term retention, or things which are correlated with monetization. So we've definitely leaned into that feature and we definitely run a lot of correlation analysis.
[00:21:03] Andy Carvell: But honestly, what I find to be a super easy kind of simple way to look at that, we do a lot of just segmented retention. So we just look at a retention curve. Instead of just looking at the blended retention curve of all users, which generally doesn't tell you that much, actually, because typically, your users are not a homogeneous group, but they have differences. But, so when we come into a new customer and we're doing this kind of audit, we segment that retention curve by any segmentation variable parameter that we can think of, more or less. There's some obvious ones that we always go to. But depending on the app or the audience and what data we have available, there might be other segmentations which is, interesting just to see…
[00:21:49] Andy Carvell: If you break out your retention curve by users who've done this, or users who came in from this acquisition channel, or really the world's your oyster. It can be demographic stuff, acquisition data that you're looking at, like down funnel, and of course, user behavior. So things that they've done, features that they've interacted with, I guess is where it comes back to your question. And just by segmenting the retention curve out by people who've used feature X versus the average… versus people who haven't used feature X. You'll see some standouts where those curves look quite drastically different. And of course, it doesn't have to be a retention curve. You could also look at monetization or other variables, but by the same kind of segmentations.
[00:22:32] Andy Carvell: But I find it really useful to look at, like users who've received a push notification versus users who have never received a push notification. You'll see the received push notification curve is always way higher. It kind of stands to reason because push is a pretty good channel for getting people to re-engage with the app. But when you see it on a curve and you can see the difference, like that gives you a good, also it's like a, just an instant snapshot of, you know, where the opportunities lie and it's really quick to be able to do those kind of segmentations and to break out those curves in products like amplitude or mixed panel, et cetera.
[00:23:09] Andrew Michael: Yeah. You make it sound really simple as well. I think like a lot of times people like they have the best intentions of doing these things, but then there's typically like, a lot of barriers that come into play. So like a lot of companies don't maybe have the, aren't in a fortunate position to have really good clean data where they can do these forms of analysis effectively.
[00:23:26] Andy Carvell: So true.
[00:23:27] Andrew Michael: How often do you find where you go into a company and you just have what you need or how, like, difficult is actually to get the data out to be able to do this?
[00:23:35] Andy Carvell: Great question. I would say even the top, you know, top quartile of companies, so, you know, the ones that are doing things really well and the ones that are, like really on top of their game. Even those folks don't have totally clean, totally like working data taxonomies, there's always homework to do on the data taxonomy. Now, if we come into a situation and it's, and they're literally not tracking anything, you know, that would actually, we wouldn't even start working with a company like that, there's something that we would kind of do in early discovery. And then we can help them to build a taxonomy and a tracking plan. That's also stuff that we can help with, but we wouldn't be able to do proper retention work with them at that point.
[00:24:16] Andy Carvell: What's more common is that we come in and we find that the taxonomy is a bit of a mess. It's maybe not documented anywhere, so we can actually help them document that. We look at parity between web, iOS, and Android. They're tracking the same things. Also, of course, that depends on product parity, but assuming the iOS and Android products are functionally identical, typically the data taxonomies won't be because the tracking will have been implemented by two different teams. We look at naming convention. There's a lot of cleanup we can do. So typically, we do that work right at the start, before we start looking at retention curves or anything else.
[00:24:54] Andy Carvell: We create a bunch of Jira tickets for their developers to prioritize those based on… these events are just missing or broken. And you're really going to need these to do analysis or to do segmentation or targeting or triggering later. So these really need to be done. These are things which would actually add additional nuance to the data or allow you to do additional personalization, whatever. So we recommend these highly. These are kind of nice to have, you want to maybe clean up your naming convention, et cetera. So we try to, you know, because we know that engineers are always, you know, they always have a huge backlog. They usually also not massive, also not massively fans of implementing tracking for days on end usually.
[00:25:34] Andy Carvell: So we try to make it like, you know, really prioritized and sort of red flag the stuff that really needs to be fixed. And we tried to do that really early on because we know there's going to be a lead time before the developers can get onto it. And so we flag that stuff. We work with the data that we've got, assuming that it's good enough that we can do. Usually the data is in a reasonable shape, but we're also flagging the stuff early that we know will help us to do deeper analysis or better user interactions later.
[00:26:04] Andrew Michael: Yeah. Yeah, because I think that's definitely like the thing that holds most companies back from doing something like this and having the analysis is just not having the clean data and I think having like a analytics taxonomy and getting started right from the beginning is probably one of the most critical components. Cause a lot of times like when people start, Oh, let's start tracking things. It's like, okay, like what do we want to track to create this huge long list and think they need to track everything. But I think also, like one of the tricks is in the beginning, like start small, but just make sure that you document everything along the way. And there's a clear lineage in terms of, like what you're tracking and what the purpose is, where it's coming from, what are the sources, like having this in place is critical.
[00:26:45] Andrew Michael: And I think this actually is something that we discussed with Elena Dorfman from Segment. And just one of the things is like in the past, they used to sort of allow people just to onboard straight away with their product and get started immediately from signup. And what they realized was like that actually caused an issue for them in churn is because the biggest value that segment is a product, it was giving you good clean data, because it allowed you to send events from one source instead of multiple different places. But ultimately like it was only valuable if you'd done the first step, right. It's like defining what those events should be and what the property should be and having a really good solid tracking plan.
[00:27:18] Andrew Michael: So I think like my advice always like whenever anyone's thinking about doing analytics at their company, it's like, that's the first place you need to start is like get everyone in your team aligned on like what the tracking plan should look like, what it needs to be tracked. Okay. You've got a list of a hundred, like start with five, start with 10, like get them implemented, get them working, like see the value and then expand from there. And like, I think less is more in analytics. Like obviously over time you grow in sophistication, but the more you grow in sophistication, the less helpful it becomes to the majority of the company as well. And like, then you start getting into the weeds and nobody knows what anything means anymore and it's like, you don't have trust and you go back in like this vicious circle.
[00:27:55] Andy Carvell: This is a fascinating topic. Yeah. Just a quick kind of side story on this. There was a big dating app that we worked with. I'm not going to name one particular one, but when we came in, we saw that they had a lot of their own data infrastructure. They built a lot of very, very cool, very advanced stuff. They essentially built their own version of Segment. That's another question of build versus buy, whether that makes sense. But they've done that. They're hosting all their own data on their own infrastructure. Their data costs were relatively low and they considered themselves very data driven as a company and they were tracking absolutely everything down to micro interactions of just scrolling the screen or having seen a certain element on a mobile app screen.
[00:28:44] Andy Carvell: I don't think that even really makes sense for a web app. It certainly doesn't make sense for a mobile app. What it also meant was first of all, they were capturing gigabytes of data per user, per session, which was just insane. They had terabytes and terabytes of data that were capturing daily from all this usage. And I'm like, what are you using this data for? And they're like, oh, but well, we might want to look back at this. We do run AP tests. We might want to examine it. How are you even going to? They had like 4,000 events. I'm like, who understands this taxonomy? Who's going to actually be looking at this data? And you know, it couldn't get, like clear answers on that because I think very few people, you know, just as you said, like it became kind of an impenetrable heap of data that nobody really could dig through and it becomes less useful for sure.
[00:29:32] Andy Carvell: It also actually was binding them to that solution because they couldn't easily go and get a segment or something like that because actually, you know, the data point costs would be prohibitive if they just try to track everything that they were currently tracking and send all that to segments. So yeah, that was a challenging situation.
[00:29:51] Andrew Michael: It's almost like they're replicating like screen recording software just through the events that they were tracking. You'd probably play it back and watch. Yeah, no, it's definitely, I think that's also like the, looking back in time as well, that's something as well as a fallacy, I think like at Hotjar, at some points, like we had this big mess and we were like, okay, like we need to fix it and we sort of said, okay, like we're going to start from fresh again. And we debated and said, like, should we save the old data? Like, what are we going to do with it? And at some point we just said like, screw it. Like, let's just roll new.
[00:30:21] Andrew Michael: And I think not once did anybody ever say like, Hey, like, let's go look back and see what happened two years ago or what happened like 12 months ago, which is like having a clean, fresh start to let us like, to forget of all the mistakes that happened in the past and like have something really good, clean and fresh to focus on and yeah, I don't know. I've never really, very rarely, like maybe once. And I was, like focusing, business intelligence, like I was, hit a business intelligence that I actually really go back and look at things, but.
[00:30:48] Andy Carvell: Yeah, I think it becomes less useful over time anyway, right? It degrades pretty quickly in terms of action ability, because probably the products moved on in the meantime. And hence, you know, the interaction data of, like how people were using the product a year ago, you know, might be kind of academically interesting, but like, if the product's not exactly the same now, then it's not necessarily so applicable. Yeah.
[00:31:11] Andrew Michael: So obviously you mentioned you're going in, you do all this work then. Now you've got some good clean data. You've got some good insights in terms of, like what's driving value and which segments are performing at VSCO. Like what was one of the interesting things that you uncovered and maybe like, let's talk through a specific use case where you went in and worked on a specific aspect of the product.
[00:31:31] Andy Carvell: Yeah. So, you know, typically, as I said, we look at the, like sort of key engagement funnels, the key payment funnels. We, of course, guided by the client in terms of what's most important for them. Sometimes it's monetization. They want to sort of, you know, often it's monetization if they're already monetizing as an app, but not always, right? Like they might just want to get more people through at the front of the funnel, or they might just need help actually kind of modeling where the best, biggest opportunities are, you know, like, because it's not always straightforward, particularly if you've got siloed teams in your organization to understand if like, you know, the…
[00:32:05] Andy Carvell: If it's going to be more valuable for you to invest in acquisition versus improving retention or working on your paywall conversion, for example, do you want to push more people through the funnel or do you want to optimize the funnel itself? Those questions aren't always as straightforward to answer as you might expect or at least to produce projections which actually give people an idea of what the potential upside of these things are. I think there's sometimes a modeling step where we can help people understand that.
[00:32:31] Andy Carvell: Yeah, so at VSCO, I think one of the first things that we looked at, and it's pretty typical that we see an opportunity in onboarding and activation. That's usually where you get the most drop-off in retention. Every retention curve looks pretty similar. It's just about how steep is that curve and does it level off, right? Yeah, so in VSCO's case, the retention was pretty good. Their activation rates were okay, but they were definitely keen to look at ways that they could get more people to activate, which in VSCO's case was uploading a photo. So it's basically getting a photo, either taking a photo on your camera and getting it straight into VSCO, or bringing a photo from your camera roll, or just in any other way, getting a photo uploaded into the app. And once you get your first photo, you're a creator on VSCO.
[00:33:15] Andy Carvell: So yeah, we looked at driving that up and we launched various experiments. We work on an experimental basis with all of our clients. In VSCO's case, as I mentioned, they were using Braze as a customer engagement platform. So we focused a lot of the work that we did with their marketing team on, how can they leverage the channels in Braze to drive various behaviors and drive up certain metrics? So when we started with activation and a photo upload, there were various different kind of experiments that we got into a backlog and discussed with the client and prioritized based on which ones we jointly thought would be the ones that might drive the most impact.
[00:33:57] Andy Carvell: Now some of that's a little bit on gut feel. And some of it's based on, we have a framework for this, a prioritization framework for prioritizing messaging and growth initiatives called the RRF. So we look at, like reach, relevance, and frequency. So how many users will this particular idea reach? This campaign idea, is it going to be targeted at just sort of power users or people like in a certain part of the funnel? Or in the case of onboarding initiatives, they typically have a very big reach because you, you're reaching users before they actually end up dropping off and not coming back to the app. And you're hitting all new users, which is usually a large cohort. So yeah, we came up with a whole bunch of different things to try there.
[00:34:38] Andy Carvell: One of the most successful actually was personalizing on time of day with a push notification around sunset. So we were able to cross-reference their location, time of day, and approximate sunset times, and to be giving them a message like, hey, beautiful sunsets today. You could use VSCO to capture the moment and capture the sunset. And so we had a push notification for that. We had also a mobile in-app message, which is kind of like a pop-up that you can deploy to a certain segment of users within the app. So if they were in the app at that time, or if they clicked on the push notification, they would get this Sunset in-app message, which had some examples of other people's Sunset pictures to inspire them. Like, you know, the sunsets look beautiful.
[00:35:24] Andy Carvell: And so I think there were learnings there around personalization, interacting with people at a certain time of day. So it should increase, relevance of the message, but also picking, like, cause we tried a few different kind of angles, like not just sunset, but we found that sunsets, you know, typically are pretty pictures with people that are very engaging with people. So we found that actually from a conversion perspective, having those, like beautiful looking sunset pictures converted a lot better than some of the other ideas we tried. So this was one of the more effective ways of getting people to upload their first photo.
[00:35:57] Andrew Michael: Very, very interesting experiment. Yeah. It's a very interesting experiment. I think that's the beauty with mobile where you can do these sorts of experiences, where you can like bring together different elements, looking at location, looking at time of day, and then being able to like have these really timely because like desktop apps, you're not really always at the desktop, but the mobile you always have it on you. So like you are one, you're going to see that notification, two like the data. I think I was just wondering, is, like, did you go the, step further of like, trying to understand the weather in this specific location, or is it just to generalize like sunsets today, like take a photo?
[00:36:32] Andy Carvell: Yeah, no, really great question. We didn't in this case.
[00:36:35] Andrew Michael: You probably discussed it there. Yeah.
[00:36:37] Andy Carvell: Yeah, and actually we have done something very similar in the case of a company called FishBrain. They have a social app for anglers, and we did exactly that. We had these weather specific notifications. We actually, we kind of… we were pulling data from a weather API, I think it was called Dark Sky. We kind of had, like a smart spreadsheet that we could use to kind of create copy for like, different kinds of weather conditions. So you know, it wasn't that we were only sending rainy day notifications, but we actually had different copy, different messages that we could use and also sort of update on a regular basis so they didn't become stale. Doing that in a Google sheet, but then Braze was pulling that stuff at send time.
[00:37:18] Andy Carvell: So basically at send time, at a certain time every day, users would get a message, hey, it's going to be rainy this weekend. Don't forget your fishing hat. This is an app for anglers, right? So people go out actually fishing in all weathers. So it was kind of ideal, you know, like, it's like, oh, it's going to be windy today. Hopefully, you know, this is not going to affect your, sorry, you're now showing my lack of angling experience, but maybe it kind of, you know, I don't know how it affects fishing, but yeah, you know, so, or it's like, it's going to be a beautiful sunny day, the day, like great day for fishing, this kind of thing. It's just a great question. Did we include whether, we didn't link it to that sunset campaign, but we have seen great impact with weather campaigns.
[00:37:56] Andrew Michael: Yeah. But I think in general, it's just more making it really contextual to the user in their circumstance and the next level of personalization as well. In that sense, it's like bringing in their surroundings and their environment into the communication that you have with them.
[00:38:09] Andy Carvell: Yeah. 100%.
[00:38:11] Andrew Michael: I see we, running up on time now, so I want to make sure I ask you a question I ask every guest like, what's one thing that you know today about churn and retention that you wish you knew when you got started with your career?
[00:38:21] Andy Carvell: Ah, okay. Great question. Yeah, when I started my career and actually when I started at SoundCloud in the retention team, I was really, like starting to think and dig into this topic in detail. So one thing that I really wish that I had understood more clearly is that retention is a slow moving metric. It's an output metric. And there are a lot of different inputs to retention all the way through the funnel. And it's, you know, you can run a lot of initiatives which have, you know, little micro improvements on retention. But it can take quite a long time also to see the results of that because you've got to wait and if you want to look at month one retention, you've got to wait at least one month to actually see the results.
[00:38:59] Andy Carvell: Even then, it's probably only going to increase a little bit from various different initiatives. There are not many initiatives that will move the needle multiple percentage points on retention. I wish I'd known that earlier because I think it's something that… which, a lot of folks, a lot of our clients still struggle with the understanding of how much, can they move this metric and how quickly can they move it.
[00:39:22] Andrew Michael: Yeah, definitely. And I think that's also a lot of the premise of the show is that there's so many different inputs that lead to the final output symmetric and like, it's not necessarily, as you said, like one silver bullet that's going to like make a huge impact, but it's really like a collection of different things that you can do to influence the final output. And they do compound though, I think over time, like as you start to work one and plus the other and things do improve like, sudden, like takes very long and then it's suddenly when you start to see changes as well. I think at least I'd… like what I've noticed working with different companies, but yeah.
[00:39:56] Andy Carvell: Yeah, good point.
[00:39:58] Andrew Michael: Well, Andy, it's been an absolute pleasure chatting with you today. Is there any final thoughts you want to leave the listeners with? Like anything they should be aware of to keep up to speed with your work or?
[00:40:07] Andy Carvell: Yeah, thanks, Andrew. One thing that springs to mind, I know we talked a lot about, like the importance of having a good tracking plan and a good data taxonomy. On the feature website, we actually have a downloadable tracking plan that people can use as a template that they can use. There's also some articles about how to put together a good data tracking plan. Since it's such a fundamental part of doing data driven growth work, I thought I'd mention it. We can include the links, I guess, in the show notes.
[00:40:32] Andrew Michael: Absolutely.
[00:40:33] Andy Carvell: Good stuff.
[00:40:34] Andrew Michael: Very nice. Well, thanks so much for joining and yeah, I wish you the best of luck now going forward.
[00:40:39] Andy Carvell: Thank you. Thanks. Thanks, Andrew. It's been a real pleasure.
[00:40:42] Andrew Michael: Cheers, Andy.
[00:40:45] Andrew Michael: And that's a wrap for the show today with me, Andrew Michael. I really hope you enjoyed it and you were able to pull out something valuable for your business. To keep up to date with Churn. FM and be notified about new episodes, blog posts and more, subscribe to our mailing list by visiting Chur.FM. Also don't forget to subscribe to our show on iTunes, Google Play or wherever you listen to your podcasts. If you have any feedback, good or bad, I would love to hear from you. And you can provide your blunt, direct feedback by sending it to Andrew@Churn.FM. Lastly, but most importantly, if you enjoyed this episode, please share it and leave a review as it really helps get the word out and grow the community. Thanks again for listening. See you again next week.
