A Product Market Fit Show | Startup Podcast for Founders
Every founder has 1 goal: find product-market fit. We interview the world's most successful startup founders on the 0 to 1 part of their journeys. We've had the founders of Reddit, Gusto, Rappi, Glean, Cohere, Huntress, ID.me and many more.
We go deep with entrepreneurs & VCs to provide detailed examples you can steal. Our goal is to understand product-market fit better than anyone on the planet.
Rated one of the world's top startup podcasts.
A Product Market Fit Show | Startup Podcast for Founders
For 5 years he struggled to get traction. Then with AI he grew 12x— in 1 year | Scott Stevenson, co-founder of Spellbook
Use Left/Right to seek, Home/End to jump to start or end. Hold shift to jump forward or backward.
Spellbook is ChatGPT for lawyers. They've raised $30M in the last 6 months. They now have over 2,000 law firms as customers. But it wasn't a straight line-- it took Scott 6 years to get here.
For the first 5 years, Scott worked on a legal tech platform called Rally. During that time, Scott along with his co-founders and their small team ran hundreds of experiments, pushed dozens of landing pages and built a process to test feature after feature.
In this episode, Scott goes through his process to test product-market fit. It was because of this process that he was able to jump on the AI-enabled opportunity and grow 12x in one year.
One of the intuitions to stay lean until you find PMF was right because we did question that a lot. It was very hard to maintain that conviction with some really difficult conversations with stakeholders on why we weren't scaling out or growing faster. I would tell myself, no, that was right. I probably could have saved myself some sleep if someone had told me that, that – more people had told me that, that was right.
PabloWelcome to the Product Market Fit Show brought to you by Mistral, a seed-stage firm based in Canada. I'm Pablo. I'm a founder turned VC. My goal is to help early stage founders like you find product market fit. Scott, welcome to the show. It's awesome to have you here.
ScottThanks for having me Pablo. Great to be here.
PabloNow you run a company that's called Spellbook. Originally that same company was called Rally Legal. I remember maybe a few years ago you spoke with me about that opportunity and I was not smart enough to see the future but maybe more so to understand your own perspective into everything you need to do to find product market fit, which at the end of the day is the key characteristic I think in the early stages. How fast can you move; how many experiments can you make, because if you just keep doing that, at some point you'll get it. That's exactly what happened here. Spellbook now is like the ChatGPT for lawyers. It could be I think a cleaner positioning, but it took a long time to get there. Maybe to start, we can start in the early days just to set context like what was Rally legal and maybe more so how did you come up with that idea?
The idea of Rally
ScottYeah, we started Rally. There's myself and two co-founders Matt and Daniel. Daniel was a lawyer. Matt was a mechanical engineer who had started a toy company. I come from an engineering and product background and I had started my first small naive startup straight out of the university. Even before this, we launched this musical instrument and I got my first little angel check of 20K. Before I knew it, half of that angel check was gone on legal fees and I was like, what just happened? It was crushing for me. I thought that was a lot of money at the time for a company to have and I was really frustrated with that. I went down to become a director of engineering at another company for a while, but this problem really stuck in my head of like, wow, it's really hard to start a company when it costs so much just to get basic legal work done. This must be a problem for a lot of people. That became my mission and I was plugged into the smart contract and Ethereum space for a while. I was like, oh, for a while I thought smart contracts are going to take over and everything's going to become a computer-based contract and it's all going to be cheap. I went to Ethereum hackathons and things like that, but we knew we needed to build something that people would actually use and that lawyers would use and lawyers didn't seem too receptive to using Ethereum and smart contracts at the time. They knew there was this pain. This set me personally on a mission to figure out how do we make legal services more accessible so we can have more entrepreneurs, so we can have faster sales cycles, better employment contracts? Everything in the world runs on contracts. It's friction that is in every single transaction. As an entrepreneur, I would always been looking for what's an impactful mission that I could dedicate my life to? When I thought about the experience I had and when I thought about the scale of this around the globe; if you go to the strike website, you see that globe with all the financial transactions happening around the world and their whole thing is increasing the GDP of the internet. I'm like, well, if we could decrease the friction of legal transactions by 90%, what does that do to the GDP of the earth? How many more people can start companies?
PabloThat's the high level to decrease friction, which I really like. What was the first pain point and value prop that you went after and how did you land on that specifically?
ScottI'll preface this with us plus really our director of growth as well. We launched over a hundred landing pages, so we've had a lot of different value props. I could tell you about a lot of experiments that we ran.
PabloWe'll definitely go through that.
The First Product
ScottThe very first thing we launched and it happened almost in a couple weeks was really a template library for founders. Actually the very first thing we did was like, screw lawyers, we're just going to bring templates to founders and whatever they need we'll have. We launched that very quick. It wasn't pretty successful.
PabloLike an employment contract, a safe doc, whatever, a bunch of NDAs?
ScottYeah, exactly. We had a bunch of people immediately sign up and pay us five bucks a month and we're like, oh geez, this still isn't – it's going to take a long while to become raw and profitable on this. It was a hard sale because what we realized is even if you could automate the creation of these documents for people, they still had no idea what they say. They still need someone to trust. They still need someone to explain it to them. We really quickly pivoted away from that. At the same time, a law firm guy named Todd Bissett in Waterloo at Miller Thomson came to us and he was like, "Oh, our law firm is actually looking for a templating technology and a way to streamline startup paperwork for our own clients. We saw what you have and it looks actually really compelling." We actually worked with him and his team for a number of weeks to actually hone the product for law firms and to interface with clients so they could then use our product to capture a bunch of information from clients. They could send a questionnaire to clients and then suck that into templates, build a whole bunch of docs, and then do their bespoke thing. Then that became – I would say the core of the real Rally product was this Rally for law firms.
PabloWhen you had this lawyer come to you and tell you about this problem set, did you just start designing with them or did you also go and run discovery and all these classic PMF motions?
ScottWe definitely went into hyper focus on their problem. We actually worked out of the office for a little while. We were alongside lawyers who were doing real legal work. We hyper-focused probably too much on my customer to be honest. Probably over-fit our product a lot.
PabloThat was your first product? This is like 2019, you've got this product for law firms to help them streamline their process of making simple documents. Is that maybe in broad stroke what it was?
ScottYeah, exactly. Then we ended up raising a pre-seed round from some east coast investors early on after we had some initial revenue. We got a handful of other customers on law firms on board who were great early supporters. This is still 2019. Then we did a lot of iteration there to get to something where we felt there was really deep PMF. This is where the long – those first customers were super exciting but then it was at the valley of despair. I'm sure you've seen the graph from YC of the typical startup trajectory. Really exciting getting these first customers; then getting to true PMF was this valley of years really.
PabloWhat happened? Walk me through that because I find this is so important to understand and hard to understand from the outside is if you find one customer, five customers that really love it, you just assume there's got to be a thousand and how can it be harder? It is got to get easier, not harder. What's your thinking on that? Why does that happen?
Experiment to Measure PMF
ScottI think the biggest thing for us is I don't think the initial customers – they love the idea of it, they love to buy it, they love the concept of it, but actually getting real usage, real engagement, real habit change, that was the hard thing. Even with these customers, getting them to truly set up templates in our system, time to value we realized was really important. It took us a long time to figure that out, but they had to do a lot of setup and configuration to get this to work. Then once they got it set up to work – so they had to basically set up these fancy legal templates in our system. Then often the legal teams would be like, "Oh, this is great, all the other lawyers are going to love this. My work's a little bit too bespoke but all the other lawyers are going to love it." We would hear that over and over and over again where they so want to improve their process, they so want to embrace technology, but when the rubber meets the road and they have a live deal they're like, oh, this is too bespoke for this templating engine and they wouldn't use it. From a usage perspective it wasn't hitting the way we wanted. I think especially if you're working with a large customers, large law firms, the danger there is the implementation cycle can be a year, can be two years to technically get something rolled out past all the committees and to start getting feedback and do all the changes. When all that time goes by, you don't know if you have product market fit or not. That's really, really difficult. I think we ended up switching to focusing on small nimble firms who just adopting is really quickly. Then our iteration cycle sped up a lot. We could easily see like, okay, we still don't really have deep PMF. The other way we measured PMF was CAC payback period. We were just of a little bit religious about what's our CAC payback period? You doing this motion. If the CAC payback period wasn't clearly well under 12 months we were like, okay, we don't have PMF.
PabloYou're going after smaller clients. You mentioned running many different experiments. You mentioned hundreds of landing pages. Can you dive in on that? What were some of the things you were testing? Then you mentioned CAC payback period. What were some of the outputs that you looked at to see if those tests worked?
ScottSplits; our director of growth was also a huge part of this also. His name is Kurt Dunphy. We rolled over a hundred landing pages with different value props. Often there was a new feature, a little feature we built to angle the product in a certain way. We monitored these experiments at a variety of levels. Our goal was to run an experiment every two weeks. We would look at a lot of different things. We wouldn't just launch landing pages, we would also launch ads with those landing pages.
PabloWhat's an example of that? Just take us through one of those experiments that you remember.
ScottYeah, so one of the experiments which actually resonated a lot from a sales standpoint was what we called our menu of services pitch. We actually turned our templating system into a menu of services or a Shopify for law firms that they could put on their website. Now not only did they have templates, they could actually advertise like, "Oh, you need a will or you need an employment agreement, come into my website, click on need an employment agreement, put in all the info." That gets submitted to the lawyer. The lawyer goes into the lawyer's template. The lawyer does their massage on it, and the product goes back to the customer. That was one of our actually hit experiments on the sales side was allowing the lawyer to almost set up a little Shopify shop.
PabloWhy do you think that resonated?
ScottIt resonated because lawyers want to make money. They want clients and they want customers. That's one of their biggest pains is where's my next dollar coming from? Where's my next customer coming from? The idea of having a digital storefront was really compelling. How we knew this worked? First we ran ads and people were clicking those ads and we were looking at the cost per click and the cost per conversion into our demo meeting. What's the cost to get a lawyer into a demo meeting? It was way lower than anything else we had run before. We knew that within two weeks of running these ads. We had so much data. We threw up an ad, threw up a landing page. What is the cost to get someone into a demo? We were like, "Wow, this is 90% lower than anything we've run before. They love this." Then we took that and then we were able to go and say, okay, let's take this experiment to the next step. Let's actually roll this feature out to all these customers, implement it and see how it goes. This is where the problems started with that value prop. Again, the time to value issue, getting connected into their website. Lawyers could get this thing up and running, but they had no idea how to drive traffic to it. They're chasing this mirage of – and I'm sure Shopify has this issue too. They probably have tons of people setting up stores and no traffic going to those stores. That was actually a killer problem for that direction for us.
PabloIt's again this notion that reality doesn't meet the idea? People expect, oh my God, it's going to drive all this traffic, but then the work it takes and all the nuances around it don't let them realize that value?
ScottExactly, and that passed the CAC payback test. I think our CAC payback was probably six months on that whole motion or less, but it didn't pass the retention and the usage test. We looked at usage retention cohorts a lot.
PabloHow much time? That one's tougher because don't you need months of data to see if it flatlines or do you get to see it faster somehow?
ScottIn this case actually – I'm probably over-explaining it because in this case, I think you could see what we call the wiggles of death, which is the customer size of the product. You see those first like, oh they logged in, oh they did something and you see a couple wiggles and then nothing. At our company we say actually most things you launch, that's the pattern you see with the wiggles of death. The user is like, "Oh, there's a new feature, I'll play with it." You see a couple little spikes of people using it and then nothing. In my mind that is the default for most companies when they launch anything. I was talking to another company the other day. They built this amazing customer portal and they were amazed that after months of hyping up this portal selling it to customers, they looked at the logs and no customer had ever logged into this portal almost anytime. It had all been their own internal users using it and they thought people were using it. I was like this is normal. This is actually what I see. This is the dirty secret of SaaS is actually getting anyone to use anything. Especially in 2024 when you're so distracted, there's so many other more exciting things to look at, getting anyone to even log into a web app is excruciatingly –
PabloReminds me of my my usage of threads.
ScottYeah, exactly. Yeah, I logged in once or twice. The wiggles of death, that was my exact usage pattern. I logged in, I went there twice.
PabloThat was one experiment. What were some of the others?
ScottIt's quite a lot. I think client questionnaires was something we focused on a lot. We had this idea of taking info from a client and feeding it into a template, but actually the questionnaire in its own was useful as a product because when you're onboarding a new legal client, you need to figure out do they conflict with any of your other clients? There's a whole bunch of KYC information you need. We also had this client questionnaire angle of like, watch your lawyers, watch your client questionnaires. That was one that did pretty well. It didn't have that amazing physiological reaction from the customer on it. Some of the others, more advanced types of templating, we even went in terms of – client portal is also something we launched like, hey, lawyers, now not only do you have all these templates on this stuff, you also have a portal that your clients can log into.
PabloMaybe one question that comes to me is, everybody talks about you just run a lot of experiments, that's how you're going to have higher odds of finding something that has true product market fit. The other current is that even there's outside pressure, but even most founders I speak with, they just want something that works. They just want something that scales. You have this like, yeah, you would want to just run experiments endlessly until something just takes off but also you just want to find something that works and scale it. How did you manage staying disciplined and what just was that process like while you're running all these experiments?
PMF Should Feel Like a Physical Reaction From Customers
ScottIt was excruciatingly hard. We had very high conviction that product market fit should feel having an almost physiological response from your customers where when they see the product their pupils are dilating, there's a neurochemical reaction happening. I remember reading a quote. I can't remember exactly where it was from. I can't remember. You might know. It was a quote that basically said, if you have product market fit, it should feel like basically the product is being pulled out of your hands faster than you can provide it. Your servers are on fire; your salespeople are burnt out. That's what it should feel like. I can't remember where I had read that but it really stuck in our heads, true product market fit. This obviously doesn't always happen, especially in enterprise sales, sometimes it's just to grind the whole time. We decided we're not scaling the company, we're not upping our burn and we're just staying incredibly lean and agile until we have a product that people are pulling out of our hands faster than we can provide it and we're having enormous growing pains and everything's on fire. I would say that to our board. Our board's awesome. They've been very supportive but they are very skeptical about this. Everyone so wants to believe that the thing that they're doing is the right thing. Everyone so wants to move on to the next stage and to start scaling, moving on to the next thing, hiring people after you raise money. There's just a lot of pressure to put that capital to work as quickly as possible and scale the thing and everyone wants to believe that you already know what the thing is.
PabloWhat are some of the arguments because this is where it gets really interesting? I think the rubber meets the road here. On the one hand, you're coming here and you're saying, look, the capital here is not like this, usage and retention, why would I scale it? It seems obvious, but I'm sure there's another argument on the other side that's what is that argument that tells you, no, you're overthinking it, you just got to scale this thing and things will take care of themselves?
ScottI do think there was a period of time where capital was in such excess and there was just this social thing happening where everyone was just watching each other and doing what everyone else was doing. This social mimicry, which I think is – key to business and investing is not mimicking what everyone else is doing, but it's actually in practice very, very hard. The book Blitzscaling was very popular at the time and everyone's like, well, if you don't scale fast enough, a competitor is going to come in or if you don't start scaling up, you're not going to learn fast enough. If you don't scale up, how is your team going to do anything if you don't hire 10 more people? I think there was largely a lack of belief at how how much a small number of people could do for a little while. I think a small number of people can do a lot, sometimes more than a larger number of people. If you're doing a new experiment every two weeks, very hard to get a 50 person team aligned on that. People get burnt out pretty quick, but small team can do that.
Having Faith in the Product and the Ability to Abandon Ideas
PabloI think that's one of the most mistaken beliefs by the way is that – it's like there's all these things you should do. There's two worlds in my mind. There's pre-product market fit and the post-product market fit. One of the most classic things I see is, for whatever reason, implementing things that make sense post-product market fit but too early. That goes for like, you should hire a VP of this or a VP of that; you need to scale, you need more developers. You actually don't need more production. You don't need more output until you figure it out in your words what the thing is. Staying lean means there's no overhead. It means you shift quickly. This is my experience is this is one of the things that I think is just not – it's hard to appreciate until you've gone through it which is if you've got a lot of people, the amount of time you need to take to communicate every single time that you decide you're abandoning this and going that way is massive. Then all of a sudden you've got people who worked on this thing, they don't want to let that thing go, they don't necessarily agree with the new direction. You're trying to just go here and then here and then here. It's like you got five people. I don't know how many people you were actually, but I assume it was like five, 10 people or something.
ScottFive to eight people, around there. Yeah.
PabloEverybody's aligned, and so that's a huge difference maker.
ScottExactly, and I have to give our engineering team so much credit for that as well and our founding engineer Matt Drover rolling with that. Even with a small team of staff, you also have to hire people who are able and willing to do that. Building things over and over and seeing it not stick, it's like not normal. People in big companies just don't – I don't know. You're sheltered from the harsh reality I think a lot of the time that most things actually just don't work and suck and fail. I think when you're up against raw reality with a small team, you see that and feel that, but I think you can live a whole life as a software engineer and not realize just how many things just completely fail and suck.
PabloDid you think everything was going to maybe fail through that period? Did you and or your team think that at times?
ScottYeah, we did. I don't think that's a good way to communicate it in a large company, but in a small company we're like, we just got to assume everything's going to fail and we have to get to the next experiment. Invalidate this one and move on to the next thing was our approach.
PabloWhat about, I mean, broader? Did you ever think, you know what? Maybe Rally isn't going to work out, we're just not going to find product market fit? Was that something that – or not really?
ScottThat honestly never really crossed my mind because I had so much faith in the problem and just there must be a way to make legal services 10X more efficient or 100X more efficient. I know that everyone hates getting that really expensive legal bill. Everyone knows how universal that experience is. I think our faith in the problem and our mission was big enough to enable us to do that. Where if it wasn't, if we weren't that motivated by the mission or we started to question now does this problem exist at all, we would've abandoned it for sure. I think that was a really important part, finding a problem that felt – actually had to be a spiky tough problem that hadn't been solved before that we felt motivated that should be solved. It helped our team stick to it and be like, it's almost good that this is a hard problem because otherwise it would've been solved already, so we're just going to keep at it.
PabloWhen does this – ChatGPT comes out, walk me through how AI starts to play a role and then the transformation into Spellbook.
ScottWe
The Introduction of AI into Spellbook
Scottstarted experimenting with GBT-2 very early just totally on the side. It's not an OKR. It's on a project. It's on a strategy. It's just playing around and hacking. That's one pattern I had noticed in our successes is that they almost all arose from this evening and weekend hacking. They never arise from the board meeting plan or the quarterly plan. They always arise from like, oh, I'm going to try this thing after hours and see what happens. We started playing with GBT-2 early on. It wasn't really good enough to do anything with. It sucked. Then when GBT-3 and GitHub Copilot came out, that's when we really saw promise. This is when the aha moment really happened and we knew immediately. As soon as I tried GitHub Copilot as a programmer, what I saw is that I could have bespoke code on my screen and the AI copilot would riff off my bespoke code to tell me what I should write next. That was an amazing moment because lawyers had the same problem with our product where they want something that's going to help automate their workflow but it was never bespoke enough. Templates were too rigid. Templating software for lawyers has been around since the 70s. It's called document automation. It was a big thing in the 70s, but it never just caught on. We were like the hundredth company to try this and everyone was like, oh lawyers, they're so bad at adopting technology. I was pissed off at our customers; not really but I was off. I was like, oh, why don't –
PabloYeah, frustrated, yes.
ScottYeah, frustrated like, well, why don't these lawyers use this product. They're foolish. It's actually, no, the lawyers were right. The rigid templating approach to automating contracts just sucked. It took me too long to realize this from our customer feedback, but every deal is unique. Manually massaging text and getting it right for your client really matters. Your client has its own unique history as a corporate entity and its own unique things it's trying to do. Everything actually is really bespoke in law. The amazing aha moment for us was like with GitHub Copilot in software engineering was like seeing that this LLM could riff off of bespoke code and generate new code. We said, oh, this is what our customers want. It wasn't like, oh, we're going to apply AI towards law. It was like we finally have the tool that solves our customer's problem and all the things that they keep saying to us, we finally have the answer to it. Almost immediately we made a V1 of that in about two weeks, actually in evenings and weekends. It wasn't even part of our main plan. We were like, yeah, we're going to hack this together. Really crappy early version, but as soon as we showed it to – this was before ChatGPT actually.
PabloThis was what? What was V1?
ScottV1 was – I don't know if you've seen GitHub Copilot but it's basically auto complete for your code. Ours was auto complete for lawyers basically. We built a word admin with a button that says draft or whatever and you could be writing a contract and you could just hit auto complete and it would complete what you were typing and that was it. That was really the first feature. We also had a feature called explain to a five-year-old where you could highlight some text and it would explain that in client friendly language. It was really the auto complete feature that took off. We demoed that to our first lawyers and it was an immediate difference from anything we had ever seen before. Their pupils dilated. NFX has this great article. I don't if you've read it, Find The Fast Moving Water. Have you seen this one before?
PabloNo, I haven't. I'm going to check that out.
ScottIt's an awesome blog post that came out around the same time. It talks about during these inflection points and points where really groundbreaking products come out, you actually see customers have a neurochemical reaction to seeing the product for the first time where their brain is reconfiguring to be like, holy cow, my whole life is about to change. We saw that. Just this one little feature showing it to lawyers, you could just see the rest of their career flash before their eyes and they're like, holy cow, something is actually here that's going to change how I practice. Sometimes you see a little bit of fear, sometimes you see excitement, but it was this wild moment. This was before ChatGPT. It was summer of 2022. We rolled that out very quickly to some beta users. They loved it.
PabloAll your KPIs, the ones you were tracking, this time were in the right place, retention, usage, et cetera?
ScottAll the KPIs were off the charts. CAC payback was below one month. There was immediate CAC payback and usage was like we had never ever seen before.
PabloWhat were some of the wording you used in those ads? How did you describe this?
ScottDo I remember? That's a great question. I don't even know. Actually I can't remember. We had an ad that was riffing off of GPT for lawyers thing but this was before anyone knew what ChatGPT was. I think it was draft and review contracts 10 times faster, writing Microsoft Word with an image of the product working. It was the image or the video of the product working that I think was a big part of it. Sometimes that image or that GIF speaks louder than words and that's really important. One thing I would say is we figured out a lot of patterns along the way that went into this. It wasn't like we just struck it lucky. There was a whole bunch of learnings that we made along the way that made us build this one. We realized getting lawyers to go to web app is almost impossible. That alone is too much friction for a product for lawyers in 2024. We built the Word at it and that was a crucial decision for us. No one else did that.
PabloI didn't even know you could build something for Word to be honest. I don't have any plug of Word?
ScottYeah, it wasn't very popular at the time. Our developers cringed. They're like really we're going to deal with these messy Microsoft APIs. This doesn't sound like fun. That spikiness and that contrarian is part of what helped us have this big lead on the market. We knew how hard it was to even get them to go to a web app. That was crucial. The other thing we learned is you have to be able to explain what your product is in a five second GIF. If you can't, it's like no one has the attention span for it anymore. We knew it had to have that element where you have a five second video and someone's like, holy cow, what is this? We knew it had to work with bespoke documents and we knew that the time to value had to be zero. That's a huge pattern that we still use today when we launch new things is no one has time for configuration controls, setting things up anymore. It has to be, you turn the thing on and it immediately works. These things became very important to now even as we launch new features, these attributes become really important aspects for how we evaluate new features and their potential fit. If you get those right, you see actually the usage charts, which most of the time you don't.
PabloI mean, this is the interesting thing and there's this broad concept like you make your own luck or however you want to talk about that. You had actually two things. One is spend a lot of time with your customers who really understood them and how they worked, but also effectively build a machine that was really good at acting on a new idea and trying to figure out delivering something really quickly and trying to figure out if there's true pull. All those things had been done for years when this moment happened and so it makes sense that it worked out in the way it did.
Basing Next Feature On Pattern Matching
ScottI think there's one way I would compress all our insights into a framework that I would say has enabled us to create these PMF moments way faster now than we could back then. The big thing, and this has been this amazing breakthrough on our team in terms of how we decide what to build next and how to replicate these moments; not every four years but every three or four months. There this framework we call basing the next features you build based on pattern matching, not based on verbal reasoning. I think this is how we screwed up so much and how many companies screw up and how I think you can make really good bets. What I mean by this is, when we first started we used verbal reasoning to explain what we're going to buy. Lawyers want to make more money, therefore, we're going to give them a store. These sorts of little simple narratives are so contagious at the board level, at the founder level, at the company level. That little phrase I just said, lawyers don't care about efficiency. They want more clients, therefore, we're going to build them a store. It's so easy to build these little narratives that are super powerful and super contagious and to go all in on them because everyone thinks they sound reasonable. We learn over and over again that actually that thing is actually really bad because the world is way more hyper-dimensional than that. There are way more characteristics about a feature that will determine whether it's successful or not. We learned that the form of the feature, whether it's in Word or not, actually matters more than almost the product itself, because If you're building a web app for lawyers, sorry, you already lost the game. Whether that story makes sense, doesn't matter. There's so many other characteristics. The characteristics we look at are like, is it zero time to value? Can it be explained in a five second GIF? Does it require zero habit change? Is it already in the workflow? The other characteristic we have is did someone on our team actually build this based on their own inspiration in their own time? That's a huge thing that we've noticed is almost everything that we've built that was really successful, someone was actually just inspired to go and hack on, on their own outside of our normal development cycles. We have 16 of these characteristics and we've learned that by instead doing pattern matching against a whole bunch of attributes, we can be way more successful. I think this applies in investing too. I think you're a pattern matcher and investors get that like, oh, I'm going to invest in this cobalt mining company because they have a contract with Tesla. If you invest in a story like that, you're just going to end up piling on with a bunch of retail investors who have this very contagious story that's really compelling and then there's this big spike and bubble and bust thing on that stock.
PabloI think that's huge because the problem that you described is exactly how it happens. This is just how almost every product idea, feature idea comes alive. You have a story and by the way, most founders are actually quite good at telling stories. The hierarchy you are, the better you are at telling stories and the better you're at convincing yourself and others, and so the stories make a lot of logical sense. They're overly simplified because they have to be and they miss a lot of the nuance. This new approach of pattern matching across characteristics that you have worked – in your case, obviously you had done enough features to have a sample size to say these are the ones that work more and these are the characteristics they had. You probably could start with some first principles around things like zero time to value, easy to communicate, probably apply broadly. If you think about your customer set, you probably can come up with some other ones. That's really helpful even at the outset. Let me ask this, how do you come up – just to make sure that I understand it, do you still – because humans think more in verbal logical reasoning than they do pattern matching, is that still what you use to come up with an idea but then you use pattern matching to test the idea or are you somehow able to use just a characteristic framework to come up with ideas in the first words?
ScottYeah, that's a good question. I think ideas can come from anywhere. They can totally come from the verbal reasoning. The characteristic framework is the gut check on validating could this actually work? Ideas can come from stories totally. They can come from all sorts of weird places. You also need stories to motivate a team. A team is not motivated by staring at a spreadsheet of numbers.
PabloTo be clear, it's not like you have a framework where you have these characteristics and you're somehow able to go from that to, oh, maybe this feature will work or do you, because that's hard and unintuitive?
ScottYeah, no, we don't use it for idea generation. It's more so we have a list of 50 things we could do, how do we rank which one to actually do? Yeah, but it's tough. We have to be good storytellers. I always call the storytelling power as this dark magic and you have to be very careful because it can screw up your own radar if you use it in the wrong way.
PabloThe worst part of being a good storyteller is you believe yourself.
ScottYeah, exactly. I have become more skeptical of simple stories though generally as like, because they're so contagious and because people rally around them so quickly those are the ones where I apply the framework more intensely because I'm like, okay, hang on, this is it.
PabloI love it because the way I see it happen from my vantage point is this is more – I mean it's important across the board but it's so common in pre-product market fit because we're trying to find something that could work. You'll have the founder tell you, so actually what we're going to try now is this thing, here's how we came up with this and here's why it makes sense and you think – what I end up thinking to myself is because I've seen it so many times, I'm like, yeah, it makes a lot of sense, I guess we'll see what happens because I don't have – but your thing is like, that makes a lot of sense. You put it against this thing and it's like actually it is likely to succeed or no, actually it's not likely to succeed. It gives you that extra data point because that's what it's all about is you have a lot of ideas, but you only have limited time. Even if you're the fastest team possible, you still are not going to get through all them and you certainly don't want to get through all of them if you don't have to. Finding a way to reorganize these not on who has the loudest voice in the room, which one is the best story, but something a little bit more that's quantitative hard data. That's great. Honestly, I hadn't heard of that at all before, so I'm really glad you shared it.
ScottHumans need stories to communicate and operate and be motivated. You need it to some level. You have to have an inner circle and your board meeting has to be this place where you actually are really careful about this drug. Actually it's a drug. It screws up your perception of reality. I know there are lots of board meetings that are driven by these stories and people nodding their head to me like, yeah, that makes sense. I think you have to create this somewhat story free zone or this safety zone where we think about things a little differently. Otherwise, it's too easy that for the boardroom to become this storytelling place where we all convince ourselves of things that are really a little bit too simplified and not really digging into the data and stuff.
PabloMy next question is, when did you – there's a difference between a new feature and changing the company's name and going all in on this. How did that transition happen?
The Growing Feature List Weighs You Down
ScottYeah, so we actually did it out of the gate. One thing, I've become very cognizant of how simple is it for the user to grasp this? I thought that by splitting this out as its own website and brand, it would just be simpler to communicate what it is. Whereas if it's like, oh, Rally launch is an AI feature, it's immediately just info overload like what's Rally? What's all this other stuff? How does this fit in? It's so confusing. If I had my time back, those a hundred landing pages, I wouldn't have put under Rally. I would've been like, spin up a new domain names. You get this feature craft and your feature list growing is actually not a good thing. It actually overwhelms customers. It starts to feel like it's overlapping with other products they're using. This is actually the first time we're like, you know what? We're just going to split it in a totally different brand and test it that way.
PabloThat's so smart. Actually I haven't heard of that. I mean, every now and then somebody spins up a new product, but just the idea of almost every new meaningful feature in this pre-product market fit phase, obviously it might as well just be a new brand. You might as well just start from zero, take all the sunk cost out and be as clear as possible. You give it the best chance of success. That resonates. That makes a lot of sense.
ScottExactly, because the growing feature list weighs down your new thing so much. Also figuring out how you're going to integrate this new thing and everything you already do and make it all make sense, it's like it's not supposed to make sense. You're trying experiments. You don't have to combine all the work you did together. Get rid of the sunk cost. All you're looking for is lighting a bottle. All the failed experiments, you don't need to keep them. Yeah, we did that. Then we also eventually changed the company name. That took a little longer to fully go all in. I remember the first time we brought these results to our board because we had not talked about this after this point. We had started experimenting this. It was just an evening, a weekend experiment. We had other stuff going on. We came back to the board. We're like, hey, we launched this little thing and here's the growth chart. It's like this is going to eclipse all the work we did in the past three years within three months. Everyone was just like, wait, what? What's going on? I think you had some element of confusion and worry like, does this company know what it's doing? You never told us about this. Where did this come from? I was like, I think we got to go all in on this. It's just like, oh my God! You built up all this context over years of what the product is and then all of a sudden you're saying leave all that behind. I had been communicating with our investors in the background too, so it wasn't complete surprise but it was a weird moment of like, hang on a minute, this is strange. Then it didn't take very long for our team and board to come to a consensus like, yeah, you know what? These are absolutely stellar growth numbers and this is going to eclipse everything we've done over the last three years.
PabloWhat did you do with your customers on Rally? Did you just abandon Rally or did you still have it and it's just in the background?
ScottNo, we still have it. We still sell it and we're still – I would say the value prop of templates still exists within our ecosystem of Spellbook. Spellbook we want it to be the ultimate AI assistant for corporate and commercial lawyers. Templating is still an aspect of what our customers want to do for sure. That templating engine will roll in slowly into Spellbook as a feature because it is actually useful for certain things that lawyers want to do.
PabloI know obviously mid-2023, you raised about 10 million and then six months after that, so earlier this year you raised another 20 or so, so clearly there's something going on. Can you give a sense of scale, like number of customers? Maybe just curious what Rally was like at that time and then how fast and how big Spellbook has gone?
ScottYeah, so Rally we had around 100 law firms. Now we're close to 2000 law firms on board around the world.
PabloWow! That's insane.
ScottYeah. A little over a year to year and a half, we went to 2000 law firms.
PabloThat's product market fit.
ScottYeah, exactly. We're in a lot of different countries. We see one and two million contracts a year right now go through our system.
PabloPerfect. Look, we'll stop it there. I usually end with two questions, but one of them I think you already answered, which is when did you first feel like you had true product market fit? You went to a lot of detail on the emotional response, so I'll skip that one. I'll ask you the other one, which is, if you could go back to about 2018 when you were just starting Rally with one piece of advice for yourself as a founder, what might that be?
ScottOne
One Piece of Advice
Scottof the intuitions to stay lean until you find PMF was right because we did question that a lot. It was very hard to maintain that conviction with some really difficult conversations with stakeholders on why we weren't scaling out or growing faster. I would tell myself, no, that was right. I probably could have saved myself some sleep if more people had told me that, that was right. The other thing is what I mentioned also. Everything doesn't have to bolt on the same product. When you're in the pre PMF stage, it's a totally different game. It has nothing to do with the post PMF stage almost. You're trying a bunch of stuff. Launch different apps. Launch different brands. Don't get stuck in that sunk cost. I think we made a big mistake in trying to bolt everything onto one foundation to make it feel like all our work was accumulating. That probably slowed us down a lot. Keep launching stuff. Don't worry about the sunk cost. It only takes one landing page and one feature to have that breakthrough moment. You don't need 10 or 20 features all bolted together.
PabloPerfect. Thanks a lot, Scott. I really appreciate it. Some great insights here and I think a lot of lessons actually for founders to take away.
ScottThanks, Pablo, for having me. I really appreciate it.
PabloI just gave you content that you like so much. You actually listened to the end. Guess what? You didn't pay a single dollar. Not only that, I didn't even put any ads in your face. You just got a bunch of content for free. Now that I've delivered that value, I'm asking for something in return. Open your app, open Apple Podcasts, open Spotify, open whatever app you use to listen to this and hit that follow button. It's actually going to help you because it's going to help you make sure you don't miss out on the next episode which you like so much that you listen to the whole thing.
