Introducing Fast Rhymes Ghostwriter

Fast Rhymes is a songwriting app for iOS & Android. When I show people Fast Rhymes for the first time, I often get asked if I'm leveraging generative AI. Especially lately, with the growing hype around this tech. However, it's not that simple.

LLMs can be very expensive, so it's important to do a thorough analysis of the cost before shipping something to your users. Building a financial scalable feature is important to both your business and your users. Your users don't want a feature they've come to love, to suddenly disappear, because you are running your company into the ground by burning cash on an expensive, state of the art, LLM. A thorough analysis needs to be done both up front and during the launch.

Do you need the latest LLM?

So, do you really need a state of the art LLM? In most cases you don't. Buying a Ferrari and driving it like a Golf makes no sense, your users don't care about the tech “wowfactor”. You're not getting the extra value you are paying for. That's why experimenting with other cheaper LLMs can give you the same or even better results for a fraction of the price, because it's all about your use case. Providing the same value to your users in a sustainable way. With this strategy, the LLM chosen for the Fast Rhymes Ghostwriter is 160 times cheaper than OpenAI's GPT-4.

Launch strategy

As mentioned above it's important to do thorough calculations on cost before shipping anything to your users. However, not everything can be calculated up front, like real world production usage. There are several launch strategies that can be used to ensure a smooth launch and mitigate unforeseen circumstances. Especially in a mobile app, where the app needs to go through app review, a release strategy is important.

For this feature, feature toggling with a soft launch was utilized. Feature toggling is a great strategy to be able to soft launch a feature to a smaller audience. This mitigates a lot of risk, you get real world usage statistics, which can help you iterate on the feature before rolling it out to the entire userbase. Feature toggles can also be helpful by allowing you to completely disable a feature in case of concerns related to cost, safety, or other issues.

Utilizing a feature toggle also gives us the opportunity to run an A/B test. This can be really powerful, gathering data on how the feature performs. An example is testing the conversion rate difference of the feature. This way the ROI of the feature can be calculated. Read more about previous experiences with A/B testing in Fast Rhymes here.

Demo

Here is a quick demo of Ghostwriter. In this first launch, the focus is on simple features that can boost productivity. For now Ghostwriter can generate a title, the next verse, the next chorus/hook and finish a lyric.

The launch

Ghostwriter was launched in the end of march as an A/B test, exposing it to 50% of the userbase. The feature has been received well, as the A/B tests are already statistically significant, and conversion increased by 150% on iOS, and 60% on Android. This is still an ongoing A/B test which will run for a couple more weeks, but the results after a bit more than a week speak for themselves. The cost of the LLM integration has been surprisingly low. Rate-limiting LLM requests was implemented by caching LLM responses for a brief amount of time, which seems to have been quite effective to avoid multiple identical requests to the LLM, which drastically keeps costs down.

Conclusion

In conclusion, implementing GenAI features requires a thorough cost and tech analysis to figure out how to scale well for your specific need. It's also important to have a good launch strategy to prevent user frustration, for this feature toggling with a soft launch worked great. Gaining insight into how the launched feature affects your business is also crucial, combing a soft launch with an A/B test was effective to see how conversion rates were affected.

To learn more about Fast Rhymes, visit: fastrhymes.com