Data Science Playbook Pt 2 - Learnings at TikTok
Recaps learnings at TikTok Shop and going from 0 to $XX GMV
China has an involution culture shaped by 9-9-6, which is that you eat, breathe and sleep as breaks - while you work 9 am - 9 pm, 6 days a week. Bytedance, TikTok’s parent company entered the US market with the acquisition of a startup called musical.ly, and then grew to replicate Douyin (the name of the Chinese TikTok app) to scale and find mass adoption in the US market. While the US market is makes roughly 50-80% of the revenue of any international consumer app that is ad-driven, China makes roughly 80% of overall TikTok revenue, in broadstrokes.
Background
After my stint on launching Notes at Instagram, I moved over to TikTok, 3 months prior to the E-commerce launch. While most tech companies in the US operate in a pod structure with design, data, product and engineering, the set up for TikTok Shop was different. It was an operations-driven company, looking to blueprint its’ success framework in Asia and bring it to the US market with minimal changes. The Operation leader (let’s called them X), was insistent on using the same rubric to find creators who would sell products with the same enthusiasm as live creators in China, sold products on Douyin. At Meta, bug identification to solution took around 1-2 days, when launching a feature, at TikTok, bugs were identified and neutralized within hours. There was a ton of velocity and eagerness on launching commerce and while TikTok’s revenue so far was ad-monetized revenue, there was increasing appetite for using a closed loop model (User sees a product → User buys a product fulfilled by TikTok Shop) vs open-loop (User sees a product → User clicks ad) and semi closed loop (User sees a product → User buys a product fulfilled by third party). Owning the entire experience meant that we had to build trust early as users tend to buy low-priced items (commodity items) first before trusting a platform to buy a TV, Microwave. The stage was set.
My role
I spent my initial few months on the Recommendation team where we assessed the number of commerce videos we could show on Short form video controlling for churn. Next, as someone who could access US user data at a granular level (which my Chinese colleagues couldn’t), we had to figure out the following
Where do we place the Shop tab with minimal disruption to organic traffic?
What products do feature first to our users?
Do we feature products with higher quality / high cost vs lower quality / low cost?
How do we acquire the right products, incubate them and grow them to be “hero products” such that there is reliability of revenue?
In the first few months, we hustled our way to the first question - understanding that we were displacing XX% of traffic that was flowing in from the iconic For You Page (FYP) to the Friends tab, while Friending was Meta’s core competency (Entertainment being TikTok’s) , it was something that TikTok was playing catchup to (as Meta did for Short Form video). Once we figured that out, there were 3 areas where commerce was served up - LIVE, Short Form Video and Shop Tab (the tab we replaced the Friends tab with, that looked exactly like Amazon’s home page).
For the next few months, I spent an inordinate amount of time working with a Senior Director (who was locally based, and spoke english → both being rarities in TikTok at the time - where I found myself lucky), to work on a strategy for the following.
How do we acquire the right products across industry verticals like Fashion, Beauty, Healthcare, Furniture, etc?
How do we in Minimum Viable way, figure out which products drove the highest traction to “incubate” them?
How do we ensure a Pareto with 80% of our revenue insured by stable “hero” products that customers kept buying over time?
Trading off between $ and Order
For this we had two metrics - GPM and OPM - GPM referred to Gross Merchandise Volume that we generated for 1000 impressions of a product and OPM referred to as Orders we generated for 1000 impressions of a product. Initially, to drive transaction frequency (or, in other words to get the user to buy more products → we focused on driving purchases of low priced products that would drive more mindshare while the long tail of users would finally buy furniture → a model that TEMU and other marketplaces also do).
Product exposure in exchange for price-match
Then, we boosted products for sellers who would help reduce the prices for us to be competitive with Amazon, TEMU in exchange for traffic. We were in the business of trading off eyeballs in exchange for ad revenue, GMV sales through product, and in this case sellers helped match the lowest price on the market for more and more exposure. By now, we had scaled GMV ground up from $0 to $XXM, in less than 4 months. I continued to advise the various business teams on (1) Price ($0-$10 vs $20-$30) , (2) Category (e.g. Furniture vs Sport) and (3) Industry (e.g. Home vs Fashion). Although Collectibles (e.g. trading cards) generated high user repeat purchases, high average order value, we deprioritized it since we had to go for on-brand categories like Fashion. There were competitive landscape considerations, given startups like Whatnot primarily drove the Collectibles (trading cards, pokémon cards) marketshare through live (a perishable card reveal use case that generates engagement through principle of scarcity - a “shiny” card that drives intrigue and therefore, sales. With TikTok Shop, the idea was to be a trendy marketplace that would capitalize on the trends that emerged from organic engagement - Fashion was on point.
Causal link between reviews driving sales
Lastly, I moved to the User Product Reviews team where the objective was to (1) Grow user reviews (2) understand the causal link between user reviews → sales. My correlative studies so far showed that it was difficult to link sales to reviews on the app - products with reviews had to have sales and therefore the two were related but I quickly wanted to establish a causal link by running a test where hid the star rating altogether to see the impact on conversion. We had seen earlier that given the price was low, conversion rate would inflect even at moderate/ low avg rating - meaning that it was entirely possible for a product with 3.5 rating to have high sales, given the price was <$10.
In addition to this, the advent of AI took over, and we had to fast follow with AI Insights (e.g. tags that summarized wisdom of crowds from reviews such as “T-shirt fits well”) and AI Summaries (long prose version of Insights), that were already mature in places like Amazon and were building perception for what a user might expect off reviews. However, the number of users scrolling and seeing detailed reviews were very low and we had to incentivize - (a) users adding more reviews after purchases (b) users seeing detailed reviews to create an appetite for these AI products. All this while, we hadn’t talked about whether the sentiment of a review would actually lead to a proclivity of user to purchase a product, on a marketplace highly weighted by price, promotion and discounting.
While this is a brief overview, watch out for Part 3 where I will discuss more on product analytics / data science journeys at consumer tech.
Please read Part 1: Instagram and Zynga for my learnings there