Product management is a relatively new job. The rise of software & e-commerce business, increased the need for Product managers. Many people ask me what is the most efficient way for someone to enter into Product Marketing. One of the most common problems in product management is that the scope of a B2B Product manager varies greatly from a Product manager in an e-commerce or a Product manager in an online marketplace.
They have different product sense, different KPIs and different challenges to cope. Even within a marketplace business (like efood), a Product manager that is responsible for mobile apps has different skillset from a PM that is responsible for our sales tool or the PM of our data pipeline squad. There are a lot of compehensive articles and essays about Product management tribes, I am preparing a big one as well so stay tuned.
In this post I will share an assessment we used to assign when we were searching for a Product manager for our Consumer Growth initiatives. Consumer growth in efood means that the PM will be responsible for product initiatives projects that focus on user acquisition and retention. These projects most of the times are correlated with the user experience in our website and our apps.
Although this assessment is old, it is still relevant.
There are many online resources about everything and anything regarding Product management, I couldn’t find many good public product assessments. I believe that sharing it, I will help professionals to understand what do we expect from a Product manager that is responsible of our growth projects and implementation roadmap for our mobile applications of Android & iOS.
Product Marketing Manager Assessment Slides
Assessment Breakdown
This assessment has 4 main pillars. We evaluate those pillars separately in order to get the total score. One of the main challenges here, is that although we don’t want to force the candidate spend many hours of his free time, we don’t want to exclude any of those pillars from the evaluation as well. The purpose here is to assess candidate’s thinking, not the length of the deliverable (we even mention it in the small letters in the deliverables section).
The case is about mobile web. Mobile website of every business is a very tricky area. There are many examples of bad and best practices but in general a website that incorporates the advantages [Google’s discoverability] and disadvantages [limited capailities compared to apps] of web browsers within the small real estate of a smartphone. Mobile web is not dead and the e-commerce business that invest in mweb gain significant value from those that disregard it.
Subscribe to my newsletter and get updates straight into your inbox.
Data understanding
This report does not allow candidates to dig deeper. It only provides overview data. The reason is that we want to understand if the candidate is able to understand how Google Analytics work and identify potential challenges & opportunities from this high level overview.
Content group has a specific definition in Google Analytics. Through this report you are able to identify which landing page group (that means which product flow) converts better, which group has significant traction and which group has the biggest potential. This report will help the candidate to prioritize her A/B tests efficiently. There are candidates that ask which specific restaurant’s page should they check. This is a clear signal that they don’t know how content group works in GA.
There is no right or wrong approach, as long as you have the right arguments to support it. For example, there are candidates that focus on the homepage. This is reasonable because homepage generates the biggest traction in terms of sessions so even the smallest CVR increase will lead to more transactions. There are others that focus on menu page landing page group because the combination of low bounce rate along with low conversion rate may probably lead to “easy wins” A/B tests. Small bounce rate means that users are trying to convert, so there must be painpoints on the flow that don’t allow users to convert.
Product Flows
Product flow optimization has variations across industries. There are services that do not require account creation of r atransaction while others do. In our case, if you want to make an order online, we need you to create an account and provide us your delivery address as well. As you can see in the report, we are talking about landing pages. Landing page is the page where user “lands”. This means that user has not seen the flow steps that occur before this landing page. So, you should be able to create an account and/or provide your address in every landing page that users “land” the most.
Best suggestions focus on how to encourage users to add everything that is necessary in a smooth and efficient way. Other good suggestions indentify and close the gaps between restaurant list and resaurant page (ex. if restaurant provides loyalty points, it should be highlighted both in restaurant list and restaurant page).
An experienced product manager will check not only the main path or the “happy path” as we define it, but some other not so happy paths as well. For example, there are suggestions to amplify restaurant reviews section is a restaurant has great reviews. Reviews check is not mandatory in order to make an order online, but it can dramatically influence users behavior.
A/B testing
When it comes to A/B testing, we mostly consider the ideation process plus the mechanics of the test. Ideation can be considered as an easy task. Ideas are everywhere, they are cheap and there are literally tons of good, bad, unconventional and innovative ideas available for free. What differentiates a good candidate is how she selects the ideas that she believe are worthwhile.
We saw a lot of A/B test suggestions that suggest bigger copy (order now -> order NOW or O r d e r N o w) or changing the color of CTA button. We don’t like that kind of suggestions. That doesn’t mean that we do not try that kind of A/B tests, but we would like to see something more relevant than an A/B test suggestion that considered as a best practice (some could call it basic).
The simplest way to provide good ideas is to do a small analysis of the biggest online delivery apps out there. There are great services across the globe that allows you to order food online, such as PedidosYa, Talabat and Foodpanda Doordash, Zomato, Swiggy etc. There is also the cross-industry inspiration, where you can generate ideas based on services of other industries such as Fintech, Social Media etc.
Prioritization
Prioritization is a hot topic. It is one of the most important skills of every Product Manager. There are many prioritization methods available, some of them are popular, some are more ROI-oriented and some of them are anti-patterns. Due to every method has its pros and cons, it help us get better context regarding Product Manager’s prioritization style. Some PMs are iteration-oriented (small and fast changes), some of them are ROI-oriented (bigger but most impactful changes). Every prioritization method is welcome because from time to time, we adjust our prioritization schema as well to meet specific business needs.
An argument about prioritization is that because PMs are not aware of our Tech stack, they will mostly fail to predict correctly the effort for the implementation of the task. Firstly, this applies to all candidates so its fair and secondly, an experienced PM can understand the difference between the effort of changing the placement of a button and the effort of changing the whole product flow. For the record, there was not one great assessment that lost points due to poor effort estimation.
Let me know if you like this assessment and feel free to try it and share your effort with me (mail me at giamalisk@gmail). Good luck!
