I Interviewed ChatGPT for a PM role, and it almost got the job

How to detect the AI’s confident but shallow answers

The unedited interview

The unedited job interview with ChatGPT is below. All of the answers were ChatGPT’s first response to the question, with the exception of the first question, where the initial response was along the lines of “I am a large language model, I don’t have a favorite app.” The Trello response was what I got on the second try.

  1. User engagement: This metric would measure how often users are using the app, including the number of tasks created, completed, and assigned to team members.
  2. Customer satisfaction: This metric would track customer feedback through surveys or ratings to determine how satisfied users are with the app.
  3. Retention rate: This metric would measure the percentage of users who continue to use the app over time.
  4. Revenue: This metric would track the amount of revenue generated from paid subscriptions or in-app purchases.
  5. User acquisition: This metric would track the number of new users who download and start using the app.
  1. Improving user engagement: This could involve adding new features or improving existing ones to make the app more useful and enjoyable to use.
  2. Increasing customer satisfaction: The product team may aim to gather and act on customer feedback to improve the user experience and make the app more valuable to users.
  3. Growing the user base: The product team may focus on acquiring new users through marketing and partnerships, as well as retaining existing users through regular updates and support.
  4. Increasing revenue: The product team may aim to generate more revenue from paid subscriptions or in-app purchases, as well as exploring new monetization opportunities.
  5. Maintaining a competitive edge: The product team may strive to stay ahead of the competition by constantly innovating and offering new and improved features for users.
  1. Talking to users: I would reach out to users to gather feedback on their experience with the app, including what they like and dislike about it. This would help me to identify areas for improvement and potential new features to add.
  2. Monitoring user behavior: I would track user behavior to see how users are using the app and where they may be encountering challenges or obstacles. This would provide insight into potential improvements that could make the app more user-friendly and effective.
  3. Keeping an eye on the competition: I would regularly monitor the competition to see what features and innovations they are offering, and how Trello compares. This would help me to identify areas where Trello could improve or differentiate itself from the competition.
  4. Collaborating with the team: I would work closely with the product team and other stakeholders to brainstorm ideas and prioritize potential improvements based on their potential impact and feasibility.
  5. Testing and iterating: I would implement a process for testing and iterating on new ideas and features, including gathering feedback from users and making adjustments based on their feedback. This would help to ensure that the improvements we make are effective and align with user needs.
  1. Adding new features: The product team could add new features to the app that make it more useful and enjoyable for users, such as the ability to customize their dashboard or create templates for frequently used tasks.
  2. Improving existing features: The product team could improve existing features to make them more user-friendly and effective, such as making it easier to assign tasks to team members or attach files.
  3. Offering integrations with other apps: The product team could explore partnerships and integrations with other apps that users may be using, such as Slack or Google Calendar, to make Trello more convenient and valuable for users.
  4. Hosting webinars or training sessions: The product team could offer webinars or training sessions to help users get the most out of Trello, including tips and best practices for using the app.
  5. Regularly updating and improving the app: The product team could commit to regularly updating and improving the app to keep it fresh and relevant for users, which would show their commitment to providing a high-quality user experience.
  1. Adding the ability to create and customize templates for tasks and projects: This would allow users to quickly create tasks and projects that follow a specific format or structure, making it easier to organize their work and stay on track.
  2. Adding a feature for tracking project budgets and expenses: This would allow users to input and track the costs associated with their projects, including expenses and resources, making it easier to manage their budgets and stay on track.
  3. Adding the ability to create and share reports: This would allow users to generate reports on their tasks and projects, including progress and completion, and share them with team members or stakeholders.
  4. Adding a feature for managing recurring tasks: This would allow users to set up tasks that repeat on a regular basis, such as weekly or monthly, making it easier to stay on top of recurring tasks and avoid missing deadlines.
  5. Adding a feature for scheduling and managing meetings: This would allow users to schedule meetings and track the status of meeting tasks, such as preparing agendas or sending reminders, making it easier to stay organized and prepared for meetings.
  1. User engagement: I would track how often users are creating and using templates, including the number of templates created and the number of tasks assigned to templates. This would help me to determine whether the feature is being used and whether it is making a positive impact on user engagement.
  2. Customer satisfaction: I would track customer feedback through surveys or ratings to determine whether users are satisfied with the new feature and how it is impacting their experience with Trello.
  3. Retention rate: I would track the retention rate of users who have created templates, to determine whether the feature is helping to retain users over time.
  4. Revenue: I would track the impact of the feature on revenue, including any increase in paid subscriptions or in-app purchases.
  5. User acquisition: I would track the number of new users who download and start using Trello after the feature is launched, to determine whether the feature is helping to acquire new users.


There are obviously many things to be said about these results — among other things, they point out how poor of a predictor interview performance is on the actual ability to do the job. Product management is about so much more than generating reasonable ideas, prioritizing them, and knowing which metrics to track — it’s much more nuanced than that.

Favorite app

This answer is quite good. It highlights Trello’s unique value proposition (“visual and intuitive way”), and is succinct and to the point. The only not so great part about the answer is the reference to “constantly updating and improving their app”, which I wouldn’t really consider something that inherently makes a product great. Moreover, it isn’t really what I would most point out about Trello, a product whose core really hasn’t changed all that much in 10+ years. If recent improvements were what made this product great, then I would expect a bit more detail on how these changes have made the product better.

Customer problem

Again, this is a good and succinct answer to the question. You could expand this by addressing different use cases, personas, or jobs-to-be done (e.g., considering the single player vs. multi player use case), but I don’t think I would fault even a more senior candidate for this response.


This answer is almost perfect. Even the more senior PM candidates that I have interviewed sometimes forget to cover the entire customer lifecycle in their dashboard. There is really only one nitpick that I have: the “engagement” point contains multiple things to measure, there isn’t really one metric covering app opens, tasks created, completed, and assigned to team members.

Product goals

Here, ChatGPT’s responses start getting a bit weaker. There is nothing inherently wrong with this answer, but it is to some extent rehashing the response to the Dashboard question, and mostly generic. You could literally give this answer for any product. Great PM candidates use this opportunity to start prioritizing what they believe to be the most important goals and why, making and calling our reasonable assumptions.


This answer isn’t wrong, but it’s not great either. ChatGPT mentions some obvious competitors, but isn’t very comprehensive. Great product managers don’t just think about alternative products, they also identify substitutes and non-consumption as alternatives. For task and project management in particular, pen and paper and Excel/PowerPoint are going to be the biggest actual competitors.

Thinking about improvement ideas

The answer here is quite boilerplate. Not only could this answer apply to any product, it is also not tied to any of the goals identified previously. Moreover, product strategy or goals as a starting point are completely missing from the response.

Most important objective

This answer might be a bit short, but it’s great. It prioritizes without hesitation and explains why user engagement is a great goal to focus on, since it is also a leading indicator for some of the other product goals.

Ideation, round one

Except for the fifth point (regularly updating and improving the app), this answer is good. The range of ideas is broad, each class of ideas has an example that is relevant to Trello (so it’s not super generic), and the set of ideas doesn’t just include product changes, it also includes “webinars or training sessions” which is more left-field, but a good sign if a product manager candidate doesn’t consider their work in isolation of the other teams.

Ideation, round two

This is a great list. It’s relevant to Trello, there are more incremental ideas (templates) as well as bigger bets (scheduling and managing meetings), along with a succinct explanation of the ideas.

Prioritization approach

This answer is good, but not great. Firstly, it is a bit “everything but the kitchen sink” (“gather as much information as possible from users and other stakeholders”), whereas in reality you often have to make prioritization decisions with imperfect information. Secondly, “goals and strategy” is a bit buried even though it should be the primary driver for prioritization. Lastly, great PM candidates respond to this question not just by describing their hypothetical approach, they would also immediately apply it to their list of ideas.

Idea prioritization

This is a quite good response. Mostly, I would fault the candidate for not calling out assumptions, and not comparing and contrasting this idea with the other ideas.

Idea validation

The only flaw that I see with this response is that it begs follow-up questions like “how exactly would you do that?” Also, there is again the phrase “gather as much information as possible” which suggests a bit of analysis paralysis. However, the overall approach is perfectly reasonable and well summarized. (One tiny nitpick is the mention of focus groups, which I haven’t really seen used much in most tech companies; product discovery research tends to be one-on-one.)

Success measures

The biggest flaw with this answer is that it is too broad. A great PM candidate would have recalled that the goal that we are pursuing is user engagement, and called that out as the number one success measure to track. In particular, revenue and user acquisition seem much farther removed from the actual feature, so they should, if anything, only be mentioned as an afterthought.


As you can see, all of these answers were decent. There were a few with more obvious shortcomings, but again, I have seen candidates with actual product management experience under their belt give far worse answers for each of these questions.



Head of Product at RevenueCat; previously at 8fit, Yammer, BCG.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store