Automating NPS Analysis

I was on the product marketing team, working on competitive intelligence. This was not assigned. I noticed the problem, built the solution on my own time, and took the results to stakeholders once I could show it worked.

The team was doing data analysis in Excel, but the data wasn't quantitative. It was unstructured text. Someone sat and read comments one at a time, tried to hold the patterns in their head, and made grouping decisions under time pressure.

That produces three problems at once. It is slow. It is inconsistent, because the themes depend on who is reading and what they already expect to find. And it is late, because by the time the write-up reaches the product team the quarter it describes is over.

A classification pipeline in Python: pandas for cleaning and preprocessing, the ChatGPT API for thematic coding at scale. Each comment is categorised by topic and sentiment, recurring pain points are identified across the whole dataset rather than a reader's working memory, and the output is a structured report.

The design decision that mattered was the taxonomy. I started with fully user-defined categories and realised that biases the result toward themes the analyst already expects. The unknown themes are the ones that change product direction, and those are exactly the ones a preset taxonomy misses. So the pipeline does both: you can preselect categories and have it sort comments into them, or let it generate categories from the data. The hybrid catches the expected and the unexpected.

I ran it in parallel with the manual process for a full quarter and compared the output before proposing it as a replacement. Trust in a tool like this has to be earned against the thing it replaces, not asserted.

Analysis time dropped from roughly eight hours to about thirty minutes, and the themes stopped depending on who was reading.

The larger shift was operational. A manual quarterly project became a system that reruns. The time the team had been spending to find insights moved to acting on them, which is what shortens the loop between what customers say and what positioning and roadmap do about it.