ChatGPT's Decline? Nitin Dhamelia on Data Accuracy and AI Investment Strategies

Did ChatGPT really lose billions of users to Gemini?

A Financial Times article from September 2024, which is now corrected and reads differently, supported by Goldman Sachs data, claimed just that. Though this error was soon rectified, it could have had long-lasting impacts on marketing and AI budgets.

The incident highlighted the need for critical thinking and subject matter expertise in differentiating fact from fiction.

After all, to err is human, to forgive is divine, and to help fix is responsible.

To help educate our collective industry of marketers, technologists, and content creators on how to better discern data, we spoke with Nitin Dhamelia, the man who helped Financial Times and Goldman Sachs identify and fix the problem.

Nitin Dhamelia is a London-based digital marketing leader with over 20 years of expertise in website optimization, SEO, e-commerce, and data analysis. His career across the Americas, EMEA, and APAC includes roles in the food and beverage manufacturing, motor vehicle manufacturing, retail, and media production industries.

Nitin has also consulted multinational enterprises to transform business strategies, fuel growth, and drive MarTech innovation through strategic insights. He is passionate about knowledge sharing to empower marketing teams with data literacy and cross-functional capabilities.

Read on for expert analysis, a step-by-step guide on discerning data authenticity, and future-focused advice.

Kamaljeet Kalsi: How did you come across this article, and what prompted you to investigate the data presented in the Financial Times regarding ChatGPT usage trends? 

Nitin Dhamelia: I've been a heavy user of ChatGPT, Gemini, and you.com since their launch in December 2022. I am also a paid subscriber to publications like Financial Times, which has a strong, influential readership.

I usually use my premium ChatGPT account on my personal phone and a web browser to move fluidly between web and app platforms.

A few weeks before the Financial Times article was published, I noticed a shift from the old chat.openai.com to chatgpt.com for premium users. I identified this subtle change because I frequently switch between devices depending on the screen real estate I need for a respective task.

When I saw that particular article with the bar chart in the Financial Times, I thought the audience might look at the chart and think, "Okay, ChatGPT usage is declining globally; let's find out what's next." 

Many may mistakenly conclude that capital expenditure (CapEx) investment in platforms like ChatGPT will lead to diminished returns due to its perceived declining usage, not realizing that this is a misconception concerning its future ROI potential. 

But the reality is that this isn't entirely correct. You can't go from peak usage — billions of users a month — to an 89% drop, especially in 2024 when companies like Tesla and others are creating their own AIs and robots powered by AI agents.

The picture didn’t seem quite right, and this curiosity led me to check for changes concerning ChatGPT and the FT.com bar chart.

Did ChatGPT actually lose billions of users to Gemini?

No. It was a simple website migration to improve the user experience for consumers using a chatbot like ChatGPT.

It’s not obvious for the average consumer to spot what's happened because it requires technical knowledge of the web and consumer experiences that appear unchanged but are indeed different.

I wrote to Financial Times. They consulted Goldman Sachs and made a correction inspired by a chart I sent.

Source: Nitin Dhamelia

Source: FT.com

Even though the article has been corrected now with a new chart of 'minutes used,' illustrating rising usage trends for ChatGPT, readers may not revisit it, having seen it once and believing there was nothing wrong.

Source: FT.com

“I would advise budget holders to continue investing their CapEx in tech platforms that rely upon ChatGPT or OpenAI APIs themselves because usage is still skyrocketing.”

Nitin Dhamelia
Marketing Insights Consultant

What was your thought process in identifying the discrepancy and conducting your own data analysis?

To better understand the reality, a quick look at FT.com's data from May to September revealed it was just a website migration. They had 10.3 billion user visits, which weren’t considered. This process is complex and technical in SEO yet appears simple on the surface. Understanding such details escapes many in the marketing industry — possibly 90% — indicating a need for education through publications like G2’s.

Old chart source: Nitin Dhamelia

The picture revealed the partial truth. We call it partial proof because their data analysis lacked depth.

It became clear that the alleged user loss wasn't due to newer platforms like Gemini but rather a missing piece of data analysis, which is a common oversight. This ties back to the importance of education and understanding.

I’d like to emphasize that after any research or data analysis, it’s crucial to distill findings into simple terms. The bottom line here is that there's an increasing usage trend. If you're managing a budget, perhaps in IT or marketing, continue investing in your APIs like ChatGPT. Keep those CapEx lines in your budget because user demand is on the rise, expecting access to services through various platforms. 

It’s vital to simplify conclusions: is there growth or decline, and should CapEx investment increase or decrease? Remember, investment applies to business budgets, such as marketing, and typical stock market investors. The takeaway from any article should guide all these stakeholder groups clearly.

“My intention was to facilitate a broader understanding and correct public interpretation of the data in published material, thus affecting readers who follow financial and technology news closely.”

Nitin Dhamelia
Marketing Insights Consultant

Can you detail the process you followed to conduct and communicate the analysis?

It was an incredibly simple method, but I think having deep knowledge of this subject area helped me understand how things operate. 

I decided to run my independent analysis by replicating the entire process, which I imagined to be from Goldman Sachs. I basically used Similarweb, the same tool that the Goldman Sachs team used. 

I used Similarweb’s freemium version, but you can do the same using Semrush or your tool of choice.

Having gathered and analyzed the data, I reflected on the discrepancy between my findings and what I believed was portrayed in public articles. Rather than discussing it on social media platforms like X (formerly known as Twitter), I contacted Financial Times directly. I wrote to them explaining the partial inaccuracy in their article and offered my chart for their consideration, suggesting it might provide a more accurate perspective.

The team at Financial Times responded to my correspondence, expressing gratitude for pointing out the oversight. They mentioned they had taken my chart into account and used it to verify and correct the information in their article. They clarified that while their updated chart came from another data source, it corroborated the same storyline, which they appreciated having been noted.

Finally, contemplating further actions, I reached out to contacts at Goldman Sachs. Although unsure if my communication would reach the appropriate individuals, I began this process to ensure both Financial Times and Goldman Sachs were informed about the necessary corrections. 

More than your average newsletter.

Every Thursday, we spill hot takes, insider knowledge, and news recaps straight to your inbox. Subscribe here

The article was for premium paid customers of Financial Times, meaning that it could have influenced investment decisions of even the smartest minds. What lessons can publishers, financial analysts, journalists, and content creators like myself learn from this incident?

Generative AI technologies like ChatGPT are incredibly powerful, and we’re all on a collective journey of learning and embracing it. 

For content creators, publishers, and analysts, connecting with field experts is crucial. Don't hesitate to reach out for a sense check. Many, like myself, are willing to help without a consulting fee because we value global knowledge growth.

Those in influential stakeholder positions at places like Financial Times can support AI budget increases, benefiting their companies long-term. Those in influential stakeholder positions at places like Financial Times can advocate for AI budget increases, benefiting their companies long-term. Since investors in Financial Times are likely also shareholders in other companies where we, as marketers, work, this support can indirectly benefit a range of industries.

Investors and other key stakeholders play a crucial role in helping marketing leaders secure the necessary budget or CapEx for AI initiatives. This investment is essential as it enhances an organization's capabilities, ultimately benefiting your department and the entire company. It's important to approach this decision from a comprehensive, 360° perspective, ensuring all aspects are taken into account for strategic growth and success.

“If clear strategies aren't established, different components of the market can become isolated and disjointed.”

Nitin Dhamelia
Marketing Insights Consultant

This disjointedness is evident in how platforms expose gaps across marketing disciplines.

For instance, platforms like LinkedIn often highlight inefficiencies within various facets of marketing, including SEO, PR, and content creation, where AI is increasingly influential.

It's unproductive to have separate groups within these areas loudly expressing a lack of understanding surrounding SEO, AI, or any other content-related issues. Instead, it's crucial for professionals across these fields — marketing, digital marketing, or content creation — to share knowledge and elevate each other's expertise collaboratively. 

At the risk of sounding esoteric, I’d like to mention that this collective growth is beneficial not just for individuals but for teams and the organization's budget in the long term. 

The key takeaway is the significance of solidarity and shared learning on this journey. As we advance together, it enriches our collective human experience and enables us to build more sophisticated and accurate tools, including AI, that ultimately serve and empower us all.

How important is it for marketing leaders to stay updated on technical changes, and how can they ensure that their teams are also informed?

For marketing leaders, it's imperative to be self aware of their areas of improvement and understand the immense speed at which marketing is moving. 

Over the past 12 months, the brand equity measurement industry has seen the topic of AI infiltrate its typically slow-moving space. This is because brand equity, while not known for rapid change, requires AI to keep up with its pace from a consumer perspective. The market must grasp the practical applications of interpreting quality data and understanding changes in data across marketing verticals. This includes revisiting fundamentals, such as optimizing search channels, content creation, content generation, and ideation.

Encouraging teams to learn independently is beneficial. For instance, if you're based in London, attending conferences like Brighton SEO or Marketing Week events can inspire and educate. Offering your team these opportunities helps them stay informed and engaged, rather than enforcing a top-down approach that may not suit diverse learning styles, especially in innovative environments. This approach ensures your team grows more effectively, stays inspired, and keeps pace with industry changes.

On the note of marketing leaders staying updated, I recall you did a Mini MBA. How have you continued to pursue your learning journey, and how has it helped?

I completed my Mini MBA in December 2023 and applied the knowledge alongside my day job. I used it to create internal brand and measurement tools. Now, measuring brand impact after a campaign doesn't take three months; it's down to about 72 hours and sometimes even less.

The skills I learned, combined with AI's capabilities, have catalyzed my workflows and efficiency. It sometimes takes just half a day — maybe three or four hours — rather than waiting three months to complete the measurement and double check everything.

Can you share how you manage your day job, learning, and consultant role?

It overlaps with my Netflix time, which I consider beneficial. I'll have my personal phone and Netflix on one side, while I catch up on some family time in the evening. Meanwhile, I'll run something on ChatGPT in the background to get ideas flowing. Later, I'll revisit those ideas and determine if they offer insights for consulting or something I need for a future project or white paper. 

AI allows me to balance things nicely, stay energized, and avoid burnout, which is crucial.

What trends do you foresee in the use of AI for large language models (LLM) and data analysis and marketing?

More departments and business functions, such as HR, sales, and supply chain, will recognize the value of a good LLM. Business leaders will increasingly opt to use external LLMs via APIs rather than developing in-house models to achieve their goals. 

An example of this is creating synthetic data for brand marketing measurement surveys, enabling rapid marketing segmentation. This process no longer requires six months to three years to achieve industry segmentation. By then, generations and demographics may have shifted making the data outdated. Businesses can now rerun the LLM to generate synthetic data to anticipate demographic changes and adjust brand messaging accordingly. 

This is a powerful approach to understanding and utilizing synthetic data for swift segmentation in marketing, ensuring a business remains forward-looking.

Follow Nitin Manhar Dhamelia to keep up with the latest marketing, data, and MarTech insights.

If you enjoyed this insightful conversation, subscribe to the G2 Tea newsletter for the latest tech and marketing thought leadership.