Questions you must ask tech salespeople selling AI-powered solutions

At the AFC UK Summer Conference 2025, artificial intelligence wasn’t just a buzzword, it was central to the day’s conversations. One of the AI sessions of the day tackled a pressing concern for every business leader today: how do you know whether an AI solution is really worth your time, data, and money?

Richard Huston, managing director and co-founder of VAMOS, and AFC’s AI advisor, led a powerful and practical discussion with Chris Kirby, founder of Tomorrow’s Journey, and John Murray, director, office of the CTO at Alfa. Their mission? To arm the audience with eight essential questions every buyer should ask when approached with an AI-powered solution.

The context: Why AI Is booming

Richard Huston opened the session by grounding the audience in two transformative AI trends:

1. Commoditisation – Tools that once required teams of engineers and years of R&D are now available via APIs from OpenAI, Anthropic, Google and others. Access is cheap, fast, and widespread.
2. Generalisation – Modern AI models are no longer narrowly focused. A single foundation model can now tackle a wide variety of tasks, from summarisation to conversation to prediction.

This combination of power and accessibility has led to what Huston called a “Cambrian explosion” of AI tools. But as he warned: just because it’s possible, doesn’t mean it’s valuable. That’s why asking the right questions matters more than ever.

The 8 questions you must ask when buying an AI-powered solution

1. Why is AI well suited to solving this problem?

John Murray cautioned against “AI for AI’s sake.” His team at Alfa built a chatbot named AskThea to help users navigate complex product documentation. Generative AI’s ability to understand intent and respond in natural language made it uniquely capable in this space, providing better results than traditional keyword search.

Chris Kirby highlighted the same benefit for car-buying journeys. Tomorrow’s Journey’s AI agent helps customers filter through countless options via conversation, drastically reducing decision time from hours to minutes.

Takeaway: AI should unlock new capabilities or significant efficiencies, otherwise it’s just expensive window dressing.

2. What is it allowing us to do that we couldn’t do previously?

AI isn’t just about faster processing – it should change the nature of how a problem is solved. At Alfa, AskThea helps onboard new staff, answer complex product questions instantly, and reduce reliance on internal support.

Kirby pointed out how their AI helps customers ask nuanced questions, like “Which electric SUVs can I lease for under £300/month that have heated seats?” – something search filters alone couldn’t handle effectively.

Takeaway: If the AI just replicates what a drop-down menu or PDF already does, then it’s not worth the investment.

3. What type of model are you using?

You don’t need to build your own model from scratch, and most vendors won’t. Kirby shared how Tomorrow’s Journey experimented with several models, including Claude and GPT-4, ultimately selecting GPT-4 Light for its performance-to-cost ratio.

Takeaway: Ask vendors what model they’re using, whether it’s proprietary or third-party, and why they chose it. Understand how it affects costs and flexibility.

4. Where is your model deployed and run?

Security and scalability are key. John Murray explained Alfa’s decision to run their AI on Amazon Web Services (AWS). Using AWS gave them enterprise-grade infrastructure, managed services, and access to the latest models, without building everything in-house.

Takeaway: Clarify whether the AI runs on a public cloud, private server, or device. Know where your data is going and how it’s protected.

5. Does this system need access to our data?

Most useful AI systems will need some form of access to your data to retrieve relevant info, personalise outputs, or take action. But it’s important to define what’s needed and how tightly that access is scoped.

Kirby explained that their agent queries Tomorrow’s Journey’s actual vehicle pricing and availability data. That’s what allows it to give precise, tailored answers.

Takeaway: Find out what kind of data access is required, and ensure it aligns with your compliance and security standards.

6. How does it use our data?

This is where trust comes in. John Murray highlighted the use of Retrieval Augmented Generation in AskThea, a method that ensures the AI only pulls from vetted internal documentation, reducing “hallucinations” and increasing reliability.

Kirby echoed this: his agent uses different “sub-agents” for different tasks, each accessing tightly scoped datasets. Product definitions in one database, and vehicle comparisons in another.

Takeaway: Be clear on whether your data is being used for training (rare in enterprise) or just reference (more common). Ask where it’s stored, how long it’s retained, and how it’s secured.

7. What kind of human oversight is there?

AI is fallible, especially generative AI. Murray emphasised that humans still need to review responses, measure satisfaction, and test for accuracy. Alfa uses user feedback, real-world observation, and technical QA to ensure performance remains high.

Takeaway: Ask vendors what happens when the AI gets it wrong, and how often they expect that to happen. What guardrails or escalation paths are in place?

8. What happens when the AI makes mistakes?

The stakes are higher when AI has more autonomy. Chris Kirby explained that while the long-term goal is a fully autonomous car-buying experience, today their AI hands over to a human as soon as the customer gets stuck or asks to speak to someone.

Alfa’s Murray likened the current state to a co-pilot not an autopilot. You need confidence, feedback loops, and clear handoff mechanisms.

Takeaway: Ensure there’s always a clear, seamless route for human takeover. Trust is built over time and backed by data.

Final thoughts: buyer beware – and be curious

As Richard Huston closed the session, he reminded the audience that AI is not magic, and no solution is perfect. But with the right mindset – and the right questions – you can separate useful innovation from shiny nonsense.

The 8 questions offered in this session aren’t just due diligence, they’re essential for any organisation serious about implementing AI responsibly, safely, and effectively.

“You should be challenging your providers,” Huston urged. “Not just to sell you a solution, but to show you how it works, where it adds value, and how you can trust it.”