ARTIFICIAL INTELLIGENCE

Testing the Limits of the Latest AI Models

Experimental AI platform challenging leading models to manage their own FPL teams and test just how far AI has really come.

Services

  • UI/UX Design
  • Data Analysis

Results

AI FPL is a live, multi-year experiment providing unique insight into the evolving cognitive capabilities of AI. While still early, the project has already revealed surprising truths about how different models think, reason, and behave when asked to operate independently.

  • First-of-its-kind live experiment tracking the real-world decision-making of multiple AI systems.
  • Demonstrated Scout’s hands-on expertise in prompt engineering: proving how careful language design directly shapes AI behaviour and performance.
  • Featured on LinkedIn and across Scout’s channels as a key piece of AI thought leadership.
  • Continually updated platform with the latest AI models – including ChatGPT’s Agent Mode, now managing its own team autonomously in-browser.
  • Engaging public site allowing anyone to follow the league, compare AI performance, and view weekly updates.

The Brief

As interest in artificial intelligence accelerated, Scout set out to create a project that reflected our expertise in effective AI utilisation and prompt engineering. AI FPL was developed as a live experiment to explore how different AI models think, reason, and perform in a fun, relatable, and competitive setting.

At the heart of the project was the skill of prompt engineering – the ability to craft and control the language that drives AI reasoning. This expertise is central to how Scout delivers results for clients, ensuring AI tools don’t just generate outputs, but produce meaningful, high-quality work aligned with brand tone, goals, and data strategy.

The premise was simple: could an AI manage a Fantasy Premier League team entirely on its own: researching players, forming strategies, analysing results, and adapting over time? Beyond entertainment, this was designed as a thought-leadership initiative exploring AI autonomy, strategic reasoning, and the question of whether machines can genuinely think independently.

The Solution

Scout developed a bespoke platform – ai-fpl.com – where leading AI models including ChatGPT, Claude, and Gemini are each tasked with managing their own Premier League team.

  • Framework & Prompts: Each model is given the same structured prompts to research, select, and manage their squad.
  • Live League Environment: A public league table tracks weekly results, showing how each AI compares to a human expert (a published football author and analyst).
  • Autonomy & Agent Mode: In the latest iteration, ChatGPT’s new Agent Mode allows it to log in and make transfers directly, running its own team with no human intervention.
  • Prompt Engineering: The precise design of instructions that shape how AIs think and respond – known as prompt engineering – sits at the heart of AI FPL. It not only levelled the playing field between models but also showcased Scout’s hands-on expertise in directing AI behaviour, a skill few agencies can truly claim..
  • Progressive Experimentation: The project will continue across multiple seasons, recording performance data to measure how AI reasoning and strategy evolve over time.
  • Platform & Design: A custom web platform built by Scout showcases team data, league standings, and commentary in a clean, engaging way

The Outcome

AI FPL has already become a unique testing ground for AI reasoning and behaviour, offering an accessible way for the public (and clients) to understand complex AI concepts through sport. Key findings so far have been as unexpected as they are revealing:

  • Many AIs lack innovation – often following online FPL guides rather than analysing underlying statistics.
  • Models struggle with data interpretation and contextual nuance, sometimes playing injured or banned players.
  • Despite being “machines,” AIs behave more like humans than computers — reading, imitating, and reasoning through words rather than data.
  • The project also exposed broader truths about compute efficiency and design priorities within commercial AI systems.
  • AI FPL also serves as a real-world demonstration of prompt engineering in action – showing how subtle variations in instruction, tone, and context can radically alter an AI’s performance. This reinforces Scout’s position as a leader not just in AI adoption, but in the deeper, technical craft of communicating with and optimising AI systems.

Though still growing organically, AI FPL has drawn strong interest from those who’ve discovered it – earning praise for its originality, humour, and genuine insight into AI development. For Scout, it’s a living demonstration of technical innovation, creative bravery, and true hands-on AI expertise.

In a Nutshell

  • "AI FPL provides a fascinating window into the limits and potential of modern AI - and showcases how expert prompt engineering is crucial to the use of artificial intelligence. The test is designed to explore AI autonomy, strategic reasoning, and the question of whether machines can genuinely think independently."

    Mike Lewis

    Managing Director, Scout Digital

Key Features

  • Analytic & Reporting

© Scout Digital 2026