Reed Shaffner

I am the Chief Technology Officer at Teamworks, which means that I oversee our engineering, design, product and data science teams.

So I get the pleasure of overseeing what we build, how we build it, and it’s a lot of fun.

I’ve had a long and wandering career. I worked at Microsoft. I worked in gaming for a number of years, building things like Words with Friends and Farmville and Cityville and then went to a company, Scopely, that now has Pokemon Go.

I started a few companies that ultimately sold, the most recent one being a company called MOJO Sports, which was in the youth sports space, so we were working on instructional content for youth and media capture in that world.

We were then acquired by TeamSnap, which is probably the largest youth sports co-ordination and registration platform in the United States.

It’s got millions of users every month that log on to co-ordinate their kids’ schedules, to capture their games and live stream them. They are even starting to get into statting in the youth world.

You have people on there getting training plans for hundreds of thousands of soccer games and practices every year.

I was the CTO there for a number of years and that brought me here to Teamworks.

When we talk to our customers, AI is at the forefront of everyone’s mind. My role is thinking about how it is going to evolve the products that we put in hands of our customers.

There’s a computer scientist in the United States, John McCarthy, who’s first credited with the phrase Artificial Intelligence.

If we think back to the last three to five years, it’s entered the popular discourse. But long before that there have been disciplines of AI. You’ve had machine learning, you’ve had computer vision – which obviously is you know massively consumed in the sports world world and frankly foundational to a lot of the models that that people are using – you have natural language processing and then increasingly you have generative AI and these large language models and all the agentic workflows that are taking place.

Right now, when most people think about AI, they think about these large language models that are transformer based. They’re being used for a lot of purposes.

Those are your Groks, Claudes, ChatGPTs and all the rest. They are predictive models.

You may have heard people say that it’s predicting the next thing, the next word, the next token in a given sequence.

But it turns out that can be used to predict a lot of things, whether that’s the next pixel in an image or the next frame in a video.

Certainly clubs have been using machine learning for a long time. If we think about computer vision, we’re seeing that used across the world of football.

From an LLM perspective, you’ll be hard pressed to find a club where there are not at least a few people today in football using LLMs and experimenting with it. We’re increasingly seeing teams that are standing up their own lakehouses and saying, ‘Hey what can I do with AI directly on top of that?’

Certainly if you’re on one of our products, there’s an increasing amount of AI embedded into it.

I’m hard-pressed to find a customer that isn’t at least experimenting with AI. The reality is it’s really good for some applications today. And others, it’s still got a lot of work to get it to to where you want.

If we think about drafting a communication or contract analysis, I would argue it’s lower stakes and easier to review the output and I think you can get value very quickly.

If you look at other areas, like injury prediction, I would argue that the effectiveness today is still going to be of questionable return. If you think about injury prediction, I really need to know everything about you.

I need to know all of your risk factors and all of your history, which I rarely have. Injuries tend to be very infrequent in nature, so it’s very hard to achieve the outcomes people are looking for.

You have to ensure the context is there, which is really hard. There are a lot of environmental factors in the moment that can contribute to a given injury.

Hallucinations are incredibly risky.

I’ll give an example. Yesterday, we were debugging using AI. We were looking at some athlete data that hadn’t mapped to the correct ID and the model did a good job of identifying the challenge.

It also created two athletes that didn’t exist. And so suddenly we’re like, ‘Wait, these two people don’t exist.’ That’s easy for us to see and we quickly caught it. But essentially the models still have an occasional tendency to quite literally hallucinate – create things that didn’t exist.

They’re very good at being confidently wrong. This is when you hear customers or companies talk about building ‘evals’ (AI evaluations) and specialised approaches to deploying these models.

A lot of that is trying to condition the input and the output such that you’re not getting those hallucinations, so you’re not treading into areas where it can’t give you factually correct answers.

But certainly if you don’t have those guardrails in place, there are very few questions you’re going to go put into any of these LLMs where it’s not going to attempt to give you an answer.

(This is an injury prediction tool that leverages data and artificial intelligence to help clubs keep players healthy and performing at their best on the field. It does this by providing a complete view of an NFL player’s experience, taking video and data from training, practice and games and using AWS technology to run millions of simulations of NFL games and specific scenarios to tell teams when players are at the highest risk of injury. It achieved a 17% reduction in concussions and 700 fewer injury absences over the course of the 2024 NFL season).

I think it succeeded because it was focused on one particular type of injury. There was an incredible dataset there that the NFL was afforded because of the cameras they have in place, the incredible amount of sensors they have on their athletes, and it was really controlled, both in terms of the input data and the quality of the analysis.

This is a good example – if you focus on one particular problem, one particular injury, with a really robust dataset, with a really robust set of findings, then you can start to say, ‘Okay, I think I can see what leads to these types of injury.’

I was listening to one of your older podcasts with Luke Bornn, one of our colleagues, and he talks about ‘would you take good data or good process?’

I think ideally you’d take both and I think in their (NFL) case it was both great data as well as rigorous process, repeatable methodology looking at that.

Weird traumatic injuries or one-offs that happen infrequently, I think it’s always going to be hard. But I think for injuries that happen frequently, with good data leading in, I absolutely believe we will be able to get better and we are getting better.

My belief here is that the specialists, the doctors, the trainers that have the background, that have the intuition of what signals should we be looking at here will still continue to play a role.

Ultimately, the data you’re putting in (to an AI model) is being used to train future inference, future responses. The way we coach our employees is that if it’s something you would not be comfortable sending to anyone in the outside world, it should not be going into any of the consumer models.

We use models today but have agreements in place so that we know where the data is being stored and whether it can be used to train the public models. You have to be very careful about that.

I assume some of you have multiple multiple Gmail accounts? Have you ever just opened up the window and you’re like, ‘Oh, I’m logged into my personal account, not my work account.’

Unfortunately, what makes this tricky is even those with the best intentions sometimes make mistakes. You’re just, ‘Oh, I’m in my personal version of Claude and I accidentally put it there.’

You have to be very careful where you’re inputting data. It’s not just medical information. This could be contract information. It could be schedules, which you don’t think about. ’It’s just a schedule. I want to optimise the schedule.’

Well, now suddenly I know where athletes are and this could be out in the public domain.

So it really runs the gamut in terms of really being deliberate about where am I giving this information and how am I consuming the model. It’s actually somewhat easier through the large providers, because it’s clear to see their terms, it’s clear to see where you’re consuming it from.

Where everyone in the space has to be really careful is you’re working with vendors. Do you have a clear picture of what their data retention policies are, what models they’re using, where are they storing it, what is it being used to train?

Being able to ask those questions of anyone that is coming in and saying, ‘Hey, I’m going to give you AI functionality,’ is absolutely critical.

Done the right way, they can be safe and secure.

If you pull up Claude, it’s got a million connections. You can just click click click and you’re feeding all of that context in. It’s a question of through what forum are you doing this though.

Could any of that ever crop up anywhere else?

You see people launching attacks where they go and try and suss out information that has been shared into the model that should not have been.

And I give this example of a fake condition that a bunch of security researchers concocted. They made up this disease of the eye characterised by red, puffy, swelling eyes.

It was completely made up. They fabricated papers about it and they started to feed it into the large models. Well, what happened?

The models started to return answers saying this was a real condition, that this bixonimania was something you could get. 

You have to be careful with have they deliberately put information into the public models to mislead and lead to questionable answers?

Certainly as a technology company we have to do this.

We have to rethink what are the types of things that we have to train our employees in? What are the policies we have to put in place? It all becomes a a new frontier that we have to think about.

The reality is that there are new ways that you can be hacked, that you can be taken advantage of.

One example is skills. A skill in the simplest terms is a set of repeatable instructions that tells these models a particular set of actions to take or constraints.

I have a skill that constructs my morning brief, so I’ve given instructions on the things I want the model to check, things I want it to ignore, people in particular I want it to listen to signal from. Then I run that as an instruction-set to the model each morning to give me my morning brief.

People think of skills as just a text. It’s a markdown file, right? And you tell the model what to do.

So why wouldn’t I just go to the internet and plug in a skill? But you have to be careful with that. You used to think of skills almost as code, and people can use vectors like that as new ways to get at information in your systems.

Frankly, it depends on the size of the club.

Larger clubs that have their own analytics teams are always trying to get that edge and are always experimenting with their own models. They’re pulling in data from models like ours and using it to supplement what they had in place.

If you’re a smaller club with fewer resources, you might be entirely dependent on one of the model providers, your outsourced analytics staff. We see people doing everything on the model side.

In American football, we’ve seen people building tools for scout bias detection.

Does the evaluation from the scout, the quantitative scores that they provided about the player, match the written language they had in place?

We’re humans. We all have biases, and maybe I just like this person. Usually there are ways to start to get at that from the language you use, relative to the quantitative output.

So we see people using this stuff in really really creative ways that go beyond the models we think of, which are ‘tell me what player out there is is really good.’

People are doing incredible things with AI today and I think that’s only going to continue to increase as we as we go forward.

I wouldn’t say it’s the the norm yet, but we are certainly starting to see it pop up more and more.

Their ultimate focus may not just be on the team performance side, it’s how can I use AI from everything from how do we win more games to the commercials. Using it in the collegiate space here – I need to go raise money from donors, how can I use AI to enrich the information that I have about my donors and improve my comms to them?

So that Head of AI may have a mandate that spans beyond the team side all the way to the commercials as well.

Some sports are ahead on different axis.

I think baseball has certainly been the sport that has embraced analytics the most. You have teams with 30-person teams on the analytics side. It is a sport rooted in Analytics. You’ve had people taking box scores for a long time, and so they’ve been at it the longest.

There are other elements, like certain things that European teams are tracking relating to human performance that are frankly ahead of their American counterparts.

The problem, even for the last 10 years, hasn’t necessarily been data. Every club has data now and force plates and GPS harnesses and cameras tracking your joints down to the centimetre.

It’s what do I do with that?

Roles are going to shift. Whereas a person may have spent hours hunting for that video or putting those visualisations together, now you essentially shift their time forward to what I would argue is far more valuable.

If I don’t have to go find the relevant clips, if I don’t have to go build the visualisation, I can truly spend my time on analysing it.

At the end of the day, it’s how do you give people the information they need to make better decisions faster? I don’t think we’re going to see this huge drop in staff. I think we’re just going be able to focus on the work that’s ultimately the most impactful.

We’re using AI in our products in a whole host of ways.

In the systems that we build, people often have to upload a lot of data, whether it’s a survey that they want to send the players, or in the collegiate world in the US needing to upload a syllabus for an athlete that has their class schedule and athlete records. Or I want to upload my inventory into our system.

That used to take a lot of time. I had to go label it and figure out how to transform it. Not any more. We can use AI – I know everything that’s in your inventory, I can map it to a SKU (an external identifier), I can take that survey, I can immediately transform it to something that you can use digitally.

I could take that syllabus and extract all the dates and assignment types and put it on a calendar. 

We’re starting to see more and more LLMs to query data. The demo that everyone does is pulling up the typical chat interface and saying, ‘Show me the player with the highest load.’ And it goes into one of the providers on that side and spits back a graph that shows the player with the highest load.

That’s really exciting, because writing that query, doing those things, used to take a lot of time.

The evolution we’re going to see over the next few months, certainly in the next few years, is an evolution from reactive to proactive.

So much of that today is I still need you to go to the system and ask a question. I need you to say, ‘Hey show me the 10 players that are most exciting for this role,’ or, ‘Show me that load.’

With the agentic workflows that are coming online that’s going to evolve. We can know the second a player is injured and start recommending next steps. We’re going to move from this world of people needing to query the system to the system being able to proactively suggest recommendations and data.

I still think the human element is going to be critically important though.

If we look at Player ID, the problem we’re trying to solve is ultimately in the world of global football, how do we help you find the best players in the world?

Every club in the world has data now. I don’t think that’s the problem. The problem in identifying players is that different metrics disagree. One is going to say a player is excellent, another is going to say, ‘Hey, this player is average.’ And so what do you believe?

That’s what we’re out there trying to solve. So instead of picking a favourite metric, we’ve built several. We’re going to continue to build more, looking at the game in a different way.

We want to build a system using different approaches, that learns which one to trust in a given situation. It’s somewhat analogous to the human side, where you have a Sporting Director who has a room full of scouts and over time they learn which scout’s recommendations to trust. 

Who’s good at evaluating younger players? Who’s good at evaluating centre-backs? That’s what we’re trying to do on the technology side. We do that in our world through something we call the composite player metric.

The way I would frame it is we built a system that has learned from years of results that predate me, asking how much weight to put on the thing it’s looking at. I think the test we have to pass is not does it describe the last season well, but does this predict what’s going to happen next season?

That’s a much harder bar to to clear but it’s ultimately the problem we’re all trying to solve. It’s not just are they going to be good next season, but are they going to be good for the next five? That’s a really hard problem to solve, but obviously it is a very consequential one.

There’s obviously huge value there. How can we go find those gems? I’m a huge world football fan. You have these player pipelines of kids and athletes all over the world. How do you go and identify that player that that is going to provide that value, that is the diamond in the rough?

And frankly, one of the things that’s really exciting about this technology and what can be possible is we can get cameras in more places around the world with higher-quality footage, so it’s going to hopefully provide opportunities for more and more athletes to be discovered.

We will continue to get better at being able to identify those players with potential, with talent.

Going back to my time in the youth world, the cameras tracking kids all over the world now are going to get so incredible. We work with a company XbotGo that has a a camera that has an AI chip on the camera, that is panning and zooming and tracking kids down to the to the youngest level.

If we’re suddenly starting to get footage from every kid playing football around the world and now we have the means and models to analyse that. I get really excited about being able to identify talent regardless of where you are.

So many of the models we look at today tend to over-favour offence, so how can we start to do even more with off-ball and looking at defensive players? I think we’ll see more and more there.

I’m excited about a lot. There’s a world in which the software you use on a daily basis is personalised to how you use it. It’s rebuilding to you. You could argue, ‘Well, if everyone’s going to have this all-knowing technology and predictions, aren’t we all just going to be seeing the same thing?’

I don’t think that’s going to be the the case at all. There’s still going to be the human element of this, especially in sport. At the end of the day there’s 11 players that go on the field that have to do what they do.

But I think people are going to find creative ways to use this to produce personal unique outcomes for their club, for their culture and all the rest.

Workflows are going to change.. We’re going to see that people don’t have to do the parts of their job they don’t want to do. They’re not going to be cutting film. You’re not going to be sitting there having to manually annotate things. That is that is going to evolve.

Still I’ll go back to this though – there are 11 men or women on game day that have to go on the field and perform, and I don’t see them running around with AI chips in their heads telling them how to play football.

Certainly for the data we deliver, we invest an inordinate amount of time and treasure in ensuring that we keep data secure, that we make sure that we’re not giving it to models that are training public things.

We work with the US military, so we deal with data that is incredibly sensitive, that obviously we can’t afford to have getting out there. Certainly for the information and the systems they (Teamworks clients) use from us, I want them to sleep better at night knowing that we take that headache off of them.

I don’t think it then becomes ‘I don’t have to worry about this anymore’ though, because clubs – especially the larger ones – are going to continue to build their own models. They’re going to continue to experiment with AI. And so I think it’s not going to be an either or.

We’ve talked about data loss and data risk and I do lose a lot of sleep at night. The sophistication of hackers and attackers is going up every day.

There may have been disbelief at first when Claude came out and said, ‘Hey, we have this new Mythos model we can’t release because it’s it’s really good at finding vulnerabilities.’

I was like, ‘Okay, this is all marketing.’ It wasn’t. We’re seeing this in our own code, where particularly some of these more advanced models are very good at finding vulnerabilities.

On the plus side, it means now we can fix them very quickly. We find things we wouldn’t have found. The other side is that that’s all available to the attackers.

So I certainly lose a lot of sleep over ‘are we doing everything we can?’ To protect the information that we are trusted with.

There’s a quote, I think it may have been Warren Buffett – ‘It’s the best time to be a scammer.’ I can go synthesis e your voice. I can go synthesise your likeness. I can synthesise your style. I can use that to to launch these human engineering attacks.

Everyone tends to think about securing it at the code level. So many exploits don’t happen because of something that went wrong in the code. It happens because we’re humans. We’re fallible.

Someone calls you, they trick you, they get you to do something you shouldn’t. And the sophistication with which those types of social engineering attacks can be launched now is is scary.

(Reed will be presenting on the main stage on Day One of TGG Live ’26 at Old Trafford on September 29th).

I’m very excited to be there. Hopefully I get to show some some new flashy things too.

I’m very excited to come over to the UK and and get to speak with everyone. I’m a huge football fan. A lot of people in our company tend to be American football fans. I am far more biased to to world soccer than anything else. Getting to speak at Old Trafford is a pretty cool life experience.

Not to ruin everything,I think more specific some of the stuff we talked about today around you know where are we seeing people use AI effectively and where we may be seeing some of that I don’t want to say horror stories, but some of the areas where it’s not going as well.

I’m optimistic that by the time I come over to see you, we’re going to have some really good data from around the world of sport on how a lot of operators are using AI. Where are they on a spend basis?

Where are they finding success? I’m hoping I’ll be able to to share those findings with you. I’m a CTO Best part of my job is sometimes I get to show off some new shiny toys. And certainly I hope I get to show off some new shiny toys.