Episode Transcript
[00:00:01] Speaker A: And the fact is, am I really going to trust a COA that I don't understand works if it's my life or my unit's life's on the line? No. No way.
We are moving from humans generating data and machines learning to machines generating data and machines learning.
[00:00:21] Speaker B: This is the Convergence, the Army's Mad Scientist podcast, and I'm your host, Rachel Melling. Mad Scientist is a US army initiative that continually explores the evolution of warfare, challenges assumptions, and collaborates with academia, industry, and government.
You can connect with us on social media ME madsci. And don't forget to subscribe to the blog the Mad Scientist Laboratory at madsci Blog Tradoc Army Mil on today's episode, we're talking with John Pan, army veteran and CEO of Xiao Labs. We'll be talking with John about the impact of artificial intelligence on decision advantage and how our adversaries are investing in this space.
As always, the views expressed in this podcast do not necessarily reflect those of the Department of Defense, Department of the Army, Army Futures Command or Training and Doctrine Command. Let's get started.
[00:01:16] Speaker C: John, welcome to the show. We're happy to have you.
[00:01:18] Speaker A: Thanks, Rachel. I'm excited to be here because I am a big fan of this podcast. Thank you.
[00:01:23] Speaker C: Thank you. Yeah, we have a lot of people in your community that listen to this podcast, so we're really happy and excited to have you here.
So before we get started, can you just talk a little bit more about yourself? So tell us about yourself and your background.
[00:01:35] Speaker A: Sure. So I'm. I'm Jon Pan. I'm the co founder and CEO of Xia Labs. We're building defense technology that helps war fighters make better decisions faster.
And it's a. It's a rather long story, but I want to share the story of how I ended up here, really at the intersection of two very different worlds, the military and then gaming.
So I grew up in New York, and during my first week of college, 911 happened.
And I remember just all of us rushing out to the streets when we heard the news and we could see the dust in the air.
I think it was the first time I heard jets, you know, right over New York City.
And like many other New Yorkers, seeing the towers fall inspired me to. To join the military, to serve.
And I joined the army as an infantry officer.
I had a great time there. I learned a lot, I grew a lot. And after five years of service, I took what I think would be an unconventional turn into the video games industry. I don't really see a lot of Veterans take that path. And I worked at game studios like Riot Games and then on the gaming teams at bigger technology firms like Amazon, Meta and most recently Walmart, where I was senior director of product for the gaming team. And let me dive deeper into how the bridge between these two worlds, military and gaming, led to the formation of XCL Labs. So like many people who serve, we stay connected with the people that we serve with. And it was last January I saw a post from Sam Lin.
He's now a retired colonel and he just simply made a post about soldiers coming back from a deployment with the 2nd Battalion, 15th Field Artillery Regiment. And he was organizing welcome home baskets through Walmart's registry system.
And Sam and I served together in then 5th Brigade, 2nd Infantry Division.
So since I was working at Walmart, I am working directly with the senior leadership team at Walmart. I thought, hey, I can really help amplify this effort.
So our entire team contributed to these welcome home baskets. And that sparked a call with Sam about what he was doing now because he just retired, I think two months before this check in call.
And I simply asked, I said, hey, I'm kind of interested in defense. I've done nothing related to defense since I left the military.
Is there any way I can apply my gaming and product background? And that's when Sam got me hooked into really Decision Advantage. So he talked a lot about ideas about using software and AI to enhance decision making cycles for targeting, for operational planning.
We did a deep dive into games and AI. We talked about Google, DeepMind's AlphaGo and all the subsequent online multiplayer games that served as the targets for AI models to beat humans on.
We talked about advanced NPCs, non player characters, how advanced they were in the game industry and why the game industry invested so much in that. And the answer is, because user acquisition costs are so high, game companies want AI that can simulate human behavior really well.
So back to Sam.
I love what he said. I started reading his articles, he's written a lot.
And I found one of his articles he wrote kind of like 10 years ago, his very first sentence was, it was quote, you know, the US Army's policies designed to mitigate risk may create a distortion for commanders by weighing tactical risk as less important than accidental and causing suboptimal decision making. So once again, decision making, decision Advantage. That's the origin story of how I got introduced to this world of military decision making and why Xia Labs was created.
[00:05:28] Speaker C: That's an awesome story. It's like a crazy journey that you've had to get kind of to get to this point and to, to where you are. And so you mentioned it throughout your intro, Decision Advantage. You know, a lot of our conversation today is going to center around this concept and we've talked a lot about this in Mad Sci, on the blog and in the podcast before. But can you elaborate a little bit more on what Decision Advantage means to you?
Does the army have it? Does DOD and our military have it? And if not, how do we get it? What's your, what's your thought on that?
[00:06:03] Speaker A: So what got me really excited about Decision Advantage from the military perspective is that I worked a lot on that topic. In the corporate world, we didn't call it Decision Advantage. We just called it our maybe annual planning process.
And you know, most companies do annual planning processes using frameworks. Now Google has one called OKR Objectives and Key Results.
At Amazon we have like an annual planning process called OP1. And then as part of that process we write a document called the prfaq. So lots of acronyms in the corporate world, just like the military. And the PRFAQ stands for Press Release and Frequently Asked Questions. So it's about thinking backwards from a product you want to build and then writing kind of what the press release will look like before you build anything.
And what got me kind of nerding out a bit about this is that Amazon's PRFAQ is remarkably similar to the military's five paragraph op order format. Right. So they are both the primary written artifacts for decisions across their entire organization and they both have standardized formats.
So let's get back to what Decision Advantage means to me, besides, I did a lot of it in the corporate world.
From the military world, Decision Advantage is simply the ability to make better and faster decisions than an opponent.
And I believe my personal opinion is that current DoD command control systems and processes have been optimized for low intensity conflict and they don't address decision advantage over a near peer competitor like the prc.
And I think furthermore, there are nuances between decision advantage between robots, drones and autonomous systems versus decision advantage between human formations.
And a lot of my work today is really focused on the latter. So getting a little bit deeper into like peeling this onion of Decision Advantage, I think Decision Advantage implies that you have advantage over someone else.
And that means we should evaluate kind of what's happening on both sides. And of course we don't have perfect information.
So I'm just sharing what I've personally experienced or what I've read or researched.
So from, from the US side, what's been very interesting for me is what when I left the military in 2010, I was just a junior officer, a captain then.
And now that I've met a lot of current serving soldiers out in the field, a lot of what they do, their processes, even some of the documents are quite similar. So it's kind of like opening up a time capsule. But. But that also shows you just how similar the so called tactical edge is even 15 years after I, I left the military. And let's consider a simple vignette, you know, a brigade in a field training exercise, deployed or otherwise. I've been just thinking, does a brigade staff and leadership truly know what is going on at any given time? You know, it's not going to be perfect information, but I think there's a lot of informational gaps and I say this because decision advantage requires you to know what's going on. So status is tracking mechanism and I'll share a couple of small stories. So one story is there's an army system that tracks every piece of equipment that units own, but there's no mechanism to deduplicate serial numbers that may be similar provided by different equipment providers.
So that's just like a little paper cut. But imagine that times 100. A lot of these stories happening across the army or even something more kind of like physical. Right? Sometimes just comms don't work and that's not in some detailed environment in the Indo Pacific. It's just like right here in a training environment.
And I've seen battalion commanders having to drive physically to a certain, you know, command post or brigade support area to get information just because sometimes the comms don't work. The primary way for units to report on their supply status and to request resupply through the lock stats. And there's inconsistent formats infrequent reporting frequencies.
And the reality is it's kind of similar to the story about what I saw back in, right before I left in 2010.
It's still kind of the same. The army is still very spreadsheet driven. Everyone's got different spreadsheets.
And because the formatting, the format of all these files are different and the columns and the data, it's hard to operationalize all of this at higher and higher echelons.
Ok, so that was really just a couple stories about maybe network process technology.
But the, the main decision making apparatus or process in the army is mdmp military decision making process.
And the main output of MDMP are course of actions or coas.
And coas are generated by staff and Reviewed and approved by the commander. MDMP is very interesting because it's both an art and a science. And I've nerded out about this topic. I even started a podcast called Decision Advantage where I talk to active duty and retired soldiers about this topic. And, you know, you can even write find articles about people writing about MDMP over the last few decades. And I. And you can actually find insights whether it's the article was from, you know, decades ago or or recently.
So the question is, if this is the main decision making process or apparatus for the army, should we use technology and can we even use technology to improve the MDMP process?
I personally believe we can and we should.
And we'll talk a little bit about kind of what the PRC is doing in this realm as well.
But diving deeper into coas for a second, there's actually no standard for what's a good or bad COA objectively within the army or the dod. And I think this is a problem that can be solved for the Army. I think we can leverage a lot of the Combat Training center data because I think at those locations they see a volume of units make decisions, and they track the results of what they've done. So this is a very, in my opinion, simple data exercise to create a benchmark.
I think we should have humans be the judge of coas in the beginning. And once we have enough benchmark data, we can leverage AI models to help evaluate coas. Okay, so how can we get Decision Advantage? I have two ideas.
First, by simply ensuring that existing tools and products in the army have data that can be shared across each application.
And I think the Army's NGC 2 effort is tackling this, but I think it'll be like, you know, it's a team effort. Everyone has to contribute to this.
Second, I think we can use, you know, I use the word AI, machine learning, and sometimes software interchangeably because I think people use the word AI too liberally.
So I might use the word software or machine learning a lot more. But the idea is a lot of the initial AI efforts in the army and across DoD have not worked quite well, specifically in the LLM realm. So there are a variety of, I'll call them kind of like chatbot prototypes, and they cannot execute military workflows.
Creating a KOA is a very complex workflow, but I would just challenge anyone to use.
They know what they are, the existing kind of army and DoD GPTs to just generate one KOA. And it's incredibly hard.
And that's why I think people thought AI was magic and AI should be able to do things like this and it just can't. And we'll dive deeper into my ideas of how we could do it. So those are some of the practical steps. If I was king for a day, we had unlimited money. The ultimate thing we can do with unlimited budget and time is really to create this ender's game like simulation from the movie from the book.
A practical application of that could be using MTO data for blue order of battle and capabilities, using TRADOC G2 Odin data for RED.
Having a primarily rigid war game where each side can execute their countries or their services equivalent of a mission essential task like company tag and so on and then simulating it thousands, hundreds of thousands of times.
If we have that system, why wouldn't we do it? Right. Because it's obvious that we should do it. How do we get the resources and the people behind to create this thing?
[00:15:10] Speaker C: Yes. So let's dive a little bit deeper into what, what we're seeing today. So you already kind of touched on MDMP and the way coas and how we're making decisions and the major decision making process of the army now. And so can you dive a little bit deeper into that and then are we taking advantage of machine speed to enhance this process in any way?
[00:15:31] Speaker A: Let's stick with MDMP and coas. There's lots of different decisions of course and I always, once again I'm really focused on brigades and below because the echelons above brigades so divisions and court, they have bigger staff. Right. Everyone's busy. But I think brigades are where the MDMP process gets pretty tough because of smaller staffs and more junior officers serving in those staff roles versus higher headquarters. So what I would say is right now brigades and below are doing MDMP very slowly. It's not amateur speed. And I think that's going to be only harder as staffs continue to be to be cut and reduced.
So if you've been in the military you've, you probably know that you'll sometimes have to do the same, if not more with less people.
And I think that's situation that everyone sees that is starting to happen.
Specifically, I've been invited to two staff exercises for army tick brigades and they have been typically four day exercises.
Now a staff exercise isn't just to conduct mdmp, it's also to train the staff on how to do it. So I do think once the staff is trained to do it, they could do it faster. But let's just say four days is average for A staff exercise day one is usually all about mission analysis step of mdmp and the primary output of mission analysis is a mission analysis brief.
Also, the output is getting the staff on the same page. So using our product, we can get a mission analysis brief in 15 minutes. I mentioned that not to brag, but to highlight the opportunity, which is not to use AI separately as a magic button, but to have it help humans do more of the automatic matable processes and let humans apply more creativity and judgment. For example, we can leverage AI to generate a starting point on mission analysis. Mission analysis really begins with reading lots of documents that specific task LLMs are good for. I previously mentioned that LLMs are not great for generating COAS, but I think LLMs are really good at generating mission analysis briefs. So use AI to generate a mission analysis brief. And that's just a starting point. It's not going to be perfect. And then you can have human staff officers come in and edit it and validate it as they see fit. If you broke down planning or MDMP into discrete steps that human can execute, I think there really could be tens of thousands of steps.
So focus software, machine learning, AI on the areas that are just more objective and requires less human judgment. For example, terrain analysis. Identify all the areas that have reverse slope or identifiable attributes by the staff or commander to best in place a brigade headquarters or a PAA or hlz. So we can use software to provide a universe of these points and then humans can make the final judgment on where to finally place it.
[00:18:45] Speaker C: So it's really interesting because like you said, like there's kind of like a, a little bit of a, we're doing it the quote unquote, like manual way at the moment, as opposed to being able to feed in that machine learning or see the impacts of machine speed on these types of processes.
And so, you know, we're focused on decision advantage obviously in relation to an opponent, in relation to an adversary, and we've touched on the PRC a little bit, but. But what about our adversaries? What about them? Why do you think China is investing so much into war games and AI and what do you think it's giving them?
[00:19:23] Speaker A: Yeah, and before I cover that, I want to go into the whole.
You know, you brought up a point about why are things still manual? Because we, we do have software there, there is software out there. But why do people, based on what I've seen, not really use it.
And I think it boils down to two things.
It seems so simple and obvious, but I don't think A lot of software was designed for, let's call it Brigades and below, right. It's been built for the Enterprise army, so higher echelons, people who are in an office, for lack of a better word, and I think people at the so called, we'll call it the front lines or the Brigades and below, they need to just get things done. So if a piece of software or a process or thing doesn't work, they'll just do whatever it takes to get it done.
And sometimes that means doing it manually. So I totally understand that. I think it's not that the technology doesn't work. I think the tools have not been built with the intuitive or great UI UX for every echelon, primarily Brigade and below.
And there's not enough of a feedback loop to get them to use it. Okay, so let's talk about the prc. So first of all, let's talk about why they're well positioned to tackle the space and then what are they doing?
So when you surface this up, the biggest technology battle right now is in AI and the prc. And their talent pool is just really good at AI.
I'm sure some folks may have heard about Meta recently creating an AI team called Superintelligence Labs.
And the big kind of news was that they were paying people compensation packages between 100 million and 200 million.
So now we're paying AI researchers as if they were basically like athletes, which is great, it's a great time to be an AI researcher. But then you look at the names of the people that they hired. It's like Shen Jia Zhao or Xiaohui Yu. I mean these folks are either Chinese, American or Chinese nationals here on a visa. And that's great, I'm glad they're on Team usa. But the point is of the top AI researchers have Chinese ethnicity.
So you know, the PRC is not lacking in talent front.
So that's the AI. So that's kind of step one is like, hey, PRC is well positioned to just invest in AI because they have the right people. And number two is really, it's about games.
So you know, why games? Well, I mentioned earlier, online multiplayer games have served as the benchmark for AI models to beat.
And when you are like the PRC and you have not been in large scale combat operations or really any form of war in decades, logically it makes sense that you want to be able to war game things. You want to simulate things because it's better than doing nothing, right? You can't just do nothing.
And culturally, just broader, not just, not just the prc. But in Asia, the Asian culture is just more open to games.
For example, in Korea, I saw a story where esports players were going to the Korean national soccer team before a match and pumping them up, right? So it's not the other way around.
In China, they have a national war gaming esports competition, including their best schools, and they compete on how to best invade Taiwan.
Right? And they have this on a, on a consistent national scale. On the US Side, I've seen some, some efforts primarily with the Marines for war gaming tournaments and, and so on.
And that's why this is a part that not many people really see. But the gaming part is super important because if you want to use AI agents to simulate human behavior or vehicles or whatnot, the environment that they are doing things in, you can call it different words, but it's primarily a game.
And okay, so what are they actually investing in? So, you know, step one and was, hey, they've got the right people. Step two is they've got the right culture to test AI in the realm of games.
The third part is what are they investing in? So the short story is both reinforcement learning and LLMs.
And I mention that because from the US side, most of our leading AI tools, we'll call it the Nipper GPTs, that's the air forces or camo GPT, that's the armies. Those are prototype AI applications based on LLMs.
But the original PRC AI algorithms, like Alpha War, named after AlphaGo, was based on reinforcement learning.
And just a reminder, try to use any dodgpts to try to generate a COA that a commander will actually use.
That's not a knock on those products. I just don't think LLM first technologies can do that specific military workflow. So let's go back to what they're doing. So they started off with reinforcement learning because LLMs were. They didn't get to that stage of technology development at the time.
And I think there's a lot that can be done with reinforcement learning that certain elements in the broader army are using. But it's more rare because it just wasn't like a primary focus. Although now reinforcement learning, I would say a lot of AR researchers are focused more on reinforcement learning than LLMs, which is kind of interesting.
So what can we do for reinforcement learning and what can we do with LLMs? So for reinforcement learning, one test we've done with Army War College is to take a rigid war game that they've created and added AI agents, reinforcement learning agents, to play both sides. In this case, it was a US versus PRC war game.
And then what the AI agents would do would be they will find a so called optimal path. You know, conflict or war doesn't necessarily, may not necessarily have like an optimal path, but you kind of get the abstraction of this idea. But you can maybe isolate certain areas where you could run simulations, you know, thousands or hundreds of thousands of times.
And RL is really the best technology and technique to apply to that problem space.
I've kind of, you know, dissed LLMs for a little bit, but I think they're actually quite still useful.
I think what's what the biggest opportunity for the DoD and the army is that right now the Frontier Labs and their LLMs, everyone, most people have acknowledged that they've trained on the entire corpus of data on the Internet, but that excludes DoD data.
So imagine, I don't think there's a repository of every, let's say, op order in the Army. There probably isn't, but that's why I keep going back to CTCs. They must have a big, they probably have the biggest database, right? Or a SharePoint drive, whatever you want to call it.
What if we can leverage, let's just say UP orders, like notional UP orders, and what happened with the units that executed the UP orders? That's an amazing data set that we can use for a variety of purposes.
So once again, I love to get in touch with CTCs. If you're listening, contact me.
Okay, so to summarize, what the PRC is Investing in is AI plus war games.
Why is AI plus war games important? Well, based on my technical belief, plus some of the PRCs is that you need to simulate entities in an environment and you have to give them rules of what they can't do.
And really what we've described is just a game. It's a game with two teams, for lack of a better word. And each team has different subcomponents with different capabilities.
And this is what we do in the video game industry.
And I think the PRC sees it because culturally they just accept games a lot more and because they've not been in large scale combat operations, they want to simulate things as much as possible.
[00:27:47] Speaker C: Yeah, that's really interesting. I didn't really know a lot about their gaming culture and how ingrained it is over there. And so I think that is something that's really interesting and a big comparison to, to us and to the way that, you know, we grow up in our military is today. And just kind of jumping off that, the simulations, the repetitions that gaming gives you is clearly it's, it's critical. And that's like a really, it's a really important lesson that we get out of gaming. And so how many hours do soldiers actually get to do combat in, in say a 20 year career?
And how can simulations, war games and other tools help with this? Even I've had, you know, interactions with zero sixes in the army who have said this is the first time in their career that they've ever had experience with war gaming. And that kind of struck me as, as surprising. And so kind of, what are your thoughts on that? And how can we integrate this into maybe lower echelons?
[00:28:49] Speaker A: So this idea I got from Colonel Kent Park, Army Colonel, and he actually proposed this question to me as well. So I started thinking about this, so I want to first credit him on it. And another thing I want to mention is, you know, I'm just using anecdotal data. I'm sure Army G1 may have the actual data, but you know, looking at a 20 year career, especially during, let's say a high op tempo during the GWA era, just based on people I know and what you've seen, you probably have four to five deployments in a 20 year career. And that's a lot, right?
And that's including, you could probably only get four to five years in because you have PME requirements, you have dwell time, other assignments and so on. So let's just say four to five deployments, let's call it four to five years of deployment time. Now how many days per deployment was it truly combat operations? Right. There's obviously different echelons, but in terms of intense combat operations, it's probably not 365 days per entire deployment.
It's probably every day during a specific mission. But what I've seen is that units have downtime as well. It's not just a constant grind for a whole deployment. So we'll just use a high estimate two years of full time daily combat operations, which seems a lot.
And that's spread over your career because it's not all happening in one deployment. Let's say it's happening over five deployments now using the Malcolm Gladwell kind of, he has this whole 10,000 hours framework.
How many soldiers really get that 10,000 hours of either being the individual trigger puller or maybe being an NCO or officer, but you're actively, for lack of a better word, managing violence?
Right. Like I, you know, I don't think many people actually can hit that 10,000 hours.
And you know, there's a debate whether it's actually 10,000 hours, maybe 7,000, but you kind of get the point. And going back to your comment about war games, you know, whether it's war games or combat simulation or like simulators, the idea is the same like how do you practice? Right. And I think as more from an individual soldier level. I remember using these tools, you know, they were more like individual soldiers simulators. And then when you move deeper into your career, maybe like a colonel, they start to do more operational level activities like war games. But one thing we already have war games and simulations, right.
So my contribution is to suggest that we need to make them work fun. And it's kind of interesting because as someone who worked on the gaming teams of these big tech firms like Amazon and Meta, we actually had to convince a lot of our leadership that games have to be fun because sometimes they think the game just becomes a software product, which it is.
And, and it's hard. And fun is hard. It's hard to pinpoint. Well, I know for sure. And you know, I recently went to another kind of training complex and they had, they had all these simulators.
It's just not fun, you know, like I bet, you know, do we need to convince any army soldier to play, you know, Call of Duty? No, we don't have to. Or if they don't like that specific game, they'll play another game. But I think you have to convince them to use an army simulation tool, right? You have to make it like a training exercise day for them to do it. So we have all this money, we have all this equipment spent. We know that, you know, simulation and war games are a good idea. My only suggestion is that maybe we just need to make them more fun.
If we made it more fun, maybe people will do it. One other story, obviously a lot of war games occur at the senior service colleges.
I went to a senior service college last year and it was two teams of 30 06s across all the services playing an operational war game. And it took four days and frankly it was very boring.
Some of them loved it. The real war gamers loved it.
A lot of people just thought, man, this is so slow.
So that's the other part. There's a whole debate obviously between digital and tabletop and I think there's really roles for both. But if you wanted just sheer repetition, I think that's where digital comes into play.
Have an opponent that increases in level of difficulty, not just always a human. Right. I know there's a whole debate about humans in the guy but that opponent could be human too. That human can automatically adjust their level of difficulty as you get better. And how do you save your results from previous rounds and so on.
So I think war games and simulations can help increase those reps, whether it's individual soldier level or, you know, NCOs and officers in, you know, managing violence or doing some other task.
And we need to make war games and simulations more fun and in certain cases leverage digital versions of them more.
[00:33:55] Speaker C: Yeah, I feel like if you're a professional wargaming or you're like a seasoned wargamer, you know, you probably think a lot of them are fun, but if you're more of a casual gamer, then maybe it takes a little bit more to kind of capture your, your interest in that way and make it fun.
And so you did mention a little bit about human and AI teaming and how important that is to, to decision making.
A lot of that is critical and it's all about trust. It's built on trust. And you've actually used and said this, that trust is the new oil.
And so what does that mean to you and how do we demonstrate trust in this process?
[00:34:35] Speaker A: I had to learn this the hard way. So to be self critical, I have to admit that, you know, after 15 years in the gaming and tech industries, when I decided to do software for DoD, I thought, you know, like, we're so smart, you know, we'll just, we'll teach, we'll teach DOD what to do.
Let's just go make some amazing software and then they'll like it. And that's why when we first started, we took this approach of automate everything, you know, just give us some information, we'll give you, we'll give you the whole, we'll give you coas to choose from, we'll give you the whole OP order.
And that's just wrong from an approach because as I said earlier, what's the point of generating a COA or anything if people won't use it? And the fact is, am I really going to trust a COA that I don't understand works if it's my life or my units? Life's on the line. No. No way.
So that mentality isn't just for our specific product or whether people are trying to do defense tech or how DoD is leveraging AI.
It's happening in the commercial world as well because they're going so fast with AI. Their whole idea is to remove humans out the process.
The biggest debate really just in technology right now is what's happening with Junior Engineers, do we need them anymore? And are senior engineers who can leverage AI just so much better at coding and output and productivity that we don't need them anymore? So there's lots of, you know, that's the lens, that's where things are going.
But that is not the right approach, I think, for commercial or for defense. So coming back to defense, I think the fact is there's art in war as well. And I don't think any AI system can really apply art in the sense of art of war. It can maybe draw art. So what we've pivoted into is instead of giving you the final answer, we're going to create decision support products where humans can check in and approve at each step. The commercial example is, you know, ChatGPT versus perplexity. You ask ChatGPT a question, it gives you an answer. Perplexity. It gives you the sources to every answer, right? Something as that, I think that's a low hanging fruit. So what we do is we're starting to integrate into actual army workflows and products people love.
You know, something, something very simple like the McCoo modified command oscillator overlay or DSM, DST decision support matrix and template. Those are actual products that commanders and staff use to make a final decision.
So we've turned to, hey, why don't we create those products and we'll show them tactically via tooltips or things you can hover on a screen to show you. This is how we calculated that. And if you don't like it, you can change it. Where did you get this piece of information?
FM4.0 Sustainment Operations, page 73.
That's just extra work on our end. But I think taking steps like that helps us gain a lot more trust. Because the fact is AI is just moving forward with or without us.
And it's just going to be so advanced that we're going to be at the stage where we will trust becomes a new oil. You'll need people or other systems to tell you, can you trust these output whether it's for defense or not?
Because it'll just get so good, right?
Or it'll appear to be so good that you need another mechanism for humans to trust before we actually do something important.
[00:37:59] Speaker C: Yeah, it's a really interesting conundrum that we're coming up on. And so yeah, it really does take that trust with us and the machine to be able to use it in our work. Now as we're coming to the end, we like to ask about where we see this Going, so what does the future look like in this area, in this decision advantage area?
And we're so relatively nascent when it comes to this capability. But what could be possible in, say, 10 years?
[00:38:30] Speaker A: The short answer is Ender's Game. And I keep coming back to that. And what's funny is that as I started to build this company, I actually rewatched that movie and read that book again.
And the more I think about it, I think that scenario might actually play out. So hear me out. Right now we're simulating humans, but also what's happening right around the corner is simulating autonomous systems.
Right? So in the near future, do we need humans to command a squadron of drones or robots?
Or will we have even more advanced AI that command themselves? How are humans in the loop now? Right, so another broad thing that's happening is we are moving from humans generating data and machines learning, basically the Internet and LLMs, to machines generating data and machines learning.
And that's now we're back into kind of like reinforcement learning, plus robotics. Right? So, okay, if humans are still somehow being involved in the direct control or command of these systems, if you believe in my kind of gaming analogy, we know that they're probably going to be younger and younger. It's kind of dystopians. It's basically Ender's game. It's the story of Ender's Game. They recruit these young people because they're better at playing games.
Right. And, you know, is warfare going to look like a game in the future?
I think an early glimpse of that is kind of dark, but it's the Ukrainian drone operators using Xbox controllers to control drones. We're already seeing that.
But what does that look like at a much bigger scale? And how does the world deal with that?
It's kind of scary, really, if you see this all play out, but it's kind of happening in real time. So I think we should think about this problem and try to navigate it as best as we can.
[00:40:30] Speaker C: Yeah, like you said, it is a little bit scary to think about thinking about 10 years from now when we could, couldn't even really think about what was, what's happening today. So it is, it is like you said, it's pretty scary. Well, Jonathan, thank you. This has been such a great conversation. We've really learned a lot, um, really building on things we've talked about before, kind of learning some new aspects. So, so we really appreciate it. And so now we're going to move into our rapid fire questions. So something a little bit different uh, save the. Save the crazy questions for the end. So the first one right off the bat, what is a trend or technology that keeps you up at night?
[00:41:06] Speaker A: Really? Everything we've talked about. I'm just so excited and sometimes scared of AI.
And right now, my singular focus in life is figuring out how to generate the best enemy. The enemy coas. How do I best generate the enemy coas? With as little information as possible.
And I know it's super nerdy, but I love talking to folks who are still active service members or retired army soldiers about this topic, because a lot of people are thinking about this and talking about it, and that's just. That's my singular focus right now.
[00:41:39] Speaker C: Yeah, it can be exciting and scary at the same time.
Nothing wrong with that. And so next, what is something about you that most people might not know?
[00:41:47] Speaker A: Well, this is a little bit embarrassing, but I would say most strangers wouldn't know that. I'm still an avid gamer, you know, now that I'm still. Now that I'm in my 40s still, and I play an embarrassing amount of games. I won't reveal the number, but. But it's a lot.
And I can barely kind of keep up with all the new games coming out on top of, you know, my professional duties.
[00:42:12] Speaker C: Yeah, it does seem kind of overwhelming with everything coming out. There's a lot. A lot going on. Well, that's awesome. Fits within. Within your line of work, so that's to be expected. And lastly, Matt would say this is the hardest question, but what is your favorite movie?
[00:42:26] Speaker A: That is such a hard question that I'll say I just watched a Superman movie and it was pretty cool, but I'm more of a TV show guy, so I'll say that my favorite TV show is very on theme with this whole podcast is Bennett Brothers.
And, you know, I watched that show 20 years ago before I joined the military, and I just watched it again. It's on Netflix, I think, a couple months ago. So it's really hard to watch that show and not feel great about being American.
And, you know, I think it's one of, like, the top 10 shows of all time, according to all these surveys. So highly recommend that show. If people haven't watched. Check it out.
[00:43:04] Speaker C: Yes, I will. We will accept that answer of a TV show over. Over a movie. I'm pretty sure we've definitely have that answer. Have had that answer in the past with other guests. So, yeah, they're definitely very popular. Very popular for our audience as well, so. So, yeah, great answer. All right, so lastly, I'd like to just ask where people can find you, where people can find your work. Where do you want to point everyone to?
[00:43:27] Speaker A: I would like to point everyone to two locations. First is our blog, where we write about a lot of this stuff. So blog.xialabs.com and the second place is really our own podcast called Decision Advantage.
And what I would love to get is other guests from other services because we're so deep into the army and mdmp. I'm very curious now. Hey, how does the Navy think about Decision Advantage? How does the Air Force think about that? How do the Marines. So a big part of this process is really leveraging all the talent and knowledge that exists within the Dodge.
So I kind of joke about this, but I listened to this show, right?
I think it was last year.
There was a Mad Scientist episode, episode 92.
I kind of researched this.
Know your enemy army doctrine starts with a threat.
And it was with General Brito, Mr. Ian Sullivan, and Colonel Rich Creed.
And the sentence that really got to me was, I think General Brito said this. It says, he said, quote, it starts with the enemy. Paragraph one is paragraph one for a reason.
Understanding the threat enables how we operate, understanding our enemy capabilities, and that can be applied to how we fight as an army. And that just that that's part of the reason why we're so into enemy COA generation right now.
Because frankly, practically speaking, a lot of staff, that's the lead. That's the last area they have time to focus on. They're going to focus on our COAS, not the enemy COAS.
But using a lot of the tradeoff, G2 materials, Odin and whatnot, I think we could take a really good shot at creating decent enemy coas.
[00:45:06] Speaker C: Yeah, I think that's, you know, a good closing as well, because, you know, as our, as our leader likes to say, the enemy gets a vote.
[00:45:13] Speaker A: Exactly.
[00:45:14] Speaker C: And so I think that is fits right into what you're talking about. Thinking about those red coas. Thinking about the adversary is one of the.
Everything starts with that. So that's where you. You got to build off of. So I think that is incredible what you guys are doing. All right, Jonathan, this has been a great conversation and we just like to say thanks for coming on the show and happy to have you.
[00:45:34] Speaker A: Thank you.
[00:45:35] Speaker B: Thanks for listening to the Convergence. I'd like to thank our guest, John Pan.
You can connect with us on social media me madsci. And don't forget to subscribe to the blog the Mad Scientist Laboratory at madsci Blog trick Tradoc Army Mil.
And finally, if you enjoyed this podcast, please consider giving us a rating or review on Apple, Spotify or wherever you accessed it. This feedback helps improve future episodes of the Convergence and allows us to reach a bigger and broader audience.
