The Quicksilver Story: Chapter 7
Welcome to Quicksilver!
This is the seventh post about the story of our company. You can jump to the beginning here: The Quicksilver Story: Chapter 1 and to the previous chapter here: The Quicksilver Story: Chapter 6
Our story is four decades long, and still going strong. It involves more than 50 games for numerous different publishers, and a number of military training apps that leverage our game-design and educational-game background. Here we will cover the first part of that story.
Several themes show up repeatedly over the years:
- Synergies between projects spanning very different markets
- The amplifying power of great tools and proprietary engines
- Relationships that pay back years or even decades later
- The importance of asking the right questions and solving the right problems
- Coming to the rescue of projects in trouble
- The value of hiring the right people for the job
- Copy Protection, and Security by Design
- Sophisticated AI across all product lines
- Always learning new skills
Military Applications: Cognitive Training
It’s easy to imagine how computer-based military simulations could train basic skills such as marksmanship. There are plenty of modern combat games that provide focus on realistic first-person combat experiences. Such simple combat skills have already been a part of training systems, so there’s no new ground to be broken there.
But the military isn’t just about shooting guns. Strategy and tactics are important skills for leaders at all levels. This is where “cognitive training” comes in.
For example, a company commander in the US Army is in control of as many as 200 personnel. Some are front-line combatants, and others are in supporting roles. How should they be organized? How should they be deployed? How can supplies be provided to them as the battle continues? It’s been said that “no plan survives contact with the enemy.” How can leaders adjust their strategies when conditions on the battlefield change?
Around the year 2000, Quicksilver connected up with the newly-created USC Institute for Creative Technologies, a DoD-funded organization focused on bringing cutting-edge research and Hollywood-style production values to military training programs. The ICT, as it was known, needed to supplement its academic staff with experienced game and AI development teams to realize its visions. This led to a long, multi-year relationship during which we created a series of sophisticated training products for the US Army, Marine Corps and Naval Postgraduate School.
Note: for those who are wondering, none of the projects here used classified or secret information. Most were based on readily downloadable documents such as US Army Field Manuals. In fact, nobody on the team had a security clearance. Although the products were used in classified settings, none of the data created for those settings was made available to the development teams. Nevertheless, the images shown here have been carefully selected and selectively obfuscated.
Military Applications: Full Spectrum Command
The software would let the commander define the organization of the units (there's never just one way to do that), make a plan, and then oversee the execution of the plan in a real-time 3D graphics simulation.
Rendering the 3D simulation was going to be a huge challenge. This was the era of the GeForce 3 video card – far less powerful than even a current-day mobile phone. We looked at game engines such as Unreal, but Unreal would never have been able to handle 200 individual humans. At the time, it was limited to 32 of them active at any given time. Due to the nature of what we were doing, there would be situations where all 200 might be visible at once (think aerial viewpoint). And there was no easy way to work around that.
We went back to what we usually do – we created our own custom engine that was designed specifically to render only what was needed. We knew that the graphics quality wouldn’t look like the top video games of the day. Character models would be less detailed, textures would be very simple, and there would be no shadows cast on the ground. But we’d be able to animate 200 unique objects at once while maintaining a reasonable frame rate.
The second challenge was AI. The simulated soldiers in our app needed to execute their orders in a manner consistent with the TTPs (tactics, techniques and procedures) defined in the Army FMs (Field Manuals). We needed to teach an AI how to move across terrain, take cover, breach rooms, and clear entire floors in buildings properly. We had a top AI programmer working for about a year to make this happen.
One big issue with the AI was revealed during development: it’s not enough for the simulated soldiers to execute orders as written. They also need to know what to do when conditions change. If they’ve been told to advance to a given position, but they come under fire, they need to either take cover in nearby low-lying terrain or switch to “bounding” movement mode where one group provides covering fire while another moves. And they had to follow reasonable adaptations of the TTPs in order to accomplish that. This required a bit of careful thought and structuring of the code to prioritize a hierarchy of potential actions.
We traveled a couple of times to an Army training base and to our local Marine base at Camp Pendleton to watch training exercises and understand the nuances of the terrain (for example, exactly where to I need to be in order to hide behind the slight rise in the terrain?). We walked these “MOUT” (military operations, urban terrain) facilities, climbed the stairs to see the fields of view, and so on.
One of the things the Army liked most about this product was the way it allowed lessons to be taught without deploying 200 real soldiers in red-blue team live-action exercises. Those are expensive, time-consuming and dangerous, so soldiers seldom get to participate in them during their years in the service. With software, on the other hand, they can play through dozens of scenarios in a single course. We delighted in creating “trick” scenarios where commanders had to discover weaknesses in the deployment; in one, the mortar team was easily ambushed. But no real soldiers had to die in order to teach the lesson.
Full Spectrum Command was a success, so much so that we were asked to do a special version for the Singapore Armed Forces. Then we were brought in again to do a platoon commander app.
Military Applications: Advanced Research: Explainable AI
Artificial Intelligence has been “the next big thing” for decades.
Around the turn of the century, computers were just beginning to become powerful enough to run good game and training AI software. Surely, the age of super-powerful AI was right around the corner.
But AI had a huge problem. With the technologies of the day, such as Neural Nets, it was almost impossible to understand WHY an AI made a particular decision. With enough training data, and with clearly-defined performance targets, early neural net systems could produce very interesting results. But the best that researchers could do to explain their output was to point at the core mathematical equations driving these systems. They could not walk through the decision process following by the network; although the network produced human-usable output, it didn’t think like a human and could not explain its own reasoning process.
One of the researchers at ICT got funding to demonstrate that it was possible to design an “explainable” AI system that would be able to describe in human-useful terms what it was doing. He enlisted Quicksilver to write a custom version of Full Spectrum Command that would record and play back its actions, along with specific explanations of how the units decided what to do. For example, a unit could explain that its orders were to breach a given building but that it was taking cover because it was under fire from a nearby building. It could then show the enemy lines of fire in a fully traversable 3D world with a “flying camera” that could see anything in the scene.
When the software was completed, the academic team used it to prepare one of the earliest Explainable AI papers, “An Explainable Artificial Intelligence System for Small-unit Tactical Behavior,” with two Quicksilver team members as co-authors. The full paper is available here: Explainable AI Paper
Military Applications: Full Spectrum Leader
In the Army, a Company is made up of a headquarters section, several platoons, and sometimes other specialized units. In Full Spectrum Command, we enabled the company commander to specify orders for multiple platoons or other units, which would then be carried out via AI. The company commander is not typically “out front,” but instead is in a central location monitoring events and giving orders, as needed. Our user interface design reflected this primarily high-level view of the battlefield.
The platoon leader’s experience is very different. Commanding several squads of eight or nine people, the platoon leader is in the thick of the action, directing forces from a first-person view of the action. This necessitates a completely different style of control that’s basically “pointing and gesturing” rather than plotting lines on a map or moving pieces around on a sand table.
For our new product, which would be known as Full Spectrum Leader, we created a radial menu overlaid on the first-person view, with an inner circle of primary categories and a second set of wedges outside that circle with the specific commands. It was very elegant and easy to use, and well-suited to the fast action of a combat game.
We also refined and enhanced our AI. In our previous product, we were able to abstract some of the behaviors of units smaller than platoons (i.e., fire teams, as they are known). But in this game, the platoon leader needed to give orders to each fire team, and also needed more refined control over what happened when some of the soldier were killed.
Full Spectrum Leader was delivered about a year after our first training simulation.
Military Applications: Patriot Missile Battery Deployment
One of the most interesting parts of our military application development efforts was learning about how military organizations and their weapons systems work. We certainly learned a great deal when we were sent to Fort Bliss, TX to design a training tool for the deployment of Patriot anti-missile defense systems.
We’ve all seen photos of the Patriot missile launcher trucks. These distinctive flatbed-style vehicles feature large, boxy launch tubes that tilt upward when deployed. It’s tempting to think that the Patriot system is just that single vehicle. But that’s far from the case. A full Patriot battery includes more than two dozen vehicles, some weighing as much as 30 tons. Moving from place to place requires massively complex planning and logistics, and laying out all of the trucks in a functional formation is quite complicated. We won’t go into detail here about how that all works.
Our training product focused on a few key aspects of the deployment process: planning, movement and final deployment. Planning requires deciding exactly which vehicles will be included. Movement is complicated – the convoy is more than 1/4 mile long and needs to move through potentially crowded urban environments as a single, cohesive unit. Handling of threats requires careful thought, and the “obvious” answers are not necessarily correct.
When we visited the base to deliver the final product, we were brought into a classroom to see it in action. The instructor took an interesting approach. He had the soldiers create a reasonable plan and then watch it play out interactively. When an event occured during the movement phase, he asked the soldier to make a choice, but stopped him and made him choose the “wrong” answer. He did this specifically to show what would happen (loss of equipment and personnel) and experience viscerally what NOT to do. This is the power of simulation. In the real world, nobody would ever make that wrong choice just to see what happened. But when it’s just a simulation on a projector in a classroom, it’s safe to make the dangerous choices and really learn about the consequences.
Coming Up Next: Chapter 8 – Next-Generation Classics
The Quicksilver Story: Chapter 8
Let’s talk! We love to take on new challenges. Tell us what you need and we’ll let you know how we can help.
You can reach us here: https://www.quicksilver.com/contact.html#contact.
For more information:
You can find us on the Web at www.quicksilver.com.