Why Do Games Take Up so Much Space?

I got several questions from readers this week, all of them related to how large games are. So I’m going to answer them all in the same column.

First question:

Hi, Shamus,

Big fan here. Love your writing. Keep it going, man!

So, the other day I saw that one of the recent Metal Gears made fun of the fact that you don’t have to change CD’s anymore. (I think it was 4…) And that got me thinking… If Metal Gear Solid (from the PS1) was made today, with today’s technologies, would the same game, audio, graphics and content fit in a single CD? What I’m asking is… can new technologies in file compressing and optimizations make old games considerably smaller, thus requiring less disks?

Thanks a lot,

Rodrigo Lima Diniz

Short answer: A very annoying “maybe”.

Data compression is the science of reducing the size of data, usually to reduce its storage or transmission footprint. We’ve gotten pretty good at it in some cases, and sort of hit a wall in others. In particular, we’re really good at compressing video.

A single frame of uncompressed video at 1080p takes about 6,220,800 bytes, or about 6 megabytes. A Blu-ray disk holds 50 gigabytes, about 8,000 times that. It sounds like a lot, but at 24 frames per second that’s only just over five minutes of full-color 1080p video. So without compression, a Blu-ray disk wouldn’t be able to hold a movie much longer than five minutes. But according to Blu-ray.com, it’s apparently possible to get up to nine hours of video onto one.

Like I said, we’ve gotten really good at video compression.

However, video data is a very special case. When you’re watching a movie, very few of the pixels change from one frame to the next, and often they don’t change very much. This, combined with the fact that it’s often possible to throw away lots of color information (particularly in dark scenes) without harming the image for the average viewer, and it’s possible to get these amazing compression rates. (If you’re curious, I wrote a longer explanation of video compression a few years ago on my blog.)

Unfortunately, videogame data is nothing like video data. The bulk of game data is comprised of texture maps. Those are also images, but unlike video frames it’s a safe bet they’re not 95% the same. Compression is all about finding and removing redundant data and repeating patterns. There’s a limit to how much you can compress something without losing any information.

When we compress raw binary data, our compression rates don’t get much better than 2:1 or perhaps 3:1. Also, our general-use compression techniques haven’t advanced much in the last 20 years, either. Our computers might be a few thousands times faster, but our (non-video) compression algorithms have barely changed.

Maybe a tricky algorithm comes along now and again to shave an extra few percent off the final size, but there haven’t been any dramatic leaps forward. There’s a theoretical limit to how small you can compress something. (Obviously it wouldn’t make any sense if you could put a zip file inside a zip file and continue to see the data get smaller. In fact, if you put a zip inside of a zip, the end result will often be a file that’s slightly larger, since the data inside is already as small as it can get.)

I don’t know how big the first Metal Gear Solid game was, other than the fact that it came on two disks. I also don’t know if the game was already compressed. Game developers don’t want to have to compress the game data if they don’t have to. It makes loading times longer (this was more of a concern in the 1990s than today) because the computer has to spend time un-compressing the data before it can be put to use. But distribution costs were a big deal, and every disk ate into your bottom line. So if Hideo Kojima was finishing up MGS1 and found himself with 2 and a half disks with of content, then they might have worked hard to squeeze all of it onto just two disks by compressing things and hurting loading times. On the other hand, if the final game was 1.9 disks big, then unless they could compress it all the way down to one disk, there would be no point in compressing anything at all.

To sum up: We’ve made almost no gains in general data compression in the last twenty years, but it might just be barely possible to fit MGS1 onto a single disk if the original was (say) a disk and a half worth of uncompressed data. We could use heavy-duty compression on the data and make it fit, and a modern CPU would be able to unpack it quickly without giving us long loading screens.

Next we have three questions, all from reader Daniel:

First: Why are so many games that release on the PS4 and XboxOne so full of technical issues despite running on more powerful consoles?

There’s always a technological shake-out period when we move to new consoles. (Lair is a good example of a game that came out during the technological shake-out of last generation.) New hardware means totally different bottlenecks for programmers to worry about. Maybe on the GameBox 500, the memory was super small, so developers targeting that console got really, really good at finding ways to save memory. Then the GameBox 1000 comes out and it’s got lots of memory, but the CPU is just a little slower than what’s needed to to support all the fancy new graphics. So now developers have to re-write their engines and master different techniques, sacrificing memory in order to save CPU cycles whenever they can. With each new generation this problem is a little worse, because the graphics engines are more complex.

On top of this, a new console might mean new tools. (Like a programming environment, or perhaps tools for artists to figure out.) It takes time for everyone to get good at using them. This last generation was a big jump. The processor in the PS4 is radically different from the PS3. I expect the technological problems will smooth out as the platforms mature. The late-period PS2 games looked a generation ahead of the early ones, despite running on the same exact hardware.

If I can dare a prediction: It’s possible the problems developers are having hitting 60fps will get solved at some point. Late in the generation, it might be commonplace to see Xbox One and Playstation 4 games hit a stable 60.

On a console is there a difference in performance depending on whether the game is on a physical disk or digitally downloaded and why?

Assuming both are on the hard drive and don’t use any sort of disk-checking or phone-home DRM, then I can’t imagine any difference between the two.

However, if you’re playing directly from a disk you’re obviously going to risk slower loading times compared to playing from the hard drive.

What determines the size (in GB) of a game? Like Wind Waker HD is 8 GB while Xenoblade Chronicles on the Wii is around 1.5 GB and Xenoblade Chronicles X is 21 GB. Splatoon is roughly the same size as Xenoblade Chronicles but doesn’t have nearly the amount of content as Xenoblade. Meanwhile MGS 5 and Witcher 3 have roughly the same size of 25 GB yet have very different play times. What’s up with that?

While it varies by genre, when you’re talking current big-budget AAA spectacle-driven content munchers, a majority of your data is going to be texture maps. This is really pronounced on games that use Megatextures. (I explain megatextures in that video over there, if you’re curious.) Wolfenstein: The Old Blood came out earlier this year. It has a PC install size of 36GB, and 31GB of that is textures. Everything else – models, video, game logic, and audio – fits into the other 5GB.

In terms of hard drive space, megatextures more than deserve their name. They eat massive amounts of storage space. But those games are outliers. To my knowledge, there are only four games that use the technology.

For most other games, the size can often come down to art style and priorities. In an open world game the art team can have one grass texture that they uses everywhere we see grass, and every fortress and castle will be made from the same brick. Or maybe the artists have a sense of style, tone, and mood, so they have one grass for the steppes, another for the grasslands, another for the swamps, etc. Or maybe they’re really picky and each environment has a dozen different grass textures. This same idea will apply to dirt, stone, different kinds of wood, foliage, mud, pavement, and so on. One team might have a minimalist style and another might have a lavish collection of textures that nom up your hard drive space.

(Have a question for the column? Ask me!.)

Shamus Young is a programmer, critic, comic, and crank.