Is putting everything in one huge 25GB file a bad idea?


Recommended Posts

55 minutes ago, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

45 minutes ago, LostCat said:

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

Seems like it would help performance generally and I guess it would help with reducing the game's disk space footprint? Honestly though, being one that likes to poke around in game assets and tweak stuff, I rather preferred when games had looser structures to their data. Unreal engine titles prior to version 3 were great like this, and Quake 3/Doom 3 pk3/4 files were just zips that you could open and modify at will. That was awesome.

 

While I'm sure there's some performance upsides to keeping game data in one big file, I'd imagine the big driver towards the one-big-impenetrable-file structure is to prevent tampering, especially in this age of Denuvo and built-in DLC.

 

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

1 minute ago, Circaflex said:

What exactly do you mean?

Like defragging the entire harddrive but just that one file :)

2 minutes ago, IanHead said:

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

I'm having serious issues with one area on the map where I would only get like 1 frame per second :( Never done that till now.

2 hours ago, Danielx64 said:

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

13 minutes ago, LostCat said:

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

Yeah, FEAR has those .Arch## files, though iirc it is split up into smaller chunks rather than one enormous one.

1 hour ago, Danielx64 said:

Like defragging the entire harddrive but just that one file :)

use Mark Russinovich's Contig or use Piriform Defraggler which also have similar capability (via its search feature).

Edited by Torolol

So you have bad performance in one particular area of a map in a particular game... You think it has to do with the area that file is located? Then wouldn't that imply hardware, not some file system anomaly? Just trying to follow the logic here... Instead of defragging, perhaps  a checkdisk is in order. Or it's just a bug.

On 2016-08-29 at 8:07 PM, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

15 minutes ago, LaP said:

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

3 hours ago, LostCat said:

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

Why?

 

My PC can still run new games. I wont upgrade it just to get USB3 support ;)

 

Not my fault if cpus did not progress much in the last 10 years. My overclocked core i5 750 is still doing the job at 1080p while paired with a 970. It is starting to show its age lately but it's still good enough.

It's not the packaging but the usage that's important. Guild Wars 2 is one 27GB asset file but has no loading issues. On the other hand, SWTOR takes minutes just to load the tiny interior of your personal starship.

  • Like 2
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Stellarium 26.2 by Razvan Serea Stellarium is a free open source planetarium for your computer. It shows a realistic sky in 3D, just like what you see with the naked eye, binoculars or a telescope. It is being used in planetarium projectors. Just set your coordinates and go. Stellarium key features: Realistic simulation of the sky, sunrise and sunset Default catalogue of over 600,000 stars Downloadable additional catalogues for up to 210 million stars Catalog data for all New General Catalogue (NGC) objects Images of almost all Messier objects and the Milky Way Artistic illustrations for all 88 modern constellations More than a dozen different cultures with their constellations Solar and lunar eclipse simulation Photorealistic landscapes (more are available on the website) Scripting support with ECMAScript (a few demo scripts are included) Extendable with plug-ins: 8 plug-ins installed by default, including: artificial satellites plug-in (updated from an on-line TLE database) ocular simulation plug-in (shows how objects look like in a given ocular) Solar System editor plug-in (imports comet and asteroid data from the MPC) telescope control plug-in (Meade LX200 and Celestron NexStar compatible) The major changes of this version: Added new sky culture Added new plugin: Planes Many improvements in plugins Many improvements in Core and GUI Many updates in sky cultures. [full release notes] Download: Stellarium 26.2 (64-bit) | 456.0 MB (Open Source) View: Stellarium Home Page | Other Operating Systems | Screenshot Get alerted to all of our Software updates on Twitter at @NeowinSoftware
    • NASA: This asteroid may not kill us but it probably won't be far off either by Sayan Sen Image by Zelch Csaba via Pexels New observations by NASA's James Webb Space Telescope have eliminated the last remaining impact threat posed by asteroid 2024 YR4, ruling out the possibility that the near-Earth object could strike the Moon in December 2032. NASA said observations collected by Webb on February 18 and 26, 2026, enabled scientists to refine the asteroid's orbit enough to "rule out a chance of lunar impact on Dec. 22, 2032." Instead, asteroid 2024 YR4 is now expected to pass the Moon at a distance of about 13,200 miles (21,200 km). The agency stressed that the update "reflects improved precision in our understanding of where the asteroid is expected to be in 2032 rather than a shift in its orbital path." The announcement closes a remarkable chapter in planetary defence that began in late 2024, when the approximately 60-metre-wide asteroid briefly became the most closely watched near-Earth object in the world. Discovered on December 27, 2024, by the ATLAS telescope in Chile, 2024 YR4 initially appeared to have a small chance of colliding with Earth on December 22, 2032. As astronomers gathered more observations, the impact probability briefly climbed to around 3%—the highest ever recorded for an asteroid of its size—before steadily falling as its orbit became better understood. By early 2025, international observations had ruled out any significant risk to Earth. However, astronomers were left with another possibility: a roughly 4% chance that the asteroid could instead strike the Moon. "The probability that asteroid 2024 YR4 will strike the Moon on 22 December 2032 is now approximately 4%," the European Space Agency (ESA) had said last year, noting that "there is a 96% chance that the asteroid will not impact the Moon." ESA said such an impact, while unlikely, would have presented an extraordinary scientific opportunity. "It is a very rare event for an asteroid this large to impact the Moon – and it is rarer still that we know about it in advance. The impact would likely be visible from Earth, and so scientists will be very excited by the prospect of observing and analysing it," said Richard Moissl, Head of ESA's Planetary Defence Office. "It would certainly leave a new crater on the surface. However, we wouldn't be able to accurately predict in advance how much material would be thrown into space, or whether any would reach Earth," he added. The asteroid also exposed an important blind spot in planetary defence. Because 2024 YR4 approached Earth from the direction of the Sun, it remained hidden from ground-based telescopes until after its closest approach. "We looked into how Neomir would have performed in this situation, and the simulations surprised even us," Moissl said. "Neomir would have detected asteroid 2024 YR4 about a month earlier than ground-based telescopes did. This would have given astronomers more time to study the asteroid's trajectory and allowed them to much sooner rule out any chance of Earth impact in 2032." He added, "As an infrared telescope, like Webb, Neomir would have also immediately given us a much better estimate for the asteroid's size, which is very important for assessing the significance of the hazard." The latest NASA observations underscore the value of space-based infrared telescopes in tracking faint asteroids. According to NASA, Webb made "among the faintest ever observations of an asteroid," extending the object's observational record by nearly eight months at a time when it had become too faint for other telescopes. That additional data allowed scientists to eliminate the remaining uncertainty surrounding its 2032 flyby. Although asteroid 2024 YR4 is now confirmed to pose no threat to either Earth or the Moon, scientists say its discovery remains one of the most significant real-world tests of the international planetary defence system, demonstrating how continued observations can rapidly transform an object once considered hazardous into one whose future path is known with high confidence. Source: NASA, ESA This article was generated with some help from AI and reviewed by an editor. Under Section 107 of the Copyright Act 1976, this material is used for the purpose of news reporting. Fair use is a use permitted by copyright statute that might otherwise be infringing.
    • Yup. Google is just scraping the entire internet for their own ad profits without sharing revenue with the sources. It's obviously stealing, but since these sites depend upon Google's search scraps to survive... As for me, I just stopped using Google for anything except Reddit searches. If Reddit's own search wasn't complete crapola, I'd never use Google search again.
  • Recent Achievements

    • Conversation Starter
      Admir earned a badge
      Conversation Starter
    • First Post
      The_Focal_Point earned a badge
      First Post
    • Apprentice
      daryld went up a rank
      Apprentice
    • Contributor
      Carltonbar went up a rank
      Contributor
    • One Month Later
      The_Focal_Point earned a badge
      One Month Later
  • Popular Contributors

    1. 1
      +primortal
      418
    2. 2
      +Edouard
      170
    3. 3
      PsYcHoKiLLa
      130
    4. 4
      Xenon
      69
    5. 5
      neufuse
      69
  • Tell a friend

    Love Neowin? Tell a friend!