Is putting everything in one huge 25GB file a bad idea?


Recommended Posts

55 minutes ago, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

45 minutes ago, LostCat said:

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

Seems like it would help performance generally and I guess it would help with reducing the game's disk space footprint? Honestly though, being one that likes to poke around in game assets and tweak stuff, I rather preferred when games had looser structures to their data. Unreal engine titles prior to version 3 were great like this, and Quake 3/Doom 3 pk3/4 files were just zips that you could open and modify at will. That was awesome.

 

While I'm sure there's some performance upsides to keeping game data in one big file, I'd imagine the big driver towards the one-big-impenetrable-file structure is to prevent tampering, especially in this age of Denuvo and built-in DLC.

 

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

1 minute ago, Circaflex said:

What exactly do you mean?

Like defragging the entire harddrive but just that one file :)

2 minutes ago, IanHead said:

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

I'm having serious issues with one area on the map where I would only get like 1 frame per second :( Never done that till now.

2 hours ago, Danielx64 said:

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

13 minutes ago, LostCat said:

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

Yeah, FEAR has those .Arch## files, though iirc it is split up into smaller chunks rather than one enormous one.

1 hour ago, Danielx64 said:

Like defragging the entire harddrive but just that one file :)

use Mark Russinovich's Contig or use Piriform Defraggler which also have similar capability (via its search feature).

Edited by Torolol

So you have bad performance in one particular area of a map in a particular game... You think it has to do with the area that file is located? Then wouldn't that imply hardware, not some file system anomaly? Just trying to follow the logic here... Instead of defragging, perhaps  a checkdisk is in order. Or it's just a bug.

On 2016-08-29 at 8:07 PM, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

15 minutes ago, LaP said:

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

3 hours ago, LostCat said:

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

Why?

 

My PC can still run new games. I wont upgrade it just to get USB3 support ;)

 

Not my fault if cpus did not progress much in the last 10 years. My overclocked core i5 750 is still doing the job at 1080p while paired with a 970. It is starting to show its age lately but it's still good enough.

It's not the packaging but the usage that's important. Guild Wars 2 is one 27GB asset file but has no loading issues. On the other hand, SWTOR takes minutes just to load the tiny interior of your personal starship.

  • Like 2
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Look up 'inflation' kid. Ask an AI for the numbers between both games.
    • Google reportedly set to lose two key Gemini and DeepMind researchers to Anthropic by Karthik Mudaliar Google is reportedly preparing to lose two more prominent artificial intelligence researchers, with Gemini contributors Jonas Adler and Alexander Pritzel planning to join rival AI developer Anthropic. According to a report from Bloomberg, both researchers are viewed internally as important contributors to Google’s flagship Gemini model family. Adler worked on Google’s AI coding efforts, while Pritzel was involved in the process used to train AI systems. Neither company has publicly confirmed the moves. The report also does not say when the researchers will formally leave Google or what positions they will hold at Anthropic. Training a large AI model requires decisions covering its architecture, data preparation, distributed computing infrastructure, and post-training methods that shape how the finished system behaves. Researchers with experience operating at the scale of Gemini are consequently difficult to replace quickly. Both Adler and Pritzel have previously contributed to Google DeepMind’s scientific research as well. They are listed among the authors of the company’s work on expanding AlphaFold protein-structure predictions across entire proteomes, alongside AlphaFold researchers including John Jumper. The reported departures arrive shortly after another important change within Google’s Gemini organization. Gemini co-lead Noam Shazeer is leaving Google for OpenAI, after returning to the search company in 2024 through its deal with Character.AI. Shazeer is particularly well known as one of the authors of the Transformer paper, whose architecture became the foundation for most modern large language models. Anthropic, meanwhile, has been recruiting recognizable figures from other leading laboratories. OpenAI co-founder and former Tesla AI director Andrej Karpathy joined Anthropic’s pre-training team in May. His move, followed by the reported recruitment of several Google researchers, suggests Anthropic is strengthening the research teams responsible for the core capabilities of future Claude models rather than concentrating solely on product and enterprise sales. The competition is complicated by the companies’ extensive commercial relationships. Anthropic competes directly with Google’s Gemini models, but it also relies on Google as an infrastructure partner. In April, Anthropic announced an expanded agreement with Google and Broadcom covering multiple gigawatts of next-generation Tensor Processing Unit capacity. TPUs are Google-designed accelerators used to train and run large AI models. via Bloomberg
    • This article makes my head hurt. Lots of confusing words
    • Google adds built-in computer control to Gemini 3.5 flash by Karthik Mudaliar Google has added Computer Use as a built-in tool in Gemini 3.5 Flash, giving developers a single model that can reason about a task and operate graphical interfaces across browsers, mobile devices, and desktop environments. The feature is available through the Gemini API and Google’s Gemini Enterprise Agent Platform, although it remains a preview feature for now. Computer Use enables an AI agent to examine screenshots and return actions such as mouse clicks, scrolling, and keyboard input. A developer’s application must execute those actions, capture the resulting screen, and send it back to Gemini, creating a continuous loop until the task is completed. Google says the integration can be used for activities including repetitive form filling, application testing, research across multiple websites, and longer enterprise workflows. Gemini 3.5 Flash can work with browser, mobile, and desktop environments, whereas Google’s earlier standalone Computer Use model was primarily positioned around browser interaction. The main change is consolidation. Computer control was previously offered through the separate Gemini 2.5 Computer Use preview model. As Neowin reported when that model was introduced, it was designed to interpret a visual interface and generate actions without requiring a website-specific API. Google later brought Computer Use to preview versions of Gemini 3 Pro and Gemini 3 Flash in January 2026. The latest release now incorporates the tool into the stable Gemini 3.5 Flash model rather than requiring developers to select a specialized model solely for interface automation. Gemini 3.5 Flash itself was announced in May as Google’s latest fast model for coding and multi-step agent workflows. It supports a one-million-token input context window and up to 65,000 output tokens, along with adjustable thinking levels that let developers trade additional reasoning for lower latency and cost. Google also added that Gemini 3.5 Flash received targeted adversarial training for computer-use scenarios. The company is also offering safeguards that can require user confirmation before sensitive or irreversible actions and automatically stop a workflow when suspected prompt injection is detected. Its developer documentation describes configurable protections for areas such as financial transactions and changes to sensitive records. Google isn't the first to bring Computer Use to its platform. Anthropic has made computer control available through Claude, while OpenAI has continued improving computer-use performance in its recent models. Microsoft has also applied the concept to business workflows, including a Computer Use capability for the Researcher agent in Microsoft 365 Copilot.
    • After I installed KB5095093, the volume on my ARM laptop won't go above 20%. It's stuck on the hearing protection level, which is pretty much useless if you want to listen to anything. I rolled back.
  • Recent Achievements

    • Dedicated
      Scoobystu earned a badge
      Dedicated
    • First Post
      Tom Schmidt earned a badge
      First Post
    • One Month Later
      D0nn13 earned a badge
      One Month Later
    • Rookie
      +ChiefOfNeo went up a rank
      Rookie
    • One Year In
      Tom Schmidt earned a badge
      One Year In
  • Popular Contributors

    1. 1
      +primortal
      463
    2. 2
      +Edouard
      177
    3. 3
      PsYcHoKiLLa
      124
    4. 4
      Michael Scrip
      79
    5. 5
      Xenon
      76
  • Tell a friend

    Love Neowin? Tell a friend!