Is putting everything in one huge 25GB file a bad idea?


Recommended Posts

55 minutes ago, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

45 minutes ago, LostCat said:

It depends.  As I understand it, it's good for the applications internal memory management.

 

Lots of first person games use large files such, though perhaps not that large.

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

Seems like it would help performance generally and I guess it would help with reducing the game's disk space footprint? Honestly though, being one that likes to poke around in game assets and tweak stuff, I rather preferred when games had looser structures to their data. Unreal engine titles prior to version 3 were great like this, and Quake 3/Doom 3 pk3/4 files were just zips that you could open and modify at will. That was awesome.

 

While I'm sure there's some performance upsides to keeping game data in one big file, I'd imagine the big driver towards the one-big-impenetrable-file structure is to prevent tampering, especially in this age of Denuvo and built-in DLC.

 

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

1 minute ago, Circaflex said:

What exactly do you mean?

Like defragging the entire harddrive but just that one file :)

2 minutes ago, IanHead said:

I did notice that Mankind Divided takes a hell of a long time to load, but the maps are huge and transitioning around them seems totally seamless.

I'm having serious issues with one area on the map where I would only get like 1 frame per second :( Never done that till now.

2 hours ago, Danielx64 said:

Looks like I need to find a way of getting that large file in one block on the harddrive. Mind you, The Division is quite large but use lots of small files.....

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

13 minutes ago, LostCat said:

I'm just sayin you can go all the way back to FEAR 1 and perhaps before that to find insanely large files in FPS land.

Yeah, FEAR has those .Arch## files, though iirc it is split up into smaller chunks rather than one enormous one.

1 hour ago, Danielx64 said:

Like defragging the entire harddrive but just that one file :)

use Mark Russinovich's Contig or use Piriform Defraggler which also have similar capability (via its search feature).

Edited by Torolol

So you have bad performance in one particular area of a map in a particular game... You think it has to do with the area that file is located? Then wouldn't that imply hardware, not some file system anomaly? Just trying to follow the logic here... Instead of defragging, perhaps  a checkdisk is in order. Or it's just a bug.

On 2016-08-29 at 8:07 PM, Danielx64 said:

The latest Deus Ex has like one massive archive file and several smaller ones. I cant decide if it a bad idea or a good one. I guess that with that huge file you would everything in one block on the harddrive so that performance would be good but IDK.

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

15 minutes ago, LaP said:

For backup it's not cool. Every time the game is updated you have to redo the whole 25GB backup. If the backup is done online or using an external disk on an old computer supporting USB2 only that's not cool.

 

Also if the file gets corrupted you have to redownload the whole thing (if you don't have a backup).

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

3 hours ago, LostCat said:

If you're still using USB2 or flaky drives but playing high end games like Deus Ex: Mankind Divided...that is weird.

Why?

 

My PC can still run new games. I wont upgrade it just to get USB3 support ;)

 

Not my fault if cpus did not progress much in the last 10 years. My overclocked core i5 750 is still doing the job at 1080p while paired with a 970. It is starting to show its age lately but it's still good enough.

It's not the packaging but the usage that's important. Guild Wars 2 is one 27GB asset file but has no loading issues. On the other hand, SWTOR takes minutes just to load the tiny interior of your personal starship.

  • Like 2
This topic is now closed to further replies.
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Posts

    • Windows 10 quietly gets one more year of support and updates by Taras Buria Windows 10 reached its end of life at the end of 2025. Microsoft kicked off the Extended Security Updates program, aimed at giving regular consumers one more year of security-only updates. By doing so, Microsoft gave users more time and money to update their computers to a newer operating system or compatible hardware. Now, with the end of the Extended Security Updates program quickly approaching, Microsoft is making an important adjustment. Users discovered that the official support article for the program now lists a new end-of-support date: The Extended Security Updates program is not a new concept. It has been an official way for business consumers to continue receiving critical updates for unsupported Microsoft products for many years. However, all this time, it was a business-only, paid feature. With Windows 10, Microsoft brought ESU to regular consumers, allowing them to get security updates for Windows 10 past October 2025 essentially for free. When Windows 10 was approaching the end of support, many guessed that Microsoft might adjust its support timelines, and this is exactly what seems to be happening. Of course, Microsoft would love everyone to switch to new computers, such as its latest Surface devices, but in the days of ever-growing hardware prices, not everyone is lucky enough to have money for a new PC. Leaving hundreds of millions of customers with a Windows version that no longer receives security updates is a major risk that Microsoft is not willing to take. If you have a Windows 10 PC to enroll in the Extended Security Updates program, check out this guide to learn how to do so.
    • Sony announces Bungie layoffs that will affect "significant number of employees" by Pulasthi Ariyasinghe Sony today announced that major layoffs are happening at its first-party studio Bungie, the developer that has spawned series like Halo, Destiny, and Marathon over the past decades. The news arrives just weeks after Bungie delivered the final update to Destiny 2, and it's that team being hit with the layoffs the most. CEO of Sony Interactive Entertainment Hermen Hulst revealed the staff reduction today, calling it "painful news." "Over the past several months, together with Bungie leadership, we reviewed the studio’s long-term direction, development priorities, resource needs, and role within our broader portfolio strategy," said Hulst, explaining the decision. "We explored multiple alternatives before concluding that a reduction was necessary to align the studio’s resources with its current priorities and long-term goals." The layoffs will be hitting "a significant number of employees" across most of the Destiny franchise development team. It doesn't look like Sony is planning to continue the series following Destiny 2's sunsetting update. The studio is said to be in early stages of looking at other projects to pivot to, but it's said that keeping the size of the team at current levels is no longer feasible. "We know this decision has a profound impact on the people affected, their families, friends, and teammates," said Bungie leadership in a separate message on social media. "While these changes are necessary to best position the studio now and for the future, that does not lessen the difficulty of this moment or the impact it has on those affected." At the same time, "some" of the Marathon development team are also affected by the layoffs. The recently released multiplayer-only extraction shooter title hasn't seen a big boom of players either, but the company is reportedly hoping that the live service experience will pick up players with future updates.
    • Microsoft adds reusable skills and finance data connectors to Copilot in Excel by Karthik Mudaliar Microsoft is giving Copilot in Excel a collection of new features aimed squarely at finance teams. The update introduces reusable instructions for common tasks, connections to services such as FactSet and Morningstar, and a better way to review what Copilot intends to do before it starts changing a workbook. The most interesting addition is 'Skills' finally coming to Copilot in Excel. Skills let companies teach Copilot how to handle a recurring process, so employees do not need to write the same detailed prompt every month. Users can create skills that can specify the steps Copilot should follow, along with the required layout, formulas, and formatting. Microsoft says users can create their own skills by saving a SKILL.md file in OneDrive. The file is written using Markdown and tells Copilot when and how to perform the task. Once it is available, a user can select the skill in the Copilot pane or mention it in a prompt using the @ symbol. There is also a library of prebuilt finance skills for customers who do not want to create their own. Microsoft plans to let developers distribute additional skills through the Microsoft Marketplace and the Microsoft 365 Admin Center, with LSEG, Ramp, Rogo, samaya.ai, Velixo, and Vena among the first partners involved. The company says that it is also expanding the external data that Copilot can access from inside Excel. New connectors are being added for CB Insights, Daloopa, FactSet, Morningstar, PitchBook, and S&P Global data through technology developed by Kensho. There is a catch, however. Accessing these services may require a separate subscription from the relevant data provider, so a Microsoft 365 Copilot licence will not necessarily unlock all of them. FactSet is also only available in preview for now, with general availability planned for July. Microsoft is also trying to make Copilot’s workbook edits easier to inspect. Users can switch to a planning mode that shows which sheets, cell ranges, formulas, and assumptions Copilot intends to work with before it begins making changes. Once the work is complete, the Show Changes pane can distinguish edits made by Copilot from those made by human collaborators. The update continues Microsoft’s push to turn Excel Copilot from a chatbot into an agent that can carry out longer tasks. The company previously added an Agent Mode capable of planning and completing multi-step Excel work. Microsoft also recently acquired financial AI startup Fintool, another indication that finance is becoming a key target for its Excel AI strategy. Prebuilt skills, personalization, workbook rules, external connectors, planning mode, and Copilot attribution in Show Changes are generally available to Microsoft 365 Copilot customers using Excel on the web, Windows, and macOS. Custom skills are initially available to Microsoft 365 Insiders on Windows and Mac starting today. Microsoft plans to make them generally available across Windows, Mac, and the web over the next month. Partner-built skills are expected during the third quarter of the year. Availability may still differ depending on region and licensing.
    • Exactly. They serve different (although related) purposes.
    • Do not enter the code under any circumstances, or you will be sorry. It's definitely and most likely a hacking attempt.  That happened to me a couple of years ago, and I kept receiving those prompts for months. It's simply the attacker trying to get you tired of the constant requests, so you just give up and enter the code, so they can log in to your account. 
  • Recent Achievements

    • First Post
      kinowa earned a badge
      First Post
    • Rookie
      krychek57 went up a rank
      Rookie
    • Grand Master
      Jaybonaut went up a rank
      Grand Master
    • One Year In
      Philsl earned a badge
      One Year In
    • Dedicated
      Scoobystu earned a badge
      Dedicated
  • Popular Contributors

    1. 1
      +primortal
      438
    2. 2
      +Edouard
      169
    3. 3
      PsYcHoKiLLa
      134
    4. 4
      Xenon
      77
    5. 5
      Michael Scrip
      75
  • Tell a friend

    Love Neowin? Tell a friend!