Microsoft announces Git Virtual File System for better performance on large repos

Git is a version control system created by Linus Torvalds in 2005 and well-known among software developers to track changes in files. The system makes it easier for developers to work together on the same project by providing a fast and reliable distributed revision control system.

But Git wasn't designed to work with repos with millions of files or hundreds of gigabytes, a problem that Microsoft has faced when trying to implement the system for some of its developer's teams. As an example provided by Microsoft on a blog post at MSDN, the Windows code base has over 3.5 million files and is over 270 GB in size. Such a huge repo caused many performance issues when running Git commands.

For example, a “git checkout” would take up to three hours to complete and a simple “git status” would take almost 10 minutes to run. But worse, a “git clone” would take 12+ hours to complete.

That's why Microsoft is developing the Git Virtual File System (GVFS), a project Gabriel Aul, former leader of the Windows Insiders Program and now a Microsoft VP, is also working on. According to the blog post on MSDN:

"(GVFS) virtualizes the file system beneath your repo and makes it appear as though all the files in your repo are present, but in reality only downloads a file the first time it is opened. GVFS also actively manages how much of the repo Git has to consider in operations like checkout and status, since any file that has not been hydrated can be safely ignored."

Furthermore, because the changes were made on the file system level, no change is necessary on the developer's IDEs or build tools. And the results achieved using GVFS are amazing: a “git checkout” that would take three hours, takes only 30 seconds, a “git status” that would take 10 minutes, takes up to five seconds, and a “git clone” tha would take 12+ hours, takes only a few minutes.

Finally, Microsoft has made GVFS open source through its GitHub account, but warns developers to "be aware that it still relies on a pre-release file system driver", which has also been made available. The company has also released some other changes made to Git to allow it to work well with GVFS.

Source: MSDN via Gabriel Aul | Image via Ittai Zeidman