Files with long path names

[tnsply100]

I'm a developer, and often have deep directory structures which leads to long paths to files. Windows XP's explorer does not seem to be able to handle files with names larger than roughly 255-260 characters. Trying to delete such a file leads to warnings about not being able to access files.

Can someone confirm whether this limitation has been fixed in Vista? If not, are there any plans from MS to fix this? I cannot imagine the most popular desktop/workstation OS forcing its users to put up with this annoyance.

Perhaps BrandonLive (or someone else who works for MS) would be able to answer this best?

Any help would be appreciated...

[+primortal Subscriber²]

I don't think must people have that deep of a directory structure for it to become an annoyance. Not sure if its an os issue or format type (FAT32, NTFS) issue.

[Evolution]

I only know that there's a bug for long file names.... not long patch names.

[tnsply100]

I don't think must people have that deep of a directory structure for it to become an annoyance. Not sure if its an os issue or format type (FAT32, NTFS) issue.

You're probably right about most people not having the problem. However, developers (Java devs with extremely long package names, for example) working on large scale projects are sure to come across this little annoyance.

Windows is probably the most popular development/workstation platform as well... so I would expect developers/administrators to be quite inconvenienced by this.

Here is a discussion that google found.... http://discuss.joelonsoftware.com/default....joel.3.57363.29

I see no real workarounds other than hacks such as ...move your directory structure to the root of your drive (to make the overall path shorter), and then do your delete/rename/etc operations.

Also, I found http://support.microsoft.com/kb/q177665/ ...which means it has been around for years and years.

[+BudMan MVC]

Well you should be able to just create reparse points to get around long path names.. These are used quite a lot in vista (junctions). An have been available in window OSs for quite some time. They have just made them easier for users in vista with the mklink command

The work around for the 260 character limit path is \\?\

https://msdn.microsoft.com/library/default....ming_a_file.asp

Maximum Path Length

In the Windows API, the maximum length for a path is MAX_PATH, which is defined as 260 characters. A path is structured in the following order: drive letter, colon, backslash, components separated by backslashes, and a null-terminating character, for example, the maximum path on the D drive is "D:\<256 chars>NUL".

Note Functions in the Windows API convert "/" to "\" as part of converting the name to an NT style name.

The Unicode versions of several functions permit a maximum path length of approximately 32,000 characters composed of components up to 255 characters in length. To specify that kind of path, use the "\\?\" prefix.

Note The maximum path of 32,000 characters is approximate, because the "\\?\" prefix can be expanded to a longer string, and the expansion applies to the total length.

For example, "\\?\D:\<path>". To specify such a UNC path, use the "\\?\UNC\" prefix. For example, "\\?\UNC\<server>\<share>". These prefixes are not used as part of the path itself. They indicate that the path should be passed to the system with minimal modification, which means that you cannot use forward slashes to represent path separators, or a period to represent the current directory. Also, you cannot use the "\\?\" prefix with a relative path. Relative paths are limited to MAX_PATH characters.

When using the API to create a directory, the specified path cannot be so long that you cannot not append an 8.3 file name.

The shell and the file system have different requirements. It is possible to create a path with the API that the shell UI cannot handle.

[tnsply100]

Well you should be able to just create reparse points to get around long path names.. These are used quite a lot in vista (junctions). An have been available in window OSs for quite some time. They have just made them easier for users in vista with the mklink command

The work around for the 260 character limit path is \\?\

Thanks for the reply, although it was a bit more involved than what I was asking for.

I should have been a bit more clear... my issue is not with the Windows API. I'm not using any Windows API to manipulate files with long path names. I'm simply talking about using Windows explorer to manually rename/delete files from my development tree on the file system.

For example, it would be nice if I could go to any folder, however deep, (again, in windows explorer), and click delete to have it and its subdirectories deleted. (without popping up the message about not being able to access files with > 255 characters in its full path).

So, I guess, my question can be rephrased as "Is Windows Explorer going to be updated anytime soon so that we can deal with files with > 255/260 characters?"

While its comforting to hear that NTFS itself can handle larger path names, it would be desirable to have Windows Explorer do the same...

I will look into "junctions" to see if they can help... Thanks again.

[Dane]

wait, sorry I dont get it. Why is there a need for such long file names?

[night_stalker_z]

wait, sorry I dont get it. Why is there a need for such long file names?

You might have something like New Text Document (124987439258956293563925613985692365923568139569158365983126591836591365193563295236895238562935).txt

[tnsply100]

wait, sorry I dont get it. Why is there a need for such long file names?

It's not the file names themselves that are the problem (as far as this thread is concerned at least). It's the length of the file's fully expanded path (the filename is only one small part of that path).

As to why you might need such long path names.....that is a consequence of how source code is structured in development environments. When dealing with large projects, you usually end up with very deep folder hierarchies... which leads to very large expanded paths.

You might have something like New Text Document (124987439258956293563925613985692365923568139569158365983126591836591365193563295236895238562935).txt

Yes, you might... but that is not the issue addressed here. The above would be an example of a long single element in the path.

What I'm talking about in this thread is the MAX_PATH restriction on the length of the *fully expanded* path... not just its file name.

c:\Users\tnsply100\file.txt <-- this is a path with length 25 (roughly). Note that the filename itself only contributes seven characters to the total length of the fully qualified path.

Windows Explorer cannot deal with folders and files where the total length of the path is over 255 or 260.

Hope that explains it clearly....

[+BudMan MVC]

The work around is junctions ;)

See

C:\>junction c:\link C:\pathtest12345678901234567890123456789001234567890

012345678900123456789001234567890012345678901234567890123456789012345678900123

456789001234567890012345678900123456789001234567890123456789012345678901234567

890012345678900123456789001234567

Junction v1.05 - Windows junction creator and reparse point viewer

Copyright © 2000-2007 Mark Russinovich

Systems Internals - http://www.sysinternals.com

Created: c:\link

Targetted at: C:\pathtest12345678901234567890123456789001234567890012345678900

123456789001234567890012345678901234567890123456789012345678900123456789001234

567890012345678900123456789001234567890123456789012345678901234567890012345678

900123456789001234567

[tnsply100]

The work around is junctions ;)

Okay.. great. Will look that over (looks pretty much like soft links in unix/linux). Thanks.

If anyone else has any information about if/when Windows Explorer is going to be updated so that we don't need any workarounds, please post here. I know there are MS employees here who have worked on Vista (BrandonLive for example), so, any information on future plans would be appreciated.

I'm a bit surprised that folks have not made much noise over this issue (one that has caused many hours of headaches in development, build/automation, deployment, debugging, etc) as opposed to histrionics over "ZOMG no pretty bootscreen!!!" or "LOLZ.... Vista's maximized black windows are FUGLY!!!"

Edited November 13, 2007 by tnsply100

[+BudMan MVC]

I think your making it sound like its easier to fix than you think ;) I know vista started work on it, vista will auto shrink the path -- here is some info on it

--

http://blogs.msdn.com/michkap/archive/2006...13/1275292.aspx

Well, it is actually one of the efforts that took place in Vista, a feature whose official name I am not entirely sure about but which can be best described as "auto path shrinking."

The idea is simple enough -- when a path turns out to be too long, start shrinking individual elements in the path one by one until it is short enough to fix within MAX_PATH.

--

keep in mind the average user her on neowin, an you would understand why something like this is not a hot topic ;)

If you have to work with long path names -- I would of thought you would of found junctions quite some time ago ;)

[tnsply100]

I think your making it sound like its easier to fix than you think ;) I know vista started work on it, vista will auto shrink the path -- here is some info on it

Hmm.. you're right. It is always easy for customers to expect something to work without knowing the full extent of the underlying work. I'm learning this now as I've recently started my first "real" job after finishing up graduate school :)

In either case, I suppose I thought it would be quite trivial to fix since I've never heard anything like this for Linux/X.org/Gnome/KDE. Then again, I didn't work on large projects on those platforms either..

Interesting info on that blog... from the comments it seems like backward compatibility with certain programs is what is holding back a real fix. I suppose they think keeping compatibility with those programs is a worth putting new band-aids on the problem such as path shrinking (must be very important programs). Anyways, I guess I'm a bit out of my depth here.

If you have to work with long path names -- I would of thought you would of found junctions quite some time ago ;)

I've only had one other development position (only an internship) before this current one .. There I just put up with the problem and manually shortened names due to apathy :p But now, I decided to look for a more permanent solution in earnest. I guess I was expecting this to be one of the issues addressed in Vista.

Edited November 13, 2007 by tnsply100

[night_stalker_z]

I think your making it sound like its easier to fix than you think ;) I know vista started work on it, vista will auto shrink the path -- here is some info on it

--

http://blogs.msdn.com/michkap/archive/2006...13/1275292.aspx

Well, it is actually one of the efforts that took place in Vista, a feature whose official name I am not entirely sure about but which can be best described as "auto path shrinking."

The idea is simple enough -- when a path turns out to be too long, start shrinking individual elements in the path one by one until it is short enough to fix within MAX_PATH.

--

keep in mind the average user her on neowin, an you would understand why something like this is not a hot topic ;)

If you have to work with long path names -- I would of thought you would of found junctions quite some time ago ;)

Isnt this similar or the same as the dos 8.3 naming convention?

[Brandon Live Veteran]

I know there are MS employees here who have worked on Vista (BrandonLive for example), so, any information on future plans would be appreciated.

I didn't really work much on Vista directly, or at least more on the indexer and XP UI for WDS - then that "other" project. I moved / got moved to the shell team right as Vista was wrapping up. Of course, I'm sure you'll understand that I can't talk about future releases. Even if I could, I'm rarely aware of plans that don't relate to my feature area - especially for something as obscure as that.

Edited November 14, 2007 by Brandon Live

[abcdefg]

XP UI for WDS

Hi, WDS's UI really sucks. Please change that.

[tnsply100]

I didn't really work much on Vista directly, or at least more on the indexer and XP UI for WDS - then that "other" project. I moved / got moved to the shell team right as Vista was wrapping up. Of course, I'm sure you'll understand that I can't talk about future releases. Even if I could, I'm rarely aware of plans that don't relate to my feature area - especially for something as obscure as that.

Thanks for the update - I can understand if you're not aware of the future plans of other teams.

However, I'm a bit baffled as to you've termed this issue as "obscure" - reading over various sites on the net over the last few days, it seems that the problem affects quite a few folks - causing problems with development/deployment. From the link the other poster presented, it seemed like the shell team was trying new ways to reduce the visibility of the MAX_PATH issue.

As software becomes more complex, I can only imagine that this issue will pop up again and again in the future. Not being able to manipulate files on the file system through explorer (albeit in large scale development/deployment environments) seems a tad bit more important than "obscure" or trivial bug.

[Brandon Live Veteran]

Hi, WDS's UI really sucks. Please change that.

I didn't write / design it =P Just did lots of work to make it function on the new indexer.

[Brandon Live Veteran]

Thanks for the update - I can understand if you're not aware of the future plans of other teams.

However, I'm a bit baffled as to you've termed this issue as "obscure" - reading over various sites on the net over the last few days, it seems that the problem affects quite a few folks - causing problems with development/deployment. From the link the other poster presented, it seemed like the shell team was trying new ways to reduce the visibility of the MAX_PATH issue.

As software becomes more complex, I can only imagine that this issue will pop up again and again in the future. Not being able to manipulate files on the file system through explorer (albeit in large scale development/deployment environments) seems a tad bit more important than "obscure" or trivial bug.

Sorry, by obscure I didn't mean that it was trivial (changing it obviously isn't) or didn't affect anybody. I simply meant that knowledge of such a change would require a great level detail when your scope is the plans for an entire OS. Put another way, it's probably not a "back of the box feature" and I'm willing to bet the only people who know the answer are the people who own that components involved.

[tnsply100]

Sorry, by obscure I didn't mean that it was trivial (changing it obviously isn't) or didn't affect anybody. I simply meant that knowledge of such a change would require a great level detail when your scope is the plans for an entire OS. Put another way, it's probably not a "back of the box feature" and I'm willing to bet the only people who know the answer are the people who own that components involved.

Ahh. Okay. I guess I'll cross my fingers and hope Windows "Seven" will have the true fix (though I don't think it will... it seems MS is already done with half of the new features for the next release according to an "insider' article on Arstechnica)

[Preppy Veteran]

I read more lies about what goes on externally than I see true stories. *shrug* Take everything with many grains of salt.

If Windows added support for MAX_PATH*N paths everywhere without special considerations (as mentioned above), how many third party applications would that break? What level of user/IT frustration and rage would it cause?

This seems nice in a locked-down closed system, but in the open Windows environment it seems like a bad plan. *shrug* =\

[Lewyx]

I read more lies about what goes on externally than I see true stories. *shrug* Take everything with many grains of salt.

If Windows added support for MAX_PATH*N paths everywhere without special considerations (as mentioned above), how many third party applications would that break? What level of user/IT frustration and rage would it cause?

This seems nice in a locked-down closed system, but in the open Windows environment it seems like a bad plan. *shrug* =\

Does anyone know whether this applies to standard .NET file handling?

How many user applications would break? As I see they are already broken. Just mentioning the most famous ones: WindowsExplorer cannot operate on long pathnames just as TotalCommander. Both give a very descriptive error message (file not found, I think). Furthermore: Nero skips the files _SILENTLY_ and the user does not even have the chance to notice that something isn't right.

Btw. if only CreateFileW was updated to accept (see: accept, not return -- I don't see the potential for buffer overflow attacks) arbitrary long (32K will once be short) pathnames (even without having to prepend \\?\) it would allow lots of current software to work just fine with long pathnames. E.g. most people I know who use Delphi (pre-.NET series), simply append a #0 to the pathname, Length() it, and pass the two params to CreateFile (whoops... lots of programming frameworks open files just exactly the same way) ... and they currently get an error; all CreateFileW had to do was to implement the logic that the now require from the application's side:

Check if length>MAX_PATH, if not then continue as before. If yes then the client program is obviously not aware of MAX_PATH (at least in this case) so there is no point in applying the limitation to it. CreateFileW can convert the buffer to an absolute pathname, append \\?\, and proceed with the magic it currently does. Tons of programs now fully fixed without coding (except for the fix in ?kernel32.dll?). Other tons of programs partially fixed.

I see why an API may not return buffers longer than MAX_PATH, I just don't agree with why it cannot accept such buffers, and why functions that return a buffer cannot be set to return larger buffers (i.e. if a buffer is provided by the caller and a parameter of the call states that the buffer is 6K long then what's the API's reason to deny returning a 2K pathname).

Furthermore, if an application could signal in a single step that it is large-pathname-aware, the API could forget the MAX_PATH limitation for it. This would be useful for languages with easy large string handling (are there still others?) since they obviously already contain code to convert between strings and buffers, most of this code doesn't care about MAX_PATH. I just had to review the conversion code to ensure that MAX_PATH is not even mentioned, add the signalling to the application's startup, and everything would work fine.

As oppesed to what now happens: I have to check each system call against MAX_PATH, convert names to absolute, prepend some fun-stuff and go (but have no way of using the intelligent and fast copy service of the shell).

So I think many promising steps could be taken even with full backward compatibility.

Furthermore:

Most modern languages promote dynamic buffers. If a counterpart was created for APIs that use pathname buffers (like CreateFileW is for CreateFile(A) in case of the Unicode-issue) then most of these languages could instantly move to arbitrary-length-pathnames by redirecting to these new functions.

In most applications I've encountered was CreateFile's length-check the only one MAX_PATH bottleneck...

[Walter123]

It is not fixed in vista because of compatibility. You can use available solutions to be able to access files with long path. I do use http://www.abtollc.com/products.aspx it just allows to work with files keeping their real long paths, thats all, no additional overhead.

Walter