How to lock a file completely from being read

The situation ist the following: In the company I work in we have many servers running. The one I work with normally has 1.2 TB hard disk space. For one application we need really large data files (12 files occupying a total of 380 GB). These files are never modified, but as you can guess, each time a defragmentation is run, these 380 GB will be moved a few megabytes forward or backward on the harddisk.

Since I found this very annoying, I wrote a small defragmentation program myself which puts the 12 files at the end of the harddisk, so that there will be no need to move them every time we run a defragmentation utility.

But the problem remains the same: When we run a defragmentation, the program moves the 12 files from the end of the harddisk to the end of the occupied space. Although I am have Admin rights on this local server, all defragmentation runs are controlled over the company network, so there is no way for me to add the 12 files to the "exclusion list" of the defragmentation programm.

According to the Microsoft defragmentation API the defragmentation can only work if the function is called with a filehandle which has at least read access.

Now my idea was to simply lock the 12 files from being opened in read mode when the defragmentation is run, so that the programm will not be able to move them on the harddisk. I tried this by simply opening the files with exclusive access and also by removing all access rights (including the ones for "SYSTEM" and "Admin"), but all methods I tried did not prevent the defragmentation programm from moving the files. (although it prevented my own defragmentation utility from defragmenting the files, since the files could not be opened for reading)

Now my question: How do I lock a file so that it cannot be opened by defragmentation utilities? Is this even possible from user mode or do I need to write a KMD?

[redbull]

In the old days (of DOS and up to win 9x) you could prevent a file from moving by giving it ReadOnly, Hidden and System attrributes.

Under win2k and xp try giving the file hidden and readonly attributes and see if that makes any difference.

Otherwise defrag tools like Norton Speed Disk allow exclusion lists.

No, read-only, hidden or system attributes set doesn't make any difference.The only thing which needs to be done to defragment a file is to open the file for reading. Most programms don't even care any more about the attributes

Our defragmentation programm allows exclusions, but only company-wide and I'm not allowed to edit the exclusion list since I'm only Admin on one server and not company-wide.

Originally Posted by MarkusO

According to the Microsoft defragmentation API the defragmentation can only work if the function is called with a filehandle which has at least read access.

Are you shure defragmentation programm uses Microsoft defragmentation API?
What are defragmentation programm?

We use Diskeeper.

Yes, I'm sure that the defragmentation API is used since every defragmentation programm has to use this API (Microsoft doesn't allow any other way). This was always like this in any WinNT version.

In WinNT 3.1 and 3.5 the API didn't yet exist, so Executive Software (Diskeeper) had to buy the WinNT source code in order to implement their own defragmentation functions. All Diskeeper users had to get all updates and Service Packs directly from Executive Software since the Microsoft files where incompatible with their version of Windows.

Starting with WinNT 4.0 Microsoft implemented a few APIs for defragmentation in Windows. Since this time all defragmentation utilities have to use this API.

[LaDidi]

I think you may use ACL to deny access to this file.
know which user has used fior defrag and deny it.
Regards.

Originally Posted by MarkusO

... and also by removing all access rights (including the ones for "SYSTEM" and "Admin"), but all methods I tried did not prevent the defragmentation programm from moving the files.

I already tried the ACL method. The Diskeeper service runs as "SYSTEM". Removing all access for both "SYSTEM" and "Admin" accounts did not help.

[LaDidi]

Have you remove access or DENY access to account ?
It's not the same result...

Have you tried LockFile or LockFileEx API ?
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/lockfile.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/lockfileex.asp

I missinterpreted what you said last time. Now I also denied any access for any user group, but the files could still be defragmented.

The next thing I tried was the LockFile and LockFileEx APIs, but neither of the prevented the file from being defragmented, in fact you could even open the files in read mode, only trying to open the files in write mode gave a lock violation.

[Mr.S]

Here is one of the solutions to accomplish your task which will work depending on the program you¡¯re using.

Resize your 1.2 TB (decrease main size) and on the end of the physical disk create additional partition, move your files (hide this partition if you want)

Since de-fragmentation works on logical partition, remove the de-fragmentation from your new drive.

The best way would be to write a KMD which hooks NtCreateFile, and then check for the filename, and if it's one of your files, return a failure code (or hook some of the directory listing API's to remove the name from the list).

There are also programs for hiding files and folders, most use the KMD method. So you can just download one of those programs and try that. They hide a folder or file from even windows explorer, so they should work fine. Just Google it.

-Lunar

[Eleven]

From the diskeeper manual

Files that are open for exclusive access, including Oracle databases, Exchange databases and the paging file, cannot be moved or defragmented by Diskeeper. This is because the Microsoft API for defragmentation will not move files open for exclusive access because files that are open in that fashion are opened for input and moving them could cause corruption. We consider the integrity of your data to be of greater importance than file defragmentation.

It also says that these files can be a nuisance to diskeeper solely because fragments within these files will cause problems:

Files that are open for exclusive access can be a problem, because the fragments of these files, being immovable, can prevent Diskeeper from consolidating enough free space to defragment the entire partition.

If the files are only read like you said there will be absolutely no impact on the effectiveness of the defrag.

You could try exploiting this for your own benefit by temporarily opening the file for exclusive access (this is assuming that during defrag these files don't need to be accessed by your software).

@Elven:
This is what I'm actually trying to manage. But even when the file is opened for exclusive access (open with GENERIC_READ + GENERIC_WRITE, set no share mode, use LockFile on whole file, denying all access for all users/usergroups including SYSTEM and ADMINISTRATOR), it will still be moved by Diskeeper for some reason. However, my "fragmentation" utility will not be able to move the file.

@Lunar_Dust:
I will try to find some of the programs you mentioned.

[Mkz]

The defragmenting method being used is probably the FSCTL_MOVE_FILE control code: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/fsctl_move_file.asp
It requires a handle to the volume, and a handle to the file with FILE_READ_ATTRIBUTES right.
If you open the file with no sharing, other processes can probably still open it with only this right.
If you deny this right to the SYSTEM account, the program might still be able to open it with backup semantics, bypassing the ACL's.

One way would be to intercept the DeviceIoControl function, check for the file being moved, and return an error if it's the files you don't want to move. But then you'd have to see the reaction of the defragmenter - would it skip this file and go on with the other ones? I doubt it.
Returning success and not doing a thing is also risky, the defragmenter might rely on the return code to update its internal copy of the volume bitmap, and assume the file has moved...

@Mkz:
I know which defragmentation method is used, since it is the only method availabe on WinNT (see above). Opening the file for reading is only required if the file is EFS encrypted. Else plain "read attribute" access is needed, like you wrote. Since reading attributes is even possible from Windows Explorer, the defragmentation program will most likely be able to get the same file access.

@Lunar_Dust:
I found a programm called "File Lock" (yes, plain name ), which is able to lock and/or hide files or directories. I tried it with O&O Defrag, Diskeeper and PerfectDisk.

The O&O Defrag service simply crashes completely when analysing a volume with a locked file (you have to reboot before the GUI can connect to the service again).

PerfectDisk simply ignores the fact that the file ist locked an defragments it, even if the file is also hidden.

Diskeeper does not defrag a locked file, finally. I will test it tonight on our server, tomorrow I can tell if it worked.

@all:
Thank you all for your ideas.