delx.au / blog

HFS+ automatically defragments files as they are used. When a fragmented file is opened, if the system has been up for 3 minutes, the file is not busy and is under 20MiB in size then it will be relocated to be in a contiguous section of the disk

So I was having a discussion with Greg yesterday about file systems and I claimed the above. I have actually made this claim, that OS X automatically defragments files, many times in the past. However this time Greg decided to call me on it. He told me: Citation needed.

I knew I’d read this somewhere, but couldn’t track my original source. So I did a little bit of research and found there wasn’t an easily accessible authoritative source anywhere.

Apple Technote HT1375 was one of the first pages I found. However it only talks about the Hot-File-Adaptive-Clustering, which is a completely different technique.

A little more searching and I found this Ars Technica article, a review of Mac OS X 10.3. This actually contains some solid details on the auto defragmentation feature. The reference for this information is only a newsgroup post on comp.macosx.general, not exactly definitive.

So I decided to go to the source. Apple publishes the source code for large components of Mac OS X in the Darwin Source Code repository. In this case we needed to look at the HFS+ driver, which is part of XNU – the OS X kernel.
Incidentally, thanks go to Apple for making this code available to everybody for random research like this.

Before long I’d narrowed it down to this file: xnu-1228.7.58/bsd/hfs/hfs_vnops.c. The function that we’re interested in is hfs_vnop_open(), which is called whenever the system needs to open a file or directory on disk for reading or writing. The last two comments in that function explain the conditions under which defragmentation occurs. Nicely written and well commented code.

Since you need a free Apple developer account to view the above link, here’s the relevant section of the source code:

/*
 * Open a file/directory.
 */
static int
hfs_vnop_open(struct vnop_open_args *ap)
{
  struct vnode *vp = ap->a_vp;
  struct filefork *fp;
  struct timeval tv;
  int error;

  /*
   * Files marked append-only must be opened for appending.
   */
  if ((VTOC(vp)->c_flags & APPEND) && !vnode_isdir(vp) &&
      (ap->a_mode & (FWRITE | O_APPEND)) == FWRITE)
    return (EPERM);

  if (vnode_isreg(vp) && !UBCINFOEXISTS(vp))
    return (EBUSY);  /* file is in use by the kernel */

  /* Don't allow journal file to be opened externally. */
  if (VTOC(vp)->c_fileid == VTOHFS(vp)->hfs_jnlfileid)
    return (EPERM);
  /*
   * On the first (non-busy) open of a fragmented
   * file attempt to de-frag it (if its less than 20MB).
   */
  if ((VTOHFS(vp)->hfs_flags & HFS_READ_ONLY) ||
      (VTOHFS(vp)->jnl == NULL) ||
#if NAMEDSTREAMS
      !vnode_isreg(vp) || vnode_isinuse(vp, 0) ||
      vnode_isnamedstream(vp)) {
#else
      !vnode_isreg(vp) || vnode_isinuse(vp, 0)) {
#endif
    return (0);
  }

  if ((error = hfs_lock(VTOC(vp), HFS_EXCLUSIVE_LOCK)))
    return (error);
  fp = VTOF(vp);
  if (fp->ff_blocks &&
      fp->ff_extents[7].blockCount != 0 &&
      fp->ff_size <= (20 * 1024 * 1024)) {
    struct timeval now;
    struct cnode *cp = VTOC(vp);
    /* 
     * Wait until system bootup is done (3 min).
     * And don't relocate a file that's been modified
     * within the past minute -- this can lead to
     * system thrashing.
     */
    microuptime(&tv);
    microtime(&now);
    if (tv.tv_sec > (60 * 3) &&
       ((now.tv_sec - cp->c_mtime) > 60)) {
      (void) hfs_relocate(vp, VTOVCB(vp)->nextAllocation + 4096,
                          vfs_context_ucred(ap->a_context),
                          vfs_context_proc(ap->a_context));
    }
  }
  hfs_unlock(VTOC(vp));

  return (0);
}