Clustered File System

Jonathan Miner
March 22, 2001

Abstract

Hard drives have grown in size, and dropped in price in recent years.  It is not difficult to configure 120GB of space for less than $200.  Consider that most operating systems can be installed in 6GB or less, and that many companies have evolved to a server-centric model where all data is stored on fileservers.  The end result is a large amount of unused disk space.  For example, lets consider a company with 1000 desktops, each with a 40GB internal drive, of which 6GB is used for the operating system and locally installed tools.  This leave 34GB per machine, for a total of 34TB of unused space.  To make matters worse, if users are allowed to store data on the local drives, and the company does not backup local drives, then the worse case is that there is 34TB of data that is NOT backed up!  This number could be larger than the largest single server in the company's data center.  Imagine the possibilities if all the local storage devices could be clustered together.

Goal

The goal of the Cluster File System project is to design, develop and implement a replacement for the existing Network File System (NFS) protocol.  The new architecture will contain an additional layer of abstration which allow local disk space on multiple machines to be grouped together as a single filesystem.  This abstration will also allow for mirroring of files and data migration between servers, with no impact to running clients.

Definitions

Client

A "client" is an end-user workstation that mounts a "cluster filesystem" from a "cluster". The client will run a "cluster client daemon" (ccd) which will provide the interface between user applications and the "cluster servers".  This is similar to the NFS daemon "biod".

Cluster

A "cluster" is defined as a collection of networked machines ("nodes") that have been administratively grouped.  Each cluster supports a single cluster filesystem.  A set of "cluster server daemons" (csd) will maintain a database that maps cluster pathnames to the physical locations of files.

Sub-Cluster

A "sub-cluster" is defined as a collection of networked machines ("nodes") that have been administratively grouped, based on physical location.  This optional grouping is used to managing the mirroring of files.  A file can not be mirrored to another node within the same sub-cluster.

Node

A "node" is a single machine will local disk space that is part of a cluster filesystem. Each node will run a "cluster node daemon" (cnd) which will provide an interface between the "cluster servers" and the local filesystems. This similar to the NFS daemon "nsfd".

Layer Diagram

Standalone
 NFS
 Cluster
application
 application
 application

NFS client (biod)
 Cluster client (ccd)


Cluster server (csd)

NFS server (nfsd)
 Node server (cnd)
filesystem
 filesystem
 filesystem
volume management
 volume management
 volume mangement
physical device
 physical device
 physical device

Operations

Basic

The all basic file and directory system calls will be supported.

Advanced

The following advanced operations will be supported by the cluster architecture:

Project Status

This project is currently in the design phase.  Comments and questions are welcome.  All design and development will be performed on Linux systems, following POSIX standards.  Code will be written in C.

Links