COS 318 - Project 6

Project 6: File System


Overview

In this project you will implement a simple UNIX-like file system with a hierarchical directory structure. Files and directories can grow and shrink, and you need to manage free disk space. Your file system will be able to allow users to browse the directory structure, create and delete new files and directories, etc. The functionality is similar to UNIX file systems, but permission control and user management are not included. Neither do we require concurrency or high performance of your file system. Since this is the final project that you work alone, you would have to do the design and implementation by yourself. You can not discuss the project design/implementation with other students. The TA will be able to help clarify the requirement and explain the concepts.

Yun Zhang will be the TA in charge of this project.

Important Dates

Precept 1: Dec 10, Wed, 20:30-21:30, COS 102.
Precept 2: reading period in Jan, location TBD
Project Due: Jan 13, Tue, 23:59, no late submission will be accepted
No design review, Q/A by appointment only, shoot me emails

Details

We provide you with the kernel we have built so far, including a number of functions to access the disk (see block.h for details), and also several shell command (explained later) to test the file system. You are required to implement most of the standard UNIX file system calls as listed in the following:

1. File system module initialization

Prototype:    void fs_init (void);
Function description:

This function initializes the data structures and resources used by the file system subsystem, and if it detects that the disk is already formatted, it automatically mounts it to the root directory.  It is invoked at kernel initialization time, after USB subsystem has been initialized, but before the block module (block.c) is initialized.  So you need to invoke block_init in fs_init.  Note that by the time fs_init is called, it disk is not necessarily formatted.  As a result, you need to devise a mechanism so that a formatted disk is recognized (need the help of fs_mkfs).

This function is the only non-syscall function you are required to implement.


2. Format a disk

Prototype:    int fs_mkfs (void);
Function description:

This function formats a disk, either a raw disk, or one that is previous formatted, and then mount the newly formatted file system to the root directory.  The size of disk (in blocks) is defined as FS_SIZE at fs.h.(You can increase the size of disk if you would like to in order to support bigger file system.)  We assume there is one and only one disk present in system.  You need to set up a flag (magic number) somewhere in the disk, so that it can be later recognized as a formatted disk (ie: You can still access a formatted disk and get its content after the shell exists and gets restarted).  The function should return 0 on success, and -1 on failure.

Note: The bootblock, kernel image and process images are all outside the file system.  The file system starts at START_SECTOR (defined in block.c),  in the disk and that block is given the block number 0 when using block_read/write to access the disk.


3. Open and possibly create a file

Prototype:    int fs_open (char *filename, int flags);
Function description:

Given a filename, fs_open() returns a file descriptor, a small, non-negative integer for use in the subsequent system calls (fs_read, fs_write, fs_lseek, etc).  The file descriptor returned by a successful call will be the lowest-numbered file descriptor not currently open for the process.

The parameter flags must include one of the following access modes:  FS_O_RDONLY, FS_O_WRONLY, FS_O_RDWR.  These request opening the file read-only, write-only, or read/write respectively.  The constants are defined in the file common.h.  Open return the new file descriptor, or -1 if an error occurred.

If a non-existing file is opened for write, it should be created.  An attempt to open a non-existing file read-only should fail.

To make your life easier, we assume filename passed to the syscalls can only be ".", "..", or a filename in the current directory.  So you don't have to parse the path as "/" separated directory and file names.  You can also assume that the length of the filename (and dirname in the future) will be less than 32 bytes (MAX_FILE_NAME). These assumptions remain the same for the following functions.

You do not need to worry about user management or access control list.

4. Close a file descriptor

Prototype:    int fs_close (int fd);
Function description:

fs_close() closes a file descriptor, so that it no longer refers to any file and may be reused.  It returns zero on success, and -1 on failure.  If the descriptor was the last reference to a file which has been removed using unlink the file is deleted.

5. Read a file

Prototype:    int fs_read (int fd, char *buf, int count);
Function description:

fs_read() attempts to read up to count bytes from file descriptor fd into the buffer starting at buf.  If count is zero, fs_read() returns zero and has no other results.  On success, the number of bytes successfully read is returned (zero indicates end of file), and the file position is advanced by this number.  It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now.  On error, -1 is returned.  In this case it is left unspecified whether the file position changes.

6. Write a file

Prototype:    int fs_write (int fd, char *buf, int count);
Function description:

fs_write() writes up to count bytes to the file referenced by the file descriptor fd from the buffer starting at bufThe file position is advanced as the number of bytes written.  

On success, the number of bytes written are returned (zero indicate nothing was written).  On error, -1 is returned.  If count is zero, 0 will be returned without causing any other effect.


7. Reposition read/write file offset

Prototype:    int fs_lseek (int fd, int offset);
Function description:

The fs_lseek() function repositions the offset of the open file associated with the file descriptor fd to the argument offset.

The fs_lseek() function allows the file offset to be set beyond the end of file (but this does not change the size of the file).  If data is later written at this point, subsequent read of data in the gap (a "hole") return bytes ('\0') until data is actually written into the gap.

Upon successful completion, fs_lseek() returns the resulting offset location as measured in bytes from the beginning of the file.  Otherwise, a value of -1 is returned.

8. Create a directory

Prototype:    int fs_mkdir (char *dirname);
Function description:

fs_mkdir() attempts to create a directory named dirname.  It returns zero on success, or -1 if an error occurred.  fs_mkdir() should fail if the directory dirname already exists.

9. Delete a directory

Prototype:    int fs_rmdir (char *dirname);
Function description:

fs_rmdir() deletes a directory, which must be empty.  On success, zero is returned; on error, -1 is returned (eg, attempt to deleting a non-empty directory).

10. Change the current directory

Prototype:    int fs_chdir (char *dirname);
Function description:

fs_chdir() changes the current directory to that specified in dirname.  On success, zero is returned.  On error, -1 is returned.

11. Make a new name for a file

Prototype:    int fs_link (char *oldpath, char *newpath);
Function description:

fs_link() creates a new link (also known as a hard link) to an existing file oldpath.  If newpath exits it will not be overwritten.  The new name may be used exactly as the old one for any operation; both names refer to the same file and it is impossible to tell which name was the `original'.

On success, zero is returned.  On error, -1 is returned.

Note because we excluded the usage of paths, oldpath and newpath are actually both filenames and can only be in the same directory.

12. Delete a file

Prototype:    int fs_unlink (char *filename);
Function description:

fs_unlink() deletes a name from the file system.  If that name was the last link to a file and no process has the file open, the file is deleted and the space it was using is made available for reuse.

If the name was the last link to a file but any process still have the file open the file will remain in existence until the last file descriptor referring to it is closed.

On success, zero is returned.  On error, -1 is returned.

13. Get file/directory status.

Prototype:    int fs_stat (char *filename, fileStat *buf);
Function description:

fs_stat() returns information about a file.  It returns a fileStat structure (defined in common.h), which contains the following fields:

typedef struct {
    int inodeNo;        /* the file i-node number */
    short type;            /* the file i-node type, DIRECTORY, FILE_TYPE (there's another value FREE_INODE which never appears here */
    char links;            /* number of links to the i-node */
    int size;            /* file size in bytes */
    int numBlocks;    /* number of blocks used by the file */
} fileStat;

Note: if your implementation will need different kind of fileStat structure, you can modify this fileStat and the corresponding code to reflect your change. Please try to keep some essential fields like file size, type so that our testing script will be able to work with your finished code.

Shell Commands

Below is a list of shell commands that you need to support in your file system. We will test your system using the following commands. You may add some more for your own purpose. But dont change the format of the given commands.

Shell commands

Arguments

Description

mkfs

 

Make a new file system, i.e., format the disk so that it is ready for other file system operations..

open

<filename> <flag>

Open a file with the given <flag>, return a file descriptor (fd) associated with this file.

<flag>: 1: FS_O_RDONLY; 2: FS_O_WRONLY; 3: FS_O_RDWR

The flag is same as the flag used by the open system call. The current file offset will be 0 when the file is opened.

read

<fd> <size>

Read <size> bytes from the file associated with <fd>, from current file offset. The current file offset will move forward <size> bytes after read.

write

<fd> <string>

Write <string> into file associated with <fd>, from current file offset. The current file offset will move forward <size> bytes after write.

lseek

<fd> <offset>

Move the current file offset associated with <fd> to a new file offset at <offset>. The <offset> means the number of bytes from the beginning of the file.

close

<fd>

Close the file associated with <fd>.

mkdir

<dirname>

Create a sub-directory <dirname> under the current directory.

rmdir

<dirname>

Remove the sub-directory <dirname>.

chdir

<dirname>

Change the current directory to <dirname>.

link

<src> <dest>

Create a link named <dest> to an existing file or directory named <src>.

unlink

<name>

Remove a link to the name. (When link count drop to 0, delete the file or directory).

stat

<name>

Show the status of the file or directory with name <name>. It should display its inode information; whether it is a file or directory; its link count; size of the file/directory; number of blocks allocated; other information stored about this file/directory.

ls

 

Show the content of the current directory.

cat

<filename>

Show the content of the file.

create

<filename> <size>

Create a file with <filename> as the name in the current directory, and fill it with <size> amount of data, then close the file. (For testing purpose)

flush

 

Flush the dirty cache of the file system to the disk. (Only needed if you are implementing the extra credit for file system cache, you would need to find a way to add this shell command into the shell.)

Extra Credit (2 points)

Implement file caching. You need to cache data blocks of files. In specific: 1. Define the data structure of caching system 2. Write a system call that when file access is invoked, OS goes to cache to look for data blocks first. This system call handles cache miss (like going to the disk for file) 3. Implement cache replacement policy LRU 4. Implement a cache flush system call fc_flush(), to flush the file cache Make sure the file system is consistent. Set a easy turn on/off switch in either your Makefile or your source code. When the extra credit is turned off, your program should still work correctly.

Design Document

Design document must be submitted with your code together. I will take pdf or plain text document. Other format such as rtf, doc, docx are strongly not recommended.

Testing

Submission

Submit to blackboard as you do in the previous projects. Please zip/tar all the files in your project into one zipped package, then submit the package. You should also submit one or more test cases that you think is diffcult along with your program. We will run a tournament - run all students' program against all the others' submitted test cases. If your test case is proved to be diffcult for others but you can solve it, you gain an advantage.

Grading Criteria