NAStore External Reference Specification

NAStore RASH
(Rapid Access Storage Hierarchy)
External Reference Specification

Purpose of RASH

The purpose of the Rapid Access Storage Hierarchy (RASH) is to transparently migrate files between user disk space and virtual volumes (VV's) in order to optimally manage disk space within the storage system.

Interfaces to RASH

User Interface

The user sees some modifications to existing commands as well as new commands.

Overall Design

Levels of Storage

RASH mediates between two layers of storage: high speed user disk storage and the VV subsystem. A unique 'Bit-file ID' (bfid) is applied to each file to provide a handle to the archived data.

Archiving

The copying of files from disk to VV is called archiving. The basic steps in archiving are:

  • The arcbuild(1M) program is run as needed. It examines files and outputs a file containing the pathnames of all resident files, which is the input to the next step.

  • The archive(1M) program takes the list of files created by arcbuild and copies each file to VV(s). Several child processes can be used for faster processing. The administrator specifies the number of children that may run in parallel. Each child process will verify that each file is still a candidate for archiving, i.e. it is still present and has not been touched recently (this is done since a large amount of time may have passed since the selection process started.). The file is copied onto VVs (as many as needed for the file size). When the copy is complete, a rash data base entry is created and the inode is marked as having been archived. If the option to release the space is selected, the allocated blocks are released. Otherwise, the blocks are left resident until step four. If the file is modified during the time it was being written to VV(s), the archive is assumed to be bad; it will not be marked as archived in the inode, though the data base entry will be kept.

  • The reclaim(1M) process frees space from disk-resident files marked as archived. If such a file is not opened by any user, the disk blocks assigned to the file are freed and the inode is marked as being non-resident.

    Restoration

    Files are restored to the user disk area automatically by reading or writing data in a file that is not resident. When a restore has been initiated, the inode is marked "restore in progress" and the read(2) or write(2) calls are blocked until the data being accessed is valid (restored). Upon completion of the restore, the "restore in progress" and "non-resident" flags are cleared. If a non-resident file is opened with O-TRUNC, the data is not restored. Instead, the inode is marked as resident and not archived. However, the information in the RASH data base is left alone.

    RASH Database

    Information about each archived file is kept in the RASH data base, which is organized as a collection of "user databases," each containing information on files owned by the user at the time of archive. Each user data base is a directory containing a data file and a B-tree index file. The information in the data file is stored in ASCII format in order to facilitate report generation. The data records contain the following information: 

    
typedef struct usr_fileinfo {   
        int             usr_version;    /* rash version                 */
        bfid_t          usr_bfid;       /* bfid                         */
        int             usr_vvno;       /* multiple volume sequence #   */
        unsigned short  usr_rflg;       /* record flags                 */
        unsigned short  usr_vflg;       /* vv flags                     */

        char            usr_vvname[VVNAMELEN+1];        /* vv name      */
        int             usr_fno;        /* file number in multifile vv  */
        unsigned int    usr_bof;        /* byte offset into the vv      */
        off_t           usr_lseek;      /* lseek write to disk location */
        off_t           usr_vvdata;     /* amnt of data on vv for fno   */
        off_t           usr_fsize;      /* file size                    */
        uid_t           usr_uid;        /* uid                          */
        gid_t           usr_gid;        /* gid                          */
        time_t          usr_ctime;      /* ctime                        */
        time_t          usr_mtime;      /* mtime                        */
        time_t          usr_arctm;      /* arctm                        */
        mode_t          usr_mode;
        time_t          usr_rectm;      /* record timestamp             */
        time_t          usr_deltm;      /* record delete time           */
        int             usr_group_no;   /* rash archive group number    */
        char            *usr_fname;     /* file name pointer            */
} USR_FILEINFO;

    VV Format

    Each VV has a RASH header structure at the beginning. All the named fields are separated by blanks. 

    
typedef struct  {
        char    hdr[4],                 b0;     /* 'RASH'               */
        char    vvname[MAXVVNAMELEN],   b1;     /* this vv              */
        char    version[INTLEN],        b2;     /* rash version         */
        char    dbuid_name[10],         b3;     /* owner when bfid assigned */
        char    dbuid[INTLEN],          b4;     /* dbuid number         */
        char    date[DATELEN],          end;    /* time label written   */
} vv_hdr_lbl_t;

    Each file on a VV has a HDR label before the data starts and a label at the end of data. If a file spans multiple VV's, each VV except the last will have an EOV label following the file segment. The last VV will have a EOF label following the file segment. This is also the case if a file does not span multiple VV's. 

    
typedef struct  {
    /* basic info */
        char    hdr[4],                 b00;    /* 'FILE'                   */
        char    label[3],               b01;    /* 'HDR' 'EOV' or 'EOF'     */
        char    version[INTLEN],        b02;    /* rash version             */
    /* file-vv mapping */
      /* multi-vv files */
        char    vv0[VVNAMELEN],         b03;    /* 1st vv for file          */
        char    vvno[5],                b04;    /* volume order number      */
        char    othervv[VVNAMELEN],     b05;    /* prev or next vv for file */
      /* multi-file vv */
        char    fno[5],                 b06;    /* which file on this vv    */
    /* file info */
        char    bfid[BFIDLEN],          b07;    /* bfid in ascii hex        */
        char    uname[10],              b08;    /* username                 */
        char    uid[INTLEN],            b09;    /* file uid in ascii hex    */
        char    gname[10],              b10;    /* group name               */
        char    gid[INTLEN],            b11;    /* file gid in ascii hex    */
        char    mode[4],                b12;    /* file mode in ascii hex   */
        char    mtime[DATELEN],         b13;    /* mtime                    */
        char    ctime[DATELEN],         b14;    /* ctime                    */
        char    arctm[DATELEN],         b15;    /* archive time             */
        char    fsize[OFFLEN],          b16;    /* file size in ascii hex   */
        char    lseek[OFFLEN],          b17;    /* lseek address hex        */
        char    vvdata[OFFLEN],         b18;    /* amt of file on this vv   */
        char    flen[4],                end;    /* length of file name hex  */
} file_hdr_lbl_t;