Lab 6 Notes

James Calloway

There are fewer steps in this lab than lab5 (roughly half) but each of them will require some serious thinking on your part.
On a higher level you will be implementing 3 system calls in this lab: dup(), dup2(), and pipe(). You will be completely re-doing the IO that you designed from labs 4 and 5.

Step 1

The structure will be as follows. Each process's PCB will have a ``filetable'' field, which is an array of FTSIZE (#defined to be 64 in myjos.h). The elements of the filetable are NULL if the corresponding fd is not opened.

If it is opened, then it contains a pointer to a "vnode" struct. For this I used two structures, one for Vnodes and one for io buffers associated with them. These are defined in jos.h and are as follows:

  typedef struct {
    IO_Buffer *iobuf;
    int num_open;
    int type;
    int mode;
  } Vnode;

    typedef struct {
      char *buf;
      int size;
      int nreaders;
      int nwriters;
      int count;
      int head;
      int tail;
      Cv empty;
      Cv full;
      j_Gsem read_serialize;
      j_Gsem write_serialize;
    } IO_Buffer;

For now, iobuf is a pointer to the console (just point it to the address of your console struct) , num_open is the number of fd's that point to this vnode, type is CONSOLE (#defined in myjos.h as 1), and mode is O_RDONLY or O_WRONLY (from fcntl.h -- read the open man page).

Rewrite your code to have this format. Do the following:

1. Create two vnodes at startup.

   One is for reading from the console, and one is for writing to the console. Just point the iobuf field to your console struct (i did some weird casting here). Take care of this in initialize_global_jos_state When you initialize your first process, have p->filetable[0] point to the console reading vnode, and have p->filetable[1] and p->filetable[2] point to the console writing vnode. Make sure the num_open fields are correct.

2. Change read/write so that they work with this structure.

   Read() and write() now perform error checking according to the vnode (i.e. check that (fd >= 0 && fd < FTSIZE), then check to make sure that p->filetable[fd] != NULL, then check to make sure that p->filetable[fd]->mode is correct). Of course check the buf argument to make sure that it is in the proper range. Then read from the console buffer or write to the console.

3. Change fstat() so that it works with the structure

   Fstat will set the buffer size according to p->filetable[fd]->mode and p->filetable[fd]->type. This will be dependant upon whether you are reading or writing.

4. Fix close()

   Change close() so that it error checks and if p->filetable[fd] != NULL, then it decrements p->filetable[fd]->num_open and then sets p->filetable[fd] to NULL.

5. Fix fork()

   Have fork() copy the p->filetable[] fields of the parent to the child (copy the pointers, don't make copies of the vnodes). Make sure that if p->filetable[i] != NULL, then when you copy it to the child, you increment the num_open field and nwriters and nreaders (the mode field is a good pointer).

Step 2

Remember the io_buffer_struct has the following look:

    typedef struct {
      char *buf;
      int size;
      int nreaders;
      int nwriters;
      int count;
      int head;
      int tail;
      Cv empty; 
      Cv full;
      j_Gsem read_serialize;
      j_Gsem write_serialize;
    } IO_Buffer;

The fields are as follows: buf is the buffer. It has size elements. nreaders will be the number of fd's that point to this io_buffer for reading. nwriters is the number of fd's that point to this io_buffer for writing. Count, head and tail are the same as in the bounded buffer threads examples.

Empty is a cv that readers block on when the buffer is empty. Full is a cv that writers block on when the buffer is full. mon.o has been compiled with libsim.a so you don't have to include your own. Read_serialize and write_serialize are semaphores which start at 1 and ensure that only one thread at a time can read from the buffer, and that only one thread at a time can write to the buffer.

I created two global io_buffers for the console which were initialized in initialize_global_jos_state(). I called them read_vnode and write_vnode. The read_vnode pointed to one io_buffer which had size 256, and the write_vnode pointed to an io_buffer whose size was 1.

The io_buffer buf fields were circular queues which worked much like the one from labs 4 and 5. You'll want to use continuation style programming in both cases. Think about what is going on in these two cases.

Some things should be apparent. First off you should be calling P() on the read and write_serialize routines. Then you should be using console_read and console_write to fill up the buffers. Write your buffer filling routines such that characters that go beyond iobuf->size are discarded. Make sure that you are keeping track of what is in the buffer using the iobuf->count field. If you get used to using the iobuf->count and iobuf->size variables it will make life a lot easier when we tear down and re-build read() and write(). Also keep semantics in mind.

Whereas the name of write_to_console_iobuffer brings writing to mind, in terms of what it is doing to the io_buffer buf field it is actually reading characters from the buffer and outputting them to the screen using console_write()

Try reading and writing from the console now. Test heavily. Try out cat (i.e. lots of characters, control D behavior, etc). Compare it with whatever in ~cs560/bin

Now I basically deleted all of the functions I had for reading and writing. My read() calls in the ISR make a jpt_fork call to io_buffer_read(PCB *p). It reads bytes from v->iobuf into the read buffer (p->regs[7]). Notice that you have all of the fields needed to do this from within your PCB struct namely:

1. The filetable for getting at the Vnode
2. The iobuffer of the vnode (see 1)
3. The size to read (as denoted by p->regs[7])
4. the buffer where we'll place things (p->regs[6])

io_buffer_read performs the following general steps:

• calls P on v->iobuf->read_serialize, making sure that only one process reads from a buffer at a time.
• reads as many characters as are in the buffer, up to p->regs[7]. While it does this, it checks for the -1 character (console EOF) and stops if it sees this character. If you want, you may have it stop after hitting '\n' too.
• calls cv_notify on iobuf->full.
• if the read is not finished (size bytes haven't been read and EOF has not been detected), then it waits on the empty cv and continues after waking up.

I've been sketchy on this description because we you'll have to do some thinking. I.e. you'll have to break this up into several procedures since you're making waiting calls. Make sure that when you wake up from a blocking call, you check to make sure that there are characters to read -- i.e. be safe, and don't assume that just because you woke up, you won't have to block instantly again.

This is a little different than what you had before. You read characters directly from the console before when you wanted to service a read call. When you read a charcter from the iobuf you'll want to place it in the read_buffer. Also, make sure that when you read a character from the buffer that you take care to update the iobuf->count field.

Now, write io_buffer_write(PCB *p)

This writes size bytes starting at buf into the IO_buffer pointed to by v->iobuf. When it's done, it calls syscall_return with the number of characters written (usually size). This should work even if the buffer is smaller than size -- i.e. it will write as many bytes as will fit into the buffer, call cv_wait(full) and then when it wakes up continue writing bytes.

This is also a little different from what you had before. Before, you wrote characters directly to the console. Make sure to update the iobuf->count field. Anyway, write this procedure so that it works -- specifically, you'll have to check to see if the buffer is full, and if so, call cv_wait(v->iobuf->full).

After you write characters to the buffer, you should call cv_notify(v->iobuf->empty) so that any thread that wants to empty the buffer will wake up and do so. As in io_buffer_read(), before you start writing, you should call P() on v->iobuf->write_serialize and V() on it when you're done.

Test all of this using the code in ~cs560/test_execs. It should behave exactly as it did before. Pay special heed to how your code behaves in the face of CONTROL-D's and \n's and the like. You must have the read and write routines working perfectly for your JOS pipes to be able to function.

Step 4

• io_buffer_read()
• and io_buffer_write()

Now, tie up some loose ends. Make sure that your IO_buffer for console reading has its nwriters field set to 1, and that the IO_buffer for console writing has its nreaders field set to 1. Take care of this in initialize_global_jos_state().

Finally, implement close(). If the fd is valid, you should decrement the num_open field of the vnode. If num_open is zero, then no processes have this particular file open, so you should

1. Decrement the nreaders or nwriters field of the iobuf, depending on what v->mode is.
2. If (nreaders == 0 && nwriters == 0) free the iobuf (this will never happen for the console, but it will happen for pipes, so you'll be testing this later).
3. free() the vnode.
4. Set the proper entry in the filetable to null.

Step 5

This should be straightforward -- you will simply make a copy of the pointer to the proper fd and increment the num_open field. And of course do error checking. You'll need to consult the man pages for error conditions. Also, think about what you'll want to do with the nreaders and nwriters fields. Make sure you also deal with what happens if filedes from the dup2() man page refers to an open file descriptor. READ the man page for these two functions. Test this out (write your own C code to test it!).

Step 6

After creating the io_buffer, you create two vnodes, one for reading and one for writing, and have them both point to the io_buffer. You then set the fd in p[0] to point to the reading vnode, and the fd in p[1] to point to the writing vnode. When setting p[0] and p[1] in the user's memory, don't forget to byte-swap.

Remember the semantics of pipe(). The user will pass as an argument to pipe an array of two integers. This address will be in p->regs[5]. You will need to calculate two file descriptors and place them back in user space. Return the two lowest file descriptors to the user, and also make sure that your code returns the appropriate errno value (consult the man page for various pipe error conditions).

Now, read() and write() should simply call io_buffer_read() and io_buffer_write() as before, and it should all work. Think about it and make sure that it makes sense. Test it by calling pipe in one process and having that process read and write from the pipe. I.e. don't try to do pipes from the shell yet. Something like pipe_test.c:

      main()
      {
        int i, j, p[2];
        char s[100];
        char s2[100];
      
        i = pipe(p);
      
        printf("pipe(p) returned %d %d %d\n", i, p[0], p[1]);
      
      
        for (j = 0; j < 5; j++) {
          sprintf(s, "I am a string %d\n", j);
          i = write(p[1], s, strlen(s));
          printf("write to the pipe completed.  i = %d\n", i);
      
          i = read(p[0], s2, 100);
          printf("write from the pipe completed.  i = %d.  s2=%s\n", i, s2);
        }
      
      }

Step 7

Try hw | cat. It should print out the hello world, etc. but chances are the cat will not exit because it will block on reading from the pipe and never unblock once hw is done.

To deal with that, you should do three things. First, in io_buffer_read() before you block, you should check and make sure that nwriters > 0. If not, then if you block, it will be forever. Why? Because there are no processes to fill the buffer. Instead, if (nwriters == 0) return from the system call.

Second, whenever you set nwriters to zero (this will happen sometimes when you close a file or exit a process), if (nreaders > 0), you should call cv_notify(empty). This will wake up any blocked processes who will then return from their read calls.

Third, when a process exits, you should make sure that all of its open fd's get closed. Now try hw | cat again. It should work. Try it again and again. Make sure that your pipe iobufs are getting freed when then two processes die. Try modifying hw so it writes more characters than the pipe buffer can hold. Make sure everything still works then. Make sure that pipe_test still works from both the shell and also when it is run as a.out.

Step 8

  
   
   main()
   {
           int n;
           char ch[81];
   
           while ( (n = read(0, ch, 80)) > 0) {
           ch[n] = '\n';
           write(1, ch, n+1);
         }
        if (n < 0) { perror("cat"); }
   }