In this lecture, we discuss the aim and schedule of the course, and take a brief look at Unix, C programming, and computer architecture.

The course

The aim of this course is to develop the necessary systems programming skills in C and Unix as a foundation to tackle the design, implementation, and integration of a large software project working in small teams. The challenge of the course is to quickly get people up to speed so there is sufficient time to get into the details of a complex software design project. The first part of the course serves to develop design, programming and other systems skills such as source-code management, testing, and debugging. The second part of the course is all about the project and team work. Good team work will lead to success. That’s the message.

A note about Unix/Linux: Unix came first. All of the others, including Solaris, AIX, BSD and all the variants of Linux, are derivations of that original effort. While we use Linux on the Sudikoff servers, you may elect to use your own copy of Linux or the Unix that Mac OSX is based on. So, unless it really matters, I will use the term Unix to refer to them all.

The syllabus in a nutshell:

  • Unix: shell, commands, shell programming;
  • C: structure, arrays, pointers, dynamic memory, files;
  • Programming tools: gcc (GNU compiler), make (maintain groups of programs), gdb (GNU debugger), git (source code management), and valgrind (profiling and memory leak checker).
  • Software development methodology: design, implement, unit test/debug, integration, demo, maintenance.
  • Thematic programming assignments: building a search engine (crawler, indexer, query engine).
  • Team project: it’s a surprise!

The course includes six Unix shell and C programming assignments for the first part of the course; these assignments are to be done individually. The last part (about 2 weeks) is devoted to the team project. There are fewer lectures in the last part of the course but the projects are run with design reviews and progress meetings where the team can brainstorm problems and come up with solutions. There will be a common project goal for all teams, but students are free to develop their own ideas beyond this common goal - show your entrepreneurial side!

Please take note of the reading assigned each week. I will list the reading week by week, rather than lecture by lecture, to give you some flexibility – but please do the reading, you’ll find it to be interesting and valuable!

Logistics

Go over the Logistics information.

About communication: be sure to track Piazza for announcements and updates.

About engagement: you will learn more from this course if you are actively engaged, where engagement = preparation + participation. Read the lecture notes and weekly reading, practice at home, and participate in classroom and Section activities. The Learning Fellows are here to help with classroom activities, and the Section Leaders are here to help outside class.

About the programming: There is a significant amount of programming in this course requiring a considerable time commitment on the part of the student. You will need to be well organized to complete all the programming assignments and project. It will be challenging, but we hope it will be fun!

About the project: You will be assigned to a team of four students and given about two weeks to complete a large project requiring strong collaboration and a problem-solving mindset. Each member is responsible for contributing to the overall system design, implementation, testing, integration, and documentation. The goal of this activity is to help you develop the confidence, skills, and habits necessary to write large computer programs while being part of a multi-person team. You will become conversant in software engineering paradigms, such as source code control with git and other open source tools that ease the software development process. In addition, you will develop vital skills in self-directed learning, problem solving, and communication. The project concludes with a demo and a review of your design and implementation. All members of the team get the same base grade, plus a team-contribution grade determined in part by a short evaluation form completed by all members of each project team.

Goals for today’s lecture

We plan to cover the following in today’s lecture:

  • The concept of a command line;
  • Logging on to a Unix machine;
  • Looking at the home directory and its files;
  • Copying files to/from remote machines;
  • Logging out; and
  • Some housekeeping business.

In the first few lectures we cover Unix, the shell, and shell programming. This is not meant to be a detailed presentation of the Unix OS and its programming tools – it would require a whole term to cover all that material in detail. We need to know enough about Unix and its tools to be able to navigate our way around the system, write some basic shell scripts, and use its programming tools.

It is important that you use these notes as a starting point, but like any budding hacker you need to do some experimenting and read up on the details. You need to go on the web and find information if there are gaps in your knowledge, and then come see the instructor or TAs for help if you’re still stuck. There are many references on the Resources page.

Caveat: Please take note that lecture notes will not always be detailed. You will need to augment these notes with your own comments and by using the references and reading assignments so you can dive deeper into the topic. We will be discussing the reading assignments during each week!

The command line

Unix was originally developed for computers with hardwired ‘terminals’, each of which was basically an electronic typewriter - a printer with a keyboard. There were no graphical displays, and the concepts of ‘windows’ and ‘mouse’ had not yet been invented. To interact with the computer, the user types on the keyboard, and the computer echoes the keystrokes on the paper. The system is controlled by typing ‘commands’, most of which print results for the user to see. The interaction occurs over the ‘command line’.

Modern Unix systems support rich graphical user interfaces, but under the hood they all support the command line. In this class, I will demonstrate everything using the MacOS command line, accessed via the MacOS application called Terminal. If you have a Mac, you can follow along. If you use Windows, consider installing Linux on your laptop (either ‘dual boot’ or ‘virtual machine’). But all of us can (and will) also use the department’s Linux servers.

When I open a new Terminal window on my Mac, I can type Unix commands at the ‘prompt’. This prompt is from the ‘shell’ – a program – that interprets your command line and runs a program representing each command. Your prompt may look different - the shell allows users to customize prompts and you’ll see a few formats in the examples below.

[cs50@tahoe ~]$ ls
Archive/  data/  dotfiles/  public_html/
[cs50@tahoe ~]$ echo Hello CS50
Hello CS50
[cs50@tahoe ~]$  

The ls command is the “list directory contents” command. The echo command is like a print statement - it echoes its arguments back.

Most commands quickly produce some output and then exit. Some will run too long - perhaps printing too much output; you can stop (kill) the command, forcing it to exit, by typing control-C ^C at the keyboard. One silly program, yes, just prints an infinite sequence of y characters until you kill it:

[xia:~] yes
y
y
y
^C
[xia:~]

Some commands ask for your input, and continue to read input until they read an “end of file” (EOF); you can cause the program to detect an EOF by typing control-D ^D at the beginning of an input line. Below I typed two lines of text, the ^D at the start of the third input line:

[cs50@flume ~]$ mail xia@cs
Subject: Hello from cs50
This is a test email
How should we end the input?
EOT
[cs50@flume ~]$ 

Here, when we typed ctrl-D the mail program detected an end of file (EOF) on its input (i.e., the keyboard), then printed EOT and exited. Moments later, the mail message landed in my inbox.

Notice the difference between ^C and ^D; the former kills the program immediately, whereas the latter causes it to detect EOF when it next reads input from the keyboard.

Logging into a remote machine using ssh

The Department of Computer Science operates a set of Linux servers in the basement of Sudikoff, and you should plan to log in to one of those servers for doing the labs. (See the list of recommended servers - a web page accessible only on campus.)

I’ll login in from my Mac using the secure shell (ssh) Unix command. The ssh command establishes a secure channel and uses public-key cryptography to authenticate the remote computer and the user.

The ssh command replaces the archaic telnet (remote communications with another computer) and rlogin (remote login) because they lack security. The ssh command is exclusively used, these days, because your session is encrypted when it’s transmitted over the network, rather than being sent in clear text.

Below, I remotely log in as user cs50 to the server named flume by giving its full IP name - flume.cs.dartmouth.edu.

xia@~> ssh -l cs50 flume.cs.dartmouth.edu
Last login: Tue Jun 21 15:03:52 2017 from 10.31.178.217
[cs50@flume ~]$ ls
Archive/  data/  dotfiles/  public_html/
[cs50@flume ~]$ ls dotfiles/
cs50-home/  fresh-accounts/  MacOS/
[cs50@flume ~]$ logout
Connection to flume.cs.dartmouth.edu closed.
xia@~> 

It is also possible to connect to a remote Linux server using X-windows, an early form of graphical user interface that has the capability to run applications on a remote server but put the windows on your local computer. I won’t be using X in this course, but you may wish to explore the option.

There are hundreds of Unix commands – but you probably only need a few dozen to get by. Each Unix command has a short abbreviated command name (e.g., LiSt directory (ls) or Secure SHell (ssh)) and its associated syntax that typically includes various arguments, and options; typically, these options (or switches as they are also known) are either a single letter preceded by a hyphen (e.g., -l) or one or more words preceded by two hyphens (e.g., --verbose). For example, the format of an ssh command line looks like this:

	ssh [options] [user@] hostname [command]

In ssh -l cs50 flume.cs.dartmouth.edu, the switch -l informs the ssh command that the username of the user logging in is cs50. (Alternately, I could have used the form ssh cs50@flume.cs.dartmouth.edu).

This is a good time to look at the formatting of these command help texts. The [ ] are used to denote optional things, like

[command]

while the things outside of the [ ] , like hostname, must be specified.

Getting Information using the online manual (man)

If you want the detailed syntax of a Unix command you can use the manual command followed by the command, as in man ssh which produced the following:

    SSH(1)                    BSD General Commands Manual                   SSH(1)

    NAME
         ssh -- OpenSSH SSH client (remote login program)

    SYNOPSIS
         ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec] 
             [-D [bind_address:]port] [-e escape_char] [-F configfile] [-I pkcs11] 
             [-i identity_file] [-L [bind_address:]port:host:hostport] [-l login_name]
             [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] 
                [-R [bind_address:]port:host:hostport] [-S ctl_path]
             [-W host:port] [-w local_tun[:remote_tun]] [user@]hostname [command]

    DESCRIPTION
         ssh (SSH client) is a program for logging into a remote machine and for executing
         commands on a remote machine.  It is intended to replace rlogin and rsh, and
         provide secure encrypted communications between two untrusted hosts over an 
         insecure network.  X11 connections and arbitrary TCP ports can also be forwarded 
         over the secure channel.

         ssh connects and logs into the specified hostname (with optional user name).  The 
         user must prove his/her identity to the remote machine using one of several methods 
         depending on the protocol version used (see below).

         If command is specified, it is executed on the remote host instead of a login shell.

         The options are as follows:

         -1      Forces ssh to try protocol version 1 only.

         -2      Forces ssh to try protocol version 2 only.

         -4      Forces ssh to use IPv4 addresses only.

         -6      Forces ssh to use IPv6 addresses only.

         -A      Enables forwarding of the authentication agent connection.  This can also
                 be specified on a per-host

    ... and a whole lot more

This is just a snippet of the man ssh output (man is short for manual). The manual output includes all the nitty gritty details on options and about the command. For most commands you can use the common option --help (two hyphens) to get a brief breakdown of the command and its switches. This doesn’t work for all commands (including ssh, interestingly), but in that case the use of -help is interpreted as an invalid entry by ssh and it lists of the options anyway.

You can use

man -k keyword

to search through the manual pages for matches on a keyword. For example:

[cs50@flume ~]$ man -k shell                                                           
bash (1)             - GNU Bourne-Again SHell
capsh (1)            - capability shell wrapper
chroot (1)           - run command or interactive shell with special root directory
chsh (1)             - change your login shell
CPAN::Admin (3pm)    - A CPAN Shell for CPAN admins
CPAN::Plugin (3pm)   - Base class for CPAN shell extensions
csh (1)              - C shell with file name completion and command line editing
... and much more

Your home directory and its files

Each user has a ‘home directory’. After you have logged in using ssh you are in your home directory - that is, the shell’s notion of your ‘current working directory’ is your home directory.

We can look at our home directory ‘path’ using the pwd (print working directory) command. You can always use the man and info commands to get more information.

[cs50@flume ~]$ pwd
/net/class/cs50
[cs50@flume ~]$ 

The tilde (~) above is shorthand for ‘home’. Let’s take a look at the contents of my home directory (using the -l switch which means long format):

[cs50@flume ~]$ ls 
Archive/  data/  dotfiles/  public_html/
[cs50@flume ~]$ ls -l
total 16
drwxr-x---  8 cs50 cs50 4096 Jun 21 14:58 Archive/
drwxr-xr-x  5 cs50 cs50 4096 Jun 11 06:29 data/
drwxr-xr-x  5 cs50 cs50 4096 Jun 11 06:29 dotfiles/
drwxr-xr-x 10 cs50 cs50 4096 Jun 19 15:15 public_html/
[cs50@flume ~]$ cd cs50
-bash: cd: cs50: No such file or directory
[cs50@flume ~]$ cs cs50
-bash: cs: command not found
[cs50@flume ~]$ cd dotfiles/
[cs50@flume ~/dotfiles]$ ls -l
total 12
drwxr-xr-x 2 cs50 cs50 4096 Mar 29 22:16 cs50-home/
drwxr-xr-x 2 cs50 cs50 4096 Mar 29 11:15 fresh-accounts/
drwxr-xr-x 2 cs50 cs50 4096 Apr 13 14:08 MacOS/
[cs50@flume ~/dotfiles]$ 

Yes, I mistyped two commands in that example: cd cs50 and cs cs50. I wanted you to see what the system says when you make a mistake … just in case you make one yourself sometime. ;-)

The cd command changes the current working directory; I finally typed it right and the shell reported that my working directory is ~/dotfiles. The same command, ls -l, listed that directory the second time.

Files can be plain files, directories, or special files (more later). We can see that each file has file permissions and other data associated with it; for example, the directory public_html:

drwxr-xr-x 10 cs50 cs50 4096 Jun 19 15:15 public_html/

You can traverse directory trees assuming you have the appropriate permission.

Unix supports a number of shells (command line interpreters). If we use the echo command we can look at the environment variable that tells us which shell is running. For this course we will use the bash (Born Again SHell) shell.

Again, the shell is the command processor for Unix systems. One way to find out what shell you’re running, try this:

[cs50@flume ~]$ echo $SHELL
/bin/bash
[cs50@flume ~]$ 

The first parameter to echo substitutes the value of the variable SHELL. More on the bash shell later.

Another useful command for copying files between machines is the scp - secure copy (remote file copy program) command. Below I find, then copy, of the schedule for CS50 from the course website.

xia@~> ssh cs50@flume.cs.dartmouth.edu
Last login: Tue Jun 21 15:11:57 2017 from 10.31.178.217
[cs50@flume ~]$ ls
Archive/  data/  dotfiles/  public_html/
[cs50@flume ~]$ cd public_html/
[cs50@flume ~/public_html]$ ls
Comics/  data@      index.html  Lectures/   Reading/   Resources/
css/     examples/  Labs/   Logistics/  README.md  Schedule.pdf
[cs50@flume ~/public_html]$ logout
Connection to flume.cs.dartmouth.edu closed.
xia@~> scp cs50@flume.cs.dartmouth.edu:~/public_html/Schedule.pdf .
Schedule.pdf                                                            100%   28KB  28.2KB/s   00:00    
xia@~> open Schedule.pdf 

Recall that ~ indicates the home directory, so ~cs50 is the home directory for user cs50. I changed my directory there, then into its public_html, and listed the files there. Now I know the ‘path name’ for the desired file - the directory name followed by slash followed by the filename. The scp command allows me to specify the host and pathname for the source and destination of the copy; here the destination is ., a shorthand for ‘current working directory. Finally, I used the MacOS command open to open the file, which launches Preview to show me the pdf.

Logging out

OK, we are ready to logout from our session on flume.cs.dartmouth.edu.

[cs50@flume ~]$ logout
Connection to flume.cs.dartmouth.edu closed.

Okay, this has been a good start. We have covered a number of important issues that we will revisit in the course.

Housekeeping - things you need to do:

  1. If you haven’t already, do lab assignment 0 today!

  2. Visit Canvas and connect to Piazza to make sure your accounts are set up and linked.

  3. Review all of the class materials on the website (Tentative Schedule, Lectures, Reading, Logistics, Reading, Resources), accessible via Canvas. Pay special attention to the CS Department User’s FAQ.

  4. Before next class, use your CS unix account to go through the above examples: execute all the commands and get a ‘feel’ for the shell.

  5. Choose an editor and become very familiar with it. This is a very important step, as the reading assignment “Learn to use a real editor” says. Your first homework assignment will be released at the next class period, so don’t delay!

One more thing: Remotely accessing CS Unix machines

Note the following, depending on your personal computer’s operating system.

Linux: The systems in the CS department are running GNU/Linux. Your Linux will likely be sufficiently compatible that you won’t have any trouble developing on your machine and delivering your assignments on a CS machine. You may want to try the X-windows connection.

Mac: Mac OSX Unix conforms to the POSIX standard for the C API, shell utilities, and threads and can compile and run your existing code. This is really exciting for Unix/Linux development. The GNU tools we will be using, such as gcc, make, etc., are either included or freely available. (You may need to install command-line tools via Xcode; if so, you will be prompted to do so when you first type gcc or make or related commands.) You can use the Terminal application. You can also use the ssh command to remotely log on to computers, as discussed above.

Windows: There are Unix-command shells for Windows and even an X-Window system for Windows systems. However, given this is a Unix course you would be far better off doing your assignments on the departmental server. The most successful approaches have been to do your editing and documentation on the Windows system and to upload (using sftp, filezilla, etc.) and build/debug on the lab systems via ssh. Many students have adopted the Sublime Text editor and used its handy sftp integration. As an alternative, visit Sudikoff 003/005 and use the MacOS computers there. As another alternative, consider installing Linux on your laptop - either to dual-boot (so you can choose Linux or Windows at boot time) or using Virtual Machine software to run one OS on top of the other.

Historical note

The name “Terminal” is still commonly used in Unix parlance, right down to the MacOS app by that name. This term refers to the fact that early Unix computers, like other computers of its time, had a small number of interactive ‘terminals’ connected to it – each directly connected by a dedicated wire to a card within the computer. Each terminal was little more than a keyboard and a screen - or in the early days, a keyboard and a printer that would print each character as it was typed, and print the characters sent by the computer.

The first such terminals were “teletypes”, adapted from the device used by newsrooms around the country, which printed characters transmitted over the phone line from a distant device. This video shows a Teletype model ASR33. From the sound of this machine, can you tell why television news shows’ opening music usually has a stocatto tap-tap-tap-tap-tap background theme?