chapter 04 -- traditional internet applications

(rev. 02/19/2017)

Notes On Chapter Four -- Traditional Internet Applications

4.1 Introduction
- Concept of a transfer protocol
- Transfer protocols of applications
4.2 Application-Layer Protocols
- A communication protocol includes rules about
  - syntax to be used in messages exchanged
  - meaning of messages to be exchanged
  - which process initiates communication
  - how possible errors are to be handled
  - when each process should terminate communication
- If there is an expectation that people will create many server and/or client programs that use the protocol, then the rules of the protocol must be documented and used as a standard by all the programmers who write the server or client programs.
- If this is not done then there is a very good chance that some of the clients and servers will not be able to communicate - due to some, perhaps slight, difference in the way that they try to conduct their communication.
4.3 Representation and Transfer
- When a protocol is complex, it is very common that there are separate documents describing
  - the required formats for representing data objects like numbers, characters and files, and
  - the rules regarding how data is transferred, e.g. how clients and servers interact, message syntax and semantics, and so forth.
4.4 Web Protocols
- Three key standards associated with the world wide web:
  - HyperText Markup Language (HTML) - spec for content and layout of web pages
  - Uniform Resource Locator (URL) - specs format and meaning of web page identifiers
  - HyperText Transfer Protocol (HTTP) - specs how data is transferred between browser (client) and server.
4.5 Document Representation with HTML
- An HTML document is a plain text file
- However HTML can specify web pages that contain multimedia
- HTML documents tell WHAT must be done, not HOW it will be done.
- HTML gives guidelines for display, not specifics.
- This gives the browser flexibility to vary the display according to the capabilities of the underlying hardware.
- The general form of an HTML document is illustrated by this example:
```
<HTML>

  <HEAD>
      <TITLE>
         text that forms the document title
      </TITLE>
  </HEAD>

  <BODY>
     body of the document appears here.
  </BODY>

</HTML> 
```
- There is no provision to include images directly in an HTML document. Instead the author places a tag like:
```
 <IMG ALIGN=RIGHT SRC="./becoat.gif"> 
```
  into the HTML document. When the browser encounters such a tag it fetches the image from the location specified by the pathname in the tag and places a copy of the image within the rendered document (what the browser displays in its window). The position of the image in the rendered document corresponds to the position of the tag in the HTML document.
4.6 Uniform Resource Locators and Hyperlinks
- The user of a browser specifies the resource s/he wants with a Uniform Resource Locator (URL). The syntax is:
```
 protocol://computer_name:port/document_name?parameters 
```
- :port, document_name and ?parameters are optional. The default for document_name is index.html
- Some popular protocols used in browsers are:
  - HyperText Transfer Protocol (HTTP),
  - Hypertext Transfer Protocol over Secure Socket Layer (HTTPS), and
  - File Transfer Protocol (FTP)
  Example URL's:
```
http://www.lycos.com/
https://duckduckgo.com/
ftp://ftp.mirror.nl/
http://www.cs.csustan.edu:80/~john/js.html
```
- An HTML document can also specify a link to another document using an anchor. Examples:
- If you put this text in your web page:
  
  <A HREF="http://www.nps.gov/glba/"> Visit beautiful Glacier Bay</A>.
  
  It will display as below. The URL http://www.nps.gov/glba/ is the URL to which the browser will link when you click on the selectable text: "Visit beautiful Glacier Bay."
- Visit beautiful Glacier Bay.
4.7 Web Document Transfer with HTTP
- Web browsers and servers communicate according to the HyperText Transfer Protocol (HTTP)
- The four basic operations available to HTTP clients are
  - GET (to get status information and an item (e.g. an HTML document),
  - HEAD (to get only status information about an item),
  - POST (to add to an item stored on the server -- e.g. a list of messages), and
  - PUT (to replace an item stored on the server).
- GET's are the most common:
```
GET /item version CRLF
```
- The server responds by sending a header, blank line and the requested document (/item).
- Example: GET /page.html HTTP/1.0 CRLF
- /page.html represents the name of the file, HTTP\1.0 is the version of HTTP that the client 'wants to use', and CRLF designates the two-character ASCII sequence carriage return, line feed.
- General form of basic response header from a web server:
```
HTTP/1.0 status_code status_string CRLF
Server: server_identification CRLF
Last-Modified: date_document_was_changed CRLF
Content-Length: datasize CRLF
Content-Type: document_type CRLF
CRLF
```
- status_code = numeric code indicating whether request could be handled & status_string = textual explanation of status_code (e.g. 200 OK, 400 Bad Request, or 404 Not Found)
- server_identification = human readable description of the server
- data_size = size in bytes of the data to follow
- document_type = what kind of document and what format used (e.g. text/html or image/jpeg)
- Example: Apache web server returns nine lines of header, a blank line, and contents of a 16-byte file:
```
HTTP/1.1 200 OK
Date: Sat, 15 Mar 2008 07:35:25 GMT
Server: Apache/1.3.37 (Unix)
Last-Modified: Tue, 1 Jan 2008 12:03:37 GMT
ETag: "78595-81-3883bbe9"
Accept-Ranges: bytes
Content-Length: 16
Connection: close
Content-Type: text/plain

This is a test.
```
4.8 Caching in Browsers
- Web browsers typically cache what they download from servers to avoid repeating downloads unnecessarily.
- For example, image files can be large, and caching them provides an important benefit.
- When a server sends a document to a client, it can add a cache timeout value or specify that an item should not be cached.
- A client can use a HEAD request to find out if a cached item is still valid.
4.9 Browser Architecture
- Web browsers are more complicated than web servers. Parts of a web browser include:
  - A controller to manage a GUI and call other modules (clients and interpreters)
  - HTML interpreter to render HTML documents
  - other interpreters (e.g. for various image formats)
  - drivers to be used by interpreters to create output
  - HTTP client to make requests from servers - using HTTP protocol
  - other clients (e.g. e-mail, FTP, HTTPS) for using other client protocols,
  - network interface for clients to use for communication
- Importantly, interpreters have to keep track of the locations of the selectable items in the display.
4.10 File Transfer Protocol (FTP)
- In the early days of networking it quickly became apparent that there was a need for general file transfer software - to transfer various types of files between various types of computers.
- Complications:
  - conventions for naming files vary.
  - file formats, protection methods, and data representation vary.
- FTP:
  - deals well with differences between computing systems,
  - transfers files in both directions
  - permits transfer of arbitrary data, and
  - deals well with file permissions and ownership.
  - allows browsing through file systems
  - utilizes plain text control messages
- FTP has been around longer than IP or TCP.
- Users are often not aware of it but FTP is used for almost all the downloading done with web browsers and other network applications.
4.11 FTP Communication Paradigm
- The client does not just pass all the characters typed by the user to the server.
- The client interprets what the user types and sends commands to the server (when necessary) that achieve the result desired by the user.
- The client and server communicate using the language of FTP -- the protocol.
- For each file transfer the client and the server form a separate connection -- distinct from the original control connection.
- The control connection persists for the duration of a session (what happens between an open and a close) but each data connection is terminated after transferring one file.
- The FTP server on the remote host acts as a client when it opens a data connection back to the local host.
- This paradigm is not compatible with the network address translation "routers" used in home networking. The workaround is for the NAT router to rewrite values in the PORT commands that FTP clients send to FTP servers.
4.12 Electronic Mail
- The typical old style e-mail transmission goes like this:
  - User employs an email interface program to compose a message and place it in an outgoing queue
  - The e-mail transfer program (mail transfer agent, MTA) running on the user's computer finds the message in the queue, opens a connection to the mail server on the destination machine and sends the message.
  - The mail server on the destination machine places the message into the intended recipient's mail box
  - The recipient uses an email interface program to view the contents of her mailbox
  - The categories of protocol standards used for Internet email:
4.13 The Simple Mail Transfer Protocol (SMTP)
- "Simple" is in the name because there was a complex predecessor that SMTP replaced.
- SMTP is the (TCP/IP) protocol used in client-server mode by e-mail transfer software.
- SMTP handles reliable delivery details.
- SMTP allows sender to ask the receiving host if the intended recipient mailbox exists.
- SMTP follows a stream paradigm
- SMTP control messages are plain text
- SMTP is only for transfer of plain text messages
- If a user on host X sends a message to three users on host Y then the e-mail transfer process only transmits one copy of the message from X to Y. On Y the e-mail transmission software puts a copy of the message in each of the three recipients mailboxes.
- It would be a waste of network bandwidth to send three copies of the message from X to Y.

4.14 ISPs, Mail Servers, and Mail Access

The typical ISP runs email server and email transfer software on its computers, and provides email transfer service for subscribers
ISP computers provide email mailboxes for ISP subscribers
Subscribers access their email using a web interface provided by the ISP or a special-purpose email interface application.
The advantage of the web interface is that one can read mail using a browser on any computer connected to the Internet
The advantage of the special-purpose application is that users with laptops can download all their email and interact with it while offline.

4.15 Mail Access Protocols (POP3, IMAP)

POP3 and IMAP are protocols for providing an interface to email
Typically an ISP runs POP3 and/or IMAP server software, and subscribers run clients
The software is designed to allow a single user to access a single mailbox
The access allows viewing mail headers, downloading messages, deleting email, and sending email.
POP3 is essentially just a protocol that allows you to download a copy of your mail from your mail host to your personal computer. POP3 asks you for the password on the mail host -- to insure that only you have access to your private e-mail.
POP3 does not work well if you want to read your e-mail on more than one host, or if you want to organize your e-mail into folders while working on more than one computer. The basic problem is that POP3 does not let you organize the mail on the server into folders. All you can do is download the newest mail in the mailbox. If you read mail on more than one client machine then you have to organize the same messages on each client machine separately.
IMAP is similar except that it can keep the primary copies of your messages on the mail host and allow you to organize your mail into sets of folders on the mail host. That is very handy if you want to read your mail from several different personal computers, workstations, and what-have-you. No matter what machine you use to read your mail, you are always working with the same copy of each message and the same set of organized folders.

4.16 Email Representation Standards (RFC2822, MIME)

RFC 2822 specifies the mail message format. A message is a plain text file consisting of:
- header
- blank line
- body
Header lines have the form: keyword: information
Keywords include: From, To, Subject, CC
Arbitrary keywords starting with X can be used - never considered bad syntax
MIME is the Multi-purpose Internet Mail Extension.
Internet e-mail was designed only to transmit printable ASCII characters (plain text).
Nowadays e-mail software encodes arbitrary kinds of data as sequences of printable ASCII characters and transmits it.
The sender puts lines such as:
```
MIME-Version: 1.0
Content-Type: Multipart/Mixed; Boundary=Mime_separator
```
in the header to tell the receiver that MIME conventions are used in the message and that the message consists of different parts separated by the string "Mime_separator."
The sender is free to choose any string for a separator. (Of course one must choose something that is not used elsewhere in the body of the message.)
The sender can put directives in the message that tell the recipient how to decode each part of the message.
Using MIME we can send attachments such as a photograph, word processor document, or spreadsheet.
MIME is very flexible: one can add new message types and encoding schemes without any need to change the software that transmits the mail. Everything that is transmitted is encoded as printable ASCII so old e-mail software has no problems -- the MIME message is transmitted exactly the same way as "old-fashioned" plain text.
(The glitch with MIME is that often senders and receivers do not use the same e-mail software and so conventions understood by one "end" may not be understood by the other "end." When in doubt just send plain text -- and not as an "attachment." It may not be "fancy" but it usually works because all e-mail software is supposed to understand plain text messages.)

4.17 Domain Name System (DNS)

Domain name service (DNS) is provided over the Internet
There are several forms of service but mainly DNS translates hostnames like cs.csustan.edu into an IP addresses like 130.17.70.80.
Most applications that use Internet services utilize DNS.
DNS is a distributed database service - there is a hierarchical network of servers in locations all over the Internet, each of which is responsible for answering questions about a portion of the database.
DNS can work in a recursive mode: if a client asks a server for information that it doesn't have, then the server may ask another server, obtain the answer, and pass it along to the client.
A DNS server has to know the address of at least one root server
Top level domains names like .edu and .com are controlled by the Internet Corporation for Assigned Names and Numbers (ICANN)
ICANN authorizes domain registrars, to which organizations must apply for approval of the domain names that they request.
Once an organization has been approved to use a domain name like csustan.edu, it can create a naming hierarchy for the hosts in its network -- with names like dijkstra.cs.csustan.edu and muon.physics.csustan.edu.
Except for the rightmost two, the segments of a domain name need not correspond to domains, sub-domains, or networks.

4.18 Domain Names that Begin with a Service Name

The domain name of a host is independent of what servers it runs. For example a host that runs a web server is not required to have a domain name that contains "www." Conversely if a host has a name that contains "www," it is not required to run a web server.

4.19 The DNS Hierarchy and Server Model

An organization runs a server that applies to its part of the hierarchy - thus controls what names are used within it.
For example CSU Stanislaus operates DNS servers that perform translations of hostnames that end with "csustan.edu"
- ns1.csustan.edu
- ns2.csustan.edu
Server databases contain information that show links to servers for adjacent portions of the database hierarchy.
There are often redundant replicates of servers, which must be kept consistent.

4.20 Name Resolution

Defined as translation of a domain name to an address
Resolver: client software that performs name resolution by making a request to a domain name server
Resolvers are typically configured to know the address of one or more local domain name servers
A resolver may choose to use either stream or datagram service

4.21 Caching in DNS Servers

Resolvers tend to look up names of 'local' computers.
That's why the practice of the DNS system is to put translation information for local computers in local DNS servers.
And that's why resolvers are configured to send DNS queries to local DNS server.
If the local DNS server has an authoritative answer, it will return that answer to the resolver.
If not, typically the local DNS server will act temporarily as a client.
If it knows how to reach a DNS server that is authoritative for the domain in question, it will ask that server.
Otherwise, it will ask a root DNS server.
When the local server gets the answer, it will return the answer to the resolver that made the query.
As an optimization DNS servers cache the information they get from other servers, in case they get the same query again soon.
Along with the translation information, servers send a cache timeout value - so receiving servers know when to delete the information they cache.
Resolvers cache DNS query results too.

4.22 Types of DNS Entries

DNS database entries consist of a domain name, type, and value.
Queries specify a type, like A for IPv4, AAAA for IPv6, and MX for Mail eXchanger.
DNS servers return only entries that match the query type.
It is desirable that DNS not always return the same IP address for a given domain name. Suppose administrators want all mail for desktop hosts at some site to go to one particular powerful "mail hub" computer at the site.
If a DNS query specifies the name of one of the desktop machines and type MX, the DNS server can respond with the IP address of the mail hub.
If the query specifies type A or AAAA, the DNS server can give an IP address for the desktop machine named in the query.

4.23 Aliases and CNAME Resource Records

and entry like this:
```
www CNAME timmy
```
in the DNS database file for foobar.com allows the host name www.foobar.com to be used as an alias for the hostname timmy.foobar.com.
We can move the web server to jimmy.foobar.com, and change the DNS entry to:
```
www CNAME jimmy
```
and no one using URL's containing www.foobar.com has to change anything.

4.24 Abbreviations and the DNS

DNS will only look up fully qualified domain names
resolvers may try out different ways to complete abbreviated domain names -- e.g. "search domains"

4.25 Internationalized Domain Names

The International Domain Names in Applications (IDNA) standard specifies how to translate arbitrary domain names into a string of ASCII characters.
DNS databases contain only ASCII versions of domain names.

4.26 Extensible Representations (XML)

An extensible protocol or standard like XML allows users to specify new formats for data.