- Concept of a transfer protocol
- How applications implement protocols
- Standard Internet protocols and the protocols they use
- A communication protocol includes rules about
- syntax to be used in messages exchanged
- meaning of messages to be exchanged
- which process initiates communication
- how possible errors are to be handled
- when each process should terminate communication
- If there is an expectation that people will create many server and/or client programs that use the protocol, then the rules of the protocol must be documented and used as a standard by all the programmers who write the server or client programs.
- If this is not done then there is a very good chance that some of the clients and servers will not be able to communicate - due to some, perhaps slight, difference in the way that they try to conduct their communication.
- When a protocol is complex, it is very common that there are
separate documents describing
- the required formats for representing data objects like numbers, characters and files, and
- the rules regarding how data is transferred, e.g. how clients and servers interact, message syntax and semantics, and so forth.
- Three key standards associated with the world wide web:
- HyperText Markup Language (HTML) - spec for content and layout of web pages
- Uniform Resource Locator (URL) - specs format and meaning of web page identifiers
- HyperText Transfer Protocol (HTTP) - specs how data is transferred between browser (client) and server.
- An HTML document is a plain text file
- However HTML can specify web pages that contain multimedia
- HTML documents tell WHAT must be done, not HOW it will be done.
- HTML gives guidelines for display, not specifics.
- This gives the browser flexibility to vary the display according to the capabilities of the underlying hardware.
- The general form of an HTML document is
illustrated by this example:
<HTML> <HEAD> <TITLE> text that forms the document title </TITLE> </HEAD> <BODY> body of the document appears here. </BODY> </HTML> - There is no provision to include images directly in an HTML
document. Instead the author places a tag
like:
<IMG ALIGN=RIGHT SRC="./becoat.gif">
into the HTML document. When the browser encounters such a tag it fetches the image from the location specified by the pathname in the tag and places a copy of the image
within the rendered document (what the browser displays in
its window). The position of the image in the rendered document
corresponds to the position of the tag in the HTML document.
- The user of a browser specifies the
resource s/he wants with a Uniform Resource Locator (URL).
The syntax is:
protocol://computer_name:port/document_name%parameters
- :port, document_name and %parameters are optional. The default for
document_name is index.html
- Some popular protocols used in browsers
are:
- HyperText Transfer Protocol (HTTP),
- Hypertext Transfer Protocol over Secure Socket Layer (HTTPS), and
- File Transfer Protocol (FTP)
Example URL's:http://www.indeed.com/ https://smallbusiness.commercebank.com/cbi/login.aspx ftp://www.cs.csustan.edu/pub/fd3 http://www.cs.csustan.edu:80/~john/js.html
- An HTML document can also specify a link to
another document using an anchor. Examples:
- If you put this text in your web page:
<A HREF="http://www.nps.gov/glba/"> Visit beautiful Glacier Bay</A>.
It will display as below. The URL http://www.nps.gov/glba/ is the URL to which the browser will link when you click on the selectable text: "Visit beautiful Glacier Bay."
- Visit beautiful Glacier Bay.
- Web browsers and servers communicate according
to the HyperText Transfer Protocol
(HTTP)
- The four basic operations available to
HTTP clients are
- GET (to get status information and an item (e.g. an HTML document),
- HEAD (to get only status information about an item),
- POST (to add to an item stored on the server -- e.g. a list of messages), and
- PUT (to replace an item stored on the server).
- GET's are the most common:
GET /item version CRLF
- The server responds by sending a header, blank line and the requested document (/item).
- Example: GET /page.html HTTP/1.0 CRLF
- /page.html represents the name of the file, HTTP\1.0 is the version of HTTP that the client 'wants to use', and CRLF designates the two-character ASCII sequence carriage return, line feed.
- General form of basic response header from a web server:
HTTP/1.0 status_code status_string CRLF Server: server_identification CRLF Last-Modified: date_document_was_changed CRLF Content-Length: datasize CRLF Content-Type: document_type CRLF CRLF
- status_code = numeric code indicating whether request could be handled & status_string = textual explanation of status_code (e.g. 200 OK, 400 Bad Request, or 404 Not Found)
- server_identification = human readable description of the server
- data_size = size in bytes of the data to follow
- document_type = what kind of document and what format used (e.g. text/html or image/jpeg)
- Example: Apache web server returns nine lines of header, a blank
line, and contents of a 16-byte file:
HTTP/1.1 200 OK Date: Sat, 15 Mar 2008 07:35:25 GMT Server: Apache/1.3.37 (Unix) Last-Modified: Tue, 1 Jan 2008 12:03:37 GMT ETag: "78595-81-3883bbe9" Accept-Ranges: bytes Content-Length: 16 Connection: close Content-Type: text/plain This is a test.
- Web browsers typically cache what they download
from servers to avoid repeating downloads unnecessarily.
- For example, image files can be large, and caching them provides an important benefit.
- When a server sends a document to a client, it can add a cache
timeout value or specify that an item should not be cached.
- A client can use a HEAD request to find out if a cached item is still valid.
- Web browsers are more complicated than
web servers. Parts of a web browser include:
-
- A controller to manage a GUI and call other modules (clients and interpreters)
- HTML interpreter to render HTML
documents
- other interpreters
(e.g. for various image formats) - drivers to be used by interpreters to create output
- HTTP client to make requests from servers - using HTTP protocol
- other clients (e.g. e-mail, FTP, HTTPS) for using other client protocols,
- network interface for clients to use for communication
-
- Importantly, interpreters have to keep track of the locations of the selectable items in the display.
- In the early days of networking it quickly became apparent that
there was a need for general file transfer
software - to transfer various types of files between
various types of computers.
- Complications:
- conventions for naming files vary.
- file formats, protection methods, and data representation vary.
- FTP:
- deals well with differences between
computing systems,
- transfers files in both directions
- permits transfer of arbitrary data, and
- deals well with file permissions and ownership.
- allows browsing through file systems
- utilizes plain text control messages
- deals well with differences between
computing systems,
- FTP has been around longer than IP or TCP.
- Users are often not aware of it but FTP is used for almost all the downloading done with web browsers and other network applications.
- The client does not just pass all the characters typed by the user to the server.
- The client interprets what the user types and sends commands to the server (when necessary) that achieve the result desired by the user.
- The client and server communicate using the language of FTP -- the protocol.
- For each file transfer the client and the server form a separate connection -- distinct from the original control connection.
- The control connection persists for the duration of a session (what happens between an open and a close) but each data connection is terminated after transferring one file.
- The FTP server on the remote host acts as a client when it opens a data connection back to the local host.
- This paradigm is not compatible with the network address translation "routers" used in home networking. The workaround is for the NAT router to rewrite values in the PORT commands that FTP clients send to FTP servers.
- The typical old style e-mail transmission goes like this:
- User employs an email interface program to compose a message and place it in an outgoing queue
- The e-mail transfer program (mail transfer agent, MTA) running on the user's computer finds the message in the queue, opens a connection to the mail server on the destination machine and sends the message.
- The mail server places the message into the intended recipient's mail box
- The recipient uses an email interface program to view the contents of her mailbox
- SMTP is the (TCP/IP) protocol used in client-server mode by e-mail transfer software.
- SMTP handles reliable delivery details.
- SMTP allows sender to ask the receiving host if the intended recipient mailbox exists.
- SMTP follows a stream paradigm
- SMTP control messages are plain text
- SMTP is only for transfer of text
- If a user on host X sends a message to three users on host Y then the e-mail transfer process only transmits one copy of the message from X to Y. On Y the e-mail transmission software puts a copy of the message in each of the three recipients mailboxes.
- It would be a waste of network bandwidth to send three copies of the message from X to Y.
- The typical ISP runs email server and transfer software on their computers and provides email transfer service for subscribers
- Also subscriber mailboxes are provided on the ISP computers.
- Subscribers access their e-mail using a web interface provided by the ISP or a special-purpose email interface application.
- The advantage of the web interface is that one can read mail using a browser on any computer connected to the Internet
- The advantage of the special-purpose application is that users with laptops can download all their email and interact with it while offline.
- POP3 is essentially just a protocol that allows
you to download a copy of your mail from your mail host to your
personal computer. POP3 asks you for the password on the
mailhost -- to insure that only you have access to your private
e-mail.
- POP3 does not work well if you want to read your
e-mail on more than one host, or if you want to organize your e-mail
into folders while working on more than one computer. The
basic problem is that POP3 does not let you organize the mail on the
server into folders. All you can do is download the newest mail in
the mailbox. If you read mail on more than one client machine then
you have to organize the same messages on each client machine
separately.
- IMAP is similar except that it can keep
the primary copies of your messages on the mail host and allow you to organize your mail into sets of folders
on the mailhost. That is very handy if you want to read
your mail from several different personal computers, workstations,
and what-have-you. No matter what machine you
use to read your mail, you are always working with the same copy of
each message and the same set of organized folders.
- RFC 2822 specifies the mail message format. A message is a plain
text file consisting of:
- header
- blank line
- body
- Header lines have the form: keyword: information
- Keywords include: From, To, Subject, CC
- Arbitrary keywords starting with X can be used - never considered bad syntax
- MIME is the Multi-purpose Inernet Mail Extension.
- Internet e-mail was designed only to transmit printable ASCII characters (plain text).
- Nowadays e-mail software encodes arbitrary kinds of data as sequences of printable ASCII characters and transmits it.
- The sender puts lines such as:
MIME-Version: 1.0 Content-Type: Multipart/Mixed; Boundary=Mime_separator
in the header to tell the sender that MIME conventions are used in the message and that the message consists of different parts separated by the string "Mime_separator." - The sender is free to choose any string for a separator. (Of course one must choose something that is not used elsewhere in the body of the message.)
- The sender can put directives in the message that tell the recipient how to decode each part of the message.
- Using MIME we can send attachments such as a photograph, word processor document, or spreadsheet.
- MIME is very flexible: one can add new message types and encoding schemes without any need to change the software that transmits the mail. Everything that is transmitted is encoded as printable ASCII so old e-mail software has no problems -- the MIME message is transmitted exactly the same way as "old-fashioned" plain text.
- (The glitch with MIME is that often senders and receivers do not use the same e-mail software and so conventions understood by one "end" may not be understood by the other "end." When in doubt just send plain text -- and not as an "attachment." It may not be "fancy" but it usually works because all e-mail software is supposed to understand plain text messages.)
- Domain name service (DNS) is provided over the Internet
- There are several forms of service but mainly DNS translates hostnames like cs.csustan.edu into an IP addresses like 130.17.70.80.
- Most applications that use Internet services utilize DNS.
- DNS is a distributed database service - there is a hierarchical network of servers in locations all over the Internet, each of which is responsible for answering questions about a portion of the database.
- DNS can work in a recursive mode: if a client asks a server for information that it doesn't have, then the server may ask another server, obtain the answer, and pass it along to the client.
- A DNS server has to know the address of at least one root server
- Top level domains names like .edu and .com are controlled by the Internet Corporation for Assigned Names and Numbers (ICANN)
- ICANN authorizes domain registrars, to which organizations must apply for approval of the domain names that they request.
- Once an organization has been approved to use a domain name like csustan.edu, it can create a naming hierarchy for the hosts in its network -- with names like dijkstra.cs.csustan.edu and muon.physics.csustan.edu.
- Except for the rightmost two, the segments of a domain name need not correspond to domains, sub-domains, or networks.
- name is independent of function as web server
- An organization runs a server that applies to its part of the hierarchy - thus controls what names are used within it.
- For example CSU Stanislaus operates DNS servers that perform
translations of hostnames that end with "csustan.edu"
- w2k3-ns01.csustan.edu,
- w2k3-ns02.csustan.edu,
- Defined as translation of a domain name to an address
- Resolver: client software that performs name resolution by making a request to a domain name server
- Resolvers are typically configured to know the address of one or more local domain name servers
- A resolver may choose to use either stream or datagram service
- As an optimization DNS servers cache the information they get from other servers.
- Along with the translation information, servers send a cache timeout value - so receiving servers know when to delete information they cache.
- Resolvers cache DNS query results too.
- Because of the use of mail hubs, it is desirable that DNS not always return the same IP address for a given domain name.
- DNS databases have MX entries.
- If a query specifies type MX the IP address given in answer may be different from the answer given to a query of type A.
- and entry like this:
www CNAME timmy
in the DNS database file for foobar.com allows the host name www.foobar.com to be used as an alias for the hostname timmy.foobar.com. - We can move the web server to jimmy.foobar.com, and change the DNS
entry to:
www CNAME jimmy
and no one using URL's containing www.foobar.com has to change anything.
- DNS will only look up fully qualified domain names
- resolvers may try out different ways to complete abbreviated domain names -- e.g. "search domains"
- The International Domain Names in Applications (IDNA) standard specifies how to translate arbitrary domain names into a string of ASCII characters.
- DNS databases contain only ASCII versions of domain names.
- An extensible protocol or standard like XML allows users to specify new formats for data.