Computer Network Summary

Summary of Computer Networks#

I. Architecture#

Application Layer: The application layer protocol defines the rules for communication and interaction between application processes, providing data transmission services for specific applications, such as HTTP, DNS, etc. The data unit is a message.
Transport Layer: The task of the transport layer is to provide general data transmission services (TCP or UDP) for communication between processes on two hosts.
Network Layer: Encapsulates the message segments or user datagrams generated by the transport layer into packets for transmission.
Data Link Layer: Assembles the IP datagrams handed down from the network layer into frames and transmits them over the link between two adjacent nodes.
Physical Layer: Transmits data in the form of bits using physical media.

Physical Layer Channel Multiplexing Technology#

Basic Concepts#

Channel, a medium for transmitting information in one direction, includes:

Unidirectional Channel: One party sends, one party receives, simplex.
Bidirectional Alternating Channel: Both parties can send messages, but not simultaneously, half-duplex.
Bidirectional Simultaneous Channel: Both parties can send information simultaneously, full-duplex.

Multiplexing refers to channel sharing, with several common channel multiplexing technologies: frequency division multiplexing, time division multiplexing, statistical time division multiplexing. Signal multiplexing and demultiplexing are performed through multiplexers and demultiplexers.

Channel Multiplexing Technologies#

1. Frequency Division Multiplexing (FDM)
Users occupy different frequency bandwidths to multiplex the same channel at the same time.

2. Time Division Multiplexing (TDM)
Divides time into equal time slots for multiplexing frames, occupying the same bandwidth at different times.

3. Statistical Time Division Multiplexing (STDM)
An improved time division multiplexing that dynamically allocates time slots as needed, rather than fixed allocation (the number of time slots is less than the number of users connected to the concentrator, ensuring that the packets in each transmitted STDM frame are full).

4. Wavelength Division Multiplexing (WDM)
Frequency division multiplexing for light, used in optical communication technologies.

5. Code Division Multiplexing (CDM)
Each user communicates using the same frequency band at the same time, but each user uses a selected different code pattern, allowing independent communication between users.

Data Link Layer#

The main functions of the data link layer:

Link management
Frame synchronization
Flow control
Error control
Distinguishing data from control information
Transparent transmission
Addressing

The three basic issues of the data link layer: Encapsulation into frames, Transparent transmission, Error detection.

Encapsulation into Frames#

Add headers and trailers to the front and back of the IP datagram to form a frame, so that the receiving end can know the start and end of the frame in the bit stream at the physical layer, which is frame delimitation;

In addition, the header and trailer must also add many control information. The link layer protocol specifies the upper limit on the length of the data portion of the frame that can be transmitted, Maximum Transmission Unit - MTU.

Transparent Transmission#

The concept of transparent transmission refers to the fact that during transmission at the data link layer, the transmitted data is not obstructed at the data link layer, and the data received by the receiving party is identical to the data sent by the sending party, meaning that the data link layer is completely transparent to the transmitted data frames;

Byte Stuffing:
During transmission, to prevent the data portion from containing frame delimiters that cause the receiving party to mistakenly believe that the received data has ended prematurely, an escape character is used to insert an escape character before any control character appearing in the data portion. The inserted escape characters are removed at the receiving party's data link layer.

Zero Bit Stuffing:
Whenever five consecutive 1s are detected, a 0 is immediately inserted to ensure that six consecutive 1s do not occur.

Error Detection#

Error detection refers to bit errors that occur during transmission;

Bit Error Rate: The ratio of erroneous bits to the total number of bits transmitted over a period of time;

Currently, the error detection method used at the data link layer is Cyclic Redundancy Check;

Note:
At the data link layer, we ensure there are no bit errors, but not transmission errors. Transmission errors also include frame loss, frame duplication, frame disorder, etc.;

II. IP Address#

Overview#

For computers to achieve network communication, they must have a network address for quick location. The IP address is the unique identity ID of a computer in the network, analogous to the need for a specific residential address for delivery in the real world.

IP Address = Network Address + Host Address (also known as Host Number and Network Number)

Subnet Division#

Class A Address: Starts with 0, first byte range: 0 - 127 (1.0.0.0 - 126.255.255.255);
Class B Address: Starts with 10, first byte range: 128 - 191 (128.0.0.0 - 191.255.255.255);
Class C Address: Starts with 110, first byte range: 192 - 223 (192.0.0.0 - 223.255.255.255);
Reserved addresses on the Internet for internal private use: 10.0.0.0 - 10.255.255.255, 172.16.0.0 - 172.31.255.255, 192.168.0.0 - 192.168.255.255.

Subnet Mask#

When dividing subnets, we divide part of the originally host number into the network number to distinguish which part is the network number and which part is the host number, which needs to be achieved through the subnet mask.
The subnet mask cannot exist alone; it must be used in conjunction with the IP address.
The only function of the subnet mask is to divide a certain IP address into two parts: the network address and the host address.

Calculating Subnet Mask#

Calculating Subnet Mask Using the Number of Subnets#

Calculation Rules:

Convert the number of subnets into binary representation.
Obtain the number of bits of that binary, denoted as N.
Obtain the class subnet mask of the IP address, and set the first N bits of the host address portion to 1 to derive the subnet mask for the subnet division of that IP address.

For example, to divide the Class B IP address 168.195.0.0 into 27 subnets:

27 = 11011
This binary is five bits, N = 5.
Set the first 5 bits of the host address of the Class B subnet mask 255.255.0.0 to 1, resulting in 255.255.248.0, which is the subnet mask for dividing the Class B IP address 168.195.0.0 into 27 subnets.

Calculating Subnet Mask Using the Number of Hosts#

Calculation Rules:

Convert the number of hosts into binary representation.
If the number of hosts is less than or equal to 254 (note to exclude the two reserved IP addresses), obtain the number of bits of that host, denoted as N, where N < 8. If greater than 254, then N > 8, indicating that the host address will occupy more than 8 bits.
Use 255.255.255.255 to set all bits of the host address of that class IP address to 1, then set the last N bits to 0 from back to front to derive the subnet mask value.

For example, to divide the Class B IP address 168.195.0.0 into several subnets, with 700 hosts in each subnet:

700 = 1010111100
This binary is ten bits, N = 10.
Set all bits of the host address of the Class B subnet mask 255.255.0.0 to 1, resulting in 255.255.255.255, then set the last 10 bits to 0 from back to front, resulting in: 11111111.11111111.11111100.00000000, which is 255.255.252.0. This is the subnet mask for the Class B IP address 168.195.0.0 divided into 700 hosts.

III. UDP#

Overview#

The User Datagram Protocol (UDP) adds multiplexing and demultiplexing functions and error detection to the IP datagram service. It is characterized by connectionless messages and unreliable transmission.
UDP only adds headers to the data handed down from the application layer and processes it specially, then hands it over to the network layer;
The user datagram received from the network layer is unpacked after removing the header and handed over to the application layer.

UDP Header Format#

The UDP header fields are simple, consisting of 4 fields, each 2 bytes long, totaling 8 bytes.

Source Port: Used when a reply is needed from the other party; can be all 0 if not needed.
Destination Port: Must be used when delivering the message at the endpoint; otherwise, who will the data be delivered to?
Length: The length of UDP, with a minimum value of 8 bytes, which only includes the header.
Checksum: Checks whether there are errors in the user datagram during transmission; if there are errors, it is discarded.

During transmission, if the receiving UDP finds that the destination port in the received message does not exist, it will discard it directly, and the Internet Control Message Protocol (ICMP) will send an "unreachable port" error message to the sender.

For details on cyclic redundancy check methods, see: Introduction and Implementation Analysis of CRC (Cyclic Redundancy Check Code).

IV. TCP#

As one of the main protocols of the transport layer, the TCP protocol is connection-oriented, end-to-end, reliable full-duplex communication, and a byte stream-oriented data transmission protocol.

TCP Segment#

Although TCP is byte stream-oriented, the data unit transmitted by TCP is the segment. A TCP segment consists of a TCP header and a data portion, with the first 20 bytes of the TCP segment header being fixed, and an additional 4n bytes added dynamically as needed, with a maximum length of 40 bytes.

Source Port and Destination Port each occupy two bytes; TCP's demultiplexing function is also implemented through ports.
Sequence Number occupies 4 bytes, with a range of [0, 232]. TCP is byte stream-oriented, and each byte is numbered in order. For example, if a segment has a sequence number field of 201 and carries a data length of 100, then the sequence number of the first data is 201, and the last one is 300. When the maximum range is reached, it starts again from 0.
Acknowledgment Number occupies 4 bytes and is the sequence number of the first byte of the next segment expected to be received from the other party. If the acknowledgment number = N, it indicates that all data before sequence number N has been correctly received.
Data Offset occupies 4 bits and indicates the starting position of the data portion of the segment, relative to the starting position of the entire segment. It indirectly indicates the length of the header.
Reserved occupies 6 bits and is reserved for future use, currently set to 0.
URG (Urgent) When URG=1, it indicates that the urgent pointer field is valid, and this segment contains urgent data that should be sent as soon as possible.
Acknowledgment Number is valid only when ACK=1; after the connection is established, all segments' ACKs are set to 1.
PSH (Push) When the receiving party receives a segment with PSH=1, it will deliver it to the receiving application as soon as possible, without waiting for the entire buffer to fill up before delivery. This is rarely used in practice.
RST (Reset) When RST=1, it indicates that a serious error has occurred in the TCP connection, and it must be reset and reconnected.
SYN (Synchronize) Used to synchronize sequence numbers when establishing a connection. When SYN=1 and ACK=0, it indicates a connection request segment. When SYN=1 and ACK=1, it indicates that the other party agrees to the connection. Used in establishing TCP connections.
FIN (Finish) Used to release a connection window. When FIN=1, it indicates that the sender of this segment will no longer send data and requests to release the unidirectional connection. Used to disconnect TCP connections.
Window occupies 2 bytes and indicates the sender's own receiving window; the window value is used to inform the other party of the allowed amount of data to be sent.
Checksum occupies 2 bytes, and the checksum field checks the range including the header and data portion.
Urgent Pointer occupies 2 bytes; when URG=1, the urgent pointer indicates the number of urgent bytes in this segment (ordinary bytes follow the urgent bytes).
Options variable length, up to 40 bytes. For example, Maximum Segment Size (MSS). MSS refers to the length of the data portion, not the entire TCP segment length; the default MSS is 536 bytes. Options such as window scaling, timestamps, etc.

Three-Way Handshake#

First: The client sends a connection request segment to the server, where SYN=1, seq=x (representing the client's initial sequence number), i.e., ISN (Initial Sequence Number). After sending, it enters the SYN_END state.
Second: The server receives the segment and sends back an acknowledgment segment, where ACK=1, ack=x+1, and also includes SYN=1, seq=y in the segment for the client to confirm. After sending, it enters the SYN_RCVD state.
Third: The client receives the segment and sends an acknowledgment segment, where ACK=1, ack=y+1. After sending, the client enters the ESTABLISHED state, and the server, upon receiving the segment, also enters the ESTABLISHED state. At this point, the connection is established.

Reason for Three-Way Handshake
To avoid resource wastage. If during the second handshake, due to network delays, the acknowledgment packet does not reach the client in time, the client will think the first handshake failed and will send a connection request again. The server, upon receiving it, will send an acknowledgment packet again. In this case, the server has created two connections, waiting for two clients to send data, while in reality, only one client is sending data, leading to resource wastage.

Four-Way Handshake#

The four-way handshake refers to the client and server each sending a request to terminate the connection while responding to each other's requests.

First Handshake: The client sends a packet with FIN=1, seq=x to the server, indicating that it has no data to transmit and wants to close the unidirectional connection. After sending, the client enters the FIN_WAIT_1 state.
Second Handshake: The server receives the request packet and sends back an acknowledgment packet with ACK=1, ack=x+1, confirming the disconnection. The server enters the CLOSE_WAIT state. After receiving this packet, the client enters the FIN_WAIT_2 state. At this point, the data connection from the client to the server has been disconnected.
Third Handshake: The server sends a packet with FIN=1, seq=y to the client, indicating that it has no data to send to the client anymore. After sending, it enters the LAST_ACK state, waiting for the client's acknowledgment packet.
Fourth Handshake: The client receives the request packet and sends an acknowledgment packet with ACK=1, ack=y+1 to the server, entering the TIME_WAIT state, possibly to retransmit the acknowledgment packet. After receiving the acknowledgment packet, the server enters the CLOSED state, and the connection from the server to the client has been disconnected. After a period, the client will also enter the CLOSED state.

Note:

By default (without changing socket options), when you call close (or closesocket, hereafter referred to as close), if there is still data in the send buffer, TCP will continue to send the data until completion.
Sending FIN only indicates that this end cannot continue sending data (the application layer can no longer call send), but it can still receive data.

Reason for Four-Way Handshake
Since TCP connections are full-duplex, both parties can actively transmit data. One party's disconnection needs to inform the other party so that the other party can carry out subsequent related operations, which is a responsible behavior.

Protocols based on TCP transmission include: FTP (File Transfer Protocol), Telnet (Remote Login Protocol), SMTP (Simple Mail Transfer Protocol), POP3 (used for receiving emails, in contrast to SMTP), HTTP Protocol, etc.

Supplement on "Three-Way Handshake, Four-Way Handshake"#

ISN#

An important function of the three-way handshake is for the client and server to exchange ISNs, allowing each party to know how to assemble data by sequence number when receiving data next.
If the ISN is fixed, an attacker can easily guess the subsequent acknowledgment numbers.
ISN = M + F(localhost, localport, remotehost, remoteport)
M is a timer that increments by 1 every 4 milliseconds. F is a hash algorithm that generates a random value based on the source IP, destination IP, source port, and destination port. The hash algorithm must not be easily deduced by external parties.

Detailed Explanation of TIME_WAIT State#

The TIME_WAIT state is set to 2 * Maximum Segment Lifetime (MSL). The reason for the TIME_WAIT state is:

A SOCKET in the LAST_ACK state may resend the FIN packet due to a timeout without receiving the ACK packet, so the purpose of this TIME_WAIT state is to retransmit potentially lost ACK packets.
Ensure the connection is closed. If the link to be closed later establishes a new connection with the same IP address and port, it may be mistakenly regarded as a new request for the old connection (a TCP session is uniquely determined by the four-tuple of [source IP, source port, destination IP, destination port]). Since the maximum lifetime of TCP segments is MSL, maintaining the TIME_WAIT state for 2MSL ensures that all unreceived or late packets in both transmission directions have disappeared or been discarded by routers, and after 2MSL, establishing a new connection will not receive application data from the original connection. Thus, by setting a TIME_WAIT state of 2MSL, a new connection will not receive application data from the original connection.

Sequence Number Wraparound#

Since ISNs are random, sequence numbers can easily exceed 2^31-1. TCP relies on comparing sequence numbers to determine issues like packet loss and out-of-order delivery, leading to the so-called TCP sequence number wraparound problem.

Half-Connection and Full-Connection#

Half-Connection Queue:
After the server receives the client's SYN for the first time, it enters the SYN_RCVD state. At this point, the connection has not been fully established, and the server will place this request in a queue, which we call the half-connection queue.

Full-Connection Queue:
Connections that have completed the three-way handshake and are established will be placed in the full-connection queue. If the queue is full, packet loss may occur.

For other details such as SYN flood attacks, SYN Cache technology, SYN Cookie technology, SYN Proxy firewalls, etc., please refer to: Do You Really Understand "Three-Way Handshake, Four-Way Handshake"?.

TCP Reliable Transmission#

Reliable transmission should meet the following criteria:

The transmission channel does not produce errors;
Ensure the correctness of the transmitted data, with no errors, no loss, no duplication, and in order.

Implementation of TCP Reliable Transmission:

Use a three-way handshake to establish a TCP connection and a four-way handshake to release the TCP connection, ensuring that the established transmission channel is reliable.
TCP uses the ARQ protocol to ensure the correctness of data transmission.
TCP employs the sliding window protocol to ensure that the receiving party can timely process the received data and control the flow.
TCP uses slow start, congestion avoidance, fast retransmission, and fast recovery for congestion control, avoiding network congestion.

Sliding Window Protocol#

The sliding window protocol means that the sender has a sending window, and data within the range of the sending window is allowed to be sent. The starting position and size of the sending window are dynamically adjusted based on the acknowledgment information received from the receiver and the size of the receiving window.
There are three scenarios for window sliding:

The front edge moves to the right, which occurs when data is sent and acknowledged.
The back edge moves to the right, allowing more data to be sent. This occurs when the receiving window increases or network congestion alleviates.
The back edge moves to the left, which occurs when the receiver wants to shrink the sending window. The TCP standard strongly discourages this situation because the sender may have already sent some data that falls outside the reduced window, leading to errors.
The front edge of the window cannot move to the left because TCP will clear out the acknowledged data outside the window from the buffer.

ARQ Protocol#

ARQ (Automatic Repeat reQuest) is a method for achieving reliable data transmission over unreliable networks using acknowledgments (ACKs, where the receiving party sends a message to inform the sender whether it has correctly received a packet) and timeouts (the period during which a sender waits for an acknowledgment). It includes mechanisms such as stop-and-wait ARQ protocol and continuous ARQ protocol, error detection, positive acknowledgment, retransmission after timeout, and negative acknowledgment and retransmission.

Stop-and-Wait ARQ:
"Stop-and-wait" means that after sending a packet, the sender stops and waits for the other party's acknowledgment. Upon receiving the acknowledgment, it sends the next packet. There are two error scenarios:

When the receiver receives an erroneous data packet, it discards the packet.
If the sender does not receive an acknowledgment within a certain time, it retransmits the packet, i.e., timeout retransmission.

Continuous ARQ Protocol:
The sliding window protocol combined with automatic repeat request technology forms the continuous ARQ protocol. The continuous ARQ protocol is divided into Go-Back-N ARQ protocol and Selective Repeat ARQ protocol based on different methods of retransmitting data after a timeout.

The sender slides the sending window forward by one packet position upon receiving an acknowledgment.
The receiver generally adopts the cumulative acknowledgment (ACK mechanism), meaning that the receiver does not need to send acknowledgments for each received packet but can send an acknowledgment for the last packet received in order after receiving several packets, indicating that all packets up to that point have been correctly received.

`Go-Back-N ARQ`#

GBN allows the sender to continuously send several frames after sending one data frame without stopping to wait for the acknowledgment frame. If an acknowledgment frame is received, the sender can continue sending data frames. This reduces waiting time and improves utilization. However, if an erroneous frame is received, all subsequent frames must be retransmitted.

`Selective Repeat/Reject ARQ`#

In contrast to Go-Back-N ARQ, in the case of receiving an erroneous frame, it does not require retransmitting all frames, only the erroneous frames need to be retransmitted, further improving utilization. However, it sacrifices space because it needs to cache the correctly received information until all information is correctly received before sending an acknowledgment.

Flow Control#

When sending data through a TCP connection, if the sender sends data too slowly, it may lead to resource wastage; if the sender sends data too quickly, the receiver may not be able to keep up, leading to data loss. Flow control refers to sending data reasonably and quickly within the range that the receiver can handle.

Flow Control Based on Sliding Window#

When establishing a TCP connection, the receiver specifies its receiving window size in the acknowledgment segment. Each time an acknowledgment segment is sent, it can dynamically adjust the size of the receiving window based on the situation and inform the sender. As shown in the figure below:

The sender sends 100 bytes of data starting from sequence number 1, and the receiver declares its receiving window size to be 300 bytes in the acknowledgment segment. The sender then sends 300 bytes of data, and the receiver adjusts its receiving window size to 50 bytes in the acknowledgment segment. After sending 50 bytes of data, the sender receives an acknowledgment segment from the receiver, which declares the receiving window to be 0.

When the receiving window is 0, the sender stops sending data until the receiver sends an acknowledgment segment indicating that the window size has changed. However, this acknowledgment segment may not be received by the sender, and if it is lost, both parties will be in a waiting state, resulting in a deadlock. To prevent this situation, TCP specifies that when the receiving window is 0, a persistent timer is started to periodically send probe packets to determine whether the receiving window's state has changed.

Additionally, the TCP standard specifies that when the receiving window is 0, it will no longer accept normal data, but can accept zero window probe segments, acknowledgment segments, and segments carrying urgent data.

Congestion Control#

TCP commonly uses four algorithms for congestion control: slow start, congestion avoidance, fast retransmission, and fast recovery.

Slow Start#

TCP maintains a congestion window, denoted as cwnd, which is related to SMSS, the maximum segment size sent. The slow start algorithm specifies that after initializing the congestion window, for each acknowledgment received for a new segment, the congestion window increases by one SMSS. The congestion window is measured in bytes, but slow start increases it by SMSS. According to the slow start algorithm, after one round of transmission, the congestion window doubles, indicating exponential growth.

Congestion Avoidance#

In addition to maintaining the congestion window cwnd variable, the slow start algorithm also maintains another variable, the slow start threshold ssthresh. When cwnd grows exponentially to be greater than or equal to ssthresh, the slow start algorithm is no longer used, and the congestion avoidance algorithm is adopted for congestion control. The congestion avoidance algorithm specifies that cwnd is increased by 1/cwnd of SMSS for each acknowledgment received. This means that instead of doubling cwnd after one round of transmission as in the slow start algorithm, it increases by one SMSS. This indicates additive growth, and when congestion occurs (timeout or receiving duplicate acknowledgments), cwnd is set to 1 SMSS. ssthresh is set to half of the current window size, but at least 2 segments.

In summary: Additive increase, multiplicative decrease.

Fast Retransmission#

If individual segments are lost in the network without congestion occurring, the sender does not receive acknowledgment packets, and after a timeout, it will retransmit the segment. The sender mistakenly believes that congestion has occurred and incorrectly starts the slow start algorithm, reducing transmission efficiency. The fast retransmission algorithm allows the sender to know about the loss of individual segments earlier. The fast retransmission algorithm requires the receiver to send acknowledgments immediately, even for out-of-order segments. As shown in the figure below:

After receiving M1, the receiver sends an acknowledgment for M1. M2 is lost, and when the receiver receives M3, M4, and M5, it sends repeated acknowledgments for M1 each time. The fast retransmission algorithm specifies that upon receiving three duplicate acknowledgments, the sender considers the M2 segment lost and immediately retransmits it without waiting for a timeout, thus avoiding the sender mistakenly believing that congestion has occurred.

Fast Recovery#

After executing the fast retransmission algorithm, the sender knows that only individual segments are lost, not that congestion has occurred. It then does not execute the slow start algorithm but executes the fast recovery algorithm: setting the threshold ssthresh = cwnd/2, while setting cwnd = ssthresh + 3 SMSS. The congestion window value is set to the threshold plus three segments because the sender has received three acknowledgment packets, indicating that three packets have left the network and reached the receiver's buffer. These three acknowledgment packets no longer occupy network resources, allowing for an appropriate increase in the congestion window size.

Difference Between Congestion Control and Flow Control#

Congestion Control: Congestion control acts on the network, preventing too much data from being injected into the network to avoid excessive network load.
Flow Control: Flow control acts on the receiver, controlling the sender's sending speed so that the receiver can keep up and prevent packet loss.

V. Differences Between TCP and UDP#

TCP is byte stream-oriented, while UDP is message-oriented.
TCP is connection-oriented (requires a three-way handshake), while UDP is connectionless.
TCP provides reliable data transmission services, with packet loss retransmission mechanisms and guarantees data order; UDP may lose packets and does not guarantee data order.
Each TCP connection can only be point-to-point; UDP supports one-to-one, one-to-many, many-to-one, and many-to-many interactive communication.
TCP requires more system resources, while UDP requires fewer.

Specific programming differences, different parameters for socket():

UDP Server does not need to call listen and accept.
UDP sends and receives data using sendto/recvfrom functions.
TCP: Address information is determined during connect/accept.
UDP: Address information must be specified each time in sendto/recvfrom functions.
UDP: The shutdown function is ineffective.

VI. HTTP Protocol#

The HTTP protocol is short for Hypertext Transfer Protocol. It is the protocol for transferring hypertext markup language (HTML) from the WEB server to the local browser, based on the TCP/IP communication protocol for data transmission.

Message Format#

Common Request Headers#

Accept: Used to inform the server of the data types supported by the client.
Accept-Charset: Informs the server of the encoding format used by the client.
Accept-Encoding: Informs the server of the data compression formats supported by the client.
Accept-Language: The language environment of the client.
Host: The hostname the client wants to access through this header.
If-Modified-Since: Informs the server of the resource's cache time.
Referer: Informs the server from which resource the client accessed the server (to prevent hotlinking).
User-Agent: Informs the server of the client's software environment.
Cookie: The client can send data to the server through this header.
Connection: Indicates whether to close the connection after processing this request or to keep it open.
Date: The current time value.

Common Response Headers#

Location: Indicates the address for redirection; this header is used with the 302 status code.
Server: Indicates the type of server.
Content-Encoding: Indicates the type of data compression sent by the server to the browser.
Content-Length: Indicates the length of the data sent by the server to the browser.
Content-Language: Indicates the languages supported by the server.
Content-Type: Indicates the type and content encoding of the data sent by the server to the browser.
Last-Modified: Indicates the last modification time of the server resource.
Refresh: Indicates timed refresh.
Content-Disposition: Informs the browser to open the resource in download mode (used when downloading files).
Transfer-Encoding: Informs the browser of the data transfer format.
Set-Cookie: Indicates the cookie information sent by the server to the browser (used for session management).
Expires: Informs the browser how long to cache the returned resource; -1 or 0 means not to cache.
Cache-Control: no-cache.
Pragma: no-cache.
The server uses the above two headers to control the browser not to cache data.
Connection: Indicates the connection status between the server and the browser. close: close the connection; keep-alive: keep the connection open.

Request Status Codes#

Status Code Classification#

1XX - Informational, the server has received the request and requires the requester to continue the operation.
2XX - Success, the request has been successfully received, understood, and processed.
3XX - Redirection, further action is needed to complete the request.
4XX - Client Error, the request contains syntax errors or cannot be completed.
5XX - Server Error, an error occurred on the server while processing the request.

Common Request Status Codes#

200 OK The request has succeeded.
204 No Content No content, the server successfully processed the request without returning any content.
301 Moved Permanently The requested resource has moved, permanently redirected.
302 Found The resource has been found, temporarily redirected.
304 Not Modified Not modified, the client's cached resource is the latest, and the client should use the cache.
400 Bad Request The client request has syntax errors and cannot be understood by the server.
401 Unauthorized Unauthorized, the client is not authorized to access the data.
403 Forbidden Forbidden, no permission to access.
404 Not Found The requested resource does not exist, possibly due to an incorrect URL.
405 Method Not Allowed The method is not allowed.
406 Not Acceptable The content characteristics of the requested resource cannot satisfy the conditions in the request header, thus unable to generate a response entity.
500 Internal Server Error Internal server error.
502 Bad Gateway Invalid gateway, the server acting as a gateway or proxy received an invalid response while executing the request.
503 Service Unavailable Service unavailable, the server could not respond due to maintenance or heavy load.
504 Gateway Timeout The gateway connection timed out; the gateway or proxy server could not respond within the specified time.

Differences Between POST and GET Requests#

GET request data is attached after the URL (i.e., the data is placed in the HTTP protocol header), separated by a ? between the URL and the transmitted data, with parameters connected by &. POST places the submitted data in the body of the HTTP packet.
GET can submit a smaller amount of data (because the data is attached after the URL), while POST can submit a larger amount of data; browsers and servers have different limits.
POST is more secure than GET; the data in GET is visible to everyone in the URL; saying GET is secure refers only to non-modifying information; GET is used to retrieve/query resource information, while POST is used to update resource information.
GET is idempotent, while POST is not.
GET encoding type is application/x-www-form-url, while POST encoding types are application/x-www-form-urlencoded or multipart/form-data.
GET historical parameters are retained in the browser history. POST parameters are not retained in the browser history.

Idempotency#

The idempotency of HTTP methods means that making a single request and making multiple requests for a resource should have the same side effects. That is, sending the same request once and sending it N times should yield the same result!
Idempotent methods: GET, DELETE, PUT.
PUT and POST are both for creating or updating, with the only difference being that POST is non-idempotent.

HTTP/1.x vs. HTTP/2.0#

HTTP/1.1#

Introduced persistent connections (persistent connection), meaning that TCP connections are not closed by default and can be reused for multiple requests without declaring Connection: keep-alive.
Introduced pipelining, allowing the client to send multiple requests simultaneously within the same TCP connection, further improving the efficiency of the HTTP protocol.
New methods added: PUT, PATCH, OPTIONS, DELETE.
The HTTP protocol is stateless, meaning that each request must include all information. Many fields in requests are repeated, wasting bandwidth and affecting speed.

HTTP/2.0#

HTTP/2 is a completely binary protocol, with header information and data both in binary format, collectively referred to as "frames": header frames and data frames. This avoids the security issues caused by plaintext transmission of content in HTTP/1.x.
Reuse of TCP connections, allowing multiple requests or responses to be sent simultaneously in one connection without needing to correspond in order, avoiding head-of-line blocking issues. This bidirectional real-time communication is called multiplexing.
HTTP/2 allows the server to proactively send resources to the client without a request, known as server push.
Introduced header compression mechanism, compressing header information using gzip or compress before sending.

HTTPS#

HTTPS communicates via HTTP, using the SSL/TLS protocol to encrypt data packets, making it the secure version of HTTP. The main differences with HTTP are:

HTTPS requires obtaining a certificate from a CA.
HTTP and HTTPS use different ports; the former uses 80, while the latter uses 443.
HTTP runs over TCP, with all transmitted content in plaintext, while HTTPS runs over SSL/TLS, which runs over TCP, with all transmitted content encrypted.
HTTPS effectively prevents operator hijacking.

VII. The Process from Entering a Domain Name to Displaying a Page in the Browser#

Event Sequence:

DNS Domain Resolution: Browser cache --> Operating system cache --> Local host file --> Router cache --> ISP DNS cache --> Top-level DNS server/root DNS server, looking up the IP address corresponding to the domain name.
Establishing a TCP Connection: TCP uses a three-way handshake to establish a connection with the server, providing reliable transmission services.
Sending HTTP Request: Sending the HTTP message to the server through the TCP connection.
The server processes the request and returns the HTTP message.
The browser renders the page: The browser first parses the HTML file to construct the DOM tree, then parses the CSS file to construct the rendering tree. Once the rendering tree is complete, the browser begins to layout the rendering tree and draws it on the screen.

Protocols Involved:

Application Layer: HTTP (WWW access protocol), DNS (domain name resolution service).
DNS maintains a mapping table of domain names (domain name) and IP addresses (IP address) to resolve the domain name.
Transport Layer: TCP (providing reliable data transmission for HTTP), UDP (DNS uses UDP for transmission).
Network Layer: IP (transmission and routing of IP data packets), ICMP (providing error detection during network transmission), ARP (mapping the default gateway IP address to the physical MAC address).

Cookie data is stored on the client's browser, while session data is stored on the server.
Cookies are not very secure; others can analyze the cookies stored locally and perform cookie spoofing.
For security reasons, sessions should be used.
Sessions are stored on the server for a certain period. With increased access, they can consume server performance. To alleviate server performance, cookies should be used.
A single cookie can store no more than 4K of data, and many browsers limit a site to a maximum of 20 cookies.

Supplement#

Sessions rely on cookies to some extent. When the server implements the session mechanism, it generates a session ID, which is sent to the client. The client includes this ID in the HTTP request header for each request to the server, and this ID is stored on the client side, with the storage container being the cookie.

IX. Long Connections, Short Connections, and WebSocket#

Differences Between Long Connections and Short Connections#

In HTTP/1.0, short connections are used by default. This means that for each HTTP operation, a connection is established, but the connection is interrupted once the task is completed. Starting from HTTP/1.1, long connections are used by default to maintain connection characteristics. In long connections, the HTTP protocol sets Connection: keep-alive in the response header.

In long connections, once a webpage is fully opened, the TCP connection used for transmitting HTTP data between the client and server will not close. If the client accesses another webpage on the same server, it will continue to use the already established connection. Keep-Alive does not maintain the connection permanently; it has a keep-alive time that can be set in different server software (such as Apache). Implementing long connections requires both the client and server to support long connections.

The long and short connections of the HTTP protocol are essentially the long and short connections of the TCP protocol.

Application Scenarios for Long Connections and Short Connections#

Long connections are often used in scenarios with frequent operations and point-to-point communication, where the number of connections cannot be too high. Each TCP connection requires three-way handshakes, which take time. If each operation involves connecting and then operating, the processing speed will significantly decrease. Therefore, after each operation, the connection is not closed, allowing the next operation to send data packets directly without establishing a TCP connection. For example, database connections use long connections; using short connections for frequent communication can lead to socket errors, and frequent socket creation is also a waste of resources.
In contrast, web services generally use short connections because long connections consume server resources. For web services with potentially thousands or even millions of client connections, short connections are more resource-efficient. If long connections were used and thousands of users were connected simultaneously, it would be unsustainable. Therefore, short connections are preferable in scenarios with high concurrency but where each user does not require frequent operations.

Short Polling and Long Polling#

Short and long polling are fundamentally different from long and short connections. Long and short connections refer to the mechanism of establishing and maintaining TCP connections between the client and server, while short and long polling refer to the way the client requests the server and the server responds.

Short Polling: Repeatedly sends HTTP requests to check whether the target event is completed. Advantages: Simple to write; disadvantages: Wastes bandwidth and server resources.
Long Polling: The server holds the HTTP request (using a loop or sleep, etc.) until the target event occurs (keeping the request waiting for data to arrive or timing out appropriately) and then returns the HTTP response. Advantages: Does not frequently request in the absence of messages; disadvantages: More complex to write.

WebSocket#

Establishing a WebSocket Connection#

The client browser first sends an HTTP request to the server, which is different from a typical HTTP request and includes some additional header information, with the additional header "Upgrade: WebSocket" indicating that this is a request for a protocol upgrade.

Differences Between WebSocket and HTTP Long Connections#

HTTP/1.1 uses Connection: keep-alive for long connections, and HTTP 1.1 defaults to persistent connections. Multiple HTTP requests can be completed in one TCP connection, but each request still requires sending headers separately. Keep-Alive does not maintain the connection permanently; it has a keep-alive time that can be set in different server software (such as Apache).
The long connection of WebSocket is a true full-duplex connection. After the initial TCP link is established, subsequent data can be sent by both parties without needing to send request headers, and this connection will persist until one party actively closes it. Unlike HTTP long connections, WebSocket allows for more flexible control over when to close the connection, rather than having the server close it immediately when the Keep-Alive time expires (which is not user-friendly).