Unknownpgr

What Happens When You Search in a Web Browser?

2020-12-25 04:30:59 | English, Korean

This post was translated from Korean into English by AI.

What Happens in a Web Browser

I once heard that companies sometimes ask a question like this during the hiring process:

Enter http://google.com in a web browser's address bar and press Enter. Explain everything you know about what happens before the web page appears.

As a computer science student, I also became curious about how much detail I could give in response, so I decided to organize what I know.

Everything below is explained in the context of UNIX/Linux-based operating systems.

In the Hardware

First, when a key on the keyboard is pressed, an internal switch closes a circuit. A hardware keyboard module detects this, encodes the signal as the appropriate keycode, and sends it to the USB module. The USB module then sends this information to the computer according to the USB protocol.

The computer then receives the USB signal. This signal is received through the computer's USB controller, which is dedicated hardware for handling USB transmission and reception. The module decodes the received signal appropriately and communicates with the CPU.

USB controllers communicate with the CPU in various ways, one notable example being Direct Memory Access (DMA). If the CPU had to read USB data directly after it arrived, doing so would consume many clock cycles. Instead, the USB controller accesses RAM directly and writes the data to a predetermined area. Meanwhile, the CPU can do other work and then read the data straight from RAM after the write is complete. This method is called DMA because the data is written directly to memory.

The communication process may vary depending on the type of computer. In a typical computer, for example, the CPU is connected to the northbridge through the Front-Side Bus (FSB); the northbridge is connected by buses to RAM and the southbridge; and the southbridge is connected to the USB driver. In the ARM Cortex M4 MCU, which I am fond of, the USB controller, core, and RAM are instead connected directly through the Advanced High-performance Bus (AHB).

Once the data has been written to RAM via DMA, the CPU must be notified. The USB controller therefore generates an interrupt signal. How this interrupt signal is handled also differs depending on the type of computer.

In a typical computer, the signal is delivered to an interrupt controller such as the 8259A (which is also hardware). Other controllers, such as the USB controller, are connected to the interrupt controller by lines called IRQs. Physically, an IRQ line is simply a wire capable of carrying one bit. An interrupt controller such as the 8259A chip has seven IRQ lines, IRQ 0 through 7, and can operate in Master or Slave mode, allowing two controllers to be connected in a cascade. Because IRQ2 on the Master is used to receive interrupts from the Slave, a total of 15 interrupts can be used. The Master ultimately sends a signal directly to the CPU through the INTR line. This INTR line is also just a single wire.

MCUs used in devices such as Raspberry Pis and mobile devices are called Systems on Chip (SoCs), and all of these components are built into the processor. For example, the Cortex M4 MCU mentioned above has an interrupt controller called the Nested Vector Interrupt Controller (NVIC) placed right next to the core. In this case, every interrupt is handled through the NVIC. Even an interrupt such as a System Exception generated by the core is not handled by the core itself. Instead, the core first sends a signal to the NVIC, which handles it. The reason for this seemingly unnecessary complexity is to let the user freely adjust interrupt priorities through the NVIC's Interrupt Vector Register.

When the core receives an interrupt signal through the INTR line, it normally jumps to a predetermined address and runs an interrupt routine. Much like a context change, this involves saving the CPU's general-purpose registers and context on the stack before branching to the designated address. Subsequent processing is normally performed in the kernel. In the case of the NVIC mentioned above, however, the NVIC itself has an interrupt vector register that specifies the address to branch to for each interrupt, so the processor runs the appropriate interrupt service routine without kernel intervention.

In the Kernel

Once the appropriate service routine is running as described above, the kernel must determine which interrupt occurred and handle it appropriately. With the NVIC, a different service routine runs for each interrupt, so no separate identification step is needed. When a fixed service routine is used, however, it is necessary to determine which Interrupt Request (IRQ) caused that routine to run. Since I have only studied Linux systems, I will explain this in terms of the Linux kernel. In this case, the kernel reads the Interrupt Status Register (ISR), which is a register in the interrupt controller rather than in the CPU.

How this register is read also depends on the type of computer. If the system has a separate address space for external devices, called I/O Space, the register can be read with a special instruction. The x86 core is a representative example with a separate I/O space. In this case, an ordinary MOV instruction cannot access registers outside the CPU; special instructions such as IN AL, DX (Input byte from I/O port in DX into AL) must be used.

By contrast, the Cortex M4 MCU mentioned earlier uses Memory Mapped IO. Its entire address space is therefore unified with the main memory address space. Not only can ordinary assembly instructions such as MOV be used, but external registers can also be accessed conveniently through pointer operations in C.

More specifically, when the core receives an interrupt signal (INTR signal), it sends back another signal (INTA, Interrupt Accepted). INTA is also a hardware-implemented signal and, like INTR, uses a single line. Upon receiving the INTA signal, the interrupt controller sends the appropriate interrupt number over the Data Bus.

After reading the interrupt register in this way, the appropriate interrupt service routine must be run. The kernel uses the interrupt number (IRQ number) as an index to find and execute the proper service routine in the interrupt vector table. Some devices have a fixed IRQ based on which device generated the interrupt. A floppy disk, for example, always requests IRQ 6. Most modern devices, however, do not work this way, and device drivers can dynamically register an IRQ through a process called probing.

It would be nice if all the desired processing could be performed while the interrupt routine is running. However, these service routines interrupt the CPU's existing work, and other interrupts may also occur, so they must finish their work and exit as quickly as possible. Various mechanisms have been proposed for this purpose, including Bottom Halves, Task Queues, Softirq, Tasklet, and Work queues.

These interrupt service routines are registered by device drivers, and each device driver performs the processing appropriate to its role. Once the interrupt finishes safely, a predefined event on the wait queue is triggered. After appropriately processing the data, the kernel uses a special kernel function (copy_to_user) to copy the key information into user memory. A direct copy is impossible because kernel memory uses actual physical memory addresses, whereas user memory uses virtual memory addresses implemented through paging. When the kernel returns, the CPU operates in user mode and the user process resumes execution.

In the User Process

The user process recognizes from the key value that the key was Enter and begins its work. Because the work to follow takes a fair amount of time, it is performed in a thread so that it does not disrupt the UX. In a UNIX operating system, a thread is merely another process that shares the same address space, so in practice you can think of this as creating a process.

The newly created thread analyzes the address entered in the address bar and determines that the address is google.com and the protocol is http. If the user entered an invalid address (e.g., omitted the protocol or used characters that cannot appear in a URL), the browser also corrects it appropriately (e.g., automatically adds a suitable protocol or performs URL encoding). The browser then constructs an appropriate HTTP request containing various pieces of information, including the requested address, IP, requested file, browser information, and cookies.

DNS

At this point, if the computer already has the domain information and knows the IP address of google.com, it sends packets there. If it does not know the domain information, however, it must use a DNS query to discover the IP address to which that domain points. I had thought this process took place in the kernel, but my research showed that it occurs at the application level. The Linux kernel has no kernel functions related to DNS queries.

First of all, because domain lookup does not take place at the kernel level, its implementation varies from program to program. The ping command, for example, refers to /etc/nsswitch.conf and then to /etc/hosts, /etc/resolv.conf, and /etc/hostname. The host command, by contrast, refers directly to /etc/resolv.conf.

The browser will likewise query the DNS servers listed in /etc/resolv.conf, and the query itself will also be performed at the application level. Of course, the browser will have an internal cache to make connections faster. At least in Chrome's case, it is clear that mappings between domains and IP addresses are stored internally, because when I once tested an ARP spoofing attack, Chrome displayed a security warning and blocked the connection.

Socket

A socket must be created for this request data to be sent over the Internet. As far as I know, the browser creates and uses a connection pool of about six sockets, reportedly because creating any more causes performance to decline.

Since a socket created this way is treated as a file in the Linux operating system, the data is passed to the kernel through an ordinary write operation. A TRAP instruction switches the process to kernel mode, and kernel code copies the data from user space into kernel space.

In the Kernel

Socket Interface Layer

The data created in user space is transmitted using TCP/IP. First, the received data is passed through the socket interface layer to the transport layer. At this point, the socket interface layer calls the transport-layer function for the appropriate protocol. Since we are using a browser that communicates via the TCP/IP protocol, the socket interface layer will call the tcp_sendmsg function.

Transport Layer

From this point on, we are in the domain of the transport layer. Because TCP is in use, the connection-establishment stage occurs first. Once the connection has been successfully established, the actual data transmission process follows. When this stage is reached, the data is copied from user space to kernel space and an sk_buff structure is created. Until this point, the data created by the application itself is not copied; only the io vector containing the data's address is copied.

Packet segmentation also occurs during this process. Packet segmentation means dividing data into multiple packets when the data to be delivered is larger than the Maximum Transmission Unit (MTU). The tcp_transmit_skb function is then called, and a pointer to the generated packet is passed to the network layer (IP layer).

Network Layer

In this stage, an IP header is added to the packet. The routing lookup table is consulted to determine which network interface should receive the packet. Since the resulting packet must be delivered to the server, it is passed to the data link layer.

Of course, if any of the packets received from the data link layer are addressed to this machine, the IP layer will call the transport layer.

Data Link Layer

The data link layer simply acts as an intermediary before packets received from the IP layer are sent to the hardware. For this reason, it is sometimes called the queuing layer. A packet enters the appropriate queue and waits. Once the hardware—in this case, the Network Interface Card (NIC)—is ready to receive the data, the packet is transferred to the NIC via DMA. The entire process up to this point takes place in process context.

Physical Layer

From this point on, we are in the domain of the physical layer. The NIC properly encodes the received packet to create a frame. A packet contains data from layer 3 and above, such as IP addresses, while a frame contains information spanning layers 3 through 2, such as MAC addresses and checksums.

The resulting frame is sent to the router (gateway). If the router is connected directly by wire, the packet can be sent straight to it. When another medium such as a wireless LAN is subject to interference, however, various collision-avoidance mechanisms are used. As far as I know, wireless LANs use Carrier Sense Multiple Access / Collision Avoidance (CSMA/CA). Under this method, a device first informs the other devices that it is going to transmit a frame (Request to Send, RTS). Once the destination device responds that it can receive the frame (Clear to Send, CTS), the sender uses the carrier exclusively for a predetermined period while transmitting the data. Ethernet uses CSMA/CD; in this case, a device transmits data when no other device is using the carrier.

In both cases, however, there is a limit to the amount of data that can be transmitted at once. Under the IEEE 802.3 Ethernet protocol, the minimum amount of data that can be transmitted is 64 bytes (for the entire frame), and the maximum is 1,518 bytes. The minimum size is necessary for efficient transmission with CSMA/CD, while the maximum size prevents a device from monopolizing the carrier indefinitely.

This is nowhere near enough for the several kilobytes to several megabytes of data we typically want to send. The data is therefore split into multiple pieces for transmission and reassembled in order by the receiver. Methods such as Automatic Repeat Request (ARQ), which uses a sliding window, are employed for this.

When a router receives frames, it places them in a buffer in order, then takes them out and processes them one by one. After taking out a frame, the router decodes it again to inspect the IP address, then sends the frame out through another port appropriate for that IP address. During this process, the layer 3 information remains unchanged, but the layer 2 information changes at every hop. This makes sense: the layer 3 information identifies the source and destination, whereas the layer 2 information identifies the current router and the next router.

After passing through several routers in this way, the data arrives at Google's server.

On the Server

The server receives the packet by following the exact reverse of the process used to send it. The HTTP request is delivered to the server through the physical layer, data link layer, network layer, and transport layer. Since the HTTP request specifies the desired data, the server returns the corresponding data. In this case, that is the HTML file for Google's website. Of course, during this process, the server accesses the file system and network card through the kernel, much as it did for the keyboard input. Unlike with the keyboard described earlier, a block device driver is used in this case.

Ultimately, this data returns to the user's computer through the exact reverse of the receiving process.

Back on the Network

It would be nice if the user were connected directly to a public network, but users are generally connected to a private network. The server therefore does not know the user's location and sends the packet to the public address of the router that serves as the gateway. The Network Address Translation (NAT) Table at the gateway then forwards the packet from the corresponding external port to the internal port.

Back in the User Process

The user now receives the HTML file through the exact reverse of the transmission process. The browser parses this data and renders it for the user. If the parser encounters a tag that requires another resource, the browser goes through this entire process again to load that resource, modifies the DOM, and renders the page again. If there is JavaScript to execute, it executes it. Although there is naturally some blocking during this process, JavaScript runs on a single thread. An event loop is used for this purpose. (File-system and network operations actually work in a multithreaded manner, but you can think of the system as running on a single thread.) Put simply, the event loop consists of a call stack and a callback queue. When an asynchronous operation involving the DOM or Ajax is needed, the corresponding task is pushed onto the callback queue and execution simply moves on. Once the call stack is completely empty, one task is taken from the callback queue and executed.

Result

After all of these steps, pressing the Enter key causes Google's home page to appear in the browser.

References


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -