Implementing HTTP webcam live streaming

2020-08-11 04:03:18 | English, Korean

Translated with the help of ChatGPT and Google Translator

While working on a project this time, I had to implement live streaming. You need to receive video from a camera and view it in a web browser. After Googling, I couldn't find a solution that would cleanly implement this. So I decided to implement this myself.

The biggest problem with video, especially live streaming, is how to receive the video and how to stream it. Fortunately, I had had similar experiences several times before and knew of two solutions.

  1. Using OpenCV, etc., convert camera video to image and send to client. The client continuously updates and displays images. Therefore, in reality, it is not a video but just a rapidly changing image.
  2. Use FFmpeg.

At first glance, solution number 1 seems quite simple and ignorant. However, in practice, the video quality and FPS are quite good, it is easy to implement (there is no need to go through the trouble of encoding), and it is especially useful when you need to do some processing on the video before sending it. However, it has the disadvantage of not being able to transmit sound. So this time I decided to implement solution number 2.

What we did before

When I used FFmpeg in the past, I didn't know much about the protocol that enabled real-time video transmission. So I had to take a fairly low-level approach, and used the following method.

  1. Convert the webcam video to webm format using FFmpeg and pipe the output stream to stdout to receive it from the Node.js server.
  2. The Node.js server stores the received stream in a buffer.
  3. When the buffer is full, put the buffer into the queue.
  4. If the client requests video, the server takes the buffer out of the queue and transmits it to the client.
  5. The client adds a buffer to the video tag using HTML5’s Media Source API.

Looking at the commit record, this was implemented in February 2019, so it's roughly right before I graduated from high school. Now that I look at it, it is a very inefficient implementation. In the first place, two people cannot receive streaming at the same time.

What I did this time

This time, instead of doing that, I decided to take full advantage of FFmpeg's features. The code is really simple, so you might understand it better if you look at the source code directly.

Below is a copy of the FFmpeg command executed in Node.js. If you have FFmpeg installed, you can just type it straight into the terminal and it will work.

ffmpeg -f v4l2 -i /dev/video0 -c:v libx264 -crf 23 -pix_fmt yuv420p -hls_time 2 -hls_list_size 5 -hls_delete_threshold 1 -hls_flags delete_segments -f hls public/video.m3u8

Let’s look at the factors one by one. There is also an explanation in the source code.

Among the above options, -hls_list_size, -hls_delete_threshold, and -hls_flags delete_segments are a set of three options. The explanation is a bit complicated, but the -hls_flags delete_segments option is an option to delete old segments as mentioned above, and the -hls_delete_threshold option determines how many segments not included in .m3u8 will be considered old. This is an option to specify.

For example, assume hls_list_size is 7 and hls_delete_threshold is 3. Then, .m3u8 will have information about a total of 7 segments: n, n+1, n+2...n+6. At this time, because hls_delete_threshold is 3, segments up to n-1, n-2, and n-3 are not considered old, but from n-4 onwards are considered old. Therefore, there will always be 10 .ts files.

The reason why the hls_delete_threshold option is necessary is because although a specific segment has been deleted from the segment list of .m3u8, there may be clients that continue to download the segment due to problems such as Internet speed.

If you give this option, the files will be updated in real time, and the client can play real-time video as if playing a regular HLS file. Below is a screenshot.


You can view the video taken from the webcam by streaming it as HTML. (I deleted the URL because I am using a private server.)


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -