Implement res.sendVideo fn #5770

binarykitchen · 2024-07-17T22:55:30Z

binarykitchen
Jul 17, 2024

I see too many use the sendFile fn the wrong way for delivering videos to the clients.

Why not implement a new sendVideo() fn with proper streaming and content-range calculations? Wondering if this has ever been raised before?

If not, do you think a PR would make sense and it's the right place here to do so?

Thanks for your thoughts.

wesleytodd · 2024-07-17T23:00:34Z

wesleytodd
Jul 17, 2024
Maintainer

Hey @binarykitchen, I don't think it has been brought up before but as someone who has worked quite extensively on both the server and client with video content (before joining Netflix the startup I was at was an SVOD service which pivoted to a live streaming service) I don't think it is something which can easily be taken on. If you could bring some more examples of use cases you have seen where folks incorrectly used sendFile it might help to make this conversation more grounded, because the general case of "sending video" is so broad a topic it is nearly impossible to know what we would implement that could help people.

0 replies

binarykitchen · 2024-07-17T23:53:41Z

binarykitchen
Jul 17, 2024
Author

Thanks @wesleytodd, here's a "horrible" draft example of what I'm after:

function streamVideo(
  videoPath: string,
  type: string,
  req: Request,
  res: Response,
  next: NextFunction,
) {
  try {
    log.debug(`Streaming video ${videoPath} now ...`);

    const stat = fs.statSync(videoPath);
    const fileSize = stat.size;
    const range = req.get("range");

    let status: number;
    let contentLength: number;

    const head = {};
    const streamOptions = {};

    if (range) {
      const parts = range.replace(/bytes=/, "").split("-");
      const firstPart = parts[0];

      if (firstPart === undefined) {
        throw new Error(`Unable to grab the first part of the video file ${videoPath}.`);
      }

      const start = parseInt(firstPart, 10);
      const end = parts[1] ? parseInt(parts[1], 10) : fileSize - 1;

      // Aka chunk size
      contentLength = end - start + 1;

      status = 206;

      head["Content-Range"] = `bytes ${start}-${end}/${fileSize}`;
      head["Accept-Ranges"] = "bytes";

      streamOptions["start"] = start;
      streamOptions["end"] = end;
    } else {
      // Unable to split it in ranges, so as a fallback, just stream the whole
      status = 200;
      contentLength = fileSize;
    }

    const stream = fs.createReadStream(videoPath, streamOptions);

    head["Content-Type"] = `video/${type}`;
    head["Content-Length"] = contentLength;

    res.writeHead(status, head);

    stream
      .pipe(res)
      .on("close", () => {
        next();
      })
      .on("error", (err) => {
        next(err);
      });
  } catch (exc) {
    next(exc)
  }
}

Something like that. It can be improved further, e.g. grab the mime type of the file and allow streaming any multimedia files, not just videos.

4 replies

wesleytodd Jul 17, 2024
Maintainer

Oh interesting. So, sendFile uses the send package under the hood, which should support range requests like this. So are we certain here that it is not doing what you need?

binarykitchen Jul 18, 2024
Author

Oh nice, didn't know about that send package. Doesn't Express.js come with this under the hood or do I have to install it separately?

binarykitchen Jul 18, 2024
Author

Just tried this send package but afraid no, it worsens it.

With send:

const stream = send(req, videoPath, { cacheControl: false });

stream
  .on("error", (err) => {
    next(err);
  })
  .on("close", () => {
    next();
  })
  .on("end", () => {
    next();
  })
  .pipe(res);

My above custom code does not have those failing HTTP requests:

Therefore, still unsure here ...

krzysdz Jul 18, 2024

@binarykitchen send is used by res.sendFile (see below), so usually there is no need to write additional code - res.sendFile(path) is sufficient in most cases.

There are multiple requests, because I was seeking through the video. All responses were successful, with status code 206.

Code from the screenshot

const path = require("node:path");
const express = require("express");

const app = express();

app.get("/", (req, res) => {
    res.send(`<!DOCTYPE html><html><body><video controls src="video"></video></body></html>`);
});
app.get("/video", (req, res) => {
    res.sendFile(path.resolve(path.join(__dirname, "./vid.mp4")));
});

const server = app.listen(0, "127.0.0.1", () => {
    const {port, address} = server.address();
    console.log(`Listening on http://${address}:${port}/`);
});

The errors on your screenshot refer to HTTP2, but HTTP2 does not work with Express and Chrome shouldn't even try to use it, unless you're using reverse proxy, which somehow causes these errors.

This is inside res.sendFile(); send function in line 446 is from send package:

express/lib/response.js

Lines 444 to 457 in 2ec589c

    
           // create file stream 
        
           var pathname = encodeURI(path); 
        
           var file = send(req, pathname, opts); 
        
           // transfer 
        
           sendfile(res, file, opts, function (err) { 
        
             if (done) return done(err); 
        
             if (err && err.code === 'EISDIR') return next(); 
        
             // next() all but write errors 
        
             if (err && err.code !== 'ECONNABORTED' && err.syscall !== 'write') { 
        
               next(err); 
        
             } 
        
           });

The sendfile function used by res.sendFile() just sets headers, pipes the file stream to response and handles errors:

express/lib/response.js

Lines 1051 to 1140 in 2ec589c

    
           // pipe the send file stream 
        
           function sendfile(res, file, options, callback) { 
        
             var done = false; 
        
             var streaming; 
        
             // request aborted 
        
             function onaborted() { 
        
               if (done) return; 
        
               done = true; 
        
               var err = new Error('Request aborted'); 
        
               err.code = 'ECONNABORTED'; 
        
               callback(err); 
        
             } 
        
             // directory 
        
             function ondirectory() { 
        
               if (done) return; 
        
               done = true; 
        
               var err = new Error('EISDIR, read'); 
        
               err.code = 'EISDIR'; 
        
               callback(err); 
        
             } 
        
             // errors 
        
             function onerror(err) { 
        
               if (done) return; 
        
               done = true; 
        
               callback(err); 
        
             } 
        
             // ended 
        
             function onend() { 
        
               if (done) return; 
        
               done = true; 
        
               callback(); 
        
             } 
        
             // file 
        
             function onfile() { 
        
               streaming = false; 
        
             } 
        
             // finished 
        
             function onfinish(err) { 
        
               if (err && err.code === 'ECONNRESET') return onaborted(); 
        
               if (err) return onerror(err); 
        
               if (done) return; 
        
               setImmediate(function () { 
        
                 if (streaming !== false && !done) { 
        
                   onaborted(); 
        
                   return; 
        
                 } 
        
                 if (done) return; 
        
                 done = true; 
        
                 callback(); 
        
               }); 
        
             } 
        
             // streaming 
        
             function onstream() { 
        
               streaming = true; 
        
             } 
        
             file.on('directory', ondirectory); 
        
             file.on('end', onend); 
        
             file.on('error', onerror); 
        
             file.on('file', onfile); 
        
             file.on('stream', onstream); 
        
             onFinished(res, onfinish); 
        
             if (options.headers) { 
        
               // set headers on successful transfer 
        
               file.on('headers', function headers(res) { 
        
                 var obj = options.headers; 
        
                 var keys = Object.keys(obj); 
        
                 for (var i = 0; i < keys.length; i++) { 
        
                   var k = keys[i]; 
        
                   res.setHeader(k, obj[k]); 
        
                 } 
        
               }); 
        
             } 
        
             // pipe 
        
             file.pipe(res); 
        
           }

binarykitchen · 2024-07-19T01:28:22Z

binarykitchen
Jul 19, 2024
Author

Oh, good, analysis @krzysdz thanks. So it's because I'm using HTTP2/SPDY servers?

Hmmm, this package hasn't been maintained for a long time and I do not see a need to use it in my app. Will consider removing it ...
Or it is because Nginx is my reverse proxy? Hmm....

0 replies

binarykitchen · 2024-07-19T02:59:59Z

binarykitchen
Jul 19, 2024
Author

I concur. SPDY was the troubled child. With classic Node.js HTTPS servers and sendFile all works fine.

Thanks for the good analysis and feedback, everyone.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement res.sendVideo fn #5770

{{title}}

Replies: 4 comments 4 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

Implement res.sendVideo fn #5770

binarykitchen Jul 17, 2024

Replies: 4 comments · 4 replies

wesleytodd Jul 17, 2024 Maintainer

binarykitchen Jul 17, 2024 Author

wesleytodd Jul 17, 2024 Maintainer

binarykitchen Jul 18, 2024 Author

binarykitchen Jul 18, 2024 Author

krzysdz Jul 18, 2024

binarykitchen Jul 19, 2024 Author

binarykitchen Jul 19, 2024 Author

binarykitchen
Jul 17, 2024

Replies: 4 comments 4 replies

wesleytodd
Jul 17, 2024
Maintainer

binarykitchen
Jul 17, 2024
Author

wesleytodd Jul 17, 2024
Maintainer

binarykitchen Jul 18, 2024
Author

binarykitchen Jul 18, 2024
Author

binarykitchen
Jul 19, 2024
Author

binarykitchen
Jul 19, 2024
Author