I also had a discussion and think it is a good idea. There was this effort having guides instead of just having the API docs plus some topic, where this could have been included. We should definitely put this close to the API docs, since I think it is crucial to effectively using streams.

Huge agreement from me. Particularly now that streams3 can be relied upon I think some 'canonical' documentation about how to run with that version of the API for those unfamiliar would be great. If we can muster a group effort this is something I'd quite like to get involved with.

Perhaps we could start a common repo or something and work on something like this in the open? You mentioned other modules, I've long thought we have a discoverability problem of stream modules conforming to the modern APIs - what about starting a list with names and uses based on the community? Final idea for a specific guide, could we could solicit examples?

Can I be of any use here? I am new to Open-Source and I had difficulty understanding the streams docs too.

Hey there: @Tanuja-Sawant are you writing this doc? If not, I would like help here. I have not contributed anything to node, but under my former username, I made contributions to other projects (@oddlyzen). I'd like my first contribution with this username to be here. Cool ❓

@bitpimpin given the amount of time that has passed since that comment, I'd say go ahead and do it if you want :)

@aqrln @mcollina Can I pick this up? I think it'll be a good first issue :)

@wuweiweiwu that'd be fantastic. We also have https://nodejs.org/en/docs/guides/backpressuring-in-streams, so you can focus more on the userland side (and also mention some of the important modules on npm).

@wuweiweiwu absolutely! Keep in mind that it may be quite a chunk of work on its own, so I'd recommend to open a work-in-progress PR early and proceed gradually.

@mcollina @aqrln Sounds good! Seems like a lot to read. Ill get more familiar and open a WIP PR soon :)

hey guys, I'd like to take up this PR if its available

Hey folks, this has been opened for a while and there seems there is no activity on it atm, so I'd like to have a go at it. I'll start posting an index below so we can hopefully start a conversation about what we want to be included, and after that maybe get a draft.

1. What are Streams?
2. Why use Streams?
   2.1 Note on performance
3. Streams vs Web streams
4. Stream history (classic, streams1, streams2, streams3)
5. Stream types (with events)
   5.1. Readable
   5.2. Writable
   5.3. Transform
   5.4. Duplex
6. How to operate with streams
   6.1. .pipe() (with caveats on error handling)
   6.2. pipeline (+ async)
   6.3. async iterators (+ using generators as transforms)
   6.4. object mode
   6.5. experimental APIS (e.g .map(), .reduce()...
   6.6 mention to backpressure (small overview with link to dedicated guide)
7. Streams in the standard library
   7.1. fs
   7.2. http
   7.3. process.std(in|out|err)
8. Interoperability of streams and Web Streams

• Not sure if it might be interesting to also have an overview of web streams in the same guide
• There are plenty other places in the standard library where streams are used (net, tls, crypto, zlib...) I tried to pick the most used ones but feel free to propose any other
• I intentionally left out mention to packages because I think there is some consensus around avoiding to endorse particular packages

EDIT: I thought adding 6.5 now that https://github.com/tc39/proposal-iterator-helpers is at stage 3 and the async version at stage 2 would be interesting

It's just a draft so any suggestions are really appreciated. There is a lot of literature already available on this topic, so my idea is to gather it in one place and format it so it can live in the node page.

I'll start posting an index below so we can hopefully start a conversation about what we want to be included, and after that maybe get a draft.

I want to know if this is the type of thing @mcollina had in mind or not, but it looks good to me.

I 100% forgot about this issue.

I cover most of this content in https://blog.platformatic.dev/a-guide-to-reading-and-writing-nodejs-streams and https://www.youtube.com/watch?v=edB964-YYpE&t=1s. Feel free to lift as much as you need and add me as a co-author of that piece/commit. A backlink would be appreciated.

I would recommend to stay away from Stage 3 and Stage 2 proposals in the guides, as they aren't kept very much up-to-date.

I cover most of this content in https://blog.platformatic.dev/a-guide-to-reading-and-writing-nodejs-streams and https://www.youtube.com/watch?v=edB964-YYpE&t=1s. Feel free to lift as much as you need and add me as a co-author of that piece/commit. A backlink would be appreciated.

tysm

I would recommend to stay away from Stage 3 and Stage 2 proposals in the guides, as they aren't kept very much up-to-date.

I'll leave it out and try to stay up to date so we may include them when they are stable in Node