feat(functions): adding a functions module with pipe() #6143

guy-borderless · 2024-10-24T07:38:26Z

Hi,
I'd like to suggest adding an std module for utilities related to functions.

related: #4386

There are common manipulations to functions implemented in numerous npm packages and toolbelts, and using them where appropriate greatly improves code quality. It would be handy to have these in deno_std.
I want to suggest for starters the following two:

pipe
set_arguments (currying / dependency injection)

These two scenarios have some typescript knowledge behind them and it makes sense to reach for a proper utility.

Both of these have many different implementations in terms of code and typings. In my opinion, the implementation most in line with the pragmatic spirit of the std should avoid functional programming jargon for simplicity and accessibility to beginners.

For discussion in this PR I have included two implementations (without tests) that I think would suit the deno_std

pipe has good typescript performance and small implementation. I'm also using it in several projects.

set_arguments has a very non-academic intuitive signature that I think suites nicely with the deno_std. This signature does not expect a data-last coding convention, so it doesn't leak to the rest of the code. I have also provided a more classic functional programming implementation of this called curry to contrast.

I can significantly simplify the typing definition of set_arguments if you'd like me to go ahead with preparing this PR.

Since functions are an important primitive in JS, I'm sure more utilities will be added.

kt3k · 2024-10-24T10:22:51Z

Are pipe and set_arguments common in JS ecosystem? I rarely see them in various code bases.

I personally think inline arrow functions express the operation like set_arguments better.

const myNewFunc = set_arguments(myFunc, 1, undefined, 3);
// vs
const myNewFunc = (x: number) => myFunc(1, x, 3);

guy-borderless · 2024-10-24T12:20:01Z

Are pipe and set_arguments common in JS ecosystem? I rarely see them in various code bases.

I personally think inline arrow functions express the operation like set_arguments better.
const myNewFunc = set_arguments(myFunc, 1, undefined, 3);
// vs
const myNewFunc = (x: number) => myFunc(1, x, 3);

Both curry and pipe are very common and implemented by all lodash style toolbelts (including lodash of course), as well as many specific packages. Your example works because you have the number primitive. In cases of a business object, which you may or may not control, it's much nicer to use something like curry or set_arguments.

pipe is a common pattern, for example in ETL code when you have a lot of linear transformations, it makes the code much more readable, as it prevents spaghetti.

kt3k · 2024-10-24T12:48:19Z

Your example works because you have the number primitive.

I don't understand this. Inline arrow function example should work with any parameters available in that scope. I think that has the same capability as what set_arguments (or curry) can do.

BTW do you suggest curry or set_arguments? or both?

pipe is a common pattern, for example in ETL code when you have a lot of linear transformations, it makes the code much more readable, as it prevents spaghetti.

Can you elaborate on more specific examples where pipe is better for readability? The given example looks too trivial to replace with simple function composition.

const func = pipe(Math.abs, Math.sqrt, Math.floor);
//
const func = (num: number) => Math.floor(Math.sqrt(Math.abs(num)))

guy-borderless · 2024-10-25T09:04:55Z

Your example works because you have the number primitive.

I don't understand this. Inline arrow function example should work with any parameters available in that scope. I think that has the same capability as what set_arguments (or curry) can do.

BTW do you suggest curry or set_arguments? or both?

pipe is a common pattern, for example in ETL code when you have a lot of linear transformations, it makes the code much more readable, as it prevents spaghetti.

Can you elaborate on more specific examples where pipe is better for readability? The given example looks too trivial to replace with simple function composition.
const func = pipe(Math.abs, Math.sqrt, Math.floor);
//
const func = (num: number) => Math.floor(Math.sqrt(Math.abs(num)))

I see set_arguments as just a less functional version of curry, so if we want to be more strict it's either, depending on the policy.

Yes, of course, you can create an arrow function to inject parameters:

const fnWithFirstDeps = (dataArg: Parameters<typeof fn>[2]) => fn(dependency1, dependency2, dataArg)

I think it's a common and rather generic need, so having a utility that obviates the need to create a local function for it and improves readability is handy, while also encouraging a good coding practice (dependency injection). Sometimes different parts of the code inject different dependencies, having a utility encourages doing the injection locally and not creating another function or re-using the above function which entangles concerns.

const fnWithFirstDeps = setArguments(fn, dependency1, dependency2,)

pipe is useful in data processing. You don't want to compose many functions to create a pipe as this creates long bracket scopes and isn't readable. Here is a less trivial example:

const extracted = {
  bedrooms:  pipe(
      selectOne.attributes,
      parseMatchingText("bedrooms"),
      removeLabel,
      parseNumber,
    ),
  bathrooms:  pipe(
      selectOne.attributes,
      h.parseMatchingText("bathrooms"),
      removeLabel,
      parseNumber,
    ),
}

kt3k · 2024-10-28T06:41:58Z

I'm also not a fan of these typings. curry and setArguments seem having limits in number of arguments, while inline arrow function composition doesn't have such limitation. The type of pipe function looks extremely hard to understand. I guess the type error message will be also harder to understand with pipe than plain function composition.

kt3k · 2024-10-28T06:46:19Z

(BTW while I'm personally not in favor of this, we are open for adding this package if there's enough community support to this idea.)

guy-borderless · 2024-10-29T04:59:40Z

I'm also not a fan of these typings. curry and setArguments seem having limits in number of arguments, while inline arrow function composition doesn't have such limitation. The type of pipe function looks extremely hard to understand. I guess the type error message will be also harder to understand with pipe than simpler function composition.

Typing for a number of arguments is very common and gives a good experience. As far as I can see, it's a theoretical, not a practical issue.

iuioiua · 2024-11-14T00:05:31Z

This PR and #4386, which has been open since February, have had near-zero community support. Perhaps, it'd be best to have this done as a 3rd party package and see if it evolves enough to provide a stronger argument for its addition sometime in the future.

Kacaii · 2024-11-21T01:23:20Z

Honestly pipe would help a ton. Looking forward to this one!

kt3k · 2024-11-21T07:09:00Z

@guy-borderless Can you limit this PR to pipe function? Also can you write test cases for it?

guy-borderless · 2024-11-22T05:06:33Z

@guy-borderless Can you limit this PR to pipe function? Also can you write test cases for it?

Of course! On it.

codecov · 2024-12-18T06:37:39Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 96.32%. Comparing base (05b6d7e) to head (9fc723a).
Report is 183 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6143      +/-   ##
==========================================
+ Coverage   96.29%   96.32%   +0.03%     
==========================================
  Files         493      544      +51     
  Lines       39530    41610    +2080     
  Branches     5833     6313     +480     
==========================================
+ Hits        38065    40082    +2017     
- Misses       1423     1485      +62     
- Partials       42       43       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

guy-borderless · 2024-12-18T06:47:01Z

Per feedback, PR was limited to pipe() and tests were added.

BlackAsLight · 2024-12-18T07:02:03Z

Per feedback, PR was limited to pipe() and tests were added.

You also got to get all the tests to pass. Looks like the title isn't named right and you're missing the copyright notice in some of the files.

guy-borderless · 2024-12-18T07:15:59Z

Per feedback, PR was limited to pipe() and tests were added.

You also got to get all the tests to pass. Looks like the title isn't named right and you're missing the copyright notice in some of the files.

The PR naming issue seems to be because the functions module is new and so a scope doesn't exist for it. Added the copyright notice, thanks for mentioning it (didn't seem to cause a test to fail)

BlackAsLight · 2024-12-18T10:05:13Z

deno.json

+  "imports": {
+    "@deno/doc": "jsr:@deno/[email protected]",
+    "@deno/graph": "jsr:@deno/graph@^0.74",
+    "@std/archive": "jsr:@std/archive@^0.225.0",
+    "@std/assert": "jsr:@std/assert@^1.0.2",
+    "@std/async": "jsr:@std/async@^1.0.3",
+    "@std/bytes": "jsr:@std/bytes@^1.0.2-rc.3",
+    "@std/cli": "jsr:@std/cli@^1.0.3",
+    "@std/collections": "jsr:@std/collections@^1.0.5",
+    "@std/crypto": "jsr:@std/crypto@^1.0.2-rc.1",
+    "@std/csv": "jsr:@std/csv@^1.0.1",
+    "@std/data-structures": "jsr:@std/data-structures@^1.0.1",
+    "@std/datetime": "jsr:@std/datetime@^0.224.5",
+    "@std/dotenv": "jsr:@std/dotenv@^0.225.0",
+    "@std/encoding": "jsr:@std/encoding@^1.0.1",
+    "@std/expect": "jsr:@std/expect@^1.0.0",
+    "@std/fmt": "jsr:@std/fmt@^1.0.0",
+    "@std/front-matter": "jsr:@std/front-matter@^1.0.1",
+    "@std/fs": "jsr:@std/fs@^1.0.1",
+    "@std/html": "jsr:@std/html@^1.0.1",
+    "@std/http": "jsr:@std/http@^1.0.2",
+    "@std/ini": "jsr:@std/ini@^1.0.0-rc.3",
+    "@std/internal": "jsr:@std/internal@^1.0.1",
+    "@std/io": "jsr:@std/io@^0.224.4",
+    "@std/json": "jsr:@std/json@^1.0.0",
+    "@std/jsonc": "jsr:@std/jsonc@^1.0.0",
+    "@std/log": "jsr:@std/log@^0.224.5",
+    "@std/media-types": "jsr:@std/media-types@^1.0.2",
+    "@std/msgpack": "jsr:@std/msgpack@^1.0.0",
+    "@std/net": "jsr:@std/net@^1.0.0",
+    "@std/path": "jsr:@std/path@^1.0.2",
+    "@std/regexp": "jsr:@std/regexp@^1.0.0",
+    "@std/semver": "jsr:@std/semver@^1.0.1",
+    "@std/streams": "jsr:@std/streams@^1.0.1",
+    "@std/testing": "jsr:@std/testing@^1.0.0",
+    "@std/text": "jsr:@std/text@^1.0.2",
+    "@std/toml": "jsr:@std/toml@^1.0.0",
+    "@std/ulid": "jsr:@std/ulid@^1.0.0",
+    "@std/url": "jsr:@std/url@^0.225.0",
+    "@std/uuid": "jsr:@std/uuid@^1.0.0",
+    "@std/webgpu": "jsr:@std/webgpu@^0.224.5",
+    "@std/yaml": "jsr:@std/yaml@^1.0.2",
+    "automation/": "https://raw.githubusercontent.com/denoland/automation/0.10.0/",
+    "graphviz": "npm:node-graphviz@^0.1.1",
+    "npm:/typescript": "npm:[email protected]"
+  },


Why did you remove the importMap here for imports? This will probably need to be reverted.

A "@std/functions": "jsr:@std/functions@^0.1.0" will also probably need to be added to the importMap for the later lint error to fix itself.

BlackAsLight · 2024-12-18T10:10:11Z

functions/pipe.ts

These exports are also going to need JSDoc comments to satisfy the linter. You can do deno task ok locally to check that it satisfies the requirements.

BlackAsLight · 2024-12-18T10:15:39Z

The PR naming issue seems to be because the functions module is new and so a scope doesn't exist for it.

I believe a change needs to happen in .github/workflows/title.yml to recognise the functions scope. Possibly other places in .github/ as well.

kt3k · 2024-12-19T03:28:58Z

deno.json

@@ -89,6 +134,7 @@
    "./ulid",
    "./uuid",
    "./webgpu",
-    "./yaml"
+    "./yaml",
+    "./functions"


Can you keep this in alphabetical order?

kt3k · 2024-12-19T03:30:08Z

functions/mod.ts

+/**
+ * Pure functions for common tasks around collection types like arrays and
+ * objects.
+ *
+ * Inspired by
+ * {@link https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.collections/ | Kotlin's Collections}
+ * package and {@link https://lodash.com/ | Lodash}.
+ *
+ * ```ts
+ * import { intersect, sample, pick } from "@std/collections";
+ * import { assertEquals, assertArrayIncludes } from "@std/assert";
+ *
+ * const lisaInterests = ["Cooking", "Music", "Hiking"];
+ * const kimInterests = ["Music", "Tennis", "Cooking"];
+ *
+ * assertEquals(intersect(lisaInterests, kimInterests), ["Cooking", "Music"]);
+ *
+ * assertArrayIncludes(lisaInterests, [sample(lisaInterests)]);
+ *
+ * const cat = { name: "Lulu", age: 3, breed: "Ragdoll" };
+ *
+ * assertEquals(pick(cat, ["name", "breed"]), { name: "Lulu", breed: "Ragdoll"});
+ * ```


This doesn't describe this module

functions-module

3d67ed4

guy-borderless requested a review from kt3k as a code owner October 24, 2024 07:38

kt3k added the feedback welcome We want community's feedback on this issue or PR label Oct 24, 2024

limit pr to pipe() and adding tests

3a08d69

guy-borderless force-pushed the functions-module branch from c7df99b to 3a08d69 Compare December 18, 2024 06:39

guy-borderless changed the title ~~functions-module~~ feat(functions): adding a functions module with pipe() Dec 18, 2024

adding copyright notice

9fc723a

guy-borderless force-pushed the functions-module branch from bd18d57 to 9fc723a Compare December 18, 2024 07:17

BlackAsLight reviewed Dec 18, 2024

View reviewed changes

kt3k reviewed Dec 19, 2024

View reviewed changes

kt3k mentioned this pull request Dec 19, 2024

chore: update title check CI #6279

Merged

kt3k changed the title ~~feat(functions): adding a functions module with pipe()~~ feat(functions): adding a functions module with pipe()_ Dec 19, 2024

kt3k changed the title ~~feat(functions): adding a functions module with pipe()_~~ feat(functions): adding a functions module with pipe() Dec 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(functions): adding a functions module with pipe() #6143

feat(functions): adding a functions module with pipe() #6143

guy-borderless commented Oct 24, 2024 •

edited

Loading

kt3k commented Oct 24, 2024

guy-borderless commented Oct 24, 2024

kt3k commented Oct 24, 2024

guy-borderless commented Oct 25, 2024 •

edited

Loading

kt3k commented Oct 28, 2024 •

edited

Loading

kt3k commented Oct 28, 2024 •

edited

Loading

guy-borderless commented Oct 29, 2024

iuioiua commented Nov 14, 2024 •

edited

Loading

Kacaii commented Nov 21, 2024

kt3k commented Nov 21, 2024

guy-borderless commented Nov 22, 2024

codecov bot commented Dec 18, 2024 •

edited

Loading

guy-borderless commented Dec 18, 2024

BlackAsLight commented Dec 18, 2024

guy-borderless commented Dec 18, 2024

BlackAsLight Dec 18, 2024

BlackAsLight Dec 18, 2024

BlackAsLight Dec 18, 2024 •

edited

Loading

BlackAsLight commented Dec 18, 2024

kt3k Dec 19, 2024

kt3k Dec 19, 2024

feat(functions): adding a functions module with pipe() #6143

Are you sure you want to change the base?

feat(functions): adding a functions module with pipe() #6143

Conversation

guy-borderless commented Oct 24, 2024 • edited Loading

kt3k commented Oct 24, 2024

guy-borderless commented Oct 24, 2024

kt3k commented Oct 24, 2024

guy-borderless commented Oct 25, 2024 • edited Loading

kt3k commented Oct 28, 2024 • edited Loading

kt3k commented Oct 28, 2024 • edited Loading

guy-borderless commented Oct 29, 2024

iuioiua commented Nov 14, 2024 • edited Loading

Kacaii commented Nov 21, 2024

kt3k commented Nov 21, 2024

guy-borderless commented Nov 22, 2024

codecov bot commented Dec 18, 2024 • edited Loading

Codecov Report

guy-borderless commented Dec 18, 2024

BlackAsLight commented Dec 18, 2024

guy-borderless commented Dec 18, 2024

BlackAsLight Dec 18, 2024

Choose a reason for hiding this comment

BlackAsLight Dec 18, 2024

Choose a reason for hiding this comment

BlackAsLight Dec 18, 2024 • edited Loading

Choose a reason for hiding this comment

BlackAsLight commented Dec 18, 2024

kt3k Dec 19, 2024

Choose a reason for hiding this comment

kt3k Dec 19, 2024

Choose a reason for hiding this comment

guy-borderless commented Oct 24, 2024 •

edited

Loading

guy-borderless commented Oct 25, 2024 •

edited

Loading

kt3k commented Oct 28, 2024 •

edited

Loading

kt3k commented Oct 28, 2024 •

edited

Loading

iuioiua commented Nov 14, 2024 •

edited

Loading

codecov bot commented Dec 18, 2024 •

edited

Loading

BlackAsLight Dec 18, 2024 •

edited

Loading