Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SocketStream.receive sometimes returns bytearray objects on Windows #776

Closed
2 tasks done
jonathanslenders opened this issue Aug 30, 2024 · 2 comments · Fixed by #777
Closed
2 tasks done

SocketStream.receive sometimes returns bytearray objects on Windows #776

jonathanslenders opened this issue Aug 30, 2024 · 2 comments · Fixed by #777
Labels
bug Something isn't working

Comments

@jonathanslenders
Copy link
Contributor

Things to check first

  • I have searched the existing issues and didn't find my bug already reported there

  • I have checked that my bug is still present in the latest release

AnyIO version

4.4.0

Python version

3.11

What happened?

StreamProtocol, here: https://github.com/agronholm/anyio/blob/master/src/anyio/_backends/_asyncio.py#L1075
incorrectly assumes that it will only get bytes objects as an argument. Unfortunately, on Windows, that's not the case. For the ProactorEventLoop, we'll get bytearray objects here.

See:

I think the quickest fix for anyio is to accept bytearray as well, and convert it into bytes, regardless of what typeshed/cpython do.

How can we reproduce the bug?

(Sorry, I don't have access to a Windows machine right now to create an example, but please let me know if an example script is needed.)
@jonathanslenders jonathanslenders added the bug Something isn't working label Aug 30, 2024
@jonathanslenders
Copy link
Contributor Author

It looks like it was fixed in cpython v3.12.0a4 thanks to this commit:
python/cpython@1bb68ba

However, it would still be good to handle the issue here for Python 3.10 and 3.11. (3.9 doesn't have the issue).

agronholm added a commit that referenced this issue Aug 30, 2024
@agronholm
Convert bytearray to bytes in StreamProtocol.receive()
e9ed945 
Fixes #776.
@agronholm agronholm mentioned this issue Aug 30, 2024
Merged
3 tasks
@agronholm
Copy link
Owner

Review welcome. As I mentioned in the CPython issue, I was able to locally reproduce this even on the latest 3.12 release.

agronholm added a commit that referenced this issue Sep 1, 2024
@agronholm
Convert bytearray to bytes in StreamProtocol.receive() (#777)
bc962ef 
Fixes #776.
mkjpryor pushed a commit to azimuth-cloud/azimuth that referenced this issue Sep 23, 2024
@dependabot
Bump anyio from 4.4.0 to 4.5.0 in /api (#297)
c5f4fa3 
Bumps [anyio](https://github.com/agronholm/anyio) from 4.4.0 to 4.5.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/anyio/releases">anyio's
releases</a>.</em></p>
<blockquote>
<h2>4.5.0</h2>
<ul>
<li>Improved the performance of <code>anyio.Lock</code> and
<code>anyio.Semaphore</code> on asyncio (even up to 50 %)</li>
<li>Added the <code>fast_acquire</code> parameter to
<code>anyio.Lock</code> and <code>anyio.Semaphore</code> to further
boost performance at the expense of safety (<code>acquire()</code> will
not yield control back if there is no contention)</li>
<li>Added support for the <code>from_uri()</code>,
<code>full_match()</code>, <code>parser</code> methods/properties in
<code>anyio.Path</code>, newly added in Python 3.13 (<a
href="https://redirect.github.com/agronholm/anyio/issues/737">#737</a>)</li>
<li>Added support for more keyword arguments for
<code>run_process()</code> and <code>open_process()</code>:
<code>startupinfo</code>, <code>creationflags</code>,
<code>pass_fds</code>, <code>user</code>, <code>group</code>,
<code>extra_groups</code> and <code>umask</code> (<a
href="https://redirect.github.com/agronholm/anyio/issues/742">#742</a>)</li>
<li>Improved the type annotations and support for <code>PathLike</code>
in <code>run_process()</code> and <code>open_process()</code> to allow
for path-like arguments, just like <code>subprocess.Popen</code></li>
<li>Changed the <code>ResourceWarning</code> from an unclosed memory
object stream to include its address for easier identification</li>
<li>Changed <code>start_blocking_portal()</code> to always use daemonic
threads, to accommodate the &quot;loitering event loop&quot; use
case</li>
<li>Bumped the minimum version of Trio to v0.26.1</li>
<li>Fixed <code>__repr__()</code> of
<code>MemoryObjectItemReceiver</code>, when <code>item</code> is not
defined (<a
href="https://redirect.github.com/agronholm/anyio/pulls/767">#767</a>;
PR by <a
href="https://github.com/Danipulok"><code>@​Danipulok</code></a>)</li>
<li>Fixed <code>to_process.run_sync()</code> failing to initialize if
<code>__main__.__file__</code> pointed to a file in a nonexistent
directory (<a
href="https://redirect.github.com/agronholm/anyio/issues/696">#696</a>)</li>
<li>Fixed <code>AssertionError: feed_data after feed_eof</code> on
asyncio when a subprocess is closed early, before its output has been
read (<a
href="https://redirect.github.com/agronholm/anyio/issues/490">#490</a>)</li>
<li>Fixed <code>TaskInfo.has_pending_cancellation()</code> on asyncio
not respecting shielded scopes (<a
href="https://redirect.github.com/agronholm/anyio/issues/771">#771</a>;
PR by <a
href="https://github.com/gschaffner"><code>@​gschaffner</code></a>)</li>
<li>Fixed <code>SocketStream.receive()</code> returning
<code>bytearray</code> instead of <code>bytes</code> when using asyncio
with <code>ProactorEventLoop</code> (Windows) (<a
href="https://redirect.github.com/agronholm/anyio/issues/776">#776</a>)</li>
<li>Fixed quitting the debugger in a pytest test session while in an
active task group failing the test instead of exiting the test session
(because the exit exception arrives in an exception group)</li>
<li>Fixed support for Linux abstract namespaces in UNIX sockets that was
broken in v4.2 (<a
href="https://redirect.github.com/agronholm/anyio/issues/781">#781</a>
&lt;<a
href="https://redirect.github.com/agronholm/anyio/issues/781">agronholm/anyio#781</a>&gt;_;
PR by <a
href="https://github.com/tapetersen"><code>@​tapetersen</code></a>)</li>
<li>Fixed <code>KeyboardInterrupt</code> (ctrl+c) hanging the asyncio
pytest runner</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/agronholm/anyio/blob/master/docs/versionhistory.rst">anyio's
changelog</a>.</em></p>
<blockquote>
<h1>Version history</h1>
<p>This library adheres to <code>Semantic Versioning 2.0
&lt;http://semver.org/&gt;</code>_.</p>
<p><strong>4.5.0</strong></p>
<ul>
<li>Improved the performance of <code>anyio.Lock</code> and
<code>anyio.Semaphore</code> on asyncio (even up
to 50 %)</li>
<li>Added the <code>fast_acquire</code> parameter to
<code>anyio.Lock</code> and <code>anyio.Semaphore</code> to
further boost performance at the expense of safety
(<code>acquire()</code> will not yield
control back if there is no contention)</li>
<li>Added support for the <code>from_uri()</code>,
<code>full_match()</code>, <code>parser</code> methods/properties
in <code>anyio.Path</code>, newly added in Python 3.13
(<code>[#737](agronholm/anyio#737)
&lt;https://github.com/agronholm/anyio/issues/737&gt;</code>_)</li>
<li>Added support for more keyword arguments for
<code>run_process()</code> and <code>open_process()</code>:
<code>startupinfo</code>, <code>creationflags</code>,
<code>pass_fds</code>, <code>user</code>, <code>group</code>,
<code>extra_groups</code> and <code>umask</code>
(<code>[#742](agronholm/anyio#742)
&lt;https://github.com/agronholm/anyio/issues/742&gt;</code>_)</li>
<li>Improved the type annotations and support for <code>PathLike</code>
in <code>run_process()</code> and
<code>open_process()</code> to allow for path-like arguments, just like
<code>subprocess.Popen</code></li>
<li>Changed the <code>ResourceWarning</code> from an unclosed memory
object stream to include its
address for easier identification</li>
<li>Changed <code>start_blocking_portal()</code> to always use daemonic
threads, to accommodate the
&quot;loitering event loop&quot; use case</li>
<li>Bumped the minimum version of Trio to v0.26.1</li>
<li>Fixed <code>__repr__()</code> of
<code>MemoryObjectItemReceiver</code>, when <code>item</code> is not
defined
(<code>[#767](agronholm/anyio#767)
&lt;https://github.com/agronholm/anyio/pulls/767&gt;</code>_; PR by <a
href="https://github.com/Danipulok"><code>@​Danipulok</code></a>)</li>
<li>Fixed <code>to_process.run_sync()</code> failing to initialize if
<code>__main__.__file__</code> pointed
to a file in a nonexistent directory
(<code>[#696](agronholm/anyio#696)
&lt;https://github.com/agronholm/anyio/issues/696&gt;</code>_)</li>
<li>Fixed <code>AssertionError: feed_data after feed_eof</code> on
asyncio when a subprocess is
closed early, before its output has been read
(<code>[#490](agronholm/anyio#490)
&lt;https://github.com/agronholm/anyio/issues/490&gt;</code>_)</li>
<li>Fixed <code>TaskInfo.has_pending_cancellation()</code> on asyncio
not respecting shielded
scopes (<code>[#771](agronholm/anyio#771)
&lt;https://github.com/agronholm/anyio/issues/771&gt;</code>_; PR by <a
href="https://github.com/gschaffner"><code>@​gschaffner</code></a>)</li>
<li>Fixed <code>SocketStream.receive()</code> returning
<code>bytearray</code> instead of <code>bytes</code> when
using asyncio with <code>ProactorEventLoop</code> (Windows)
(<code>[#776](agronholm/anyio#776)
&lt;https://github.com/agronholm/anyio/issues/776&gt;</code>_)</li>
<li>Fixed quitting the debugger in a pytest test session while in an
active task group
failing the test instead of exiting the test session (because the exit
exception
arrives in an exception group)</li>
<li>Fixed support for Linux abstract namespaces in UNIX sockets that was
broken in v4.2
(<a
href="https://redirect.github.com/agronholm/anyio/issues/781">#781</a>
<a
href="https://redirect.github.com/agronholm/anyio/issues/781">agronholm/anyio#781</a>_;
PR by <a
href="https://github.com/tapetersen"><code>@​tapetersen</code></a>)</li>
<li>Fixed <code>KeyboardInterrupt</code> (ctrl+c) hanging the asyncio
pytest runner</li>
</ul>
<p><strong>4.4.0</strong></p>
<ul>
<li>Added the <code>BlockingPortalProvider</code> class to aid with
constructing synchronous
counterparts to asynchronous interfaces that would otherwise require
multiple blocking</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/agronholm/anyio/commit/7f35ce79e3eb9db2a2731619b48fe289d8dd872f"><code>7f35ce7</code></a>
Bumped up the version</li>
<li><a
href="https://github.com/agronholm/anyio/commit/108cc8300aa5a0d5be2834a8bbd3eb1938b8c104"><code>108cc83</code></a>
[pre-commit.ci] pre-commit autoupdate (<a
href="https://redirect.github.com/agronholm/anyio/issues/788">#788</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/d1aea98c4f552d04458e49f1484b9c018f2a3ca8"><code>d1aea98</code></a>
Fixed KeyboardInterrupt hanging the asyncio test runner (<a
href="https://redirect.github.com/agronholm/anyio/issues/779">#779</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/c1aff53d8ae5ddc2cf1b8558614a7a2d60c12518"><code>c1aff53</code></a>
[pre-commit.ci] pre-commit autoupdate (<a
href="https://redirect.github.com/agronholm/anyio/issues/785">#785</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/89d8b4c9609e22306c597cea98ecf3fba45fb663"><code>89d8b4c</code></a>
Use sphinx_rtd_theme also as an extension</li>
<li><a
href="https://github.com/agronholm/anyio/commit/4e9f18dfaa6e7db55231a35fcd93e18c343519a4"><code>4e9f18d</code></a>
Enabled uvloop to be used in the test suite on Python 3.13</li>
<li><a
href="https://github.com/agronholm/anyio/commit/7de644113250a817dd3224e6df9b3658cb09802d"><code>7de6441</code></a>
Pin Sphinx to a compatible version with sphinx-rtd-theme</li>
<li><a
href="https://github.com/agronholm/anyio/commit/41647f4ba7389dc756b0401033af1e3f41ad8a60"><code>41647f4</code></a>
Fixed feed_data after feed_eof assertion errors on asyncio (<a
href="https://redirect.github.com/agronholm/anyio/issues/752">#752</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/38890e607c726be6ea29ea2af445e8df43df5725"><code>38890e6</code></a>
Accept abstract namespace paths for unix domain sockets (<a
href="https://redirect.github.com/agronholm/anyio/issues/782">#782</a>)</li>
<li><a
href="https://github.com/agronholm/anyio/commit/0c8ad519e136c4b136b51333863966a445080a97"><code>0c8ad51</code></a>
Delegated the implementations of Lock and Semaphore to the async backend
clas...</li>
<li>Additional commits viewable in <a
href="https://github.com/agronholm/anyio/compare/4.4.0...4.5.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=anyio&package-manager=pip&previous-version=4.4.0&new-version=4.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Convert bytearray to bytes in StreamProtocol.receive() agronholm/anyio
2 participants
@agronholm @jonathanslenders