[Bug]: Used the browser when asked to review a github PR instead of using the api

[morganmcg1]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Describe the bug and reproduction steps

That task was is getting stuck pretty quickly with this prompt below to review some tests in a PR. It used the browser to have a look at the PR, but probably should have just git cloned and started work from there.

Prompt:

Review the tests for my scorers in this PR and suggest ways to improve.

The weave library is a LLM Ops library for tracking and evaluating your AI projects. One aspect of it are Scorers, which are used to grade the outputs of your model. Some are based on ML models, some are LLM judges and some are just statistical calculations.

Look at the new tests added to this repo and suggest ways to improve it.


https://github.com/wandb/weave/pull/3006

OpenHands Installation

app.all-hands.dev

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

[enyst]

I've seen it use the browser for github, unless I prompt it to remind it about the github token it has, and the ability to use the API. It shouldn't get stuck over it, anyway, that might be a separate issue and stuck how.

It's strange though. This prompt should have told it to use the API. 🤔