Autobuild with working-directory doesn't always work

[melwynpkb]

Hi,

I'm working in a monorepo setup and was using the following CodeQL setup to run CodeQL on a sub directory
`

- name: Initialize CodeQL
  uses: github/codeql-action/init@v3
  with:
    languages: ${{ matrix.language }}
    build-mode: ${{ matrix.build-mode }}
    config: |
      queries:
        # See https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning#using-queries-in-ql-packs
        - name: Security and maintainability
          uses: security-and-quality
              
- if: matrix.language == 'csharp'
  name: Build solution
  uses: github/codeql-action/autobuild@v3
  with:
    working-directory: code/<redacted>      

- name: Perform CodeQL Analysis
  uses: github/codeql-action/analyze@v3
  with:
    category: "/language:${{matrix.language}}"`

Initially, in some runs, the autobuild step correctly picked up a .sln file from the specified working directory but since yesterday, in several runs, the option can be seen to be overriden by the autobuild script and it uses the github.workspace directory i.e., the root of the monorepo.

From the logs, I can see a difference in the environment variables and how the autobuild.sh is run

not-working.log
working.log

I'm using manual build as a workaround for now.

[aibaars]

That's very curious indeed, thanks for reporting! In the non-working log I spotted some lines that are not present in the other log. I'm not sure how that is possible, because both jobs seem to run with the same CodeQL version (2.17.3). Perhaps something changed in the github/codeql-action/autobuild action. Could you check in your logs which version and commit SHA of the github/codeql-action was used.

2024-05-29T23:08:02.3798433Z ##[group]Attempting to automatically build csharp code
2024-05-29T23:08:02.3808922Z [command]/opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/codeql database trace-command --use-build-mode /home/runner/work/_temp/codeql_databases/csharp
2024-05-29T23:08:02.3932022Z Picked up JAVA_TOOL_OPTIONS:  -Dhttp.keepAlive=false -Dmaven.wagon.http.pool=false
2024-05-29T23:08:03.2916771Z Running command in /home/runner/work/<redacted>/<redacted>: [/opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/csharp/tools/autobuild.sh]
2024-05-29T23:08:05.0520089Z [2024-05-29 23:08:05] [build-stdout] CodeQL C# autobuilder
2024-05-29T23:08:05.4171776Z [2024-05-29 23:08:05] [build-stdout] Working directory: /home/runner/work/<redacted>/<redacted>

Could you try re-running the failed working with debug logging enabled? That should make the log more verbose and it also attaches a debug artifact to the job with much more data. I see you redacted your project name, I assume this is a private repository. You may want to create an enterprise support ticket and attach the more verbose logs there instead of using a public issue ticket.

[aibaars]

Comparing the logs in more details I spotted:

The working log has a lot of additional environment variables indicating that the CodeQL tracer was started by the codeql-action/init step.

 CODEQL_SCRATCH_DIR: /home/runner/work/_temp/codeql_databases/working
  CODEQL_VERBOSITY: warnings
  CODEQL_DIST: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql
  CODEQL_PLATFORM: linux64
  CODEQL_PLATFORM_DLL_EXTENSION: .so
  CODEQL_JAVA_HOME: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/java
  CODEQL_EXTRACTOR_CSHARP_ROOT: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/csharp
  CODEQL_EXTRACTOR_CSHARP_WIP_DATABASE: /home/runner/work/_temp/codeql_databases/csharp
  CODEQL_EXTRACTOR_CSHARP_DIAGNOSTIC_DIR: /home/runner/work/_temp/codeql_databases/csharp/diagnostic/extractors/csharp
  CODEQL_EXTRACTOR_CSHARP_LOG_DIR: /home/runner/work/_temp/codeql_databases/csharp/log
  CODEQL_EXTRACTOR_CSHARP_SCRATCH_DIR: /home/runner/work/_temp/codeql_databases/csharp/working
  CODEQL_EXTRACTOR_CSHARP_TRAP_DIR: /home/runner/work/_temp/codeql_databases/csharp/trap/csharp
  CODEQL_EXTRACTOR_CSHARP_SOURCE_ARCHIVE_DIR: /home/runner/work/_temp/codeql_databases/csharp/src
  CODEQL_EXTRACTOR_CSHARP_BUILD_MODE: autobuild
  DOTNET_GENERATE_ASPNET_CERTIFICATE: false
  CODEQL_EXTRACTOR_CSHARP_THREADS: 2
  CODEQL_EXTRACTOR_CSHARP_RAM: 6905
  LGTM_INDEX_FILTERS: exclude:**/*
include:code/<redacted>/
include:code/shared/*Shared*/
  CODEQL_TRACER_LOG: /home/runner/work/_temp/codeql_databases/log/build-tracer.log
  CODEQL_TRACER_DIAGNOSTICS_DIR: /home/runner/work/_temp/codeql_databases/diagnostic/tracer
  CODEQL_TRACER_LANGUAGES: csharp
  SEMMLE_PRELOAD_libtrace: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/${LIB}_${PLATFORM}_trace.so
  SEMMLE_PRELOAD_libtrace32: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/lib32trace.so
  SEMMLE_PRELOAD_libtrace64: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/lib64trace.so
  CODEQL_RUNNER: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/runner
  LD_PRELOAD: /opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/tools/linux64/${LIB}_${PLATFORM}_trace.so

In addition the working log has

[command]/opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/csharp/tools/autobuild.sh 

where the non-working log has

[command]/opt/hostedtoolcache/CodeQL/2.17.3/x64/codeql/codeql database trace-command --use-build-mode /home/runner/work/_temp/codeql_databases/csharp

This difference seems related to the following if statement in the github/codeql-action :

codeql-action/src/autobuild.ts

Lines 170 to 177 in 19a81bf

	if (
	config.buildMode &&
	(await features.getValue(Feature.AutobuildDirectTracing, codeQL))
	) {
	await codeQL.extractUsingBuildMode(config, language);
	} else {
	await codeQL.runAutobuild(config, language);
	}

It looks like the new "build mode" feature is not working properly for your repository. I'll ask the team to have a look.

[aibaars]

@melwynpkb The team has confirmed the problem, and have disabled the feature. Could you check if things are working again for you? Thanks!

[melwynpkb]

@melwynpkb The team has confirmed the problem, and have disabled the feature. Could you check if things are working again for you? Thanks!

thanks @aibaars for looking at this quickly. It's a long holiday weekend here in NZ. I'll verify and report back on Tuesday.

[melwynpkb]

@aibaars can confirm autobuild is running against the specified working directory again now. Since, the old behavior was restored with a feature flag, how would this be addressed in the new 'build mode' feature?

I'm happy to close this issue for now but will continue using manual mode until the bug in the 'build mode' feature is fixed and rolled out.