Databricks bazel 7.6.1 logging

[PetrKudr]

Added logging of Starlark function calls with very basic filters:

• Everything is controlled by two environment variables:
  • DB_LOG_STARLARK_CALLS - if present and is not equal to "no", then calls will be logged. It also controls what extra information about a call is logged:
    • params - log all passed parameters
    • callstack - log callstack
    • result - log call result
    • values can be combined using ; as a separator
  • DB_LOG_STARLARK_CALLS_FILTER - allows to filter functions by name and call parameters (look at the example for details)
    • The syntax is as following

      filters = filter (';' filter)*
      filter = function_name (':' or_set_of_params)?
      or_set_of_params = and_set_of_params  ('|' and_set_of_params)*
      and_set_of_params = ident ('&' ident)*
      

    • a function call is logged if any and_set_of_params present in the call

An example:

export DB_LOG_STARLARK_CALLS='params;callstack;result'
export DB_LOG_STARLARK_CALLS_FILTER='cc_toolchain;_is_enabled:module_maps|header_modules'

This means that

• All calls that are logged will be logged with the passed parameters, the callstack and the result
• The logging will be applied to
  • all cc_toolchain calls
  • _is_enabled calls which have module_maps or header_modules values among parameters

An example of the output:

(5945f3dc): evaluating call cc_toolchain(kwargs) at /home/petr.kudriavtsev/.cache/bazel/_bazel_petr.kudriavtsev/e9dd0175ad7cf03dc6b8a55a9baa2df4/external/rules_cc/cc/toolchains/cc_toolchain.bzl:18:24
  positional args: []
  named args: [name, dummy_cc_wasm_toolchain_cc, all_files, :empty, compiler_files, :empty, dwp_files, :empty, linker_files, :empty, objcopy_files, :empty, strip_files, :empty, supports_param_files, 0, toolchain_config, :cc_toolchain_config, toolchain_identifier, dummy_wasm_cc]
  call stack:
    (5945f3dc) <toplevel>(/home/petr.kudriavtsev/.cache/bazel/_bazel_petr.kudriavtsev/e9dd0175ad7cf03dc6b8a55a9baa2df4/external/rules_rust/rust/private/dummy_cc_toolchain/BUILD.bazel:25:13)
    (5945f3dc) cc_toolchain(/home/petr.kudriavtsev/.cache/bazel/_bazel_petr.kudriavtsev/e9dd0175ad7cf03dc6b8a55a9baa2df4/external/rules_cc/cc/toolchains/cc_toolchain.bzl:18:24)
  result: None

[PetrKudr]

I know that this is dubious because of many things, but I didn't want to start messing with Reportable events. Just wanted to do something small that'll help me with debugging Starlark

[fbenkstein-db]

While I believe that this is useful, I'd rather not merge this as-is:

• the feature uses environment variables for configuration instead of command line flags
• the information is printed in an unstructured or at least non-standard format which will make it hard to reuse
• the implementation creates a custom language for filtering the log events rather than use something standard

I believe this can (and should) be implemented outside of Bazel with a custom client to the Bazel Starlark Debugging Protocol. That would also allow extending the log without having to make a new Bazel release. The debugging protocol already supports using Starlark expressions for conditional breakpoints which could be used for filtering.

There also appears to be an additional feature enabled with the DB_LOG_TOOLCHAIN_FEATURES environment variable that isn't documented in the PR description. It also prints to stderr. If the information is generally useful, it should probably go to the profile instead. Of course, this should also be a command line flag instead of an environment variable.