Skip to content

Improve Spin support for new instances#7

Open
pl-at-shopify wants to merge 3 commits into
masterfrom
spin_tweaks
Open

Improve Spin support for new instances#7
pl-at-shopify wants to merge 3 commits into
masterfrom
spin_tweaks

Conversation

@pl-at-shopify
Copy link
Copy Markdown

@pl-at-shopify pl-at-shopify commented Oct 14, 2021

This PR adds basic support for bootstrapping a new Spin instance.

Motivation

This is like calling dev up where all the dependencies get installed without the need to read the README. In addition, the snapshotting system will benefit new instance creation.

Test plan

spin create sorbet@spin_tweaks

Should not break anything.

Expect having the listed packages and bazel installed in the first minutes of the instance creation automatically.

Copy link
Copy Markdown
Member

@vinistock vinistock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks awesome, thank you for putting it together. One question I have is: performing all the setup for Sorbet may be a little slow, since we need to compile a bunch of stuff until it's actually ready. Does this mean that creating the workspace will take long?

Comment thread .spin/packages.yml
Comment thread .spin/bootstrap
# Running the scripts here helps Spin to do it in background
# and then make a snapshot to save time at new instance creation.

./bazel version
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this line doing?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

./bazel lazy loads the real Bazel on the first run. As I've spent some time debugging spin logs I find it easier to split steps like installing a package and actually using it to relax the mental load.

Comment thread .spin/bootstrap
Comment on lines +9 to +11
export PATH="/home/spin/.bazel_binaries/4.1.0/bin_t:$PATH"
# Sets up go to definition in VS Code
tools/scripts/build_compilation_db.sh
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two lines depend on first compiling Sorbet in debug mode. The bazel version changes from time to time, so we need to export the correct version.

Also, building the compilation database requires Sorbet to be compiled first.

We would need something like this:

# Compile Sorbet
# This downloads the correct Bazel version and prints the version to the screen
./bazel build //main:sorbet --config=dbg

# After we build, we need to parse the output here to find out the correct version
BAZEL_VERSION=./bazel version # needs some output processing to get the right version

# Then we can export and build the compilation db
export PATH="/home/spin/.bazel_binaries/${BAZEL_VERSION}/bin_t:$PATH"

tools/scripts/build_compilation_db.sh

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough review. I guess, you can change the script to make it more useful when we do that buddy repository from https://github.com/Shopify/spin/issues/2922.

Comment thread .spin/bootstrap
Comment on lines +14 to +28
# TODO: useful aliases

# alias compile_sorbet="./bazel build //main:sorbet --config=dbg"

# comp_run() {
# compile_sorbet && sorbet_run $@
# }

# sorbet_run() {
# bazel-bin/main/sorbet $@ --silence-dev-message
# }

# bazel_test_find() {
# ./bazel query ‘tests(//...)’ | grep $@ | xargs ./bazel test --config=dbg --test_output=errors
# }
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to append these automatically to the user's zshrc as a last step without messing up the user's own dotfiles config?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was asking myself the same. Did not find a good built-in answer yet.

We can add something like

mkdir -p ~/zsh.d/
for f in ~/zsh.d/*
do
  source $f
done

to the default spin zsh config and then just mv files into ~/zsh.d/.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will make a PR for this.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged! Can start hacking 🛠

Co-authored-by: Vinicius Stock <vinistock@users.noreply.github.com>
KaanOzkan pushed a commit that referenced this pull request Nov 18, 2021
* Add ENFORCE that would have made error more obvious

* Add failing test

Fails with this backtrace:

    Exception::raise(): core/Loc.cc:270 enforced condition exists() has failed: Can't take source of Loc that doesn't exist

    Backtrace:
      #3 0x1725725 sorbet::core::Loc::source()
      #4 0x120e0e1 sorbet::infer::Environment::processBinding()
      #5 0x11fae5b sorbet::infer::Inference::run()
      #6 0xed1a88 sorbet::realmain::pipeline::CFGCollectorAndTyper::preTransformMethodDef()
      #7 0xed185b sorbet::ast::CALL_MEMBER_impl_preTransformMethodDef<>::call<>()
      #8 0xecf90c sorbet::ast::TreeMapper<>::mapMethodDef()
      #9 0xece3ff sorbet::ast::TreeMapper<>::mapIt()
      #10 0xecf81f sorbet::ast::TreeMapper<>::mapClassDef()
      #11 0xece37a sorbet::ast::TreeMapper<>::mapIt()
      #12 0xed1409 sorbet::ast::TreeMapper<>::mapInsSeq()
      #13 0xecef9f sorbet::ast::TreeMapper<>::mapIt()
      #14 0xebc5d9 sorbet::ast::TreeMap::apply<>()
      #15 0xea2971 sorbet::realmain::pipeline::typecheckOne()
      #16 0xeb3dc6 sorbet::realmain::pipeline::typecheck()::$_4::operator()()
      #17 0xeb3acd std::__1::__invoke<>()
      #18 0xeb3a7d std::__1::__invoke_void_return_wrapper<>::__call<>()
      #19 0xeb3a4d std::__1::__function::__alloc_func<>::operator()()
      #20 0xeb2bee std::__1::__function::__func<>::operator()()
      #21 0x177e015 std::__1::__function::__value_func<>::operator()()
      #22 0x177ddb5 std::__1::function<>::operator()()
      #23 0x17fc90e sorbet::WorkerPoolImpl::multiplexJob()
      #24 0xea5633 sorbet::realmain::pipeline::typecheck()
      #25 0xafb232 sorbet::realmain::realmain()
      #26 0xaf70f2 main
      #27 0x7fba7de990b3 __libc_start_main
      #28 0xaf702e _start

* Fix failing test
paracycle pushed a commit that referenced this pull request Aug 10, 2022
* Fix crash on mixes_in_class_methods for undeclared

The crash looked like this:

    Exception::raise(): Should never happen

    Backtrace:
      #3 0xccda75 sorbet::core::ClassOrModule::isClass()
      #4 0x13de160 sorbet::resolver::(anonymous namespace)::ResolveConstantsWalk::resolveClassMethodsJob<>()
      #5 0x13d2407 sorbet::resolver::(anonymous namespace)::ResolveConstantsWalk::resolveConstants<>()
      #6 0x13d0a15 sorbet::resolver::Resolver::run()
      #7 0xeaf34b sorbet::realmain::pipeline::resolve()
      #8 0xb1b1ae sorbet::realmain::realmain()
      #9 0xb15a22 main
      #10 0x7ffff7c65083 __libc_start_main
      #11 0xb1592e _start

* Fix error message
paracycle pushed a commit that referenced this pull request Aug 10, 2022
* Add a failing test

    + exec test/lsp_test_runner --single_test=test/testdata/lsp/go_to_type_definition.rb
    Pausing
    Resuming
    [doctest] doctest version is "2.4.1"
    [doctest] run with "--help" for options
    [2022-08-01 22:09:09.445] [fatalFallback] [error] Exception::raise(): ./core/TypePtr.h:223 enforced condition store != 0 has failed: (no message provided)

    [2022-08-01 22:09:09.490] [fatalFallback] [error] Backtrace:
      #3 0xc859b2
      #4 0xdbc197 sorbet::core::TypePtr::tag()
      #5 0xef9fe7 sorbet::realmain::lsp::(anonymous namespace)::locsForType()::$_13::operator()()
      #6 0xef95d3 sorbet::typecaseHelper<>()
      #7 0xef8f85 sorbet::typecase<>()
      #8 0xef8b0a sorbet::realmain::lsp::(anonymous namespace)::locsForType()
      #9 0xef859f sorbet::realmain::lsp::TypeDefinitionTask::runRequest()
      #10 0xda699c sorbet::realmain::lsp::LSPRequestTask::run()
      #11 0xf160f4 sorbet::realmain::lsp::(anonymous namespace)::TypecheckerTask::run()
      #12 0xf1454e sorbet::realmain::lsp::LSPTypecheckerCoordinator::asyncRunInternal()
      #13 0xf14620 sorbet::realmain::lsp::LSPTypecheckerCoordinator::syncRun()
      #14 0xf12d44 sorbet::realmain::lsp::LSPLoop::runTask()
      #15 0xf12698 sorbet::realmain::lsp::LSPLoop::processRequests()
      #16 0xf0410d sorbet::realmain::lsp::SingleThreadedLSPWrapper::getLSPResponsesFor()
      #17 0xcbf398 sorbet::test::getLSPResponsesFor()
      #18 0xcbcece sorbet::test::getLSPResponsesFor()
      #19 0xcf1709 sorbet::test::TypeDefAssertion::check()
      #20 0xbbff18 sorbet::test::_DOCTEST_ANON_FUNC_71()
      #21 0x1d060a9 doctest::Context::run()
      #22 0xbc2380 main
      #23 0x7f162d27c083 __libc_start_main
      #24 0xbb34ae _start

    [2022-08-01 22:09:09.496] [fatalFallback] [error] Backtrace:
      #3 0x1d8f6e8 std::terminate()
      #4 0xbcb94b
      #5 0xdbc1c9
      #6 0xef9fe7 sorbet::realmain::lsp::(anonymous namespace)::locsForType()::$_13::operator()()
      #7 0xef95d3 sorbet::typecaseHelper<>()
      #8 0xef8f85 sorbet::typecase<>()
      #9 0xef8b0a sorbet::realmain::lsp::(anonymous namespace)::locsForType()
      #10 0xef859f sorbet::realmain::lsp::TypeDefinitionTask::runRequest()
      #11 0xda699c sorbet::realmain::lsp::LSPRequestTask::run()
      #12 0xf160f4 sorbet::realmain::lsp::(anonymous namespace)::TypecheckerTask::run()
      #13 0xf1454e sorbet::realmain::lsp::LSPTypecheckerCoordinator::asyncRunInternal()
      #14 0xf14620 sorbet::realmain::lsp::LSPTypecheckerCoordinator::syncRun()
      #15 0xf12d44 sorbet::realmain::lsp::LSPLoop::runTask()
      #16 0xf12698 sorbet::realmain::lsp::LSPLoop::processRequests()
      #17 0xf0410d sorbet::realmain::lsp::SingleThreadedLSPWrapper::getLSPResponsesFor()
      #18 0xcbf398 sorbet::test::getLSPResponsesFor()
      #19 0xcbcece sorbet::test::getLSPResponsesFor()
      #20 0xcf1709 sorbet::test::TypeDefAssertion::check()
      #21 0xbbff18 sorbet::test::_DOCTEST_ANON_FUNC_71()
      #22 0x1d060a9 doctest::Context::run()
      #23 0xbc2380 main
      #24 0x7f162d27c083 __libc_start_main
      #25 0xbb34ae _start

    libc++abi: terminate_handler unexpectedly returned
    ===============================================================================
    test/lsp_test_runner.cc:497:
    TEST CASE:  LSPTest

    test/lsp_test_runner.cc:497: FATAL ERROR: test case CRASHED: SIGABRT - Abort (abnormal termination) signal

* Set type to untyped if null
egiurleo pushed a commit that referenced this pull request May 17, 2024
* Pull in sorbet/bazel-toolchain#8

Required to make Clang 15 available.

* llvm toolchain: 12.0.0 → 15.0.7

* Silence this warning globally

* Fix "unused but set variable" warnings

* Stop linking against mimalloc on macOS release builds

* Fix path to sanitizer archives

Clang moved these at some point.

Note that we only ever ran the sanitize build on x86_64--it never ran
over arm64 builds.

That might be nice to add some day, but I'm not blocking on it.

* Add doc on how to build and publish LLVM

* tools/scripts/format_cxx.sh

* Silence some warning for abseil for now

* Backport a patch from upstream to fix -Warray-parameter warning

I couldn't get this silenced by adding `--copt=-Wno-array-parameter`
because it looks like abseil builds this with a bunch of specific
warning flags, and those take precedence over ours.

The issue is fixed upstream, and when we upgrade we'll get this change
for free, but as mentioned in the previous commit we can't upgrade
abseil until after we upgrade clang.

* Ignore unknown warnings on emscripten build

* Defer throwing an exception until all threads have exited

In clang 15, the sanitizer was catching a stack-use-after-return error
from this multithreaded codepath. The test which caught this was

    //test/cli:test_folder-input-not-found

(Because it is multithreaded, the bug only manifested about 30% of the
time, depending on the order that threads were scheduled by the OS.)

The bug was that we were attempting to (atomically) write to
`pendingJobs` from children threads after the function that spawned
those threads exited.

A sequence of steps that could happen:

- A worker thread would realize that the input was not a directory. It
  would return a result requesting that the parent thread throw an
  exception, and queue up a bunch of jobs that would have the effect of
  shutting down the other threads.

- The parent thread would get scheduled first. It would see that a child
  asked to throw, and it would throw, in effect returning from the
  function before all children were done.

- The children would wake up and attempt to decrement the `pendingJobs`
  variable in the worker threads' closure by reference. That stack slot
  is not there anymore because the enclosing function raised.

The fix is simply wait for all the threads to finish, and only raise the
requested exception after we're sure that all the children threads have
finished processing and shut down.

For posterity, this is what the test failure looked like

    +=================================================================
    +==1781049==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f435a57c140 at pc 0x55cac4f42fd8 bp 0x7f43420ef940 sp 0x7f43420ef938
    +WRITE of size 8 at 0x7f435a57c140 thread T7 (options.findFil)
    +    #0 0x55cac4f42fd7 in unsigned long std::__1::__cxx_atomic_fetch_add[abi:v15007]<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order) realmain.cc
    +    #1 0x55cac6e4d9b8 in std::__1::__atomic_base<unsigned long, true>::operator+=[abi:v15007](unsigned long) common.cc
    +    #2 0x55cac6e3f5a6 in std::__1::__function::__func<appendFilesInDir(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, absl::flat_hash_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, absl::container_internal::StringHash, absl::container_internal::StringHashEq::Eq, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, sorbet::WorkerPool&, bool, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_0, std::__1::allocator<appendFilesInDir(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, absl::flat_hash_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, absl::container_internal::StringHash, absl::container_internal::StringHashEq::Eq, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, sorbet::WorkerPool&, bool, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&)::$_0>, void ()>::operator()() common.cc
    +    #3 0x55cac629c0e1 in std::__1::__function::__value_func<void ()>::operator()[abi:v15007]() const driver.cc
    +    #4 0x55cac6265778 in std::__1::function<void ()>::operator()() const (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x43e7778) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #5 0x55cac6ed26ac in std::__1::__function::__func<sorbet::WorkerPoolImpl::multiplexJob(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::function<void ()>)::$_2, std::__1::allocator<sorbet::WorkerPoolImpl::multiplexJob(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::function<void ()>)::$_2>, bool ()>::operator()() WorkerPoolImpl.cc
    +    #6 0x55cac6ed12c5 in std::__1::__function::__func<sorbet::WorkerPoolImpl::WorkerPoolImpl(int, spdlog::logger&)::$_0, std::__1::allocator<sorbet::WorkerPoolImpl::WorkerPoolImpl(int, spdlog::logger&)::$_0>, void ()>::operator()() WorkerPoolImpl.cc
    +    #7 0x55cac629c0e1 in std::__1::__function::__value_func<void ()>::operator()[abi:v15007]() const driver.cc
    +    #8 0x55cac6265778 in std::__1::function<void ()>::operator()() const (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x43e7778) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #9 0x55cac6ee3e33 in Joinable::trampoline(void*) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x5065e33) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #10 0x7f435c953608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x8608) (BuildId: c6d0d79d906d62bb768421fc6dada0d5e729f177)
    +    #11 0x7f435c84c352 in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x11f352) (BuildId: 87b331c034a6458c64ce09c03939e947212e18ce)
    +
    +Address 0x7f435a57c140 is located in stack of thread T0 at offset 320 in frame
    +    #0 0x55cac6e3bb4c in appendFilesInDir(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, absl::flat_hash_set<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, absl::container_internal::StringHash, absl::container_internal::StringHashEq::Eq, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, sorbet::WorkerPool&, bool, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>>&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&, std::__1::vector<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, std::__1::allocator<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>> const&) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x4fbdb4c) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +
    +  This frame has 20 object(s):
    +    [32, 40) '__p.addr.i.i.i.i'
    +    [64, 65) '__af.i.i.i'
    +    [80, 104) '__hold.i.i.i'
    +    [144, 160) 'ref.tmp.i.i.i'
    +    [176, 177) 'ref.tmp.i'
    +    [192, 208) 'basePath'
    +    [224, 225) 'recursive.addr'
    +    [240, 244) 'numWorkers'
    +    [256, 272) 'jobq'
    +    [288, 304) 'resultq'
    +    [320, 328) 'pendingJobs' <== Memory access at offset 320 is inside this variable
    +    [352, 384) 'ref.tmp24'
    +    [416, 464) 'agg.tmp41'
    +    [496, 584) 'agg.tmp42'
    +    [624, 656) 'threadResult'
    +    [688, 704) 'ref.tmp73'
    +    [720, 728) 'ref.tmp89'
    +    [752, 832) '__enforceTimer'
    +    [864, 880) 'agg.tmp199'
    +    [896, 904) 'ref.tmp225'
    +HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
    +      (longjmp and C++ exceptions *are* supported)
    +SUMMARY: AddressSanitizer: stack-use-after-return realmain.cc in unsigned long std::__1::__cxx_atomic_fetch_add[abi:v15007]<unsigned long>(std::__1::__cxx_atomic_base_impl<unsigned long>*, unsigned long, std::__1::memory_order)
    +Shadow bytes around the buggy address:
    +  0x0fe8eb4a77d0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a77e0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a77f0: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7800: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7810: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +=>0x0fe8eb4a7820: f5 f5 f5 f5 f5 f5 f5 f5[f5]f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7830: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7840: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7850: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7860: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +  0x0fe8eb4a7870: f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5 f5
    +Shadow byte legend (one shadow byte represents 8 application bytes):
    +  Addressable:           00
    +  Partially addressable: 01 02 03 04 05 06 07
    +  Heap left redzone:       fa
    +  Freed heap region:       fd
    +  Stack left redzone:      f1
    +  Stack mid redzone:       f2
    +  Stack right redzone:     f3
    +  Stack after return:      f5
    +  Stack use after scope:   f8
    +  Global redzone:          f9
    +  Global init order:       f6
    +  Poisoned by user:        f7
    +  Container overflow:      fc
    +  Array cookie:            ac
    +  Intra object redzone:    bb
    +  ASan internal:           fe
    +  Left alloca redzone:     ca
    +  Right alloca redzone:    cb
    +Thread T7 (options.findFil) created by T0 here:
    +    #0 0x55cac4eadd9c in pthread_create (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x302fd9c) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #1 0x55cac6ee4267 in runInAThread(std::__1::basic_string_view<char, std::__1::char_traits<char>>, std::__1::function<void ()>, std::__1::optional<int>) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x5066267) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #2 0x55cac6ecf33f in sorbet::WorkerPoolImpl::WorkerPoolImpl(int, spdlog::logger&) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x505133f) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #3 0x55cac6ece73c in sorbet::WorkerPool::create(int, spdlog::logger&) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x505073c) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #4 0x55cac6a300a6 in sorbet::realmain::options::readOptions(sorbet::realmain::options::Options&, std::__1::vector<std::__1::unique_ptr<sorbet::pipeline::semantic_extension::SemanticExtension, std::__1::default_delete<sorbet::pipeline::semantic_extension::SemanticExtension>>, std::__1::allocator<std::__1::unique_ptr<sorbet::pipeline::semantic_extension::SemanticExtension, std::__1::default_delete<sorbet::pipeline::semantic_extension::SemanticExtension>>>>&, int, char**, std::__1::vector<sorbet::pipeline::semantic_extension::SemanticExtensionProvider*, std::__1::allocator<sorbet::pipeline::semantic_extension::SemanticExtensionProvider*>> const&, std::__1::shared_ptr<spdlog::logger>) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x4bb20a6) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #5 0x55cac4f0379c in sorbet::realmain::realmain(int, char**) (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x308579c) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #6 0x55cac4eff162 in main (/pay/home/jez/.cache/bazel/_bazel_jez/2cc40ac946cd09178597ebd947795951/execroot/com_stripe_ruby_typer/bazel-out/k8-dbg/bin/main/sorbet+0x3081162) (BuildId: c972efcad340ab8e0e5ea671063eb36b)
    +    #7 0x7f435c751082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 87b331c034a6458c64ce09c03939e947212e18ce)
    +
    +==1781049==ABORTING
    ================================================================================
jesse-shopify pushed a commit that referenced this pull request Apr 30, 2026
* Add failing test

Crash from my debug log while working on a sorbet-runtime change:

    timestamp="2026-04-15T16:58:15.415401" level=debug Trying to see if fast path is available after 1 file changes
    timestamp="2026-04-15T16:58:15.415904" level=error Exception::raise() msg="main/lsp/LSPFileUpdates.cc:231 enforced condition !evictedFiles.empty() has failed: (no message provided)"

    timestamp="2026-04-15T16:58:15.459623" level=error Backtrace:
      #3 0xb81f0cc2396c std::__1::vector<>::operator[][abi:v15007]()
      #4 0xb81f0ccd1f00 sorbet::realmain::lsp::LSPFileUpdates::fastPathFilesToTypecheck()
      #5 0xb81f0ccc446c sorbet::realmain::lsp::LSPIndexer::getTypecheckingPathInternal()
      #6 0xb81f0ccc4ae8 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPath()
      #7 0xb81f0cde3000 sorbet::realmain::lsp::SorbetWorkspaceEditTask::getTypecheckingPath()
      #8 0xb81f0cde3858 sorbet::realmain::lsp::SorbetWorkspaceEditTask::canPreempt()
      #9 0xb81f0cce7504 sorbet::realmain::lsp::LSPLoop::runLSP()
      #10 0xb81f0cc1844c sorbet::realmain::realmain()
      #11 0xb81f0cbef374 main
      #12 0xebf55fee84c4
      #13 0xebf55fee8598 __libc_start_main
      #14 0xb81f0cbef230 _start

    timestamp="2026-04-15T16:58:15.459844" level=error Sorbet raised uncaught exception type="sorbet::SorbetException" what="main/lsp/LSPFileUpdates.cc:231 enforced condition !evictedFiles.empty() has failed: (no message provided)"
    timestamp="2026-04-15T16:58:15.477194" level=error Backtrace:
      #3 0xb81f0cbef664 __cxa_get_exception_ptr
      #4 0xb81f0cbef5e0 __cxxabiv1::exception_cleanup_func()
      #5 0xb81f0cc40984 sorbet::Exception::raise<>()
      #6 0xb81f0cc1f710 sorbet::Exception::enforce_handler<>()
      #7 0xb81f0cc2396c std::__1::vector<>::operator[][abi:v15007]()
      #8 0xb81f0ccd1f00 sorbet::realmain::lsp::LSPFileUpdates::fastPathFilesToTypecheck()
      #9 0xb81f0ccc446c sorbet::realmain::lsp::LSPIndexer::getTypecheckingPathInternal()
      #10 0xb81f0ccc4ae8 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPath()
      #11 0xb81f0cde3000 sorbet::realmain::lsp::SorbetWorkspaceEditTask::getTypecheckingPath()
      #12 0xb81f0cde3858 sorbet::realmain::lsp::SorbetWorkspaceEditTask::canPreempt()
      #13 0xb81f0cce7504 sorbet::realmain::lsp::LSPLoop::runLSP()
      #14 0xb81f0cc1844c sorbet::realmain::realmain()
      #15 0xb81f0cbef374 main
      #16 0xebf55fee84c4
      #17 0xebf55fee8598 __libc_start_main
      #18 0xb81f0cbef230 _start

Test output from this failing test:

    [2026-04-22 23:24:15.485] [fatalFallback] [error] Exception::raise() msg="main/lsp/LSPFileUpdates.cc:236 enforced condition !evictedFiles.empty() has failed: (no message provided)"

    [2026-04-22 23:24:15.556] [fatalFallback] [error] Backtrace:
      #3 0xc9423562d1bc sorbet::test::Expectations::Expectations()
      #4 0xc94235711588 sorbet::realmain::lsp::LSPFileUpdates::fastPathFilesToTypecheck()
      #5 0xc94235707a14 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPathInternal()
      #6 0xc94235708094 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPath()
      #7 0xc9423570571c sorbet::realmain::lsp::SorbetWorkspaceEditTask::getTypecheckingPath()
      #8 0xc94235705f74 sorbet::realmain::lsp::SorbetWorkspaceEditTask::canPreempt()
      #9 0xc94235818830 sorbet::realmain::lsp::LSPLoop::runLSP()
      #10 0xc942358102c8 sorbet::realmain::lsp::MultiThreadedLSPWrapper::MultiThreadedLSPWrapper()::$_0::operator()()
      #11 0xc94235810270 std::__1::__invoke[abi:v15007]<>()
      #12 0xc94235810228 std::__1::__invoke_void_return_wrapper<>::__call<>()
      #13 0xc94235810204 std::__1::__function::__alloc_func<>::operator()[abi:v15007]()
      #14 0xc9423580f4d0 std::__1::__function::__func<>::operator()()
      #15 0xc94235ec3244 std::__1::__function::__value_func<>::operator()[abi:v15007]()
      #16 0xc94235eb8748 std::__1::function<>::operator()()
      #17 0xc9423637638c Joinable::trampoline()
      #18 0xf2a5d712595c
      #19 0xf2a5d718bb4c

    [2026-04-22 23:24:15.557] [fatalFallback] [error] Sorbet raised uncaught exception type="sorbet::SorbetException" what="main/lsp/LSPFileUpdates.cc:236 enforced condition !evictedFiles.empty() has failed: (no message provided)"
    [2026-04-22 23:24:15.575] [fatalFallback] [error] Backtrace:
      #3 0xc9423548dec4 __cxa_get_exception_ptr
      #4 0xc9423548de40 __cxxabiv1::exception_cleanup_func()
      #5 0xc94235630b24 sorbet::Exception::raise<>()
      #6 0xc94235630a20 sorbet::Exception::enforce_handler<>()
      #7 0xc9423562d1bc sorbet::test::Expectations::Expectations()
      #8 0xc94235711588 sorbet::realmain::lsp::LSPFileUpdates::fastPathFilesToTypecheck()
      #9 0xc94235707a14 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPathInternal()
      #10 0xc94235708094 sorbet::realmain::lsp::LSPIndexer::getTypecheckingPath()
      #11 0xc9423570571c sorbet::realmain::lsp::SorbetWorkspaceEditTask::getTypecheckingPath()
      #12 0xc94235705f74 sorbet::realmain::lsp::SorbetWorkspaceEditTask::canPreempt()
      #13 0xc94235818830 sorbet::realmain::lsp::LSPLoop::runLSP()
      #14 0xc942358102c8 sorbet::realmain::lsp::MultiThreadedLSPWrapper::MultiThreadedLSPWrapper()::$_0::operator()()
      #15 0xc94235810270 std::__1::__invoke[abi:v15007]<>()
      #16 0xc94235810228 std::__1::__invoke_void_return_wrapper<>::__call<>()
      #17 0xc94235810204 std::__1::__function::__alloc_func<>::operator()[abi:v15007]()
      #18 0xc9423580f4d0 std::__1::__function::__func<>::operator()()
      #19 0xc94235ec3244 std::__1::__function::__value_func<>::operator()[abi:v15007]()
      #20 0xc94235eb8748 std::__1::function<>::operator()()
      #21 0xc9423637638c Joinable::trampoline()
      #22 0xf2a5d712595c
      #23 0xf2a5d718bb4c

Looks like a match.

* Remove ENFORCE

I don't know of a convenient way to validate what we were trying to
validate here. Not sure whether it's actually useful to validate, so
let's just remove the ENFORCE.
amomchilov pushed a commit that referenced this pull request May 4, 2026
* Make `//tools:compdb` manual

This was causing conflicts when using `//...`. We don't actually care
about having `//tools:compdb` picked up from `//...` for any case we
care about.

* First stab

* # This is a combination of 7 commits.
# This is the 1st commit message:

Have to forcibly enable colorized output

The wrapper script that `bazel_clang_tidy` uses redirects the clang-tidy
output to a file, so we have to forcibly enable it if we want it on.

I noticed that this is the same as what our llvm_toolchain does for
clang itself (forcibly passing `-fcolor-diagnostics`) so it should be
fine for us to also do this for clang-tidy, though it is annoying that
we can't have it inferred.

# This is the commit message #2:

Add TODOs, might leave it here for a while

# This is the commit message #3:

wip: Update note about editor config

# This is the commit message #4:

Wire up clang-tidy in CI

# This is the commit message #5:

This isn't necessary

# This is the commit message #6:

Need to allow the clang_tidy_profile to be uploaded

# This is the commit message #7:

Tidy up the README

* Have to forcibly enable colorized output

A note on forcibly enabling color: the wrapper script that
`bazel_clang_tidy` uses redirects the clang-tidy output to a file, so we
have to forcibly enable it if we want it on.

I noticed that this is the same as what our llvm_toolchain does for
clang itself (forcibly passing `-fcolor-diagnostics`) so it should be
fine for us to also do this for clang-tidy, though it is annoying that
we can't have it inferred.

* Remove some unused NOLINT comments

No clue what these were doing, but I wanted to keep score for how many
`NOLINT` I had to add to get things passing, but these are redundant.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants