Rubinius v4.11 Release Notes

Release Date: 2020-01-06 // about 3 years ago
  • ๐Ÿ”– Version 4.11 (2020-01-05)

    ๐Ÿ›  Fixed building OSX packages. (Brian Shirai)

    Try re-enabling Travis OSX. (Brian Shirai)

    Re-disable Travis OSX. STDIN is not a TTY. (Brian Shirai)

    โœ‚ Removed obsolete compiler define RBX_PROFILER. (Brian Shirai)

    Attempt to avoid breaking LLVM compiler flags. (Brian Shirai)

    โšก๏ธ Updated gems. (Brian Shirai)

    โœ… More instruction tests. (Brian Shirai)

    โœ… More interpreter tests. (Brian Shirai)

    โœ… Tests for check_frozen instruction. (Brian Shirai)

    ๐Ÿšš Moved builtin/ to class/. (Brian Shirai)
    The name 'builtin' is not very descriptive. Now that we are going to have
    managed objects that are both object-oriented and simple data, and both are
    'built in', we need more descriptive names for the directories, so the new
    names are 'class' and 'data', which parallels the names used in the code.

    ๐Ÿ›  Fixed path to website releases data. (Brian Shirai)

    ๐Ÿง For Rubinius build on aarch64-linux-gnu (Raspberry Pi 3). (#3705) (medols)

    Specs for rb_obj_method_arity. (Brian Shirai)

    Added rb_obj_method_arity. Fixes #3708. (Brian Shirai)

    โšก๏ธ Updated daedalus-core gem. (Brian Shirai)

    โž• Added C-API rb_memhash. (Brian Shirai)

    โšก๏ธ Updated rubysl-openssl version. (Brian Shirai)

    โšก๏ธ Updates to issues/PR templates. (Brian Shirai)

    Try putting under .github. (Brian Shirai)

    โช Revert "Try putting under .github." (Brian Shirai)
    โช This reverts commit 320f504.

    โšก๏ธ Updated README. (Brian Shirai)

    Don't use enum as last arg for varargs. /ht @heftig. (Brian Shirai)

    ๐Ÿ›  Fixed Bundler, brew wars. (Brian Shirai)

    ๐Ÿ›  Fixed setting ThreadGroup for signal handler Thread. (Brian Shirai)
    ๐Ÿšฆ In Rubinius, trapped signals are handled in a dedicated Thread.

    ๐Ÿšฆ In Ruby, the Thread where a signal is processed isn't defined, but the
    ๐Ÿšฆ assumption is that the signal handler runs in Thread.main (as observed in Ruby

    ๐Ÿšฆ Also undefined in Ruby is what ThreadGroup a Thread created in a signal
    ๐Ÿšฆ handler
    should be added to. Going with the assumption that the signal handler
    proc is run in Thread.main, we assume the new Thread should be added to

    ๐Ÿšฆ We now set the signal handler Thread's group to the same group as Thread.main,
    ๐Ÿšฆ so that threads created in a signal handler proc have a group and are added to
    a group.

    • ๐Ÿ‘ Better backtrace rendering for copy-paste. (Brian Shirai)
      Rubinius renders backtraces with the receiver+method right justified and the
      source location trailing. Previously, the source location was split when
      exceeding the available terminal width to avoid cluttering the left side where
      the receiver+method was rendered to aid visibility of the most important
      backtrace information.

    Unfortunately, splitting up the source location makes copy-pasting the path to
    edit a source file very tedious.

    Now, the source location is rendered as an unbroken sequence of characters but
    starting on a new line if the total backtrace line exceeds the terminal width.
    This preserves the readability of the receiver+method, while also preserving
    the ability to copy-paste the source location.

    • ๐Ÿ‘Œ Improved backtrace rendering contrast. (Brian Shirai)
      Rubinius introduced colorized backtraces to improve readability.

    Unfortunately, color is problematic due to the perception of colors and the
    interaction of color with terminal colors, which something which Rubinius has
    no control over.

    The goal of readability in backtraces is an important one. There are two
    primary pieces of information in each backtrace line: 1. receiver+method; and

    1. source code location, file+line number.

    The current rendering approach is to bold the receiver+method and separate the
    source line on a new line when the total backtrace line exceeds the terminal
    width. This provides contrast that improves scanning the backtrace on primary
    information, the receiver+method, and easily selecting secondary information,
    the source location, when necessary.

    • Align method names in backtrace. (Brian Shirai)
      It is a very common convention (in Rubinius, at least) to fully render a
      method as 'A::B::C{#, .}name', where A::B:: is an optional namespace and C is
      the class of the receiver and # indicates an instance method, while .
      indicates a module method.

    As such, this convention carries over without much question to rendering
    backtraces. However, the context of consuming a backtrace requires different
    conventions, but ones that are not foreign.

    So, for rendering backtraces, the conventions are retained but the ordering is
    ๐Ÿ”„ changed slightly to improve processing the information. There are now four
    basic components to a backtrace line (for an object-oriented method): 1. the
    method name; 2. the instance/module indicator; 3. the fully-scoped receiver
    name; and 4. the source code location.

    The first thing that needs to be located and digested is the method name, so
    the method names are right aligned allowing both easily reading the method
    name and quick indexing for the second important piece of information, the
    ๐Ÿš… receiver name. The source location trails and may be on a new line if the
    total backtrace line rendering exceeds the terminal width.

    ๐Ÿ‘ This rendering better orders information in the backtrace based on the
    hierarchy of important information (method name > instance/module indicator >
    receiver name > source location). It also balances the tensions between making
    an individual line of a backtrace readable and making the whole thing
    scannable and readable. When rendering the full method name as A::B::C#name,
    the variability of the A::B::C component pushed many names too far to the
    right, leaving a very jagged left edge that is more difficult to scan, as well
    as burying the method name (the primary element) embedded in the middle of a
    bunch of character noise.

    ๐Ÿ›  Fixed rendering method name. (Brian Shirai)

    Catch C++ exceptions sooner. (Brian Shirai)
    We attempt to catch C++ exceptions as close to where they occur as possible.
    This means at the last executed managed code (eg method) so that as much
    context as possible can be given.

    โž• Added extra C++ exception catch to new interpreter. (Brian Shirai)

    ๐Ÿ‘‰ Use llvm-config --libs instead of --libfiles. (Brian Shirai)

    ๐Ÿ›  fix case where IO#gets blocks forever on read to char boundary (Chuck Remes)
    This was a tough one. WEBrick would hang when reading an HTTP
    header. The cause was that IO::EachReader#read_to_separator_with_limit
    blocked forever trying to "peek ahead" to ensure we are reading
    to the end of a proper character boundary. In this case the socket
    had no more data to recv but it also does not set EOF, therefore
    ๐Ÿšš the read blocked indefinitely. The simple fix was to move to a
    non-blocking read and skip the read entirely if the string
    already had a valid encoding. However, this broke several specs
    for reading multi-byte chars. To resolve that problem required
    making sure that when read_nonblock returned an empty buffer that
    0๏ธโƒฃ this buffer was set to ASCII_8BIT instead of whatever the default
    ๐Ÿ›  external coding was set to (usually UTF-8). The fixes are all
    relatively simple but it took many hours to chase this down and
    figure out the minimal approach to fix it.

    DRY up allocation of 8-bit string buffers (Chuck Remes)

    โšก๏ธ Update (Brian Shirai)
    ๐Ÿ›  Fixed Markdown formatting of checkbox list.

    Update (Brian Shirai)
    ๐Ÿ›  Fixed Markdown formatting of checkbox list.

    ๐Ÿ— Enable build from network for RPi3 (macbookpr02010)

    add test for instructions::send_stack_with_block (Chuck Remes)

    โž• add test for instructions::rotate (Chuck Remes)

    โž• add work in progress test for instructions::send_vcall (Chuck Remes)

    ๐Ÿ‘ better tests for instructions::send_stack (Chuck Remes)
    The test_send_stack_execute_returns_null tests the situation where
    no value is returned from the method, e.g. nullptr.

    Second test test_send_stack_method_no_method_error tests code path
    where the method is not found and raises a NoMethodError. This had
    the side effect of leaving arguments on the stack. This is a
    problem that requires a bug fix to the instruction.

    • clean up stack before trying to execute method (Chuck Remes)
    • ๐Ÿ‘Œ verify stack is cleaned up before NoMethodError raised (Chuck Remes)
    • clean up stack before executing method in case it raises error (Chuck Remes)
    • โœ… test to verify handling of method returning nullptr (Chuck Remes)
    • ๐Ÿ‘Œ verify stack is cleaned up when NoMethodError raised (Chuck Remes)
    • clean up stack before executing method (Chuck Remes)
    • ๐Ÿ– handle method that returns nullptr properly (Chuck Remes)
    • test for stack cleanup on instructions::send_stack_with_splat when NoMethodError raised (Chuck Remes)
    • clean up stack before executing method (Chuck Remes)
    • finish other tests for instructions::send_stack_with_splat (Chuck Remes)
    • ๐Ÿ†“ free allocated buffes on an early return to prevent memory leak (Chuck Remes)
    • ๐Ÿ‘ allow for scan-build to substitute CC and CXX compilers (Chuck Remes)
    • ๐Ÿ›  fix typo (Chuck Remes)
    • add tests for instructions::send_super_stack_with_block (Chuck Remes)
    • ๐Ÿ‘ป clean up stack prior to calling method that may raise exception (Chuck Remes)
    • add tests for instructions::send_super_stack_with_splat (Chuck Remes)
    • ๐Ÿ‘Œ verify stack cleared before method call that could raise exception (Chuck Remes)
    • โž• add tests for instructions::send_vcall (Chuck Remes)
    • โž• add test for instructions::set_const (Chuck Remes)
    • add test for instructions::set_const_at (Chuck Remes)
    • โž• add tests for instructions::set_ivar (Chuck Remes)
    • add tests for instructions:::set_local, set_stack_local, and store_my_field (Chuck Remes)
    • compatible with musl libc (SquallATF)
      Mapping unsigned to uint to compatible with musl libc
    • โœ… freebsd supports birthtime so test for it (Chuck Remes)
    • โšก๏ธ update US/Eastern to use modern zone terminoloy America/New_York (Chuck Remes)
    • ๐Ÿ›  fix two small FFI bugs (Chuck Remes)
      ๐Ÿ› Bug 1 was trying to find the string index of the integer 0. For this
      to work the system calls :to_str on the Fixnum which fails. The fix
      is to specify the nul string explicitly.

    ๐Ÿ› Bug 2 was more of an issue of convenience. The FFI.config facility
    ๐Ÿ— builds FFI::Struct class layouts from data in runtime/platform.conf.
    Some structs have char arrays. The original code allocated the proper
    space but it did not associate an array with a :char_array field.
    Therefore trying to access the field to assign to it directly did
    not work.
    ๐Ÿ’Ž struct =
    struct[:sun_path] = "/some/path" # failed!
    This fix allows the direct assignment of string fields built using
    the FFI.config convenience method.

    • start work on yield_stack instruction (Chuck Remes)
    • โšก๏ธ update gems list to include rubysl-socket 2.2.1 (Chuck Remes)
    • ๐Ÿ‘€ rescue subprocess EPIPE so we do not see unnecessary backtrace during spec run (Chuck Remes)
    • ๐Ÿ‘‰ make sure errno from child process is preserved (Chuck Remes)
    • preserve errno for backtick child subprocesses (Chuck Remes)
    • โž• add tests for instructions:ret (Chuck Remes)
    • ๐Ÿคก mock module that handles +const_missing+ call (Chuck Remes)
    • โž• add tests for instructions::find_const (Chuck Remes)
    • Correctly handle nested calls to inhibit GC. (Brian Shirai)
    • โž• Added --gc-stack-check. (Brian Shirai)
      When the process is going to checkpoint to run GC, all the threads will
      ๐ŸŒฒ log their stacks.

    ๐ŸŒฒ Presently, if a thread is already suspended, it won't log its stack.
    When the GC triggering and checkpointing is reworked, the threads that
    were suspended when GC occurred will check a flag that GC has run when
    ๐ŸŒฒ resuming to log their stacks.

    Emit the kind of thread for --gc-stack-check. (Brian Shirai)

    โž• Add MarkStack abstraction for GC. (Brian Shirai)
    ๐Ÿ”ง Use the --gc-stack-check configure option to enable a version of
    MarkStack that stores the parent object (when available) for every
    object added to the mark stack.

    โž• Add deprecations for version 3.x to 4.0 transition. (Brian Shirai)

    Preserve C-API handle references for several GC cycles. (Brian Shirai)
    ๐Ÿš€ In the MRI C-API, there are no explicit 'retain', 'release' operations.
    When a managed object is created in Rubinius, a static (ie non-moving)
    pointer is returned. The pointer is a MemoryHandle object that
    references the managed memory object.

    When a GC cycle occurs, if nothing has marked the managed memory object
    ๐Ÿš€ as reachable, the MemoryHeader object would be released and the managed
    memory object's space reclaimed.

    The problem with this is that any time the C-API calls back into the VM,
    a GC cycle may occur. The following code illustrates this:

    VALUE c_func(VALUE a, VALUE b) {
    VALUE new_obj = c_api_create_obj(a, b);

    /* Many C-API functions are implemented as wrappers for Ruby code,
     * so this call may result in a GC operation and 'new_obj' will not
     * be referenced anywhere, causing the GC to assume it is
     * unreachable, and therefore garbage.
    return new_obj;


    To address this, we track how many GC cycles have occurred while the
    object is not marked. To make this slighty more robust (ie not retain
    true garbage for too long), we could use a flag that the MemoryHandle
    has just been created and only retain it for several cycles if it has
    just been created.

    โž• Add ability to search mark stack for pointers. (Brian Shirai)

    Start moving to a dedicated collector thread. (Brian Shirai)

    ๐Ÿ“‡ Rename CollectorThread to Collector. (Brian Shirai)
    ๐Ÿšš Also move GCInhibit from Memory to Collector::Inhibit.

    More Collector cleanup. (Brian Shirai)

    Invoke GC using a new mechanism. (Brian Shirai)
    Nothing about GC has changed, but the sequence to invoke it uses the new
    mechanism being built.

    ๐Ÿ‘ท Actually run the collector on the collector worker thread. (Brian Shirai)

    ๐Ÿšš Move more GC to Collector. (Brian Shirai)

    GC cleanup round 1. (Brian Shirai)

    More moving to the Collector. (Brian Shirai)

    Introduce Heap, MainHeap, MemoryTracer. (Brian Shirai)

    Start replacing saw_object with trace_object. (Brian Shirai)

    Detangle some GC function names. (Brian Shirai)

    ๐Ÿ— aarch64 build (macbookpr02010)

    โœ‚ Remove legacy GC code. (Brian Shirai)

    ๐Ÿ›  Fix resetting Immix allocator after collection. (Brian Shirai)

    Clean up a bit more old GC related code. (Brian Shirai)

    โœ‚ Removed Address. (Brian Shirai)
    The Address construct was interesting, but it obscured the memory
    โž• address alignment and added another (incompatible with MemoryHeader,
    DataHeader, ObjectHeader) concept that complicated interacting with
    Immix when there is no reason for Immix to be special.

    โœ‚ Remove collect flag and directly request collection. (Brian Shirai)

    ๐Ÿšš Move pinned capability to only third region. (Brian Shirai)

    Split MemoryHandles from References. (Brian Shirai)

    Pass address of object into collector. (Brian Shirai)

    โช reverting to rubinius commit 363b0e7 prior to add arm64v8changes (macbookpr02010)

    โž• add arm64v8 changes (macbookpr02010)

    Some write barrier cleanup. (Brian Shirai)

    โž• Add --log-concurrent-update and --raise-concurrent-update. (Brian Shirai)
    ๐ŸŒฒ The first compile-time option will log (level warn) when an object
    โšก๏ธ created by one thread is updated by a different thread. The second
    ๐Ÿ‘ป compile-time option will raise an exception. The options can be used
    together or separately.

    โž• Added ConcurrentUpdateError. (Brian Shirai)
    ๐Ÿ‘ป Also standardized exception handling into MachineException::guard.

    โž• Added '-Xmachine.concurrent.update.{log, raise}' options. (Brian Shirai)

    Pass Object** to MemoryVisitor functions. (Brian Shirai)
    This will enable Object#become and other useful capabilities.

    ๐Ÿ”’ Hold logger lock during fork(). (Brian Shirai)
    A thread that does not stop for fork() because it's not mutating memory
    ๐Ÿ”’ may still use the logger, which causes the logger lock to be in an
    ๐Ÿ”’ indeterminate state across a fork() call, possibly resulting in the lock
    ๐Ÿ”’ being locked in the child. By explicitly holding the lock, we can safely
    ๐Ÿ”’ unlock it in the child and the lock is in a determinate state.

    โž• Added Object#become. (Brian Shirai)
    This is not complete yet, but works for variables.

    ๐Ÿ›  Fixed write_barrier. (Brian Shirai)

    ๐Ÿ‘‰ Make MemoryVisitor recursive. (Brian Shirai)

    ๐Ÿ‘‰ Make Collector recursive. (Brian Shirai)

    Rework location of checkpoints. (Brian Shirai)

    โœ‚ Remove call frame return value. (Brian Shirai)
    This was added to save a transient root while the collector ran, but
    it's not needed with the new method of checkpointing in the ret
    instruction and loop back branches.

    โž• Added Thread-local Heap Container & Allocator. (Brian Shirai)

    Allocate using THCA. (Brian Shirai)

    Rework checkpoint. (Brian Shirai)

    Limit size of object allocated by THCA. (Brian Shirai)

    Clean up timestamp files generated by Makefile (ljulliar)

    Simplify check_stop. (Brian Shirai)
    With main collection always running on a dedicated thread, there is no
    need for each thread to check whether they may have to run the collector
    when stop is set.

    โž• Add function table to LexicalScope. (Brian Shirai)

    Define functions. (Brian Shirai)

    Rework checking stack overrun. (Brian Shirai)

    Added r_load_{0,1,nil,false,true} and goto_{past,future. (Brian Shirai)
    The goto_past instruction is a backward branch, which allows explicitly
    enabling or disabling looping constructs.

    โž• Added call instruction variants. (Brian Shirai)
    The call_send instruction looks for a function and falls back to
    dispatching a method. This will replace the send_stack instruction that
    is used for implicit self sends in code that could have functions.

    The call instruction only looks for a function.

    The call_0 instruction is for 0-arity functions.

    The send_vcall instruction will be modified to look for functions.

    ๐Ÿ›  Fixed a couple deadlocks. (Brian Shirai)

    โž• Add specs and verification for new instructions. (Brian Shirai)

    โž• Add more support for tagged nil. (Brian Shirai)
    ๐Ÿ›  And fixed other stuff.

    ๐Ÿ›  Fixed more locking issues. (Brian Shirai)

    ๐Ÿท Added push_tagged_nil instruction. (Brian Shirai)

    ๐Ÿ›  Fix nil checks. (Brian Shirai)

    ๐Ÿ›  More fixed nil checks. (Brian Shirai)

    ๐Ÿท Tag nil on return if untagged. (Brian Shirai)
    When nil values are created outside the instruction set (managed code),
    for example, in the C-API, the untagged nil value will be tagged with
    the code ID when the value is returned to managed code.

    Clean up some MemoryHeader ToDos. (Brian Shirai)

    ๐Ÿ‘ Better exception message for NoMethodError on nil. (Brian Shirai)
    The current type coercion protocol, which is modeled on MRI C functions,
    calls and rescues many NoMethodError on nil exceptions, so this is
    optionally enabled until those mechanisms can be improved.

    โž• Added exception logging. (Brian Shirai)
    ๐ŸŽ‰ Initially, this logs exceptions raised from managed code.

    Give Bignum::from a clue. (macbookpr02010)

    ๐Ÿ”€ some synch (macbookpr02010)

    ๐Ÿ”€ some sync (macbookpr02010)

    Replaced primitive for Fixnum#/. (Brian Shirai)

    ๐Ÿšš Don't remove code caches for now. (Brian Shirai)
    The core library, standard library, and codetools are all external
    ๐Ÿ— resources. During package building, a 'rake clean' is executed, which
    ๐Ÿšš would remove these resources.

    Define Fixnum#!, Fixnum#!=. (Brian Shirai)

    Simplify r_load_int instruction. (Brian Shirai)
    ๐Ÿšš Correctness checks like this can be moved into the bytecode verifier to
    ensure that, for example, b_if_int precedes this instruction with no
    mutation of the register between the check and r_load_int.

    โž• Added more integer instructions. (Brian Shirai)

    Replace primitive for Fixnum#>>, Fixnum#<<. (Brian Shirai)

    Replaced primitive for Fixnum.===. (Brian Shirai)

    Replaced more Fixnum primitives. (Brian Shirai)

    Start unifying Fixnum, Bignum under Integer. (Brian Shirai)

    โž• Added n_promote. (Brian Shirai)
    Given two values, this instruction will return the 'lowest common type'.
    The types the instruction understands are Fixnum, Bignum, and Float. If
    both operands are the same type, no conversion is made. Fixnum promotes
    to Bignum, and both Fixnum and Bignum promote to Float.

    โž• Added missing n_promote instruction files. (Brian Shirai)

    โž• Added insn stubs for extened integer and float. (Brian Shirai)

    0๏ธโƒฃ Make --without-rpath the default. (Brian Shirai)

    Replace native_int with intptr_t. (Brian Shirai)

    โšก๏ธ Updated contributing guide (Brian Shirai)

    Start replacing Bignum, Float primitives. (Brian Shirai)

    Replaced more Fixnum, Bignum primitives. (Brian Shirai)