nayaread

Topic

Compilers

19 articles

Showing 19 of 19
The Cloudflare Blog

How we found a bug in Go's arm64 compiler

84 million requests a second means even rare bugs appear often. We'll reveal how we discovered a race condition in the Go arm64 compiler and got it fixed.

22 min read
Shopify Engineering

Ruby 3.2’s YJIT is Production-Ready - Shopify

YJIT, a just-in-time (JIT) implementation on top of CRuby built at Shopify, is now production-ready and delivering major improvements to performance and speed. Maxime shares the updates that have been made in this newest version of YJIT, and future plans for further optimization.

12 min read
The GitHub Blog

Introducing stack graphs

Precise code navigation is powered by stack graphs, a new open source framework that lets you define the name binding rules for a programming language.

Douglas Creager19 min read
Shopify Engineering

Our Experience Porting the YJIT Ruby Compiler to Rust - Shopify

In this post, I want to give a nuanced perspective on our experience porting YJIT from C to Rust. I'll talk about the positives, but also discuss the things that we found challenging or suboptimal in our experience.

19 min read
Shopify Engineering

Introducing Ruvy - Shopify

We’ve recently open sourced a project called Ruvy! Ruvy is a toolchain that takes Ruby code as input and creates a WebAssembly module that will execute that Ruby code. There are other options for creating Wasm modules from Ruby code. The most common one is ruby.wasm. Ruvy is built on top of ruby.wasm to provide some specific benefits. We created Ruvy to take advantage of performance improvements from pre-initializing the Ruby virtual machine and Ruby files included by the Ruby script as well as not requiring WASI arguments to be provided at runtime to simplify executing the Wasm module. WASI is a standardized collection of imported Wasm functions that are intended to provide a standard interface for Wasm modules to implement many system calls that are present in typical language standard libraries. These include reading files, retrieving the current time, and reading environment variables. To provide context for readers not familiar with WASI arguments, WASI arguments are conceptually similar to command line arguments. Code compiled to WASI to read these arguments is the same code that would be written to read command line arguments for code compiled to target machine code. WASI arguments are distinct from function arguments and standard library code uses the WASI API to retrieve these arguments. Using Ruvy At the present time, Ruvy does not ship with precompiled binaries so its build dependencies need to be installed and then Ruvy needs to be compiled before it can be used. The details for how to install these dependencies is available in the README. After building Ruvy, you can run: The content of ruby_examples/hello_world.rb  is: When running Ruvy, the first line builds and executes the CLI to take the content of ruby_examples/hello_world.rb and creates a Wasm module named index.wasm that will execute puts “Hello world” when index.wasm’s exported _start function is invoked. To use additional Ruby files, you can run: Where the content of ruby_examples/use_preludes_and_stdin.rb is: And the prelude directory contains two files. One with the

7 min read
The Cloudflare Blog

DNS parser, meet Go fuzzer

Here at CloudFlare we are heavy users of the github.com/miekg/dns Go DNS library and we make sure to contribute to its development as much as possible. Therefore when Dmitry Vyukov published go-fuzz and started to uncover tens of bugs in the Go standard library, our task was clear.

18 min read
Shopify Engineering

When is JIT Faster Than A Compiler? - Shopify

I had this conversation over and over before I really understood it. It goes: “X language can be as fast as a compiled language because it has a JIT compiler!” “Wow! It’s as fast as C?” “Well, no, we’re working on it. But in theory, it could be even FASTER than C!” “Really? How would that work?” “A JIT can see what your code does at runtime and optimize for only that. So it can do things C can’t!” “Huh. Uh, okay.” It gets hand-wavy at the end, doesn’t it? I find that frustrating. These days I work on YJIT, a JIT for Ruby. So I can make this extremely NOT hand-wavy. Let’s talk specifics. I like specifics. Wait, What’s JIT Again? An interpreter reads a human-written description of your app and executes it. You’d usually use interpreters for Ruby, Python, Node.js, SQL, and nearly all high-level dynamic languages. When you run an interpreted app, you download the human-written source code, and you have an interpreter on your computer that runs it. The interpreter effectively sees the app code for the first time when it runs it. So an interpreter doesn’t usually spend much time fixing or improving your code. They just run it how it’s written. An interpreter that significantly transforms your code or generates machine code tends to be called a compiler. A compiler typically turns that human-written code into native machine code, like those big native apps you download. The most straightforward compilers are ahead-of-time compilers. They turn human-written source code into a native binary executable, which you can download and run. A good compiler can greatly speed up your code by putting a lot of effort into improving it ahead of time. This is beneficial for users because the app developer runs the compiler for them. The app developer pays the compile cost, and users get a fast app. Sometimes people call anything a compiler if it translates from one kind of code to another—not just source

10 min read
Shopify Engineering

RubyConf 2021: The Talks You Might Have Missed - Shopify

Shopify loves Ruby and opportunities to get together with other engineers who love Ruby to learn, share, and build relationships. In November, Rubyists from Shopify’s Ruby and Rails infrastructure teams gathered in Denver at RubyConf 2021 to immerse themselves in all things Ruby with a community of their peers. If you weren’t there or want to revisit the content, we’ve compiled a list of the talks from our engineers.  A History of Compiling Ruby by Chris Seaton Love Ruby compilers? Chris does. “Why is it worth looking at Ruby compilers? Why is it worth looking at compilers at all? Well, I think compilers are fascinating. I’ve been working on them for a couple of decades. I think one of the great things about compilers, you can talk to anyone who’s a developer about compilers, because we all use compilers. Everyone’s got an opinion on how the languages should be designed. You can have conversations with anyone at every level about compilers, and compilers are just really fun. They may seem like a deeply technical topic, but they’re conceptually fairly simple. They take a file as input, they do something internally, and they produce a file as output.” In this talk, Chris dives into the history of Ruby compilers, the similarities and differences, and what we can learn from them. Learn more about Chris’ work on TruffleRuby: https://shopify.engineering/understanding-programs-using-graphs Some Assembly Required by Aaron Patterson  In typical Aaron style, this talk is filled with puns and humor while being educational and thought-provoking. Aaron shares why he wrote a JIT compiler for Ruby. Why did he write a JIT compiler?  To see if he could. “I wanted to see if I could build this thing. For me, programming is a really creative and fun endeavor. I love to program. And many times I’ll just write a project just to see if I can do it. And this is one of those cases. So, I think maybe people are asking, ‘does this

12 min read
The GitHub Blog

Using the GitHub Copilot SDK for Java

Enterprise Java developers have a new superpower—drive GitHub Copilot from idiomatic Java code with annotations, virtual threads, and more. The post Using the GitHub Copilot SDK for Java appeared first on The GitHub Blog.

Edward Burns9 min read
Shopify Engineering

Building a ShopifyQL Code Editor - Shopify

In October 2022, Shopify released ShopifyQL Notebooks, a first-party app that lets merchants analyze their shop data to make better decisions. It puts the power of ShopifyQL into merchants’ hands with a guided code editing experience. In order to provide a first-class editing experience, we turned to CodeMirror, a code editor framework built for the web. Out of the box, CodeMirror didn’t have support for ShopifyQL–here’s how we built it. ShopifyQL Everywhere ShopifyQL is an accessible, commerce-focused querying language used on both the client and server. The language is defined by an ANTLR grammar and is used to generate code for multiple targets (currently, Go and Typescript). This lets us share the same grammar definition between both the client and server despite differences in runtime language. As an added benefit, we have types written in Protobuf so that types can be shared between targets as well. All the ShopifyQL language features on the front end are encapsulated into a typescript language server, which is built on top of the ANTLR typescript target. It conforms to Microsoft's language server protocol (LSP) in order to keep a clear separation of concerns between the language server and a code editor. LSP defines the shape of common language features like tokenization, parsing, completion, hover tooltips, and linting. When code editors and language servers both conform to LSP, they become interoperable because they speak a common language. For more information about LSP, read the VSCode Language Server Extension Guide. Connecting The ShopifyQL Language Server To CodeMirror CodeMirror has its own grammar & parser engine called Lezer. Lezer is used within CodeMirror to generate parse trees, and those trees power many of the editor features. Lezer has support for common languages, but no Lezer grammar exists for ShopifyQL. Lezer also doesn’t conform to LSP. Because ShopifyQL’s grammar and language server had already been written in ANTLR, it didn’t make sense to rewrite what we had as a Lezer grammar. Instead, we

9 min read
Shopify Engineering

Announcing go-lua - Shopify

Today, we’re excited to release go-lua as an Open Source project. Go-lua is an implementation of the Lua programming language written purely in Go. We use go-lua as the core execution engine of our load generation tool. This post outlines its creation, provides examples, and describes some challenges encountered along the way.

9 min read
Shopify Engineering

Contributing support for a Wasm instruction to Winch - Shopify

Jeff Charles, a Senior Shopify Developer, shares key learnings from his successful pull request to Winch. The pull request aimed to add support for various WebAssembly instructions to Wasmtime's baseline (non-optimizing) compiler.

12 min read
Shopify Engineering

Optimizing Ruby Lazy Initialization in TruffleRuby with Deoptimization - Shopify

Shopify's involvement with TruffleRuby began half a year ago, with the goal of furthering the success of the project and Ruby community. TruffleRuby is an alternative implementation of the Ruby language (where the reference implementation is CRuby, or MRI) developed by Oracle Labs. TruffleRuby has high potential in speed, as it is nine times faster than CRuby on optcarrot, a NES emulator benchmark developed by the Ruby Core Team. I’ll walk you through a simple feature I investigated and implemented. It showcases many important aspects of TruffleRuby and serves as a great introduction to the project! Introduction to Ruby Lazy Initialization Ruby developers tend to use the double pipe equals operator ||= for lazy initialization, likely somewhat like this: Syntactically, the meaning of the double pipe equals operator is that the value is assigned if the value of the variable is currently not set. The common use case of the operator isn’t so much “assign if” and more “assign once”. This idiomatic usage is a subset of the operator’s syntactical meaning so prioritizing that logic in the compiler can improve performance. For TruffleRuby, this would lead to less machine code being emitted as the logic flow is shortened. Analyzing Idiomatic Usage To confirm that this usage is common enough to be worth optimizing for, I ran static profiling on how many times this operator is used as lazy initialization. For a statement to count as a lazy initialization for these profiling purposes, we had it match one of the following requirements: The value being assigned is a constant (uses only literals of int, string, symbol, hash, array or is a constant variable). An example would be a ||= [2 * PI]. The statement with the ||= operator is in a function, an instance or class variable is being assigned, and the name of the instance variable contains the name of the function or vice versa. The function must accept no params. An example would be def get_a; @a ||= func_call. These criteria

10 min read
Shopify Engineering

How Batman can Help you Build Apps - Shopify

Batman.js is Shopify’s new open source CoffeeScript framework, and I’m absolutely elated to introduce it to the world after spending so much time on it. Find Batman on GitHub here. Batman emerges into a world populated with extraordinary frameworks being used to great effect. With the incredible stuff being pushed out in projects like Sproutcore 2.0 and Backbone.js, how is a developer to know what to use when? There’s only so much time to play with cool new stuff, so I’d like to give a quick tour of what makes Batman different and why you might want to use it instead of the other amazing frameworks available today. Batman makes building apps easy Batman is a framework for building single page applications. It’s not a progressive enhancement or a single purpose DOM or AJAX library. It’s built from the ground up to make building awesome single page apps by implementing all the lame parts of development like cross browser compatibility, data transport, validation, custom events, and a whole lot more. We provide handy helpers for development to generate and serve code, a recommended app structure for helping you organize code and call it when necessary, a full MVC stack, and a bunch of extras, all while remaining less than 18k when gzipped. Batman doesn’t provide only the basics, or the whole kitchen sink, but a fluid API that allows you to write the important code for your app and none of the boilerplate. A super duper runtime At the heart of Batman is a runtime layer used for manipulating data from objects and subscribing to events objects may emit. Batman’s runtime is used similarly to SproutCore’s or Backbone’s in that all property access and assignment on Batman objects must be done through someObject.get and someObject.set, instead of using standard dot notation like you might in vanilla JavaScript. Adhering to this property system allows you to: transparently access “deep” properties which may be simple data or computed

9 min read