Migrating Our iOS Construct System from Buck to Bazel | by Qing Yang | The Airbnb Tech Weblog | Feb, 2024

Qing Yang
The Airbnb Tech Blog

How Airbnb achieved a clean and clear migration from Buck to Bazel on iOS, with minimal interference to developer workflows

By: Qing Yang, Andy Bartholomew

At Airbnb, we’re dedicated to offering one of the best expertise for our engineers. To supply a cohesive and environment friendly construct expertise throughout all platforms, we’ve determined to undertake Bazel as our construct system. Bazel is a strong construct system broadly utilized within the business. In alignment with Airbnb’s tech initiatives, each our backend and frontend groups initiated the migration course of to Bazel. Within the first Bazel put up, we begin with our iOS improvement migrating from Buck to Bazel.

We’ll describe the migration strategy which concerned two primary items of labor: migrating the construct configuration and migrating the IDE integration. Such a transition can probably disrupt engineers’ workflows or hinder the event of latest options, however we had been in a position to efficiently migrate them with out disrupting the day-to-day developer expertise. Our intention is to assist others who’re presently present process or planning an analogous migration.

On the subject of construct configuration, Buck and Bazel exhibit vital similarities. They share a comparable listing construction, make use of comparable command line invocation, and, importantly, each make the most of the Starlark language. These similarities current a possibility for configuration sharing between the 2 construct methods. This may permit us to reuse our Buck configurations in Bazel, whereas avoiding slowdowns through the “overlap” section once we had been within the means of migrating and nonetheless actively utilizing each construct methods.

Sadly, there’s a significant drawback: Buck and Bazel make use of distinct guidelines with completely different parameters. For example, Buck presents guidelines similar to apple_library and apple_binary, whereas Bazel, relying on the exterior rule units, options guidelines like swift_library and apple_framework. Even in circumstances the place the 2 methods have guidelines with the identical title, similar to genrule, the syntax for configuring these guidelines is usually unalike. The completely different design philosophies of those two methods end in varied incompatibilities as effectively. For example, Bazel doesn’t have the read_config perform to learn command line choices in a macro.

Hiding the Variations with rules_shim

After conducting an in-depth evaluation of each Buck and Bazel, we devised a complete structure for the construct configuration to leverage the similarities and handle the variations between every system.

The construct configuration layers

On the core of this structure lies the rules_shim layer, which introduces two units of guidelines: one for Buck and one other for Bazel. These rule units act as wrappers across the native and exterior guidelines, providing unified interfaces to the layers above.

How does rules_shim work, precisely? By making use of native repositories, we will level the rules_shim repository to completely different implementations relying on the construct system.

That is what the outcome seems like in Buck’s .buckconfig:

[repositories]
rules_shim = rules_shim/buck

[buildfile]
title = BUILD

Be aware that we’ve additionally configured Buck to make use of BUILD because the config file, and renamed the present BUCK information to BUILD, so the identical configuration will be acknowledged by each Buck and Bazel.

In Bazel’s WORKSPACE, we do the next:

local_repository(
title = "rules_shim",
path = "rules_shim/bazel"
)

In a daily BUILD file, we use my_library to wrap across the native guidelines and supply the identical interface for every utility:

load("@rules_shim//:defs.bzl", "my_library", …)

The app-specific guidelines layer solely must know the interface, not the implementation. In consequence, at any time when we execute Buck or Bazel instructions, the construct system is ready to retrieve the corresponding implementation from the rules_shim layer. A notable benefit of this design is that we will simply take away the rules_shim/buck after the migration.

Unifying the genrule interface

Inside our iOS codebase, we closely depend on generated code to handle boilerplate and scale back the upkeep burden for engineers. Given the completely different syntax for genrule scripts between the 2 construct methods, we additionally designed a unified interface for genrule. In consequence, the identical genrule script can perform throughout each construct methods. As you’ll have guessed, the conversion course of is carried out within the rules_shim layer.

We designed the predefined variables within the unified genrule interface.

Changing read_config with choose

Conditional configuration is unavoidable, as a result of there are at all times completely different variants of a constructed product, similar to debug builds and launch builds. Buck supplies a perform known as read_config that reads command line choices in a macro, whereas Bazel doesn’t have this because of the system’s strict separation of loading phase. It’s price noting that Buck does help the select perform, though it’s undocumented. We’ve migrated all situations of read_config to choose-based circumstances.

deps = choose(
"//:DebugBuild": non_production_deps,
"//:ReleaseBuild": [],
# SELECT_DEFAULT is outlined in rules_shim to accommodate
# the completely different default strings utilized by Buck and Bazel
SELECT_DEFAULT: non_production_deps,
),

Total, this design achieved the utilization of a single construct configuration for each construct methods, with minimal adjustments to our BUILD information themselves. In observe, iOS engineers at Airbnb hardly ever have to manually modify BUILD information, that are robotically up to date from an evaluation of the underlying supply code. Nevertheless, in circumstances the place it does happen, they’ll depend on the unified interface without having to pay attention to the particular underlying construct system.

iOS Engineers at Airbnb primarily work together with the construct system by Xcode. Since first adopting Buck, we have now been using Buck-generated Xcode workspaces for native improvement. Over time, we’ve developed varied productivity-boosting options on high of this setup, together with the Dev App, a small improvement utility centered on a single module; Buck Native, which makes use of Buck as an alternative of Xcode for constructing and leverages distant cache; and Focus Xcode workspace, which considerably improves IDE efficiency by loading solely the modules being labored on.

Within the Bazel ecosystem, a number of options exist for producing Xcode workspaces. Nevertheless, on the time of our analysis, none of them totally met our necessities. Moreover, any IDE integration must help not solely constructing, but additionally modifying, indexing, testing, and debugging. Given the confirmed observe file and stability of our present workspace setup, we deemed the danger of adopting a very new one to be exceedingly excessive. Therefore, we determined to develop our personal generator to create a workspace near our present setup. We selected XcodeGen, a preferred instrument on this space, as a result of it generates Xcode tasks from a YAML configuration, serving as an abstraction layer to separate the construct system implementation particulars.

The move of producing the Xcode undertaking

We carried out this migration course of in three phases.

Firstly, we utilized buck question to assemble all the required info from the codebase and generate an Xcode workspace, changing the buck project command. This new workspace invoked buck construct through the construct course of. By preserving the construct system unchanged, we had been ready to make sure compatibility and consider the efficiency of the brand new generator.

Secondly, we carried out a parallel implementation in Bazel utilizing bazel question and bazel construct, incorporating a easy --bazel choice within the era script that permits switching between the 2 construct methods inside Xcode. Aside from the construct system, the person interface remained an identical, guaranteeing that every one IDE operations continued to perform as earlier than.

Lastly, after a enough variety of customers opted for Bazel and all Bazel-powered options underwent intensive testing, we made the --bazel choice the default, ending for a clean transition to Bazel. Though we didn’t have to, we might simply roll again if points had occurred. Just a few weeks later, we eliminated Buck help from the generated undertaking.

The tip results of this migration is spectacular. In comparison with the Buck-generated undertaking(buck undertaking), the era time with XcodeGen has been lowered by 60%, and the open time for Xcode has decreased by greater than 70%. In consequence, this new workspace setup obtained high rankings in an inside developer expertise survey, showcasing the numerous enhancements achieved by this course of.

“All issues in laptop science will be solved by one other stage of indirection.” — David Wheeler

Wherever we relied on Buck, we launched a standard interface abstraction and injected separate implementations to deal with the variations between Buck and Bazel. Due to the “indirection” precept, we had been in a position to check and replace every implementation with out dramatically rewriting the code, and we efficiently transitioned from Buck to Bazel seamlessly throughout all use circumstances, together with native improvement, CI testing, and releases. The migration course of was executed with out disrupting engineers’ workflows and, in reality, allowed us to ship a number of new options, together with SwiftUI Previews help.

Since Bazel grew to become our iOS construct system, we have now noticed notable enhancements in construct instances, significantly for incremental builds. This shift has enabled us to leverage shared infrastructure, similar to distant cache, alongside different construct platforms inside Airbnb. Consequently, we have now fostered elevated collaboration throughout platforms.

Migrating our iOS construct system is simply the primary of numerous Bazel migrations underway or accomplished at Airbnb. We’ve repos for JVM-based languages (Java/Kotlin/Scala), for JavaScript, and for Go, that are both utilizing Bazel already, or will probably be sooner or later. We imagine a single construct instrument throughout our complete codebase will permit us to extra successfully leverage our investments in construct tooling and coaching. Sooner or later, we’ll be sharing classes realized from these different Bazel migrations.