Rust/C++ interop within the Android Platform

One of many predominant challenges of evaluating Rust to be used inside the Android platform was making certain we may present adequate interoperability with our current codebase. If Rust is to fulfill its objectives of bettering safety, stability, and high quality Android-wide, we’d like to have the ability to use Rust anyplace within the codebase that native code is required. To perform this, we have to present nearly all of performance platform builders use. As we mentioned beforehand, we’ve an excessive amount of C++ to think about ignoring it, rewriting all of it’s infeasible, and rewriting older code would possible be counterproductive because the bugs in that code have largely been mounted. This implies interoperability is essentially the most sensible approach ahead.

Earlier than introducing Rust into the Android Open Supply Undertaking (AOSP), we would have liked to display that Rust interoperability with C and C++ is adequate for sensible, handy, and secure use inside Android. Including a brand new language has prices; we would have liked to display that Rust would be capable to scale throughout the codebase and meet its potential in an effort to justify these prices. This submit will cowl the evaluation we did greater than a yr in the past whereas we evaluated Rust to be used in Android. We additionally current a follow-up evaluation with some insights into how the unique evaluation has held up as Android initiatives have adopted Rust.

Present language interoperability in Android focuses on effectively outlined foreign-function interface (FFI) boundaries, which is the place code written in a single programming language calls into code written in a unique language. Rust assist will likewise deal with the FFI boundary as that is in line with how AOSP initiatives are developed, how code is shared, and the way dependencies are managed. For Rust interoperability with C, the C utility binary interface (ABI) is already adequate.

Interoperability with C++ is more difficult and is the main focus of this submit. Whereas each Rust and C++ assist utilizing the C ABI, it isn’t adequate for idiomatic utilization of both language. Merely enumerating the options of every language ends in an unsurprising conclusion: many ideas should not simply translatable, nor will we essentially need them to be. In spite of everything, we’re introducing Rust as a result of many options and traits of C++ make it tough to write down secure and proper code. Due to this fact, our purpose is to not contemplate all language options, however slightly to research how Android makes use of C++ and be sure that interop is handy for the overwhelming majority of our use instances.

We analyzed code and interfaces within the Android platform particularly, not codebases usually. Whereas this implies our particular conclusions might not be correct for different codebases, we hope the methodology will help others to make a extra knowledgeable resolution about introducing Rust into their massive codebase. Our colleagues on the Chrome browser group have performed an identical evaluation, which you’ll find right here.

This evaluation was not initially meant to be revealed exterior of Google: our purpose was to make a data-driven resolution on whether or not or not Rust was a good selection for programs growth in Android. Whereas the evaluation is meant to be correct and actionable, it was by no means meant to be complete, and we’ve identified a few areas the place it might be extra full. Nevertheless, we additionally notice that preliminary investigations into these areas confirmed that they might not considerably affect the outcomes, which is why we determined to not make investments the extra effort.

Exported features from Rust and C++ libraries are the place we contemplate interop to be important. Our objectives are easy:

• Rust should be capable to name features from C++ libraries and vice versa.
• FFI ought to require a minimal of boilerplate.
• FFI mustn’t require deep experience.

Whereas making Rust features callable from C++ is a purpose, this evaluation focuses on making C++ features obtainable to Rust in order that new Rust code could be added whereas making the most of current implementations in C++. To that finish, we have a look at exported C++ features and contemplate current and deliberate compatibility with Rust through the C ABI and compatibility libraries. Varieties are extracted by operating objdump on shared libraries to search out exterior C++ features they use¹ and operating c++filt to parse the C++ sorts. This provides features and their arguments. It doesn’t contemplate return values, however a preliminary evaluation² of these revealed that they might not considerably have an effect on the outcomes.

We then classify every of those sorts into one of many following buckets:

Supported by bindgen

These are usually easy sorts involving primitives (together with pointers and references to them). For these sorts, Rust’s current FFI will deal with them appropriately, and Android’s construct system will auto-generate the bindings.

Supported by cxx compat crate

These are dealt with by the cxx crate. This presently consists of std::string, std::vector, and C++ strategies (together with pointers/references to those sorts). Customers merely must outline the categories and features they need to share throughout languages and cxx will generate the code to do this safely.

Native assist

These sorts should not immediately supported, however the interfaces that use them have been manually reworked so as to add Rust assist. Particularly, this consists of sorts utilized by AIDL and protobufs.

We now have additionally carried out a local interface for StatsD as the present C++ interface depends on methodology overloading, which isn’t effectively supported by bindgen and cxx³. Utilization of this technique doesn’t present up within the evaluation as a result of the C++ API doesn’t use any distinctive sorts.

Potential addition to cxx

That is presently widespread information constructions equivalent to std::optionally available and std::chrono::period and customized string and vector implementations.

These can both be supported natively by a future contribution to cxx, or through the use of its ExternType amenities. We now have solely included sorts on this class that we consider are comparatively simple to implement and have an affordable probability of being accepted into the cxx venture.

We do not want/intend to assist

Some sorts are uncovered in at the moment’s C++ APIs which are both an implicit a part of the API, not an API we anticipate to need to use from Rust, or are language particular. Examples of sorts we don’t intend to assist embody:

• Mutexes – we anticipate that locking will happen in a single language or the opposite, slightly than needing to move mutexes between languages, as per our coarse-grained philosophy.
• native_handle – this can be a JNI interface sort, so it’s inappropriate to be used in Rust/C++ communication.
• std::locale& – Android makes use of a separate locale system from C++ locales. This kind primarily seems in output on account of e.g., cout utilization, which might be inappropriate to make use of in Rust.

Total, this class represents sorts that we don’t consider a Rust developer ought to be utilizing.

HIDL

Android is within the strategy of deprecating HIDL and migrating to AIDL for HALs for brand spanking new companies.We’re additionally migrating some current implementations to secure AIDL. Our present plan is to not assist HIDL, preferring emigrate to secure AIDL as an alternative. These sorts thus presently fall into the “We do not want/intend to assist” bucket above, however we break them out to be extra particular. If there may be adequate demand for HIDL assist, we could revisit this resolution later.

Different

This accommodates every kind that don’t match into any of the above buckets. It’s presently principally std::string being handed by worth, which isn’t supported by cxx.

One of many main causes for supporting interop is to permit reuse of current code. With this in thoughts, we decided essentially the most generally used C++ libraries in Android: liblog, libbase, libutils, libcutils, libhidlbase, libbinder, libhardware, libz, libcrypto, and libui. We then analyzed the entire exterior C++ features utilized by these libraries and their arguments to find out how effectively they might interoperate with Rust.

Total, 81% of sorts are within the first three classes (which we presently totally assist) and 87% are within the first 4 classes (which incorporates these we consider we are able to simply assist). Nearly the entire remaining sorts are these we consider we don’t must assist.

Along with analyzing common C++ libraries, we additionally examined Mainline modules. Supporting this context is vital as Android is migrating a few of its core performance to Mainline, together with a lot of the native code we hope to enhance with Rust. Moreover, their modularity presents a possibility for interop assist.

We analyzed 64 binaries and libraries in 21 modules. For every analyzed library we examined their used C++ features and analyzed the varieties of their arguments to find out how effectively they might interoperate with Rust in the identical approach we did above for the highest 10 libraries.

Right here 88% of sorts are within the first three classes and 90% within the first 4, with nearly the entire remaining being sorts we don’t must deal with.

With nearly a yr of Rust growth in AOSP behind us, and greater than 100 thousand traces of code written in Rust, we are able to now look at how our unique evaluation has held up primarily based on how C/C++ code is presently known as from Rust in AOSP.⁴

The outcomes largely match what we anticipated from our evaluation with bindgen dealing with nearly all of interop wants. Intensive use of AIDL by the brand new Keystore2 service ends in the first distinction between our unique evaluation and precise Rust utilization within the “Native Assist” class.

A number of present examples of interop are:

• Cxx in Bluetooth – Whereas Rust is meant to be the first language for Bluetooth, migrating from the present C/C++ implementation will occur in phases. Utilizing cxx permits the Bluetooth group to extra simply serve legacy protocols like HIDL till they’re phased out through the use of the present C++ assist to incrementally migrate their service.
• AIDL in keystore – Keystore implements AIDL companies and interacts with apps and different companies over AIDL. Offering this performance can be tough to assist with instruments like cxx or bindgen, however the native AIDL assist is easy and ergonomic to make use of.
• Manually-written wrappers in profcollectd – Whereas our purpose is to supply seamless interop for many use instances, we additionally need to display that, even when auto-generated interop options should not an choice, manually creating them could be easy and easy. Profcollectd is a small daemon that solely exists on non-production engineering builds. As an alternative of utilizing cxx it makes use of some small manually-written C wrappers round C++ libraries that it then passes to bindgen.

Bindgen and cxx present the overwhelming majority of Rust/C++ interoperability wanted by Android. For a number of the exceptions, equivalent to AIDL, the native model supplies handy interop between Rust and different languages. Manually written wrappers can be utilized to deal with the few remaining sorts and features not supported by different choices in addition to to create ergonomic Rust APIs. Total, we consider interoperability between Rust and C++ is already largely adequate for handy use of Rust inside Android.

In case you are contemplating how Rust may combine into your C++ venture, we advocate doing an identical evaluation of your codebase. When addressing interop gaps, we advocate that you simply contemplate upstreaming assist to current compat libraries like cxx.

Our first try at quantifying Rust/C++ interop concerned analyzing the potential mismatches between the languages. This led to a number of fascinating info, however was tough to attract actionable conclusions from. Quite than enumerating all of the potential locations the place interop may happen, Stephen Hines recommended that we as an alternative contemplate how code is presently shared between C/C++ initiatives as an affordable proxy for the place we’ll additionally possible need interop for Rust. This supplied us with actionable info that was simple to prioritize and implement. Trying again, the information from our real-world Rust utilization has bolstered that the preliminary methodology was sound. Thanks Stephen!

Additionally, because of:

• Andrei Homescu and Stephen Crane for contributing AIDL assist to AOSP.
• Ivan Lozano for contributing protobuf assist to AOSP.
• David Tolnay for publishing cxx and accepting our contributions.
• The numerous authors and contributors to bindgen.
• Jeff Vander Stoep and Adrian Taylor for contributions to this submit.