Appcelerator Blog

The Leading Resource for All Things Mobile

Update on Hyperloop and Ti.Next, and Introducing HAL

0 Flares 0 Flares ×

NOTE: The information in this post is dated. For the very latest on Hyperloop, see here.

tl;dr: A ton of progress has been made on Hyperloop® and Ti.Next over the past six months and we have a big set of exciting changes coming in 2015.

Background
We announced an ambitious project we called Hyperloop over a year and half ago, the foundation of which would become what we have termed “Ti.Next” for the next generation architecture for the Titanium SDK. We open-sourced an early version of our work in a set of projects on Github and we have used that version of our Hyperloop compiler for about a year in Titanium (starting with 3.3.0 over a year ago and through the current release) in several key iOS modules (such as iOS Core Motion). You’ve been using our work if you’re using an SDK version from the past year.

Lessons Learned
We learned a lot by using this early version of Hyperloop. As usual, we also found some problems in the design and some practical challenges with the development workflow. The key concepts are valid and we’re excited about what we’ve learned. But we also needed to work out some of the issues — such as how to make it easier for developers of the platform SDK to actually debug the generated code or how to mix between generated code from Hyperloop JavaScript and native handwritten code, for example.

The first version of the Hyperloop architecture focused on a front-end compiler which could generate back-end code from a unified AST and the back-end code was targeted for the JavaScriptCore engine using the public C API. The back-end code generator was a set of templates that were clumsy and error-prone and hard to debug. Using the C API presented a number of additional challenges across different platforms—for example, how to better handle memory management. While it worked, it was too hard to maintain, debug and enhance it. We also couldn’t easily unit test this piece of the framework which made it a little too hard to work on.

Version 2
We’ve been working on version 2 of the Hyperloop framework and I’m thrilled to talk about it today. First off, the major body of work for this piece of Hyperloop was spearheaded and championed by a tremendous engineering effort by many very talented individuals that deserve all the credit. Namely, Matt Langston, Kota Iguchi, Russ McMahon, Pedro Enrique, Ingo Muschenetz, Neeraj Gupta, Christopher Williams and Gary Mathews. Also, in addition to HAL, we had key contributions from Tony Lukasavage, Chris Barber and Dawson Toth on other key components of Hyperloop. A big shout out to Andrew McElroy from our partner Codex Labs also for his help as well.

They came up with a very elegant solution to solve the back-end compiler portion of Hyperloop. We call it the Hyperloop Abstraction Layer, or HAL for short.

HAL basically is a cross platform, C++11 framework that represents the core AST of JavaScript, as represented in the JS specification. It’s an object-oriented framework that leverages numerous C++11 specification capabilities and does a great job of modeling the JS language.

For example, in the C framework, all types are effectively a monolithic type called a JSValueRef. It’s an opaque container in C (basically a raw pointer type to an opaque struct) that you must pass to helper functions (such as JSValueToObject or JSValueToNumber) to make it useful. This required the back-end compiler to generate a ton of boilerplate code to deal with it. It also means we have to generate a lot of protection code and memory management code, or worse, write a bunch of helper libraries which we did in version 1.

With HAL, when we encounter a JS Token such as a String or a Number, HAL also has that same exact class called a JSString or a JSNumber. It acts just like it should based on the JS spec and follows the same rules. HAL also does all the correct type handling and conversion code (for example, just pass a C++ std::string or a C char * and it knows how to handle it transparently).

The other advantage that HAL provides is transparent memory management. We put all the smarts of handling internal JS engine garbage collection and C++ memory management (smart pointers) inside the HAL framework itself. In addition, HAL is almost entirely stack-based (versus heap-based). In HAL, we are using the sophisticated C++11 memory move functions where possible and HAL almost entirely operates on the stack. This has many advantages… but namely, it requires a much smaller footprint, it’s faster and it doesn’t cause heap fragmentation (something of bigger concern on smaller mobile devices).

HAL is also flexible with mixing native code and pure JavaScript code. For example, in HAL, you can simply pass a string of valid JS code and you can turn that easily into a HAL object as if it was written in native C++. And vice versa, in HAL you could pass a pure HAL-enabled native function, scalar value or C++ class and work with it in pure JS code as if it was built-in to the JS engine. The key advantage of this is that we can mix and match generated and handwritten code, in either C++ or JS.

Let me give you an example of where that could be used. Let’s say we have a Titanium API that needs to have many different internal native operations to implement. For speed, we might write this initially in pure JS code, compiled into native with Hyperloop. However, let’s say after profiling this code, it would be more efficient for this API to be ahead-of-time compiled in C++ or even assembly to achieve better performance. In HAL, we will be able to support these sort of dynamic compilation units by mixing and matching different source inputs flexibly.

HAL was developed to be cross-platform out of the box and we used C++11 specifically to achieve this. A lot of work has been done by all the modern OS platforms and tools to support portable C++ with the C++11 specification. Apple iOS, Google Android and Microsoft Windows all support C++11. This allowed us to build a very solid cross-platform HAL framework that could be compiled on all the major platforms and unit tested for each. It also allowed us to use a common C++ unit testing framework (in our case, we selected Google’s GTest tooling) as well as CMake for a common build infrastructure. Our continuous integration builds are tested with each check-in across each OS using Travis and Jenkins, and this solves our testing issues from version 1.

How do you get it?
There are a few additional steps we have to take before we can make HAL available on Github. Namely, we have some patent issues that are in progress that have to be complete before we can make it publicly available. We have numerous existing granted patents around our compiler technologies and have filed updates and new patents around our inventions and we need to have that process completed before we can make it publicly available. Our current goal is to make it available in January, pending finalizing the issues above.

HAL will be licensed under the Apache Public License just like the Titanium SDK.

However, you can get a sneak peek of HAL being used in our Titanium Windows Developer Preview Release.

What’s next?
Our next big effort is to re-work the front-end compiler to use HAL in the back-end code. With that, we can start to bring HAL into iOS and Android more directly.

Additionally, our near-term plan is to use HAL to create a new module SDK that will allow devs to extend Titanium in JavaScript (for iOS, Android and Windows) with full access to native APIs. This will also make it easier for you to start using Hyperloop capabilities in Ti.current sooner rather than later.

There are also some exciting things we prototyped with HAL that will take some time to explore more. For example, we believe with HAL we could generate an engineless application if the application used “strict” JS code. This could have profound implications if we could make this work in real applications. There’s a ton of optimizations that we believe we can continue to innovate around once we get the basics working well.

Stay tuned. 2015 looks to be a very exciting year for the Titanium SDK.

0 Flares Twitter 0 Facebook 0 Google+ 0 LinkedIn 0 Email -- 0 Flares ×

12 Comments

  1. Thank you for the update

  2. Brandon Martin

    Wow! Very cool.

  3. Amazing .. a great news in this Christmas time

  4. Thanks for the update. Sounds pretty cool.

  5. Very cool!!

    Makes me even more sure that I selected the right platform for mobile development when I switched from another framework more than a year ago 😉

  6. Krish

    Thanks for information, amazing technology insights. Waiting for first version of HAL, looking for tutorials on how to use with Ti.Current if possible.

    Do you still plan to use Nodejs as runtime for Ti.NEXT? We use many external, open source Modules like NEDB, Backbone, underscore, hoping that we will not have any impact on using JS libraries.

    Looking for good 2015.

  7. olivier

    Thank you for the information! It was necessary, I was getting very worried about the future of the platform.

    But I hope it does not go too delaying adding features and platforms: Android wear, apple watch, etc.

  8. Manoj Kumar

    Great!!! Thanks for the update.

  9. Max Khrichtchatyi

    Thanks! Happy New Year :-)

  10. Ran

    Sounds impressive!

  11. Very cool, 2015 looks to become a fantastic year indeed! Thanks for the insights!

  12. Waiting for the new Titanium SDK! great job!

Comments are closed.

Sign up for updates!

Become a mobile leader. Take the first step to scale mobile innovation throughout your enterprise.
Get in touch
computer and tablet showing Appcelerator software
Start free, grow from there.
0 Flares Twitter 0 Facebook 0 Google+ 0 LinkedIn 0 Email -- 0 Flares ×