Source of Image: dojoblog
When we describe a typical operating system kernel on a typical machine (be it physical or virtual), we are normally talking about a distinct piece of software which runs in a separate processor mode (kernel mode) and address space from the rest of the software running on that machine. This operating system kernel generally provides critical low-level functions which are leveraged by the other software installed on the box. The kernel is generally a generic piece of code which is trivially tailored (if at all) to the application software stack it is supporting on the machine. This generic kernel normally provides a wide range of rich functions, many of which may be unneeded by the particular applications it is being asked to support.
In fact, if you look at the total software stack on most machines today, it is often difficult to figure out just what application will be run on that machine. You are likely to find a wide swath of hundreds, if not thousands, of low-level utilities, plus multiple databases, a web server or two, and a number of specialized application programs. The machine may actually be charged with running a single applica- tion, or it may be intended to run dozens simultaneously. Careful analysis of the startup scripts will yield hints as to the final solution set which will be run on the machine, but it is far from certain, as a suitably privileged user may elect to invoke any of a number of applications present on the box.
The Unikernel Difference
The footprint of a unikernel-based machine, however, is quite different. A unikernel’s role in a machine (or virtual machine image) is similar to that of other kernels, but the implementation parameters are significantly different.
For example, an analysis of a unikernel-based machine’s code does not suffer from the ambiguity of most other software stacks. When you look at a unikernel system, you will find one and only one application present. The multiple applications of a standard software stack are gone, as are the aforementioned plethora of generic utilities and support functions.
But the trimming of excess doesn’t stop there. Not only is the application stack trimmed to the bone, but the operating system functions are likewise reduced. For example, multi-user support is gone. Multiple process support is gone. Advanced memory management is gone.
Think that’s radical? Consider this: the whole notion of a separate operating system layer is gone as well! No longer is there a separate address space for the kernel and a separate address space for the application. Why? Because the kernel functions and the application are now part of the same program. In fact, the entire software stack is comprised of a single software program, providing all needed application code and operating system functions. And, if that weren’t enough, the operating system functions contained in the unikernel provide only those functions needed to power the application in question – all other unneeded operating system functions have been removed entirely.
A Software Stack Reflecting The Realities of a New Century
This singularity of purpose is the radical rethink behind unikernels. For decades in this industry, we have worked with the concept that the best architecture for any machine is to start with a generic multiuser operating system base, load on a wide array of useful utilities, add every application we might possibly want to use, and then top it off with some type of packaging software to manage this entire mess.
Thirty-five years ago, this approach was entirely sensible. Hardware was expensive. Virtu- alization options were very limited or unavailable. Security was limited to making sure the guy seated next to you in the computing center wasn’t watching you type your password. A single machine had to handle many users run- ning many applications at the same time in order to be cost effective. When I was in college (a millennium or two ago), before the arrival of the personal computer, the college computer center had one insanely expensive machine (by today’s standards) – a DEC PDP-11/34a, for the gray-headed who might care – equipped with 248 kilobytes (no, that’s not a typo) of usable memory and 25 megabytes of disk space to support an entire campus of computer science, engineering, and mathematics students. That single machine had to service every function which a couple hun- dred students could think of in a given semester.
Compare that ancient dinosaur of computing history to a modern smart phone and you dis- cover that the phone has multiple orders of magnitude more computing power than that machine had. So why are we still creating machine images using the same rules we used back in the computing stone age? Doesn’t it make sense to rethink the software stack to match the new realities of computing?
In the modern world, hardware is cheap. Virtualization is omnipresent and efficient. And just about every computing device is connected to a vast worldwide network of potentially malicious hackers. Think about it: a DNS server really doesn’t need gigabytes of bloat to do its job. An app server doesn’t really need to have a thousand utilities available for the use of the desperado who just exploited a hole which gave him virtual command line access. And a webserver doesn’t need to validate the command line logins of 500 different timesharing users. So why are we still using an archaic software stack concept which supports all these undesirable scenarios?
The Brave New World of the Unikernel
So what should a modern stack look like? How about this: a singular application image, virtualized, highly secure, ultra-lightweight, with incredibly fast startup times. That’s something that a unikernel can deliver. Let’s break it down:
A Singular Image: The hundreds of utilities and masses of applications layered on top of a generic kernel is replaced with a single executable containing all the needed application and operating system code in a single image. It contains nothing more than what is needed.
Virtualized: Just a few years ago, you’d be lucky to fit anything more than a handful of virtual machines on a single server. The limits in memory in hardware combined with the old- school, memory-hungry software stack simply didn’t allow room for too many VMs to coexist on a single server at one time. With today’s beefy servers equipped with many gigabytes of memory, we no longer need to settle for a handful of VMs per server. If each VM image were small enough, we could run hundreds – or even thousands – of VM appliances simulta- neously on a single server.
Secure: In the age of the cloud, we find malicious hackers routinely penetrating servers everywhere, even the servers of the largest corporations and government agencies. Such violations frequently exploit flaws in a network-enabled service to break into the lower levels of the software stack. From there, the malefactor can use utilities and other pro- grams present on the box to commit their nefarious acts. In a unikernel stack, there are no other pieces of software to assist the malevolent hacker. The hacker has to be bright enough to penetrate the application, but then be brighter still to do evil things without any resident tools to leverage. It does not make the software completely secure, but it does raise the security bar significantly – and that’s an advancement which is long overdue in the cloud.
Ultra-lightweight: A normal VM could con- sume gigabytes of disk space and memory just to make a few services available to the network. With a unikernel, it’s possible to gut those requirements. For example, MirageOS (a popular unikernel system) has a functional DNS server which uses 449 kilobytes of disk space – yes, less than half of a megabyte. ClickOS, a network appliance unikernel sys- tem from NEC labs, has actual network devices that use just 6 megabytes of memory while successfully processing over 5 million packets per second. These are by no means atypical examples of unikernel-based devices. Given the diminutive footprint of unikernels, the notion of stacking hundreds or thousands of these tiny VMs on a single host server no longer seems farfetched.
Fast startup: Normal VMs can take serious time to boot up. It can take a minute or more on modern hardware for a full operating system and application stack to come online.
However, this is not the case for a unikernel- based VM. Most unikernel VMs boot in less than a tenth of a second. For example, ClickOS network VMs have been documented to boot up in under 30 milliseconds. That’s quick enough to birth a responding VM as soon as the service request appears on the network (which is one of the things which the Jitsu project does – see http://unikernel.org/files/2015-nsdi-jitsu.pdf).
But Don’t Containers Already Do This?
Containers go a long way toward creating lightweight, fast VMs. But containers still rely on a shared beefy operating system underneath the covers. That is still a lot to lock down from a security standpoint. And it is clear that we need to step up our security in the cloud, rather than pursue the same old security methods which are rapidly becoming problematic in the cloud. Beyond that, the final footprint of unikernels is still smaller than what containers can provide. So containers move in the right direction, but unikernels manage to go farther in the direction which we need in tomorrow’s cloud.
So How Do Unikernels Work?
As mentioned earlier, a traditional machine is built from the bottom up: you select a general-purpose kernel, add a slew of utilities, and then add the applications. Unikernels are the exact opposite: they are built top-down. Focus on the application you need to run and then add just enough operating system functions to make it come to life.
Most unikernel systems rely on a compile- and-link system which takes the source code of the application and links in libraries which provide only those operating system functions that application requires, resulting in a single compiled image that can be run in a VM as-is without any additional software.
How Do You Debug the Result?
Since there is no operating system or utilities in the final product, most unikernel systems use a staged approach to development. Often, a compile while in the development phase will yield an executable suit- able for testing on Linux or other Unix-like operating system. This executable can be run and debugged like any standard program. Once you are satisfied with the result, you recompile with the production switch turned on, creating the final image meant to be run standalone in a VM.
The lack of debugging tools on the production machine isn’t as awkward as it may sound at first. The limitations on debugging a production image are mitigated by the fact that in most organizations, developers are not allowed to actually debug on a production machine anyway. Instead, they gather up logs and other information, recreate a failure on a development platform, make corrections, and redeploy the result. That exact sequence is readily available in the unikernel world. You just need to make sure that your production image will produce enough logging output to facilitate forensic reconstruction of failures – and you may be doing that already in your standard application.
What Unikernel Systems are available?
There are a good selection of unikernel systems already in place, supporting a large number of languages, but the number of unikernel projects continues to grow. Some of the more popular unikernel systems include:
• MirageOS: One of the oldest unikernels, it uses the Ocaml language.
• HaLVM: Another early unikernel, it is built on Haskell.
• LING: This long-established project employs Erlang.
• ClickOS: Optimized for network appliances, this has bindings for C, C++, and Python.
• OSv: A slightly different unikernel, it is based on Java with additional bindings which cover a large number of languages. Most JAR files reportedly drop in and run.
• Rumprun: Leveraging the modular code from the NetBSD project, it targets most any “POSIXy” application which doesn’t need to fork. It is exceptionally well positioned for migrating existing applications into the unikernel world.
Are Unikernels a Panacea?
Unikernels are far from being a panacea. As they are single-process entities working in a single address space with no advanced memory management, there are certain programs which will not easily lend themselves to existence as a unikernel. However, a huge number of services running in datacenters across the world will easily fit the bill. By converting these services into lightweight unikernels, we can reassign the server capacity we freed up to the heavier tasks which could benefit from additional resources.
The number of tasks which lend themselves to being unikernels is larger than you might think. In 2015, Martin Lucina announced the successful creation of a “RAMP” stack. A variant of the common “LAMP” stack (Linux. Apache, MySQL, PHP/Python), the “RAMP” stack employs NGINX, MySQL, and PHP each built on Rumprun. Rumprun is an instance of a Rump kernel, which is a unikernel system based on the modular operating system functions found in the NetBSD project. So even this very common solution stack can be successfully converted into unikernels.
For More Information
To learn more about unikernels, check out http://www.unikernel.org/ or watch the talk I delivered at Southeast Linuxfest in 2015 at https://www.youtube.com/watch?v=8UgiPODw3CY .
About the Author:
Russell has spent over two decades evangelizing Open Source, serving most recently as the evangelist for the Xen Project. Since his introduction to Linux in 1995, he has relent- lessly promoted the concept of Open Source to anyone who would listen.
He has over 200 pieces published, including columns for In- foworld and Processor magazines, one book published, plus one on the way. He has spoken over 100 times at Open
Source conferences, including the biggest Linux conferences in North America. A for- mer panelist on The Linux Show weekly webcast, he also has many years of experi- ence employing Open Source software in solutions for clients.
He is currently looking for a new opportunity as an evangelist and/or community person for a FOSS project.
The article comes from BSD Mag Vol. 10 No. 03