Cloud Native Application

Faster, Lower, Better with Quarkus in k8s

Yann Albou
10 min read
#kubernetes
Faster, Lower, Better with Quarkus in k8s

This blog post was originally published on Medium

Why having a very fast startup, low memory footprint, native compilation with standard frameworks fit perfectly in Docker and Kubernetes improving scalability, resiliency and security.

TL;DR;

In this blog post, I show you how Quarkus can run a Java application natively (by compiling Java bytecode to machine code with GraalVM) inside Docker using the most minimal image (and most secure) in Docker “FROM SCRATCH” and deploying it on kubernetes.

Quarkus ?

A Kubernetes Native Java stack tailored for OpenJDK HotSpot and GraalVM, crafted from the best of breed Java libraries and standards.

Quarkus is a framework developed by RedHat which was designed for the container world and which has the following characteristics:

GraalVM ?

GraalVM is a universal virtual machine for running applications written in JavaScript, Python, Ruby, R, JVM-based languages like Java, Scala, Groovy, Kotlin, Clojure, and LLVM-based languages such as C and C++. GraalVM removes the isolation between programming languages and enables interoperability in a shared runtime. It can run either standalone or in the context of OpenJDK, Node.js or Oracle Database.

GraalVm is developed by Oracle and has capability to run JVM based language natively (by compiling Java bytecode to machine code) and supports other languages like JavaScript,Python, Ruby ,C,C++ ,R etc.

Demo

To demonstrate this approach I use a demo application based on https://github.com/quarkusio/quarkus-quickstarts/tree/master/getting-started

This is a minimal CRUD service exposing a couple of endpoints over REST. Under the hood, this demo uses RESTEasy to expose the REST endpoints.

you can find the sources in the following GitHub repository: sokube/quarkus-scratch

Multi-stage Docker build

In order to generate an image with the strict minimum, the usage of a multi-stage docker build with the last stage based image being “scratch” is appropriated:

https://github.com/sokube/quarkus-scratch/blob/master/Dockerfile :

### Image for getting maven dependencies and then acting as a cache for the next image
FROM maven:3.6.3-jdk-11 as mavencache
ENV MAVEN_OPTS=-Dmaven.repo.local=/mvnrepo
COPY pom.xml /app/
WORKDIR /app
RUN mvn test-compile dependency:resolve dependency:resolve-plugins

### Image for building the native binary
FROM oracle/graalvm-ce:19.3.1-java11 AS native-image
ENV MAVEN_OPTS=-Dmaven.repo.local=/mvnrepo
COPY --from=mavencache /mvnrepo/ /mvnrepo/
COPY . /app
WORKDIR /app
ENV GRAALVM_HOME=/usr
RUN gu install native-image &&\
    ./mvnw package -Pnative -Dmaven.test.skip=true &&\
    # Prepare everything for final image
    mkdir -p /dist &&\
    cp /app/target/*-runner /dist/application

###*/ Final image based on scratch containing only the binary
FROM scratch
COPY --chown=1000 --from=native-image /dist /work
# it is possible to add timezone, certificat and new user/group
# COPY --from=xxx /usr/share/zoneinfo /usr/share/zoneinfo
# COPY --from=xxx /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/
# COPY --from=xxx /etc/passwd /etc/passwd
# COPY --from=xxx /etc/group /etc/group
EXPOSE 8080
USER 1000
WORKDIR /work/
CMD ["./application", "-Djava.io.tmpdir=/work/tmp"]

This multistage docker build is composed of 3 parts:

  1. Run maven dependencies plugin: to create an intermediate image with all maven jars to act as a cache for the next image. So that next time you build this docker image, you don’t need to load again all dependencies.
  2. Create the binary executable: With the Graalvm image + the native-image goal of quarkus-maven-plugin it will produced a 64 bit Linux executable. The important part is in the pom.xml: In order to create a nativeImage that can run natively on linux distribution that didn’t contain libc (like Alpine linux, …) the “native-image” command use the “additionalBuildArg” to compile using the “–static” argument.
  3. Create the final image using the scratch based image: “scratch” is a Docker’s reserved name to indicate to the build process to skip this line and go to the next Dockerfile command. This is not an image you can pull or run (although it appears in the DockerHub’s repository). So this is the base ancestor for all other images but it is an empty image without any folders/files, shell, libraries, … To understand how Docker interpret a “scratch” based image read https://www.mgasch.com/post/scratch/ So because of the previous step with the “–static” argument, the executable can run inside a docker image built with “from scratch”…

Build and Run the docker image

To execute this multi-stage build use the following command line:

docker build -t quarkus-app .

It is quite long and CPU-intensive to compile a native executable with GraalVm. This is therefore preferable to delegate this process to your CI/CD pipeline. The good news it that, without native compilation and during your development phase, the Quarkus hot reload is very efficient.

To run the generated image:

docker run -it --rm --name quarkus -p 8888:8080 quarkus-app

It will generate the following output:

2020-03-15 18:19:39,643 INFO  [io.quarkus] (main) getting-started 1.0-SNAPSHOT (running on Quarkus 1.2.1.Final) started in 0.028s. Listening on: http:
//0.0.0.0:8080
2020-03-15 18:19:39,644 INFO  [io.quarkus] (main) Profile prod activated.
2020-03-15 18:19:39,644 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy]

Notice the startup time of 0.028s

You can test the application in your browser : http://localhost:8888/hello/greeting/SoKube

Do not think this is a simple hello World demo: Under the hood the Quarkus framework use CDI and Resteasy as you would do with a real java application that needs to serve business requests…

Another interesting test is to limit memory and cpus:

docker run -it --rm --name quarkus -p 8888:8080 --cpus="0.05" --memory="4m" --memory-swap="4m" quarkus-app

With 0.05 of a CPU and 4m of memory the application starts in 2.503s :

2020-03-15 18:35:18,137 INFO  [io.quarkus] (main) getting-started 1.0-SNAPSHOT (running on Quarkus 1.2.1.Final) started in 2.503s. Listening on: [http://0.0.0.0:8080](http://0.0.0.0:8080/)
2020-03-15 18:35:18,137 INFO  [io.quarkus] (main) Profile prod activated.
2020-03-15 18:35:18,137 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy]

Still impressive in regard of the allocated resources!

Kubernetes Quarkus Deployment

To deploy on Kubernetes I use a local k3s distribution with k3d. for more information see a previous article I made “k3d + k3s = k8s perfect match for dev and testing”.

So first create the kubernetes cluster with:

k3d create --name quarkus-cluster --api-port 6555 --publish 8085:80
export KUBECONFIG="$(k3d get-kubeconfig --name='quarkus-cluster')"
kubectl cluster-info

Deploy the application using https://github.com/sokube/quarkus-scratch/blob/master/deploy.yaml

Clone this Git repo and change directory to “quarkus-scratch”

k3d import-images --name quarkus-cluster quarkus-app
kubectl apply -f deploy.yaml

The first line imports the previously created image called “quarkus-app” in the k3s cluster. And the second line deploys the application.

Then you should be able to reach the application using: http://localhost:8085/hello/greeting/SoKube

K8s request and limit

In the Pod spec I defined the request and limit resources to use a maximum of 1 millicore of CPU (1000 millicore = 1 CPU) and 4Mi of Memory

 resources:
          limits:
            memory: "4Mi"
            cpu: "1m"
          requests:
            cpu: "1m"
            memory: "4Mi"

The command “kubectl logs -l app=quarkus” shows the startup logs:

2020-03-17 10:42:26,239 INFO  [io.quarkus] (main) getting-started 1.0-SNAPSHOT (running on Quarkus 1.2.1.Final) started in 0.170s. Listening on: [http://0.0.0.0:8080](http://0.0.0.0:8080/)
2020-03-17 10:42:26,241 INFO  [io.quarkus] (main) Profile prod activated.
2020-03-17 10:42:26,241 INFO  [io.quarkus] (main) Installed features: [cdi, resteasy]

Still very fast despite the resource limitations ! Try such a scenario with your JEE or SpringBoot app :D

Scale your application

With a so fast startup and low memory footprint you can easily scale your application with 50 replicas on your laptop:

kubectl scale deployment/quarkus --replicas 50
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
quarkus   50/50   50           50          1h

and then scale it down also very quickly:

kubectl scale deployment/quarkus --replicas 1
NAME      READY   UP-TO-DATE   AVAILABLE   AGE
quarkus   1/1     1            1           1h

OK fun but why ?

Deploying such an application is not for the “Wahoo effect” (at least not only!).

  • CPU and Memory are the common resources used to charge for cloud or on premise environments. Those resources aren’t limitless and the billing may increase incredibly. Moreover it exists low cost solutions like Google Cloud Platform (GCP) on which you can order a VM using “Shared-core machine types”. This can be a cost-effective method for running small, non-resource intensive applications, for instance an “E2-micro 2vCPU, 1GB memory” is only 6.5 $ per month.
  • Fast startup and shutdown for scale up and rollout deployments is a key aspect for Kubernetes. Having a long startup time for your application will force you to configure probes with high timeout (except since k8s 1.16 with the new startup probe which is used for the first startup and then liveness and readiness probes are used). But in all cases it will simplify the rollout of new application versions and the rollback as well.
  • Low memory footprint on your laptop or your dev environment makes it easier to develop locally. Combining lightweight Kubernetes distribution like k3s with native applications make it possibles to have a full k8s platform being closer to the real production situation but locally!
  • Security is a major concern for production, and having a dedicated 64-bit Linux executable in a minimalist docker image (without shell, files, folders, or libraries) greatly reduces security concerns. As showed in the multi-stage build file, don’t forget to keep other security best practices like not being root…
  • Serverless architectures and products can benefit from faster startup times with real applications. Functions are executed on demand with a “cold start” when no “warm” container is available. So, in those situations, to avoid hight latency it is important to have a very fast startup time included the first request response time.

Perfect ?

Relying on Quarkus using native compilation and deploying on kubernetes is good but not sufficient!

Your application needs to be designed as a Cloud Native Application (CNA) and I am not talking about micro-services. You can write a CNA as a “normal” service but that respects some principles and avoid, for instance, a startup init of your application that load during several minutes a huge cache in memory… The design of your application is very important, none of the frameworks, tools, products can compensate for a bad design!

A drawback of the native compilation are the restrictions, especially around the use of reflection and dynamic class loading. This makes it harder (at least for now) to move all applications to native binaries, but with every release of Graal, compatibility is improving. It is why Quarkus supports a limited list of extensions but it is growing and already contains lot of extensions.

Another aspect related to the minimalist docker image is debuging ! No way to exec a command in the container! So how to debug an image that doesn’t contain a shell, tools like curl, wget …or even ls, chmod, chown, mkdir… ? I won’t go into detail on how to achieve this but the short story is to use an image like busybox and inject from this image to the minimalist image the shell and the needed tools…

Conclusion

Quarkus fit perfectly in the Cloud era, where containers, Kubernetes, micro-services or services, function-as-a-service and applications natively designed for the Cloud allow to reach high levels of productivity and efficiency.

Services, instant scalability, and high density platforms like Kubernetes require applications with a small memory footprint and fast start-up. Java was not well positioned because it favors processing times at the expense of the CPU and RAM. Combining Quarkus with GraalVM, it is not anymore the case!

Kubernetes is here to stay, so let’s prepare our developments, applications and platforms for maximum flexibility, efficiency and security.

To go further, discover our Cloud Native applications training and our Cloud Native applications expertise.