https://memgraph.com/blog/ship-it-on-arm64-or-is-it-aarch64

* * * *
[615e]
Platform
Core
 
Memgraph
Arrow icon
In-memory graph database for streaming data.
The Memgraph Toolbox /Ecosystem
 
Lab
Arrow icon
A user interface for graph data visualization.
 
MAGE
Arrow icon
A growing open-source graph algorithm repository.
 
GQLAlchemy
Arrow icon
An object graph mapper (OGM) for Python.
 
Download Platform
 
How it works
Arrow icon
Check under the hood and get a glimpse at the inner workings of
Memgraph.
Cloud
Use cases
 
Fraud detection
Arrow icon
Analyse the behavior of multiple users over time to detect anomalies
and fraud.
 
Recommendation Engine
Arrow icon
Combine multiple data sources to recommend products and services to
the right people at the right time.
 
Resource and Process Optimisation
Arrow icon
Analyse data from various data sources in real-time to improve
productivity and reduce costs.
 
Unique Case
Arrow icon
Not sure Memgraph is the right fit for your use case? Set up a call
and explore let's explore the possibilities together.
Resources
 
Playground
[62889e64dc]
Master graph algorithms in minutes through guided lessons and
sandboxes on real-world problems in the browser.
 
Community
[62889e64dc]
Join a growing community of graph developers and data scientists
building graph based apps.
 
Blog
[62889e64dc]
As blogs do. Dive into Memgraph topics.
 
Code with Buda
[62889e64dc]
Watch Memgraph's CTO demonstrate the power of graphs.
 
Email courses
[62889e64dc]
Upgrade your Cypher or Graph Modelling skills in weekly
bite-sized lessons.
DocsPricing
Discord logoGitHub logo
Free Download
Discord logoGitHub logo
[61fa30e301] 
Check out the new Python Object Graph Mapper (OGM) libraryGQLAlchemy
Right arrow icon[61fa30e301]Close icon

Categories

Tutorials

 

Ship it on ARM64! Or is it AARCH64?

Cypher QL

Memgraph Lab

Tutorials

Product

Tutorials

Graph Streaming

Python

Company

 
Tutorials
 
/

Ship it on ARM64! Or is it AARCH64?

by
Jure Bajic
July 14, 2022
Ship it on ARM64! Or is it AARCH64?

In Memgraph, we were always building packages for x86_64
architecture, but when Apple's M1 chip became prevalent in the new
laptops we had to adapt. The article on how we adapted to build our
Debian packages for ARM64, the architecture of the M1 chips, would
probably not be a hit, so let's discuss how we accomplished
supporting Docker images on different architectures.

It is important to mention that we are currently building two
separate DEB packages - one for x86_64 and the other for the ARM64
architecture.

How to build multi-architecture Docker images?

Before we start with the heavy lifting, please stay calm and trust me
it's not as hard as all the fancy words make it out to be!

I've already mentioned Docker, right? Well, you definitely need to
have it installed to be able to build Docker images. Everything I'll
be describing was done on Ubuntu 20.04 so if you are using some other
distribution you might have to adapt a bit. Make sure your Docker
version is at least 19.03. or greater.

First, enable the Docker CLI extension buildx that will allow you to
build images for multiple architectures. You can do that by following
these steps:

 1. Download the latest binary from their GitHub repository.
 2. Move the downloaded binary (mine is buildx-v0.8.2.linux-amd64 )
    into the path

~/.docker/cli-plugins like this:
mkdir -p ~/.docker/cli-plugins
mv buildx-v0.8.2.linux-amd64  ~/.docker/cli-plugins/

Note: Make sure the binary name is docker-buildx, you can rename it
or link the one you downloaded to the docker-buildx in the same
directory by running:

ln -s ~/.docker/cli-plugins/buildx-v0.8.2.linux-amd64 ~/.docker/cli-plugins/docker-buildx

 3. Change the files permission, so it can be executable, by running:

chmod 755 ~/.docker/cli-plugins/docker-buildx

 4. Now install the plugin by running:

~/.docker/cli-plugins/docker-buildx install

Voila, buildx plugin is successfully installed. To check which
architectures are now supported, run:

docker buildx inspect

And the output should look something like this:

Name:   default
Driver: docker

Nodes:
Name:      default
Endpoint:  default
Status:    running
Platforms: linux/amd64, linux/386

But wait, these are only two architectures! Well, I did promise
multi-architecture, just not a bunch of them. Just kidding - what you
need to do now is to install a builder instance that will be used by
buildx to produce images for multiple architectures.

The builder instance that I've used is from tonistiigi/binfmt
repository, and to install it, just run the command mentioned in the
README docs:

docker run --privileged --rm tonistiigi/binfmt --install all

And if you run the command docker buildx inspect now, the output
should look like this:

Name:   default
Driver: docker

Nodes:
Name:      default
Endpoint:  default
Status:    running
Platforms: linux/amd64, linux/386, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/arm/v7, linux/arm/v6

And here they are - the two architectures we are interested in -
linux/amd64 and linux/arm64. Buildx automatically recognizes the
builder image we have and can use it to build a Docker image. We also
need to instantiate this builder instance with the following command:

docker buildx create --name super_builder --use

And to test out our new builder, let's build Memgraph for both
architectures using this Dockerfile (you can see the same Dockerfile
in our repository):

FROM debian:bullseye

ARG BINARY_NAME
ARG EXTENSION
ARG TARGETARCH

RUN apt-get update && apt-get install -y \
  openssl libcurl4 libssl1.1 libseccomp2 python3 libpython3.9 python3-pip \
  --no-install-recommends \
  && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

RUN pip3 install networkx==2.4 numpy==1.21.4 scipy==1.7.3

COPY "${BINARY_NAME}${TARGETARCH}.${EXTENSION}" /

# Install memgraph package
RUN dpkg -i "${BINARY_NAME}${TARGETARCH}.deb"

# Memgraph listens for Bolt Protocol on this port by default.
EXPOSE 7687
# Snapshots and logging volumes
VOLUME /var/log/memgraph
VOLUME /var/lib/memgraph
# Configuration volume
VOLUME /etc/memgraph

USER memgraph
WORKDIR /usr/lib/memgraph

ENTRYPOINT ["/usr/lib/memgraph/memgraph"]
CMD [""]

The file looks like a standard Dockerfile, but the power of buildx is
to recognize for which architecture it is building and to use that
knowledge to our advantage. For every platform argument we specify in
the build command, buildx will generate different variable values for
the TARGETARCH argument that we use in Dockerfile to copy the Debian
package with the right architecture into our image and install it.
You can check out other buildx variables here.

The only thing we are missing are two Debian packages for AMD64 and
ARM64, and we can get those easily by running:

curl -L https://download.memgraph.com/memgraph/v2.3.0/debian-11-aarch64/memgraph_2.3.0-1_arm64.deb > memgraph-arm64.deb

And now for the grand finale, lets build those images:

docker buildx build --build-arg BINARY_NAME="memgraph-" --build-arg EXTENSION="deb" --platform linux/amd64,linux/arm64 --tag test/memgraph:2.3.0 --output type=local,dest=$PWD/images .

In due time you should see the images for both AMD64 and ARM64 in the
images directory. Well that wasn't so difficult, right?

Do it in GithubActions!

You know what they say - why spent 10 minutes doing something when
you can spend 8 hours automatizing it. So let's do just that -
automatize the build process of Docker images. Let's take a look at
Memgraph's release_docker.yaml workflow to see how it's done. So the
setup is pretty standard, so let's go through the steps to see what's
happening:

 1. Check out the repository.
 2. Use already existing GithubAction for setting up the QEMU
    emulator.
 3. And another GithubAction for setting Docker buildx CLI extension.
 4. Login to Dockerhub using credentials that need to be set up in
    GitHub secrets, to enable the upload of multiple images directly
    to Dockerhub
 5. Download Memgraph AMD64 and ARM64 Debian packages using curl.
 6. Finally, run the slightly modified Docker buildx command, the
    command now immediately pushes to the registry by replacing
    --output flag with --push. Wow, this was actually faster than
    what we did locally.

Which architecture naming convention to use?

And finally to answer the most troubling question of them all - the
one that has caused so many Slack threads, disrupted so many casual
coffee breaks and destroyed friendships - what naming convention to
use? Well, as you can see, it's complicated.

So our Debian packages all have architecture AMD64, which is by some
also called x86_64, For example, our RPM packages of the same
architecture have x86_64 architecture suffix. And for the ARM64
architecture we were conflicted should we use arm64 or aarch64 naming
for Debian packages. After reading through some documentation, and
after reading even more documentation on naming conventions of Debian
and RPM packages, we found that usually, the Debian packages have an
arm64 suffix while RPM has an aarch64 suffix. And since our Docker
image uses Debian 11 base image, we went with arm64 suffix.

Therefore we are using x86_64 and aarch64 suffix for our RPM
packages, and amd64 and arm64 for our Debian packages. And the good
thing is that Docker follows the same convention for Docker images we
use for Debian packages, and since we only have docker images based
on Debian 11 - it all fits so nicely. Friendships restored!

Conclusion

After successfully going through these instructions, you are probably
left wondering why would anyone do this locally when it's so easier
to do it with GithubActions. I would say the biggest benefit of going
through that on your own is to get more familiar with the technology
and options you can use in the CI. But the go-to approach when
releasing the product for multiple architectures should definitely be
a CI solution, in this case, GithubActions.

And now we can relax and be proud of the fact that our users, when
pulling images from the Docker registry on whichever architecture
they are currently using, will automatically get the one they should.

Share Blog
Sign up for our Newsletter

Get the latest articles on all things graph databases, algorithms,
and Memgraph updates delivered straight to your inbox

Table of Contents
 
Share Blog
Sign up for our Newsletter

Get the latest articles on all things graph databases, algorithms,
and Memgraph updates delivered straight to your inbox

Continue Reading

We don't have anything related to this article, but check out our
blog.

Subscribe to our newsletter

Stay up to date with product updates, tips, tricks and industry
related news.
[                    ]
[Sign up]
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Memgraph logo (footer)
#Github iconTwitter iconFacebook iconLinkedin icon

Free Cypher Course

Arrow icon
 
How it Works
Right arrow icon 
Products
Right arrow icon 
Why Memgraph
Right arrow icon 
Documentation
Right arrow icon
 
Community
Right arrow icon 
Blog
Right arrow icon 
Pricing
Right arrow icon 
Contact us
Right arrow icon
Company
About UsCareersLegalPress RoomNewsletterGraph Modeling
Community
DiscordDiscourseTwitterFacebook
(c) 2022 Memgraph Ltd. All rights reserved. Terms & Privacy