Easy Raspberry PI Cluster Setup with Cloudmesh from MacOS

Set up many SD Cards directly from MacS that are preconfigured to create PI clusters.
By Gregor von Laszewski (laszewski@gmail.com, laszewski.github.io), Richard Otten, Anthony Orlowski Adam … | 2021.02.23

In this tutorial, we explain how to easily set up a cluster of Pis on a Mac while burning preconfigured SD Cards. We assume you use an SD Card reader/writer that is plugged into your manager PI that we configure initially with Pi Imager.

Learning Objectives

  • Learn how to use cloudmesh burn to create many SD Cards for a cluster
  • Test the cluster after burning
  • Focus on the setup for a Mac

Topics covered

1. Introduction

Over time we have seen many efforts to create Clusters using Pi’s as a platform. There are many reasons for this. You have full control over the PIs, you use an inexpensive platform, and you use a highly usable platform and provides an enormous benefit to get educated about cluster computing in general.

There are different methods of how to set up a cluster. This includes setups that are known under the terms headless, network booting, and booting from SD Cards. Each of the methods has its advantages and disadvantages. However, the last method is most familiar to the users in the Pi community that come from single Pis. While reviewing the many efforts that describe a cluster set up most of them contain many complex steps that require a significant amount of time as they are executed individually on these Pis. Even starting is non-trivial as a network needs to be set up to access them.

Despite the much improved Pi imager and the availability of Pi bakery, the process is still involved. So we started asking:

Is it possible to develop a tool that is specifically targeted to burn SDCards for the PIs in a cluster one at a time, so we can just plug the cards in, and with minimal effort start the cluster that simply works?

You are in luck, we have spent some time to develop such a tool and present it at as part of PiPlanet1 . No more spending hours upon hours to replicate the steps, learn complex DevOps tutorials, but instead get a cluster set up easily with just a few commands.

For this, we developed cms burn which is a program that you can execute either on a “manager” Pi (or in a Linux or macOS computers) to burn cards for your cluster.

We have set up on GitHub a comprehensive package that can be installed easily we hope that it is useful to you. All of this is discussed in detail at the cloudmesh-pi-burn README2. There you can also find detailed instructions on how to use a Mac or Linux computer to burn directly from them. To showcase to you how easy it is to use we demonstrate here the setup of a cluster with five nodes.

2. Requirements

  • 5 Raspberry Pis
  • 5 SD Cards
  • Network Switch (unmanaged or managed)
  • 5 Ethernet Cables
  • Wifi Access
  • Monitor, Mouse, Keyboard (for desktop access on Pi)
  • 1 SD Card Burner(s) (we recommend one that supports USB 3.0 speeds)

For a list of possible part choices, please see:

3. The Goal

We will be creating the following setup using 5 Raspberry Pis (you need a minimum of 2, but our method works also for larger numbers of PIs). Consequentially, you will also need 5 SD cards for each of the 5 Pis. You will also want a network switch (managed or unmanaged) with 5 ethernet cables (one for each Pi).

Figure 1 shows our network configuration. From the five Raspberry Pis, one is dedicated as a manager and four as workers. We use WiFi between the manager PI to allow for you to set it up anywhere in your house or dorm (other configurations are discussed in the README).

We use an unmanaged network...

Read more »