Built one of our projects, and want to expand upon it? Perhaps you want to build a second one, but lack the hardware? Fear not! One of the great benefits of balenaCloud is that you can combine different projects and run everything on one device. In this short guide, we’re going to take a look at how to run multiple projects on a single device, allowing you to do more with the hardware you already have.
Contents
- Introduction
- The idea behind multiple projects on one device
- How to merge separate multi-container projects (in theory)
- Let’s merge balenaSound and Pi-hole
- Let’s merge balenaSound, Pi-hole and balenaSense
- Making it permanent (git submodules)
- Conclusion
Introduction
If you’ve already built one of our projects, be it balenaSound, Dash or Sense, or you already have your own project running on balenaCloud, awesome! This Bud’s one’s for you, that’s the perfect start to this guide.
If you haven’t built a project yet, check out these guides first, as we’re gonna be building on top of those in order to show you how to complete more than one project with only one device.
I wanted to share this idea as it’s a question that comes up quite frequently on our support and in our forums. Whilst it isn’t immediately obvious how to do it, it’s totally possible and quite simple once you know how. The instructions here have been written in a generic manner with a followup example, with the aim that you can use them to combine any two projects, not only balena ones.
The idea behind multiple projects on one device
When running any project on any device, that software is unlikely to be using all the resources available. For example when running Pi-hole on a Raspberry Pi, we’re not using the audio output, so why not add balenaSound functionality on the same device? Similarly when running balenaSense, you’re not using the display output, so why not add balenaDash functionality to show your dashboard on a display connected to the same device.
The same goes for any project that you’ve developed, too. With balena, one device doesn’t have to mean one project; as long as your two (or more) applications aren’t fighting for the same hardware resources, there’s no reason why they cannot share the device they’re living on.
How to merge separate multi-container projects (in theory)
Each project, if it’s a multi-container project, contains a docker-compose.yml file, and some subfolders, one for each service (container) that make up that project. The Docker compose file is essentially a list of all the services that make up that application, so it points to each of the subfolders. Very simply, what we have to do to combine projects is to merge the compose file from each project and add all the subfolders from one to the other.
There’s a bit more to it than this, but that’s the general idea.
Merging two multi-container applications
If the project you’re merging is made up of only one service, it will have a Dockerfile in the root instead of a docker-compose.yml, and that means we can add that project as a subfolder within our main project.
Merging a single container application with a multi-container application
Possible conflicts along the way
Probably the most common conflict you’ll come across between two projects is that of networking. Say for example you are merging two projects that each have their own web server: both could be listening for requests on port 80. It’s not possible for multiple services to listen on the same network port, so one will have to be moved. The services can listen on the same port internally, but they must be mapped to different ports externally. There are ways around this by using software such as Traefik, but that’s for another post.
Altering duplicated ports to enable both services to be accessible externally
Let’s merge balenaSound and Pi-hole
OK, enough talk! Let’s hit this. To work through this, I’m assuming you’ve got a terminal available with git installed along with balena-cli. If you’ve already successfully deployed one of these projects in the past, you should be ready to go. Initially we’re going to simply copy and paste services to merge them (no git installation required), but later in the guide we’re going to look at doing it with Git submodules for a more maintainable solution.
Step 1: Let’s create a new directory for our merged project, I’m calling mine balena-sound-hole. In the CLI, I use this command (and other commands in their respective steps):
mkdir balena-sound-hole
Step 2: Next, download the Pi-hole project and the balenaSound project to a new folder on your computer, using either git clone or by downloading the .zip file from each respective repo, in which case you can skip the git clone commands below, and instead extract the downloaded zip files into the new folder.
git clone [email protected]:klutchell/balena-pihole.git
git clone [email protected]:balena-io-projects/balena-sound.git
I now have three directories in this folder: balena-sound, balena-pihole and balena-sound-hole.
Step 3: I’m going to copy the services from each of the two apps into my new balena-sound-hole app.
cp balena-pihole/docker-compose.yml balena-sound-hole/
cp -R balena-pihole/pihole balena-sound-hole/pihole
cp -R balena-pihole/dnscrypt-proxy balena-sound-hole/dnscrypt-proxy
cp balena-sound/docker-compose.yml balena-sound-hole/docker-compose-sound.yml
cp -R balena-sound/bluetooth-audio balena-sound-hole/bluetooth-audio
cp -R balena-sound/airplay balena-sound-hole/airplay
cp -R balena-sound/spotify balena-sound-hole/spotify
Step 4: Let’s merge the two docker-compose.yml files. I copied the file from each project into our new project, renaming the balena-sound one to docker-compose-sound.yml in the process. Open both files from the balena-sound-hole directory side by side in your editor of choice.
There are two things for us to do here. First, we copy all the service definitions from docker-compose-sound.yml to docker-compose.yml. It’s important to note how these blocks are indented and keep this formatting. You should end up with a services: block, that has pihole:, dnscrypt-proxy:, bluetooth-audio:, airplay: and spotify: blocks underneath it. Note that we are merging the services blocks from each file, so there should only be one services: header.
Now, as both of these projects have volumes (data storage areas) defined in their respective docker-compose.yml files, we need to merge these as well. The resultant file should only have a single volumes: block, under which it should have pihole_config:, dnsmasq_config:, dnscrypt_config:, spotifycache:, and bluetoothcache:.
Step 5: It’s time for the big moment: you can now run balena push <appName> from this directory in the same way you did when you deployed your first application. Replace <appName> with the name of your balenaCloud application.
A successful outcome
If the project built successfully, you should now find that your device is running the entire balenaSound app alongside the Pi-hole app, greatly adding to what your single device is able to do!
EGADS! A multifunction device!
Let’s merge balenaSound, Pi-hole and balenaSense
OK, so that worked and seemed almost too easy! Let’s push our luck and add balenaSense to the app we just created, running 3 different applications on one device. Let’s call this new app balena-sense-sound-hole? Maybe not.
Step 1: Working from the same directory you cloned balenaSound and Pi-hole into previously, let’s clone balenaSense, too.
git clone [email protected]:balena-io-projects/balena-sense.git
Step 2: Working through the same process as before, we’re going to copy all the services from balenaSense to our new app.
cp -R balena-sense/grafana balena-sound-hole/grafana
cp -R balena-sense/influxdb balena-sound-hole/influxdb
cp -R balena-sense/sensor balena-sound-hole/sensor
cp -R balena-sense/telegraf balena-sound-hole/telegraf
cp balena-sense/docker-compose.yml balena-sound-hole/docker-compose-sense.yml
Step 3: As before, we now take the services from the balenaSense compose file and add them to the main one. This time though, note that the service grafana is going to want to use port 80. This will conflict with Pi-hole, which also uses port 80, so let’s change that to something else using the method mentioned earlier in the guide to make it available on port 3000.
Also as before, there is a data volume used by balenaSense called sense-data. Copy this to the existing volumes block and add it below the rest.
Step 4: Now, push the application again! You should find that you are now building containers for all three applications!
Another successful outcome
Good lord! That’s a lot of containers.
That turned out to be really straightforward, no special consideration required other than changing the port for the balenaSense Grafana instance to prevent it clashing with Pi-hole. I now have one Raspberry Pi 3 device running all three balena applications.
Making it permanent (git submodules)
Now, we merged these projects manually by copy and pasting the components into a new project, but this isn’t necessarily the best way of doing things. What if there are updates to the individual projects and you want to update your app to include the latest changes? We can do this by creating a new Git repository and pulling in the other projects as submodules.
The process is very similar, in that we still have to pull in all the separate projects, and then create a new docker-compose.yml file for the root of the project. A Git installation is required for this part of the guide.
Step 1: First, create a new directory for your project and cd into it. I called mine balena-mashup.
mkdir balena-mashup
cd balena-mashup
Step 2: Initialize this directory as a Git repository.
git init
Step 3: Add the three projects as submodules.
git submodule add [email protected]:balena-io-projects/balena-sound.git
git submodule add [email protected]:balena-io-projects/balena-sense.git
git submodule add [email protected]:klutchell/balena-pihole.git
Step 4: Create the docker-compose.yml file in the root of your new project directory (`balena-mashup/docker-compose.yml’) in the same way as described before. You’ll need to copy the services and volumes from each of the 3 projects just as before, but with a small difference; the build paths need to be prefixed with their folder name, as they’re now being run as a submodule.
For example, the bluetooth-audio service by default says build: ./bluetooth-audio but this needs to be updated to a path relative to our new docker-compose.yml file, so in this case it would be build: ./balena-sound/bluetooth-audio. Apply the same premise to all the services
Step 5: Push the project in the usual way: balena push <appName>.
You’ve created your own Git repo with your project setup here, so you can go ahead and git commit the changes in the usual way.
Step 6: Now, when the time comes, you can run git submodule foreach git pull origin master and Git will update each of the individual projects with all the latest changes – neat, huh? After they’ve updated, just commit and push again to update your devices.
Conclusion
Hopefully this guide has served to demonstrate how simple it can be to use one device for multiple purposes on balena. There are of course a whole load of different combinations so we can’t possibly cover every single one, but hopefully with these examples you now have the knowledge to go ahead and try it for yourself.
Thanks for reading! If you decided to test this theory by merging some projects of your own, we’d love to hear how it went. Similarly, if you get stuck combining projects we haven’t covered here, or have any questions, let us know in our forums at https://forums.balena.io, on Twitter @balena_io, on Instagram @balena_io or on Facebook.