#295 Raspberry Pi Server based on Docker, with VPN, Dropbox backup, Influx, Grafana, etc: IOTstack

If you've ever dreamed of a Raspberry IOT

server

with Dropbox

backup

and secure remote access from anywhere via your own VPN, all

based

on Docker containers, you have to watch this video. Great marine youtubers, here's the guy who always has a Swiss accent with a new episode and New ideas on sensors and microcontrollers. Remember that if you subscribe you will always sit in the front row. Having a Raspberry Pi as a home

server

is a great addition to our IOT stuff, but setting it up requires a lot of time and knowledge. And if? We could use the latest tools to reduce this effort and add some conveniences.

Let's try it by installing Docker with many containers like Mosquito Not Red Gravano in Flux TB and others. Increase the life expectancy of your SD card by disabling swap and install. lock to ram automatically

backup

s all valuable data to the cloud or in our case to Dropbox, set up a VPN to remotely and securely access our home network from anywhere in the world. Additionally, you will learn many useful things about Docker and containers. Let's get started. First I want to thank Graham from South Africa, he did the heavy lifting of what we will look at today and created a github repository that includes the proper documentation for you.

I just had to express all my wishes and dreams. My first wish was to use Docker containers on my Raspberry Pi because it is a modern way to organize applications, but what is stalker and how does it work on normal raspbian. We install applications to the file system. They are often complex to install and have many dependencies that also need to be installed, especially by beginners. You can go crazy if something goes wrong and it sure takes a long time. A better concept is to use so-called containers, since in logistics they hide what is inside and can be easily transported because the outside is standardized.

Docker is the de facto standard for those containers, when we install Docker we get a Linux plugin that can handle containers. The containers themselves contain pre-built applications as if there were no threats or influences. Database specialists configure these containers and periodically update them to the latest version that we don't have to worry about too much. that and the best thing is that you can go to Docker Hub Comm, search for the containers you need and Docker automatically downloads them to your Esprit. Fortunately, the containers only contain what is required. By the way, this is the main difference from virtual machines, which also include a complete operating system. system and are therefore much larger copied into our

raspberry

, you can start and stop containers as well as delete them, let's continue with our wish list and the IOT server for our lab, as usual, consists of at least mosquito node-red in flux TB and Gravano, what are they?

First we go to Docker Hub and search for Mosquito, we find many containers. Luckily the first one is an official image which is good, it was updated an hour ago so it looks like it's well maintained and includes versions for many different platforms including perfect build, with a single click we get additional information like where The container stores its configuration and lock data and how to start it. Unfortunately these directories are inside the container and you cannot access these files from outside also if you delete the Container these files are gone and you lost all your valuable data and settings.

Later we will see how we can map those directories to our standard file system to keep them out of containers. This is great because then we can separate data that is specific to our installation and therefore valuable from data that is available online, there is no need to backup publicly available data because we cannot restore it quickly , we only need to backup our own data, which is a fraction of the general SD card site. I really like this concept because it is a simple and efficient protection against a faulty SD card and later we will see how easy it is to more or less automatically install all the standard components.

Next we need the speed of the node. Here we do not find any official image and neither is the most popular version obsolete. If we search a little, we get the new version. This is because Node-Red recently had a significant update to version 1.0. The new container also contains a cool 4 arm version and uses the well-known port 1880. I can go ahead and find the rest of the containers and start them to keep it simple, but it's too much work for a lazy guy like me. Fortunately, we have a more elegant way to achieve our goal. We can use Docker Compose, which greatly reduces our efforts.

It will even be easier than that. Docker Compose is a framework that contains scripts to automatically commission and start containers. If we look at the Docker Compose file on my Raspberry Pi, we first see all the different containers here called services. Let's look at the read note and find the name we give to the container, as well as the name in Docker Hub, then we see that we want the container to restart if something terrible happens. Also, the port mapping is here, so we could map, for example, internal port 18 to a different part outside the container if necessary, lastly we see the volumes here, we map the internal data directory to a directory outside the container to save our valuable data when the container is deleted.

The same applies to all other containers if we run this Yama file with the simple

docker

command. compose all containers are downloaded, installed and started completely automatically, but give us more. Wrote a menu to automate the generation of this Yama file

based

on our needs and added some additional scripts in the last few weeks. He worked hard to have it all. ready for us, all we have to do is take a new SD card, install raspbian and 2a update just the normal, then we follow your wording and start downloading the whole project with this command. We now find all the files in the IOT stack directory when we start the menu dot Sh, we see a selection of containers to choose from, maybe this will change in the future if Graham decides to add other things, first install Docker and then create the Yama file for the stack.

He can choose which containers he wants to install. as well as the additional nodes for the node rate, by the way, the hole creates a hole in your network where all the advertising falls, no longer displayed in your browsers. I strongly suggest him to install port ainur as it helps to manage his containers. After this step, our Docker Compose file is created and you can start the entire stack with Docker Compose. It takes a while, but we can use the time for a cup of coffee. The time is much faster than running Peters Cargill's script because the containers are already precompiled and we just have to copy it to our Raspberry and if we start our container stack next time, Docker will not be downloaded again and the startup is much faster. faster after some time we can test if the containers started, let's connect to the reading node. on port 1880 yes it works of course it has no flows included we have to create or add them using copy and paste as shown in video number 255.

You can also connect to Ainur port on port 9000 or Gravano on port 3000 , but how to connect. for

influx

DB has no web interface, we have to connect to the terminal inside a container. Fortunately, this is very simple. This command creates a message inside the container and if you type flux you can see your databases etc. if you restart the menu dot Sh. you will find some of the most common

docker

commands and you will also find several commands here: disable sharing and install lock on ram, run them if you want, what are they for? Log files are continuously written to your SD card and thus reduce its lifespan if you install Lock to ram log files are collected in RAM and only written every hour to the SD card much better and if you have a Raspberry Pi 4 with a lot of memory you will see that even if it does not need all the memory, it starts to create a swap file with two undesirable effects: one, the swap file is often written to the SD card and the performance of these writes and reads is very slow and two, it shortens the life of the SD card, so I delete the swap file completely and make sure it never uses more than the built-in memory, otherwise your Raspberry Pi will crash.

Now our Raspberry is at a similar level to what it was after applying the Peters Cargill script, based solely on the modern container concept. In Graham's description, you will find the commands to update the containers from time to time. Sometimes this is the charm of Docker: you always get the latest version when you install it and can easily update it, but of course we want more. We still have two items on the list. Automatic backup and remote access. What happens if your SD card breaks? you lose everything of course, you can regularly create a backup to a second SD card using an RPI clone for example, it's not very elegant.

My proposal is different with our concept of containers. We strictly separate valuable data from data that can be quickly restored from the Internet because valuable. The data is minimal compared to the size of the entire SD card, we can easily backup it to the cloud and because Graham put all the valuable data in a directory called volumes, we only need to upload this directory to Dropbox. You can do this as often as you like. If you want, of course, you can also use Google or another service for that purpose if you know that Dropbox so far does not support Raspberrys, but fortunately Andreyeva Pretty wrote a Dropbox, uploaded a link to the article on how to install it and customized the charger It's in the description.

The important thing is that you need an API key to access your Dropbox, which must be entered during the uploader installation. Also provided in the scripts is a script that exports the

influx

database and saves the volumes directory to your Dropbox. folder, you can start it by typing docker underscore backup dot SH but of course you want it to run automatically, this is done in the Chrome tab, we start with the command Krone tap - e and on this line this starts your backup always at 11:00 o'clock p.m. Of course, you can also write this line that starts the backup every hour.

Now we can save the changes with the X&Y control and you can check in your Dropbox if the backup arrives regularly. Now we are safe. In case of SD failure, we just create a new one. one that uses everything we learned before will take us less than an hour and then we will stop all the containers and copy the Dropbox volumes folder again if we are happy, also in Flux DB it works, if not we have to delete all the databases and restore the backup as shown in Gramps until the last step is installing pi VPN.

It's simple, find the link to the setup in Gramsci above, but how does it work? We need two things: a connection from our PC or smartphone to our Raspberry and encryption of our traffic like that, only our devices can connect to our home network and no one can read what you do and yes, you heard right, not only can we not connect to a Rasberry, but we can connect to all the other devices on our home network and even to the entire Internet that way, you always have a secure connection, even if you are connected to a public hotspot and the Internet thinks you feel at home , you can watch all geo-blocked content from your country, even if you're in a hotel in Timbuktu or another remote location, but first we have to overcome some obstacles.

Let's start with a first question: how can we connect to our Raspberry that is safely behind our firewall? Hopefully no one can overcome this protection and, in any case, we do not even know the IP address of our home if we do not have a fixed one which is the typical problem for most of us to know our IP address we could enter my IP address in Google it works like It is expected that Google knows everything anyway but unfortunately it changes from time to time because our Internet provider does not have enough IP addresses for all of us, so we use a trick: we use a free service like Tok DNS.

Here we create a subdomain with a fancy name. This domain will always be the same if we regularly call doc DNS from our Raspberry. dark DNS obtains our IP address and stores it until the next call. We get this normal call by adding this line to crontab; updates our IP address every five minutes. If we now connect to our fancy DNS doc domain, our traffic is redirected to our real IP address we now have a connection to our home, the first problem is almost solved, but because we are outside thefirewall, no one lets us in unless we open a port and forward their traffic to our

raspberry

, this would be very insecure, but because we installed PI VPN on our raspberry only our encrypted traffic is accepted on this port now the first problem is completely solved and we can continue to find out how we encrypt our traffic during installation pi VPN generates a key that must be transported to the smartphone or PC of your choice, the best is to use a USB stick, the worst, but the most convenient, as always, is send it by email.

Now you install this key on your remote device and because it also contains the information about our fancy duct dns domain, you can just turn on the VPN and you will be securely connected to your home network, so I can see my weather station data from wherever I am or watch my favorite team's football match, which is only available in Switzerland, that's all for today, no. summary this time in the past I saw that most of you anyway stop every time I start with a summary, so it is up for discussion whether I should include it in the future as well.

I hope this video was useful or at least interesting to you, if true, please consider it. supporting the channel to ensure its future existence you find the links in the description thank you bye