Understanding Docker, A Comprehension

 

I was looking for documentation for Docker recently to help someone understand how they were trying to set up their app, and most of what I came across was either too little information or much more detailed than what I was looking for. So I’m making this for those who are somewhere in the middle and trying to bridge the gap. If you went through my Nextcloud setup and had some issues, this will help you understand what you were doing and will enable you to troubleshoot it a bit on your own.

First you need to understand what docker IS though. Docker is an engine that allows you to containerize applications into their own space, without affecting the rest of the system. If you’re familiar with virtualization you already somewhat understand the concept, only you don’t have to spin up and build a whole virtual machine, this application actually runs using your host kernel and resources. It just creates kind of its own file system space that is separate from your host.

This post is going to strictly cover Linux because it’s 1:1 on how the containers are managed. You can run Docker on Windows and Mac too, but there are other factors at play. I’m not going into that for the sake of simplicity.

Now, let’s look at a real practical use. I’m going to use the Nextcloud image that we used in my  Personal Cloud  post now. You’ll notice that these images all mention tags, this is just the version number. The actual Nextcloud container supports putting tags on to bind your server to a particular version number, but the Linux Server people have theirs set up a bit more simply. We’ll just stick with latest.

Ok, now let’s look at the basic command:

docker run -d \
  --name=nextcloud \
  -e PUID=1000 \
  -e PGID=1000 \
  -e TZ=Europe/London \
  -p 443:443 \
  -v /path/to/appdata:/config \
  -v /path/to/data:/data \
  --restart unless-stopped \
  ghcr.io/linuxserver/nextcloud:latest

It’s important to note, the backslash (\) is a word wrap indicator in Linux. So you can open a text editor and put your command in as many lines as you want, just put that at the end. In reality that command reads as:

docker run -d --name=nextcloud -e PUID=1000 -e PGID=1000 -e TZ=Europe/London -p 443:443 -v /path/to/appdata:/config -v /path/to/data:/data --restart unless-stopped ghcr.io/linuxserver/nextcloud:latest

Just, don’t put spaces AFTER the slash, the command will break entirely.

The ‘-d’ in the command just means to run as detached, otherwise you would be running your command and it will run in real time until you hit CTRL^C which will kill the process as well. If you plan on your image running by itself, you should make sure to add this option.

Another important thing to note is the PUID and PGID, which is entirely dependent on your system. If you are the only admin there is a 99% chance that this will be 1000 because there are other types of “system accounts” that are used prior to that. You can verify by typing this into your command line (do not invoke sudo, run as yourself):

id $user

You will see the uid and gid come back with a numerical value and a name value. This will also probably show you all the gid’s that you are a part of. When you set these values, you are effectively saying: “Whatever the admin account is in the container is equal to this account on the host.” This will effectively declare that you are the owner of that container. This is really useful in creating persistent data, which we’ll get to in a bit. You’ll notice 1000 was taken by autologin when I set up this machine, and my GID is 1001, this is precisely why this is an important step. Don’t worry though, I have a disk encryption passphrase that is supposed to take a quindecillion years to break (90 zeros).

Name and time zone are relatively self-explanatory, if you don’t specify a name it will generate one using two random words. Time zone is UNIX epoch format so you’ll likely need to consult a list like this in order to find that. I’ll cheat a bit though and tell you that CST is America/Chicago. Skipping to the end, you’ll see the restart parameter, which should also be pretty self-explanatory. I pick unless-stopped instead of always (another option) because if there are issues and you need to troubleshoot it’s much easier when the container just stays off when you want it to.

Next comes the part that can trip people up. This is the bindings that relate between the host and container. Now obviously you do not have an actual folder path to /path/to/data. You would instead maybe have that data in /mnt/nextcloud/data so you would fill in your own values. The important part is you figure out which side is the host and container. I am going to borrow the “righty tighty” line and just remember that containers are TIGHT-knit isolated spaces. I’ll let you look back at the command and see if you can visualize that.

Did you get it? Ok cool so /mnt/nextcloud/data on my host has some files (ls -la will show permissions and hidden), if I enter the container using:

docker exec -it nextcloud bash

Then I view the files located at /data they should be precisely the same thing. But if you look in your container, you won’t see yourself as the user, you’ll see “abc” because that is the admin account they use inside the container.

This is where that UID stuff comes into play. Now abc owns the data as far as the container is concerned, and you have translated that account to mean your own on the host, so now you running that container, give yourself permission to manipulate the data. Otherwise you’ll have probably docker owning the data (960) and that isn’t even a real user.

The port is where you actually go to get to the admin side. You can keep 443 as the port, OR if you use a lot of apps with a reverse proxy, you might specify something like this:

-p 8081:80 \
-p 4341:443 \

Your container will then use the values as it’s prompted by the reverse proxy but you can get to it locally using the IP and specified port in order to get to that point.

That’s essentially the basic makeup of how Docker works. There is a lot more to it, even my personal cloud post mentions creating a network – but if you just want to get going running your own containerized apps either by self-hosting or developing, you’ll be going in the right direction.

 

 

Backing Up Your Mobile Data

 

You may have noticed that I didn’t include Mobile devices in my previous post. That is because I feel it deserves a post of its own, partially because the data is managed in ways that make this, well, not an easy task. Especially considering the fact that “offline backup” is not a term synonymous with the modern mobile ecosystem.

Between photos, contacts, calendars, and random items you’ve downloaded in between you may have trouble finding a reliable way to get everything in one go. To be honest the easiest way is to rely on the services that provide the experience – so Apple/Google or some other major service you might prefer (like some of the ones in my previous post). Of course most major cloud providers will have their own apps and they should have some sort of auto-upload function. The only caveat to this is (depending on the service) it’s only files – not app data or contacts, so you’ll likely need to rely on more apps to provide that functionality for your preferred service.

There are a few ways to get around having to use a service you may not quite trust with some of your data. Perhaps one the easiest is Syncthing . It does require you to have another target device that you use as a “server” but this can honestly just be your laptop. This app is installed both on your computer/server and your phone. It will then generate a unique ID that is entered on the device to be paired and you start setting up your folders. If you pair this with any number of backup/restore apps on the store, it should back up to your other device. Of course you’ll want to save your ID’s in a password manager.

The good news is, it adds your Camera folder by default.

For messages, your best bet would be one of the many SMS backup apps like this one for Android. You can then set the backup folder to sync via Syncthing. For Apple, it’s likely that your messages are going to be synced via iMessage, but if not – iTunes is probably going to be your best bet for an offline backup.

Two other choices for Android Backup include Fonedog and Migrate. Fonedog will take backups of your devices on your computer, and if you do not select encrypted backups, it will copy your photos and videos as they are. Contacts, calendar, etc are backed up in .info files but can be exported from the app. It is paid, but it does do a lot, and is probably worth it. You can select different components though if you only need the backup/restore.

(Slideshow)

The other option is Migrate, this is ONLY for rooted devices using a custom recovery. It creates a zip file for you to flash in recovery to restore your data. I like this because it’s 100% offline. It has a helper app you have to grant permission for it to function and it does give a warning to test uninstalling and restoring apps before you try wiping your phone. I thought that was nice.

I’ve tested this one out briefly and it seems to work as intended. The apps are backed up and contacts are in a .vcf file, though the sms and call logs are in .db files which can be opened using a database viewer.

Backing Up Your Data

Whether it’s family photos, professional work or sensitive legal documents, everyone in the modern age has some amount of data that is precious to them. Not everyone is entirely informed on the best ways to preserve that data though. In previous endeavors, I’ve seen people come to tears that their computers drive had failed after storing the last five years of their childrens’ lives on their computers. The fact of the matter is, the materials in these machines are finite and you have to expect them to fail. There isn’t a set time-frame so the sooner you take action, the better off you’ll be.

In this article I’m going to discuss a few different options, considering there really isn’t a one-size-fits-all solution. The option (or options) you decide on will depend entirely on how precious the data is to you, and how much it will cost to maintain the integrity of the data. Also it should be noted that I’m targeting the consumer class here, larger organizations I expect to have much more complex backup solutions.

I’ll start off by mentioning the most common solution – external hard drives. There are a few things to be aware of when choosing to use external hard drives. First, they fail too. Just moving all your data to a drive will not keep it safe if that is the drive that fails. Second, natural disasters – they can be damaged or lost in a fire. A decent solution to this if you want to be completely offline is to purchase two drives, one that you use consistently, and one that you store in a safety deposit box and update maybe twice a month or whatever time-frame you deem reasonable. Obviously you would be subjected to rent on the box, see your local banks.

Another option would be your standard-issue cloud storage provider. I’m not going to go into this one too much because most users are aware of services like Dropbox. However, there are a few things to consider on this, first is your cost, you’ll be paying a subscription to get any reasonable amount of storage. Second is privacy. It’s one thing to nonchalantly recognize that everything you do is being tracked. It’s an entirely different thing to read a person’s first-hand testimonial to the practice. No matter your opinion on the subject – you need to be aware that anything you upload will be visible to someone.

There are a few services that boast privacy such as Tresorit and Sync . You can also opt for a backup software that you designate the file structure such as Spideroak or Sugarsync.

Luckily there is a happy median to these solutions. I personally use a software called Duplicati. This software has been around for quite a while and has a large community of contributors that help to improve it and fix bugs. One reason I recommend this software is it runs on all the major operating systems – it just runs in the background and you open up the interface in your favorite web browser. You can set it to run the backup job on a schedule, and best of all you can encrypt those backups in-transit to many types of cloud storage. And there you have it, cloud + privacy. Win/win. The most important thing here is to store your encryption passphrase in a password manager (you DO use one, don’t you) so that it’s safe because it’s a two-way street – without that passphrase you can’t access that data either.

If you have the technical know-how, you can build out a Nextcloud server (a cloud storage solution that YOU manage) in your own home for a reasonable price, I do this personally, and have Duplicati backing up to a cloud storage I use every six hours. This is more advanced and requires you to have your own hardware, but it’s also the best way to have your cloud be completely private.