Ubuntu 21.10 dual boot with encryption and LVM

This guide shows how to install Ubuntu in a dual boot configuration with encryption and LVM. The method was tested only on Ubuntu 21.10. The behaviour of the installer in earlier or later versions or other flavours of Ubuntu may be different and not work.

Why would you want to do this? If you have an existing Windows installation (or any other OS) that you don’t want to mess with but you also want to install Ubuntu and keep your data encrypted then this might be useful to you. Encryption is essential to avoid your data being easily accessed if your device is lost or stolen and is required to comply with GDPR for anyone handling personal data.

Encryption has its own problems and risks, but this guide is not addressing those. I’m assuming you understand that and just want to know how to do it.

Doesn’t the installer do that anyway I here you ask? Well, no. When you fire up the installer you get some options (depending on what is already on the disk):

• Install Ubuntu alongside Windows Boot Manager – lets you shrink the Windows installation and install Ubuntu without encryption.
• Erase disk and install Ubuntu – as it says, but provides options to use LVM and encryption.
• Something else – gives you more control, but does not provide options for LVM and encryption.

If you are installing both Windows and Ubuntu from scratch then consider using the ‘Erase disk’ option and running Windows in a VM.

However, if you want to leave the existing Windows alone and dual boot with encrypted Ubuntu on LVM just using the installer then you’re out of luck. This feature has been requested for several years (and related) but has met stubborn resistance. The basic argument against seems to be that encryption doesn’t solve every problem – which is true, but doesn’t help those wanting or needing encryption.

There are various guides around that achieve this with lots of command line work. The approach here is to use the installer as much as possible and minimize use of the command line.

We are going to follow the results produced by selecting the ‘Erase disk’ option and using LVM and encryption:

• Unencrypted /boot and /boot/efi partitions
• Encrypted partition (LUKS2) used as a LVM Physical Volume
• Two LVM Logical Volumes: one for swap, one for root

Two things that you might question:

• Unencrypted /boot – this is a possible attack vector and other guides show how to encrypt this.
• A swap partition rather than swap file. A test install with ‘Erase disk’ and no LVM or encryption creates a swap file but a test install with ‘Erase disk’ and LVM/encryption creates a swap partition. I haven’t dug into why that is the case. Certainly, if you don’t use LVM then a swapfile is easier to alter than a partition. But if you do use LVM then I’m not clear on the pros and cons of swap file vs swap partition.

However, we’re going to assume this is a reasonable pattern to follow since it is a default one produced by the installer.

Right, enough preamble.

Steps

Do be careful to follow the steps in order.

Make sure you backup any important data. Messing about with partitions, LVM, formatting and encryption provides plenty of opportunities to trash your data! One wrong command and your data is toast. I’d suggest experimenting with this in a throw-away VM first. Your data is your responsibility. What follows are my notes of what worked for me. Use at your own risk.

Follow the installation guide until you get to Step 4. Then select ‘Try Ubuntu’ rather than ‘Install’ so you get a full desktop environment. Once booted, click on the Installer to start the installation.

At Step 6, select ‘Something else’

My system has 4 partitions: EFI and 3 Windows-related ones which are /dev/nvme0n1p1 to /dev/nvme0n1p4

We’re going to create two partitions in the free space on the disk. One for /boot, one for encrypted content.

Click on the free space then the + icon. Fill in the details for /boot : size 538MB, use as ext4

Click OK. This is partition 5.

Click + again and use the rest of the space for the encrypted volume.

• In ‘Use as’ select ‘physical volume for encryption’
• Choose a security key that you can remember! You need this every time you boot.
• I suggest you enable the recovery key. However, the key does not get written to the file it says so click the eye icon and copy/paste that key into a file called recovery.key We’ll want this later.

Click OK and it will go off and encrypt the partition. This is partition 6.

Unfortunately the installer does not let us configure logical volumes in this ‘something else’ menu option, but it can install to them if they exist, so we need to resort to the command line to create them. Fortunately, we don’t need to do much.

Leave the Installer as it is. Start a terminal window and copy these lines one by one. If your encrypted partition is not p6 or your device is not nvme0n1 then adjust appropriately!

The previous encryption step created /dev/mapper/nvme0n1p6_crypt. Now create a PV (Physical Volume):

$ sudo pvcreate /dev/mapper/nvme0n1p6_crypt

Create a VG (Volume Group) called vgubuntu on the PV we just created:

$ sudo vgcreate vgubuntu /dev/mapper/nvme0n1p6_crypt

Create a LV (Logical Volume) called swap_1 of size 2GB in the VG vgubuntu

$ sudo lvcreate --size 2G -n swap_1 vgubuntu

Create another LV called root using all of the free space in the VG vgubuntu

$ sudo lvcreate -l 100%FREE -n root vgubuntu

Your output should look like this:

Return to the installer and click the ‘back’ button. This causes the installer to re-examine the disks and partitions.

Click the ‘Something else’ option again and you should see the new LV’s showing up.

So now we can go ahead and tell it what we want to use:

Select the root LV /dev/mapper/vgubuntu_root. Click Change and configure as Ext4, format and mount as /

Select the swap LV /dev/mapper/vgubuntu_swap-1. Click Change and select swap.

Select the p5 partition /dev/nvme0n1p5 that we created earlier. Click Change, select ext4, format, mount as /boot

Note: the existing EFI partition /dev/nvme0n1p1 will automatically be mounted as /boot/efi

It should look something like this:

Go ahead and click ‘Install Now’. You’ll get a warning like this:

Assuming you are happy with what it says, click ‘Continue’.

The installation will start and ask you things like setting the timezone. STOP. Leave that for now.

Because we used the installer to do the encryption and then used ‘back’, it seems to forget that a file /etc/crypttab needs to be created on the new disk. Fortunately the file is not hard to create but we need to do it after the installation has started but before it reaches the point where it updates grub.

Open a terminal and run:

$ echo "nvme0n1p6_cryptUUID=`lsblk /dev/nvme0n1p6 -o UUID -d -n` none luks,discard" > crypttab

use cat to check the result looks like this: (Obviously your UUID will be different)

and then copy it to the newly mounted disk:

$ sudo cp crypttab /target/etc/crypttab

Now return to the installer and continue. You may get an error about setting the recovery key. Just click OK for now.

Wait for the installer to complete. It will ask whether you want to ‘Continue testing’ or ‘Reboot now’. Don’t click either one yet! (When you do, the disks are unmounted.)

Now we’ll deal with the error about setting the recovery key. Assuming you saved the key to the file recovery.key, run the following command:

$ sudo cryptsetup luksAddKey /dev/nvme0n1p6 recovery.key

You will be prompted for an existing passphrase: enter the one you used when first setting up the encrypted device.

You also need to keep a record of that key. I suggest you copy it to the root user’s home directory on the new system for now. Obviously you need to save that somewhere else later so you can use it if you forget your usual passphrase.

$ sudo cp recovery.key /target/root/recovery.key

Now reboot. You should be prompted for your passphrase to unlock the disk and then boot into your shiny new Ubuntu system with LUKS2 encryption and LVM. Remember to save your recovery key somewhere safe.

Summary

The GUI installer offers an option to install Ubuntu with encryption and LVM if the entire disk is used but it does not provide this capability alongside existing installations. However, by using a few simple terminal commands alongside the GUI installer it is possible to overcome this and install Ubuntu on an encrypted partition with LVM independently of any other installations on the disk.

Ubuntu VM Clone – No Network

If you clone a Virtual Machine in VirtualBox it will most likely fail to find its network interface when it boots.

In most cases, all you need is to login on the console and then

# rm /etc/udev/rules.d/70-persistent-net.rules
# reboot

See http://mickvaites.com/2009/06/ubuntu-changing-network-device-id-udevrules-d/ for more details or search for “ubuntu udev rules clone”

Accessing Google Calendar using the php API

Google provides a good UI for their calendars for manual use, and calendars can easily be embedded in other sites, but being able to access them programmatically adds another dimension of usefulness.

Many of Google’s services have API’s in various languages.  These notes relate to using the Google Calendar API v3 with php.

Scenario

I wanted to use a php script to grab data from a Google Calendar, either to write to a file or serve up as part of a web page.

Warning

Firstly, a word of warning.  I wasted some time looking at old documentation.  There are multiple versions of documentation that apparently relate to the same thing but with differing information.  Old documentation has not been updated or removed and there is no indication that it is old.

If you want to use the php API, DO NOT use the instructions at https://developers.google.com/google-apps/calendar/  These point to old code on http://google-api-php-client.googlecode.com/ which is deprecated.

The place to look is https://developers.google.com/api-client-library/ and the code is on github at https://github.com/google/google-api-php-client

[Looking at https://developers.google.com/api-client-library/php/guide/migration, it seems the github version is 1.0.0 of the php library for the Calendar v3 API]

Setup

Follow the instructions at https://developers.google.com/api-client-library/php/start/get_started to create a project in the Google Developer’s Console and install the client library.

Authentication and Authorization

The API key approach does not provide access to private data.  In my application, I do want to access private data so we need the OAuth 2.0 approach.  This is not as complicated as it sounds!

There are three different types of client id’s that can be used:

• Web application client IDs
• Installed application client IDs
• Service Account client IDs

Although the notes warn you of the importance of choosing the right type, they don’t provide guidance. See here or here (but beware of following other links from that page to old documentation)

For my purposes, I want an automated process to access some calendars, so a Service Account is most appropriate.

Create a Service Account

In Google Developer Console select your project, then ‘API’s & Auth’, then ‘Credentials’, then ‘Create New Client ID’ and choose ‘Service Account’.  You will be prompted to save the private key for the id. Make sure you do save it since it can’t be downloaded later (although you can generate new keys).

You will need the ‘Client ID’ and ‘Email Address’ in the next steps.

Sharing the Calendar

The Client ID, Email Address and Private Key File will be used by the php script to authenticate to the calendar service, but it needs to be authorized to access private data.  Just because we can connect to the service doesn’t mean it should give us whatever data we ask for!  We need to share the calendar that we want to access with the Service Account ID we have just created.

Log in to the Calendar using the normal web GUI. Locate the calendar on the left panel under ‘My Calendars’, hover over it to reveal the dropdown arrow, then select ‘Share this Calendar’.  Under the ‘Share with specific people’ section, enter the Email Address from the previous step and check it has enough access for your intended use.  For me, that means ‘See all event details’. Then click ‘Add Person’, then ‘Save’.

The calendar is now shared with the service account, so our php script can access it.

Before closing the calendar, go to the ‘Calendar Details’ link, and look for the ‘Calendar Address’ section.  You need to copy the Calendar ID – looks like an email address: long string of random stuff @ group.calendar.google.com

PHP script

At long last, we’re ready to start some coding.

See: https://developers.google.com/api-client-library/php/guide/aaa_oauth2_service

And the example here: https://github.com/google/google-api-php-client/blob/master/examples/service-account.php

Putting all the pieces together, we need the Client ID, Email Address, Private Key File and Calendar ID to plug into this script.  Hopefully it works…  Note that many errors result in a 404 status and the error message is not very useful.

I didn’t find much documentation for the PHP API, but the code is easy to read at https://github.com/google/google-api-php-client/blob/master/src/Google/Service/Calendar.php

Listing Events

To explore the API in the Developer Console, click on ‘API’s & Auth’, then ‘API’, and select the Calendar API.  See the section on calendar.events.list   Some useful options include:

• singleEvents – convert recurring events to single events
• timeMax, timeMin – for searching by the start time of events
• fields – to control which fields are returned

Sample Code

(Edited based on comments and questions – last update Jan 8, 2015)

#!/usr/bin/php
<?php

require_once "Google/Client.php";
require_once "Google/Service/Calendar.php";

// Service Account info
$client_id = 'replace with Service Account Client Id';
$service_account_name = 'replace with Service Account Email Address';
$key_file_location = 'replace with Service Account Private Key filename';

// Calendar id
$calName = 'replace with Calendar ID';


$client = new Google_Client();
$client->setApplicationName("Calendar test");

$service = new Google_Service_Calendar($client);

$key = file_get_contents($key_file_location);
$cred = new Google_Auth_AssertionCredentials(
 $service_account_name,
 array('https://www.googleapis.com/auth/calendar.readonly'),
 $key
);

$client->setAssertionCredentials($cred);

$cals = $service->calendarList->listCalendarList();
print_r($cals);
//exit;

// Convert recurring events to single events
// Look for events in the next week - now to now+1week
$params = array(
 'singleEvents' => TRUE,
 'timeMin' => (new DateTime())->format(DateTime::RFC3339),
 'timeMax' => (new DateTime())->add(new DateInterval('P1W'))->format(DateTime::RFC3339),
);
$events = $service->events->listEvents($calName, $params);

foreach ($events->getItems() as $event) {
 echo "Summary:  ", $event->getSummary(), "\n";
 echo "Location: ", $event->getLocation(), "\n";
 echo "Start:    ", fmt_gdate($event->getStart()), "\n";
 echo "End:      ", fmt_gdate($event->getEnd()), "\n";
}

function fmt_gdate( $gdate ) {
  if ($val = $gdate->getDateTime()) {
    return (new DateTime($val))->format( 'd/m/Y H:i' );
  } else if ($val = $gdate->getDate()) {
    return (new DateTime($val))->format( 'd/m/Y' ) . ' (all day)';
  }
}

Ubuntu Oracle Java

Ubuntu ships with OpenJDK.  This seems to work well enough for most things, but some apps just don’t work without Oracle’s version of Java.  One such is the Proxmox console program.

If you’re a Proxmox developer, then you could figure out what features of Proxmox don’t work with OpenJDK and fix them.  If, like me, your interest is more pragmatic and you just want to get the Proxmox console working, then switching to Oracle’s Java packages does the job.

Conveniently, the nice chaps at Duinsoft have figured out what you need to do and created a script update-sun-jre to do the work.  I used the repository and all works nicely.  Thanks guys!

Oh, and Proxmox 3 is great – but that’s another post.

LXC containers

If you want to provide some separation between different activities on your Linux server but don’t want the overhead of running full virtualization software, then containers might be of interest.

LXC provides lightweight virtualization sometimes described as “chroot on steroids”.

This Ubuntu page shows how to run lxc on Ubuntu.

Another selection of links is provided here

There are lots of options, but getting started is pretty easy with lxc-create

Since the system was already using LVM, I opted to use LVM storage.  lxc-create does all the work when called with “-B lvm”

I’ve not tried this, but might be useful http://jim.studt.net/depository/index.php/letting-your-lxc-containers-use-consoles

Backups

Backups: tedious and unproductive until you need them, and then you realize why you did – or should have done – them! There are numerous options available for managing backups.  For this post, I’m interested in Linux solutions for servers to backup to an off-site location. I have rsnapshot running locally which provides easy comparison between versions of a file and simple restore.  Clean, simple, efficient and no complex or unusual storage formats. Now I want to get a copy off-site and suitably protected.  The off-site location is semi-trusted: trusted enough that to look after the data reliably but I don’t want them (or anyone who might get access to their system) to be able to browse through it easily. Clearly we need something that works reliably (meaning restores must work), but the main concern is to get something running rather than produce a wonderful survey and comparison of backup products … but still have nothing running! This Ubuntu page on Backing up Your System is a good start with numerous links. My requirements:

• incremental backups
• works remotely 
• efficient on network transfers
• does not use root for login
• encrypted

For what it’s worth, Ubuntu uses Déjà Dup as its backup solution which uses duplicity as the backend. Despite having a more than 10-year active history the product is still marked as beta but it seems to be widely used.

I have chosen to use duplicity with scp as the remote storage method.

Steps:

1. Configure ssh with keys to allow the server running duplicity to connect to the remote site (not covered here, but widely documented)
2. Create a simple script to run duplicity with whatever parameters you want and use the scp option
3. Run the script daily from cron

My rsnapshot is configured to save under /data/rsnaphost/<time>/<server>

I want a remote copy of the daily.0 tree, so my script is basically:

#!/bin/bash
export PASSPHRASE='my long passphrase'
me='my_email_address'
id='id_for_scp'
my_server='local_server_name'
log=/var/log/duplicity/log-`date +%F`
duplicity --full-if-older-than=1M /data/rsnapshot/daily.0/$my_server scp://$backup@$remote_server:/backups/$my_server >> $log
mail $me -s 'Backup log' < $log

Do make sure you keep the passphrase safe – without it, your backups are useless!