UPDATE: February 14, 2024
I think I've managed to optimize this process a bit.
Building a repo and generating metadata seems to be problematic with apt-mirror. So I tried a 3rd party application called "aptly". Important to note, support for aptly is... limited. And the only evidence it's alive are the occasional commits I see on the aptly-dev github. (Only one person has contributed this year so far, "neolynx". Most others have not touched the project in a year or more, according to their commit page.)
As such, if you want to use aptly to help you create a local usb repository that pretty much any apt-based linux system can use, there are a two minor hurdles to overcome. Aptly, as of the time of this post I used aptly version 1.3 (the version available in Ubuntu official repos) to complete my tasks. Github shows aptly version 1.5 as the latest.
- Version 1.3 does not have the ability to read and create metadata for z-standard compressed files.
- Version 1.3 does not have the ability to properly read gpg2 generated keys.
- Version 1.3 does not have the ability to publish a snapshot with multiple repo components IDENTIFIED (although access to the desired .deb files is STILL there).
- Version 1.3 does not have the ability to properly ask for your gpg passphrase when publishing your snapshots and repos. (The workaround is to temporarily remove your passphrase, publish your repo/snapshot, and then reapply your passphrase before making your repo/snapshot available for Users.
- I have NOT tested version 1.5 for these failures.
Let's get started.
Code:
sudo apt update
sudo apt install aptly
Create your mirror.
Code:
aptly mirror create <name> <archive url> <distribution> [<component1> ...]
Example for 64-bit Ubuntu 20.04 assuming you want all components.
Code:
aptly -architectures="amd64" mirror create focal-main http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
If you, for example, want the full catalog of for focal (focal-updates, focal-security, etc.) then you will swap "focal" in the command above for the desired one. And pick and choose your preferred components. If you're feeling yourself, so to speak, you can open multiple terminals and run each mirror creation concurrently. Please note, this particular command will download over 400GB of data. Additional steps that create and publish snapshots and the .iso creation, will practically DOUBLE that size. These are big boy numbers. Make sure your internet connection and working hard drive are up for handling these tasks.
Then we tell the mirrors to update, and this will be the long part.
Code:
aptly mirror update your-mirror-name
Ubuntu example.
Code:
aptly mirror update focal-main
Once your mirrors are complete you will verify them and then create snapshots.
Verify.
Code:
aptly mirror show <name>
My 20.04 example. You should get something similar to this, that confirms the components you downloaded, and the architectures (assuming you specified any).
Code:
$aptly mirror show focal-main
Name: focal-main
Archive Root URL: http://archive.ubuntu.com/ubuntu/
Distribution: focal
Components: main, restricted, universe, multiverse
Architectures: amd64
Download Sources: no
Download .udebs: no
Last update: 2024-02-13 16:48:25 UTC
Number of packages: 60252
Information from release file:
Acquire-By-Hash: yes
Architectures: amd64 arm64 armhf i386 ppc64el riscv64 s390x
Codename: focal
Components: main restricted universe multiverse
Date: Thu, 23 Apr 2020 17:33:17 UTC
Description: Ubuntu Focal 20.04
Label: Ubuntu
Origin: Ubuntu
Suite: focal
Version: 20.04
Then create immutable snapshot.
Code:
aptly snapshot create <name> from mirror <mirror-name>
Example for the 20.04 repo we just downloaded.
Code:
aptly snapshot create focal-main-snapshot from mirror focal-main
At this point you will need to generate a passwordless gpg1 key, and save it to your preferred keychain. (If you are NOT publishing this repository online, or for other people to have access to it then the password on the key is not strictly necessary. This key can be specifically created, and even given it's own keychain specifically for aptly offline-publishing functionality.)
Note "gpg1" and gpg version on output. This is *necessary* for aptly version 1.3.
Code:
$ gpg1 --gen-key
gpg (GnuPG) 1.4.23; Copyright (C) 2015 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Please select what kind of key you want:
(1) RSA and RSA (default)
(2) DSA and Elgamal
(3) DSA (sign only)
(4) RSA (sign only)
Your selection?
If you know what you're doing, select the one you want. If you don't know what you're doing, press 1 and hit enter.
Then enter 4096 because why the fuck not.
Code:
Your selection? 1
RSA keys may be between 1024 and 4096 bits long.
What keysize do you want? (2048)
Now select the lifetime of the key. For my example, I am selecting 1 day since this key is only being produced for the purposes of this post.
Pay attention to what you are doing here now. You will input the NUMBER for how many you want, THEN the letter to specify how many of WHAT you want. Numbers without letters are automatically assumed to be days!!! For example, "3" is 3 DAYS. "3w" is 3 WEEKS. "3m" is 3 MONTHS. "3y" is 3 YEARS.
Code:
Requested keysize is 4096 bits
Please specify how long the key should be valid.
0 = key does not expire
<n> = key expires in n days
<n>w = key expires in n weeks
<n>m = key expires in n months
<n>y = key expires in n years
Key is valid for? (0)
You will verify Yes, by typing y and hitting Enter. Because it defaults to No.
Next you will provide it necessary arbitrary information to facilitate function with other people. Put whatever you want in the requested fields if this key will never be used or requested by others.
(If it will be used by other people and you will be using this in some sort of professional capacity, then I do recommend providing real identifier information so that you can be reached.) Whatever you put in the comments, ensure that it's something to remind you what the key is actually for.
Assuming you followed the prompts you will end up here:
Code:
You selected this USER-ID:
"Mario Lawrence (Lawrence Industries, Inc. Focal Repository Publishing Key) <mario.lawrence@email.com>"
Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O
You need a Passphrase to protect your secret key.
Enter passphrase:
At this point simply press Enter. And do NOT enter a passphrase.
GPG will bitch and moan (a little) about no password being a bad idea. Ignore this and just drag a window around real fast to generate some entropy. It should output your key information in short order. We're done with GPG.
Now we can get back to aptly.
You will now publish your snapshot.
Code:
aptly publish snapshot <name> [[<endpoint:>]<prefix>]
Ubuntu example. NOTE. For aptly version 1.3 the following DOES NOT work.
Code:
aptly publish snapshot -architectures="amd64" -distribution="focal" -component="main,restricted,universe,multiverse" focal-main-snapshot
You will not be able to publish all your components properly. I'm uncertain why this is the case. I'm fairly certain I followed aptly's documentation correctly. So, if you can overlook my hubris, I think this is a bug for version 1.3. But you will see however the applications under the additional components are infact still available to apt!
So you will publish with the command as follows, NOT specifying a component.
Code:
aptly publish snapshot -architectures="amd64" -distribution="focal" focal-main-snapshot
This process will take similarly long to creating the mirror.
When it's complete you will navigate to the default aptly folder /home/user/.aptly/public/ and here you will find the structure that apt will invoke when used.
Now we will make this exactly folder structure an .iso file! If you do not have sufficient space on your hard drive for an additional 400+GB of data, then I recommend connecting the usb drive that you intend to turn into the repo. Depending on your distribution, when it's connected it will automount (on ubuntu-based systems this is typically /media/user/).
You will now perform the final step of the creation. Now creating an iso file that respects the folder structure and filenames thankfully isn't too complicated, but required me to look through the documentation very carefully.
Here's the required command:
Code:
mkisofs -R -relaxed-filenames -joliet-long -iso-level 3 -l -o /path/to/your/usb/drive/desired-iso-name.iso /home/user/.aptly/public/
This command should ensure that your exact filename structure is maintained, and it will provide you with continuous progress. See "man mkisofs" for more information.
That's it. You've successfully created an offline usb ubuntu repo!
If you want to test the repo, be sure to backup and modify your /etc/apt/sources.list file. Comment out your other sources and add a line that says...
Code:
deb [arch=amd64 trusted=yes] file:/path/to/your/usb/drive focal main
Save your changes and run apt.
You will likely see it complain about the GPG key. This is fine to ignore (unless you plan to publish the repo to others), in which case you can simply apt --add-key. (Which I think going into, is beyond the scope of my intent.)
Thanks guys. I hope this helps someone in the future!
