Pinebook Pro boot from NVME - drop requirement for booting from uSD card
 

Following the Slackware Pinebook Pro installation instructions it has a section "Boot from NVME - drop requirement for booting from uSD card" noting that this is not yet fully worked out.

After a lot of trial and error I now have this working. First I wrote the Slackware bootloader to the SPI flash storage as described in my previous post. This allowed the SD card with the Slackware installer to boot. I formatted the NVME drive as a swap partition (/dev/nvme0n1p1) and a single data partition with ext4 format, /dev/nvme0n1p2 (maybe a separate /boot partition would work, I didn't try that). After completing the installation to the NVME storage the installer places the kernel and initrd on the SD card. The EMMC card was present in the computer but switched to disable it.

I copied the files from the boot partition on the SD card over to the /boot directory on the ext4 root partition. Editing /boot/extlinux/extlinux.conf on the NVME drive was where all the trial and error was then needed. I found the following entry worked and boots up from the NVME without the SD card present.

Changing the PROMPT value to 90 gave enough time to see the menu and make a choice - it otherwise moves too quickly when running from the NVME drive, much faster then when running from the SD card.

Using root=LABEL=SLKroot in the APPEND line works too as the installer already labels up the partition - that would be more easily generalisable than the /dev/nvme0n1p2 version. Including console=tty1 ensured some errors were written to the screen to help understand what was happening before I got the APPEND options right. I'm not sure the combination above is optimal, but it works! Including rootfstype was crucial to the kernel being able to reach the root partition as it boots up.

I appreciate it! I've revised the installation guide to include a reference to this post. You can adhere to the standard Boot Loader installation instructions and implement this alteration during the post-installation phase by choosing 'Shell' from the installer's exit menu.

AND with the system up and running, what happens when you insert sd card?

Is it accessible to mount? like a flash drive, floppy drive, whatever external mountable device.

I have a mixed opinion about this feature.

1. If for some reason the internal /boot partition or / partition do not allow the system to boot, you still need a SD card to rescue the system. I find having the installer available on the SD card convenient for this reason. It is more convenient while running -current.

2. With that said, my NVMe drive boots the system extremely fast when compared to the SD card, as expected. I plan to boot from the NVMe drive once we have a 15.1 release of Aarch64. I also encrypted my NVMe drive with LUKS + LVM as described here: https://docs.slackware.com/slackware...ckware_aarch64

The SD card becomes a "floppy" or "flash" drive and can be mounted for whatever use case. I plan to store my LUKS encryption key on my SD card. If you wanted to do so, you could put another OS on the SD Card and dual boot, sort of. There are many other options. You may wish to leave the boot order within u-boot alone so that it looks for the SD card first and contains a rescue image.

I have also found that the built in SD card reader continues to work normally and an SD card can be mounted there.

Yes, the case of the NVME drive holding the system but not booting can be managed if you keep the bootable SD card available. I used this method multiple times while I worked out the contents of extlinux.conf on the NVME drive.

One thing I found was you can get the case that the NVME drive takes priority over the internal SD card reader. If the internal drive doesn't boot properly that can leave you in a mess. There's probably U-Boot commands to handle this. But I also found that putting the Slackware installer SD card in an external card reader plugged into the USB3 port (left side of PinebookPro) can overcome this. The external USB card reader was found and enabled booting ahead of the NVME and internal SD card reader. This was using the standard Slackware U-boot install copied to the SPI flash where I expect this order was configured, but I didn't look further at that.

It's been little while since I installed, pretty sure usb3 port/card reader is what I did along with installing towboot.

I'm thinking I'll lose being able to boot if I flash to new slackware u-boot.

Thank you for replying, you both say it can be done, I'd like to see it done.

Like what does messages say after sd card is inserted, what /dev/??? sd card is identified as.

x86_64 slackware, messages shows the following when plugging in sd card in card reader
plugging sd card in sd card slot on pbp messages says
and I don't see any /dev/ device

The naming difference arises from how the system assigns device names based on the interface they are connected through (USB, PCI, SATA, etc.).
USB-connected devices are identified as generic 'mass storage devices' irrespective of the physical media they're hosting.

As it's a direct MMC device, not connected through a USB interface it'll be /dev/mmcblk1 or /dev/mmcblk0

There's more to this than it seemed. Some cards are not readable in the internal SD slot but work fine in an external reader. Another one works perfectly in the internal reader.

The one that now does not read is the one I installed Slackware from - so I know this did work in the internal reader. There's some software issue here now the machine has booted off the installed system.

Details are below - if anyone has some ideas what I could try to help explore this please let me know.

Some searching which is aligned with the dmesg output below is that there's a speed set for the SD card. Working cards give messages such as "dwmmc_rockchip fe320000.mmc: Successfully tuned phase to 220". Cards that won't read don't have this message. The posts linked below suggest this might be about a speed set in the device tree - is the tree used during installation different to the one booting after install? These posts seem to point towards this as an issue - probably nothing to do with the original subject of this post!

eMMC doesn't work after kernel start - Does this apply to the internal SD too on the mmcblk1 device?

Kernel doesn't work with eMMC after start

Here's some output for cards that do/don't work, in the internal and an external SD card reader.

(1) Slackware installer SD card in internal reader - not readable. dmesg output:

Chcking using lsblk

Card details via fdisk /dev/mmcblk1


(2) Slackware installer SD card in external USB reader. Perfectly readable, dmesg gives

lsblk gives:


fdisk /dev/sdc output:


(3) dmesg for an alternative SD card in the internal reader which reads perfectly:

lsblk output:

Card details from fdisk /dev/mmcblk1

After lots of reading and trying things out I think I've worked out what's happening and found a work-around.

SD cards have a range of operating frequencies depending on their capabilities.

If you boot from the SD the Rockchip hardware tunes to the capability of the card in place and it all works. If no SD card is present (as when booting using the NVME drive) the current behaviour is to set operation of the SD card socket for the highest frequency available in the host hardware - and this then seems to be locked in once the machine is running. The hardware should recognise a card plugged in later and if necessary drop frequency to match, but it does not. Hence some cards work when used as the boot drive, but fail if plugged in later following NVME boot. Other cards that can handle the maximum frequency of the socket work fine both ways. Some clues about all this are here.

This problem can be avoided by setting a lower maximum operating frequency for the SD socket. This can be done my modifying the device tree using this (as root):

That reduces the SD operating frequency to 12.5MHz, down from its standard value of 50MHz. After a reboot the SD card that previously failed to mount then works perfectly. Probably there's a performance penalty for data transfer to the SD card, but I didn't notice anything major. It might be possible to go higher (e.g. 25MHz) without causing the card to fail - I've not tried that.

The real solution would be for the RK3399 to adapt frequency properly, but the above is a work-around which allows more (all?) SD cards to mount even if they are not particularly high spec cards. Rather than modifying the device tree it should also be possible to change this frequency from uboot (extlinux.conf file) using something like

But I've not worked that out yet.

@drmozes

I was not posting about what 'mass storage devices', I was posting what messages acknowledges when something system related occurs.

I was also indicating there was no new /dev/ device of any kind after sd card was inserted.

I will try to follow up difdif investigations soon.

I'm curious as to how you guys are connecting M.2 NVME drives.

It's pretty obvious how one would mount one drive with a simple PCIe to NVME sled or dock. In my case my Pine64 is used for NAS so I would need multiple M.2 drives and slots. I'm using an actual PSU instead of the little wall wart supply but have doubts about the current ability to experience any real bandwidth gains should I go to a 4 x M.2 sled or dock. Any comments?

Pinebook Pro boot from NVME - drop requirement for booting from uSD card
 

The pinebook pro has an addon that enables a NVMe drive via a M.2 slot. A small ribbon cable and a mounting board interface the drive to the main board.

The Rockpro64 PCIe lane most likely doesn't have enough bandwidth to use more that one or two NVMe drives. I may be wrong about that.

Most users drop in a SATA or Network card in the PCIe slot of the Rockpro64. Pine64 does sell a NVMe card, but it only has input for one drive. The pine64 wiki hardware compatibility page shows only two tested nvme drives.