Alternative update approach to slackpkg - request for testing/feedback
SlackwareThis Forum is for the discussion of Slackware Linux.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Is it possible to set up separate blacklists for the install and upgrade lists? Or alternatively just apply the blacklist to the install but not the upgrade list?
E.g. If I add the alienbob repo to the default.conf for slackscan I can scan and upgrade my currently install packages from that repo. However, I only have a handful of packages installed from that repo which means that all the remaining packages come up for installation when I generate an install list.
It seems to me the options from here are to:
1. Blacklist all the packages in the alienbob repo I dont use individually so that upgrade would still work on the ones I do have and the others dont show up for installing. Not ideal IMO but it should work.
2. Filter the install list manually with "slackscan -i - | grep -v alienbob" or something along those lines and then use the remaining output directly with upgradepkg --install-new. Simpler than option 1 but skips the slackup use.
3. Ask here if it's possible to blacklist matches in the install list, without affecting the upgrade list. No harm in asking?
Or maybe there's some other option I'm not seeing.
I'm not itching to drop slackpkg just yet, although I can appreciate the concept you are going for with the simplicity of the non-interactive setup. I think these tools could be scripted/automated fairly easily for notifications and updates so thanks for sharing. I'll be playing around with them a bit more.
Is it possible to set up separate blacklists for the install and upgrade lists? Or alternatively just apply the blacklist to the install but not the upgrade list?
A whitelist approach would probably work best in that situation.
e.g. if all I wanted from eric was blender then I could use a whitelist like this:
Or at lesat that would work if upgradepkg supported https:// URLS. I'll look at breaking the command-generating code out of slackup so that it can be used in this manner.
'slackup' doesn't support whitelists at present. Really it was just a demo for using slackscan that turned out to work better than I expected, so I just ran with it. It's never going to be as flexible as creating your own solution around slackscan and greps, but it'll work well for the most simple cases. Perhaps I'll enhance it to do something like the above, but I'm wary of making it too complex. The idea was to keep these tools small and simple. More full featured and complex tools are already available elsewhere.
In hindsight I should have probably given slackscan top billing, and demoted slackup to just being a demo of using it again.
Last edited by GazL; 10-03-2021 at 06:12 AM.
Reason: oops, missed the '-f' on grep.
@GazL
What is the advantage of your scripts over slapt-get?
None.
Different sort of tool entirely. It's kind of like comparing sed with emacs: they both edit files, but that's about it.
As I said in post one, slackscan was designed with the UNIX philosophy in mind; that may appeal to some folk and other simply won't care.
I'm not expecting these tools to usurp slackpkg, slackpkg+, slapt-get or any other update tools. I wrote them for my own use and I'm just making them available as an option for anyone who may see a use for them.
A whitelist approach would probably work best in that situation.
I ended up doing something like that afterwards with a separate script that just calls slackscan and then filters each list separately with whitelists for each. I've only generated lists as dry runs so far but it looks like the output would work for updating if I would feed it into xargs. Little steps so far.
I'm not very good with awk — I usually use sed or perl5 — but this will turn the output of slackscan into curl and upgradepkg commands, much like my slackup script does.
Code:
#!/bin/awk -f
#
# slackscan_update_generator,
# Version 2.4
#
# This awk-script takes a list of Slackware packages on stdin
# and converts them to a stream of pkgtools updatepkg commands.
# It is intended for use with the output generated with
# slackscan's -i and -u options.
#
BEGIN {
FS="/"
url = 0
print "# Package update commands generated by slackscan_update_generator"
print "#"
}
/^https?:\/\/.*\/[^/]+-[^-/]+-[^-/]+-[0-9]+[^-/.]*\.t[^-/.]z$/ {
print "curl -O '"$0"'"
print "curl -O '"$0".asc'"
url = 1
}
/^(.*\/)*[^/]+-[^-/]+-[^-/]+-[0-9]+[^-/.]*\.t[^-/.]z$/ {
if ( url == 1 )
f = $NF
else
f = $0
print "gpg --verify \\"
print " '"f".asc' \\"
print " '"f"' \\"
print "&& upgradepkg --install-new '"f"' "
print ""
url = 0
}
Thanks for that awk script, it cleans up the install and update lists nicely. The remove list is formatted just as package names so that can be processed easily.
Since slackscan just generates lists and doesn't do anything potentially harmful I've been running it as my regular user and filtering it through blacklists/whitelists and that awk script to make an "update.sh" file. root can look over that file after and execute it if it looks in order. I've also been messing about with scripting dbus lately so I hacked together a notifications popup that can show some more details with kdialog. Add's a bunch of unnecessary complication but makes for a nice update notifier when I'm logged in as my regular user in kde plasma I'll attach a screen shot to show my little setup.
Now I just have to verify the update.sh against my slackpkg+ output to make sure its all the same, then I can try out the generated scripts...
Edit: Also gotta update my mirror first so just ignore that package list if anyones looking at it.
I've decided to split the two tools slackscan and slackup into separate repos as judging by the posts made to this thread there's been far more interest in the slackscan back-end than my slackup front-end, so shipping them separately seems like a sensible move.
Well I'm happy to report that after syncing my local mirrors, the first run of slackscan and the slackscan_update_generator scripts were a success!
I opted to blacklist the kernels/modules and did that bit manually along with mkinitrd and elilo. I'm using a whitelist/blacklist combo for the install and upgrade lists and just a blacklist for removals. Seems to work out equivalent to how I have slackpkg+ configured since slackpkg reports the system is up to date as well (it just wants to remove the slackup package ).
I'm not using the slackup script but the post-install script triggering feature sounded interesting. That would be nice for automating the initrd and handling .new files in etc.
The suggestion I have is moving the blacklist/protect list options to slackscan, while keeping the gpg checking, download parsing, and post install script triggering in slackup. Then slackscan could output the filtered lists to slackup for handling, while still leaving the option for additional scripting in between. I think that would be more flexible for custom repos but then I guess I'm just basing this off my use case. I'm already happy with this setup so no need to change the design unless you feel inclined.
Well I'm happy to report that after syncing my local mirrors, the first run of slackscan and the slackscan_update_generator scripts were a success!
Good to hear.
Quote:
Originally Posted by 0XBF
I'm not using the slackup script but the post-install script triggering feature sounded interesting. That would be nice for automating the initrd and handling .new files in etc.
That was exactly what I intended it for.
The post-install script for kernel-modules is best for building an initrd as the output of slackscan is sorted by package-name, so kernel-modules always comes after kernel-generic/huge: mine also copies it and the bzImage into /boot/efi/...
Quote:
Originally Posted by 0XBF
The suggestion I have is moving the blacklist/protect list options to slackscan, while keeping the gpg checking, download parsing, and post install script triggering in slackup. Then slackscan could output the filtered lists to slackup for handling, while still leaving the option for additional scripting in between. I think that would be more flexible for custom repos but then I guess I'm just basing this off my use case. I'm already happy with this setup so no need to change the design unless you feel inclined.
When slackscan was originally created it only worked with local directories not URLS. Adding the support for remote package locations adds a lot of complexities...
With my local mirror I can have this in the conf to add 'xv':
/local/slackware/mirror/slackware64-current/slackware64/
/local/slackware/mirror/slackware64-current/extra/xv
... so there's no need to blacklist anything, but there's no CHECKSUMS.md5 in the xv directory so if you use a remote URL you have to use the one the next level up in extra/ and then you have all the other packages under /extra to worry about in a blacklist. The "include then exclude" approach will handle this situation with appropriate entries, but its a complexity that simply wasn't needed in the original use case of working on local dirs.
My initial thoughts are still that filtering is best done by whatever script uses slackscan, but I'll give some more thought to whether the filtering can be improved without making slackscan too complex or unwieldy to use. I don't want it to turn into emacs!
Thanks for taking an interest, and for your testing and feedback. It's much appreciated.
Does slackscan use the md5 sums from a remote CHECKSUMS.md5 or just file paths?
The reason I ask is the following. I use slackrepo to build SBo packages, it produces a repo, the metadata for this repo is produced by the gen_repos_files.sh script. This script works very slow, so I'd like to get rid of it and slackscan is the only program I know that works with remote repos without metadata. So, my idea is to generate just CHECKSUMS.md5 instead of all metadata. And it would be even better not to compute the sums.
I was wondering if you could post the script to show notifications popup with kdialog?
I am have my own script hat automate's upgrading installing and allows me to make sure everything is updated on my home network. I am just trying to figure out how to get dbus to show a notification.
Quote:
Originally Posted by 0XBF
Thanks for that awk script, it cleans up the install and update lists nicely. The remove list is formatted just as package names so that can be processed easily.
Since slackscan just generates lists and doesn't do anything potentially harmful I've been running it as my regular user and filtering it through blacklists/whitelists and that awk script to make an "update.sh" file. root can look over that file after and execute it if it looks in order. I've also been messing about with scripting dbus lately so I hacked together a notifications popup that can show some more details with kdialog. Add's a bunch of unnecessary complication but makes for a nice update notifier when I'm logged in as my regular user in kde plasma I'll attach a screen shot to show my little setup.
Now I just have to verify the update.sh against my slackpkg+ output to make sure its all the same, then I can try out the generated scripts...
Edit: Also gotta update my mirror first so just ignore that package list if anyones looking at it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.