LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 10-02-2021, 03:23 PM   #16
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033

Ok, I'll give that some thought. It might have to be conditional on a '-v' option, otherwise it could interfere with redirection, piping of the lists.
 
Old 10-02-2021, 07:34 PM   #17
0XBF
Member
 
Registered: Nov 2018
Distribution: Slackware
Posts: 781

Rep: Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895
Is it possible to set up separate blacklists for the install and upgrade lists? Or alternatively just apply the blacklist to the install but not the upgrade list?

E.g. If I add the alienbob repo to the default.conf for slackscan I can scan and upgrade my currently install packages from that repo. However, I only have a handful of packages installed from that repo which means that all the remaining packages come up for installation when I generate an install list.

It seems to me the options from here are to:
1. Blacklist all the packages in the alienbob repo I dont use individually so that upgrade would still work on the ones I do have and the others dont show up for installing. Not ideal IMO but it should work.
2. Filter the install list manually with "slackscan -i - | grep -v alienbob" or something along those lines and then use the remaining output directly with upgradepkg --install-new. Simpler than option 1 but skips the slackup use.
3. Ask here if it's possible to blacklist matches in the install list, without affecting the upgrade list. No harm in asking?

Or maybe there's some other option I'm not seeing.

I'm not itching to drop slackpkg just yet, although I can appreciate the concept you are going for with the simplicity of the non-interactive setup. I think these tools could be scripted/automated fairly easily for notifications and updates so thanks for sharing. I'll be playing around with them a bit more.
 
1 members found this post helpful.
Old 10-03-2021, 12:44 AM   #18
MadMaverick9
Member
 
Registered: Aug 2010
Posts: 353
Blog Entries: 4

Rep: Reputation: Disabled
@GazL

What is the advantage of your scripts over slapt-get?

One thing I like about slapt-get is the ability to do:
Code:
slapt-get --no-upgrade --install --simulate kernel-generic kernel-huge kernel-modules kernel-source
Install a new kernel without deleting the previous one.
 
Old 10-03-2021, 05:15 AM   #19
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
Quote:
Originally Posted by 0XBF View Post
Is it possible to set up separate blacklists for the install and upgrade lists? Or alternatively just apply the blacklist to the install but not the upgrade list?
A whitelist approach would probably work best in that situation.
e.g. if all I wanted from eric was blender then I could use a whitelist like this:
Code:
slackware64-current/slackware64/
alien/slackbuilds/blender
And then use slackscan something like this:
Code:
slackscan -i in.list -u up.list -r rm.list
grep -f in.whitelist in.list | grep -v -f in.blacklist | xargs -r -- upgradepkg --install-new 
grep -v -f up.blacklist up.list | xargs -r -- upgradepkg
grep -v -f rm.blacklist rm.list | xargs -r -- removepkg
Or at lesat that would work if upgradepkg supported https:// URLS. I'll look at breaking the command-generating code out of slackup so that it can be used in this manner.


'slackup' doesn't support whitelists at present. Really it was just a demo for using slackscan that turned out to work better than I expected, so I just ran with it. It's never going to be as flexible as creating your own solution around slackscan and greps, but it'll work well for the most simple cases. Perhaps I'll enhance it to do something like the above, but I'm wary of making it too complex. The idea was to keep these tools small and simple. More full featured and complex tools are already available elsewhere.

In hindsight I should have probably given slackscan top billing, and demoted slackup to just being a demo of using it again.

Last edited by GazL; 10-03-2021 at 06:12 AM. Reason: oops, missed the '-f' on grep.
 
1 members found this post helpful.
Old 10-03-2021, 05:23 AM   #20
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
Quote:
Originally Posted by MadMaverick9 View Post
@GazL
What is the advantage of your scripts over slapt-get?
None.

Different sort of tool entirely. It's kind of like comparing sed with emacs: they both edit files, but that's about it.

As I said in post one, slackscan was designed with the UNIX philosophy in mind; that may appeal to some folk and other simply won't care.

I'm not expecting these tools to usurp slackpkg, slackpkg+, slapt-get or any other update tools. I wrote them for my own use and I'm just making them available as an option for anyone who may see a use for them.
 
7 members found this post helpful.
Old 10-03-2021, 12:04 PM   #21
0XBF
Member
 
Registered: Nov 2018
Distribution: Slackware
Posts: 781

Rep: Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895
Quote:
Originally Posted by GazL View Post
A whitelist approach would probably work best in that situation.
I ended up doing something like that afterwards with a separate script that just calls slackscan and then filters each list separately with whitelists for each. I've only generated lists as dry runs so far but it looks like the output would work for updating if I would feed it into xargs. Little steps so far.
 
1 members found this post helpful.
Old 10-03-2021, 12:22 PM   #22
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
I'm not very good with awk — I usually use sed or perl5 — but this will turn the output of slackscan into curl and upgradepkg commands, much like my slackup script does.

Code:
#!/bin/awk -f
#
#  slackscan_update_generator,
#  Version 2.4
#
#  This awk-script takes a list of Slackware packages on stdin
#  and converts them to a stream of pkgtools updatepkg commands.
#  It is intended for use with the output generated with
#  slackscan's -i and -u options.
#

BEGIN {
    FS="/"
    url = 0

    print "# Package update commands generated by slackscan_update_generator"
    print "#"
}

/^https?:\/\/.*\/[^/]+-[^-/]+-[^-/]+-[0-9]+[^-/.]*\.t[^-/.]z$/ {
    print "curl -O '"$0"'"
    print "curl -O '"$0".asc'"
    url = 1
}

/^(.*\/)*[^/]+-[^-/]+-[^-/]+-[0-9]+[^-/.]*\.t[^-/.]z$/ {
    if ( url == 1 )
        f = $NF
    else
        f = $0

    print "gpg --verify \\"
    print "  '"f".asc' \\"
    print "  '"f"' \\"
    print "&& upgradepkg --install-new '"f"' "
    print ""

    url = 0
}
so you can do something like
Code:
slackscan -i - | grep -f includes.list | grep -vf excludes.list | slackscan_update_generator | sh
should work equally well for -i and -u. xargs can handle -r without any outside help.

Last edited by GazL; 10-03-2021 at 12:23 PM.
 
1 members found this post helpful.
Old 10-03-2021, 02:47 PM   #23
0XBF
Member
 
Registered: Nov 2018
Distribution: Slackware
Posts: 781

Rep: Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895
Thanks for that awk script, it cleans up the install and update lists nicely. The remove list is formatted just as package names so that can be processed easily.

Since slackscan just generates lists and doesn't do anything potentially harmful I've been running it as my regular user and filtering it through blacklists/whitelists and that awk script to make an "update.sh" file. root can look over that file after and execute it if it looks in order. I've also been messing about with scripting dbus lately so I hacked together a notifications popup that can show some more details with kdialog. Add's a bunch of unnecessary complication but makes for a nice update notifier when I'm logged in as my regular user in kde plasma I'll attach a screen shot to show my little setup.

Now I just have to verify the update.sh against my slackpkg+ output to make sure its all the same, then I can try out the generated scripts...

Edit: Also gotta update my mirror first so just ignore that package list if anyones looking at it.
Attached Thumbnails
Click image for larger version

Name:	Screenshot_20211003_143910.jpg
Views:	66
Size:	249.9 KB
ID:	37364   Click image for larger version

Name:	Screenshot_20211003_143935.jpg
Views:	66
Size:	220.8 KB
ID:	37365  

Last edited by 0XBF; 10-03-2021 at 02:50 PM.
 
1 members found this post helpful.
Old 10-03-2021, 03:40 PM   #24
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
Excellent. Looks like you made good use of it.

I've decided to split the two tools slackscan and slackup into separate repos as judging by the posts made to this thread there's been far more interest in the slackscan back-end than my slackup front-end, so shipping them separately seems like a sensible move.

When ready they'll both be on my github.
 
Old 10-03-2021, 10:24 PM   #25
0XBF
Member
 
Registered: Nov 2018
Distribution: Slackware
Posts: 781

Rep: Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895Reputation: 895
Thumbs up

Well I'm happy to report that after syncing my local mirrors, the first run of slackscan and the slackscan_update_generator scripts were a success!

I opted to blacklist the kernels/modules and did that bit manually along with mkinitrd and elilo. I'm using a whitelist/blacklist combo for the install and upgrade lists and just a blacklist for removals. Seems to work out equivalent to how I have slackpkg+ configured since slackpkg reports the system is up to date as well (it just wants to remove the slackup package ).

I'm not using the slackup script but the post-install script triggering feature sounded interesting. That would be nice for automating the initrd and handling .new files in etc.

The suggestion I have is moving the blacklist/protect list options to slackscan, while keeping the gpg checking, download parsing, and post install script triggering in slackup. Then slackscan could output the filtered lists to slackup for handling, while still leaving the option for additional scripting in between. I think that would be more flexible for custom repos but then I guess I'm just basing this off my use case. I'm already happy with this setup so no need to change the design unless you feel inclined.

Thanks again for the useful scripts.
 
1 members found this post helpful.
Old 10-04-2021, 03:48 AM   #26
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
Quote:
Originally Posted by 0XBF View Post
Well I'm happy to report that after syncing my local mirrors, the first run of slackscan and the slackscan_update_generator scripts were a success!
Good to hear.

Quote:
Originally Posted by 0XBF View Post
I'm not using the slackup script but the post-install script triggering feature sounded interesting. That would be nice for automating the initrd and handling .new files in etc.
That was exactly what I intended it for.

The post-install script for kernel-modules is best for building an initrd as the output of slackscan is sorted by package-name, so kernel-modules always comes after kernel-generic/huge: mine also copies it and the bzImage into /boot/efi/...

Quote:
Originally Posted by 0XBF View Post
The suggestion I have is moving the blacklist/protect list options to slackscan, while keeping the gpg checking, download parsing, and post install script triggering in slackup. Then slackscan could output the filtered lists to slackup for handling, while still leaving the option for additional scripting in between. I think that would be more flexible for custom repos but then I guess I'm just basing this off my use case. I'm already happy with this setup so no need to change the design unless you feel inclined.
When slackscan was originally created it only worked with local directories not URLS. Adding the support for remote package locations adds a lot of complexities...

With my local mirror I can have this in the conf to add 'xv':
/local/slackware/mirror/slackware64-current/slackware64/
/local/slackware/mirror/slackware64-current/extra/xv

... so there's no need to blacklist anything, but there's no CHECKSUMS.md5 in the xv directory so if you use a remote URL you have to use the one the next level up in extra/ and then you have all the other packages under /extra to worry about in a blacklist. The "include then exclude" approach will handle this situation with appropriate entries, but its a complexity that simply wasn't needed in the original use case of working on local dirs.

My initial thoughts are still that filtering is best done by whatever script uses slackscan, but I'll give some more thought to whether the filtering can be improved without making slackscan too complex or unwieldy to use. I don't want it to turn into emacs!


Thanks for taking an interest, and for your testing and feedback. It's much appreciated.
 
Old 01-18-2022, 07:18 AM   #27
slalik
Member
 
Registered: Nov 2014
Location: Moscow
Distribution: Slackware
Posts: 233

Rep: Reputation: 203Reputation: 203Reputation: 203
Does slackscan use the md5 sums from a remote CHECKSUMS.md5 or just file paths?

The reason I ask is the following. I use slackrepo to build SBo packages, it produces a repo, the metadata for this repo is produced by the gen_repos_files.sh script. This script works very slow, so I'd like to get rid of it and slackscan is the only program I know that works with remote repos without metadata. So, my idea is to generate just CHECKSUMS.md5 instead of all metadata. And it would be even better not to compute the sums.
 
Old 01-18-2022, 09:51 AM   #28
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,915

Original Poster
Rep: Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033Reputation: 5033
It doesn't use the hash values, but it will check for valid format, so it'll expect 32 hex characters two spaces and then a relative path.

This ought to work as a cheap way of generating the file.
Code:
find /wherever -name "*.t?z" -type f -printf "00000000000000000000000000000000  %P\n"

Last edited by GazL; 01-18-2022 at 10:00 AM.
 
1 members found this post helpful.
Old 01-18-2022, 10:04 AM   #29
slalik
Member
 
Registered: Nov 2014
Location: Moscow
Distribution: Slackware
Posts: 233

Rep: Reputation: 203Reputation: 203Reputation: 203
Quote:
Originally Posted by GazL View Post
It doesn't use the hash values, but it will check for valid format, so it'll expect 32 hex characters two spaces and then a relative path.

This ought to work as a cheap way of generating the file.
Code:
find /wherever -name "*.t?z" -type f -printf "00000000000000000000000000000000  %P\n"
Great! Thank you very much!
 
1 members found this post helpful.
Old 01-22-2022, 07:19 PM   #30
reddog83
Member
 
Registered: Apr 2018
Distribution: Slackware 15.0/Current
Posts: 461

Rep: Reputation: 237Reputation: 237Reputation: 237
I was wondering if you could post the script to show notifications popup with kdialog?
I am have my own script hat automate's upgrading installing and allows me to make sure everything is updated on my home network. I am just trying to figure out how to get dbus to show a notification.



Quote:
Originally Posted by 0XBF View Post
Thanks for that awk script, it cleans up the install and update lists nicely. The remove list is formatted just as package names so that can be processed easily.

Since slackscan just generates lists and doesn't do anything potentially harmful I've been running it as my regular user and filtering it through blacklists/whitelists and that awk script to make an "update.sh" file. root can look over that file after and execute it if it looks in order. I've also been messing about with scripting dbus lately so I hacked together a notifications popup that can show some more details with kdialog. Add's a bunch of unnecessary complication but makes for a nice update notifier when I'm logged in as my regular user in kde plasma I'll attach a screen shot to show my little setup.

Now I just have to verify the update.sh against my slackpkg+ output to make sure its all the same, then I can try out the generated scripts...

Edit: Also gotta update my mirror first so just ignore that package list if anyones looking at it.
 
2 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] I have run slackpkg update gpg instead of slackpkg update amikoyan Slackware 13 08-15-2021 02:12 AM
SARPI on Pi3 - Ran slackpkg update & slackpkg upgrade-all and now won't boot, can't find init petejc Slackware - ARM 11 03-25-2020 04:30 AM
[SOLVED] Slackpkg upgrade-all returns 'no packages to upgrade' after slackpkg-update has downloaded files san2ban Slackware 8 11-01-2019 05:44 AM
[SOLVED] [ENCHANCEMENT] slackpkg+: do not show the notices "pkglist is older than 24h..." and "remember to re-run 'slackpkg update''..."... yars Slackware 1 01-09-2016 09:56 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 02:47 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration