slackpkg vs. third-party package repository

phenixia2003 · 03-15-2013, 12:01 PM

Hello,

Quote:

Originally Posted by Alien Bob

Of course, the slackpkg functionality that still only works with Slackware repositories is slackpkg's "install-new"

The code below manages the installation of any compat32 package (from multilib repository) which is not currently installed. In other words, this allows the installation of newly added package(s) to the layer-32, or the entire layer-32.

Code:

if [ "$CMD" == "install-new" ] ; then
  
  ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
  
  grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

  NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
  
  if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
    LIST=""
    
    for pkg in $NEWCOMPAT32PKGS ; do
      LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
    done
  fi
fi

Hope this help.

--
SeB

zerouno · 03-19-2013, 09:29 AM

Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.

If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

phenixia2003 · 03-20-2013, 06:13 AM

Hello,

Quote:

Originally Posted by zerouno

Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.

If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

Well, I finally updated givepriority() so that, it now supports enhanced priority rules (priority given to package(s) from a given repository). This kind of priority uses the following syntax:

Code:

repository_name:pattern

For instance, to give priority to package "openjdk" from slacky repository, you will use the rule "slackpkgplus_slacky:openjdk".

Code:

  # Found packages in repository. 
  # This function selects the package from the higher priority
  # repository directories.
  #
  # This Modified version supports enhanced priority rule (priority 
  # given to package(s) from a given repository). This kind of priority 
  # uses the following syntax :
  #
  #   repository_name:pattern
  # 
  #
function givepriority {
        local DIR
        local ARGUMENT=$1
	local PKGDATA
	local CPRIORITY
	local DIR
	local PKG

	unset NAME
        unset FULLNAME
	unset PKGDATA
	
        for CPRIORITY in ${PRIORITY[@]} ; do
		[ "$PKGDATA" ] && break
		
		if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
		  DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
		  PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
		 
		  if echo "$ARGUMENT" | grep -q "$PAT" ; then
		    PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
		  fi
		else	
		  PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
		fi
		  
		if [ "$PKGDATA" ]; then
		  NAME=${PKGDATA[1]}
		  FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
		fi
        done
}

So, with this change, the code in my previous post for the extension that I suggested (ie slackpkg install [repository-name:]package-name) must be changed as below :

Code:

if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

	NEWINPUTLIST=""

	for pref in $INPUTLIST ; do
		if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
			repository=$(echo "$pref" | cut -f1 -d":")
			package=$(echo "$pref" | cut -f2- -d":")

			PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
		else
			package=$pref
		fi
		
		NEWINPUTLIST="$NEWINPUTLIST $package"
	done

	INPUTLIST=$NEWINPUTLIST	
fi

Now, about the upgrade/upgrade-all problem. Since givepriority() now supports enhanced priorities, I suggest to allow the user to defines prioritary packages in a variable declared into slackpkgplus.conf. These priorities would be merged with the variable PRIORITY at slackpkg+ init time.

For instance if the user want apache-ant from slacky, and openjdk from alienbob, it will defines this inside slackpkgplus.conf :

Code:

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )

The code to merge these priorities with those defined by PRIORITY will be as below :

Code:

  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
    
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done

With this, when the user will issue the commands :

Code:

$ slackpkg install apache-ant
$ slackpkg install openjdk

slackpkg+ will install the apache-ant from slacky and openjdk from alienbob.

And, when the user will issue the commands :

Code:

$ slackpkg upgrade apache-ant
$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

slackpkg+ will get the latest apache-ant from slacky repository, and the latest openjdk from alienbob repository.

Note, the user can always install/upgrade another version using the extension I suggest. For instance, the command :

Code:

$ slackpkg install slacky:openjdk
$ slackpkg upgrade slacky:openjdk

Will imply install/upgrade of openjdk using the version from slacky repository.

However, since PKGS_PRIORITY includes "alienbob:openjdk", the commands :

Code:

$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

Will imply upgrade of openjdk using the version from alienbob repository.

Well, this starts to be a bit complex, so, to summarize you will find the full code (and a version of slackpkgplus.conf that includes the variable PKGS_PRIORITY) below :

Code:

declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
  
    # -- merge priorities from PKGS_PRIORITY with PRIORITY ...
  
  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
    
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done
  
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
    
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
	egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
	echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
	rm $2-tmp
      done
    fi

  }

    # Found packages in repository. 
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority 
    # given to package(s) from a given repository). This kind of priority 
    # uses the following syntax :
    #
    #   repository_name:pattern
    # 
    #
  function givepriority {
	  local DIR
	  local ARGUMENT=$1
	  local PKGDATA
	  local CPRIORITY
	  local DIR
	  local PKG

	  unset NAME
	  unset FULLNAME
	  unset PKGDATA
	  
	  for CPRIORITY in ${PRIORITY[@]} ; do
		  [ "$PKGDATA" ] && break
		  
		  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
		    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
		    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
		  
		    if echo "$ARGUMENT" | grep -q "$PAT" ; then
		      PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
		    fi
		  else	
		    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
		  fi
		    
		  if [ "$PKGDATA" ]; then
		    NAME=${PKGDATA[1]}
		    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
		  fi
	  done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

	  NEWINPUTLIST=""

	  for pref in $INPUTLIST ; do
		  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
			  repository=$(echo "$pref" | cut -f1 -d":")
			  package=$(echo "$pref" | cut -f2- -d":")

			  PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
		  else
			  package=$pref
		  fi
		  
		  NEWINPUTLIST="$NEWINPUTLIST $package"
	  done

	  INPUTLIST=$NEWINPUTLIST
	  
#    echo "PRIORITY  set to ${PRIORITY[*]}"
#    read userkey
  fi

  if [ "$CMD" == "install-new" ] ; then 
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
    
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
    
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
      
      for pkg in $NEWCOMPAT32PKGS ; do
	LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
  
fi

Here is the slackpkgplus.conf I used :

Code:

SLACKPKGPLUS=on

#MIRRORPLUS['slacky']=http://darkstar.ist.utl.pt/slackware/addon/slacky/slackware64-14.0/
MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )

Hope this help.

--
SeB

phenixia2003 · 03-22-2013, 11:01 AM

Hello,

I slightly modified the function givepriority() so that it is now possible to use regexp in enhanced priority rules. With that, you can give priority for an entire repository through the variable PKGS_PRIORITY. For instance, to give priority to the multilib repository, you simply have to add this in slackpkgplus.conf :

Code:

PKGS_PRIORITY=( multilib:.* )

I also changed the code that merges the content of the variables PKGS_PRIORITY and PRIORITY. This was needed to preserve the priority orders given in PKGS_PRIORITY. Indeed, my previous code was wrong and the priorities were reversed.

Here is the full code (changes in bold) :

Code:

declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
  

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
  
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
    
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
    
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
	if [ -z "$PREFIX" ] ; then
	  PREFIX=( slackpkgplus_${repository}:$package )
	else
	  PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
	fi
      fi
    done
    
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

  
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
    
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
	egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
	echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
	rm $2-tmp
      done
    fi

  }


    # Found packages in repository. 
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority 
    # given to package(s) from a given repository). This kind of priority 
    # uses the following syntax :
    #
    #   repository_name:pattern
    # 
    #
  function givepriority {
	  local DIR
	  local ARGUMENT=$1
	  local PKGDATA
	  local CPRIORITY
	  local DIR
	  local PKG

	  unset NAME
	  unset FULLNAME
	  unset PKGDATA
	  
	  for CPRIORITY in ${PRIORITY[@]} ; do
		  [ "$PKGDATA" ] && break
		  
		  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
		    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
		    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
		  

		    if echo "$ARGUMENT" | grep -q "$PAT" ; then
		      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
			
		    fi
		  else	
		    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
		  fi
		    
		  if [ "$PKGDATA" ]; then
		    NAME=${PKGDATA[1]}
		    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
		  fi
	  done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

	  NEWINPUTLIST=""

	  for pref in $INPUTLIST ; do
		  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
			  repository=$(echo "$pref" | cut -f1 -d":")
			  package=$(echo "$pref" | cut -f2- -d":")

			  PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
		  else
			  package=$pref
		  fi
		  
		  NEWINPUTLIST="$NEWINPUTLIST $package"
	  done

	  INPUTLIST=$NEWINPUTLIST
	  
  fi

  if [ "$CMD" == "install-new" ] ; then 
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
    
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
    
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
      
      for pkg in $NEWCOMPAT32PKGS ; do
	LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
  
fi

Cheers.

--
SeB

zerouno · 04-23-2013, 06:14 AM

I want to repackage slackpkg+

Eric, what do you think of the phenixia2003 ideas?

Alien Bob · 04-23-2013, 12:58 PM

I have not implemented phenixia2003's ideas here locally, yet. But his ideas sound good and would give slackpkg+ some needed flexibility.
One thing that I would implement but my ARM port does not leave me any time: to allow more than just the Slackware GPG key. It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

Other than that, I have been using slackpkg+ on my desktop machine (with a lot of alien and SBo packages, and multilib) without a single issues (except the failure to use GPG checking).

Eric

zerouno · 04-24-2013, 09:31 AM

Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

Alien Bob · 04-24-2013, 10:01 AM

Quote:

Originally Posted by zerouno

Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

No I have not yet done so, and I do not have short-term plans in that direction.

Eric

phenixia2003 · 04-24-2013, 10:07 AM

Hello,

Quote:

Originally Posted by Alien Bob

It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

I had an idea about that. This is not perfect, but I guess that's a good starting point.

First, you have to add the array MIRRORPLUSKEY in slackpkgplus.conf. In this array you must declare the key of each repositories declared into MIRRORPLUS. When a repository does not provide a GPG-KEY, then you must use the keyword NULL. Here is the slackpkgplus.conf I use.

Code:

SLACKPKGPLUS=on

MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

MIRRORPLUSKEY['slacky']="NULL"
MIRRORPLUSKEY['alienbob']="Eric Hameleers <alien@slackware.com>"
MIRRORPLUSKEY['multilib']="Eric Hameleers <alien@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

With this, you must use the new slackpkgplus.sh below (changes since my last post are in bold):

Code:

declare -A MIRRORPLUS
declare -A MIRRORPLUSKEY

if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then

     # -- In this version CHECKGPG can be set to "on". However, CHECKMD5
     #    must be set to off, because the CHECKSUMS.md5 files on alien
     #    repositories does not includes entries for .asc files.
     #
  CHECKGPG=on
  CHECKMD5=off
  
  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
  

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
  
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
    
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
    
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
	if [ -z "$PREFIX" ] ; then
	  PREFIX=( slackpkgplus_${repository}:$package )
	else
	  PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
	fi
      fi
    done
    
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

  
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
    
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
	egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
	$DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
	echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
	rm $2-tmp
      done
    fi

  }


    # Found packages in repository. 
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority 
    # given to package(s) from a given repository). This kind of priority 
    # uses the following syntax :
    #
    #   repository_name:pattern
    # 
    #
  function givepriority {
	  local DIR
	  local ARGUMENT=$1
	  local PKGDATA
	  local CPRIORITY
	  local DIR
	  local PKG

	  unset NAME
	  unset FULLNAME
	  unset PKGDATA
	  
	  for CPRIORITY in ${PRIORITY[@]} ; do
		  [ "$PKGDATA" ] && break
		  
		  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
		    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
		    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
		  

		    if echo "$ARGUMENT" | grep -q "$PAT" ; then
		      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
			
		    fi
		  else	
		    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
		  fi
		    
		  if [ "$PKGDATA" ]; then
		    NAME=${PKGDATA[1]}
		    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
		  fi
	  done
  }
  
	# -- Loads the GPG-KEY from the Slackware repository
	#
  function loadSlackwareGpgKey() {
	getfile ${SOURCE}GPG-KEY $TMPDIR/gpgkey 
	gpg --yes --batch --delete-key "$SLACKKEY" &>/dev/null
	gpg --import $TMPDIR/gpgkey &>/dev/null && \
	echo -e "\t\t\tSlackware Linux Project's GPG key added"
  }
  
    # -- Loads the GPG-KEY from the external repository ($1)
    #
  function loadRepositoryGpgKey() {
	local l_repo=$1
	local l_url=${MIRRORPLUS[$l_repo]}
	local l_mirrorkey=${MIRRORPLUSKEY[$l_repo]}
	
	if [ "$l_mirrorkey" == "NULL" ] ; then
		echo ""
		echo "(WARNING) ************************************************"
		echo "(WARNING) *    No key for external repository $l_repo    *"
		echo "(WARNING) ************************************************"
		echo ""
		return
	fi
	  
	getfile ${l_url}GPG-KEY $TMPDIR/gpgkey
	
	gpg --yes --batch --delete-key "$l_mirrorkey" &>/dev/null
	gpg --import $TMPDIR/gpgkey &>/dev/null && \
	echo -e "\t\t\t$l_repo GPG key added"
  }
  
  # -- This override updatefilelists() from original core-functions.sh
  #
  #
function updatefilelists()
{
	if checkchangelog ; then
		echo -e "\
\n\t\tNo changes in ChangeLog.txt between your last update and now.\n\
\t\tDo you really want to download all other files (y/N)? \c"
		answer
		if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
			cleanup
		fi
	fi
	echo
	cp ${TMPDIR}/ChangeLog.txt ${WORKDIR}/ChangeLog.txt

	#
	# Download MANIFEST, FILELIST.TXT and CHECKSUMS.md5
	#

	# That will be download MANIFEST.bz2 files
	#
	echo -e "\t\tList of all files"
	for i in ${PRIORITY[@]} ; do 
	
	    
	    # -- IMPORTANT:
	    #     Enhanced priority rules (ie: repository:name) must be
	    #     ignored, otherwise we'll get some nasty error messages.
		
	  if ! echo "$i" | grep -q "[a-zA-Z0-9]\+[:]" ; then
		getfile ${SOURCE}${i}/MANIFEST.bz2 $TMPDIR/${i}-MANIFEST.bz2 && \
		DIRS="$DIRS $i"
	  fi
	done

	ISOK="1"
	echo -e "\t\tChecksums"
	
	  # -- IMPORTANT:
	  #      The version of getfile() for slackpkg+ concatenates the files 
	  #      CHECKSUMS.md5 from the selected repositories. Therefore, the 
	  #      code to check signature of this file must be removed.
	  #
	getfile ${SOURCE}CHECKSUMS.md5 ${TMPDIR}/CHECKSUMS.md5		
	
#	getfile ${SOURCE}CHECKSUMS.md5.asc ${TMPDIR}/CHECKSUMS.md5.asc
	if ! [ -e "${TMPDIR}/CHECKSUMS.md5" ]; then
		echo -e "\
\n\t\tWARNING: Your mirror appears incomplete and is missing the\n\
\t\t         CHECKSUMS.md5 file. We recommend you change your mirror\n\
\t\t         so that package integrity can be verified against \n\
\t\t         CHECKSUMS.md5.\n"
		sleep 10
#	else
#		if [ "$CHECKGPG" = "on" ]; then
#			ISOK=$(checkgpg ${TMPDIR}/CHECKSUMS.md5)
#			if [ "$ISOK" = "0" ]; then 
#				read userkey
#				rm $TMPDIR/CHECKSUMS.md5
#				rm $TMPDIR/CHECKSUMS.md5.asc
#				echo -e "\
#\n\t\tERROR: Verification of the  gpg signature on CHECKSUMS.md5\n\
#\t\t       failed! This could mean that the file is out of date\n\
#\t\t       or has been tampered with.\n"
#				cleanup
#			fi
#		elif [ "$SLACKKEY" != "" ]; then
#			echo -e "\
#\n\t\tWARNING: Without CHECKGPG, we can't check if this file is\n\
#\t\t         signed by:\n\
#\n\t\t         $SLACKKEY.\n\
#\n\t\t         Enabling CHECKGPG is highly recommended for best\n\
#\t\t         security.\n"
#				sleep 10
#		fi
	fi

	ISOK="1"
	echo -e "\t\tPackage List"
	getfile ${SOURCE}FILELIST.TXT ${TMPDIR}/FILELIST.TXT
	if [ "$CHECKMD5" = "on" ]; then
		CHECKSUMSFILE=${TMPDIR}/CHECKSUMS.md5
		ISOK=$(checkmd5 ${TMPDIR}/FILELIST.TXT)
	fi
	if [ "$ISOK" = "1" ]; then 
		if ! [ -e $WORKDIR/LASTUPDATE ]; then
			echo "742868196" > $WORKDIR/LASTUPDATE
		fi
		LASTUPDATE=$(cat $WORKDIR/LASTUPDATE)
		ACTUALDATE=$(date -d "$(head -1 $TMPDIR/FILELIST.TXT)" "+%s")
		if [ $ACTUALDATE -lt $LASTUPDATE ]; then
			echo -e "\
\n\t\tFILELIST.TXT seems to be older than the last one.\n\
\t\tDo you really want to continue (y/N)? \c"
			answer
			if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
				cleanup
			fi
			echo
		fi
		echo $ACTUALDATE > $WORKDIR/LASTUPDATE
	else
		rm $TMPDIR/FILELIST.TXT
	fi
	
	if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
		FILELIST="$TMPDIR/CHECKSUMS.md5"
	elif [ -e $TMPDIR/FILELIST.TXT ]; then
		if [ "$ISOK" = "0" ]; then
			echo -e "\
\n\t\tERROR: CHECKSUMS.md5 signature doesn't match!\n\
\t\t       We strongly recommend that you change your mirror\n\
\t\t       to prevent security problems.\n"
			cleanup
		fi
		sleep 10
	  	FILELIST="$TMPDIR/FILELIST.TXT"
	else
		echo -e "\
\n\t\tERROR: No CHECKSUMS.md5 and no FILELIST.TXT.\n\
\t\t       We strongly recommend that you change your mirror\n\
\t\t       to prevent security problems.\n"
		cleanup
	fi 

	# Download all PACKAGES.TXT files
	# 
	echo -e "\t\tPackage descriptions"
	for i in $DIRS; do
		getfile ${SOURCE}${i}/PACKAGES.TXT $TMPDIR/${i}-PACKAGES.TXT
	done

	# Format FILELIST.TXT
	#
	echo -e "\tFormatting lists to slackpkg style..."
	echo -e "\t\tPackage List: using $( basename $FILELIST ) as source"
	grep "\.t[blxg]z$" $FILELIST| \
		awk -f /usr/libexec/slackpkg/pkglist.awk |\
		sed -e 's/^M//g' > ${TMPDIR}/pkglist
	cp ${TMPDIR}/pkglist ${WORKDIR}/pkglist		

	# Create the slackware tree under TEMP directory
	for i in $( cut -f7 -d\  ${WORKDIR}/pkglist | sort -u ) ; do
	  if ! [ -d ${TEMP}/${i} ]; then
	    mkdir -p ${TEMP}/${i}
	  fi
	done

	# Format MANIFEST
	#
		
	# bunzip and concatenate all MANIFEST files
	#
	MANFILES=""
	for i in $DIRS; do
		echo "(DEBUG) calling bunzip2 with $TMPDIR/${i}-MANIFEST.bz2"
		read userkey
		
		if [ -s $TMPDIR/${i}-MANIFEST.bz2 ] ; then
			bunzip2 -c $TMPDIR/${i}-MANIFEST.bz2 | awk -f /usr/libexec/slackpkg/filelist.awk | \
				gzip > ${TMPDIR}/${i}-filelist.gz
		fi
	done
	cp ${TMPDIR}/*-filelist.gz ${WORKDIR}/

	if [ -r ${WORKDIR}/filelist.gz ]; then
		rm ${WORKDIR}/filelist.gz
		ln -s ${WORKDIR}/${MAIN}-filelist.gz ${WORKDIR}/filelist.gz
	fi

	# Concatenate PACKAGE.TXT files
	#
	echo -e "\t\tPackage descriptions"
	for i in $DIRS; do
		cat $TMPDIR/${i}-PACKAGES.TXT >> $TMPDIR/PACKAGES.TXT
	done
	cp $TMPDIR/PACKAGES.TXT ${WORKDIR}/PACKAGES.TXT

	if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
		cp $TMPDIR/CHECKSUMS.md5 $WORKDIR/CHECKSUMS.md5 2>/dev/null
	fi

	if [ -e $TMPDIR/CHECKSUMS.md5.asc ]; then
		cp $TMPDIR/CHECKSUMS.md5.asc \
			$WORKDIR/CHECKSUMS.md5.asc 2>/dev/null
	fi
}
  
  
  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

	  NEWINPUTLIST=""

	  for pref in $INPUTLIST ; do
		  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
			  repository=$(echo "$pref" | cut -f1 -d":")
			  package=$(echo "$pref" | cut -f2- -d":")

			  PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
		  else
			  package=$pref
		  fi
		  
		  NEWINPUTLIST="$NEWINPUTLIST $package"
	  done

	  INPUTLIST=$NEWINPUTLIST
	  
  fi

  if [ "$CMD" == "install-new" ] ; then 
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
    
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
    
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
      
      for pkg in $NEWCOMPAT32PKGS ; do
	LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
  
    # -- New code to handle the command "update"...
    
  if [ "$CMD" == "update" ] ; then
	if [ "$UPARG" = "gpg" ] || [ "$GPGFIRSTTIME" = "0" ]; then
		#
		# Creates .gnupg directory if doesn't exist
		# without this dir, gpg got an error.
		#
		if ! [ -e ~/.gnupg ]; then
			mkdir ~/.gnupg
		fi
	
		loadSlackwareGpgKey

		for PREPO in $REPOPLUS; do
			loadRepositoryGpgKey $PREPO
		done
		
		if [ "$UPARG" = "gpg" ]; then
			cleanup
		fi
	fi
	
	echo "Updating the package lists..."
	updatefilelists
	
	cleanup
	exit
  fi 
  
fi

Important:

I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).

The getfile() in slackpkg+ concatenates the files CHECKSUMS.md5 from the selected repositories. Therefore, the code to check signature - in updatefilelists() - has been removed (colored in red in the code above). This should/could be moved into getfile().

I fixed a problem caused by the "enhanced priority rules" in updatefilelists() that leads to some nasty error messages (The correction is colored in green in the code above)

Now, with this new version, if you install apache-ant from alienbob repository you will get this :

Code:

slackpkg -batch=on -default_answer=y install alienbob:apache-ant

Looking for apache-ant in package list. Please wait... DONE

apache-ant-1.8.2-noarch-1alien.tgz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: apache-ant-1.8.2-noarch-1alien.tgz
--2013-04-24 16:39:10--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8382749 (8.0M) [application/x-gzip]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz'

100%[================================================>] 8,382,749    192KB/s   in 55s    

2013-04-24 16:40:11 (149 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz' saved [8382749/8382749]

--2013-04-24 16:40:11--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 198 [text/plain]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc'

100%[================================================>] 198         --.-K/s   in 0s      

2013-04-24 16:40:11 (615 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc' saved [198/198]

        Installing apache-ant-1.8.2-noarch-1alien...
Verifying package apache-ant-1.8.2-noarch-1alien.tgz.
Installing package apache-ant-1.8.2-noarch-1alien.tgz:
PACKAGE DESCRIPTION:
# Apache Ant (Java-based build tool analogous to Make)
#
# Ant is a platform-independent build tool for java.
#
# Ant is extended using Java classes.  The configuration files are
# XML- based, calling out a target tree where tasks get executed.
# Each task is run by an object that implements a Task interface.
#
# Homepage: http://ant.apache.org/
#
Executing install script for apache-ant-1.8.2-noarch-1alien.tgz.
Package apache-ant-1.8.2-noarch-1alien.tgz installed.

Hope this helps.

Greetings.

--
SeB

Alien Bob · 04-24-2013, 03:19 PM

Quote:

Originally Posted by phenixia2003

I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).

I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Eric

Alien Bob · 04-24-2013, 04:54 PM

Quote:

Originally Posted by Alien Bob

No I have not yet done so, and I do not have short-term plans in that direction.

Well I changed my mind, and the restricted_slackbuilds repository now has a 'modularized' version as well. See http://taper.alienbase.nl/mirrors/pe...icted_sbrepos/

Quote:

Originally Posted by Alien Bob

I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Eric

phenixia2003 · 04-25-2013, 05:21 AM

Hello,

Quote:

Originally Posted by Alien Bob

Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Great, so I removed the line "CHECKMD5=off" in my code and all is ok. However,I got another trouble because there's no file ChangeLog.txt at http://taper.alienbase.nl/mirrors/pe...os/14.0/x86_64.

I also made some changes to get something better.

First, I think that's not a good idea to have one array for the urls of repositories, and another for the keys.

So, I choose to merge these data into the array MIRRORPLUS. Now, when you declare a new repository you will have to supply the key through a field "?key=<key>" right after the url.

For instance, now my slackpkgplus.conf is as below :

Code:

SLACKPKGPLUS=on

  # MIRRORPLUS allow you to declare external repositories. Its syntax is as below:
  #
  #   MIRRORPLUS['<repository-name>']="<repository-url>?key=<repository-key>"
  #
  # When a repository has no GPG-KEY, the field "?key=<repository-key>" can
  # be undefined or must be set to "?key=null"
  #  
MIRRORPLUS['alienbob']="http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['multilib']="http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['slacky']="repository.slacky.eu/slackware64-14.0/?key=null"
#MIRRORPLUS['robby']="http://rlworkman.net/pkgs/14.0/?key=Robby Workman <rworkman@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

Note that Robby's repository works but only the i486 packages can be installed because the files required by slackpkg (CHECKSUMS.md5,...) are not at architecture level.

At runtime, slackpkgplus.sh parses the array MIRRORPLUS to extract the urls and keys of the defined repositories.

When a repository has no key, and you run "slackpkg update gpg" with CHECKGPG turned on, then a warning is printed on the standard output :

Code:

(WARNING) ************************************************************************************
(WARNING) * There is no GPG-KEY for the external repository <reponame> while CHECKGPG is on. *
(WARNING) * Therefore, any attempt to install/upgrade a package from this repository         *
(WARNING) * will fail.                                                                       *
(WARNING) ************************************************************************************

You will find a copy of slackpkgplus.conf and slackpkgplus.sh in attachment.

Greetings.

--
SeB

TobiSGD · 04-25-2013, 07:26 AM

phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?

Code:

slackpkg update

is just ignoring the additional repos, no error messages also.

phenixia2003 · 04-25-2013, 08:54 AM

Hello,

Quote:

Originally Posted by TobiSGD

phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?

Code:

slackpkg update

is just ignoring the additional repos, no error messages also.

slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:


MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"

if your local mirror has no gpg-key, do not put the field "?key=...", or set this to "?key=null". But keep in mind that in this case, you must set CHECKGPG to "off", or run slackpkg with option -checkgpg=off.

I tested this with a local copy of Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).

If you run into any trouble, let me know.

Greetings.
--
SeB

Alien Bob · 04-25-2013, 09:20 AM

Quote:

Originally Posted by phenixia2003

slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:


MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"

Slackpkg uses a "file://" syntax for that, please try to follow the original syntax where possible. Your solution only uses one slash.

Quote:

Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).

I will re-generate the multilib repository files with my new script, so that the CHECKSUMS.md5 will have entries for the .asc files.

Eric