[SOLVED] How to convert the rows of data into the columns?

H_TeXMeX_H · 09-29-2012, 11:53 AM

On 64-bit you must you PIC, see here:
http://www.gentoo.org/proj/en/base/a...?part=1&chap=3

w1k0 · 09-29-2012, 04:28 PM

After I improved the binary-empty.c code I tested two simple plugins: binary-empty.bin and empty.wmi as well as a few the most useful more complicated plugins. Here’s the excerpt from the wminfo documentation that sums up the tests I performed using wminfo-benchmark-extended:

Quote:

I tested the tolerance of the wminfo-benchmark for very simple plugins as well as for the complicated ones.

First I ran 25 instances of empty-binary.wmi (it runs the plugin written in C that displays one line of the text) and empty.wmi (it is the plugin written in bash that displays one line of the text) plugins repeating that 5 times and performing the tests for three periods: 60, 180, and 300 seconds (each plugin-time combination gave 125 results; the total test lasted for 45 minutes).

The first table shows three results for each combination of plugin and time: the lowest CPU usage, the median, and the highest CPU usage:

Code:

             | 60             | 180            | 300
-------------+----------------+----------------+---------------
             | min  med  max  | min  med  max  | min  med  max
-------------+----------------+----------------+---------------
binary-empty | 0.25 0.26 0.28 | 0.24 0.24 0.25 | 0.24 0.25 0.25
empty        | 0.27 0.28 0.30 | 0.24 0.25 0.26 | 0.25 0.25 0.26

The second table shows the tolerances referred to the median value:

Code:

             | 60             | 180            | 300
-------------+----------------+----------------+---------------
binary-empty | -0.01 +0.02    | -0.00 +0.01    | -0.01 +0.00
empty        | -0.01 +0.02    | -0.01 +0.01    | -0.00 +0.01

The above plain plugins work very stable. Since the time equal 180 seconds the tolerance is not bigger than -0.01 +0.01. These two values represent also the wminfo-benchmark accuracy -- it measures the CPU usage exact to ±0.01%.

Next I tested six pairs of the plugins: date.wmi and date-us.wmi, conky.netmon.wmi and netmon.wmi, conky.sysmon.wmi and sysmon.wmi, conky-thinkpad.wmi and thinkpad.wmi, conky.top.wmi and top.wmi, and conky.uptime.wmi and uptime.wmi -- in each pair the first plugin uses less of the CPU power than the other. These are the most useful plugins designed as a replacements of respectively: wmCalClock, wmnet, wmsm.app, wmpower+, and wmtop Window Maker dockable applications. In each pair I ran 25 instances of the respective plugins repeating that 5 times and performing the tests for three periods: 60, 180, and 300 seconds (each plugin-time combination gave 125 results; the total test of each pair of the plugins lasted for 45 minutes).

The first table shows three results for each combination of plugin and time: the lowest CPU usage, the median, and the highest CPU usage:

Code:

               | 60             | 180            | 300
---------------+----------------+----------------+---------------
               | min  med  max  | min  med  max  | min  med  max
---------------+----------------+----------------+---------------
date           | 2.07 2.15 2.22 | 2.16 2.18 2.20 | 2.17 2.18 2.20
date-us        | 3.21 3.37 3.57 | 3.33 3.39 3.40 | 3.35 3.39 3.39
---------------+----------------+----------------+---------------
conky.netmon   | 0.75 1.05 1.17 | 0.88 0.96 1.14 | 0.89 0.96 1.10
netmon         | 4.07 5.07 5.83 | 3.32 4.69 5.39 | 3.48 3.87 5.34
---------------+----------------+----------------+---------------
conky.sysmon   | 1.21 1.33 1.38 | 1.21 1.29 1.34 | 1.23 1.29 1.33
sysmon         | 2.37 2.61 2.87 | 2.36 2.58 2.71 | 2.36 2.58 2.65
---------------+----------------+----------------+---------------
conky.thinkpad | 1.63 1.91 2.53 | 1.88 2.04 2.33 | 1.92 2.12 2.41
thinkpad       | 2.57 2.84 3.81 | 2.78 2.93 3.42 | 2.77 3.06 3.57
---------------+----------------+----------------+---------------
conky.top      | 1.10 1.15 1.21 | 1.10 1.12 1.17 | 1.09 1.10 1.24
top            | 1.41 1.52 1.63 | 1.40 1.45 1.47 | 1.35 1.39 1.47
---------------+----------------+----------------+---------------
conky.uptime   | 0.67 0.71 0.74 | 0.66 0.67 0.71 | 0.66 0.67 0.70
uptime         | 0.96 0.98 1.00 | 0.80 0.82 0.89 | 0.84 0.86 0.87

The second table shows the tolerances referred to the median value:

Code:

               | 60             | 180            | 300
---------------+----------------+----------------+---------------
date           | -0.08 +0.07    | -0.02 +0.02    | -0.01 +0.02
date-us        | -0.16 +0.20    | -0.06 +0.01    | -0.04 +0.00
---------------+----------------+----------------+---------------
conky.netmon   | -0.30 +0.12    | -0.08 +0.18    | -0.07 +0.14
netmon         | -1.00 +0.76    | -1.37 +0.70    | -0.39 +1.47
---------------+----------------+----------------+---------------
conky.sysmon   | -0.12 +0.05    | -0.08 +0.05    | -0.06 +0.04
sysmon         | -0.24 +0.26    | -0.22 +0.13    | -0.22 +0.07
---------------+----------------+----------------+---------------
conky.thinkpad | -0.28 +0.62    | -0.16 +0.29    | -0.20 +0.29
thinkpad       | -0.17 +0.97    | -0.15 +0.49    | -0.29 +0.51
---------------+----------------+----------------+---------------
conky.top      | -0.05 +0.06    | -0.02 +0.06    | -0.01 +0.14
top            | -0.11 +0.11    | -0.05 +0.02    | -0.04 +0.08
---------------+----------------+----------------+---------------
conky.uptime   | -0.04 +0.03    | -0.03 +0.04    | -0.01 +0.03
uptime         | -0.02 +0.02    | -0.02 +0.07    | -0.02 +0.01

The tolerances depend on the complication of the plugins. The highest diversity have: netmon.wmi, thinkpad.wmi, and conky.thinkpad.wmi plugins. The results of the tests lasting for 180 seconds are usually close to the results of the tests lasting for 300 seconds. The exceptions are: conky.top.wmi and uptime.wmi plugins.

The bottom line is: there is difficult to guess the tolerance of the measurments for the given plugin without measuring it.

This is wminfo-benchmark-extended based on wminfo-benchmark which I used to perform the mentioned above tests:

wminfo-benchmark-extended

Code:

#!/bin/bash

duration="060 180 300"

if [ "$1" != "" ]
then
    directory="$(echo $1 | sed -E 's#/$##;s#(.*)#\1/#')"
fi

if [ "$directory" == "" ]
then
    directory="./"
fi

cd $directory

wrapper() {
for plugin in *.wmi
do
    TIMEFORMAT=$(printf '%-24.24s: %s' "$plugin" '%2R real, %2U user, %2S sys, %P%% CPU')
    command=$(sed -n 's/^# command: \(wminfo .*\)/\1/p' "$plugin")
    # eval will correctly handle quotes
    # or any shell construct in the command
    eval time "$command" &
done
}

timer() {
    # both the time reports and Terminated messages go on stderr
    wrapper 2>&1 | grep -v Terminated | sort -k1,1 >> $time-$step &

    sleep $time
    killall wminfo

    # wait for all the reports to be printed before exiting
    wait
}

for time in $duration
do
    for step in $(seq 1 5)
    do
        for instance in $(seq -w 1 25)
        do
            timer &
        done
        wait
        cat $time-$step | sed 's/:/ :/' | sort -n -k9 > $time-${step}s
    done
    cat $time-1s $time-2s $time-3s $time-4s $time-5s | sort -n -k9 > $time-t-5s
done

ntubski · 09-29-2012, 04:29 PM

Quote:

Originally Posted by H_TeXMeX_H

On 64-bit you must you PIC, see here:
http://www.gentoo.org/proj/en/base/a...?part=1&chap=3

As far as I can tell, that link is only saying you need PIC for shared libraries.

Quote:

On certain architectures (AMD64 amongst them), shared libraries must be "PIC-enabled".

w1k0 · 09-29-2012, 04:29 PM

Quote:

Originally Posted by ntubski

I had noticed the missing return, but it didn't make sense that it would be reported as file not found. Then I realized it was a bug in my popen patch.

The behavior of the wminfo was irrational. Now it works well. Thank you for the new patch.

[EDIT]
Don’t overlook my post #92.
[/EDIT]

PTrenholme · 09-29-2012, 08:00 PM

I noticed that wminfo was using conky, and that conky "liked" lua, so, for what it's worth, here's a lua port of the last bash code I posted.

Code:

#!/bin/lua
b=" "
s="*"
B={{b,b,b,b},{b,b,b,s},{b,b,s,b},{b,b,s,s},{b,s,b,b},{b,s,b,s},{b,s,s,b},{b,s,s,s},{s,b,b,b},{s,b,b,s}}
D=os.date("+%H|%M|%S")
z=string.byte("0")-1
Hl=D:byte(2)-z
Hr=D:byte(3)-z
Ml=D:byte(5)-z
Mr=D:byte(6)-z
Sl=D:byte(8)-z
Sr=D:byte(9)-z
--print(D:sub(2))
print("HH|MM|SS")
print(" "      .. B[Hr][1] .. "| " ..             B[Mr][1] .. "| "            .. B[Sr][1])
print(B[Hl][2] .. B[Hr][2] .. "|"  .. B[Ml][2] .. B[Mr][2] .. "|" .. B[Sl][2] .. B[Sr][2])
print(B[Hl][3] .. B[Hr][3] .. "|"  .. B[Ml][3] .. B[Mr][3] .. "|" .. B[Sl][3] .. B[Sr][3])
print(B[Hl][4] .. B[Hr][4] .. "|"  .. B[Ml][4] .. B[Mr][4] .. "|" .. B[Sl][4] .. B[Sr][4])

On the other hand, I looked at your wminfo 3.1 tarball, and saw that all your conky.plugin/*.wmi files were bash scripts, so perhaps a lua port won't make much difference.

w1k0 · 09-30-2012, 01:08 AM

@PTrenholme,

Your Lua script has incredible performance. I tested the interpreted version and the compiled one. Both of them use exactly the same amount of the CPU power. Here’s the summary of the performance tests:

Code:

The CPU usage of the respective plugins used by wminfo is the following:

    plugin                   | author     | language | CPU usage
    -------------------------+------------+----------+----------
    binclock-vertical-01.wmi | H_TeXMeX_H | C        | 1.11%
    binclock-vertical-02.wmi | PTrenholme | C        | 1.12%
    binclock-vertical-03.wmi | PTrenholme | C        | 1.12%
    -------------------------+------------+----------+----------
    binclock-vertical-04.wmi | PTrenholme | Lua      | 1.21%
    binclock-vertical-05.wmi | PTrenholme | gawk     | 1.70%
    binclock-vertical-06.wmi | PTrenholme | gawk     | 1.72%
    binclock-vertical-07.wmi | PTrenholme | bash     | 2.71%
    binclock-vertical-08.wmi | grail      | bash     | 3.06%
    binclock-vertical-09.wmi | ntubski    | bash     | 3.13%
    binclock-vertical-10.wmi | PTrenholme | bash     | 3.16%
    binclock-vertical-11.wmi | grail      | Ruby     | 3.62%
    binclock-vertical-12.wmi | grail      | Ruby     | 3.63%
    -------------------------+------------+----------+----------
    binclock-vertical-13.wmi | w1k0       | bash     | 9.11%

The performance was measured with wminfo-benchmark from contrib directory
running for 180 seconds. To learn about the tolerances of the measurements
see README.wminfo-benchmark file from the contrib directory.

***

Before I ask you my question I have to describe how I use conky...

I started to use conky since wminfo 2.5.0. Earlier I prepared and improved the plugins written in bash and using different system-related programs and files such as top or /proc/stat. Despite of the improvements most of them used too much of the CPU power so I decided to try conky.

I prepared .conkyrc file that stores the information in the temporary file (in the example below for user w1k0):

Code:

overwrite_file /tmp/.w1k0.conky.tmp
temperature_unit celsius

out_to_x no
background no
cpu_avg_samples 2
net_avg_samples 2
no_buffers yes
out_to_stderr no
update_interval 3.0
update_interval_on_battery 6.0
uppercase no
use_spacer none
out_to_console no
extra_newline no
short_units

TEXT

SYSMON: CPU: $cpu%
SYSMON: $cpubar
SYSMON: MEM: $memperc%
SYSMON: SWP: $swapperc%
SYSMON: PRC: $processes

TOP: ${top cpu 1} ${top name 1}
TOP: ${top cpu 2} ${top name 2}
TOP: ${top cpu 3} ${top name 3}
TOP: ${top cpu 4} ${top name 4}
TOP: ${top cpu 5} ${top name 5}

As a result conky stores in the /tmp/.w1k0.conky.tmp file something like:

Code:

SYSMON: CPU: 24%
SYSMON: ##________
SYSMON: MEM: 39%
SYSMON: SWP: 1%
SYSMON: DSK: 62.0K
TOP:  12.51 X               
TOP:   1.17 X               
TOP:   1.00 bittorrent      
TOP:   1.00 wmtop           
TOP:   0.67 mc

Sample conky.top.wmi plugin looks like:

Code:

#!/bin/bash

# wminfo plugin: top CPU usage related command output

# command: wminfo -p conky.top.wmi -u 3 -b#000000 -f#ff0000

datafile="/tmp/.$USER.conky.tmp"

grep '^TOP: ' $datafile | sed -E 's/TOP:  //;s/\.[0-9]+//'

echo

It reads data from the /tmp/.w1k0.conky.tmp file, processes them, and displays the following information:

Code:

12 X               
 1 X               
 1 soffice.bin     
 1 bittorrent      
 0 wmtop

In the wminfo window it looks like:

Code:

+---------+
|12 X     |
| 1 X     |
| 1 soffic|
| 1 bittor|
| 0 wmtop |
+---------+

I was right: the conky-dependent plugins use less of the CPU power than their regular counterparts so among eleven plugins which I use constantly seven are conky-dependent and two following plugins use the CPU power just a bit.

***

My conky.plugin/*.wmi files are bash scripts because conky just prepares the raw information for them and they format it to the data useful for wminfo as in the example above.

Quote:

Originally Posted by PTrenholme

On the other hand, I looked at your wminfo 3.1 tarball, and saw that all your conky.plugin/*.wmi files were bash scripts, so perhaps a lua port won't make much difference.

I have the problem with your above statement.

I invented the method to store conky data into the temporary file myself and I wrote some bash plugins which read and process these data providing them to wminfo. If I understand you right I do something strange. Should I change the method of providing the data from conky to wminfo?

***

Quote:

Originally Posted by PTrenholme

I noticed that wminfo was using conky, and that conky "liked" lua, so, for what it's worth, here's a lua port of the last bash code I posted.

The above statement isn’t clear as well.

Is there any connection between your Lua script and conky? Should I use conky to run it?

I had the problem with your Lua script for a while because at the beginning I tried to read it with conky in order to invent the method allowing conky to store the binary clock data in the temporary file. Finally I realized that there is sha-bang at the beginning of the script so I ran it with lua and it started to work. New wminfo runs it without any problems as well.

H_TeXMeX_H · 09-30-2012, 02:19 AM

Quote:

Originally Posted by ntubski

As far as I can tell, that link is only saying you need PIC for shared libraries.

Yes, mostly. It also tells you what PIC is, and the errors you can expect in x86_64 if you don't use it.

However, you are right in that it doesn't say why you need it. Well, I can't find anything on exactly why.

PTrenholme · 09-30-2012, 11:20 AM

Well, conky does have several lua-specific hooks you can put into a configuration file. (e.g, lua_load scrip [ script...], etc.) that conky can use to make lua scripts "real" plug-ins. Use of those might make things run better. (Note, however, that this is all speculation on my part. What I know is just what I've read in the conky manual file.) From the description in the man page, it looks like one need to create a lau_draw_hook_pre function to calculate anything that needs to be updated before that window is redrawn, and so on.

Anyhow, I need to go walk my daughter's dog now, and then watch grown men bash each other in the Sunday games. (I.e., "Watch American Football" on TV.) So, tomorrow I'll install wminfo and see if I can set up a lua plugin that works with conky in it. (No promises: It may be harder then it appears right now.)

w1k0 · 09-30-2012, 12:57 PM

@PTrenholme,

As for Lua I started to look after that when you published your Lua script. Earlier I didn’t bother trying Lua because conky worked for me without that providing the data for wminfo plugins written in bash. Now I realized that Lua could improve conky performance.

As for your first paragraph I read the same about Lua in the man page yesterday and I watched some examples on the websites before I realized that your script is conky-independent and I can run it as is. The conky man page doses information about Lua thriftily so implementing that can be a challenge.

As for your second paragraph it can be interesting experience (I mean: walking dog, watching football, and setting up Lua plugin with conky). I can’t help with dog and with football. I can merely publish the current pre-release of wminfo:

[EDIT]
http://linux-bsd-unix.strefa.pl/wminfo-pre3.1.0.tar.gz.

Updated 03:41:03 CEST
[/EDIT]

As for our thread: the next post will bear #100 so we will achieve our next goal with it.

PTrenholme · 10-01-2012, 06:21 PM

And for #100, we have

:

I haven't gotten very far today with the lua stuff. I got distracted looking at the wmcloclmon C program in the dockapps/ WindowMaker site. (That's a really nice binary/digital clock program.)

w1k0 · 11-13-2012, 05:45 PM

Finally I finished the new wminfo release. Taking into consideration the great improvement in the comparison to the 3.0.0 version I decided to bump the version number to 4.0.0.

The wminfo 4.0.0 is so good thanks to the assistance and the inspirations that came from PTrenholme and ntubski. I’d like to thank them once again for their kind help. The program includes of course all the binary clocks by PTrenholme, H_TeXMeX_H, grail, and ntubski.

The tarball is available here:

http://dockapps.windowmaker.org/file.php/id/367

I submitted it also to:

https://freecode.com/search?q=wminfo

and

http://slackbuilds.org/repository/14.0/desktop/wminfo/

If you’re in a hurry you can download the unofficial SlackBuild for the new wminfo from here:

http://linux-bsd-unix.strefa.pl/index.en.html#wminfo

w1k0 · 02-06-2014, 09:24 PM

Hi guys and gals,

I just released wminfo 4.1.2.

It is available here:

● http://linux-bsd-unix.strefa.pl/index.en.html#wminfo

It will be available sooner or later here:

● http://freecode.com/projects/wminfo

and here:

● http://slackbuilds.org/repository/14...?search=wminfo

Unfortunately dockapps.windowmaker.org site is still unavailable:

○ http://dockapps.windowmaker.org/file.php/id/367

○ http://dockapps.windowmaker.org

***

Here is the short description of the new release of wminfo:

This release includes a lot of new plugins such as: beatclock.wmi (Swatch Internet Time), dice.wmi (a dice to toss), clock.wmi (semi-analog clock), fuzzy-clock.wmi (the time in plain English), acrobats-*.wmi, cyclist.wmi, golfer.wmi, ostrich.wmi (different animated plugins). Other highlights include: improved online script, improved FORECAST entries in conky.conf, rewritten cpumon.wmi and iching.wmi plugins, and a lot of minor changes.

Have fun!