[SOLVED] awk: how to achieve a 64character column file from a list of FORTH commands (and other things) ?

MadeInGermany · 06-24-2023, 11:18 AM

I have examined it.
But there is no abbreviation rule, e.g. it keeps the
." ELLIPSE PERIM = "
instead of changing it to
." ELL PERI = "

So I suggest to keep the separate forthbuild64column.
Of course you can use a pipe:

Code:

awk -f hp71bccfilter FILEIN.fth | awk -f forthbuild64column > FILEOUT.SRC

The previous solution wrapped at 56 + a new line; that could exceed the 64 length.
The following wraps close to the required 64 (but still does not split lines):

Code:

#!/usr/bin/awk -f
BEGIN { wrap=64 }
# Format the : block ;
$1 == ":" {
  line=""
  flag=1
}
flag {
  nline=(line $0)
  if (length(nline) > wrap) {
    print line
    line=$0
  } else {
    line=nline }
  if ($NF == ";") {
    flag=0
    print line
  }
  next
}
{ print }

floppy_stuttgart · 10-18-2023, 11:25 AM

Quote:

Originally Posted by MadeInGermany

The following variant looks for the ending semicolon only in the formatted block, and then prints it unconditionally.

Code:

#!/usr/bin/awk -f
BEGIN{wrap=54}
/^:/{
        line=""
        flag=1
}
flag{
        line=line $0
        if (/;$/){
          flag=0
          print line
          next
        }
        if (length(line)>wrap){
                print line
                line=""
        }
        next
}
{print}

That code was used for a time ( despite it eats few lines and more than 64 column output). Example below

original

Code:

-1 CONSTANT TRUE
0 CONSTANT FALSE 
: >= < 0= ;
: 0<> 0= 0= ;
: .R >R S->D R> D.R ;
: D0= OR 0= ;
: D= D- D0= ;
: 2, , , ; 
: 2! SWAP OVER ! 5+ ! ;
: 2@ DUP 5+ @ SWAP @ ; 
: 2CONSTANT CREATE , , DOES> 2@ ;
: 2VARIABLE CREATE 0 , 0 , DOES> ;
: AT-XY 27 EMIT 37 EMIT SWAP EMIT EMIT ;
: BLANK 20 FILL ;
: CELLS 5 * ;
: CELL+ 5+ ;
: CELL- 5- ;
: CHAR+ 2+ ;
: CHAR- 2- ;
: CHAR BL WORD CHAR+ C@ ;
: NIP SWAP DROP ;
: AGAIN POSTPONE FALSE POSTPONE UNTIL ; IMMEDIATE
: [CHAR] 
 ?COMP CHAR POSTPONE LITERAL ; IMMEDIATE
: CHARS 2* ; 
: CHARI= 
 OVER XOR DUP 0= IF 2DROP TRUE EXIT THEN 
 DUP 32 <> IF 2DROP FALSE EXIT THEN
 OR [CHAR] a [ CHAR z 1+ ] LITERAL WITHIN
;
: CHARI<> 
 CHARI= 0=
;
: EQUALSI 
 ROT OVER <> IF DROP 2DROP FALSE EXIT THEN
 OVER + >R
 BEGIN DUP R@ U< WHILE 
 >R COUNT R> COUNT ROT 
 CHARI<> IF 2DROP RDROP FALSE EXIT THEN
 REPEAT 2DROP RDROP TRUE
;
: COMPARE S= 0<> ;
: ?DO
 COMPILE 2DUP COMPILE = [COMPILE] IF COMPILE 2DROP 
 [COMPILE] ELSE [COMPILE] DO ['] THEN ; IMMEDIATE 
: LOOP 
 DUP ['] THEN <> IF [COMPILE] LOOP EXIT THEN
 >R [COMPILE] LOOP R> EXECUTE ; IMMEDIATE
: +LOOP 
 DUP ['] THEN <> IF [COMPILE] +LOOP EXIT THEN
 >R [COMPILE] +LOOP R> EXECUTE ; IMMEDIATE
: MS 0 DO LOOP ;
: N>$ S->D STR$ ;
: ON TRUE SWAP ! ;
: OFF FALSE SWAP ! ;
: PAGE 27 EMIT 69 EMIT ;
: PERFORM @ EXECUTE ;
: -ROT ROT ROT ;
: TO 
 ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE ! EXIT 
 THEN ! ; IMMEDIATE
: +TO ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE +! EXIT THEN 
+! ; IMMEDIATE
: UNLOOP R> DROP R> DROP ; 
: VALUE CREATE , DOES> @ ; 
HEX 
: VLIST E0005
 D 1 DO
 DUP @
 BEGIN DUP
  COUNT 1F AND 1-
  DUP >R 2* SWAP DUP >R
  + C@ 7F AND R> R>
  TYPE EMIT CR 5- @ ?DUP 0=
 UNTIL
 5+ LOOP DROP ;
DECIMAL
: WITHIN 
 OVER - >R - R> U< ;

filtered

Code:

-1 CONSTANT TRUE
0 CONSTANT FALSE 
: >= < 0= ;
: 0<> 0= 0= ;
: .R >R S->D R> D.R ;
: D0= OR 0= ;
: D= D- D0= ;
: 2! SWAP OVER ! 5+ ! ;
: 2CONSTANT CREATE , , DOES> 2@ ;
: 2VARIABLE CREATE 0 , 0 , DOES> ;
: AT-XY 27 EMIT 37 EMIT SWAP EMIT EMIT ;
: BLANK 20 FILL ;
: CELLS 5 * ;
: CELL+ 5+ ;
: CELL- 5- ;
: CHAR+ 2+ ;
: CHAR- 2- ;
: CHAR BL WORD CHAR+ C@ ;
: NIP SWAP DROP ;
: CHARI=  OVER XOR DUP 0= IF 2DROP TRUE EXIT THEN  DUP 32 <> IF 2DROP FALSE EXIT THEN
 OR [CHAR] a [ CHAR z 1+ ] LITERAL WITHIN;
: CHARI<>  CHARI= 0=;
: EQUALSI  ROT OVER <> IF DROP 2DROP FALSE EXIT THEN OVER + >R
 BEGIN DUP R@ U< WHILE  >R COUNT R> COUNT ROT  CHARI<> IF 2DROP RDROP FALSE EXIT THEN
 REPEAT 2DROP RDROP TRUE;
: COMPARE S= 0<> ;
: ?DO COMPILE 2DUP COMPILE = [COMPILE] IF COMPILE 2DROP 
: LOOP  DUP ['] THEN <> IF [COMPILE] LOOP EXIT THEN >R [COMPILE] LOOP R> EXECUTE ; IMMEDIATE
: +LOOP  DUP ['] THEN <> IF [COMPILE] +LOOP EXIT THEN >R [COMPILE] +LOOP R> EXECUTE ; IMMEDIATE
: MS 0 DO LOOP ;
: N>$ S->D STR$ ;
: ON TRUE SWAP ! ;
: OFF FALSE SWAP ! ;
: PAGE 27 EMIT 69 EMIT ;
: PERFORM @ EXECUTE ;
: -ROT ROT ROT ;
: TO  ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE ! EXIT 
: +TO ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE +! EXIT THEN 
: VLIST E0005 D 1 DO DUP @ BEGIN DUP  COUNT 1F AND 1-  DUP >R 2* SWAP DUP >R
  + C@ 7F AND R> R>  TYPE EMIT CR 5- @ ?DUP 0= UNTIL 5+ LOOP DROP ;
DECIMAL
: WITHIN  OVER - >R - R> U< ;

For example the HEX (line 66) disappeared. Anybody saying again thats not modern.. then look at the next Forth development ;-) https://8th-dev.com/

Any suggestion for improvement is more than welcome (I am awk limited). However, hand work is my currently plan B.

Target is:
- same structure as the file above top
- maximum 64 columns

Ideally the result

Code:

-1 CONSTANT TRUE
0 CONSTANT FALSE
: >= < 0= ;
: 0<> 0= 0= ;
: .R >R S->D R> D.R ;
: D0= OR 0= ;
: D= D- D0= ;
: 2, , , ;
: 2! SWAP OVER ! 5+ ! ;
: 2@ DUP 5+ @ SWAP @ ; 
: 2CONSTANT CREATE , , DOES> 2@ ;
: 2VARIABLE CREATE 0 , 0 , DOES> ;
: AT-XY 27 EMIT 37 EMIT SWAP EMIT EMIT ;
: BLANK 20 FILL ;
: CELLS 5 * ;
: CELL+ 5+ ;
: CELL- 5- ;
: CHAR+ 2+ ;
: CHAR- 2- ;
: CHAR BL WORD CHAR+ C@ ;
: NIP SWAP DROP ;
: AGAIN POSTPONE FALSE POSTPONE UNTIL ; IMMEDIATE
: [CHAR] ?COMP CHAR POSTPONE LITERAL ; IMMEDIATE
: CHARS 2* ;
: CHARI= OVER XOR DUP 0= IF 2DROP TRUE EXIT THEN DUP 32 <> IF
2DROP FALSE EXIT THEN OR [CHAR] a [ CHAR z 1+ ] LITERAL WITHIN ;
: CHARI<> CHARI= 0= ;
: EQUALSI ROT OVER <> IF DROP 2DROP FALSE EXIT THEN OVER + >R
BEGIN DUP R@ U< WHILE >R COUNT R> COUNT ROT CHARI<> IF 2DROP
RDROP FALSE EXIT THEN REPEAT 2DROP RDROP TRUE ;
: COMPARE S= 0<> ;
: ?DO COMPILE 2DUP COMPILE = [COMPILE] IF COMPILE 2DROP
[COMPILE] ELSE [COMPILE] DO ['] THEN ; IMMEDIATE
: LOOP DUP ['] THEN <> IF [COMPILE] LOOP EXIT THEN >R [COMPILE]
LOOP R> EXECUTE ; IMMEDIATE
: +LOOP DUP ['] THEN <> IF [COMPILE] +LOOP EXIT THEN >R
[COMPILE] +LOOP R> EXECUTE ; IMMEDIATE
: MS 0 DO LOOP ;
: N>$ S->D STR$ ;
: ON TRUE SWAP ! ;
: OFF FALSE SWAP ! ;
: PAGE 27 EMIT 69 EMIT ;
: PERFORM @ EXECUTE ;
: -ROT ROT ROT ;
: TO ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE ! EXIT THEN !
; IMMEDIATE
: +TO ' >BODY STATE @ IF POSTPONE LITERAL POSTPONE +! EXIT THEN
+! ; IMMEDIATE
: UNLOOP R> DROP R> DROP ;
: VALUE CREATE , DOES> @ ;
HEX
: VLIST E0005 D 1 DO DUP @ BEGIN DUP COUNT 1F AND 1- DUP >R 2*
SWAP DUP >R + C@ 7F AND R> R> TYPE EMIT CR 5- @ ?DUP 0= UNTIL
5+ LOOP DROP ;
DECIMAL
: WITHIN OVER - >R - R> U< ;

MadeInGermany · 10-19-2023, 01:18 AM

Try my last post#16
It won't print format lines >64, and won't omit a line.

But it still cannot handle an input line like

Code:

: AGAIN POSTPONE FALSE POSTPONE UNTIL ; IMMEDIATE

because it expects the terminating semicolon to be the last word in the line

Code:

  if ($NF == ";") {

Quick fix: also check the second to last word

Code:

  if ($NF == ";" || $(NF-1) == ";") {

floppy_stuttgart · 10-19-2023, 01:59 AM

I will have a look. Thanks. My current thoughts are, after a good night sleep (different from my previous spec)
a) a ":" seen in a line should bring it to a carriage return (=new line starting with : )
b) ";" should be ignored (since it can have IMMEDIATE after ; )
c) width 64 is important (my hardware want this maximum length)
d) all identified character must have a blank between them (no concatenation of a character seen at line end with another character seen at next line start)
I will now review the awk codes

MadeInGermany · 10-19-2023, 03:27 AM

Then it would chain terms like HEX and DECIMAL. Intended?

The to-be-chained lines start with a space character. Is this always the case? Could be an easier criterion.

floppy_stuttgart · 10-19-2023, 05:19 AM

Yes. Good remark. Then I was rethinking (I am a bit slow)
In fact, we must think in 2 topics
a) how to form a text for readability
b) how to form a text for compatibilty to Forth INTERPRETER
- never a <CR> after a : because the compiler think it has no words. this mean, better to start a new line with : if it is the last word
- some words needs a word after them example POSTPONE cannot be at the end of the line because it act on the next word, it mean POSTPONE XXXX at the end of the
line is fine (NOT POSTPONE alone). Other words are COMPILE or CONSTANT or VARIABLE which need another word after them before the carriage return is done

From a readability point of view, a : at the beginning of the line is good.

But from Forth point of view, all words ( ; or : or IMMEDIATE or > etc.) are all Forth words which must be separated by a BLANK for the INTERPRETER to work correctly.
so, the code below will work because recognized by the interpreter (tested on gforth with another added file = it will not work alone in gforth).
But its no fun to read. Usually, the : should be at the beginning of the line.

Code:

-1 CONSTANT TRUE 0 CONSTANT FALSE : >= < 0= ; : 0<> 0= 0= ; : .R
>R S->D R> D.R ; : D0= OR 0= ; : D= D- D0= ; : 2, , , ; : 2!
SWAP OVER ! 5+ ! ; : 2@ DUP 5+ @ SWAP @ ; : 2CONSTANT CREATE , ,
DOES> 2@ ; : 2VARIABLE CREATE 0 , 0 , DOES> ; : AT-XY 27 EMIT 37
EMIT SWAP EMIT EMIT ; : BLANK 20 FILL ; : CELLS 5 * ; : CELL+
5+ ; : CELL- 5- ; : CHAR+ 2+ ; : CHAR- 2- ; : CHAR BL WORD CHAR+
C@ ; : NIP SWAP DROP ; : AGAIN POSTPONE FALSE POSTPONE UNTIL ;
IMMEDIATE : [CHAR] ?COMP CHAR POSTPONE LITERAL ; IMMEDIATE
: CHARS 2* ; : CHARI= OVER XOR DUP 0= IF 2DROP TRUE EXIT THEN
DUP 32 <> IF 2DROP FALSE EXIT THEN OR [CHAR] a [ CHAR z 1+ ]
LITERAL WITHIN ; : CHARI<> CHARI= 0= ; : EQUALSI ROT OVER <> IF
DROP 2DROP FALSE EXIT THEN OVER + >R BEGIN DUP R@ U< WHILE >R 
COUNT R> COUNT ROT CHARI<> IF 2DROP RDROP FALSE EXIT THEN REPEAT
2DROP RDROP TRUE ; : COMPARE S= 0<> ; : ?DO COMPILE 2DUP
COMPILE = [COMPILE] IF COMPILE 2DROP [COMPILE] ELSE [COMPILE] DO
['] THEN ; IMMEDIATE : LOOP DUP ['] THEN <> IF [COMPILE] LOOP
EXIT THEN >R [COMPILE] LOOP R> EXECUTE ; IMMEDIATE : +LOOP DUP
['] THEN <> IF [COMPILE] +LOOP EXIT THEN >R [COMPILE] +LOOP R>
EXECUTE ; IMMEDIATE : MS 0 DO LOOP ; : N>$ S->D STR$ ; : ON TRUE
SWAP ! ; : OFF FALSE SWAP ! ; : PAGE 27 EMIT 69 EMIT ; : PERFORM
@ EXECUTE ; : -ROT ROT ROT ; : TO ' >BODY STATE @ IF
POSTPONE LITERAL POSTPONE ! EXIT THEN ! ; IMMEDIATE : +TO ' 
>BODY STATE @ IF POSTPONE LITERAL POSTPONE +! EXIT THEN +! ;
IMMEDIATE : UNLOOP R> DROP R> DROP ; : VALUE CREATE , DOES> @ ;
HEX : VLIST E0005 D 1 DO DUP @ BEGIN DUP COUNT 1F AND 1- DUP >R
2* SWAP DUP >R + C@ 7F AND R> R> TYPE EMIT CR 5- @ ?DUP 0= UNTIL
5+ LOOP DROP ; DECIMAL : WITHIN OVER - >R - R> U< ;

MadeInGermany · 10-19-2023, 06:30 AM

The following keeps adding lines if they start with a space or semicolon

Code:

#!/usr/bin/awk -f
BEGIN { wrap=64 }
# End the : block ;
/^[^ ;]/ {
 if (flag) print line
 flag=0
}
# Start the : block ;
$1 == ":" {
  line=""
  flag=1
}
flag {
# Add a line, ensure a separating space
  nline=(line ($0 ~ /^[ :]/ ? "" : " " ) $0)
  if (length(nline) > wrap) {
    print line
    line=$0
  } else {
    line=nline
  }
  next
}
{ print }
END { if (flag) print line }

floppy_stuttgart · 10-19-2023, 07:49 AM

That looks very good. Thanks.
Lets stay with it (with minimum handwork required; room for improvement: no blank at any start or end of the line, full 64 columns for all words ..).

MadeInGermany · 10-19-2023, 09:18 AM

The leading space can be eliminated. I think it is sufficient for the case when in a : block a line is not appended.
Just add one line.

Code:

...
    line=$0
# Delete leading space
    sub(/^ */, "", line)
...

Still the code can only join lines.
Splitting lines, in order to always fill to the 64 size, would be a new challenge. E.g. I guess that no split may occur in a quoted string like "hello world".

floppy_stuttgart · 10-20-2023, 02:12 AM

Quote:

Originally Posted by MadeInGermany

I guess that no split may occur in a quoted string like "hello world".

Yes.. by looking at each detail, the effort is getting exponential. Lets stay with this.

MadeInGermany · 10-20-2023, 06:26 AM

Greater but not exponential.
Here is a join+split version:

Code:

#!/usr/bin/awk -f
BEGIN { wrap=64 }
# Print the collected : block ;
function prt_blk(){
  so=out=st=tokn=""
  for (i=1; i <= bi; i++) {
    tokn=(tokn st blk[i])
# Collect a string as one token
    if (gsub("\"", "\"", tokn) % 2) {
      st=" "
      continue
    }
#   printf "debug tokn=(%s)\n", tokn
    nout=(out so tokn)
    so=" "
    if (length(nout) > wrap) {
      print out
      out=tokn
    } else {
      out=nout
    }
    st=tokn=""
  }
  print out
}
# End the : block ;
/^[^ ;]/ {
 if (flag) prt_blk()
 flag=0
}
# Start the : block ;
$1 == ":" {
  line=""
  flag=1
  bi=0
}
# Collect the : block ;
flag {
  for (fld=1; fld <= NF; fld++) { blk[++bi]=$fld }
  next
}
{ print }
END { if (flag) prt_blk() }

floppy_stuttgart · 10-20-2023, 10:59 AM

Thanks. However in special non listed cases, it screw up the result.
So far, the best result (used) is #22 plus #24 .