[SOLVED] Tried to write something "serious" and have failed once again

jsbjsb001 · 07-23-2019, 10:02 AM

I didn't add any more code to my program - I tried figuring it out in a new file. I also tried reading the man page for strtok(), but it just didn't make much sense to me.

michaelk · 07-23-2019, 10:25 AM

In my program example I used a comma delimited string in the configuration file and thought an easy way to parse it was using strtok(). At the moment the program is only using two command line arguments and at the moment getopt() I think would add some additional confusion.

Granted I wrote it on the fly and most of the functions used were to get the parsed strings into a proper type required by execv(). I have been programming for a "few" years but I do not consider myself a programmer compared to many on this site. It does take time and effort...

rtmistler · 07-23-2019, 12:00 PM

Quote:

Originally Posted by jsbjsb001

I didn't add any more code to my program - I tried figuring it out in a new file. I also tried reading the man page for strtok(), but it just didn't make much sense to me.

This is compiled and tested:

Code:

#include <stdio.h>
#include <string.h>

void main(void)
{
    char sample[64] = "This,is,mysample,string";
    char *pChunk;
    char delim[2] = ",";

    printf("Original string: %s\n", sample);

    pChunk = strtok(sample, delim);

    while(pChunk != NULL)
    {
        printf("Section: %s\n", pChunk);
        pChunk = strtok(NULL, delim);
    }
}

When run:

Code:

$ ./tokey
Original string: This,is,mysample,string
Section: This
Section: is
Section: mysample
Section: string

Detail of importance from the manual page is:

Code:

Description
The strtok() function parses a string into a sequence of tokens.
On the first call to strtok() the string to be parsed should be
specified in str. In each subsequent call that should parse the
same string, str should be NULL.

This is why the subsequent calls to strtok() use NULL as the *str input, because it has been set up already by the first call.

Note that the token must be a string, cannot be just a single character, or a pointer to one character, it has to be a NULL terminated string. But you could also not use a variable and instead have the calls appear as:

Code:

pChunk = strtok(sample, ",");
pChunk = strtok(NULL, ",");

GazL · 07-23-2019, 05:32 PM

I really don't like the design of strtok(): specifically having to include two separate invocations in the code (one with str and one with NULL) makes looping over a string more awkward than it needs to be.

Interestingly, one can streamline it down to a single invocation with creative use of strtok_r() and saveptr, but it's not strictly speaking "correct" usage.

Example to show what I mean:

Code:

 
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

#define MAX_WORDS 100

int main( int argc, char *argv[] )
{
    static char string[] = { "This is a string containing lots of words" };
    char *words[MAX_WORDS] = { NULL } ;

    
    size_t count = 0;

    /* Strictly speaking this is not correct use of strtok_r() as POSIX says that
     * the first argument should point to the start of the string on the initial
     * call of strtok_r() and be NULL on any follow-up calls, however as the only 
     * "state" it uses is 'saveptr' I don't see why one can't simply do this instead
     */
    
    char *saveptr = string;

    while ( count < MAX_WORDS && (words[count] = strtok_r( NULL, " ", &saveptr)) )
        count++;

    
    printf("Found %zu words.\n", count );

    for ( size_t i = 0 ; i < count ; i++ )
        printf("Word %zu: %s\n", i, words[i]);
    
    return EXIT_SUCCESS;
}

Other than not following the rules as stated in the POSIX docs, can anyone think up a real-world reason why such an approach would be a bad idea?

jsbjsb001 · 07-24-2019, 01:33 AM

Thank you RT & GazL.

Thanks for that example RT. I wrote out what you done in your example in a new file like your example, and other than gcc complaining about main() not being "int main()" it worked (I changed "void main(void)" to "int main(void)"). While it does make at least a little more sense now; I'm not clear what exactly what the following is actually saying;

Quote:

On the first call to strtok() the string to be parsed should be specified in str.

What's "str" for starters?

Here's the example of strtok() I typed out based on your example;

Code:

#include <stdio.h>
#include <string.h>

int main(void) {

    char sample[64] = "This,is,my,sample";
    char *chunk;
   
    printf("Original string: %s\n", sample);

    chunk = strtok(sample, ",");

    while(chunk != NULL) {
         printf("Token: %s\n", chunk);
         chunk = strtok(NULL, ",");
    }
    
    return 0;
}

Which produces;

Code:

[james@jamespc devel]$ ./strtok_example 
Original string: This,is,my,sample
Token: This
Token: is
Token: my
Token: sample

So I tried to build an array using strcpy(), strcat() and strtok(), but I couldn't get all of the "tokens" into the other array. I created a new file once again without trying to do anything else, and I've included some comments where the relevant code didn't work, and caused the program to hang or put nothing into the other array.

Here's the code so far;

Code:

#include <stdio.h>
#include <string.h>

int main(void) {
  
    char string[100] = "ffmpeg:-i:something:blah:blah:blah";
    char result[100];
    int i;
    char *chunk;
    
    printf("Original string: %s\n", string);
    
    chunk = strtok(string, ":");

    while( chunk != NULL ) {
       // chunk = strtok(string, ":"); - ends up with nothing in "result"
        chunk = strtok(NULL, ":");
	printf("Token: %s\n", chunk);
	if (chunk) {
	//  for ( i = 0; chunk != NULL; ++i ) { - results in the program hanging where you need to press CTRL C to stop it
	      i = 0;
	      i++;
	      strcpy(result, chunk);
	      strcat(result, string);
	//  }
	}
	    
    }
    
    printf("result = %s\n", result);
 // copy string in "s2" to "s1"
 //   strcpy(s1, s2);

 // concatenate string in "s2" in "s1"
 //   strcat(s1, s2);
   
    return 0;

}

Here's the output;

Code:

[james@jamespc devel]$ ./build_array 
Original string: ffmpeg:-i:something:blah:blah:blah
Token: -i
Token: something
Token: blah
Token: blah
Token: blah
Token: (null)
result = blahffmpeg

I honestly don't know why it's just putting the first and last "tokens" into the "result" array. I also have no idea how to put a space between the "tokens" either.

GazL · 07-24-2019, 04:56 AM

Quote:

Originally Posted by jsbjsb001

What's "str" for starters?

...and that's when I shot him, your honour!

Code:

NAME
       strtok, strtok_r - extract tokens from strings

SYNOPSIS
       #include <string.h>

       char *strtok(char *str, const char *delim);

DESCRIPTION
       The  strtok()  function breaks a string into a sequence of zero or more
       nonempty tokens.  On the first call  to  strtok(),  the  string  to  be
       parsed should be specified in str.  In each subsequent call that should
       parse the same string, str must be NULL.

The Synopsis isn't there simply for decoration.

michaelk · 07-24-2019, 06:05 AM

str=string
strcpy= string copy
strcat= string concatenate

rtmistler · 07-24-2019, 07:48 AM

Quote:

Originally Posted by jsbjsb001

I honestly don't know why it's just putting the first and last "tokens" into the "result" array. I also have no idea how to put a space between the "tokens" either.

The flow of your code does this:

Before the loop, you extract the first chunk "ffmpeg", but you never use it or print it. Notice how my loop prints the chunks first before the next call to strtok()?
1. Can you make that old Jewish man "Ach-Hah!" sound? Applies here and sorry if people find the joke offensive, can't resist/my way of saying if you review that code concisely, you'll see why this is so.
That commented out strtok() results in nothing because you are constantly re-seeding the initial string. Sorry, but the function is weird, the manual page does point this out, you either have to learn/re-learn it each time, versus not. To do my example, I had to re-review the manual page and do some trial and error, just so you know. I don't normally use that function, and "just because", not because I avoid it, just because I don't always manipulate strings like that.
You're next (uncommented out) call to strtok() is fine.
strcpy() STARTS at the beginning, thus you are getting a newly wiped string each time.
The strcat() and the result is probably because the oddness of strtok() where it likely replaces the delimiters with a NULL. You could see this if you chose to printf() string at each loop iteration. It is what I would do, and I'd also force it to print out any and all contents of that entire array so that I could learn. I honestly do not know, but that is my guess here. (In fact, I may do just that after posting this, because I sort of want to know, but never had checked.)

Those are the errors.

My continued debug points and instruction points are as follows:

You may have been able to discern some of these things by adding some printf() statements to show up during the loop iterations. You likely may have continued questions, but you'd be closer than you were at the time of your composition.
Once again, I've said to stay fundamental, but I'll now say the acronym, K.I.S.S. Keep It Simple (won't say, not calling you that and this is why I do not use that acronym faithfully.) It is meant to be humorously self-deprecating, however not serious, just a dope slap on the forehead to remind ones self that you're making this SO much harder for yourself that you need to.
1. Hardcode the #@$%! array for the exec() call to ffmpeg.
2. Exclamation points! Creating a daemon to launch and monitor your processes. The section titled Creating Child Processes. Actual code example of an array form of an argument list used to invoke a call to execvp(), and in fact you can see how I modified that argument list to fill in variable terms
3. Didn't write that to be smart, to self-advertise, but instead to be able to point people to it to show actual compiled, tested, and working code which does "something", and in this case, very applicable to what you're doing.

michaelk · 07-24-2019, 07:55 AM

In addition or in other terms.

Code:

strcpy(result, chunk);

strcpy copies the contents of chunk into result which basically means
result=chunk

Code:

strcat(result, chunk);

strcat appends the contents of chunk to the end of result which basically means
result=result+chunk
A similar command in bash would look like
result=$result$chunk

rtmistler · 07-24-2019, 07:57 AM

It's quick/dirty, but it does prove out the hypothesis. Note that my calling of a function using a char * and then literally treating it as an array is not good coding, but in this particular case we know the full scope of the array.

Code:

#include <stdio.h>
#include <string.h>
#include <ctype.h>

void Debug_String(char *pstr)
{
    for(int i=0; i<64; i++)
    {
        if(isprint(pstr[i]))
            printf("%c", pstr[i]);
        else
            printf("<%02X>", pstr[i]);
    }
}

void main(void)
{
    char sample[64] = "This,is,mysample,string";
    char *pChunk;
    char delim[2] = ",";

    printf("Original string: %s\n", sample);

    pChunk = strtok(sample, delim);

    while(pChunk != NULL)
    {
        printf("Section: %s\n", pChunk);
        Debug_String(sample);
        pChunk = strtok(NULL, ",");
    }
}

Output result:

Code:

$ ./tokey
Original string: This,is,mysample,string
Section: This
This<00>is,mysample,string<00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00>Section: is
This<00>is<00>mysample,string<00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00>Section: mysample
This<00>is<00>mysample<00>string<00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00>Section: string
This<00>is<00>mysample<00>string<00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00><00>

I've learned something about how strtok() operates today. Hope you have too.

K.I.S.S.

jsbjsb001 · 07-24-2019, 11:11 AM

Thanks again guys!

Quote:

Originally Posted by michaelk

str=string
strcpy= string copy
strcat= string concatenate

Yeah, I guessed what strcpy and strcat stood for (that same as you said above that is), but I wasn't quite sure what the man page meant by "str".

Quote:

Originally Posted by rtmistler

...
It is meant to be humorously self-deprecating, however not serious, just a dope slap on the forehead to remind ones self that you're making this SO much harder for yourself that you need to.

Hardcode the #@$%! array for the exec() call to ffmpeg.
Exclamation points! Creating a daemon to launch and monitor your processes. The section titled Creating Child Processes. Actual code example of an array form of an argument list used to invoke a call to execvp(), and in fact you can see how I modified that argument list to fill in variable terms
Didn't write that to be smart, to self-advertise, but instead to be able to point people to it to show actual compiled, tested, and working code which does "something", and in this case, very applicable to what you're doing.

I'm not honestly trying to make things harder, but I'm not a pro like you guys either. It's one thing when you've got a book that explains any examples line by line, but when you haven't got that luxury, it's a lot harder without that safety net.

I wasn't trying and haven't yet tried to use execv() to execute ffmpeg yet. I did have a quick look at your blog already, but I'll have to re-read it, as it was a little beyond me when I had a look at it before. So I'll try and give it a "fuller" read soon. I was just trying to figure out strcpy(), strcat() and strtok() for now - I haven't even touched my program itself yet.

I'm sure you wouldn't self-advertise RT

Quote:

Originally Posted by rtmistler

...
I've learned something about how strtok() operates today. Hope you have too.
...

We'll see. At least I know how to write code that doesn't work, so if that counts, then I'll say yes...
only kidding RT, well, I know more than what I did about it before, which was basically nothing.

But anyway, I've just been having a play around with strtok() and strcat(), and I've gotten the original string into my second little array thanks to RT's post #68. I got rid of the calls to strcpy(). But if I declare the "result" array as a pointer array, I get what looks like some garbage just before "ffmpeg" though - but otherwise it seems to work fine (if I just declare the "result" array as a normal char array, and not as a pointer array, being "char *result[100];").

Code:

#include <stdio.h>
#include <string.h>

int main(void) {
  
    char string[100] = "ffmpeg: -i: something: blah: blah: blah";
    char result[100];
    char *chunk;
    
    printf("Original string: %s\n", string);
    
    chunk = strtok(string, ":");
    printf("chunk = %s\n", chunk);
    strcat(result, chunk);

    while( chunk != NULL ) {
        chunk = strtok(NULL, ":");  
	printf("Token: %s\n", chunk);
	if (chunk) {
	   printf("chunk = %s\n", chunk);
	   strcat(result, chunk);
	}
	   
    }
    
    printf("result = %s\n", result);
 
 // concatenate string in "s2" in "s1"
 //   strcat(s1, s2);
   
    return 0;

}

output;

Code:

[james@jamespc devel]$ ./build_array 
Original string: ffmpeg: -i: something: blah: blah: blah
chunk = ffmpeg
Token:  -i
chunk =  -i
Token:  something
chunk =  something
Token:  blah
chunk =  blah
Token:  blah
chunk =  blah
Token:  blah
chunk =  blah
Token: (null)
result = ffmpeg -i something blah blah blah

But if I declare the "result" array as a pointer array, I get;

Code:

Original string: ffmpeg: -i: something: blah: blah: blah
chunk = ffmpeg
Token:  -i
chunk =  -i
Token:  something
chunk =  something
Token:  blah
chunk =  blah
Token:  blah
chunk =  blah
Token:  blah
chunk =  blah
Token: (null)
result = ���~0ffmpeg -i something blah blah blah

rtmistler · 07-24-2019, 12:21 PM

Quote:

Originally Posted by jsbjsb001

But if I declare the "result" array as a pointer array, I get;

So what was the question?

I'm guessing that your question would be "Why did that happen?"

A: You can't create sometin' from nuttin'!

Declaring something as a pointer means you need to either allocate memory for it to point to, or point it to existing memory.

strcpy() and strcat() only move information around, they do not allocate memory for you.

jsbjsb001 · 07-26-2019, 05:29 AM

Yes, that's what I was wondering RT. Thank you for the answer.

So since I've managed to at least partly solve that issue, I've integrated that code into my program. I've changed the name of the config file to "ffcliFront.conf", as it seemed more appropriate. I also renamed the pointer called "chunk" to "tokenPtr" to make it's purpose a little more clear. I've also added a "Usage" statement as suggested before, as well as a forth argument for the output file. But it keeps putting the argument for the output file on a new line in the array to store the args for ffmpeg's command-line no matter what I try - so I commented that out, and was thinking I could maybe include that in the execv() line instead.

While the code I've added does seem to work; if the second argument (argv[1]) for getting the id for the relevant string from the config file has the same characters it in that another line in the config file does, it doesn't get the right string, and appears to keep looping until it reaches the last line in the config file. So for example, if I have the following in the config file;

Code:

test, ffmpeg, ehe, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey
test1, ffmpeg, ehe1, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey
test2, ffmpeg, ehe2, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

and I type in;

Code:

./ffcliFront test testinput out

I get;

Code:

argv[1] = test
Line from config file: test, ffmpeg, ehe, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

Found string: test, ffmpeg, ehe, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

tokenPtr = test
Token:  ffmpeg
tokenPtr =  ffmpeg
Token:  ehe
tokenPtr =  ehe
Token:  te
tokenPtr =  te
Token:  teyeyyes
tokenPtr =  teyeyyes
Token:  y
tokenPtr =  y
Token:  eye
tokenPtr =  eye
Token:  yeh
tokenPtr =  yeh
Token:  eyh
tokenPtr =  eyh
Token:  eeee
tokenPtr =  eeee
Token:  eyeye
tokenPtr =  eyeye
Token:  e
tokenPtr =  e
Token:  ey

tokenPtr =  ey

Token: (null)
Line from config file: test1, ffmpeg, ehe1, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

Found string: test1, ffmpeg, ehe1, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

tokenPtr = test1
Token:  ffmpeg
tokenPtr =  ffmpeg
Token:  ehe1
tokenPtr =  ehe1
Token:  te
tokenPtr =  te
Token:  teyeyyes
tokenPtr =  teyeyyes
Token:  y
tokenPtr =  y
Token:  eye
tokenPtr =  eye
Token:  yeh
tokenPtr =  yeh
Token:  eyh
tokenPtr =  eyh
Token:  eeee
tokenPtr =  eeee
Token:  eyeye
tokenPtr =  eyeye
Token:  e
tokenPtr =  e
Token:  ey

tokenPtr =  ey

Token: (null)
Line from config file: test2, ffmpeg, ehe2, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

Found string: test2, ffmpeg, ehe2, te, teyeyyes, y, eye, yeh, eyh, eeee, eyeye, e, ey

tokenPtr = test2
Token:  ffmpeg
tokenPtr =  ffmpeg
Token:  ehe2
tokenPtr =  ehe2
Token:  te
tokenPtr =  te
Token:  teyeyyes
tokenPtr =  teyeyyes
Token:  y
tokenPtr =  y
Token:  eye
tokenPtr =  eye
Token:  yeh
tokenPtr =  yeh
Token:  eyh
tokenPtr =  eyh
Token:  eeee
tokenPtr =  eeee
Token:  eyeye
tokenPtr =  eyeye
Token:  e
tokenPtr =  e
Token:  ey

tokenPtr =  ey

Token: (null)
Line from config file: 

Line from config file: 

Line from config file: 

ffargs = -i testinput ffmpeg ehe2 te teyeyyes y eye yeh eyh eeee eyeye e ey

But if I specify "test1" or "test2" instead; it doesn't loop 2 or 3 times, and gives the correct output. I've tried several different things to fix it, like putting a break; statement in the second while loop, tried adding an if statement both inside and outside of both while loops to check if argv[1] is equal to argv[1], and if tokenPtr is equal to argv[1] and similar, but as usual, I'm getting nowhere. It just ends up breaking the program in some way, like not putting all of the line in the config file into the array, or putting absolutely nothing into the array. I'm outa ideas. I was also trying to put in a check, so that if you type in a string that doesn't match any of the strings at the beginning of any of the lines in the config file, it can issue and error message and quit - but again whatever I try fails - I commented out that code below.

Here's my code as it currently stands;

Code:

// includes
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>

char readConfigFile(char *argv[]);
//char readConfigFile(char *filename, char *formatID);
char execFfmpeg(char *argv[], char ffargs[]);

// main function
int main(int argc, char *argv[]) {
  
    if ( argc < 4 ) {
       fprintf (stderr, "Missing option or filename\n");
       puts("USAGE: ffcliFront <format_id> <input_file> <output_file>");
       return 1;
    }
          
    readConfigFile(argv);
    //readConfigFile(argv[1], argv[2]);

    return 0;
    
}

//char readConfigFile(char *filename, char *formatID) {
char readConfigFile(char *argv[]) {

    char ffFormatID[128];
    char ffargs[128];
    char *tokenPtr;
    
    // check if input file exists
    FILE *inputfile;
        
    if ( (inputfile = fopen(argv[2], "r")) == NULL ) {
       fprintf(stderr, "Input file not found, exiting.\n");
       return 1;        
    }
    fclose(inputfile);
       
    // read config file
    FILE *configfile;
      
       if ( (configfile = fopen("ffcliFront.conf", "r")) == NULL ) {
       fprintf(stderr, "Configuration file not found, exiting.\n");
       return 1;        
    }
    
    printf("argv[1] = %s\n", argv[1]);
       
    if ( (configfile = fopen("ffcliFront.conf", "r")) != NULL) {
       while( fgets(ffFormatID, 128, configfile ) != NULL) {
             printf("Line from config file: %s\n", ffFormatID);
	     if ( (strstr(ffFormatID, argv[1]) ) != NULL) {
	        printf("Found string: %s\n", ffFormatID);
		tokenPtr = strtok(ffFormatID, ",");
		printf("tokenPtr = %s\n", tokenPtr);
                strcpy(ffargs, "-i ");
		strcat(ffargs, argv[2]);
		 
                while( tokenPtr != NULL ) {
		      tokenPtr = strtok(NULL, ",");  
	              printf("Token: %s\n", tokenPtr);
	              if (tokenPtr) {
	                 printf("tokenPtr = %s\n", tokenPtr);
	                 strcat(ffargs, tokenPtr);
		      }
			 
		}
		
                //  strcat(ffargs, argv[3]); 
	     }
	  //    else if ( argv[1] != argv[1] ) {
		//      fprintf(stderr, "\"%s\" format_id not found in configuration file\n", argv[1]);
		   //   return 1;
	  //    }
	      
       }
       
    }
     
    fclose(configfile);
             
    printf("ffargs = %s\n", ffargs);
        
    // pass char array with ffmpeg args & call execFfmpeg() fuction with args from config file
   // execFfmpeg(argv, ffargs);
 
    return 0;
}      

//char execFfmpeg(char *argv[], char ffargs[]) {
  
    //char *ffcmdline[128];  
     
    //printf("%s\n", ffargs); 
    // run fmpeg with args from config file & filename supplied to this program. 
    // execv ("/usr/bin/ffmpeg", argv[3], ffcmdline);
	
    //return 0;
//}

rtmistler · 07-26-2019, 05:50 AM

Is there a question?

jsbjsb001 · 07-26-2019, 05:56 AM

Quote:

Originally Posted by rtmistler

Is there a question?

Yes, how can I stop it from continuing to loop with the same string specified for the format_id? And also, how can I get it to check if the format_id string exists in the config file, and if it doesn't, give the error message and then quit? And also, can I specify argv[3] in execv(), instead of trying to add it to the array for ffmpeg's command-line args? Or is it better to put it into the array instead?

Like I said before, I have tried to do all of this, but I've failed every time, so I could really use some help with the above. Thanks.