GNU Make and Linux file permissions

aryan1 · 12-31-2009, 05:26 AM

Hi All,

I am recently having problems with the compilation of my C++ source files with make.

The modifications to the source files are recognized and, hence, the sources get re-compiled successfully in a certain machine while the modifications can not be recognized at all on another machine, and the old object files are kept being used to create the binaries.

git is used for version control. In our configuration, even though the machines where software development takes place can pull from each other, one server machine is used as a centric repository from which everybody pulls.

I am suspecting that, after pulling from git, somehow, Linux file permissions do not allow the developer to compile the source successfully.

Any ideas are appreciated.

Thanks.

ozanbaba · 12-31-2009, 09:35 AM

Quote:

Originally Posted by aryan1

Hi All,

I am recently having problems with the compilation of my C++ source files with make.

The modifications to the source files are recognized and, hence, the sources get re-compiled successfully in a certain machine while the modifications can not be recognized at all on another machine, and the old object files are kept being used to create the binaries.

git is used for version control. In our configuration, even though the machines where software development takes place can pull from each other, one server machine is used as a centric repository from which everybody pulls.

I am suspecting that, after pulling from git, somehow, Linux file permissions do not allow the developer to compile the source successfully.

Any ideas are appreciated.

Thanks.

does file modification times that are recorded on the file system changes?

ForzaItalia2006 · 01-01-2010, 04:38 PM

Hey aryan1,

Quote:

Originally Posted by aryan1

Hi All,
I am suspecting that, after pulling from git, somehow, Linux file permissions do not allow the developer to compile the source successfully.

(GNU) Make doesn't really care about file permissions, but only about modification times when determining if an object file or executable should be rebuilt. If you were facing a file permission issue, make would have complained with an error :-)

As I'm thinking that the project which you're working on is a bit bigger(?), you should possibly try to minimize the makefile for a reproducable test case and then use the -d option for make to get some debugging options, for the case you can't determine if the modification time changed as ozanbaba suggested ...

Hope that helps,

- Andi -

theNbomr · 01-04-2010, 02:52 PM

If you touch the source file(s), does make then rebuild the object code?

--- rod.

bigearsbilly · 01-05-2010, 02:04 PM

you aren't mixing files from windows as well are you?
of course, we could guess all day - or maybe,
perhaps you could share with us the error messages you get?

CoderMan · 01-06-2010, 02:02 PM

Quote:

Originally Posted by aryan1

Hi All,

I am recently having problems with the compilation of my C++ source files with make.

The modifications to the source files are recognized and, hence, the sources get re-compiled successfully in a certain machine while the modifications can not be recognized at all on another machine, and the old object files are kept being used to create the binaries.

git is used for version control. In our configuration, even though the machines where software development takes place can pull from each other, one server machine is used as a centric repository from which everybody pulls.

I am suspecting that, after pulling from git, somehow, Linux file permissions do not allow the developer to compile the source successfully.

Any ideas are appreciated.

Thanks.

We want to help, but you have given us almost no helpful information. Git should have nothing to do with this, because the git repository should be own by the person compiling, and the compiler only needs read access to the source code to be able to create the object files. Unless, for some freaky reason, the git repository is also tracking object files, which is highly unlikely.

Does this repository just use a Makefile (not Autotools?) It could be that the Makefile was written badly, with improper dependency configuration or something like that. But since you didn't post it... who knows.

Also might help to know what kind of system is having the problems.

aryan1 · 01-08-2010, 01:52 AM

Quote:

Originally Posted by theNbomr

If you touch the source file(s), does make then rebuild the object code?

--- rod.

First of all, sorry for this late reply.

I did not try that specifically.

However, Codeblocks is used for development.

aryan1 · 01-08-2010, 01:54 AM

Quote:

Originally Posted by bigearsbilly

you aren't mixing files from windows as well are you?

No. All development takes place in Linux.

Quote:

perhaps you could share with us the error messages you get?

There is no error -- it appears to compile successfully.

aryan1 · 01-08-2010, 02:01 AM

Quote:

Originally Posted by CoderMan

Git should have nothing to do with this, because the git repository should be own by the person compiling, and the compiler only needs read access to the source code to be able to create the object files. Unless, for some freaky reason, the git repository is also tracking object files, which is highly unlikely.

This problem was originally reported by another person. Now, she says that everything began to compile as expected. I have no idea why it suddenly began to work as expected ...

Since I am new to git, all "freaky" reasons are possible.

Quote:

Does this repository just use a Makefile (not Autotools?) It could be that the Makefile was written badly, with improper dependency configuration or something like that. But since you didn't post it... who knows.

It just uses a Makefile.

Quote:

Also might help to know what kind of system is having the problems.

Ubuntu 9.10

I read somewhere else that "This sort of problem often happens when the clocks of the various systems are out of sync. Thus, all the systems should be configured to sync their clock with ntp." ?

aryan1 · 01-08-2010, 02:16 AM

Quote:

Originally Posted by CoderMan

It could be that the Makefile was written badly, with improper dependency configuration or something like that.

Can you please provide some info in terms of your comments, http links, books, etc. about the guidelines to write flawless Makefiles ?

Thanks.

CoderMan · 01-08-2010, 11:38 PM

Quote:

Originally Posted by aryan1

Can you please provide some info in terms of your comments, http links, books, etc. about the guidelines to write flawless Makefiles ?

Thanks.

Well, its not anything real deep. Mainly what I was thinking about was making sure you have all your targets and dependencies lined up correctly, so that everything that is supposed to rebuild actually does. If you don't know what that means, then I'd suggest a beginner's tutorial.

If you actually want to write perfect Makefiles, my suggestion is: don't. Use a build system. Build systems write the Makefiles for you (or use methods other than Makefiles). I use "Autotools" personally, but I have also used "Cmake", and I've heard of another one called "Scons".

manavendra · 01-09-2010, 01:55 AM

Sometimes, when nothing else seems to work, start from a clean build.

Code:

$ make clean
$ make

aryan1 · 01-11-2010, 05:45 AM

Quote:

Originally Posted by CoderMan

Well, its not anything real deep. Mainly what I was thinking about was making sure you have all your targets and dependencies lined up correctly, so that everything that is supposed to rebuild actually does. If you don't know what that means, then I'd suggest a beginner's tutorial.

If you actually want to write perfect Makefiles, my suggestion is: don't. Use a build system. Build systems write the Makefiles for you (or use methods other than Makefiles). I use "Autotools" personally, but I have also used "Cmake", and I've heard of another one called "Scons".

If I 'touch' all files, 'make' SEEMS to detect the timestamp changes for all touched files, and re-build the object files. However, I can not see the changes in program output.

Here are my Makefiles:

Makefile (I call 'make' for this file):

Code:

all:
	make -f Makefile_Ses
	make -f Makefile_Repo
	make -f Makefile_Code

clean:
	make -f Makefile_Ses clean
	make -f Makefile_Repo clean
	make -f Makefile_Code clean

Makefile_Ses:

Code:

LIB_DIR = ../common/lib
OBJ_DIR = ../common/src
SRC_DIR = ../common/src
INCLUDE_DIR = ../common/inc
INCLUDE_DIR_BOOST = ../../third_party

#build ses library
SRC_MODULES = Ses.cpp Tcp.cpp Data.cpp Str.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))
LIB_ST = $(LIB_DIR)/libSes.a

DEBUG = -g
CFLAGS = -Wall -c $(DEBUG) -I $(INCLUDE_DIR) -I $(INCLUDE_DIR_BOOST)

$(LIB_ST): $(OBJ_FILES)
	ar -cru $(LIB_ST) $(OBJ_FILES)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)
	rm -f $(LIB_DIR)/*

Makefile_Repo:

Code:

BASE_DIR = $(HOME)
LIB_DIR = ../common/lib
OBJ_DIR = ./src
SRC_DIR = ./src
INCLUDE_DIR_REPO = ./inc

#build ses library
SRC_MODULES = Entry.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))
LIB_ST = $(LIB_DIR)/libRepo.a

DEBUG = -g
CFLAGS = -Wall -c $(DEBUG) -I $(INCLUDE_DIR_REPO)

$(LIB_ST): $(OBJ_FILES)
	ar -cru $(LIB_ST) $(OBJ_FILES)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)
	rm -f $(LIB_DIR)/*

Makefile_Code:

Code:

LIB_DIR = ../common/lib
OBJ_DIR = ./src
SRC_DIR = ./src
INCLUDE_DIR_CODE = ./inc
INCLUDE_DIR_SES = ../common/inc
INCLUDE_DIR_BOOST = ../../third_party

#build ses library
SRC_MODULES = Ses.cpp RegisterPlugin.cpp HeaderPrs.cpp Prs.cpp StdPrs.cpp PrsBase.cpp Js.cpp
OBJ_MODULES = $(patsubst %.cpp, %.o, ${SRC_MODULES})

SRC_FILES = $(addprefix $(SRC_DIR)/, $(SRC_MODULES))
OBJ_FILES = $(addprefix $(OBJ_DIR)/, $(OBJ_MODULES))

LIBS = -lRepo
LIBS += -lNsConf
LIBS += -lSes
LIBS += -lclassreg
LIBS += -lstringtokenizer
LIBS += -lz

LIB_ST = libh.so

DEBUG = -g
CFLAGS = -Wall -fPIC -c $(DEBUG) -I $(INCLUDE_DIR_CODE) -I $(INCLUDE_DIR_SES) -I $(INCLUDE_DIR_BOOST)
LFLAGS = -shared -Wl

$(LIB_ST): $(OBJ_FILES)
	@echo "Creating $(LIB_ST)"
	gcc $(LFLAGS),-soname,$(LIB_ST).1 -o $(LIB_ST).1.0 $(OBJ_FILES) -L$(LIB_DIR) -L../../common/lib -L../../conf/configLib/lib $(LIBS)

$(OBJ_DIR)/%.o:$(SRC_DIR)/%.cpp
	gcc ${CFLAGS} -o $@ $<

clean:
	rm -f $(OBJ_FILES)

For example, when I modify PrsBase.cpp, which is built in Makefile_Code, I CAN observe from the output of 'make' that the modifications to PrsBase.cpp are detected, and PrsBase.cpp is re-built. However, when I execute the application which loads libh.so plugin, which is built in Makefile_Code, I can not see the expected output from PrsBase.cpp. Somehow, the changes do not go into libh.so in the line "gcc $(LFLAGS),-soname,$(LIB_ST).1 -o $(LIB_ST).1.0 $(OBJ_FILES) -L$(LIB_DIR) -L../../common/lib -L../../conf/configLib/lib $(LIBS)" in Makefile_Code.

theNbomr · 01-11-2010, 12:23 PM

So, then it looks like a possible Makefile problem. When you invoke the Makefile, do you see the commandline that builds the object library? Does it run cleanly (no errors or warning messages emitted)?
Another possibility is a linking loader issue. The line you pointed out only rebuilds a shared object library. Does the runtime executable actually load that new version of the library, or does the new library need to be copied to somewhere that the linking loader will be able to find it? LD_LIBRARY_PATH or ldconfig may be needed to get the new version correctly loaded.
--- rod.

aryan1 · 01-12-2010, 03:02 AM

Quote:

Originally Posted by theNbomr

Another possibility is a linking loader issue. The line you pointed out only rebuilds a shared object library. Does the runtime executable actually load that new version of the library, or does the new library need to be copied to somewhere that the linking loader will be able to find it? LD_LIBRARY_PATH or ldconfig may be needed to get the new version correctly loaded.
--- rod.

I finally resolved the problem.

First of all, the problem had nothing to do with Makefiles.

The machine where these source files are compiled using 'make' had two separate users on it both of whom were developing software for the same project.

In our case, all binaries are invoked via a shell script with root priviliges, which, right at the beginning, checks /etc/ld.so.conf file to see if $HOME/project_folder/plugin_libs line exists. If not, it adds it into /etc/ld.so.conf, and run 'ldconfig'.

Since first user had already added $HOME/project_folder/plugin_libs line to ld.so.conf when the second user checks the existence of $HOME/project_folder/plugin_libs line with a different $HOME value, another $HOME/project_folder/plugin_libs line with a different $HOME value
is added to ld.so.conf.

Since the first appearing line in ld.so.conf has presedence over the the line coming after it, the plugins from the other user's project folder kept being loaded - the modified ones were ignored.