thread debugging gdbserver gdb

Hello,
I am using the AVR32 AP7000 cpu from Atmel. My problem is that I am not able to debugging multi-threaded application on the target.

I am using buildroot to create images to the CPU. This buildroot is only a couple of weeks old. In this buildroot I have tried to enable for thread debugging support, and I have tried to used NPTL instead of linuxthread.

The target is then using gdbserver, and the host avr32-linux-gdb

avr32-linux-gdb is of version 6.7.1.

The problem is that the multi-threaded application works fine when not debugging it, just run it normaly. When I tries to debugging it, the debugger only exits with return code 0, even if it should not because I do use while(1); , breakpoints and printf in my code(Do not se any print on the screen). The termination of the program happens when using the pthread_creat() function.


Hope some have some answers.

Hi,

some weeks ago I read your post regarding multithreaded debugging issue on Atmel ATSTK1000 board. I have experienced the same behavior and want to ask you whether you have found any solution. I made a small progress but still have some malfanctions with GDBServer.
I am interested in solving this issue as well.

I have not found a solution...

Hi,

I have tried to solve this issue by
1.
- rebuilding the uClibc with following features set in the
/target/device/Atmel/uClibc.config.avr32

DODEBUG=y
DODEBUG_PT=y
PTHREADS_DEBUG_SUPPORT=y
2.
-rebuilding the GDB with following configure parameters
./configure --target=avr32-linux --with-sysroot=/home/src/buildroot_221/build_avr32/staging_dir/lib

3.
- rebuilding GDBServer with configure parameters

./configure --host=avr32-linux --prefix=/home/src/buildroot_221/build_avr32/staging_dir/lib/


After these steps my GDBServer doesn't exit with simple message while creating new thread but produces the following one
--------------------------------------------------------------------
/home/avr32 # gdbserver HOST:2345 main.elf
Process main.elf created; pid = 442
Listening on port 2345
Remote debugging from host 192.168.0.4
gdb: error initializing thread_db library: generic error
gdb: error initializing thread_db library: generic error
--------------------------------------------------------------------
while hitting the breakpoint at main() of the application that is going to launch another thread.

and
--------------------------------------------------------------------
Program received signal SIG32, Real-time event 32.
0x2aad0c18 in sigsuspend ()
from /home/src/buildroot_221/build_avr32/staging_dir/lib/lib/libc.so.0
--------------------------------------------------------------------
while creating new thread.

I don't know, yet what I do wrong but I am still going to spend some time and effort to find a solution.

I would appreciate if you share some of your experience with that.

I do think that SIG32 is a common problem. If I was you, I would tried to get some help from Atmel support. I do belive that this is a problem for many people. I think this is strange that it is not solved after so long time.

Hi,

I was able to manage this issue for both buildroots 2.2.1 and 2.3.0 and now multithreded debugging is working fine. The solution is quite trivial.
I will try to write explanation next week when I noly find free time slot for that.

Hi,

after doing the following steps I am able to debug multithreded applications and multiprocess ones spawned by fork():

1. In the first step I have removed all the Hardy packages for avr32 linux containing avr32-linux-gdb, avr32-linux-gcc, and all the sharable libraries. The goal is to use the toolchain and libraries from the avr32 buildroot (to be found in staging_dir/bin/ and staging_dir/lib/) and not to mix it up with default Hardy packages installed on your development system.

2. I modified the file /target/device/Atmel/uClibc.config.avr32
with the following entries

# HAS_NO_THREADS is not set
UCLIBC_HAS_THREADS=y
PTHREADS_DEBUG_SUPPORT=y
LINUXTHREADS_OLD=y

3. I enabled the support for multithreaded debugging in buildroot main config
--> make menuconfig
--> Toolchain
--> (*) Thread library debugging

4. In the last step for buildroot/toolchain configuration I modified the file /buildroot/toolchain/gdb/gdb.mk as follow

######################################################################
#
# gdb on host
#
######################################################################

GDB_HOST_DIR:=$(TOOL_BUILD_DIR)/gdbhost-$(GDB_VERSION)

$(GDB_HOST_DIR)/.configured: $(GDB_DIR)/.unpacked
mkdir -p $(GDB_HOST_DIR)
(cd $(GDB_HOST_DIR); \
gdb_cv_func_sigsetjmp=yes \
bash_cv_have_mbstate_t=yes \
$(GDB_DIR)/configure \
--cache-file=/dev/null \
--prefix=$(STAGING_DIR) \
--with-sysroot=$(STAGING_DIR)/lib \
--build=$(GNU_HOST_NAME) \
--host=$(GNU_HOST_NAME) \
--target=$(REAL_GNU_TARGET_NAME) \
$(DISABLE_NLS) \
--without-uiout $(DISABLE_GDBMI) \
--disable-tui --disable-gdbtk --without-x \
--without-included-gettext \
--enable-threads \
--disable-werror \
)

The new entry "--with-sysroot=$(STAGING_DIR)/lib \"
instructs avr32-linux-gdb where to find the sharable libraries and so on like libthread_db etc.

5. After these modification I built the whole system from the buildroot main directory --> make.

6. After the build process is completed I deployed the avr32 linux to my NFS directory on my Ubuntu machine.

7. The last step to prepare toolchain was to make in the development machine in /usr/local/bin a symbolic link to the avr32-linux-gdb residing in the staging_dir/bin. This link is used by AVR32 Studio to start avr32-linux-gdb when the debug session is launched.

8. The multithreaded application has to be now compiled with the avr32 linux toolchain from /staging_dir/bin and linked against the sharable libraries from /staging_dir/lib.


9. Run the application and set the breakpoint in the code executed by the first created thread.
Now it should be possible to debug in single step mode all the created threads, debug one thread while others are running, stop the running thread and switch between them as you will.

The debug session can be launched using AVR32 Studio or with avr32-linux-gdb as command line tool.

10. For debugging child process in "multiprocess application" put a call to sleep() function in the code where the child process executes after fork().
Run the application on the target system and check the process id by issuing ps command and then using gdbserver attach to the child process.
eg.
gdbserver localhost:2345 --attach 460

Connect to your gdbserver from avr32-linux-gdb and from now on the single step debugging of the child process is possible.


All the steps described above I applied with success to both buildroots 2.2.1 and 2.3.0.

The attached files show how it looks like while using AVR32 Studio and command line avr32-linux-gdb.