Haproxy ,memcached ,mysql strange problem

Eazy-Snatch · 05-13-2008, 02:42 AM

The situation is this.
Gentoo servers 2.6.22 kernels here we use apache1.3 php4.4.4 ######
server1 --- Haproxy + web
server2 --- Web
server3 --- Memcached
server4 --- mysql
###### Debian servers 2.6.24 kernel apache2 php5 ###
New server --- Web

#######################################

With haproxy we make server 1 2 to work with load balancing web applications now we want to upgrade our hardware and I make the new one server but when I put it in Haproxy config file after 10-15 min Mysql get 12-15 load.I make on new server apache2 + php5 then I downgrade to apache 1.3 with php4.4.4 then I recompile memcached clinet.

Here the haproxy.conf

Code:

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        #debug
        #quiet
        user nobody
        group nobody

defaults
        mode    http
        option  httplog
        option  dontlognull
        #2x exp
        option          abortonclose
        option          redispatch
        retries 10
        maxconn 2048
        contimeout      5000
        clitimeout      50000
        srvtimeout      50000

listen Ferm IP:80
        mode http
        stats enable
        stats auth user:passs1sds
        stats hide-version
        stats scope   .
        stats uri     /load?stats
        stats realm   Statistics
        stats refresh 5s
        balance roundrobin
#       cookie SID prefix
        cookie JSESSIONID prefix
#       cookie SERVERID insert indirect
        option httpclose
        option forwardfor
#       option persist
        option httpchk HEAD http://x.x.x.x/check.txt HTTP/1.0
        server a    server3 cookie D check inter 10000 backup
#        server b  Newserver   cookie C weight 50 minconn 10 maxconn 600 check inter 1000
        server c server1     cookie A weight  10 minconn 40 maxconn 600 check inter 1000
        server d  server2   cookie B weight  20 minconn 30 maxconn 600 check inter 1000
        redispatch

Memcached start with :

Code:

/usr/bin/memcached -d -p 11211 -l x.x.x.x -m 4096 -c 20480 -u memcached -P /var/run/memcached/memcached-11211.pid

Then when I uncoment Newserver after some time our mysql return too many connections.

Our theory is that memcached crash some way then when some user search something in the site he don't get data from memcached and search directly in Mysql for that our DB server get this load but we can't debug it.

Some ideas ...

Greetings

Eazy-Snatch · 05-14-2008, 02:54 AM

We found new errors in Memcached server

Code:

<8 set fail1210688324 0 0 7
<7 set fail1210688327 0 0 7
<11 set fail1210688335 0 0 7
<11 set fail1210688338 0 0 7
<9 set fail1210688345 0 0 7