Hello,
we're running a Magento 2 Shop on a VPS.
Debian 8, 4 cores, 4GB RAM, 50GB HDD (I believe), Nginx, MySQL, PHP7
According to Google analytics, we do have around 30 - 50 visitors online each day and very few orders.
I believe this traffic might be coming from bots, spiders or similar.
Now I've tried already several things like:
NGinx VHost config:
Code:
limit_req_zone $binary_remote_addr zone=mylimit:10m rate=2r/s;
if ($http_user_agent ~* (360Spider|....|ZmEu|ZumBot|ZyBorg) )
{
return 410;
}
But that is obviously not a great solution.
What might be a good thing to do?
I was trying to install Varnish - but that would probably result into supplying bots with our website faster.
Things like bot traps seem to simply add IP addresses to an ever growing file with IP addresses.
Increasing our server CPU / RAM is probably an option .. but spending more money because of useless traffic sounds rather stupid.
It's probably possible to optimize some MySQL setting - but that won't solve the problem, just postpone it.
Here is the process list from before the server was suspended - maybe someone can tell something?
Code:
=================================================================================================================================================================================
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
0 1030474 0.0 0.0 0 0 ? S Oct31 0:00 \_ [kthreadd/6493]
0 1030475 0.0 0.0 0 0 ? S Oct31 0:00 | \_ [khelper/6493]
0 1030400 0.0 0.0 28520 1888 ? Ss Oct31 0:01 init -z
0 1031796 0.0 0.0 38872 604 ? Ss Oct31 0:00 \_ /lib/systemd/systemd-udevd
0 1031804 0.0 0.0 32912 1760 ? Ss Oct31 0:00 \_ /lib/systemd/systemd-journald
0 1032196 0.0 0.0 25848 484 ? Ss Oct31 0:00 \_ /usr/sbin/cron -f
0 136404 0.0 0.0 40584 760 ? S 11:09 0:00 | \_ /usr/sbin/CRON -f
0 137071 0.0 0.0 4280 340 ? Ss 11:09 0:00 | | \_ /bin/sh -c [ -x /usr/lib/php/sessionclean ] && /usr/lib/php/sessionclean
0 137289 0.0 0.0 4280 356 ? S 11:09 0:00 | | \_ /bin/sh -e /usr/lib/php/sessionclean
0 137291 0.0 0.0 4280 300 ? S 11:09 0:00 | | \_ /bin/sh -e /usr/lib/php/sessionclean
0 152893 0.0 0.0 239912 7580 ? D 11:14 0:00 | | | \_ php7.0 -c /etc/php/7.0/cli/php.ini -d error_reporting='~E_ALL' -r foreach(ini_get_all("session") as $k => $v) echo "$k=".$v["local_value"]."\n";
0 137492 0.0 0.0 18840 400 ? S 11:09 0:00 | | \_ sort -rn -t: -k2,2
0 137493 0.0 0.0 18840 396 ? S 11:09 0:00 | | \_ sort -u -t: -k 1,1
0 137494 0.0 0.0 4280 200 ? S 11:09 0:00 | | \_ /bin/sh -e /usr/lib/php/sessionclean
0 139190 0.0 0.0 40584 760 ? S 11:10 0:00 | \_ /usr/sbin/CRON -f
1000 139794 0.0 0.0 4280 332 ? Ss 11:10 0:00 | \_ /bin/sh -c /usr/bin/php /var/www/html/website.com/www.website.com/bin/magento cron:run | grep -v "Ran jobs by schedule" >> /var/www/html/website.com/www.website.com/var/log/magento.cron.log
1000 140104 0.0 0.0 244344 9876 ? D 11:10 0:00 | \_ /usr/bin/php /var/www/html/website.com/www.website.com/bin/magento cron:run
1000 140331 0.0 0.0 11072 392 ? S 11:10 0:00 | \_ grep -v Ran jobs by schedule
102 1032199 0.0 0.0 25692 336 ? Ss Oct31 0:00 \_ /lib/systemd/systemd-resolved
0 1032200 0.0 0.0 186904 848 ? Ssl Oct31 0:00 \_ /usr/sbin/rsyslogd -n
0 1032221 0.0 0.0 12608 268 tty2 Ss+ Oct31 0:00 \_ /sbin/agetty --noclear tty2 linux
0 1032225 0.0 0.0 12608 276 tty1 Ss+ Oct31 0:00 \_ /sbin/agetty --noclear --keep-baud console 115200 38400 9600 vt102
0 1032312 0.0 0.0 55132 852 ? Ss Oct31 0:00 \_ /usr/sbin/sshd -D
105 1032360 2.8 0.3 2693860 526548 ? Sl Oct31 32:55 \_ /usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid
0 1032463 0.0 0.0 318036 13976 ? Ss Oct31 0:02 \_ php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
33 987874 1.6 0.0 474524 99592 ? D 10:15 0:59 | \_ php-fpm: pool www
33 1016693 1.4 0.0 354096 60836 ? S 10:24 0:43 | \_ php-fpm: pool www
33 1016743 1.3 0.0 374520 78768 ? D 10:24 0:41 | \_ php-fpm: pool www
33 1016744 1.4 0.0 453872 81844 ? D 10:24 0:44 | \_ php-fpm: pool www
33 21539 1.2 0.0 361392 69844 ? D 10:37 0:28 | \_ php-fpm: pool www
33 21974 1.1 0.0 376620 85500 ? D 10:38 0:25 | \_ php-fpm: pool www
33 23073 1.2 0.0 379480 88980 ? D 10:38 0:27 | \_ php-fpm: pool www
33 26419 1.4 0.0 448912 79940 ? D 10:39 0:29 | \_ php-fpm: pool www
33 28036 1.0 0.0 347384 58272 ? D 10:40 0:22 | \_ php-fpm: pool www
33 74470 1.2 0.0 377432 88124 ? D 10:51 0:17 | \_ php-fpm: pool www
33 74501 1.0 0.0 371108 80508 ? D 10:51 0:14 | \_ php-fpm: pool www
33 74672 1.0 0.0 377524 88432 ? D 10:51 0:15 | \_ php-fpm: pool www
33 74693 0.9 0.0 394020 102048 ? D 10:51 0:13 | \_ php-fpm: pool www
33 91402 1.1 0.0 388964 98756 ? D 10:55 0:13 | \_ php-fpm: pool www
33 91445 1.2 0.0 386976 98340 ? S 10:55 0:13 | \_ php-fpm: pool www
33 116460 0.4 0.0 358264 68300 ? D 11:02 0:03 | \_ php-fpm: pool www
33 120928 0.2 0.0 432288 52632 ? D 11:04 0:01 | \_ php-fpm: pool www
33 120953 0.4 0.0 360416 74924 ? D 11:04 0:02 | \_ php-fpm: pool www
33 122171 0.8 0.0 366392 77856 ? S 11:05 0:05 | \_ php-fpm: pool www
33 122220 0.5 0.0 368504 80148 ? D 11:05 0:03 | \_ php-fpm: pool www
0 1032478 0.0 0.0 105008 2156 ? Ss Oct31 0:00 \_ nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
33 1032479 0.0 0.0 106396 5400 ? S Oct31 0:37 \_ nginx: worker process
33 1032480 0.0 0.0 106352 5404 ? S Oct31 0:34 \_ nginx: worker process
33 1032481 0.0 0.0 106332 5256 ? S Oct31 0:41 \_ nginx: worker process
33 1032482 0.0 0.0 106440 5368 ? S Oct31 0:36 \_ nginx: worker process
=================================================================================================================================================================================