LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
Reload this Page [cowrie - elastic stack] filebeat trying to send logs to estack server - server replies with reset.
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 02-09-2018, 04:48 AM   #1
//////
Member
 
Registered: Nov 2005
Location: Land of Linux :: Finland
Distribution: Arch Linux && OpenBSD 7.4 && Pop!_OS && Kali && Qubes-Os
Posts: 824

Rep: Reputation: 350Reputation: 350Reputation: 350Reputation: 350
[cowrie - elastic stack] filebeat trying to send logs to estack server - server replies with reset.

hello all.

ive got a little problem with my estack server ...

Code:
tcpdump -vvv tcp port 5045
tcpdump: listening on enp0s31f6, link-type EN10MB (Ethernet), capture size 262144 bytes
12:03:10.931682 IP (tos 0x0, ttl 64, id 63945, offset 0, flags [DF], proto TCP (6), length 60)
    thug.home.44298 > fed.home.osp: Flags [S], cksum 0xa4e7 (correct), seq 1390210213, win 29200, options [mss 1460,sackOK,TS val 1893476962 ecr 0,nop,wscale 7], length 0
12:03:10.931713 IP (tos 0x0, ttl 64, id 51973, offset 0, flags [DF], proto TCP (6), length 40)
    fed.home.osp > thug.home.44298: Flags [R.], cksum 0x1a06 (correct), seq 0, ack 1390210214, win 0, length 0
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
... as you can see fed.home is replying with reset to the syn of thug.home.

ive got arno iptables script generated firewall, and i have made these two custom rules to accept filebeat boxes ( thug.home ) connections.
Code:
iptables -A INPUT -p tcp -s 192.168.10.33 -d 192.168.10.22 --dport 5045 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -p tcp -s 192.168.10.22 --sport 5045 -d 192.168.10.33 -m  conntrack --ctstate ESTABLISHED -j ACCEPT
pings are ok.

Code:
ping -c 3 192.168.10.33
PING 192.168.10.33 (192.168.10.33) 56(84) bytes of data.
64 bytes from 192.168.10.33: icmp_seq=1 ttl=64 time=0.491 ms
64 bytes from 192.168.10.33: icmp_seq=2 ttl=64 time=0.384 ms
64 bytes from 192.168.10.33: icmp_seq=3 ttl=64 time=0.393 ms

--- 192.168.10.33 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2079ms
rtt min/avg/max/mdev = 0.384/0.422/0.491/0.053 ms
here is nmap output from fed.home:
Code:
nmap -sA -A -sV 192.168.10.33 -p 22,23

Starting Nmap 7.60 ( https://nmap.org ) at 2018-02-09 12:40 EET
Nmap scan report for thug.home (192.168.10.33)
Host is up (0.00033s latency).

PORT   STATE      SERVICE VERSION
22/tcp unfiltered ssh
23/tcp unfiltered telnet
MAC Address: 00:22:15:04:D3:6E (Asustek Computer)
Warning: OSScan results may be unreliable because we could not find at least 1 open and 1 closed port
Device type: general purpose
Running: Apple Mac OS X 10.7.X, FreeBSD 8.X
OS CPE: cpe:/o:apple:mac_os_x:10.7.4 cpe:/o:freebsd:freebsd:8.0
OS details: Apple Mac OS X 10.7.4 (Lion) (Darwin 11.4.0), FreeBSD 8.0-CURRENT
Network Distance: 1 hop

TRACEROUTE
HOP RTT     ADDRESS
1   0.33 ms thug.home (192.168.10.33)

OS and Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 3.38 seconds
here is tcpdump capture from thug.home:
Code:
tcpdump -vvv tcp port 5045
tcpdump: listening on enp2s0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:50:10.979015 IP (tos 0x0, ttl 64, id 19292, offset 0, flags [DF], proto TCP (6), length 60)
    thug.home.46236 > fed.home.osp: Flags [S], cksum 0x95b6 (incorrect -> 0x4d2d), seq 2241528375, win 29200, options [mss 1460,sackOK,TS val 1896296975 ecr 0,nop,wscale 7], length 0
12:50:10.979564 IP (tos 0x0, ttl 64, id 57265, offset 0, flags [DF], proto TCP (6), length 40)
    fed.home.osp > thug.home.46236: Flags [R.], cksum 0xf2d9 (correct), seq 2391754938, ack 2241528376, win 0, length 0
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
any clues to how fix it are appreciated.
ill post conf files if you need em.
Attached Thumbnails
Click image for larger version Name: net.png Views: 27 Size: 43.4 KB ID: 26960   Click image for larger version Name: yjPha6A.png Views: 25 Size: 53.6 KB ID: 26961  
Last edited by //////; 02-09-2018 at 04:59 AM.
 
Old 02-10-2018, 10:16 AM   #2
//////
Member
 
Registered: Nov 2005
Location: Land of Linux :: Finland
Distribution: Arch Linux && OpenBSD 7.4 && Pop!_OS && Kali && Qubes-Os
Posts: 824

Original Poster
Rep: Reputation: 350Reputation: 350Reputation: 350Reputation: 350
logstash-plain.log is filled with these errors:
Code:
# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space


[2018-02-10T18:14:21,078][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
[2018-02-10T18:14:28,325][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-02-10T18:14:28,328][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-02-10T18:14:28,448][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-02-10T18:14:28,472][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space


[2018-02-10T18:14:28,474][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
[2018-02-10T18:14:36,115][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-02-10T18:14:36,119][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-02-10T18:14:36,273][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2018-02-10T18:14:36,305][FATAL][logstash.runner          ] The given configuration is invalid. Reason: Expected one of #, input, filter, output at line 6, column 1 (byte 132) after ## JVM configuration

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space


[2018-02-10T18:14:36,307][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit
EDIT: it was YAML indentation bug, its fixed now.

but this is the error filebeat generates:
Code:
2018-02-10T19:01:00+02:00 INFO Non-zero metrics in the last 30s: beat.info.uptime.ms=30000 beat.memstats.gc_next=10334512 beat.memstats.memory_alloc=5982840 beat.memstats.memory_total=19083712 filebeat.harvester.open_files=3 filebeat.harvester.running=3 libbeat.config.module.running=0 libbeat.pipeline.clients=1 libbeat.pipeline.events.active=4117 registrar.states.current=3
2018-02-10T19:01:03+02:00 ERR  Failed to connect: dial tcp 192.168.10.22:5045: getsockopt: connection refused
telnet 192.168.10.22 5045 (connection to logstash server) produces error :

Code:
[root@thug filebeat]# telnet 192.168.10.22 5045
Trying 192.168.10.22...
telnet: connect to address 192.168.10.22: Connection refused
[root@thug filebeat]#
Last edited by //////; 02-10-2018 at 11:18 AM. Reason: fixed error.
 
Old 02-11-2018, 09:09 AM   #3
//////
Member
 
Registered: Nov 2005
Location: Land of Linux :: Finland
Distribution: Arch Linux && OpenBSD 7.4 && Pop!_OS && Kali && Qubes-Os
Posts: 824

Original Poster
Rep: Reputation: 350Reputation: 350Reputation: 350Reputation: 350
here is a tcpdump capture from fed.home ...

Code:
[root:/home/vile]# tcpdump -vvv tcp port 5045
tcpdump: listening on enp0s31f6, link-type EN10MB (Ethernet), capture size 262144 bytes
16:26:27.787768 IP (tos 0x0, ttl 64, id 58755, offset 0, flags [DF], proto TCP (6), length 1437)	thug.home.53072 > fed.home.osp: Flags [P.], cksum 0xe8c3 (correct), seq 843221664:843223049, ack 572675798, win 229, options [nop,nop,TS val 2938872302 ecr 1445023217],	length 1385
16:26:27.790167 IP (tos 0x0, ttl 64, id 18938, offset 0, flags [DF], proto TCP (6), length 58)		fed.home.osp > thug.home.53072: Flags [P.], cksum 0x95b4 (incorrect -> 0x91cf), seq 1:7, ack 1385, win 1452, options [nop,nop,TS val 1445024221 ecr 2938872302],			length 6
16:26:27.790519 IP (tos 0x0, ttl 64, id 58756, offset 0, flags [DF], proto TCP (6), length 52)		thug.home.53072 > fed.home.osp: Flags [.], cksum 0xc8e4 (correct), seq 1385, ack 7, win 229, options [nop,nop,TS val 2938872305 ecr 1445024221],							length 0

16:26:28.791038 IP (tos 0x0, ttl 64, id 58757, offset 0, flags [DF], proto TCP (6), length 1094)	thug.home.53072 > fed.home.osp: Flags [P.], cksum 0x75ba (correct), seq 1385:2427, ack 7, win 229, options [nop,nop,TS val 2938873305 ecr 1445024221],						length 1042
16:26:28.792838 IP (tos 0x0, ttl 64, id 18939, offset 0, flags [DF], proto TCP (6), length 58)		fed.home.osp > thug.home.53072: Flags [P.], cksum 0x95b4 (incorrect -> 0x85e5), seq 7:13, ack 2427, win 1452, options [nop,nop,TS val 1445025223 ecr 2938873305],			length 6
16:26:28.793172 IP (tos 0x0, ttl 64, id 58758, offset 0, flags [DF], proto TCP (6), length 52)		thug.home.53072 > fed.home.osp: Flags [.], cksum 0xbcf7 (correct), seq 2427, ack 13, win 229, options [nop,nop,TS val 2938873308 ecr 1445025223],							length 0

16:26:32.787229 IP (tos 0x0, ttl 64, id 58759, offset 0, flags [DF], proto TCP (6), length 1300)	thug.home.53072 > fed.home.osp: Flags [P.], cksum 0x6ad2 (correct), seq 2427:3675, ack 13, win 229, options [nop,nop,TS val 2938877302 ecr 1445025223],						length 1248
16:26:32.789130 IP (tos 0x0, ttl 64, id 18940, offset 0, flags [DF], proto TCP (6), length 58)		fed.home.osp > thug.home.53072: Flags [P.], cksum 0x95b4 (incorrect -> 0x61c4), seq 13:19, ack 3675, win 1452, options [nop,nop,TS val 1445029220 ecr 2938877302],			length 6
16:26:32.789487 IP (tos 0x0, ttl 64, id 58760, offset 0, flags [DF], proto TCP (6), length 52)		thug.home.53072 > fed.home.osp: Flags [.], cksum 0x98d8 (correct), seq 3675, ack 19, win 229, options [nop,nop,TS val 2938877304 ecr 1445029220],							length 0
^C
9 packets captured
9 packets received by filter
0 packets dropped by kernel
[root:/home/vile]#
... no resets on sight and packets have data on em, just couple of checksum errors.

Code:
[2018-02-11T15:51:08,235][WARN ][logstash.runner          ] SIGTERM received. Shutting down.
[2018-02-11T15:51:13,269][WARN ][logstash.shutdownwatcher ] {"inflight_count"=>0, "stalling_thread_info"=>{"other"=>[{"thread_id"=>36, "name"=>"[main]<beats", "current_call"=>"[...]/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:199:in `run'"}, {"thread_id"=>27, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>28, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>29, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>30, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>31, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>32, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>33, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}, {"thread_id"=>34, "name"=>nil, "current_call"=>"[...]/logstash-core/lib/logstash/util/wrapped_synchronous_queue.rb:90:in `read_batch'"}]}}
[2018-02-11T15:51:13,272][ERROR][logstash.shutdownwatcher ] The shutdown process appears to be stalled due to busy or blocked plugins. Check the logs for more information.
[2018-02-11T15:51:15,271][INFO ][logstash.pipeline        ] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x7f30363@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[2018-02-11T15:51:22,374][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[2018-02-11T15:51:22,377][INFO ][logstash.modules.scaffold] Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[2018-02-11T15:51:22,549][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"6.2.0"}
[2018-02-11T15:51:22,578][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2018-02-11T15:51:23,043][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[2018-02-11T15:51:23,169][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://0.0.0.0:9200/]}}
[2018-02-11T15:51:23,170][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://0.0.0.0:9200/, :path=>"/"}
[2018-02-11T15:51:23,217][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://0.0.0.0:9200/"}
[2018-02-11T15:51:23,236][INFO ][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>nil}
[2018-02-11T15:51:23,236][WARN ][logstash.outputs.elasticsearch] Detected a 6.x and above cluster: the `type` event field won't be used to determine the document _type {:es_version=>6}
[2018-02-11T15:51:23,237][INFO ][logstash.outputs.elasticsearch] New Elasticsearch output {:class=>"LogStash::Outputs::ElasticSearch", :hosts=>["//0.0.0.0:9200"]}
[2018-02-11T15:51:23,238][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/opt/logstash/vendor/geoip/GeoLite2-City.mmdb"}
[2018-02-11T15:51:23,248][INFO ][logstash.filters.geoip   ] Using geoip database {:path=>"/opt/logstash/vendor/geoip/GeoLite2-ASN.mmdb"}
[2018-02-11T15:51:23,438][INFO ][logstash.inputs.beats    ] Beats inputs: Starting input listener {:address=>"0.0.0.0:5045"}
[2018-02-11T15:51:23,465][INFO ][org.logstash.beats.Server] Starting server on port: 5045
[2018-02-11T15:51:23,468][INFO ][logstash.pipeline        ] Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x3dec01bb@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 sleep>"}
[2018-02-11T15:51:23,472][INFO ][logstash.agent           ] Pipelines running {:count=>1, :pipelines=>["main"]}
[2018-02-11T15:51:28,255][INFO ][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[http://0.0.0.0:9200/], :added=>[http://192.168.10.22:9200/]}}
[2018-02-11T15:51:28,257][INFO ][logstash.outputs.elasticsearch] Running health check to see if an Elasticsearch connection is working {:healthcheck_url=>http://192.168.10.22:9200/, :path=>"/"}
[2018-02-11T15:51:28,265][WARN ][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://192.168.10.22:9200/"}
atleast there are no terminal errors, just warns.

Code:
[root:/home/vile]# curl -XGET '192.168.10.22:9200/cowrie?pretty'
{
  "cowrie" : {
    "aliases" : { },
    "mappings" : { },
    "settings" : {
      "index" : {
        "creation_date" : "1518024972229",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "Z0MWusQkSo-y2CBaKr_U0A",
        "version" : {
          "created" : "6010399"
        },
        "provided_name" : "cowrie"
      }
    }
  }
}
[root:/home/vile]#
i am unsure how to load data to cowrie. i think it should be easy, atleast i got servers / clients configured and there are no errors in the logs.
Last edited by //////; 02-11-2018 at 09:18 AM.
 
Old 02-12-2018, 05:02 AM   #4
goldprogrammer
LQ Newbie
 
Registered: Feb 2018
Posts: 2

Rep: Reputation: Disabled
have the same problem.
 
  


Reply

Tags
cowrie, elastic stack, filebeat, kibana



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: How to Install Elastic Stack on CentOS 7 LXer Syndicated Linux News 0 01-20-2017 08:42 PM
How to send email alerts when someone logs onto server? tearsforhari Linux - Newbie 13 07-17-2015 06:20 PM
Enable Audit logs to send logs to syslog-ng (remote server) Iyyappan Linux - Server 5 01-07-2014 04:15 PM
What is there a way to send Linux application logs to a LogRythm server? roger.price Red Hat 1 09-24-2013 01:37 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 05:40 PM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration