Web logging centralized


I think we all have this issue :

  • how to centralize web log from nginx / apache ?
  • And how to use this log ?

I activate Tools>logging on Rancher node and I get my log in ElasticSearch. That nice, but not exploitable because nginx log are in single field but I need to filter on nginx data that are in this field like IP adresse or http code status.

Ex :
88.126.125.XX - - [03/Apr/2020:23:38:54 +0000] “GET /search?filter%5Bgenre%5D%5B0%5D=UC_GENRE_GUERRE HTTP/1.1” 200 34472 “https://xxxxx.com/films/sympathie-pour-le-diable” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.149 Safari/537.36”

All information is in a sinle field “log” and I can’t filter with native ElasticSearch function.

So I’m thinking about logstash but I’m sure it should be simplier way to get log directly in ElasticSeach when activating logging functionality in Rancher. Web applications are for sure the most app in rancher.

Is there a simple way to have structured log in ElasticSearch when using Rancher logging panel ?