[Kea-users] kea-dhcp-ddns.conf log output format and %m

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[Kea-users] kea-dhcp-ddns.conf log output format and %m

Rick Dicaire
Hi, new to list and kea, using 1.6 from ISC's cloudsmith.io deb repo on
Ubuntu 18.04 LTS.
I have a working dhcp4 and ddns configuration.
I'm now trying to customize logging to output json for easy ingestion by
ELK stack. I'm working with dhcp-ddns first.
I'm using this pattern:

"pattern":
"{\"date\":\"%D{%Y-%m-%dT%H:%M:%S.%q%z}\",\"kea.ddns.log.level\":
\"%p\",\"kea.ddns.log.component\":\"%c\",\"kea.ddns.log.pid\":\"%i\",\"kea.ddns.log.data\":
\"%m\"}\n"

which seems to work *except* for the multiline output of %m (log message):

{"date":"2019-10-12T23:43:26.431-0400","kea.ddns.log.level":
"INFO","kea.ddns.log.component":"kea-dhcp-ddns.d2-to-dns","kea.ddns.log.pid":"21365","kea.ddns.log.data":
"DHCP_DDNS_ADD_SUCCEEDED DHCP_DDNS Request ID
0000017160126516693CF3917DFD4650B91CEB8437EC50975E886813C72F8606E30D7C:
successfully added the DNS mapping addition for this request: Type: 0
(CHG_ADD)
Forward Change: yes
Reverse Change: yes
FQDN: [ddns-192-168-17-105.dhcp.ldev.]
IP Address: [192.168.17.105]
DHCID:
[0000017160126516693CF3917DFD4650B91CEB8437EC50975E886813C72F8606E30D7C]
Lease Expires On: 20191013044326
Lease Length: 3600
"}

Single line outputs of %m are fine:
{"date":"2019-10-12T23:40:58.595-0400","kea.ddns.log.level":
"INFO","kea.ddns.log.component":"kea-dhcp-ddns.commands","kea.ddns.log.pid":"21365","kea.ddns.log.data":
"COMMAND_ACCEPTOR_START Starting to accept connections via unix domain
socket bound to /tmp/kea-dhcp-ddns-ctrl.sock"}
{"date":"2019-10-12T23:40:58.597-0400","kea.ddns.log.level":
"INFO","kea.ddns.log.component":"kea-dhcp-ddns.dctl","kea.ddns.log.pid":"21365","kea.ddns.log.data":
"DCTL_CONFIG_COMPLETE server has completed configuration: listening on
127.0.0.1, port 53001, using UDP"}
{"date":"2019-10-12T23:40:58.597-0400","kea.ddns.log.level":
"INFO","kea.ddns.log.component":"kea-dhcp-ddns.dhcpddns","kea.ddns.log.pid":"21365","kea.ddns.log.data":
"DHCP_DDNS_STARTED Kea DHCP-DDNS server version 1.6.0 started"}

As I understand json, multiline is not valid so I'm trying to figure out if
the format of %m can be modified to output as single line?

Thanks

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.

Kea-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/kea-users
Reply | Threaded
Open this post in threaded view
|

Re: [Kea-users] kea-dhcp-ddns.conf log output format and %m

Francis Dupont
Rick Dicaire writes:
> Hi, new to list and kea, using 1.6 from ISC's cloudsmith.io deb repo on
> Ubuntu 18.04 LTS.
> I have a working dhcp4 and ddns configuration.
> I'm now trying to customize logging to output json for easy ingestion by
> ELK stack. I'm working with dhcp-ddns first.
> I'm using this pattern:

=> the pattern is given to log4cplus with as reference:
https://log4cplus.sourceforge.io/docs/html/classlog4cplus_1_1PatternLayout.html

> "pattern":
> "{\"date\":\"%D{%Y-%m-%dT%H:%M:%S.%q%z}\",\"kea.ddns.log.level\":
> \"%p\",\"kea.ddns.log.component\":\"%c\",\"kea.ddns.log.pid\":\"%i\",\"kea.dd
> ns.log.data\":
> \"%m\"}\n"

=> hum, you assume here that anything in between two (escaped) quotes makes
a valid JSON string. Of course this is not true (reference here is ECMA 404)
and the multi-line issue is only one of the possible problems.

> As I understand json, multiline is not valid so I'm trying to figure out if
> the format of %m can be modified to output as single line?

=> log4cplus offers some formating but only a subset of printf so
nothing powerful enough. I am afraid you have to do the to JSON translation
on the other side i.e. in log files or (new)syslog.
 BTW there are already a lot of log filters so if you find a suitable one
please share here. I know for instance that Jenkins has a log file to XML
filter tool...
 I am looking for direct solution in log4j (log4cplus was designed from this).
There are some ideas using %m but we know it is not enough. Some suggest
to write a JSON Appender, can be done with not trivial coding...
(reference https://github.com/michaeltandy/log4j-json, I found more but
for log4j2).
 I can see 3 problems to do this in Kea:
 - there is no hook in Kea for logging i.e. no easy place to insert code
 - the JSON code is in another and later library (backward dependency)
 - it requires significant manpower to develop.

Regards

Francis Dupont <[hidden email]>
_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.

Kea-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/kea-users
Reply | Threaded
Open this post in threaded view
|

Re: [Kea-users] kea-dhcp-ddns.conf log output format and %m

Rick Dicaire


On Sun, Aug 2, 2020 at 12:56 PM Francis Dupont <[hidden email]> wrote:

=> log4cplus offers some formating but only a subset of printf so
nothing powerful enough. I am afraid you have to do the to JSON translation
on the other side i.e. in log files or (new)syslog.


Francis, thanks for responding!

I don't have a JSON requirement, it was just the easiest way to parse for my logstash filtering.
I could work with straight log text, again if it were single line output.
kea-dhcp4 logging seems to use single line output for each event,
easy to json format it, or otherwise parse the single line output it produces.

Do you know what the rationale was to have kea-dhcp-ddns log output using multiline format?

Thanks

_______________________________________________
ISC funds the development of this software with paid support subscriptions. Contact us at https://www.isc.org/contact/ for more information.

To unsubscribe visit https://lists.isc.org/mailman/listinfo/kea-users.

Kea-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/kea-users