Access log include double quote at the user-agent field

[Pjack]

Hi

I use some tool to do penetration test and see if modsecurity v3 will block the request successful.
In some testing case, user-agent includes double-quote in it, ie:

int2.diamond.trendmicro.com 118.163.170.73 - - [24/May/2017:07:38:35 +0000] "POST /api/v1/logs/ HTTP/1.1" 403 0- "'"<script>alert(1);</script>" 14956115151.000000 - /var/log/modsec/20170524/20170524-0738/20170524-073835-14956115151.000000 0 9454.000000 md5:d87287f684b5e55193b7402d421835bd

Modsecurity blocked the request succesfully, however, it will output the origin user-agent string into the access log too.
It cause our log parser (fluentd + elastisearch) failed in this case, and we don't have good solution for it.
I would like to support libmodsecurity can convert the double-quote as \x22 , so does other character between 0x7F-0x1F. I see this idea in nginx's access log.

https://serverfault.com/questions/584881/parseable-nginx-accesslog-files-with-delimiters

Thanks!

[zimmerle]

Hi @Pjack, Is this the nginx log or the audit log? For the auditlogs you can use straight the JSON. If that is on nginx logs, we may have a bug.

[Pjack]

Hi @zimmerle ,

It is the index of audit log, sorry that I did not use precise wording.

The log is generated by std::string Transaction::toOldAuditLogFormatIndex
https://github.com/SpiderLabs/ModSecurity/blob/v3/master/src/transaction.cc#L1318

We hopes to parse it into Elasticsearch server.

Thanks!

[victorhora]

@Pjack Can you confirm if the issue also happens if you enable JSON audit logs? (SecAuditLogFormat)

[Pjack]

Hi @victorhora ,

SecAuditLogFormat is invalid in v3.
Here is my settings,

SecAuditEngine RelevantOnly
SecAuditLogType Parallel
SecAuditEngine RelevantOnly
SecAuditLogParts ABIJDEFHZ
SecAuditLog /var/log/modsec/audit.log
SecAuditLogStorageDir /var/log/modsec
SecAuditLogRelevantStatus "^(?:5|4(?!04))"

Transaction::toOldAuditLogFormatIndex is only invoked under parallel mode.
https://github.com/SpiderLabs/ModSecurity/blob/v3/master/src/audit_log/writer/parallel.cc#L153

[victorhora]

Ah right you're using v3 @Pjack ! My bad :)

Well, afaik, the only way I can think of now to avoid plain double quotes for going to the logs would be using the sanitize directives but I'm not sure if it would work well for you and these aren't currently implemented on v3 so this might be needed to be implemented as a new feature altogether. Unless I'm missing something and someone else comes up with a better idea :)

[Pjack]

Hi @victorhora ,

Sorry that I don't know what is sanitize directive that I can't answer you.
I just know how nginx handle this situation, FYI.
https://serverfault.com/questions/584881/parseable-nginx-accesslog-files-with-delimiters

Thanks for your response and please consider to implement it in near future.

[victorhora]

Support for AuditLogFormat was added as of 63bef3d. Enabling JSON audit logging should solve this issue.