New db-parser format and other improvements

I was lagging behind my schedule very much, but due to Bazsi's pressure I finally took the time to backport some db-parser related changes from syslog-ng premium edition to the open-source edition. As Bazsi's stated in his last blog post, he introduced a new release model for syslog-ng and he also published a roadmap for the next releases. The next release features my tag support patches and the db-parser improvements.

I have added new parsers to the db-parser:

• IPv6 to parser IPv6 addresses with all the special notation styles
  
• IPvANY to match either IPv4 or IPv6 addresses
  
• NUMBER parser now handles hexadecimal numbers with the 0x or 0X prefix
  
• DOUBLE parser to match and parse non integers
  

Also I introduced patterndb format version 2, which is more flexible and hopefully easier to use. Meanwhile I removed support for version 1 format. Converting between the formats is very straight forward, but I also plan to add an XSLT for conversions.

The new format has support and flexibility for couple of interesting things:

• rules for one program can be separated into multiple rulesets
  
• rulesets can be defined without program pattern where these act as a fail-back rule sets if no program pattern is matching or when message does not have program part
  
• one rule can have any number of pattern attached to it (sometimes the same message has multiple format, just think about multilingual messages)
  
• tags can be attached to rules (IMPORTANT: as of now tags are not considered, but at least the format has place for them!)
  

The new patterndb format looks like this:

<patterndb version="2" pub_date="2009-06-04">
 <ruleset name="testset" id="4182799f-9d1e-48a2-aa1e-aa4bdac6aff8">
  <rules>
   <rule provider="balabit" id="4a3b027a-3c5c-4c55-a28b-ce8052e8a327" class="system">
     <tags>
      <tag>login</tag>
    </tags>
    <patterns>
     <pattern>HERE COMES THE PATTERN</pattern>
    </patterns>
   </rule>
  </rules>
</patterndb>

I am trying to find some time in the near future to add the missing tag support and provide some more details on the new format.