[Novalug] Looking for sample system and event logs..

[Jay Hart]

Scott,

Will syslog not work for you?

I once wrote a script that would parse the logfiles of the different levels of
logging (crit, kernel, emerg, etc) that you can separate.

For example, say I get a entry in the emerg logfile, well, that same entry
also gets "logged" in all the lower level files, so you can diff out the files
and see what is going on.

Does this make any sense to you?  I could dig up the old talk I once gave
about this, if you need it.

Jay

>
> Hi,
>
> Sorry if this an odd request, and I hope it's not inappropriate, but I'm
> looking at developing a real-time on-line logfile anomaly detection
> engine, and am hoping that some list members might be kind enough to
> provide me with some samples of their logs. I can't develop the
> algorithms without having typical logs to work with, and our own
> environment just isn't that complex, and so I'm hoping for a wider
> variety of activity characteristics to develop and evaluate against.
>
> Any system, web, or application logs you are willing to provide would
> work fine (I'll even take Windoze if its offered..), so long as you can
> provide at least a few 100k records that would be "representative" of
> normal on your system. More ideally, disjoint samples of the same log
> from different timeframes (i.e. a week or month apart) would be perfect.
> I'm willing to sign an NDA if you're worried about disclosing private
> information, or we can talk offline about how you could make your logs
> anonymous before providing them.
>
> Even if you don't have logs to offer, if you're interested in trying the
> thing out when it's done let me know. The real trick on my end is going
> to be to account for the differences in the way log entries cluster
> without forcing the user to be a machine learning expert to operate the
> tool.
>
> Thanks for listening, and I hope someone can help out
>
> 	-- Scott
>
> _______________________________________________
> Novalug mailing list
> Novalug at calypso.tux.org
> http://calypso.tux.org/cgi-bin/mailman/listinfo/novalug
>