Ability to injest Syslog and other Log formats
in progress
J
James Stull
With the SIEM, I would really like to see the ability to injest logs from syslog sources and from various devices. Such as Ubiquiti, pfSense, fortigate, and other firewalls/switches. Also other devices would be good such as printers and IoT devices.
While these can generate a lot of noise, if we know how to best configure we can filter the noise out prior to shipping them to you.
Chris Bisnett
in progress
We've started work on Syslog ingestion. This will allow us to ingest logs from any device or application that can send Syslog formatted event data to a Syslog collector. Initially we will add this functionality into our Huntress agent to collect these logs from the local network. In the future we may also add publicly accessible Syslog collection endpoints for setups that have the ability to support encryption and where it is easier to not have to run a local agent for collection.
We will parse this data into the standard ECS format just like we do for Windows Event logs. This way there will be common fields that can be searched and will return different types of log data when the field matches.
We are looking for folks who are interested to help us test this in the next week or two.
J
James Stull
Chris Bisnett any perticular devices/brands you are looking to start with?
Chris Bisnett
planned
Ingesting Syslog data is planned to start in the next week or two. While this seems pretty easy on the surface, when you dig in you quickly realize that all of the different network devices and systems that can send logs to a Syslog endpoint all do it differently and support various configuration settings, it quickly becomes clear this will initially support some systems and will iterate over the next few months.
Obviously dropping as much of the noisy data at the source is the best option, but in cases where that can't be setup, we plan on being able to drop noisy data at the ingest site and this won't count against the log volume.
J
James Stull
Chris Bisnett Awesome plan.
Yup, no two log sources are the same. I'm not sure if you looked at it or not, but you may want give the open source project graylog a look at. They can ingest logs on just about anything. It may have some ways that can help shorten your timelines.
Honestly, what I think it the hardest part is going to be the filtering. I would think a lot of that will depend on us in some ways unless you can filter it out on a per vendor basis. But even then I bet a lot of huntress clients will just ship everything at you in order not to miss anything, I could see the filtering getting overwhelmed in larger deployments.
In short, this is not an easy project.