ioc_writer.managers package

Submodules

ioc_writer.managers.downgrade_11 module

downgrade_11.py from ioc_writer Created: 12/17/15

Purpose: Provide a single reference class for converting an OpenIOC 1.1 document to OpenIOC 1.0.

This downgrade process is lossy as there are conditions, parameters and link metadata which may be present in the 1.1 indicator that cannot be expressed in the 1.0 indicator. The data that is lost is detailed below:

Data that will be removed in the downgrade:

#.For items directly underneath the top-level Indicator node (OpenIOC/criteria/Indicator/@operator='OR'for a valid MIR IOC):

  1. Any IndicatorItems under the top which use the preserve-case attribute will be removed.
  2. Any IndicatorItems which use the conditions ‘begins-with’, ‘ends-with’, ‘greater-than’, ‘less-than’, or ‘matches’ will be removed.
  3. Any Indicator nodes which contains a IndicatorItem node underneath it which match the conditions described above in 1) & 2) will be removed.

#.Metadata:

  1. Any links which contain link/@href will lose the @href attribute.

  1. Parmeters:

    1. Any parmeters which point to a Indicator node will be removed.
    2. Any parmeters which point to a IndicatorItem node which do not have param/@name='comment‘ set will be removed.

  2. General:

    1. The published date, OpenIOC/@published-date, will be removed.

Usage example:

iocm = DowngradeManager()
iocm.insert(iocs_dir)
errors = iocm.convert_to_10()
output_dir = './iocs'
iocm.write_iocs(output_dir)
iocm.write_pruned_iocs(output_dir, iocm.pruned_11_iocs)
iocm.write_pruned_iocs(output_dir, iocm.null_pruned_iocs
exception ioc_writer.managers.downgrade_11.DowngradeError

Bases: ioc_writer.ioc_api.IOCParseError

Exception raised when there is an error in the conversion

class ioc_writer.managers.downgrade_11.DowngradeManager

Bases: ioc_writer.managers.IOCManager

Convert the OpenIOC 1.1 documents into a 1.0 format. The converts IOCs are stored in self.iocs_10. IOCs which would have all nodes removed from under their top-level OR would be added to self.null_pruned_iocs IOCs which have at least one node, but not all nodes, removed would be added to self.prunded_11_iocs.

convert_branch(old_node, new_node, ids_to_skip, comment_dict=None)

Recursively walk a indicator logic tree, starting from a Indicator node. Converts OpenIOC 1.1 Indicator/IndicatorItems to Openioc 1.0 and preserves order.

Parameters:
  • old_node – An Indicator node, which we walk down to convert
  • new_node – An Indicator node, which we add new IndicatorItem and Indicator nodes too
  • ids_to_skip – set of node @id values not to convert
  • comment_dict – maps ids to comment values. only applied to IndicatorItem nodes
Returns:

returns True upon completion.
Raises:

DowngradeError if there is a problem during the conversion.
convert_to_10()

converts the iocs in self.iocs from openioc 1.1 to openioc 1.0 format. the converted iocs are stored in the dictionary self.iocs_10 :return: A list of iocid values which had errors downgrading.

write_iocs(directory=None, source=None)

Serializes IOCs to a directory.

Parameters:
  • directory – Directory to write IOCs to. If not provided, the current working directory is used.
  • source – Dictionary contianing iocid -> IOC mapping. Defaults to self.iocs_10. This is not normally modifed by a user for this class.
Returns:
write_pruned_iocs(directory=None, pruned_source=None)

Writes IOCs to a directory that have been pruned of some or all IOCs.

Parameters:
  • directory – Directory to write IOCs to. If not provided, the current working directory is used.
  • pruned_source – Iterable containing a set of iocids. Defaults to self.iocs_10.
Returns:

ioc_writer.managers.upgrade_10 module

upgrade_10.py from ioc_writer Created: 12/17/15

Purpose: Provide a single reference class for converting OpenIOC 1.0 documents to 1.1.

This upgrade process is a non-lossy process, as all conditions that can be expressed in OpenIOC 1.0 can also be expressed in OpenIOC 1.1.

Usage Example:

iocm = UpgradeManager()
iocm.insert(iocs_dir)
iocm.convert_to_11()
output_dir = './iocs'
iocm.write_iocs(output_dir)
exception ioc_writer.managers.upgrade_10.UpgradeError

Bases: ioc_writer.ioc_api.IOCParseError

Exception raised when there is an error in the conversion

class ioc_writer.managers.upgrade_10.UpgradeManager

Bases: object

convert_branch(old_node, new_node, comment_dict=None)

recursively walk a indicator logic tree, starting from a Indicator node. converts OpenIOC 1.0 Indicator/IndicatorItems to Openioc 1.1 and preserves order.

Parameters:
  • old_node – Indicator node, which we walk down to convert
  • new_node – Indicator node, which we add new IndicatorItem and Indicator nodes too
  • comment_dict – maps ids to comment values. only applied to IndicatorItem nodes
Returns:

True upon completion
Raises:

UpgradeError if there is a problem during the conversion.
convert_to_11()

converts the iocs in self.iocs from openioc 1.0 to openioc 1.1 format. the converted iocs are stored in the dictionary self.iocs_11

insert(filename)

Parses files to load them into memory and insert them into the class.

Parameters:filename – File or directory pointing to .ioc files.
Returns:A list of .ioc files which could not be parsed.
parse(fn)

Parses a file into a lxml.etree structure with namespaces remove. This tree is added to self.iocs.

Parameters:fn – File to parse.
Returns:
write_iocs(directory=None, source=None)
Parameters:
  • directory – Directory to write IOCs to. If not provided, the current working directory is used.
  • source – Dictionary contianing iocid -> IOC mapping. Defaults to self.iocs_11.
Returns:

Module contents

__init__.py.py from ioc_writer Created: 12/17/15

Purpose: Provide a generic IOC management class for parsing a large set of IOCs into memory.

class ioc_writer.managers.IOCManager

Bases: object

Generic class for managing IOC objects in memory. This base class just provides a mechanism for loading the .ioc files into memory and storing them in a dictionary. This is designed to be subclassed. The original parsing can be extended by a subclass which just needs to have a callback function registered which will consume a IOC object.

The following is a subclass example:

class IOCTestManager(managers.IOCManager):
    def __init__(self):
        managers.IOCManager.__init__(self)
        self.child_count = {}
        self.register_parser_callback(self.parse_callback)

    def parse_callback(self, ioc_obj):
        c = ioc_obj.top_level_indicator.getchildren()
        self.child_count[ioc_obj.iocid] = len(c)
insert(filename)

Parses files to load them into memory and insert them into the class.

Parameters:filename – File or directory pointing to .ioc files.
Returns:A list of .ioc files which could not be parsed.
parse(ioc_obj)

parses an ioc to populate self.iocs and self.ioc_name

Parameters:ioc_obj –
Returns:
register_parser_callback(func)

Register a callback function that is called after self.iocs and self.ioc_name is populated.

This is intended for use by subclasses that may have additional parsing requirements.

Parameters:func – A callable function. This should accept a single input, which will be an IOC class.
Returns: