Kenna Data Importer (JSON Connector)

Richard Ehrhart

April 04, 2019 17:37

Last updated: May 10, 2021 11:32

Kenna Data Importer is a connector which ingests a standard JSON format for bringing vulnerability data into the platform. It is a collection of assets and their vulns/findings, as well as vulnerability definitions for those vulns/findings.

Setting up a KDI Connector via the UI

See the section below on how to generate a KDI formatted file. Once that's done, here's the process to upload via UI:

1) Log into the instance as an administrator.

2) Create a new KDI connector by browsing to Connectors > Kenna Data Importer.

3) Use the Upload and Run button to select the JSON file and upload.

Wait while the file uploads.

Note: Failure messages are stored in the connector's page.

4) Once uploaded, you should see the assets and vulnerabilities begin to populate.

5) To upload the file via the API, get the Connector ID for you new connector by clicking on the connector name.

Screen_Shot_2020-08-28_at_8.42.15_AM.png

KDI JSON FORMAT

Currently, there are two KDI formats. In Version 1, the scanner identifier was used to create a connection between scanner vulnerabilities and vulnerability definitions. Version 2 was created to provide a more natural easily understood human readable connection between scanner vulnerability definitions and the vulnerability definitions.

Note: Currently, there is no date set to cease support of Version 1.

KDI JSON Format Version 2

Changes from Version 1

A new required vuln_def_name field has been added to the vuln and findings sections. The name field in the vuln_def section is now required and matches the vul_def_name. This creates a unique composite key using scanner_type and vuln_def_name. A new top-level version field has been added and the value is 2. The new fields are highlighted below.

KDI Info

Field	Type	Required	Description
skip_autoclose	boolean	Yes	If true, suppresses the closing of vulnerabilities missing from assets in this payload.
version	integer	Yes	The version of the KDI format. The value should be '2'.
reset_tags	boolean	No	Used to determine if ALL tags should be wiped from an asset before the tags defined in the KDI upload files are applied. As an example, set this parameter to true if tags frequently change for a KDI asset Tags are additive-only if this parameter is not listed in the KDI or if it is set to false. Caution: If true, all tags will be removed from the matching asset regardless of source. If an asset has tags both from a KDI and scanner connector, the scanner connector would need to run again before the scanner tags would be regenerated.
assets	[asset]	Yes	Contains all the asset information.
vuln_defs	[vuln_def]	Yes	Contains all the CVE or CWE vulnerabilities.

asset

Field	Type	Required	Description
file, ip_address, mac_address, hostname, ec2, netbios, url, fqdn, image_id, container_id, external_id, database	locator_field	At least one	See locator field for more details on each field.
application	string	Conditional	This is the application identifier or name from the scanner perspective. It is required if url or file are used as primary locator. This field allows findings to be assigned to Applications in Kenna.
tags	[string]		An array of strings that correspond to tags on an asset.
owner	string		The owner of the asset.
os	string		The operating system of the asset.
os_version	string		Version of the asset's operating system.
priority	integer		[0..10] Defaults to 10, which is recommended unless you have a documented risk appetite for assets. This field is used to adjust the asset score.
asset_type	string		Value of either "image" or "container" required if image_id or container_id used as locators.
vulns	[vuln]	Yes	Array of vulnerabilities. If an asset contains no open vulnerabilities this can be an empty array, but to avoid vulnerabilities from being closed, use the skip_autoclose flag.
findings	[finding]	Yes	Contains all the findings information.

locator_field

Field	Type	Required	Description
file	string	At least one	Path of the affected file.
ip_address	string	At least one	IP address of the internal facing asset.
mac_address	string	At least one	MAC address of the asset.
hostname	string	At least one	Host name of the affected asset.
ec2	string	At least one	Amazon EC2 instance ID or name.
netbios	string	At least one	NetBIOS name.
url	string	At least one	URL pointing to the asset.
fqdn	string	At least one	Fully qualified domain name of affected asset.
image_id	string	At least one	Generally the sha identifier of the image. Must be paired with asset_type field. Must be paired with asset_type field. Custom Locator Orders must be modified to include this field in order for it to be imported.
container_id	string	At least one	Generally the sha identifier of the container. Must be paired with asset_type field. Custom Locator Orders must be modified to include this field in order for it to be imported.
external_id	string	At least one	External ID of the asset. This is often used as an internal organizational tracking name for the asset.
database	string	At least one	Database name.

vuln

Field	Type	Required	Description
scanner_identifier	string	Yes	Uniquely identifies data coming from a scanner.
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier (see above).
scanner_score	integer	Yes	Score given by the scanner. [0..10] Used for scoring in some cases where the Kenna algorithm is not used. Normalized to a Kenna risk score by multiplying x 10.
override_score	integer		The risk score [0..100] for an informational vulnerability.
created_at	string		ISO8601 timestamp indicating when the vulnerability was first found by the scanner. Defaults to current date if not provided.
last_seen_at	string	Yes	ISO8601 timestamp indicating when the vulnerability was last observed.
last_fixed_on	string		ISO8601 timestamp indicating when the vulnerability was last fixed.
status	string	Yes	Vulnerability remediation status. Valid values are: "open", and "closed". If skip_autoclose is set to false, open vulnerabilities that already exist in Kenna will be closed and this field changes from "open" to "closed".
port	integer		Port that the vulnerability is referring to.
vuln_def_name	string	Yes	The name of the vulnerability definition. The name of the vulnerability definition. Matches the name field in vuln_def section.

finding

Field	Type	Required	Description
scanner_identifier	string	Yes	Uniquely identifies data coming from a scanner.
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier (see above).
created_at	string		ISO8601 timestamp indicating when the vulnerability was first found by the scanner. Defaults to current date if not provided.
due_date	string		ISO8601 timestamp. Indicates when find is due to be remediated.
last_seen_at	string	Yes	ISO8601 timestamp indicating when the vulnerability was last observed.
severity	integer		Score given by the scanner. [0..10] Used for scoring. Normalized to a Kenna risk score by multiplying x 10.
triage_state	string		Possible values are: "new", "in_progress", "triaged", "resolved", "false_positive", "risk_accepted", "duplicate", "not_a_security_issue". If skip_autoclose is set to false, open vulnerabilities that already exist in Kenna will be closed and this field changes status to resolved. This field defaults to "new" if no status is provided.
additional_fields	[]		Array of hash objects with the field name being the key and value being the desired information to pass any scanner-specific fields about the finding.
vuln_def_name	string	Yes	The name of the vulnerability definition. Matches the name field in vuln_def section.

vuln_def

Field	Type	Required	Description
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier to form a unique key (see above).
cve_identifiers	string		Comma delimited list with format CVE-000-0000. Only one set of identifiers will be saved per vuln_def.
wasc_identifiers	string		Comma delimited list with format WASC-00. Only one set of identifiers will be saved per vuln_def.
cwe_identifiers	string		Comma delimited list with format CWE-000. Only one set of identifiers will be saved per vuln_def.
name	string	Yes	Title or short name of the vulnerability and is used with scanner_type as a key. This name matches the vul_def_name field in vuln/finding sections.
description	string		Full description of the vulnerability. Note: If the value of the field is blank, a blank description is displayed in Kenna. If the field is omitted entirely, a default value of "No description was provided" is substituted. If either a description or name is not provided, the vulnerability is created as a generic "Informational" vulnerability. The substituted value is sufficient to avoid this.
solution	string		Steps or links for remediation.

KDI Version 2 Example:

Here is an example of a minimal generated KDI version 2 file that is ready for import into the Kenna platform.

{
  "skip_autoclose": false,
  "version": 2,
  "assets": [
    {
      "file": "app/models/data_access/asset.rb",
      "application": "conduit",
      "priority": 0,
      "vulns": [
        {
          "scanner_identifier": 16,
          "vuln_def_name": "File Access",
          "external_unique_id": 105,
          "scanner_type": "Brakeman",
          "created_at": "2018-05-03T00:00:00+00:00",
          "last_seen_at": "2018-05-03T00:00:00+00:00",
          "due_date": "2018-06-02T00:00:00+00:00",
          "status": "open",
          "details": "foobar",
          "last_fixed_on": "2018-05-03T00:00:00+00:00",
          "scanner_score": 2,
          "override_score": 3
        }
      ],
      "findings": [
        {
          "scanner_identifier": 16,
          "vuln_def_name": "File Access",
          "external_unique_id": 105,
          "scanner_type": "Brakeman",
          "created_at": "2018-05-03T00:00:00+00:00",
          "last_seen_at": "2018-05-03T00:00:00+00:00",
          "due_date": "2018-06-02T00:00:00+00:00",
          "triage_state": "new",
          "last_fixed_on": "2018-05-03T00:00:00+00:00",
          "severity": 2,
          "override_score": 3,
          "additional_fields": {
            "test_field": "test data"
          }
        }
      ]
    },
    {
      "file": "app/controllers/clients_controller.rb",
      "application": "conduit",
      "vulns": [
        {
          "scanner_identifier": 10,
          "vuln_def_name": "Auth Blacklist",
          "external_unique_id": 2,
          "scanner_type": "Qualsys",
          "created_at": "2018-05-03T00:00:00+00:00",
          "last_seen_at": "2018-05-03T00:00:00+00:00",
          "last_fixed_on": "2018-05-03T00:00:00+00:00",
          "scanner_score": 2,
          "override_score": 3,
          "status": "open"
        }
      ],
      "findings": [
        {
          "scanner_identifier": 10,
          "vuln_def_name": "Auth Blacklist",
          "external_unique_id": 2,
          "scanner_type": "Qualsys",
          "created_at": "2018-05-03T00:00:00+00:00",
          "last_seen_at": "2018-05-03T00:00:00+00:00",
          "last_fixed_on": "2018-05-03T00:00:00+00:00",
          "scanner_score": 2,
          "override_score": 3,
          "status": "false_positive",
          "additional_fields": {
            "test_field": "test data"
          }
        }
      ]
    }
  ],
  "vuln_defs": [
    {
      "name": "File Access",
      "scanner_type": "Brakeman",
      "wasc_identifiers": "WASC-17",
      "solution": "Fix it!"
    },
    {
      "name": "Auth Blacklist",
      "scanner_type": "Qualsys",
      "wasc_identifiers": "WASC-15"
    }
  ]
}

KDI JSON FORMAT Version 1

Note: Currently, there is no date set to cease support of Version 1.

KDI Info

Field	Type	Required	Description
skip_autoclose	boolean	Yes	If true, suppresses the closing of vulnerabilities missing from assets in this payload.
reset_tags	boolean	No	Used to determine if ALL tags should be wiped from an asset before the tags defined in the KDI upload files are applied. As an example, set this parameter to true if tags frequently change for a KDI asset Tags are additive-only if this parameter is not listed in the KDI or if it is set to false. Caution: If true, all tags will be removed from the matching asset regardless of source. If an asset has tags both from a KDI and scanner connector, the scanner connector would need to run again before the scanner tags would be regenerated.
assets	[asset]	Yes	Contains all the asset information.
vuln_defs	[vuln_def]	Yes	Contains all the CVE or CWE vulnerabilities.

asset

Field	Type	Required	Description
file, ip_address, mac_address, hostname, ec2, netbios, url, fqdn, image_id, container_id, external_id, database	locator_field	At least one	See locator field for more details on each field.
application	string	Conditional	This is the application identifier or name from the scanner perspective. It is required if url or file are used as primary locator. This field allows findings to be assigned to Applications in Kenna.
tags	[string]		An array of strings that correspond to tags on an asset.
owner	string		The owner of the asset.
os	string		The operating system of the asset.
os_version	string		Version of the asset's operating system.
priority	integer		[0..10] Defaults to 10, which is recommended unless you have a documented risk appetite for assets. This field is used to adjust the asset score.
asset_type	string		Value of either "image" or "container" required if image_id or container_id used as locators.
vulns	[vuln]	Yes	Array of vulnerabilities. If an asset contains no open vulnerabilities this can be an empty array, but to avoid vulnerabilities from being closed, use the skip_autoclose flag.
findings	[finding]	Yes	Contains all the findings information.

locator_field

Field	Type	Required	Description
file	string	At least one	Path of the affected file.
ip_address	string	At least one	IP address of the internal facing asset.
mac_address	string	At least one	MAC address of the asset.
hostname	string	At least one	Host name of the affected asset.
ec2	string	At least one	Amazon EC2 instance ID or name.
netbios	string	At least one	NetBIOS name.
url	string	At least one	URL pointing to the asset.
fqdn	string	At least one	Fully qualified domain name of affected asset.
image_id	string	At least one	Generally the sha identifier of the image. Must be paired with asset_type field. Must be paired with asset_type field. Custom Locator Orders must be modified to include this field in order for it to be imported.
container_id	string	At least one	Generally the sha identifier of the container. Must be paired with asset_type field. Must be paired with asset_type field. Custom Locator Orders must be modified to include this field in order for it to be imported.
external_id	string	At least one	External ID of the asset. This is often used as an internal organizational tracking name for the asset.
database	string	At least one	Database name.

vuln

Field	Type	Required	Description
scanner_identifier	string	Yes	Each unique scanner_identifier will need a corresponding entry in the vuln_defs section below, this typically should be the external identifier used by your scanner. Used to identify the unique data coming in from the scanner named in scanner_type. scanner_identifier is combined with scanner_type for form a unique key. The 2 keys are listed with each vuln/finding under asset and then listed again as a matching pair in the vuln_defs section along with additional information.
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier (see above).
scanner_score	integer	Yes	Score given by the scanner. [0..10] Used for scoring in some cases where the Kenna algorithm is not used. Normalized to a Kenna risk score by multiplying x 10.
override_score	integer		The risk score [0..100] for an informational vulnerability.
created_at	string		ISO8601 timestamp indicating when the vulnerability was first found by the scanner. Defaults to current date if not provided.
last_seen_at	string	Yes	ISO8601 timestamp indicating when the vulnerability was last observed.
last_fixed_on	string		ISO8601 timestamp indicating when the vulnerability was last fixed.
status	string	Yes	Vulnerability remediation status. Valid values are: "open", and "closed". If skip_autoclose is set to false, open vulnerabilities that already exist in Kenna will be closed and this field changes from "open" to "closed".
port	integer		Port that the vulnerability is referring to.

finding

Field	Type	Required	Description
scanner_identifier	string	Yes	Each unique scanner_identifier will need a corresponding entry in the vuln_defs section below, this typically should be the external identifier used by your scanner. Used to identify the unique data coming in from the scanner named in scanner_type. scanner_identifier is combined with scanner_type for form a unique key. The 2 keys are listed with each vuln/finding under asset and then listed again as a matching pair in the vuln_defs section along with additional information.
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier (see above).
created_at	string		ISO8601 timestamp indicating when the vulnerability was first found by the scanner. Defaults to current date if not provided.
due_date	string		ISO8601 timestamp. Indicates when find is due to be remediated.
last_seen_at	string		ISO8601 timestamp indicating when the vulnerability was last observed.
severity	integer		Score given by the scanner. [0..10] Used for scoring. Normalized to a Kenna risk score by multiplying x 10.
triage_state	string		Possible values are: "new", "in_progress", "triaged", "resolved", "false_positive", "risk_accepted", "duplicate", "not_a_security_issue". If skip_autoclose is set to false, open vulnerabilities that already exist in Kenna will be closed and this field changes status to resolved. This field defaults to "new" if no status is provided.
additional_fields	[]		Array of hash objects with the field name being the key and value being the desired information to pass any scanner-specific fields about the finding.

vuln_def

Field	Type	Required	Description
scanner_identifier	string	Yes	Each unique scanner identifier will need a corresponding entry in the vuln/finding sections. This typically should be the external identifier used by your scanner. scanner_identifier is combined with scanner_type for form a unique key. The 2 keys are listed with each vuln/finding under asset and then listed again as a matching pair in the this section along with additional information.
scanner_type	string	Yes	Identifies the scanner the data came from. Paired with scanner_identifier to form a unique key (see above).
cve_identifiers	string		Comma delimited list with format CVE-000-0000. Only one set of identifiers will be saved per vuln_def.
wasc_identifiers	string		Comma delimited list with format WASC-00. Only one set of identifiers will be saved per vuln_def.
cwe_identifiers	string		Comma delimited list with format CWE-000. Only one set of identifiers will be saved per vuln_def.
name	string	Conditional	Title or short name of the vulnerability and is used as the vuln/finding display value if CVE/CWE/WASC not provided.
description	string		Full description of the vulnerability.
solution	string		Steps or links for remediation.

KDI Version 1 Example:

Here is an example of a minimal generated KDI version 1 file that is ready for import into the Kenna platform.

{
 "skip_autoclose": false,
 "assets": [
   {
    "url": "www.myco.com",
    "tags": [
      "AppID:DES",
      "Find_type:Internal Reporter"
    ],
    "vulns": [],
    "findings": [
      {
      "scanner_type": "PenTestA",
      "scanner_identifier": "PSRC-7783",
      "created_at": "2020-06-12-00:00:00",
      "severity": 3,
      "last_seen_at": "2020-06-27-12:19:18",
      "triage_state": "new",
      "additional_fields": {
        "line_number": "156",
        "source_file": "testfolder/testfile.rb"
      }
    }
    ]
},
  {
    "ec2": "i-02aadcccfda719968",
    "ip_address": "172.31.42.121",
    "priority": 9,
    "tags": [
     "AWS"
    ],
    "vulns": [
{
      "scanner_type": "AWS Inspector",
      "scanner_identifier": "aws-vuln-id-1",
      "created_at": "2018-11-10-18:08:57",
      "last_seen_at": "2018-11-10-18:08:57",
      "status": "open"
    },
    {
      "scanner_type": "AWS Inspector",
      "scanner_identifier": "aws-vuln-id-2",
      "created_at": "2018-11-10-18:08:57",
      "last_seen_at": "2018-11-10-18:08:57",
      "status": "open"
    },
    {
      "scanner_type": "AWS Inspector",
      "scanner_identifier": "aws-vuln-id-3",
      "created_at": "2018-11-10-18:08:57",
      "details": "some details about how CVE-2018-10853 and CVE-2018-18074 are impacting asset",
      "last_seen_at": "2018-11-10-18:08:57",
      "status": "open"
    }
    ]
  }
  ],
  "vuln_defs": [
  {
    "scanner_identifier": "aws-vuln-id-1",
    "scanner_type": "AWS Inspector",
    "cve_identifiers": "CVE-2018-17456",
    "name": "Name of vulnerability involving CVE-2018-17456",
    "description": "Description of vuln involving CVE-2018-17456",
    "solution": "Do something good to fix CVE-2018-17456"
  },
  {
    "scanner_identifier": "aws-vuln-id-2",
    "scanner_type": "AWS Inspector",
    "cve_identifiers": "CVE-2018-6555",
    "name": "Name of vulnerability involving CVE-2018-6555",
    "description": "Description of vuln involving CVE-2018-6555",
    "solution": "Do something good to fix CVE-2018-6555"
  },
  {
    "scanner_identifier": "aws-vuln-id-3",
    "scanner_type": "AWS Inspector",
    "cve_identifiers": "CVE-2018-10853, CVE-2018-18074",
    "name": "Name of vulnerability involving CVE-2018-10853, CVE-2018-18074",
    "description": "Description of vuln involving CVE-2018-10853, CVE-2018-18074"
  },
  {
    "scanner_identifier": "PSRC-7783",
    "scanner_type": "PenTestA",
    "cwe_identifiers": "CWE-200",
    "name": "Name of vulnerability involving CWE-200",
    "description": "Description of vuln involving CWE-200"
  }
  ]
}

Kenna Data Importer (JSON Connector)

Setting up a KDI Connector via the UI

KDI JSON FORMAT

KDI JSON Format Version 2

Changes from Version 1

KDI Info

asset

locator_field

vuln

finding

vuln_def

KDI Version 2 Example:

KDI JSON FORMAT Version 1

KDI Info

asset

locator_field

vuln

finding

vuln_def

KDI Version 1 Example:

Comments

Articles in this section