ndjson

Parse an NDJSON file.

Introduction

Use the ndjson enrichment to parse data from an NDJSON (newline delimited JSON) file. The JSON file must contain an NDJSON array as the top level object. Each line of the JSON array will be treated as a row of data. Define which data from the JSON to parse using the enrichment.

Creating the enrichment

For more information on creating an enrichment, see Using custom scripts.

Configuring the enrichment

To configure the enrichment, fill in the following fields. Required fields are marked with an asterisk (*).

Header

Enter the name of the fields used to identify the header of the JSON file. These fields must match the keys in the JSON file that you want to parse. The values entered here will be used as the header of the data extract.

Sample

If you do not specify any values in the Header field, enter the number of lines to search through to find the headers.

Mapping

Enter the name of the fields to parse. In the left field, enter the key in the NDJSON file to parse (this must match the name of the header provided in the field above). In the right field, enter the name of the source field (this is usually the name as the value entered into the left mapping field and the header).

You can parse date from sub-nodes within a JSON array by appending a node with .sub_node. For example, to access a sub-node called mailHost within an array called init-param, enter init-param.mailHost in the left field.

You can parse data from arrays using the format node[index] where node is the name of the node with the array and index is the array.

Missing

Enter a value to use as an identifier for missing values. For example, enter null to treat any null values in the JSON file as missing.

Subtable

Enter the name for a subtable that you want to contain the enriched data. The enrichment is applied to the whole data extract, then the enriched data is output into the subtable you have named here.

This subtable is a temporary table, which means it only exists for this custom script. You can apply additional instructions within the same custom script to the subtable. However, the subtable cannot be used in any other custom scripts.

Example

Below is an example of the contents of an NDJSON file to parse into a data extract using the ndjson enrichment.

{"servlet-name": "cofaxCDS","servlet-class": "org.cofax.cds.CDSServlet","init-param": {"mailHost": "mail1",}}
{"servlet-name": "cofaxEmail","servlet-class": "org.cofax.cds.EmailServlet","init-param": {"mailHost": "mail1","mailHostOverride": "mail2"}}
{"servlet-name": "cofaxAdmin","servlet-class": "org.cofax.cds.AdminServlet"}
{"servlet-name": "fileServlet","servlet-class": "org.cofax.cds.FileServlet"}
{"servlet-name": "cofaxTools","servlet-class": "org.cofax.cms.CofaxToolsServlet","init-param": {"mailHost": "mail2","betaServer": true}}

Enrichment configuration

Header

• servlet-class

• init-param

Sample

1000

Mapping

Header to map	Type	Mapping
servlet-class	string	servlet-class
init-param	string	init-param.mailHost

Missing

Enter the value null

Data table after enrichment

servlet-class	init-param
org.cofax.cds.CDSServlet	mail1
org.cofax.cds.EmailServlet	mail2
org.cofax.cds.AdminServlet	null
org.cofax.cds.FileServlet	null
org.cofax.cms.CofaxToolsServlet	mail2