Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/smooks/smooks-dfdl-cartridge

Smooks cartridge leveraging Apache Daffodil to parse files and unparse XML
https://github.com/smooks/smooks-dfdl-cartridge

apache-daffodil dfdl dfdl-schema flat-file smooks-cartridge

Last synced: about 2 months ago
JSON representation

Smooks cartridge leveraging Apache Daffodil to parse files and unparse XML

Awesome Lists containing this project

README 

        
= Smooks DFDL Cartridge



image:https://img.shields.io/maven-central/v/org.smooks.cartridges/smooks-dfdl-cartridge[Maven Central]

image:https://img.shields.io/nexus/s/org.smooks.cartridges/smooks-dfdl-cartridge?server=https%3A%2F%2Foss.sonatype.org[Sonatype Nexus (Snapshots)]

image:https://github.com/smooks/smooks-dfdl-cartridge/workflows/CI/badge.svg[Build Status]



// tag::smooks-dfdl-cartridge[]

The DFDL cartridge opens up Smooks to a wide array of data formats (e.g., SWIFT, ISO8583, HL7). In fact, this cartridge forms the foundation of the https://github.com/smooks/smooks-edi-cartridge[EDI and EDIFACT cartridges]. The DFDL cartridge deserializes (i.e., parses) non-XML data and serializes (i.e., unparses) XML according to the structure described in a https://daffodil.apache.org/docs/dfdl/[DFDL] schema. Take the subsequent DFDL schema as an example:



.csv.dfdl.xsd

[source,xml]

----



  



  

    

      

    

  



  

    

      

        

          

            

              

            

          

        

        

          

            

              

            

          

        

      

    

  



----



This schema describes the structure of CSV data like the one below:



.input.csv

[source,csv]

----

last,first,middle,DOB

smith,robert,brandon,1988-03-24

johnson,john,henry,1986-01-23

jones,arya,cat,1986-02-19

----



A Smooks config parsing the above CSV using the DFDL cartridge would be written as:



.smooks-config.xml

[source,xml]

----



    



    ...



----



`+dfdl:parser+` is a reader and its `+schemaUri+` attribute references the DFDL schema driving the parsing behaviour. Assuming _input.csv_ is the source, `+dfdl:parser+` will generate the event stream:



[source,xml]

----



    

        last

        first

        middle

        DOB

    

    

        smith

        robert

        brandon

        1988-03-24

    

    

        johnson

        john

        henry

        1986-01-23

    

    

        jones

        arya

        cat

        1986-02-19

    



----



Shown in the next snippet is a https://github.com/smooks/smooks/blob/master/README.adoc#pipeline[pipeline] enclosing the `+dfdl:unparser+` visitor:



.smooks-config.xml

[source,xml]

----



    ...



    

        

            

                

            

        

        

            

                

            

        

    



----



In contrast to the `+dfdl:parser+` `+schemaUri+` attribute , the `+schemaUri+` schema in `+dfdl:unparser+` drives the unparsing behaviour. `+dfdl:unparser+` replaces each node in the event stream with its serialized CSV counterpart, essentially implementing a pass-through application.



The DFDL cartridge supports variables, on disk caching, and trace debugging. Consult the link:src/main/resources/META-INF/xsd/smooks/dfdl-1.0.xsd[XSD documentation] for further information. It is strongly advised to learn DFDL before authoring a DFDL schema. Resources to get started with DFDL include:



* https://ogf.org/ogf/doku.php/standards/dfdl/dfdl[Open Grid Forum DFDL page]

* https://daffodil.apache.org/community/[Apache Daffodil user mailing list]

* https://daffodil.apache.org/docs/dfdl/[DFDL specification]



== Parser reader options



=== Indent



Indent the generated event stream to make it easier to read. Useful for troubleshooting. The default value is `false`. Example:



.smooks-config.xml

[source,xml]

----



    



----



=== Cache on disk



Persist DFDL schema on disk to reduce compilation time in subsequent runs. The default value is `false`. Example:



.smooks-config.xml

[source,xml]

----



    



----



=== Validation mode



Validation modes for validating the resulting infoset against the DFDL schema. The following values are supported:



[cols="1,1"]

|===

| Value | Description



| Off | Turn off all validation against the DFDL schema.

| Limited | Perform XSD validation of facets, minLength, maxLength, enumeration, minInclusive, minExclusive, maxInclusive, maxExclusive, and maxOccurs constraints. Validation failures will be printed in the log but will not interrupt parsing or unparsing.

| Full | Perform full schema validation using Xerces. A validation failure will abort parsing and throw a `org.smooks.api.SmooksException`.

|===



The default value is `Off`. Example:



.smooks-config.xml

[source,xml]

----



    



----



Validation failures can be retrieved from the Smooks execution context as shown below:



[source,java]

----

...



org.smooks.Smooks smooks = new org.smooks.Smooks();

org.smooks.api.ExecutionContext executionContext = smooks.createExecutionContext();

smooks.filterSource(executionContext, source, sink);



List diagnostics = executionContext.get(org.smooks.cartridges.dfdl.parser.DfdlParser.DIAGNOSTICS_TYPED_KEY);

...

----



=== Debugging



Enable/disable trace debugging. The default value is `false`. Example:



.smooks-config.xml

[source,xml]

----



    



----



=== Schematron validation



Apply standalone or embedded https://www.schematron.com/[Schematron] rules within the DFDL schema. Note that Schematron validation leads to the https://issues.apache.org/jira/browse/DAFFODIL-2386[input stream being loaded into memory] therefore such validation is not recommended for large streams.



Standalone Schematron rules are applied like this:



.smooks-config.xml

[source,xml]

----



    

        

    



----



Embedded rules are applied as follows:



.smooks-config.xml

[source,xml]

----



    

        

    



----



== Unparser visitor options



=== Cache on disk



Persist DFDL schema on disk to reduce compilation time in subsequent runs. The default value is `false`. Example:



.smooks-config.xml

[source,xml]

----



    



----



=== Validation mode



Validation modes for validating the input infoset against the DFDL schema. The following values are supported:



[cols="1,1"]

|===

| Value | Description



| Off | Turn off all validation against the DFDL schema.

| Limited | Perform XSD validation of facets, minLength, maxLength, enumeration, minInclusive, minExclusive, maxInclusive, maxExclusive, and maxOccurs constraints. Validation failures will be printed in the log but will not interrupt parsing or unparsing.

| Full | Perform full schema validation using Xerces. A validation failure will abort unparsing and throw a `org.smooks.api.SmooksException`.

|===



The default value is `Off`. Example:



.smooks-config.xml

[source,xml]

----



    



----



=== Debugging



Enable/disable trace debugging. The default value is `false`. Example:



.smooks-config.xml

[source,xml]

----



    



----



== Maven Coordinates



.pom.xml

[source,xml]

----



    org.smooks.cartridges

    smooks-dfdl-cartridge

    1.0.1

    

----



== XML Namespace



....

xmlns:dfdl="https://www.smooks.org/xsd/smooks/dfdl-1.0.xsd"

....

// end::smooks-dfdl-cartridge[]



== License



Smooks DFDL Cartridge is open source and licensed under the terms of the Apache License Version 2.0, or the GNU Lesser General Public License version 3.0 or later. You may use Smooks DFDL Cartridge according to either of these licenses as is most appropriate for your project.



`+SPDX-License-Identifier: Apache-2.0 OR LGPL-3.0-or-later+`