Thoughts on the current state of ASN.1 and XML technologies.

Archive for category Uncategorized

XBinder Version 2.6 Release

We recently released a new major version of our XBinder XML Schema Compiler product.  XBinder generates C, C++, Java, or C# code from XML Schema (XSD) definitions, making it easier to create and consume compliant XML documents in a programmatic way.

New features in this release include the following:

    • Generation of C/C++ Code to Better Support 64-bit Architectures
    • Generation of Visual Studio Projects for 64-bits
    • Ability to Use Qt Types
    • Use of Newer Versions of Visual Studio for Windows
    • XML Validation in XBinder Editor
    • Generation of Visual Studio Projects in XBinder Editor

A full list of changes, and additional details on the above changes, are available in the release notes.

No Comments

V2X API Updates

The V2X ASN.1 encode/decode API provides functions for encoding and decoding messages defined in the SAE International Dedicated Short Range Communications (DSRC) Message Set Dictionary J2735 standard and equivalent ETSI standards – CAM and DENM.  An updated version of the API is now available for download from the V2X API product page on our web-site.  The updated API was built with ASN1C v7.2.2 and adds the following new features:

  • Generated XER (XML) and JSON code was added making it possible to output binary data to these formats and conversely decode data in these formats back to binary form.
  • The updated API contains generated “print-to-stream” functions rather than standard print functions.  This makes it possible to set up print callbacks to print to output devices or mediums other than stdout (for example, to a GUI display).
  • Versions of the C/C++ API have been added for Linux x86 32-bit and Linux ARM 32-bit platforms.

Also note that the name of the product has changed.  It was formerly known as V2X DLL, but the name was changed to V2X API as the former gave the impression it was a Windows-only product.

 

No Comments

Re-using Decoded Items in a Subsequent Encoding

This blog post attempts to provide advice about re-using items from a decoded message in a subsequent encoding of a different message.

Let’s look at the employee sample in the c/sample_ber/employee directory of the ASN1C SDK. The ASN.1 specification for this sample is fairly simple and looks like this:

<code>
Employee DEFINITIONS ::= BEGIN
EXPORTS;

PersonnelRecord ::= [APPLICATION 0] IMPLICIT SET {
   Name,
   title [0] IA5String,
   number EmployeeNumber,
   dateOfHire [1] Date,
   nameOfSpouse [2] Name,
   children [3] IMPLICIT SEQUENCE OF ChildInformation
}

ChildInformation ::= SET {
   Name,
   dateOfBirth [0] Date
}

Name ::= [APPLICATION 1] IMPLICIT SEQUENCE {
   givenName IA5String,
   initial IA5String,
   familyName IA5String
}

EmployeeNumber ::= [APPLICATION 2] IMPLICIT INTEGER

Date ::= IA5String

END
</code>

Now, suppose the ChildInformation piece and the Name piece need to be used in a different message, called a FamilyRecord, that is going to be encoded after a PersonnelRecord message is decoded. We can change the ASN.1 specification that defines the PersonnelRecord so it looks like this:

<code>
Employee DEFINITIONS AUTOMATIC TAGS ::= BEGIN

EXPORTS;

IMPORTS Name, ChildInformation FROM Common;

PersonnelRecord ::= SET {
   employeeName Name,
   title IA5String,
   number EmployeeNumber,
   dateOfHire Date,
   nameOfSpouse Name,
   children SEQUENCE OF ChildInformation
}

EmployeeNumber ::= INTEGER

Date ::= IA5String

END
</code>

So we can see now that instead of defining ChildInformation and Name, the specification imports them from a module named Common. The other changes are that we are using an explicit element name called employeeName just to make things neater, and we are using AUTOMATIC TAGS for sanity preservation.

The Common module would look like this:

<code>
Common DEFINITIONS AUTOMATIC TAGS ::= BEGIN

EXPORTS ChildInformation, Name;

ChildInformation ::= SET {
   childName Name,
   dateOfBirth Date
}

Name ::= SEQUENCE {
   givenName IA5String,
   initial IA5String,
   familyName IA5String
}

END
</code>

And we also need to create a specification that defines FamilyRecord:

<code>
Family DEFINITIONS AUTOMATIC TAGS ::= BEGIN

EXPORTS;

IMPORTS Name, ChildInformation FROM Common;

FamilyRecord ::= SET {
nameOfSpouse Name,
ageOfSpouse INTEGER (18..MAX),
children SEQUENCE OF ChildInformation
}

END
</code>

So we have a module named Common that defines ChildInformation and Name. And we have two other modules, named Employee (which defines the PersonnelRecord PDU) and Family (which defines the FamilyRecord PDU) that both make use of these two definitions in the Common module.

Now, suppose we have a need to write some C code that decodes a PersonnelRecord and uses the name of the employee’s spouse and and the information about the employee’s children in a new encoding of a FamilyRecord. Below is a complete C program that can accomplish this. The sections that I’m going to talk about in a little more detail are indicated with numbers in square brackets, e.g., [1], [2], [3], etc.

<code>
/*
This program does the following:
Reads and decodes an already-encoded PersonnelRecord message.
Uses pieces of that decoded PersonnelRecord to populate the structures for a new FamilyRecord message.
Encodes that FamilyRecord message.
*/

#include "Employee.h"
#include "Family.h"
#include "rtxsrc/rtxDiag.h"
#include "rtxsrc/rtxFile.h"

#define MAXMSGLEN (1024)

int main()
{
   PersonnelRecord tEmployee;
   FamilyRecord tFamily;
   OSCTXT tDecodeContext, tEncodeContext;

   /* Receives the encoded (i.e., not yet decoded) PersonnelRecord message from the message.dat file */
   OSOCTET* pachEmployeeMessage;

   /* Receives the encoded FamilyRecord message from this program's encode call */
   OSOCTET achFamilyMessage[MAXMSGLEN];

   /* Receives a pointer to the encoded FamilyRecord message in order to print it and then write it to a file */
   OSOCTET *pachFamilyMessage;

   OSSIZE iLength;
   int iStatus;
   FILE* ptOutputFile;
   const char szInputFileName[] = "EmployeeMessage.dat";
   const char szOutputFileName[] = "FamilyMessage.dat";
   OSBOOL bTrace = TRUE, bVerbose = FALSE;

   /* Initialize the context structure for the decoding. */
   if (rtInitContext (&amp;tDecodeContext) != 0) { /* [1] */
      printf ("Error initializing decode context\n");
      return -1;
   }
   rtxSetDiag (&amp;tDecodeContext, bVerbose);

   /* Read the input file into a memory buffer. */
   iStatus = rtxFileReadBinary (&amp;tDecodeContext, szInputFileName, &amp;pachEmployeeMessage, &amp;iLength); /* [2] */
   if (0 != iStatus) {
      printf ("Error opening %s for read access\n", szInputFileName);
      return -1;
   }
   iStatus = xd_setp64 (&amp;tDecodeContext, pachEmployeeMessage, iLength, 0, 0, 0);
   if (0 != iStatus) {
      rtxErrPrint (&amp;tDecodeContext);
      return iStatus;
   }

   /* Clear the structures that will receive the decoded message. */
   asn1Init_PersonnelRecord (&amp;tEmployee);

   /* Decode the PersonnelRecord message. */
   iStatus = asn1D_PersonnelRecord (&amp;tDecodeContext, &amp;tEmployee, ASN1EXPL, 0); /* [3] */
   if (0 == iStatus) {
      if (bTrace) {
         printf ("Decode of PersonnelRecord was successful\n");
         printf ("Decoded record:\n");
         asn1Print_PersonnelRecord ("Employee", &amp;tEmployee);
      }
   }
   else {
      printf ("decode of PersonnelRecord failed\n");
      rtxErrPrint (&amp;tDecodeContext);
      return -1;
   }

   /* Now use the spouse's name and the children's names in a new FamilyRecord message. */

   /* Initialize the context structure for the encoding. */
   iStatus = rtInitContext (&amp;tEncodeContext); /* [4] */
   if (0 != iStatus) {
      printf ("encoding context initialization failed\n");
      rtxErrPrint (&amp;tEncodeContext);
      return iStatus;
   }
   rtxSetDiag (&amp;tEncodeContext, bVerbose);

   /* Populate the structures for the FamilyRecord message. */ /* [5] */
   tFamily.nameOfSpouse = tEmployee.nameOfSpouse;
   tFamily.ageOfSpouse = 30;
   tFamily.children = tEmployee.children;

   /* Encode the FamilyRecord message. */
   xe_setp (&amp;tEncodeContext, achFamilyMessage, sizeof(achFamilyMessage));
   if ((iLength = asn1E_FamilyRecord (&amp;tEncodeContext, &amp;tFamily, ASN1EXPL)) &gt; 0) /* [6] */
   {
      pachFamilyMessage = xe_getp (&amp;tEncodeContext);
      if (bTrace) {
         if (XU_DUMP (pachFamilyMessage) != 0)
         printf ("dump of ASN.1 message failed.");
      }
   }
   else {
      rtxErrPrint (&amp;tEncodeContext);
      return iLength;
   }

   /* Write the encoded message out to the output file */ /* [7] */

   if (0 != (ptOutputFile = fopen (szOutputFileName, "wb"))) {
      fwrite (pachFamilyMessage, 1, iLength, ptOutputFile);
      fclose (ptOutputFile);
   }
   else {
      printf ("Error opening %s for write access\n", szOutputFileName);
      return -1;
   }

   /* Now free up our contexts. */ /* [8] */
   rtFreeContext (&amp;tDecodeContext);
   rtFreeContext (&amp;tEncodeContext);

   return 0;
}
</code>

In part [1] we’re initializing a context structure for decoding.

In part [2] we’re reading a file that contains an encoded PersonnelRecord. The byte array pachEmployeeMessage will have the bytes of the encoded message.

In part [3] we’re decoding the PersonnelRecord into the tEmployee structure.

In part [4] we’re initializing a context structure for encoding. Note that we’re using different context structures for decoding and encoding.

Part [5] is the crucial part. Here we’re populating the members of the tFamily structure before we use it to encode a FamilyRecord message. For two of those members we’re using members of the tEmployee structure, which contains the decoded information from the PersonnelRecord message. In both cases the members are structures in the generated C code, so the assignment results in a shallow copy of the structure from tEmployee to tFamily. So all pointers within the tEmployee structures stay the same in the tFamily structures. The crucial part to remember here is that the memory used for the decoding of the PersonnelRecord message (i.e., the tEmployee structure) must remain intact until we’re completely done with the tFamily structure, since the tFamily structure now has pointers to that memory.

In part [6] we’re encoding a FamilyRecord message using the tFamily structure that we just populated in part [5].

In part [7] we’re writing the encoded FamilyRecord message out to a file.

In part [8] we’re freeing the two contexts that we used, one for decoding and one for encoding. As pointed out in part [5] it’s crucial that the context, and hence the memory, used for the decoding remain intact until we’re completely done with the encoding, since the structure used for the encoding has pointers to the memory used for the decoding.

No Comments

Adding Information to the J2735 Specification

As some background, see the following article:

http://dsrc-tools.com/map-spat/index.php/knowledge-base/regional-data-extensions-generic-lane-object/

This article discusses how to add some information to the J2735 specification.

This specific blog post discusses how to add information for a new region to the J2735 specification, generate code for it using ASN1C, and then reference the new material in your own code. In this example we will add a GenericLane definition for a new region named Mars. That’s Mars as in the planet, not the town in western PA.

To start, the J2735 ASN.1 specification defines a type called a RegionId as follows:

RegionId ::= INTEGER (0..255)
   noRegion     RegionId ::= 0  -- Use default supplied stubs
   addGrpA      RegionId ::= 1  -- USA
   addGrpB      RegionId ::= 2  -- Japan
   addGrpC      RegionId ::= 3  -- EU
   -- NOTE: new registered regional IDs will be added here
   -- The values 128 and above are for local region use

This definition is in the DSRC module in the ASN.1 file. If you read the background material (you did, didn’t you?), you know that the DSRC module is not to be modified by users of the J2735 specification.

But the background material also indicates that there is a module named REGION in the ASN.1 file that can be modified by developers. So let’s make the first line of this module look like this:

marsRegion RegionId ::= 222

The background material says to let SAE know what number you’ve chosen so it can be tracked. Since there is no intention that I know of to define intelligent traffic systems on the planet Mars, this number 222 isn’t actually being reported to SAE and remains available for anyone else to use.

Now a few lines down let’s modify the Reg-GenericLane definition so it looks like this:

Reg-GenericLane           DSRC.REG-EXT-ID-AND-TYPE ::= {
   { Mars.GenericLane-mars IDENTIFIED BY marsRegion },
   ...  }

Then at the end of the ASN.1 file let’s add a module named Mars that looks like this:

-- ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-
-- 
-- Begin module: Mars
-- 
-- ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-
 
Mars DEFINITIONS AUTOMATIC TAGS::= BEGIN

GenericLane-mars ::= SEQUENCE {
   laneType LaneType,
   distanceFromViking1 INTEGER, -- Since this is a distance, it could be constrained to be positive
   hue RedHue,
   ...
}

LaneType ::= ENUMERATED {
   martian-lane (1),
   rover-lane (2),
   ...
}

RedHue ::= ENUMERATED {
   light-red (1),
   deep-red (2),
   ...
}

END

That’s it as far as the ASN.1 changes are concerned. Now let’s generate some C++ code using ASN1C with a command like this, assuming our changed ASN.1 definitions are in a file named J2735Mars.asn:

asn1c j2735Mars.asn -uper -cpp -table-unions -genwriter -genreader -genprint -pdu GenericLane -gentest

Let’s look at what those qualifiers do:

  • -uper: Specifies that we’re using unaligned PER (Packed Encoding Rules).
  • -table-unions: Specifies that the code generation should take into account inter-dependencies within the ASN.1 definitions. These inter-dependencies are known as table constraints.
  • -genwriter: Specifies that we want to generate a writer program (writer.cpp) that will encode an instance of our message.
  • -genreader: Specifies that we want to generate a reader program (reader.cpp) that will decode an instance of our message.
  • -genprint: Specifies that we want to generate methods to print the contents of the message.
  • -pdu GenericLane: Specifies that our writer and reader programs are to work with an instance of GenericLane (as opposed to any of the other definitions in the ASN.1 file).
  • -gentest: Specifies that the writer is to generate random test data to populate our GenericLane instance.

You can also add -genmake to generate a makefile that will build the writer and reader binaries.

If you generate the code and then build it, you will have a writer binary that will encode the message into a file named message.dat, and you will have a reader binary that will decode the message that’s in the message.dat file. For my specific situation my message looked like this:

GenericLane {
   laneID = 252
   name = 'AMXrH7pmSZHULHTJ:Us_gUAqMrI'
   ingressApproach = 1
   egressApproach = 3
   laneAttributes {
      directionalUse = { 2, 01xxxxxx }
      sharedWith = { 10, 0x72 01xxxxxx }
      laneType {
         sidewalk = { 16, 0x48 0x61 }
      }
      regional {
         regionId = 254
         regExtValue = 0x04083677ccd172888708
      }
   }
   maneuvers = { 12, 0x5E 0100xxxx }
   nodeList {
      computed {
         referenceLaneId = 181
         offsetXaxis {
            large = -9096
         }
         offsetYaxis {
            small = 253
         }
         rotateXY = 18514
         scaleXaxis = 1124
         scaleYaxis = -1034
         regional[0] {
            regionId = 119
            regExtValue = 0x040831403e5e9dfbcb99
         }
      }
   }
   connectsTo[0] {
      connectingLane {
         lane = 189
         maneuver = { 12, 0x53 0100xxxx }
      }
      remoteIntersection {
         region = 16975
         id = 25828
      }
      signalGroup = 245
      userClass = 0
      connectionID = 70
   }
   overlays[0] = 9
   regional[0] {
      regionId = 222
      regExtValue {
         GenericLane-mars {
            laneType = martian-lane
            distanceFromViking1 = 3814
            hue = light-red
         }
      }
   }
}

Keep in mind that this data is randomly generated test data, and it will be different with each code generation. Note the “regional” section at the end, which is where we added our information about lanes on Mars. The test data generator chose our Mars definition, probably because it’s the only one there. But you can see values for the fields we added for traffic lanes on Mars.

The writer.cpp file calls the genTestInstance() method of the GenericLanePDU object, which is an instance of the class ASN1C_GenericLane. The code for this genTestInstance() method is in the generated file DSRCTest.cpp. The end of the method is where the new items are given values.

No Comments

Canonical OER

With ASN1C v7.1 it’s possible to encode and decode OER in C according to the canonical OER rules. Canonical OER can be utilized by setting the ASN1CANON flag in the context before the encoding or decoding is done:

rtxCtxtSetFlag(&ctxt, ASN1CANON);

The Canonical sample in c/sample_oer illustrates using this flag.

With the upcoming ASN1C v7.2 it will be possible to generate canonical OER code directly by using the new -coer qualifier to the ASN1C command (or by choosing the new COER option in the ASN1C GUI). Setting the flag in the context will still enable canonical rules for code generated with -oer instead of -coer. Also in v7.2 it will be possible to generate C++ code for OER instead of just C code.

No Comments