Thoughts on the current state of ASN.1 and XML technologies.

Compact code generation in ASN1C

ASN.1 is used in a lot of different areas and a new area that is within the Internet of Things (IoT).  In particular Narrowband IoT (NB-IoT) uses ASN.1 UPER-based messaging.

One characteristic of these devices is they are small, so code size is critical.  We have been working on ways to make our ASN1C generated code and run-time libraries as compact as possible for applications such as these.   In our latest ASN1C v7.2.1 patch release, we are now including a new set of compact libraries for Linux.  These can be found in the c/lib_compact directories.  They are built with gcc using maximum space optimization settings and with a lot of non-critical code stripped out.  The compact libraries are roughly 25% smaller than the standard libraries.

In addition to using the compact libraries, additional steps can be taken to reduce the size of the generated code.  We touched on some of these in a past blog post entitled “Optimizing PER Encoding and Code Footprint“.  We would also recommend using the following command-line options (the equivalent GUI option is in parentheses):

  • -compact  (Generate compact code)
  • -noinit  (uncheck the Generate Initialization Functions checkbox)
  • -noenumconvert (do not generate enum-to-string conversion functions – should only be enabled if print functions are generated)

Other options that you may or may not be able use:

  • -lax (Do not generate constraint checks)
  • -strict-size (Interpret size constraints strictly)

If all of these measures are employed, users could potentially see the size of their application reduced by one half or more.

 

No Comments

ASN1C 7.2 Improved Comment Handling

In version 7.2, we improved our handling of ASN.1 comments, as follows.

  • When using the “Pretty-print ASN.1” (-asn1)  option, comments from type assignments and elements (SEQUENCE/SET/CHOICE components) are now included in the output.  Previously, pretty-printed ASN.1 did not include any ASN.1 comments in the output.
  • When generating C/C++ code, we previously put ASN.1 comments only for types into the C/C++ comments.  We now include ASN.1 comments from elements as well.
  • When writing comments, we now try to preserve the position of the comment as it appeared in the ASN.1.  We formerly printed all comments before the type assignment with which we associated the comment, even if the comment actually appeared after the type assignment.

When we output ASN.1 comments and ASN.1 syntax, whether the context is pretty-printed ASN.1 or C/C++ comments, we are not simply writing out everything as it appeared in the input.  This means we have to associate comments with syntax.  Since ASN.1 comments don’t have a syntactic relationship to other parts of the ASN.1 syntax, such associations involve a heuristic.  In the example below, the comment is potentially associated with either BigNumber or SmallNumber, though common practice suggests it’s most likely related to SmallNumber.

BigNumber ::= INTEGER (500..1000)

-- This is the type to use for speeds

SmallNumber ::= INTEGER (0..30)

Here’s a rough description of the heuristic rules we use:

  • If the start of a comment comes after some other ASN.1 syntax appearing on the same line, the comment is considered related to that syntax.
  • If the start of a comment is preceded, on the same line, only by whitespace, the comment may be related either to syntax that precedes or succeeds the comment.
    • If the comment is followed by a type assignment or an element, the first such comment that is not indented, relative to the type assignment or element, is associated with that type assignment or element.  Successive comments are also associated with the same item, regardless of indentation.
    • Any comments that preceded the first non-indented comment (all of which are indented) are associated with something which precedes those comments.  If these comments are immediately preceded by a type assignment or an element, they are associated with that type assignment or element.  In any case, they will not be associated with an element or type assignment that follows those comments.

Some examples:

-- comment for Person
Person ::= SEQUENCE {
   -- comment for age
   age INTEGER, -- another comment for age
      -- yet another comment for age
   -- comment for name
   name UTF8String
} -- another comment for Person
   -- yet another comment for Person

-- comment for Winnings
Winnings::= INTEGER (500..1000)

It is possible that these heuristics will associate a comment differently than a human reader would have.  Consider this example:

BigNumber ::= INTEGER (500..1000)

   -- SmallNumber is used for speeds

SmallNumber ::= INTEGER (0..30)

Because of the indentation, the comment will be associated with BigNumber, but it obviously actually relates to SmallNumber.  Since we try to preserve location when printing, we’ll print the comment after the definition of BigNumber, which can give the reader a hint that the comment might actually relate to something else (if the content of the comment were different, this might not be so obvious to the reader).

No Comments

Performance Improvements in ASN1C v7.2

One of the new features announced in the release of ASN1C v7.2 was improved C/C++ PER encode and decode performance.  This blog post provides some details on the improvements.

We measured a number of different messages types, but two that stood out were the improvement in encoding and decoding unaligned PER messages for LTE RRC and DSRC – two commonly used specifications.   For LTE, we encoded a sample of DL-DCCH-Message messages.  For DSRC, a set of BasicSafetyMessage messages were used.

The time in milliseconds to decode and encode 100000 records between v71x and v72x is shown in the following charts:

The chart on the left shows decoding and on the right encoding.  As can be seen, decoding performance for LTE RRC improved by over 100% while for DSRC, the gain was approximately 50%.  Encoding performances gains were not as dramatic with improvements of 28% and 18% respectively.

So to what to attribute these gains?  The main improvement was in doing calculations that were being done in the run-time to be done at compile time and thus generating simpler code for things such as constraint checking.  Similar improvements were made in the generation of code specifically targeted as “unaligned PER” (or UPER as it is commonly known) through the use of the -uper command-line option.  This resulted in the removal of code that checked for aligned or unaligned PER at run-time.

Finally, we introduced a new kind of simplified memory management called static memory blocks that contains some limitations on use but which is much faster because rather than having to carve up blocks, it simply always sequentially allocates going forward.  Further details on this are available at the following URL:

https://www.obj-sys.com/docs/acv72/CCppHTML/ch08s04.html

 

No Comments

Adding Information to the J2735 Specification

As some background, see the following article:

http://dsrc-tools.com/map-spat/index.php/knowledge-base/regional-data-extensions-generic-lane-object/

This article discusses how to add some information to the J2735 specification.

This specific blog post discusses how to add information for a new region to the J2735 specification, generate code for it using ASN1C, and then reference the new material in your own code. In this example we will add a GenericLane definition for a new region named Mars. That’s Mars as in the planet, not the town in western PA.

To start, the J2735 ASN.1 specification defines a type called a RegionId as follows:

RegionId ::= INTEGER (0..255)
   noRegion     RegionId ::= 0  -- Use default supplied stubs
   addGrpA      RegionId ::= 1  -- USA
   addGrpB      RegionId ::= 2  -- Japan
   addGrpC      RegionId ::= 3  -- EU
   -- NOTE: new registered regional IDs will be added here
   -- The values 128 and above are for local region use

This definition is in the DSRC module in the ASN.1 file. If you read the background material (you did, didn’t you?), you know that the DSRC module is not to be modified by users of the J2735 specification.

But the background material also indicates that there is a module named REGION in the ASN.1 file that can be modified by developers. So let’s make the first line of this module look like this:

marsRegion RegionId ::= 222

The background material says to let SAE know what number you’ve chosen so it can be tracked. Since there is no intention that I know of to define intelligent traffic systems on the planet Mars, this number 222 isn’t actually being reported to SAE and remains available for anyone else to use.

Now a few lines down let’s modify the Reg-GenericLane definition so it looks like this:

Reg-GenericLane           DSRC.REG-EXT-ID-AND-TYPE ::= {
   { Mars.GenericLane-mars IDENTIFIED BY marsRegion },
   ...  }

Then at the end of the ASN.1 file let’s add a module named Mars that looks like this:

-- ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-
-- 
-- Begin module: Mars
-- 
-- ^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-^-
 
Mars DEFINITIONS AUTOMATIC TAGS::= BEGIN

GenericLane-mars ::= SEQUENCE {
   laneType LaneType,
   distanceFromViking1 INTEGER, -- Since this is a distance, it could be constrained to be positive
   hue RedHue,
   ...
}

LaneType ::= ENUMERATED {
   martian-lane (1),
   rover-lane (2),
   ...
}

RedHue ::= ENUMERATED {
   light-red (1),
   deep-red (2),
   ...
}

END

That’s it as far as the ASN.1 changes are concerned. Now let’s generate some C++ code using ASN1C with a command like this, assuming our changed ASN.1 definitions are in a file named J2735Mars.asn:

asn1c j2735Mars.asn -uper -cpp -table-unions -genwriter -genreader -genprint -pdu GenericLane -gentest

Let’s look at what those qualifiers do:

  • -uper: Specifies that we’re using unaligned PER (Packed Encoding Rules).
  • -table-unions: Specifies that the code generation should take into account inter-dependencies within the ASN.1 definitions. These inter-dependencies are known as table constraints.
  • -genwriter: Specifies that we want to generate a writer program (writer.cpp) that will encode an instance of our message.
  • -genreader: Specifies that we want to generate a reader program (reader.cpp) that will decode an instance of our message.
  • -genprint: Specifies that we want to generate methods to print the contents of the message.
  • -pdu GenericLane: Specifies that our writer and reader programs are to work with an instance of GenericLane (as opposed to any of the other definitions in the ASN.1 file).
  • -gentest: Specifies that the writer is to generate random test data to populate our GenericLane instance.

You can also add -genmake to generate a makefile that will build the writer and reader binaries.

If you generate the code and then build it, you will have a writer binary that will encode the message into a file named message.dat, and you will have a reader binary that will decode the message that’s in the message.dat file. For my specific situation my message looked like this:

GenericLane {
   laneID = 252
   name = 'AMXrH7pmSZHULHTJ:Us_gUAqMrI'
   ingressApproach = 1
   egressApproach = 3
   laneAttributes {
      directionalUse = { 2, 01xxxxxx }
      sharedWith = { 10, 0x72 01xxxxxx }
      laneType {
         sidewalk = { 16, 0x48 0x61 }
      }
      regional {
         regionId = 254
         regExtValue = 0x04083677ccd172888708
      }
   }
   maneuvers = { 12, 0x5E 0100xxxx }
   nodeList {
      computed {
         referenceLaneId = 181
         offsetXaxis {
            large = -9096
         }
         offsetYaxis {
            small = 253
         }
         rotateXY = 18514
         scaleXaxis = 1124
         scaleYaxis = -1034
         regional[0] {
            regionId = 119
            regExtValue = 0x040831403e5e9dfbcb99
         }
      }
   }
   connectsTo[0] {
      connectingLane {
         lane = 189
         maneuver = { 12, 0x53 0100xxxx }
      }
      remoteIntersection {
         region = 16975
         id = 25828
      }
      signalGroup = 245
      userClass = 0
      connectionID = 70
   }
   overlays[0] = 9
   regional[0] {
      regionId = 222
      regExtValue {
         GenericLane-mars {
            laneType = martian-lane
            distanceFromViking1 = 3814
            hue = light-red
         }
      }
   }
}

Keep in mind that this data is randomly generated test data, and it will be different with each code generation. Note the “regional” section at the end, which is where we added our information about lanes on Mars. The test data generator chose our Mars definition, probably because it’s the only one there. But you can see values for the fields we added for traffic lanes on Mars.

The writer.cpp file calls the genTestInstance() method of the GenericLanePDU object, which is an instance of the class ASN1C_GenericLane. The code for this genTestInstance() method is in the generated file DSRCTest.cpp. The end of the method is where the new items are given values.

No Comments

Canonical OER

With ASN1C v7.1 it’s possible to encode and decode OER in C according to the canonical OER rules. Canonical OER can be utilized by setting the ASN1CANON flag in the context before the encoding or decoding is done:

rtxCtxtSetFlag(&ctxt, ASN1CANON);

The Canonical sample in c/sample_oer illustrates using this flag.

With the upcoming ASN1C v7.2 it will be possible to generate canonical OER code directly by using the new -coer qualifier to the ASN1C command (or by choosing the new COER option in the ASN1C GUI). Setting the flag in the context will still enable canonical rules for code generated with -oer instead of -coer. Also in v7.2 it will be possible to generate C++ code for OER instead of just C code.

No Comments