Monday, May 13, 2019

2's complement and BCD hex values in JavaScript

I'm sure this is will not be a highly accessed post as these are not a commonly required functions, but in my current job, we process data from IoT GPS devices which send data in compact formats.  To process this data using Node.js in an AWS Lambda function, I had the need to convert a 2's complement hex values and BCD (binary-coded decimal) hex value to regular numbers in JavaScript.  The examples I found were in Java, so I converted it to JavaScript and am sharing here in case anyone else needs it.

       const negativeHexTestPattern = /^[89ABCDEF]/i


        /* parse signed 2's complement hex string to number */
        hexToTwosComplement(hex) {
                let result = parseInt(hex, 16)

                // Check if high bit is set.
                if (negativeHexTestPattern.test(hex)) {
                        // Negative number
                        const subtrahend = (2 ** (hex.length * 4))
                        result -= subtrahend
                }
                return result
        }


        /* parse packed binary-coded decimal (BCD) format where unused
         * trailing digits (4-bits) are filled with all ones (1111). */
        hexBCDToString(hex) {
                let decoded = ''
                for (let i = 0; i < hex.length; i++) {
                        if ((parseInt(hex[i],16) & 0x0F) !== 0x0F) {
                                decoded += hex[i]
                        } else {
                                break
                        }
                }
                return decoded
        }

Here's a few test cases to demonstrate.
                it('converts positive hex values to twos complement', () => {
                        const val = new Processor().hexToTwosComplement('13f6b4eb')
                        chai.expect(val).to.equal(334935275)
                })

                it('converts negative hex values to twos complement', () => {
                        const val = new Processor().hexToTwosComplement('ec094b15')
                        chai.expect(val).to.equal(-334935275)
                })

                it('converts negative 2 byte hex values to twos complement', () => {
                        const val = new Processor().hexToTwosComplement('ffae')
                        chai.expect(val).to.equal(-82)
                })

Wednesday, March 27, 2019

Templated version of JavaScript Object.assign

In a recent article I provided a revised version of the JavaScript Object.assign function that also merged nested objects.  In this article, I'm going to revise it slightly so that it only assigns attributes according to a template.  The deepAssign method is useful for merging in partial data, some of which may be nested, into a larger set of (state) data.   The templateAssign is useful for pruning the data you're going to store to only what you need / care about.

Why would you need to do this?


In GraphQL you can specify the attributes you want so probably don't need to, but in other techniques (like json:api) you get the full object which may contain way more data than you need, and if the data set it large it can consume a lot of memory unnecessarily.  In the application this function is derived from, we have a lot of data and I've seen quite a few "Out of memory" errors in our application monitoring.  So eliminating unnecessary data is valuable.

The standard Object.assign (and the Object.deepAssign) method will combine all the object attributes, but if the incoming data contains extra attributes you don't care about, it can be handy to prune it to only what you care about.  This templateAssign function will accept a template object as the first argument and only copy over attributes from the additional source objects that are defined in the template.

If this is to be used in a map-reduction system like Redux, be sure to clone the template object into a new object via Object.assign.

Example:
Let's assume we only want attribute "a" either at the top level or a nested level

> const template =  { a: undefined, deep: { a: undefined } }
> const obj1 = { a : "a", deep: { a: "a" } }
> const obj2 = { b : "b", deep: { a: "b", b: "b" } }
> const obj3 = { c : "c", deep: { b: "b", c: "c" } }

Standard assign, doesn't merge nested "deep" object, and keeps all (top-level) attributes
> Object.assign({}, obj1, obj2, obj3)
{ a: 'a', deep: { b: 'b', c: 'c' }, b: 'b', c: 'c' }

deepAssign, merges everything (top-level and nested objects)
> Object.deepAssign({}, obj1, obj2, obj3)
{ a: 'a', deep: { a: 'b', b: 'b', c: 'c' }, b: 'b', c: 'c' }

templateAssign, merges all levels, but only keeps attributes in the template
> Object.templateAssign(Object.assign({}, template), obj1, obj2, obj3)
{ a: 'a', deep: { a: 'b' } }

Here's the code:

if (!Object.prototype.templateAssign) {
    Object.prototype.templateAssign = function(...objs) {
        let target = objs.shift();
        let source = objs.shift();
        
        if (source) {
            for(const attribute in source) {
                if (attribute in source &&  typeof(source[attribute]) === "object") {
                    target[attribute] = Object.templateAssign(Object.assign({}, target[attribute] || {}), source[attribute]);
                } else if (attribute in target &&
                           source.hasOwnProperty(attribute) &&
                           source[attribute] !== undefined) {
                    target[attribute] = source[attribute];
                }
            }
        }
        if (objs.length > 0) {
            return Object.templateAssign(target, ...objs);
        } else {
            return target;
        }
    };
}

Tuesday, March 26, 2019

Make NVM work like RVM

I've used RVM (Ruby Version Manager) for years and it has a great feature of automatically switching your Ruby version as you navigate to to project folders to use the Ruby version specified for that project. For NVM (Node Version Manager) you have to manually tell nvm to switch node versions via nvm use.    If you add the following code to your bash configuration, nvm will switch automatically like rvm does when it finds a .nvmrc file.  (note: I am not the original author of this code.  I tweaked it from another source, but I don't recall where that was).

# fix NVM to work like RVM
#
find-up () {
    path=$(pwd)
    while [[ "$path" != "" && ! -e "$path/$1" ]]; do
        path=${path%/*}
    done
    echo "$path"
}

cdnvm(){
    cd $@;
    nvm_path=$(find-up .nvmrc | tr -d '[:space:]')
 
    # If there are no .nvmrc file, use the default nvm version
    if [[ ! $nvm_path = *[^[:space:]]* ]]; then
  
        declare default_version;
        default_version=$(nvm version default);
  
        # If there is no default version, set it to `node`
        # This will use the latest version on your machine
        if [[ $default_version == "N/A" ]]; then
            nvm alias default node;
            default_version=$(nvm version default);
        fi
  
        # If the current version is not the default version, set it to use the default version
        if [[ $(nvm current) != "$default_version" ]]; then
            nvm use default;
        fi
  
    elif [[ -s $nvm_path/.nvmrc && -r $nvm_path/.nvmrc ]]; then
        declare nvm_version
        nvm_version=$(<"$nvm_path"/.nvmrc)
  
        # Add the `v` suffix if it does not exists in the .nvmrc file
        if [[ $nvm_version != v* ]]; then
            nvm_version="v""$nvm_version"
        fi
  
        # If it is not already installed, install it
        if [[ $(nvm ls "$nvm_version" | tr -d '[:space:]') == "N/A" ]]; then
            nvm install "$nvm_version";
        fi
  
        if [[ $(nvm current) != "$nvm_version" ]]; then
            nvm use "$nvm_version";
        fi
    fi
}
alias cd='cdnvm'

Monday, March 25, 2019

Activity LEDs

In some of my applications I get ongoing data from websockets and I want some visual indication that the websocket is connected and receiving data, so I added an LED indicator much like a network router or switch would have.

Whereever I want the LED, I add div to show the LED.

<div class="led" id="ws-led">
<div class="led-red-off">
</div>
</div>

then add some CSS rules to make it look like an LED indicator

div.led{
 display     : inline-block;
 vertical-align    : bottom;
}

div.led-red-off,
div.led-red-on,
div.led-green-off,
div.led-green-on {
 border     : 0;
 border-radius    : 50%;
 height     : 1em;
 width     : 1em;
 vertical-align    : middle;
 background-repeat   : no-repeat;
 display     : inline-block;
}

div.led-red-on {
 background    : #F44336; opacity: 1;
}
div.led-green-on {
 background    : #4CAF50; opacity: 1;
}
div.led-red-off {
 background    : #F44336; opacity: .5;
}
div.led-green-off {
 background    : #4CAF50; opacity: .5;
}

and finally, some JS code to make them blink.

    var wsLED = document.getElementById('ws-led');

    // connected to the Websocket server (general)
    function connected(greeting) {
        // change the LED from initial red to green
        if (wsLED) wsLED.firstChild.setAttribute("class", "led-green-off");
    }

    // connection to Websocket server lost
    function disconnected() {
        // if we loose the websocket connection, change the LED to red
        if (wsLED) wsLED.firstChild.setAttribute("class", "led-red-on");
    }

    /**
     * logWSActivity
     *
     * log some WS activity
     */
    function logWSActivity(...args) {
        console.log(...args);
        flashLED(wsLED);
    }

    /**
     * flashLEDs
     *
     * The visual effect of the flashing LEDs is done by switching the
     * CSS class for a 10th of a second per activity.  If a new
     * activity happens within that time, the timeout to turn it back
     * off is reset to a new 10th of a second.
     */
    var ledOffTimeout;
    function flashLED(led) {
        // flash activity LED
        if (led) {
            if (ledOffTimeout) { clearTimeout(ledOffTimeout); }
            led.firstChild.setAttribute("class", "led-green-on");
            ledOffTimeout = setTimeout(function() {
                led.firstChild.setAttribute("class", "led-green-off"); }, 100);
        }
    }

JavaScript String functions (toCamelCase, dasherize, and titleize)

I had the need for a few String functions in JavaScript to manage some variances between data sources so I figured I'd share them.  For example, from one source attributes in the JSON data would use underscores to separate words and in another it would uses dashes (my_attribute vs my-attribute).  Also for simple data display, I wanted to titleize the attribute name for a label (My Attribute). 


/**
 * String.toCamelCase
 *
 * Convert a string to camel case, including hyphens and underscores
 */
String.prototype.toCamelCase = function() {
    return this.replace(/^([A-Z])|[\s-_](\w)/g, function(match, p1, p2, offset) {
        if (p2) return p2.toUpperCase();
        return p1.toLowerCase();        
    });
};

/**
 * String.dasherize
 *
 * Dasherize a string, including periods and underscores
 */
if (!String.prototype.dasherize) {
    String.prototype.dasherize = function() {
        return this.replace(/^([A-Z])|[\s\._](\w)/g, function(match, p1, p2, offset) {
            if (p2) return "-" + p2.toLowerCase();
            return p1.toLowerCase();        
        });
    };
}

/**
 * String.capitalize
 *
 * Capitalize the first letter of a string
 */
if (!String.prototype.capitalize) {
    String.prototype.capitalize = function() {
        return this.charAt(0).toUpperCase() + this.slice(1);
    };
}

/**
 * String.titleize
 *
 * Capitalize the first letter of every word in a string, also separates
 * words by spaces if formally separated by dashes or underscores
 *
 */
if (!String.prototype.titleize) {
    String.prototype.titleize = function() {
        return this.replace(/^([A-Z])|[\s-_](\w)/g, function(match, p1, p2, offset) {
            if (p2) return " " + p2.toUpperCase();
            return p1.toLowerCase();        
        }).capitalize();
    };
}

Deep Copy version of Javascript Object.assign

I was working on some Redux work and needed a reducer that would merge in some sparse updates to the current state of an object.

If you're learning Redux you may be familiar with the tutorial example of a TODO list item where it's changing one attribute of the TODO list:

return Object.assign({}, state, {visibilityFilter: action.filter});

Now, instead of changing a single top-level attribute like the visibility filter, assume you have some data that needs to be merged into the existing object both at the top level and in some nested attributes.  For example, presume your current state looks like:

{ a : "a", deep: { a: "a" }

and you get new data for that state that need to be merged in that looks like:

{ b : "b", deep: { b: "b" } }

The Object.assign function will do a shallow copy merge and the result will be:

> Object.assign({}, { a : "a", deep: { a: "a" } }, { b : "b", deep: { b: "b" } })
{ a: 'a', deep: { b: 'b' }, b: 'b' }

The new value of deep simply replaced the first value.  The nested value of deep wasn't merged together.  The method below will correctly merge nested values as follows:

> deepAssign({}, { a : "a", deep: { a: "a" } }, { b : "b", deep: { b: "b" } })
{ a: 'a', deep: { a: 'a', b: 'b' }, b: 'b' }

Here's the code:

function deepAssign(...objs) {
  const target = objs.shift();
  const source = objs.shift();

  if (source) {
    if (source instanceof Array) {
      for (const element of source) {
        if (element instanceof Array) {
          target.push(deepAssign([], element));
        } else if (element instanceof Object) {
          target.push(deepAssign({}, element));
        } else {
          target.push(element);
        }
      }
    } else {
      for (const attribute in source) {
        // eslint-disable-next-line no-prototype-builtins
        if (source.hasOwnProperty(attribute) && source[attribute] !== undefined) {
          if (source[attribute] instanceof Array) {
            target[attribute] = target[attribute] || [];
            for (const element of source[attribute]) {
              if (element instanceof Array) {
                target[attribute].push(deepAssign([], element));
              } else if (element instanceof Object) {
                target[attribute].push(deepAssign({}, element));
              } else {
                target[attribute].push(element);
              }
            }
          } else if (source[attribute] instanceof Object) {
            if (source[attribute].toString() === '[object Object]') {
              // simple data object so deep copy it
              target[attribute] = deepAssign((typeof target[attribute] === 'object') ? target[attribute] : {}, source[attribute]);
            } else {
              // instance of some class, so just copy over the object
              target[attribute] = source[attribute];
            }
          } else {
            target[attribute] = source[attribute];
          }
        }
      }
    }
  }

  if (objs.length > 0) {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    return deepAssign(target, ...objs);
  } else {
    return target;
  }
};


Update 7/15/19 : Original version didn't handle Arrays correctly.  They become objects.  They now copy correctly including nested array and object in the arrays.  You can't just use the spread operator (newArray = [...oldArray]) to copy it or it wouldn't copy the nested objects as new objects

Update 8/27/21 : Objects that weren't simple data objects were being lost, so now complex objects are brought over to the new copy.