Showing posts with label web development. Show all posts
Showing posts with label web development. Show all posts

Friday, July 07, 2023

Migrating away from Angular flex-layout

In a blog post from October 22, 2022 the Angular team announced

Layout has made significant advancements since Angular’s early days. Based on advancements to native layout solutions and removal of support for IE11, the Angular team will stop publishing new releases of the experimental @angular/flex-layout library starting in v15.

They then proceeded to layout migration alternatives.

  • CSS Flexbox
  • CSS Grid
  • TailwindCSS

This has resulted in considerable consternation for developers.  I do not wish to restate what others have already said about the problem and will instead simply point you to a good article on the topic entitled, Why the Deprecation of Flex-Layouts Is Concerning for Angular Developers. What I intend for this article is provide a simple and cleaner migration alternative off of the @angular/flex-layout package. The migration is actually a step back in time to utilize a set of CSS Flexbox classes defined for AngularJS and I need to give a shout-out to my team member, Pam Tingiris, for remembering and formulating this solution.

I am a solutions architect over a team of developers. During the past few years we have developed a number of Angular applications that are farily large and involved. When we started we were obviously using an older version of Angular without the @angular/flex-layout package. We then followed the Angular team's recommendation to utilize the @angular/flex-layout package so we migrated to it. While the migration was time consuming and tedious just due to the volume of changes to make, it was actually fairly straight-forward and was simply the inverse of what I'm recommended for you today. Fortunately, in our code repository we were able to go back and find the old AngularJS flex layout classes that we had previously migrated away from and deleted. I'm included it at the end of this post.

The migration is mostly a straight conversion from the @angular/flex-layout injected DOM stylings to CSS classes. For example, let's look at the initial example provided on the @angular/flex-layout GitHub page with an addition of a fxFlex div child:

<div fxLayout="row" fxLayoutAlign="space-between">
  <div fxFlex></div>
</div>

With the CSS classes, that simply becomes:

<div class="layout-row layout-align-space-between">
  <div class="flex"></div>
</div>

As you can see, it's a pretty straight-forward process. When you review the CSS at the bottom of the page, you will see there are classes for most every layout variation you might use. For example, fxLayoutAlign="space-between center" becomes class="layout-align-space-between-center" and fxFlex="50" becomes class="flex-50".

For the most part, we standardized our layout gaps to 1rem (example fxLayoutGap="1rem") so we added a single layout-gap class at the end of the original CSS file (.layout-gap { gap: 1rem }), but for the few occasions when we needed different spacing in a component we'd just add a custom class in the component's CSS such as .layout-gap-3rem { gap: 3rem }.

Now, you might be thinking to yourself, that's all well and good for basic flex layout directives, but what about this part of flex-layout?

The real power of Flex Layout, however, is its responsive engine. The Responsive API enables developers to easily specify different layouts, sizing, visibilities for different viewport sizes and display devices.

Here's where it's a little harder, but still not terrible. We'll utilize the Material CDK layout package to create a BreakpointObserver service.

You inject this service into any component that needs media queries for responsive displays.

  constructor(
    public breakpoints: BreakpointsService,
  ) { }

Now, if you have a page that is laid out in rows, but you want it to switch to columns for a phone display, you might have something like:

<div fxLayout="row" fxLayout.xs="column" fxLayoutAlign="space-between">
</div>

Utilizing the breakpoints, this will become:

<div class="layout layout-align-space-between"> [ngClass]="{'layout-column' : breakpoints.screen('xs')}">
</div>

Admittedly not as nice as the concise fxLayout.xs, but it does the job. The built in breakpoint names of the BreakpointObserver like XSmall don't correspond to the flex-layout names like xs, so we named them to match. Our BreakpointsService looks like:

/********************************************************************************
 * BreakpointsService (a BreakpointObserver)
 *
 * Implementation of the Material CDK BreakpointObserver
 * see https://material.angular.io/cdk/layout/overview#breakpointobserver
 * This defines a set of viewport breakpoints (media queries) that allow us
 * to react to changes in viewport sizes for a responsive UI.
 ********************************************************************************/

import { Injectable, OnDestroy } from '@angular/core';
import { BreakpointObserver, Breakpoints} from '@angular/cdk/layout';
import { takeUntil } from 'rxjs/operators';
import { BehaviorSubject, Subject } from 'rxjs';

@Injectable({
  providedIn: 'root'
})
export class BreakpointsService implements OnDestroy {
  destroyed = new Subject<void>();

  mediaQueryMap = new Map([
    ['xs', Breakpoints.XSmall ],
    ['sm', Breakpoints.Small],
    ['md', Breakpoints.Medium],
    ['lg', Breakpoints.Large],
    ['xl', Breakpoints.XLarge],
    ['gt-xs', '(min-width: 600px)'],
    ['gt-sm', '(min-width: 960px)'],
    ['gt-md', '(min-width: 1280px)'],
  ]);

  protected matchesSubject = new BehaviorSubject([]);
  matches = this.matchesSubject.asObservable();

  constructor(public breakpointObserver: BreakpointObserver) {
    breakpointObserver
      .observe([
        Breakpoints.XSmall,
        Breakpoints.Small,
        Breakpoints.Medium,
        Breakpoints.Large,
        Breakpoints.XLarge,
        '(min-width: 600px)',
        '(min-width: 960px)',
        '(min-width: 1280px)',
      ])
      .pipe(takeUntil(this.destroyed))
      .subscribe(result => {
        const matchedBreakpoints = Object.entries( result.breakpoints ).filter( e => e[1] );
        this.matchesSubject.next(matchedBreakpoints);
      });
  }

  ngOnDestroy() {
    this.destroyed.next();
    this.destroyed.complete();
  }

  /***
   * Screen sizes for responsive styling
   * @param size 'xs', 'gt-xs'
   * */
  screen(size) {
    let screenSize = false;
    const matches = this.matches.subscribe(m => {

      const mq = this.mediaQueryMap.get(size);
      screenSize = m.filter( match => match[0] === mq ).length === 1;

    } );
    return screenSize;
  }

}

And that's it! Now it's just a process of finding all the "fx" directives and any screen size rules, and swapping them out for classes. Once you remove the FlexLayoutModule from your app.module.ts, you'll quickly discover anything you might have missed.

import { FlexLayoutModule } from '@angular/flex-layout';

@NgModule({
  imports: [
    FlexLayoutModule,

In conclusion, we found this migration path to be largely a 1-to-1 replacement. We've seen so many other packages come into favor and then fall out of favor over time (like bootstrap) that we didn't want to go down the route of switching to TailwindCSS just to have it ripped out from under us again in a few years. Hopefully you find this to be a helpful alternative.

Flex Layout CSS classes

/* flex layout classes - instructions https://material.angularjs.org/latest/layout/introduction */

.layout-align, .layout-align-start-stretch {
  justify-content: flex-start;
  align-content: stretch;
  align-items: stretch
}

.layout-align-start, .layout-align-start-center, .layout-align-start-end, .layout-align-start-start, .layout-align-start-stretch {
  justify-content: flex-start
}

.layout-align-center, .layout-align-center-center, .layout-align-center-end, .layout-align-center-start, .layout-align-center-stretch {
  justify-content: center
}

.layout-align-end, .layout-align-end-center, .layout-align-end-end, .layout-align-end-start, .layout-align-end-stretch {
  justify-content: flex-end
}

.layout-align-space-around, .layout-align-space-around-center, .layout-align-space-around-end, .layout-align-space-around-start, .layout-align-space-around-stretch {
  justify-content: space-around
}

.layout-align-space-between, .layout-align-space-between-center, .layout-align-space-between-end, .layout-align-space-between-start, .layout-align-space-between-stretch {
  justify-content: space-between
}

.layout-align-space-evenly, .layout-align-space-evenly-center, .layout-align-space-evenly-end, .layout-align-space-evenly-start, .layout-align-space-evenly-stretch {
  justify-content: space-evenly
}

.layout-align-center-start, .layout-align-end-start, .layout-align-space-around-start, .layout-align-space-between-start, .layout-align-start-start, .layout-align-space-evenly-start {
  align-items: flex-start;
  align-content: flex-start
}

.layout-align-center-center, .layout-align-end-center, .layout-align-space-around-center, .layout-align-space-between-center, .layout-align-start-center, .layout-align-space-evenly-center {
  align-items: center;
  align-content: center;
  max-width: 100%
}

.layout-align-center-center > *, .layout-align-end-center > *, .layout-align-space-around-center > *, .layout-align-space-between-center > *, .layout-align-start-center > *, .layout-align-space-evenly-center > * {
  max-width: 100%;
  box-sizing: border-box
}

.layout-align-center-end, .layout-align-end-end, .layout-align-space-around-end, .layout-align-space-between-end, .layout-align-start-end, .layout-align-space-evenly-end {
  align-items: flex-end;
  align-content: flex-end
}

.layout-align-center-stretch, .layout-align-end-stretch, .layout-align-space-around-stretch, .layout-align-space-between-stretch, .layout-align-start-stretch, .layout-align-space-evenly-stretch {
  align-items: stretch;
  align-content: stretch
}

.flex {
  flex: 1
}

.flex, .flex-grow {
  box-sizing: border-box
}

.flex-grow {
  flex: 1 1 100%
}

.flex-initial {
  flex: 0 1 auto;
  box-sizing: border-box
}

.flex-auto {
  flex: 1 1 auto;
  box-sizing: border-box
}

.flex-none {
  flex: 0 0 auto;
  box-sizing: border-box
}

.flex-noshrink {
  flex: 1 0 auto;
  box-sizing: border-box
}

.flex-nogrow {
  flex: 0 1 auto;
  box-sizing: border-box
}

.flex-0, .layout-row > .flex-0 {
  flex: 1 1 100%;
  max-width: 0;
  max-height: 100%;
  box-sizing: border-box
}

.layout-row > .flex-0 {
  min-width: 0
}

.layout-column > .flex-0 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 0%;
  box-sizing: border-box
}

.flex-5, .layout-row > .flex-5 {
  flex: 1 1 100%;
  max-width: 5%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-5 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 5%;
  box-sizing: border-box
}

.flex-10, .layout-row > .flex-10 {
  flex: 1 1 100%;
  max-width: 10%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-10 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 10%;
  box-sizing: border-box
}

.flex-15, .layout-row > .flex-15 {
  flex: 1 1 100%;
  max-width: 15%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-15 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 15%;
  box-sizing: border-box
}

.flex-20, .layout-row > .flex-20 {
  flex: 1 1 100%;
  max-width: 20%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-20 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 20%;
  box-sizing: border-box
}

.flex-25, .layout-row > .flex-25 {
  flex: 1 1 100%;
  max-width: 25%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-25 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 25%;
  box-sizing: border-box
}

.flex-30, .layout-row > .flex-30 {
  flex: 1 1 100%;
  max-width: 30%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-30 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 30%;
  box-sizing: border-box
}

.flex-35, .layout-row > .flex-35 {
  flex: 1 1 100%;
  max-width: 35%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-35 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 35%;
  box-sizing: border-box
}

.flex-40, .layout-row > .flex-40 {
  flex: 1 1 100%;
  max-width: 40%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-40 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 40%;
  box-sizing: border-box
}

.flex-45, .layout-row > .flex-45 {
  flex: 1 1 100%;
  max-width: 45%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-45 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 45%;
  box-sizing: border-box
}

.flex-50, .layout-row > .flex-50 {
  flex: 1 1 100%;
  max-width: 50%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-50 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 50%;
  box-sizing: border-box
}

.flex-55, .layout-row > .flex-55 {
  flex: 1 1 100%;
  max-width: 55%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-55 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 55%;
  box-sizing: border-box
}

.flex-60, .layout-row > .flex-60 {
  flex: 1 1 100%;
  max-width: 60%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-60 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 60%;
  box-sizing: border-box
}

.flex-65, .layout-row > .flex-65 {
  flex: 1 1 100%;
  max-width: 65%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-65 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 65%;
  box-sizing: border-box
}

.flex-70, .layout-row > .flex-70 {
  flex: 1 1 100%;
  max-width: 70%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-70 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 70%;
  box-sizing: border-box
}

.flex-75, .layout-row > .flex-75 {
  flex: 1 1 100%;
  max-width: 75%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-75 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 75%;
  box-sizing: border-box
}

.flex-80, .layout-row > .flex-80 {
  flex: 1 1 100%;
  max-width: 80%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-80 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 80%;
  box-sizing: border-box
}

.flex-85, .layout-row > .flex-85 {
  flex: 1 1 100%;
  max-width: 85%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-85 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 85%;
  box-sizing: border-box
}

.flex-90, .layout-row > .flex-90 {
  flex: 1 1 100%;
  max-width: 90%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-90 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 90%;
  box-sizing: border-box
}

.flex-95, .layout-row > .flex-95 {
  flex: 1 1 100%;
  max-width: 95%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-95 {
  max-height: 95%
}

.flex-100, .layout-column > .flex-95 {
  flex: 1 1 100%;
  max-width: 100%;
  box-sizing: border-box
}

.flex-100 {
  max-height: 100%
}

.layout-column > .flex-100, .layout-row > .flex-100 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 100%;
  box-sizing: border-box
}

.flex-33 {
  max-width: 33.33%
}

.flex-33, .flex-66 {
  flex: 1 1 100%;
  max-height: 100%;
  box-sizing: border-box
}

.flex-66 {
  max-width: 66.66%
}

.layout-row > .flex-33 {
  flex: 1 1 33.33%
}

.layout-row > .flex-66 {
  flex: 1 1 66.66%
}

.layout-column > .flex-33 {
  flex: 1 1 33.33%
}

.layout-column > .flex-66 {
  flex: 1 1 66.66%
}

.layout-row > .flex-33 {
  max-width: 33.33%
}

.layout-row > .flex-33, .layout-row > .flex-66 {
  flex: 1 1 100%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-row > .flex-66 {
  max-width: 66.66%
}

.layout-row > .flex {
  min-width: 0
}

.layout-column > .flex-33 {
  max-height: 33.33%
}

.layout-column > .flex-33, .layout-column > .flex-66 {
  flex: 1 1 100%;
  max-width: 100%;
  box-sizing: border-box
}

.layout-column > .flex-66 {
  max-height: 66.66%
}

.layout-column > .flex {
  min-height: 0
}

.layout, .layout-column, .layout-row {
  box-sizing: border-box;
  display: flex
}

.layout-column {
  flex-direction: column
}

.layout-row {
  flex-direction: row
}

.layout-padding-sm > *, .layout-padding > .flex-sm {
  padding: 4px
}

.layout-padding, .layout-padding-gt-sm, .layout-padding-gt-sm > *, .layout-padding-md, .layout-padding-md > *, .layout-padding > *, .layout-padding > .flex, .layout-padding > .flex-gt-sm, .layout-padding > .flex-md {
  padding: 8px
}

.layout-padding-gt-lg > *, .layout-padding-gt-md > *, .layout-padding-lg > *, .layout-padding > .flex-gt-lg, .layout-padding > .flex-gt-md, .layout-padding > .flex-lg {
  padding: 16px
}

.layout-margin-sm > *, .layout-margin > .flex-sm {
  margin: 4px
}

.layout-margin, .layout-margin-gt-sm, .layout-margin-gt-sm > *, .layout-margin-md, .layout-margin-md > *, .layout-margin > *, .layout-margin > .flex, .layout-margin > .flex-gt-sm, .layout-margin > .flex-md {
  margin: 8px
}

.layout-margin-gt-lg > *, .layout-margin-gt-md > *, .layout-margin-lg > *, .layout-margin > .flex-gt-lg, .layout-margin > .flex-gt-md, .layout-margin > .flex-lg {
  margin: 16px
}

.layout-wrap {
  flex-wrap: wrap
}

.layout-nowrap {
  flex-wrap: nowrap
}

.layout-fill {
  margin: 0;
  width: 100%;
  min-height: 100%;
  height: 100%
}

.layout-gap { gap: 1rem }

Friday, January 06, 2023

Reporting errors to Slack from Java Spring Boot and .NET Framework applications

Watching server logs for errors is a cumbersome way to find problems in server code.  

Back when I was actively doing Ruby on Rails for personal development, I used the "exceptional" gem which would report exceptions to a now defunct online service (exceptional.io) to monitor any errors that occurred in your application.  It was a nice tool, but it had 2 drawbacks for me.  

  1. It was just for Ruby on Rails
  2. It required a 3rd party site to store the exception data.  

At the time, I was working at IBM so I needed a tool that worked for Java applications, and they didn't look kindly on potentially confidential information being stored outside of the IBM network, so I created my own version of Exception that I named IBMExceptional (you can read a little bit about it and see some screenshots on my IBM portfolio page).  The site for monitoring the errors was a Rails application, but I created a Java client for it that provided 2 reporting options.  

First, I provided an IBMExceptional exception class that could be extended for application specific exceptions.     

        class MyProjectException extends IBMExceptional {
            @SuppressWarnings("unused")
            public MyProjectException() {
                super();
            }

            public MyProjectException(String message, Throwable cause) {
                super(message, cause);
            }

            public MyProjectException(String message) {
                super(message);
            }

            @SuppressWarnings("unused")
            public MyProjectException(Throwable cause) {
                super(cause);
            }
        }

This base exception class would report any instances of thrown exceptions to IBMExceptional.

        try {
            throw new MyProjectException("This is a test exception.");
        } catch (MyProjectException e) {
            // Handle exception as desired, but it was reported to IBM Exceptional
        }

The advantage of this option is that the exception reporting is transparent and automatic.  The disadvantage is that only exceptions that are subclassed from the IBMExceptional base class will be reported. 

Second, I provided a log appender, IBMExceptionalAppender, so that any logged errors would be reported to IBMExceptional.  You added a few configuration lines to log4j.properties like

        log4j.rootCategory=info, exceptional  
        log4j.appender.exceptional=com.ibm.IBMExceptionalAppender

Then calls to the logger with an exception would be reported to IBMExceptional.

        try {
            throw new NullPointerException("Fake Null pointer");
        } catch (NullPointerException e) {
            logger.error("Error description", e);
        }

The advantage of this option is that all exception types can be reported.  The disadvantage is that it requires that the exceptions are logged by the programmer.

This worked well and was all well and good while I remained at IBM, but I have since moved on.  For my current job at USIC, I've led the development of a Java API server using Spring Boot (feeding Angular UIs) and a C# .NET (Framework) application and decided I would similarly like to easily monitor when errors occur on the servers, but this time, instead of having some web site to monitor the errors, I thought it would be much simpler and be sufficient to just send them to dedicated Slack channels.  

For Java we went the route of a log appender, for .NET we could tie into the System.Web.HttpApplication Application_Error handler.

Java Spring Boot application with log appender

Spring Boot uses logback rather than log4j, so in logback-spring.xml we defined a new appender for errors.

	<appender name="SLACK" class="com.usicllc.jsonapi.log.SlackAppender">
		<profile>${profile}</profile>
		<filter class="ch.qos.logback.classic.filter.LevelFilter">
			<level>ERROR</level>
			<onMatch>ACCEPT</onMatch>
			<onMismatch>DENY</onMismatch>
		</filter>
	</appender>

Then the SlackAppender class utilizes the JSlack library and looks like:

public class SlackAppender extends AppenderBase<ILoggingEvent> {
	@Override
	protected void append(ILoggingEvent event) {

		String errorMessage = event.getFormattedMessage();
		StringBuffer message = new StringBuffer("*Error: " + errorMessage + "*\n\n");
		String att = null;

		Map<String, String> mdcMap = event.getMDCPropertyMap();
		if (mdcMap != null) {
			String reqUrl = mdcMap.get("REQUEST_URL");
			String reqMethod = mdcMap.get("REQUEST_METHOD");
			String reqQueryString = mdcMap.get("REQUEST_QUERY_STRING");
			String reqUserAgent = mdcMap.get("REQUEST_USER_AGENT");
			String reqRealIp = mdcMap.get("REQUEST_X_REAL_IP");
			String reqUser = mdcMap.get("REQUEST_USER");
			String reqBody = mdcMap.get("REQUEST_BODY");

			if (reqMethod != null) {
				message.append("*REQUEST_METHOD:* " + reqMethod + "\n");
				message.append("*REQUEST_URL:* " + reqUrl + "\n");
				if (reqQueryString != null) {
					try {
						message.append("*REQUEST_QUERY_STRING:* " + URLDecoder.decode(reqQueryString, "UTF-8") + "\n");
					} catch (UnsupportedEncodingException e) {
					}
				}
				if (reqBody != null) {
					message.append("*REQUEST_BODY:* " + reqBody + "\n");
				}
				message.append("*REQUEST_USER_AGENT:* " + reqUserAgent + "\n");
				message.append("*REQUEST_X_REAL_IP:* " + reqRealIp + "\n");
				message.append("*REQUEST_USER:* " + reqUser + "\n");
			}
		}

		// If we have an exception being reported, then add it to the message
		//
		if (event.getThrowableProxy() != null) {
			StackTraceElementProxy[] stArray = event.getThrowableProxy().getStackTraceElementProxyArray();
			if (stArray.length > 0) {
				// first 10 lines of the stack trace are added to the messages as a code block
				// Full stack trace is also added as an attachment
				message.append("*Stack Trace:*\n```");
				String exceptionMessage = event.getThrowableProxy().getMessage();
				StringBuffer stackTrace = new StringBuffer("*Error Message: " + exceptionMessage + "*\n");

				for (int i = 0; i < stArray.length; i++) {
					StackTraceElement t = stArray[i].getStackTraceElement();
					String traceLine = t.toString();
					if (i < 10) {
						// 1st 10 lines are added to the main message in a code block
						message.append(traceLine + "\n");
					}
					if (traceLine.startsWith("com.usicllc.jsonapi")) {
						// emphasize where the error was in our code
						stackTrace.append("*" + traceLine + "*\n");
					} else {
						stackTrace.append(traceLine + "\n");
					}
				}

				if (stArray.length > 10) {
					message.append("...\n");

					// Full stack trace is more than the 10 lines shown (usually the case)
					// Add the full stacktrace as an attachment
					Attachment attachment = Attachment.builder()
							.title("Full Stack Trace")
							.text(stackTrace.toString())
							.build();
					att = "[{\"title\" :\""+attachment.getTitle()+":\n"+"\",\"text\": \""+attachment.getText().replace("\\", "\\\\")+"\"}]";
				}
				message.append("```");
			}

		}
		String payload = "{\"channel\": \"" + this.getSlackChannel() 
			+ "\",\"username\": \"jsonapi\",\"text\": \""
			+ message.toString().replace("\\", "\\\\") 
			+ "\",\"attachments\":" + att + "}";

		WebhookResponse webhookResponse;
		try {
			webhookResponse = Slack.getInstance().send(
				this.getSlackUrl(), payload);
			if (webhookResponse.getCode() != 200) {
				logger.debug("Unable to send Slack Exception - http error:" + webhookResponse.toString());
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}

The stack trace is added both with the first 10 lines in a code block in the message body and then the full stack trace as an attachment because its more readable as a code block due to no line wrapping and the use of a monospaced font, but Slack will limit the message size and most stack traces will exceed the limit so adding the full stack trace an attachment provides access to everything albeit in a less readable format.  

Now, it could have just logged the exception message and stack trace, but it's certainly more helpful to know what request caused the problem and that's where we took advantage of the logback MDC (Mapped Diagnoatic Context) to retrieve request data in a Spring GenericFilterBean named PreRequestProcessingFilter Spring will inject this into the request filter chain and the filter code looks like:

@Component
public class PreRequestProcessingFilter extends GenericFilterBean implements CurrentUser {
	Logger logger = LoggerFactory.getLogger(PreRequestProcessingFilter.class);

	@SuppressWarnings("unchecked")
	@Override
	public void doFilter(ServletRequest req, ServletResponse res, FilterChain filterChain) {
		try {
			String bodyJson = null;
			RequestWrapper wrapper = null;
			String method = ((HttpServletRequest) req).getMethod();
			String contentType = ((HttpServletRequest) req).getContentType();

			// Get a copy of the request body for JSON POST and PATCH requests
			if (contentType != null &&
					(contentType.startsWith("application/json") || contentType.startsWith("application/vnd.api+json")) &&
					(method.equalsIgnoreCase("POST") || method.equalsIgnoreCase("PATCH"))) {
				wrapper = new RequestWrapper((HttpServletRequest) req);
				byte[] body = StreamUtils.copyToByteArray(wrapper.getInputStream());
				Map<String, Object> jsonRequest = new ObjectMapper().readValue(body, Map.class);
				bodyJson = jsonRequest.toString();
			}

			String user = getUserName();
			StringBuffer requestURL = ((HttpServletRequest) req).getRequestURL();
			if (requestURL != null) {
				MDC.put("REQUEST_URL", requestURL.toString());
			}

			MDC.put("REQUEST_METHOD", method);
			MDC.put("REQUEST_QUERY_STRING", ((HttpServletRequest) req).getQueryString());
			MDC.put("REQUEST_USER_AGENT", ((HttpServletRequest) req).getHeader("User-Agent"));
			MDC.put("REQUEST_X_REAL_IP", ((HttpServletRequest) req).getHeader("X-Real-IP"));
			MDC.put("REQUEST_USER", user);
			MDC.put("REQUEST_BODY", bodyJson);
			filterChain.doFilter((wrapper != null ? wrapper : req), res);

		} catch (IOException | ServletException e) {
			e.printStackTrace();
		} finally {
			MDC.clear();
		}
	}
}

The big trick in that filter is obtaining the POST or PATCH request body without consuming it and thereby prevening the regular request processing from getting it. Normally, you can only read the request body once. So we create a RequestWrapper and a ServletInputStreamWrapper which allow us to extract a copy of the request's input stream (body). The code for these classes looks like:

public class RequestWrapper extends HttpServletRequestWrapper {

    private byte[] body;

    public RequestWrapper(HttpServletRequest request) throws IOException {
        super(request);

        this.body = StreamUtils.copyToByteArray(request.getInputStream());
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        return new ServletInputStreamWrapper(this.body);

    }
}


public class ServletInputStreamWrapper extends ServletInputStream {
    private InputStream inputStream;

    public ServletInputStreamWrapper(byte[] body) {
        this.inputStream = new ByteArrayInputStream(body);
    }

    @Override
    public boolean isFinished() {
        try {
            return inputStream.available() == 0;
        } catch (Exception e) {
            return false;
        }
    }

    @Override
    public boolean isReady() {
        return true;
    }

    @Override
    public void setReadListener(ReadListener listener) {
        // no need to implment this method
    }

    @Override
    public int read() throws IOException {
        return this.inputStream.read();
    }
}

With all this in place, we now get nice Slack messages for errors providing request data and stack trace.  The final result looks like:



.NET Framework Application with Application_Error handler

For the .NET Framework application its much simpler. In our Global.asax.cs file (where we create our application subclass of System.Web.HttpApplication) we define the Application_Error hook which allows us access to the request data, and then we simply send it to Slack utilizing the Slack.Webhooks package.

    public class MvcApplication : System.Web.HttpApplication
    {
        private static readonly bool slackEnabled = Convert.ToBoolean(ConfigurationManager.AppSettings["slackEnabled"]);
        private static readonly string slackURL = ConfigurationManager.AppSettings["slackURL"];
        private static readonly string slackChannel = ConfigurationManager.AppSettings["slackChannel"];
        private static readonly SlackClient slackClient = new SlackClient(slackURL);     
        private static readonly string environment = ConfigurationManager.AppSettings["environmentMode"];

        protected void Application_Error(object sender, EventArgs e)
        {
            Exception ex = null;
            // make sure whatever we do here doesn't generate any exceptions and generate a loop
            try
            {
                ex = Server.GetLastError();
                
                string requestType = HttpContext.Current.Request.RequestType;
                string requestURL = Convert.ToString(HttpContext.Current.Request.Url);
                string requestReferrer = Convert.ToString(HttpContext.Current.Request.UrlReferrer);
                string requestBrowser = HttpContext.Current.Request.Browser.Browser + " " + HttpContext.Current.Request.Browser.Version;
                string requestPlatform = HttpContext.Current.Request.Browser.Platform;
                bool requestIsAuthenticated = HttpContext.Current.Request.IsAuthenticated;
                string requestUserAgent = HttpContext.Current.Request.UserAgent;
                string currentUser = User.Identity.Name?.Split('\\')[1];
                string requestUserAddress = HttpContext.Current.Request.UserHostAddress;
                string requestForwardedForAddress = HttpContext.Current.Request.Headers.Get("X-Forwarded-For");

                Stream requestInputStream = HttpContext.Current.Request.InputStream;
                requestInputStream.Seek(0, SeekOrigin.Begin);
                string requestBody = new StreamReader(requestInputStream).ReadToEnd();

                string logInfoString =
                      "\n   User: " + currentUser
                    + "\n   Request type: " + requestType
                    + "\n   Request URL: " + requestURL
                    + "\n   Referrer: " + requestReferrer
                    + "\n   Browser: " + requestBrowser
                    + "\n   Platform: " + requestPlatform
                    + "\n   UserAgent: " + requestUserAgent
                    + (requestIsAuthenticated ? "" : ("\n   IsAuthenticated: " + requestIsAuthenticated))
                    + "\n   Source address: " + requestUserAddress + (string.IsNullOrEmpty(requestForwardedForAddress) ? "" : 
                      (" / " + requestForwardedForAddress));

                string logInfoString = requestInfoString +
                      (string.IsNullOrEmpty(requestBody) ? "" : ("\n   Request body: " + requestBody));

                log.Error(logInfoString, ex);

                SendSlackMessage("Application Error:", requestInfoString, ex, requestBody);

            }
            catch (Exception exc) {
                // If we have issues getting any of the info above, at least log the exception
                try
                {
                    log.Error("Original exception: ", ex);
                    log.Error("Additionally, there was an exception in Application_Error: " + exc);
                }
                catch (Exception) { }      
            }
        }

        public static void SendSlackMessage(string title, string message, Exception exception, string requestBody)
        {
            // don't send to slack if disabled or running locally
            if (!slackEnabled || environment == "local") return;

            // add time and make the title bold so that messages posted in quick succession can be easily told apart
            // (*stuff* makes it bold)
            string slackText = "*" + DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss") + (title != null ? (" - " + title) : "") + "*";

            if (message != null)
            {
                slackText += "\n" + message;
            }

            List<SlackAttachment> slackAttachments = new List<SlackAttachment>();

            if (!string.IsNullOrWhiteSpace(requestBody))
            {
                var requestBodyAttachment = new SlackAttachment
                {
                    Text = requestBody
                };

                slackAttachments.Add(requestBodyAttachment);
            }

            if (exception != null)
            {
                var slackAttachment = new SlackAttachment
                {
                    Text = exception.ToString(),
                    Color = "#C20202"
                };

                slackAttachments = new List<SlackAttachment> { slackAttachment };
            }

            // local defaults. Comment out first line of the method to test locally if needed
            string slackUser = ".NET Application";
            string slackUserEmoji = Emoji.Wrench;

            var slackMessage = new SlackMessage
            {
                Channel = slackChannel,
                Text = slackText,
                IconEmoji = slackUserEmoji,
                Username = slackUser,
                Attachments = slackAttachments
            };

            slackClient.PostAsync(slackMessage);
        }



Monday, March 25, 2019

Activity LEDs

In some of my applications I get ongoing data from websockets and I want some visual indication that the websocket is connected and receiving data, so I added an LED indicator much like a network router or switch would have.

Whereever I want the LED, I add div to show the LED.

<div class="led" id="ws-led">
<div class="led-red-off">
</div>
</div>

then add some CSS rules to make it look like an LED indicator

div.led{
 display     : inline-block;
 vertical-align    : bottom;
}

div.led-red-off,
div.led-red-on,
div.led-green-off,
div.led-green-on {
 border     : 0;
 border-radius    : 50%;
 height     : 1em;
 width     : 1em;
 vertical-align    : middle;
 background-repeat   : no-repeat;
 display     : inline-block;
}

div.led-red-on {
 background    : #F44336; opacity: 1;
}
div.led-green-on {
 background    : #4CAF50; opacity: 1;
}
div.led-red-off {
 background    : #F44336; opacity: .5;
}
div.led-green-off {
 background    : #4CAF50; opacity: .5;
}

and finally, some JS code to make them blink.

    var wsLED = document.getElementById('ws-led');

    // connected to the Websocket server (general)
    function connected(greeting) {
        // change the LED from initial red to green
        if (wsLED) wsLED.firstChild.setAttribute("class", "led-green-off");
    }

    // connection to Websocket server lost
    function disconnected() {
        // if we loose the websocket connection, change the LED to red
        if (wsLED) wsLED.firstChild.setAttribute("class", "led-red-on");
    }

    /**
     * logWSActivity
     *
     * log some WS activity
     */
    function logWSActivity(...args) {
        console.log(...args);
        flashLED(wsLED);
    }

    /**
     * flashLEDs
     *
     * The visual effect of the flashing LEDs is done by switching the
     * CSS class for a 10th of a second per activity.  If a new
     * activity happens within that time, the timeout to turn it back
     * off is reset to a new 10th of a second.
     */
    var ledOffTimeout;
    function flashLED(led) {
        // flash activity LED
        if (led) {
            if (ledOffTimeout) { clearTimeout(ledOffTimeout); }
            led.firstChild.setAttribute("class", "led-green-on");
            ledOffTimeout = setTimeout(function() {
                led.firstChild.setAttribute("class", "led-green-off"); }, 100);
        }
    }

JavaScript String functions (toCamelCase, dasherize, and titleize)

I had the need for a few String functions in JavaScript to manage some variances between data sources so I figured I'd share them.  For example, from one source attributes in the JSON data would use underscores to separate words and in another it would uses dashes (my_attribute vs my-attribute).  Also for simple data display, I wanted to titleize the attribute name for a label (My Attribute). 


/**
 * String.toCamelCase
 *
 * Convert a string to camel case, including hyphens and underscores
 */
String.prototype.toCamelCase = function() {
    return this.replace(/^([A-Z])|[\s-_](\w)/g, function(match, p1, p2, offset) {
        if (p2) return p2.toUpperCase();
        return p1.toLowerCase();        
    });
};

/**
 * String.dasherize
 *
 * Dasherize a string, including periods and underscores
 */
if (!String.prototype.dasherize) {
    String.prototype.dasherize = function() {
        return this.replace(/^([A-Z])|[\s\._](\w)/g, function(match, p1, p2, offset) {
            if (p2) return "-" + p2.toLowerCase();
            return p1.toLowerCase();        
        });
    };
}

/**
 * String.capitalize
 *
 * Capitalize the first letter of a string
 */
if (!String.prototype.capitalize) {
    String.prototype.capitalize = function() {
        return this.charAt(0).toUpperCase() + this.slice(1);
    };
}

/**
 * String.titleize
 *
 * Capitalize the first letter of every word in a string, also separates
 * words by spaces if formally separated by dashes or underscores
 *
 */
if (!String.prototype.titleize) {
    String.prototype.titleize = function() {
        return this.replace(/^([A-Z])|[\s-_](\w)/g, function(match, p1, p2, offset) {
            if (p2) return " " + p2.toUpperCase();
            return p1.toLowerCase();        
        }).capitalize();
    };
}

Deep Copy version of Javascript Object.assign

I was working on some Redux work and needed a reducer that would merge in some sparse updates to the current state of an object.

If you're learning Redux you may be familiar with the tutorial example of a TODO list item where it's changing one attribute of the TODO list:

return Object.assign({}, state, {visibilityFilter: action.filter});

Now, instead of changing a single top-level attribute like the visibility filter, assume you have some data that needs to be merged into the existing object both at the top level and in some nested attributes.  For example, presume your current state looks like:

{ a : "a", deep: { a: "a" }

and you get new data for that state that need to be merged in that looks like:

{ b : "b", deep: { b: "b" } }

The Object.assign function will do a shallow copy merge and the result will be:

> Object.assign({}, { a : "a", deep: { a: "a" } }, { b : "b", deep: { b: "b" } })
{ a: 'a', deep: { b: 'b' }, b: 'b' }

The new value of deep simply replaced the first value.  The nested value of deep wasn't merged together.  The method below will correctly merge nested values as follows:

> deepAssign({}, { a : "a", deep: { a: "a" } }, { b : "b", deep: { b: "b" } })
{ a: 'a', deep: { a: 'a', b: 'b' }, b: 'b' }

Here's the code:

function deepAssign(...objs) {
  const target = objs.shift();
  const source = objs.shift();

  if (source) {
    if (source instanceof Array) {
      for (const element of source) {
        if (element instanceof Array) {
          target.push(deepAssign([], element));
        } else if (element instanceof Object) {
          target.push(deepAssign({}, element));
        } else {
          target.push(element);
        }
      }
    } else {
      for (const attribute in source) {
        // eslint-disable-next-line no-prototype-builtins
        if (source.hasOwnProperty(attribute) && source[attribute] !== undefined) {
          if (source[attribute] instanceof Array) {
            target[attribute] = target[attribute] || [];
            for (const element of source[attribute]) {
              if (element instanceof Array) {
                target[attribute].push(deepAssign([], element));
              } else if (element instanceof Object) {
                target[attribute].push(deepAssign({}, element));
              } else {
                target[attribute].push(element);
              }
            }
          } else if (source[attribute] instanceof Object) {
            if (source[attribute].toString() === '[object Object]') {
              // simple data object so deep copy it
              target[attribute] = deepAssign((typeof target[attribute] === 'object') ? target[attribute] : {}, source[attribute]);
            } else {
              // instance of some class, so just copy over the object
              target[attribute] = source[attribute];
            }
          } else {
            target[attribute] = source[attribute];
          }
        }
      }
    }
  }

  if (objs.length > 0) {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    return deepAssign(target, ...objs);
  } else {
    return target;
  }
};


Update 7/15/19 : Original version didn't handle Arrays correctly.  They become objects.  They now copy correctly including nested array and object in the arrays.  You can't just use the spread operator (newArray = [...oldArray]) to copy it or it wouldn't copy the nested objects as new objects

Update 8/27/21 : Objects that weren't simple data objects were being lost, so now complex objects are brought over to the new copy.

Tuesday, April 05, 2016

Re-enable disabled disable_with fields

I have some forms that do JavaScript based validation before submitting to the server.  The submit button is using the disable_with construct, so when the submit button is pressed, it is correctly disabled, but when the client-side validation fails, it's stuck disabled.

To remedy this situation, I added these lines to the JavaScript code that hides the error messages after they've been displayed for 5 seconds.

// re-enable 'disable_with' fields after validation errors
var disabled_with = jQuery('[data-disable-with][disabled]');
if (disabled_with) {
    jQuery.rails.enableElement(disabled_with);
    disabled_with.val(function(){ return jQuery(this).text()}).prop('disabled', false);
}

The first line is the commonly proposed solution that I found elsewhere online, but when I tried it (in Chrome), I found that it didn't work, leaving the button disabled with the disabled text displayed. However, one important thing it did was to insert the original button text as HTML content of the input element.  I added the second line to use that text as the button value and then actually enable the button.


Saturday, January 16, 2016

Setup of Ruby Development Environment VM

I recently needed to provide some development environment setup instructions to a friend, so I figured it would be a good idea to document them here in case I need them again or if it helps anyone else.

Let me start off with a few premises:

  • I like doing Ruby on Rails development.
  • Ruby development on Windows is not pretty.
  • The computer I'm using most often is Windows-based (provided for my "real" job which is not Ruby development).
  • My personal computers are Macs.
  • I want to be able to work from any computer but don't want to hassle with keeping the environments in sync.
  • I've used multiple flavors of Linux over the years back to the SLS days (Linux version 0.9x), and am very comfortable with it.
With those premises in mind, I've adopted the use of a virtual machine (VM) to be my development environment.  It allows me to separate my personal development environment from other clutter (or even probing by workplace compliance scans), and allows me to readily move it from machine to machine as necessary.  I also usually host my applications on Linux-based servers, so it provides me more confidence in an environment that matches the deployment environment.

I personally don't like living in a VM window to work.  I also feel that I'm already running a windows manager (Windows or OSX) on my host operating system, so why incur the overhead of running another one in the VM.  Therefore, I run a "headless" Linux in the VM, and open X windows on the host by running a simple X server on the host OS.

Step 1 - Setup an X server on the host OS

For Windows, I recommend VcXsrv (see http://sourceforge.net/projects/vcxsrv/), but if you're already using Cygwin for other UNIX applications, the Cygwin Xserver (see http://x.cygwin.com/docs/ug/setup-cygwin-x-installing.html) works well, or alternatively you could try Xming (http://sourceforge.net/projects/xming/files/latest/download).

For OSX, I use XQuartz (see http://www.xquartz.org/)

Step 2 - Install VMWare (or VirtualBox, Parallels, etc.)

I'm not going to provide instructions for every virtualization software out there.  I use VMWare so that's what I'm describing.  Other solutions work just as well.

For Windows, VMWare Player does everything you need, and it's free!

For OSX, there's no free player, so you have to purchase VMWare Fusion.  (but, hey, it's the only thing you need to buy!).

Step 3 - Download a Linux distribution ISO

Again, there are plenty of options (Ubuntu, Red Hat, SuSE, etc), but I'm using Ubuntu here as it is frequently used for hosting.  You want the Server version (not Desktop) as it will be more light-weight without all the X11 + window manager overhead.  Also, I recommend the LTS (longer term service) version so you don't have to deal with upgrading your OS all the time (besides security fixes of course).

Step 4 - Create a New Virtual Machine

Select the VMWare option to create a new virtual machine, point it to the ISO and it should be pretty straight forward.

You don't have to be excessive with the resources you allocate for the VM.  The default recommendations by VMWare will probably suffice, but if you have a little more to give, then bump it up a little.  For example, the VMWare recommendation for the memory is 1GB, my system has 16GB so I gave the VM 2GB.  I allowed it 4 out of the 8 processor cores, and give it up to 40GB of disk space.

Step 5 - Forward local ports to VM

While this is not strictly necessary, it is useful.

For Windows, in earlier versions of VMWare Player, the Virtual Network Editor was included, but you needed to know the right command to access it.  In case the current omission is unintentional and they add the network editor back in future releases, the method to access the network editor for VMPlayer was:

  1. Open command prompt as administrator
  2. cd "C:\Program Files (x86)\VMware\VMware Player"
  3. rundll32.exe vmnetui.dll VMNetUI_ShowStandalone

As it stands currently, that will give you an error that the vmnetui.dll is not found.  So instead, you need to extract the Virtual Network Editor from the VMWare Workstation package.  This is done by:

  1. Download the latest VMWare Workstation package.
  2. unpack it locally with this command :
    "
    VMWare-worksation-full-10.0.1-xxxxxx.exe /e .\ext"
    note the xxxxxx is the release id you got at download
  3. Go to the newly created "ext" directory and open the "core.cab" (use your favorite zip util)
  4. Get the "vmnetcfg.exe" from there, and copy it to the vmwareplayer install directoryGet the "_vmnetcfglib.dll" file and rename it "vmnetcfglib.dll", and then copy it to the same directory than for vmnetcfg.exe
OR  

Download vmnetcfg.exe and vmnetcfglib.dll and put them in your VMWare Player install directory.

Once you have the Virtual Network Editor, select the NAT adapter (should be VMnet8), click on the "NAT Settings..." button, and add whatever ports you'll be using as shown below.  In this example, my VM IP is 192.168.216.131 and I'm forwarding ports 22 for SSH and 3000 as the default Rails server port.  Add port 80 if you'll be running your apps through a normal HTTP server like Apache or nginx (ex. via Passenger).




For Macs using VMWare Fusion, edit the vmnet8 NAT config file using the terminal and your favorite editor

sudo emacs /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf

or
 
sudo vim /Library/Preferences/VMware\ Fusion/vmnet8/nat.conf
 
 
Near the bottom you’ll see something like this
 
[incomingtcp]
 
# Use these with care - anyone can enter into your VM through these...
# The format and example are as follows:
# = :
#8080 = 172.16.3.128:80
 
This is where we’ll be putting the port forwarding.  We'll assuming the same VM IP and that we still want to forward to the Rails server port 3000 and SSH port 22.   Since OSX is likely already running an SSH server on port 22, we'll forward from port 2222 instead.  So, add the lines:

3000 = 192.168.216.131:3000
2222 = 192.168.216.131:22 


Restart VMWare Fusion and it should be forwarding to these ports for you.  Alternatively, you could restart the VMWare networking using the command line:


sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --stop
sudo /Applications/VMware\ Fusion.app/Contents/Library/vmnet-cli --start


Optionally, if you want to ensure your VM always gets the same IP address, you can edit /Library/Preferences/VMware\ Fusion/vmnet8/dhcpd.conf and add a clause to the bottom with the format of:


host <some-name> {
    hardware ethernet <MAC-ADDRESS>;
    fixed-address <IP-address>;
}

You can determine the VMWare assigned MAC address for your VM by looking in your VM's *.vmx file and look for a line like:

ethernet0.generatedAddress = "00:0c:29:42:f0:54"

Then to assign a static IP address of 192.168.216.131 to this VM, you could add the clause:

host ubuntu {
    hardware ethernet 00:0c:29:42:f0:54;
    fixed-address
192.168.216.131;
}


Remember to restart the Fusion Networking services either by restarting Fusion or using the command line option.
Most of the OSX instructions came from http://encyclopediaofdaniel.com/blog/fusion-dhcp-port-forwarding/ where you can also find information on a Ruby gem to make the changes for you.

Step 6 - Connect using SSH with X forwarding

X11 forwarding needs to be enabled on both the client side and the server side.

On the client side, the -X (capital X) option to ssh enables X11 forwarding, and you can make this the default (for all connections or for a specific conection) with ForwardX11 yes in ~/.ssh/config.

On the server side, X11Forwarding yes must specified in /etc/ssh/sshd_config. Note that the default is no forwarding (some distributions turn it on in their default /etc/ssh/sshd_config), and that the user cannot override this setting.

The xauth program must be installed on the server side. If there are any X11 programs there, it's very likely that xauth will be there. In the unlikely case xauth was installed in a nonstandard location, it can be called through ~/.ssh/rc (on the server!).

Note that you do not need to set any environment variables on the server. DISPLAY and XAUTHORITY will automatically be set to their proper values. If you run ssh and DISPLAY is not set, it means ssh is not forwarding the X11 connection.

To confirm that ssh is forwarding X11, check for a line containing Requesting X11 forwarding in the ssh -v -X output. Note that the server won't reply either way.

(credit to http://unix.stackexchange.com/questions/12755/how-to-forward-x-over-ssh-from-ubuntu-machine)

Tuesday, January 07, 2014

Bishop Swap Puzzle Fixed!

I implemented a Bishop Swap Puzzle back in 2007 (see my prior post), but it has had a bug in it where bishops of the same color could pass through each other.  I had noted it as a 'TODO' in my code comments while I was writing it, but I had neglected to go back and fix it and forgot all about it until I was trying it out again recently.

I'm happy to say that I've now fixed the bug and it works correctly, so happy gaming!


Once you've completed the Bishop Swap puzzle, be sure to check out the others!

Monday, October 29, 2012

attachment_fu as a gem for Rails 3.2

As I've mentioned before, I develop the web site for St. Francis Society Animal Rescue.  As I've also described in that earlier article, it started as a Rails 1.2 app, then Rails 2.1, then 2.3.  Right now it's in Rails 3.1, but I'm about to switch it to Rails 3.2.  Having some history, it was developed to use attachment_fu to upload all the images for the cats and dogs.  While I know many people have abandoned attachment_fu for paperclip, or carrierwave, or dragonfly, etc. attachment_fu has continued to work just fine for me so if it ain't broke, don't fix it.  Plus, there are currently over 13,000 animal images that have been uploaded, and I don't really want to hassle with converting them over to a new attachment system.

Enter Ruby 1.9.x

The last official update to the attachment_fu github repository was on April 25, 2009.  While that update is for Ruby 1.9 compatibility fixes, if you try to use it as is for Ruby 1.9.3, you'll find it won't work.  To that end, I've forked off a new repository that will fix additional Ruby 1.9.3 incompatibilities.  If you want to continue to use attachment_fu as a plugin with Ruby 1.9.x and Rails prior to 3.2, you can use my github repository in your project simply by issuing this command in your project:

git submodule add https://github.com/pothoven/attachment_fu.git vendor/plugins/attachment_fu

Enter Rails 3.2.x

attachment_fu has always functioned as a plugin (vendor/plugins), the problem is that Rails 3.2 will give you this error if you continue to use it as a plugin:

DEPRECATION WARNING: You have Rails 2.3-style plugins in vendor/plugins! Support for these plugins will be removed in Rails 4.0. 
Move them out and bundle them in your Gemfile, or fold them in to your app as lib/myplugin/* and config/initializers/myplugin.rb. 
See the release notes for more on this: http://weblog.rubyonrails.org/2012/1/4/rails-3-2-0-rc2-has-been-released.

To that end, I've also updated my fork of attachment_fu to function as a gem! Simply add this line to your Gemfile
 
gem 'pothoven-attachment_fu'

I need to acknowledge Christophe Porteneuve for doing most of the gem conversion work. I just pulled in his updates and fixed a few problems with it that I encountered. Please feel free to let me know if you have any problems using the gem.

Thursday, October 04, 2012

Self-marking required fields in Rails

David Sulc wrote a nice article entitled, Self-marking required fields in Rails 3 that utilized a key element of Ryan Bates' Railscast on Validations in Rails 3.  In Ryan's original solution, he creates a new helper method named mark_required that uses the Rails validation reflection to determine if the model requires that the field to be present (via validates :presence => true or validates_presence_of).  If the field is required, this helper method adds a '*', otherwise it does nothing. Using this helper method in your code, your code would look like:

<%= f.label :name %><%= mark_required(@user, :name) %>
<%= f.text_field :name %>  

David improves upon this idea by enhancing the ActionView::Helpers::FormBuilder label helper instead of adding a new helper.  This eliminates the need to add the extra call to the mark_required helper for every label, which can be easily forgotten.  If the field is required, the enhanced label helper appends a "*" to the label text to indicate that the field is required.  So now your code is back to the normal form of:
<%= f.label :name %>
<%= f.text_field :name %>  

That's a nice improvement, but I thought it would be an even better design practice to simply give the label a required CSS class, and then allow the designer to be able to easily modify how required fields should be indicated.   My version of config/initializers/form_builder.rb looks like:

class ActionView::Helpers::FormBuilder
  alias :orig_label :label

  # add a 'required' CSS class to the field label if the field is required
  def label(method, content_or_options = nil, options = nil, &block)
    if content_or_options && content_or_options.class == Hash
      options = content_or_options
    else
      content = content_or_options
    end

    if object.class.validators_on(method).map(&:class).include? ActiveModel::Validations::PresenceValidator
      if options.class != Hash
        options = {:class => "required"}
      else
        options[:class] = ((options[:class] || "") + " required").split(" ").uniq.join(" ")
      end
    end

    self.orig_label(method, content, options || {}, &block)
  end
end
With this revised version in place, all labels of required fields are given a required CSS class.  If you want to indicate required fields with an asterisk after the label, you can do that by adding this CSS rule to your stylesheet,

/* add required field asterisk */
label.required:after {
    content: " *";
}
However, by extracting the method of indicating that the field is required from being hard-coded in the helper to instead using a CSS class, anything you can do with CSS is open to you and can be quickly and easily modified without updating the helper code.  You can change colors, fonts, borders, add images before or after, you could even use the dreaded text-decoration:blink.  The options are nearly endless.

Update for Rails 4! 


Thanks to the input from alex_m and Dan in the comments below, the follow revised version should work for Rails 4 (with ActiveAdmin):

class ActionView::Helpers::FormBuilder
  alias :orig_label :label

  # add a 'required' CSS class to the field label if the field is required
  def label(method, content_or_options = nil, options = nil, &block)
    if content_or_options && content_or_options.class == Hash
      options = content_or_options
    else
      content = content_or_options
    end

    if object.class.respond_to?(:validators_on) &&
      object.class.validators_on(method).map(&:class).include?(ActiveRecord::Validations::PresenceValidator)

      if options.class != Hash
        options = {:class => "required"}
      else
        options[:class] = ((options[:class] || "") + " required").split(" ").uniq.join(" ")
      end
    end

    self.orig_label(method, content, options || {}, &block)
  end
end

Thanks again alex_m and Dan!

Thursday, May 10, 2012

Rails Rumble 2010

In my long absence from posting, I see I totally skipped 2010.
For the 2010 Rails Rumble we created Commendable Kids.

Here are the results for our entry CommendableKids.com:
  • Finished 4th place out of 180 teams
  • Winners of the Appearance Category
  • Runners Up from Chargify as Most Potential to Monetize
    Read More
  • #1 team from the U.S.
  • Tech Crunch Top 5
    Read the Review

St. Francis Society Animal Rescue

One of my side-projects is the development of the St. Francis Society Animal Rescue web site.  My friend Brian Burridge and I originally converted the site from a static web site to a Ruby on Rails site several years ago (2008 timeframe).  I believe it was initially a Rails 1.2 project.  Besides the public site, it includes a fairly involved back-end administration component that beyond just allowing content management of the web site, performs all the animal rescue administration functions (detailed animal information with health records, adoption records, etc.).  At the time we were both fairly new to Ruby on Rails and we decided to use ActiveScaffold to build this administration component.

At some point we upgraded to Rails 2.1 and then in 2010, Brian left the project to be able to better focus on his other numerous projects and I further upgraded it to Rails 2.3.x.  These Rails version upgrades were more effort than a typical Rails upgrades may be due to various gem dependencies, most specifically ActiveScaffold.  ActiveScaffold is really a pretty nice framework for admin sites, but it had limitations that required work-arounds and those work-arounds often didn't work when upgrading.

During the end of 2011 to the beginning of 2012, I did a more drastic upgrade.  I migrated to Rails 3.1.x.  However, this wasn't just a simple migration, I decided to re-write the entire application.  The most involved part of this re-write is what most people will never see, the administration area.  I decided to totally abandon ActiveScaffold.  Brian told me about ActiveAdmin that he was using on some of his other projects, but after fighting with ActiveScaffold for long, I opted to stay clear of such a major framework dependency and just wrote the entire admin from scratch with straight Rails.

Comparisons of the old and new administration pages.

Here's a comparison of the listing of the cats.  I reduced the amount of information displayed on the list to reduce some clutter, and I've made some of the frequently changed values available to be changed directly on the list (save instantly via Ajax) to eliminate the need to open the edit form just to change a status.


Filtering the list required an extension to ActiveScaffold that was problematic to upgrade, and it resulted in a very large area added to the top of the list.  Now it uses a jQuery UI dialog.


ActiveScaffold constructed its forms very vertically which didn't utilize the space well.  I now have full control of the layout allowing me to organize things better.



Responsive Web Design

Last summer at the 2011 front-end design conference I had the privilege of having Ethan Marcotte introduce me to the idea of responsive web design.  So, I took that to heart and designed both the public site and the admin area to be responsive.  So, if we look at the cat listing page again and compare it between a browser and an iPhone, you can see that the navigation menu has collapsed and the table has dropped several columns.

There are also intermediate changes for tablets, but it's time I wrap this up.

I'm only touching on a very few of the page layouts and features.  Beyond a new look and feel, the re-write of the application also brought performance improvements.  Here is a New Relic report on the week I switched it over from the old to the new.

Note: Thursday was the transition day, so it should be ignored.  The jump in CPU percentage is due to also switching from a shared hosting environment on DreamHost to a Linode slice since DreamHost doesn't support Rails 3.1 at this time -- at least not on my server)




I encourage you to take a look at the St. Francis Society Animal Rescue public site, particularly if you're in the Tampa Bay area and are interested in a new pet.  Be sure to try it out in different sizes to see how the responsive layout works, and please let me know what you think of it.

Adding and removing multiple associated sub-models dynamically on a single form in Rails

The last couple blog posts have been about PeepNote ... its beginning and its ending.  Now I thought I'd share a snippet of code that I developed for it the form of a simple contact manager.

One of the things Brian pointed out in his concluding remarks about PeepNote that I referred to in my last post was,

As I began interacting more with potential customers I realized that my target audience was not what I thought it was. It wasn’t people like me who were heavily using Twitter for career networking and wanted to keep track of how I met people and what I knew about them. Instead, the only people that would pay for the service were companies. Companies that wanted to use it to track potential customers; a CRM.
Part of the CRM functionality that we had incorporated into PeepNote was to add contact information about your peeps.  The idea of a contact manager is well understood by most audiences and are thus commonly used as a demonstration application.  So, what's special about this one and why I'm I writing about it?

This particular contact manager demonstrates (and improves upon) Ryan Bates “Handle Multiple Models in One Form” recipe (#13) from the Advanced Rails Recipes book.  In Ryan’s original recipe, he added discrete field values (individual tasks on a to-do list) to the form. Any tasks that did not have an identifier associated with it was a new task, and any missing task identifiers from those currently in the database were assumed to be deleted.  This is great when you're essentially adding single values to a list, but what about adding multiple pieces of information that need to stay together as a single entity (like the parts of a contact information - street address, city, state, phone, etc)?

The solution is to use a JavaScript variable to assign temporary unique identifiers to newly added nested models (addresses, phone numbers, urls, etc). These newly added models are assigned a negative value in order for the contact controller to distinguish between new and existing records.  So, new records with a negative id are added, existing records are updated as necessary, and any records with ids that are no longer in the submitted list are removed.

This is one of the problems I solved for PeepNote and have extracted into a simple stand-alone Rails application.   Rather than provide a lengthy description here, I placed the source code on GitHub and have a running demo on Heroku.

PeepNote conclusion

I realize it's been a very long time since my last post, but I am still around.

In regards to that last post, I thought I'd share a blog entry by my RailsRumble teammate discussing the status of PeepNote:

PeepNote: The Rumble, the Startup, and now…the Conclusion 

(http://brianburridge.com/2012/03/12/peepnote-the-conclusion/)