Friday, July 07, 2023

Migrating away from Angular flex-layout

In a blog post from October 22, 2022 the Angular team announced

Layout has made significant advancements since Angular’s early days. Based on advancements to native layout solutions and removal of support for IE11, the Angular team will stop publishing new releases of the experimental @angular/flex-layout library starting in v15.

They then proceeded to layout migration alternatives.

  • CSS Flexbox
  • CSS Grid
  • TailwindCSS

This has resulted in considerable consternation for developers.  I do not wish to restate what others have already said about the problem and will instead simply point you to a good article on the topic entitled, Why the Deprecation of Flex-Layouts Is Concerning for Angular Developers. What I intend for this article is provide a simple and cleaner migration alternative off of the @angular/flex-layout package. The migration is actually a step back in time to utilize a set of CSS Flexbox classes defined for AngularJS and I need to give a shout-out to my team member, Pam Tingiris, for remembering and formulating this solution.

I am a solutions architect over a team of developers. During the past few years we have developed a number of Angular applications that are farily large and involved. When we started we were obviously using an older version of Angular without the @angular/flex-layout package. We then followed the Angular team's recommendation to utilize the @angular/flex-layout package so we migrated to it. While the migration was time consuming and tedious just due to the volume of changes to make, it was actually fairly straight-forward and was simply the inverse of what I'm recommended for you today. Fortunately, in our code repository we were able to go back and find the old AngularJS flex layout classes that we had previously migrated away from and deleted. I'm included it at the end of this post.

The migration is mostly a straight conversion from the @angular/flex-layout injected DOM stylings to CSS classes. For example, let's look at the initial example provided on the @angular/flex-layout GitHub page with an addition of a fxFlex div child:

<div fxLayout="row" fxLayoutAlign="space-between">
  <div fxFlex></div>
</div>

With the CSS classes, that simply becomes:

<div class="layout-row layout-align-space-between">
  <div class="flex"></div>
</div>

As you can see, it's a pretty straight-forward process. When you review the CSS at the bottom of the page, you will see there are classes for most every layout variation you might use. For example, fxLayoutAlign="space-between center" becomes class="layout-align-space-between-center" and fxFlex="50" becomes class="flex-50".

For the most part, we standardized our layout gaps to 1rem (example fxLayoutGap="1rem") so we added a single layout-gap class at the end of the original CSS file (.layout-gap { gap: 1rem }), but for the few occasions when we needed different spacing in a component we'd just add a custom class in the component's CSS such as .layout-gap-3rem { gap: 3rem }.

Now, you might be thinking to yourself, that's all well and good for basic flex layout directives, but what about this part of flex-layout?

The real power of Flex Layout, however, is its responsive engine. The Responsive API enables developers to easily specify different layouts, sizing, visibilities for different viewport sizes and display devices.

Here's where it's a little harder, but still not terrible. We'll utilize the Material CDK layout package to create a BreakpointObserver service.

You inject this service into any component that needs media queries for responsive displays.

  constructor(
    public breakpoints: BreakpointsService,
  ) { }

Now, if you have a page that is laid out in rows, but you want it to switch to columns for a phone display, you might have something like:

<div fxLayout="row" fxLayout.xs="column" fxLayoutAlign="space-between">
</div>

Utilizing the breakpoints, this will become:

<div class="layout layout-align-space-between"> [ngClass]="{'layout-column' : breakpoints.screen('xs')}">
</div>

Admittedly not as nice as the concise fxLayout.xs, but it does the job. The built in breakpoint names of the BreakpointObserver like XSmall don't correspond to the flex-layout names like xs, so we named them to match. Our BreakpointsService looks like:

/********************************************************************************
 * BreakpointsService (a BreakpointObserver)
 *
 * Implementation of the Material CDK BreakpointObserver
 * see https://material.angular.io/cdk/layout/overview#breakpointobserver
 * This defines a set of viewport breakpoints (media queries) that allow us
 * to react to changes in viewport sizes for a responsive UI.
 ********************************************************************************/

import { Injectable, OnDestroy } from '@angular/core';
import { BreakpointObserver, Breakpoints} from '@angular/cdk/layout';
import { takeUntil } from 'rxjs/operators';
import { BehaviorSubject, Subject } from 'rxjs';

@Injectable({
  providedIn: 'root'
})
export class BreakpointsService implements OnDestroy {
  destroyed = new Subject<void>();

  mediaQueryMap = new Map([
    ['xs', Breakpoints.XSmall ],
    ['sm', Breakpoints.Small],
    ['md', Breakpoints.Medium],
    ['lg', Breakpoints.Large],
    ['xl', Breakpoints.XLarge],
    ['gt-xs', '(min-width: 600px)'],
    ['gt-sm', '(min-width: 960px)'],
    ['gt-md', '(min-width: 1280px)'],
  ]);

  protected matchesSubject = new BehaviorSubject([]);
  matches = this.matchesSubject.asObservable();

  constructor(public breakpointObserver: BreakpointObserver) {
    breakpointObserver
      .observe([
        Breakpoints.XSmall,
        Breakpoints.Small,
        Breakpoints.Medium,
        Breakpoints.Large,
        Breakpoints.XLarge,
        '(min-width: 600px)',
        '(min-width: 960px)',
        '(min-width: 1280px)',
      ])
      .pipe(takeUntil(this.destroyed))
      .subscribe(result => {
        const matchedBreakpoints = Object.entries( result.breakpoints ).filter( e => e[1] );
        this.matchesSubject.next(matchedBreakpoints);
      });
  }

  ngOnDestroy() {
    this.destroyed.next();
    this.destroyed.complete();
  }

  /***
   * Screen sizes for responsive styling
   * @param size 'xs', 'gt-xs'
   * */
  screen(size) {
    let screenSize = false;
    const matches = this.matches.subscribe(m => {

      const mq = this.mediaQueryMap.get(size);
      screenSize = m.filter( match => match[0] === mq ).length === 1;

    } );
    return screenSize;
  }

}

And that's it! Now it's just a process of finding all the "fx" directives and any screen size rules, and swapping them out for classes. Once you remove the FlexLayoutModule from your app.module.ts, you'll quickly discover anything you might have missed.

import { FlexLayoutModule } from '@angular/flex-layout';

@NgModule({
  imports: [
    FlexLayoutModule,

In conclusion, we found this migration path to be largely a 1-to-1 replacement. We've seen so many other packages come into favor and then fall out of favor over time (like bootstrap) that we didn't want to go down the route of switching to TailwindCSS just to have it ripped out from under us again in a few years. Hopefully you find this to be a helpful alternative.

Flex Layout CSS classes

/* flex layout classes - instructions https://material.angularjs.org/latest/layout/introduction */

.layout-align, .layout-align-start-stretch {
  justify-content: flex-start;
  align-content: stretch;
  align-items: stretch
}

.layout-align-start, .layout-align-start-center, .layout-align-start-end, .layout-align-start-start, .layout-align-start-stretch {
  justify-content: flex-start
}

.layout-align-center, .layout-align-center-center, .layout-align-center-end, .layout-align-center-start, .layout-align-center-stretch {
  justify-content: center
}

.layout-align-end, .layout-align-end-center, .layout-align-end-end, .layout-align-end-start, .layout-align-end-stretch {
  justify-content: flex-end
}

.layout-align-space-around, .layout-align-space-around-center, .layout-align-space-around-end, .layout-align-space-around-start, .layout-align-space-around-stretch {
  justify-content: space-around
}

.layout-align-space-between, .layout-align-space-between-center, .layout-align-space-between-end, .layout-align-space-between-start, .layout-align-space-between-stretch {
  justify-content: space-between
}

.layout-align-space-evenly, .layout-align-space-evenly-center, .layout-align-space-evenly-end, .layout-align-space-evenly-start, .layout-align-space-evenly-stretch {
  justify-content: space-evenly
}

.layout-align-center-start, .layout-align-end-start, .layout-align-space-around-start, .layout-align-space-between-start, .layout-align-start-start, .layout-align-space-evenly-start {
  align-items: flex-start;
  align-content: flex-start
}

.layout-align-center-center, .layout-align-end-center, .layout-align-space-around-center, .layout-align-space-between-center, .layout-align-start-center, .layout-align-space-evenly-center {
  align-items: center;
  align-content: center;
  max-width: 100%
}

.layout-align-center-center > *, .layout-align-end-center > *, .layout-align-space-around-center > *, .layout-align-space-between-center > *, .layout-align-start-center > *, .layout-align-space-evenly-center > * {
  max-width: 100%;
  box-sizing: border-box
}

.layout-align-center-end, .layout-align-end-end, .layout-align-space-around-end, .layout-align-space-between-end, .layout-align-start-end, .layout-align-space-evenly-end {
  align-items: flex-end;
  align-content: flex-end
}

.layout-align-center-stretch, .layout-align-end-stretch, .layout-align-space-around-stretch, .layout-align-space-between-stretch, .layout-align-start-stretch, .layout-align-space-evenly-stretch {
  align-items: stretch;
  align-content: stretch
}

.flex {
  flex: 1
}

.flex, .flex-grow {
  box-sizing: border-box
}

.flex-grow {
  flex: 1 1 100%
}

.flex-initial {
  flex: 0 1 auto;
  box-sizing: border-box
}

.flex-auto {
  flex: 1 1 auto;
  box-sizing: border-box
}

.flex-none {
  flex: 0 0 auto;
  box-sizing: border-box
}

.flex-noshrink {
  flex: 1 0 auto;
  box-sizing: border-box
}

.flex-nogrow {
  flex: 0 1 auto;
  box-sizing: border-box
}

.flex-0, .layout-row > .flex-0 {
  flex: 1 1 100%;
  max-width: 0;
  max-height: 100%;
  box-sizing: border-box
}

.layout-row > .flex-0 {
  min-width: 0
}

.layout-column > .flex-0 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 0%;
  box-sizing: border-box
}

.flex-5, .layout-row > .flex-5 {
  flex: 1 1 100%;
  max-width: 5%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-5 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 5%;
  box-sizing: border-box
}

.flex-10, .layout-row > .flex-10 {
  flex: 1 1 100%;
  max-width: 10%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-10 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 10%;
  box-sizing: border-box
}

.flex-15, .layout-row > .flex-15 {
  flex: 1 1 100%;
  max-width: 15%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-15 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 15%;
  box-sizing: border-box
}

.flex-20, .layout-row > .flex-20 {
  flex: 1 1 100%;
  max-width: 20%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-20 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 20%;
  box-sizing: border-box
}

.flex-25, .layout-row > .flex-25 {
  flex: 1 1 100%;
  max-width: 25%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-25 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 25%;
  box-sizing: border-box
}

.flex-30, .layout-row > .flex-30 {
  flex: 1 1 100%;
  max-width: 30%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-30 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 30%;
  box-sizing: border-box
}

.flex-35, .layout-row > .flex-35 {
  flex: 1 1 100%;
  max-width: 35%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-35 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 35%;
  box-sizing: border-box
}

.flex-40, .layout-row > .flex-40 {
  flex: 1 1 100%;
  max-width: 40%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-40 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 40%;
  box-sizing: border-box
}

.flex-45, .layout-row > .flex-45 {
  flex: 1 1 100%;
  max-width: 45%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-45 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 45%;
  box-sizing: border-box
}

.flex-50, .layout-row > .flex-50 {
  flex: 1 1 100%;
  max-width: 50%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-50 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 50%;
  box-sizing: border-box
}

.flex-55, .layout-row > .flex-55 {
  flex: 1 1 100%;
  max-width: 55%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-55 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 55%;
  box-sizing: border-box
}

.flex-60, .layout-row > .flex-60 {
  flex: 1 1 100%;
  max-width: 60%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-60 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 60%;
  box-sizing: border-box
}

.flex-65, .layout-row > .flex-65 {
  flex: 1 1 100%;
  max-width: 65%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-65 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 65%;
  box-sizing: border-box
}

.flex-70, .layout-row > .flex-70 {
  flex: 1 1 100%;
  max-width: 70%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-70 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 70%;
  box-sizing: border-box
}

.flex-75, .layout-row > .flex-75 {
  flex: 1 1 100%;
  max-width: 75%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-75 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 75%;
  box-sizing: border-box
}

.flex-80, .layout-row > .flex-80 {
  flex: 1 1 100%;
  max-width: 80%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-80 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 80%;
  box-sizing: border-box
}

.flex-85, .layout-row > .flex-85 {
  flex: 1 1 100%;
  max-width: 85%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-85 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 85%;
  box-sizing: border-box
}

.flex-90, .layout-row > .flex-90 {
  flex: 1 1 100%;
  max-width: 90%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-90 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 90%;
  box-sizing: border-box
}

.flex-95, .layout-row > .flex-95 {
  flex: 1 1 100%;
  max-width: 95%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-column > .flex-95 {
  max-height: 95%
}

.flex-100, .layout-column > .flex-95 {
  flex: 1 1 100%;
  max-width: 100%;
  box-sizing: border-box
}

.flex-100 {
  max-height: 100%
}

.layout-column > .flex-100, .layout-row > .flex-100 {
  flex: 1 1 100%;
  max-width: 100%;
  max-height: 100%;
  box-sizing: border-box
}

.flex-33 {
  max-width: 33.33%
}

.flex-33, .flex-66 {
  flex: 1 1 100%;
  max-height: 100%;
  box-sizing: border-box
}

.flex-66 {
  max-width: 66.66%
}

.layout-row > .flex-33 {
  flex: 1 1 33.33%
}

.layout-row > .flex-66 {
  flex: 1 1 66.66%
}

.layout-column > .flex-33 {
  flex: 1 1 33.33%
}

.layout-column > .flex-66 {
  flex: 1 1 66.66%
}

.layout-row > .flex-33 {
  max-width: 33.33%
}

.layout-row > .flex-33, .layout-row > .flex-66 {
  flex: 1 1 100%;
  max-height: 100%;
  box-sizing: border-box
}

.layout-row > .flex-66 {
  max-width: 66.66%
}

.layout-row > .flex {
  min-width: 0
}

.layout-column > .flex-33 {
  max-height: 33.33%
}

.layout-column > .flex-33, .layout-column > .flex-66 {
  flex: 1 1 100%;
  max-width: 100%;
  box-sizing: border-box
}

.layout-column > .flex-66 {
  max-height: 66.66%
}

.layout-column > .flex {
  min-height: 0
}

.layout, .layout-column, .layout-row {
  box-sizing: border-box;
  display: flex
}

.layout-column {
  flex-direction: column
}

.layout-row {
  flex-direction: row
}

.layout-padding-sm > *, .layout-padding > .flex-sm {
  padding: 4px
}

.layout-padding, .layout-padding-gt-sm, .layout-padding-gt-sm > *, .layout-padding-md, .layout-padding-md > *, .layout-padding > *, .layout-padding > .flex, .layout-padding > .flex-gt-sm, .layout-padding > .flex-md {
  padding: 8px
}

.layout-padding-gt-lg > *, .layout-padding-gt-md > *, .layout-padding-lg > *, .layout-padding > .flex-gt-lg, .layout-padding > .flex-gt-md, .layout-padding > .flex-lg {
  padding: 16px
}

.layout-margin-sm > *, .layout-margin > .flex-sm {
  margin: 4px
}

.layout-margin, .layout-margin-gt-sm, .layout-margin-gt-sm > *, .layout-margin-md, .layout-margin-md > *, .layout-margin > *, .layout-margin > .flex, .layout-margin > .flex-gt-sm, .layout-margin > .flex-md {
  margin: 8px
}

.layout-margin-gt-lg > *, .layout-margin-gt-md > *, .layout-margin-lg > *, .layout-margin > .flex-gt-lg, .layout-margin > .flex-gt-md, .layout-margin > .flex-lg {
  margin: 16px
}

.layout-wrap {
  flex-wrap: wrap
}

.layout-nowrap {
  flex-wrap: nowrap
}

.layout-fill {
  margin: 0;
  width: 100%;
  min-height: 100%;
  height: 100%
}

.layout-gap { gap: 1rem }

Friday, January 06, 2023

Reporting errors to Slack from Java Spring Boot and .NET Framework applications

Watching server logs for errors is a cumbersome way to find problems in server code.  

Back when I was actively doing Ruby on Rails for personal development, I used the "exceptional" gem which would report exceptions to a now defunct online service (exceptional.io) to monitor any errors that occurred in your application.  It was a nice tool, but it had 2 drawbacks for me.  

  1. It was just for Ruby on Rails
  2. It required a 3rd party site to store the exception data.  

At the time, I was working at IBM so I needed a tool that worked for Java applications, and they didn't look kindly on potentially confidential information being stored outside of the IBM network, so I created my own version of Exception that I named IBMExceptional (you can read a little bit about it and see some screenshots on my IBM portfolio page).  The site for monitoring the errors was a Rails application, but I created a Java client for it that provided 2 reporting options.  

First, I provided an IBMExceptional exception class that could be extended for application specific exceptions.     

        class MyProjectException extends IBMExceptional {
            @SuppressWarnings("unused")
            public MyProjectException() {
                super();
            }

            public MyProjectException(String message, Throwable cause) {
                super(message, cause);
            }

            public MyProjectException(String message) {
                super(message);
            }

            @SuppressWarnings("unused")
            public MyProjectException(Throwable cause) {
                super(cause);
            }
        }

This base exception class would report any instances of thrown exceptions to IBMExceptional.

        try {
            throw new MyProjectException("This is a test exception.");
        } catch (MyProjectException e) {
            // Handle exception as desired, but it was reported to IBM Exceptional
        }

The advantage of this option is that the exception reporting is transparent and automatic.  The disadvantage is that only exceptions that are subclassed from the IBMExceptional base class will be reported. 

Second, I provided a log appender, IBMExceptionalAppender, so that any logged errors would be reported to IBMExceptional.  You added a few configuration lines to log4j.properties like

        log4j.rootCategory=info, exceptional  
        log4j.appender.exceptional=com.ibm.IBMExceptionalAppender

Then calls to the logger with an exception would be reported to IBMExceptional.

        try {
            throw new NullPointerException("Fake Null pointer");
        } catch (NullPointerException e) {
            logger.error("Error description", e);
        }

The advantage of this option is that all exception types can be reported.  The disadvantage is that it requires that the exceptions are logged by the programmer.

This worked well and was all well and good while I remained at IBM, but I have since moved on.  For my current job at USIC, I've led the development of a Java API server using Spring Boot (feeding Angular UIs) and a C# .NET (Framework) application and decided I would similarly like to easily monitor when errors occur on the servers, but this time, instead of having some web site to monitor the errors, I thought it would be much simpler and be sufficient to just send them to dedicated Slack channels.  

For Java we went the route of a log appender, for .NET we could tie into the System.Web.HttpApplication Application_Error handler.

Java Spring Boot application with log appender

Spring Boot uses logback rather than log4j, so in logback-spring.xml we defined a new appender for errors.

	<appender name="SLACK" class="com.usicllc.jsonapi.log.SlackAppender">
		<profile>${profile}</profile>
		<filter class="ch.qos.logback.classic.filter.LevelFilter">
			<level>ERROR</level>
			<onMatch>ACCEPT</onMatch>
			<onMismatch>DENY</onMismatch>
		</filter>
	</appender>

Then the SlackAppender class utilizes the JSlack library and looks like:

public class SlackAppender extends AppenderBase<ILoggingEvent> {
	@Override
	protected void append(ILoggingEvent event) {

		String errorMessage = event.getFormattedMessage();
		StringBuffer message = new StringBuffer("*Error: " + errorMessage + "*\n\n");
		String att = null;

		Map<String, String> mdcMap = event.getMDCPropertyMap();
		if (mdcMap != null) {
			String reqUrl = mdcMap.get("REQUEST_URL");
			String reqMethod = mdcMap.get("REQUEST_METHOD");
			String reqQueryString = mdcMap.get("REQUEST_QUERY_STRING");
			String reqUserAgent = mdcMap.get("REQUEST_USER_AGENT");
			String reqRealIp = mdcMap.get("REQUEST_X_REAL_IP");
			String reqUser = mdcMap.get("REQUEST_USER");
			String reqBody = mdcMap.get("REQUEST_BODY");

			if (reqMethod != null) {
				message.append("*REQUEST_METHOD:* " + reqMethod + "\n");
				message.append("*REQUEST_URL:* " + reqUrl + "\n");
				if (reqQueryString != null) {
					try {
						message.append("*REQUEST_QUERY_STRING:* " + URLDecoder.decode(reqQueryString, "UTF-8") + "\n");
					} catch (UnsupportedEncodingException e) {
					}
				}
				if (reqBody != null) {
					message.append("*REQUEST_BODY:* " + reqBody + "\n");
				}
				message.append("*REQUEST_USER_AGENT:* " + reqUserAgent + "\n");
				message.append("*REQUEST_X_REAL_IP:* " + reqRealIp + "\n");
				message.append("*REQUEST_USER:* " + reqUser + "\n");
			}
		}

		// If we have an exception being reported, then add it to the message
		//
		if (event.getThrowableProxy() != null) {
			StackTraceElementProxy[] stArray = event.getThrowableProxy().getStackTraceElementProxyArray();
			if (stArray.length > 0) {
				// first 10 lines of the stack trace are added to the messages as a code block
				// Full stack trace is also added as an attachment
				message.append("*Stack Trace:*\n```");
				String exceptionMessage = event.getThrowableProxy().getMessage();
				StringBuffer stackTrace = new StringBuffer("*Error Message: " + exceptionMessage + "*\n");

				for (int i = 0; i < stArray.length; i++) {
					StackTraceElement t = stArray[i].getStackTraceElement();
					String traceLine = t.toString();
					if (i < 10) {
						// 1st 10 lines are added to the main message in a code block
						message.append(traceLine + "\n");
					}
					if (traceLine.startsWith("com.usicllc.jsonapi")) {
						// emphasize where the error was in our code
						stackTrace.append("*" + traceLine + "*\n");
					} else {
						stackTrace.append(traceLine + "\n");
					}
				}

				if (stArray.length > 10) {
					message.append("...\n");

					// Full stack trace is more than the 10 lines shown (usually the case)
					// Add the full stacktrace as an attachment
					Attachment attachment = Attachment.builder()
							.title("Full Stack Trace")
							.text(stackTrace.toString())
							.build();
					att = "[{\"title\" :\""+attachment.getTitle()+":\n"+"\",\"text\": \""+attachment.getText().replace("\\", "\\\\")+"\"}]";
				}
				message.append("```");
			}

		}
		String payload = "{\"channel\": \"" + this.getSlackChannel() 
			+ "\",\"username\": \"jsonapi\",\"text\": \""
			+ message.toString().replace("\\", "\\\\") 
			+ "\",\"attachments\":" + att + "}";

		WebhookResponse webhookResponse;
		try {
			webhookResponse = Slack.getInstance().send(
				this.getSlackUrl(), payload);
			if (webhookResponse.getCode() != 200) {
				logger.debug("Unable to send Slack Exception - http error:" + webhookResponse.toString());
			}
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
}

The stack trace is added both with the first 10 lines in a code block in the message body and then the full stack trace as an attachment because its more readable as a code block due to no line wrapping and the use of a monospaced font, but Slack will limit the message size and most stack traces will exceed the limit so adding the full stack trace an attachment provides access to everything albeit in a less readable format.  

Now, it could have just logged the exception message and stack trace, but it's certainly more helpful to know what request caused the problem and that's where we took advantage of the logback MDC (Mapped Diagnoatic Context) to retrieve request data in a Spring GenericFilterBean named PreRequestProcessingFilter Spring will inject this into the request filter chain and the filter code looks like:

@Component
public class PreRequestProcessingFilter extends GenericFilterBean implements CurrentUser {
	Logger logger = LoggerFactory.getLogger(PreRequestProcessingFilter.class);

	@SuppressWarnings("unchecked")
	@Override
	public void doFilter(ServletRequest req, ServletResponse res, FilterChain filterChain) {
		try {
			String bodyJson = null;
			RequestWrapper wrapper = null;
			String method = ((HttpServletRequest) req).getMethod();
			String contentType = ((HttpServletRequest) req).getContentType();

			// Get a copy of the request body for JSON POST and PATCH requests
			if (contentType != null &&
					(contentType.startsWith("application/json") || contentType.startsWith("application/vnd.api+json")) &&
					(method.equalsIgnoreCase("POST") || method.equalsIgnoreCase("PATCH"))) {
				wrapper = new RequestWrapper((HttpServletRequest) req);
				byte[] body = StreamUtils.copyToByteArray(wrapper.getInputStream());
				Map<String, Object> jsonRequest = new ObjectMapper().readValue(body, Map.class);
				bodyJson = jsonRequest.toString();
			}

			String user = getUserName();
			StringBuffer requestURL = ((HttpServletRequest) req).getRequestURL();
			if (requestURL != null) {
				MDC.put("REQUEST_URL", requestURL.toString());
			}

			MDC.put("REQUEST_METHOD", method);
			MDC.put("REQUEST_QUERY_STRING", ((HttpServletRequest) req).getQueryString());
			MDC.put("REQUEST_USER_AGENT", ((HttpServletRequest) req).getHeader("User-Agent"));
			MDC.put("REQUEST_X_REAL_IP", ((HttpServletRequest) req).getHeader("X-Real-IP"));
			MDC.put("REQUEST_USER", user);
			MDC.put("REQUEST_BODY", bodyJson);
			filterChain.doFilter((wrapper != null ? wrapper : req), res);

		} catch (IOException | ServletException e) {
			e.printStackTrace();
		} finally {
			MDC.clear();
		}
	}
}

The big trick in that filter is obtaining the POST or PATCH request body without consuming it and thereby prevening the regular request processing from getting it. Normally, you can only read the request body once. So we create a RequestWrapper and a ServletInputStreamWrapper which allow us to extract a copy of the request's input stream (body). The code for these classes looks like:

public class RequestWrapper extends HttpServletRequestWrapper {

    private byte[] body;

    public RequestWrapper(HttpServletRequest request) throws IOException {
        super(request);

        this.body = StreamUtils.copyToByteArray(request.getInputStream());
    }

    @Override
    public ServletInputStream getInputStream() throws IOException {
        return new ServletInputStreamWrapper(this.body);

    }
}


public class ServletInputStreamWrapper extends ServletInputStream {
    private InputStream inputStream;

    public ServletInputStreamWrapper(byte[] body) {
        this.inputStream = new ByteArrayInputStream(body);
    }

    @Override
    public boolean isFinished() {
        try {
            return inputStream.available() == 0;
        } catch (Exception e) {
            return false;
        }
    }

    @Override
    public boolean isReady() {
        return true;
    }

    @Override
    public void setReadListener(ReadListener listener) {
        // no need to implment this method
    }

    @Override
    public int read() throws IOException {
        return this.inputStream.read();
    }
}

With all this in place, we now get nice Slack messages for errors providing request data and stack trace.  The final result looks like:



.NET Framework Application with Application_Error handler

For the .NET Framework application its much simpler. In our Global.asax.cs file (where we create our application subclass of System.Web.HttpApplication) we define the Application_Error hook which allows us access to the request data, and then we simply send it to Slack utilizing the Slack.Webhooks package.

    public class MvcApplication : System.Web.HttpApplication
    {
        private static readonly bool slackEnabled = Convert.ToBoolean(ConfigurationManager.AppSettings["slackEnabled"]);
        private static readonly string slackURL = ConfigurationManager.AppSettings["slackURL"];
        private static readonly string slackChannel = ConfigurationManager.AppSettings["slackChannel"];
        private static readonly SlackClient slackClient = new SlackClient(slackURL);     
        private static readonly string environment = ConfigurationManager.AppSettings["environmentMode"];

        protected void Application_Error(object sender, EventArgs e)
        {
            Exception ex = null;
            // make sure whatever we do here doesn't generate any exceptions and generate a loop
            try
            {
                ex = Server.GetLastError();
                
                string requestType = HttpContext.Current.Request.RequestType;
                string requestURL = Convert.ToString(HttpContext.Current.Request.Url);
                string requestReferrer = Convert.ToString(HttpContext.Current.Request.UrlReferrer);
                string requestBrowser = HttpContext.Current.Request.Browser.Browser + " " + HttpContext.Current.Request.Browser.Version;
                string requestPlatform = HttpContext.Current.Request.Browser.Platform;
                bool requestIsAuthenticated = HttpContext.Current.Request.IsAuthenticated;
                string requestUserAgent = HttpContext.Current.Request.UserAgent;
                string currentUser = User.Identity.Name?.Split('\\')[1];
                string requestUserAddress = HttpContext.Current.Request.UserHostAddress;
                string requestForwardedForAddress = HttpContext.Current.Request.Headers.Get("X-Forwarded-For");

                Stream requestInputStream = HttpContext.Current.Request.InputStream;
                requestInputStream.Seek(0, SeekOrigin.Begin);
                string requestBody = new StreamReader(requestInputStream).ReadToEnd();

                string logInfoString =
                      "\n   User: " + currentUser
                    + "\n   Request type: " + requestType
                    + "\n   Request URL: " + requestURL
                    + "\n   Referrer: " + requestReferrer
                    + "\n   Browser: " + requestBrowser
                    + "\n   Platform: " + requestPlatform
                    + "\n   UserAgent: " + requestUserAgent
                    + (requestIsAuthenticated ? "" : ("\n   IsAuthenticated: " + requestIsAuthenticated))
                    + "\n   Source address: " + requestUserAddress + (string.IsNullOrEmpty(requestForwardedForAddress) ? "" : 
                      (" / " + requestForwardedForAddress));

                string logInfoString = requestInfoString +
                      (string.IsNullOrEmpty(requestBody) ? "" : ("\n   Request body: " + requestBody));

                log.Error(logInfoString, ex);

                SendSlackMessage("Application Error:", requestInfoString, ex, requestBody);

            }
            catch (Exception exc) {
                // If we have issues getting any of the info above, at least log the exception
                try
                {
                    log.Error("Original exception: ", ex);
                    log.Error("Additionally, there was an exception in Application_Error: " + exc);
                }
                catch (Exception) { }      
            }
        }

        public static void SendSlackMessage(string title, string message, Exception exception, string requestBody)
        {
            // don't send to slack if disabled or running locally
            if (!slackEnabled || environment == "local") return;

            // add time and make the title bold so that messages posted in quick succession can be easily told apart
            // (*stuff* makes it bold)
            string slackText = "*" + DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss") + (title != null ? (" - " + title) : "") + "*";

            if (message != null)
            {
                slackText += "\n" + message;
            }

            List<SlackAttachment> slackAttachments = new List<SlackAttachment>();

            if (!string.IsNullOrWhiteSpace(requestBody))
            {
                var requestBodyAttachment = new SlackAttachment
                {
                    Text = requestBody
                };

                slackAttachments.Add(requestBodyAttachment);
            }

            if (exception != null)
            {
                var slackAttachment = new SlackAttachment
                {
                    Text = exception.ToString(),
                    Color = "#C20202"
                };

                slackAttachments = new List<SlackAttachment> { slackAttachment };
            }

            // local defaults. Comment out first line of the method to test locally if needed
            string slackUser = ".NET Application";
            string slackUserEmoji = Emoji.Wrench;

            var slackMessage = new SlackMessage
            {
                Channel = slackChannel,
                Text = slackText,
                IconEmoji = slackUserEmoji,
                Username = slackUser,
                Attachments = slackAttachments
            };

            slackClient.PostAsync(slackMessage);
        }



Monday, June 28, 2021

The CVS ExtraCare Coupon Clipper

In the same spirit as the Publix Digital Coupon Clipper for which I've received a lot of positive feedback and requests for similar functions for other stores, I also announce the CVS ExtraCare coupon clipper. These digital coupon sites don't like to provide a "Clip All" function.  I presume this is for some advertising purpose so you're forced to look at each one.  I like to have all the coupons clipped and if I happen to buy something there's a coupon it will just apply it at checkout. 

Plus with CVS being notorious for spitting out a whole roll of receipt paper with printed coupons, if they're all pre-clipped electronically on your CVS ExtraCare card I don't believe it will print them so you can save some trees. 

Please go to https://pothoven.net/CvsClipper.html for installation instructions, and please drop me a comment here if you try it out to let me know how it works for you.

Return of the Publix Coupon Clipper

My Publix Coupon Clipper bookmarklet has been offline for a while now due to changes that had been made to the to website code to disable it.  I like to think the changes made were specifically targeted at my bookmarklet, but it may have been purely coincidental too.  Regardless, my handy clipper was rendered inoperable with no way to circumvent what they had done to disable it.

For those reading this who are unfamiliar with it, I introduced the coupon clipper in January 2014 as a "Clip All" function that was (and still is) missing from the site to quickly and easily clip all the digital coupons so they'll all be ready and waiting when you checkout.  I revised it November 2015 to deal with some page changes, but in December of 2016 a change was added to the site that permanently disabled it.

Fortunately for us, they've re-written the coupon site using Vue instead of JQuery and the code that prevented the coupon clipper from working did not make the transition.  I've made the necessary changes to make it work with the new page, and it's working again!  So while it lasts...

Please go to https://pothoven.net/PublixClipper.html for installation instructions, and please drop me a comment here if you try it out to let me know how it works for you.

Wednesday, December 09, 2020

Angular Clipboard Service

It's about time I post something new! Since my last post I've had to learn and build applications in the Ember, React, and now Angular JavaScript Frameworks.  Along the way I've learned some interesting tips and tricks I can share here.


I needed the ability to place content in the clipboard, but needed it lots of placed throughout the application. So here's a simple Clipboard service that will place any string into the clipboard.  I incorporated the use of MatSnackBar to notify the user that the value has been placed in the clipboard.

import {Injectable, Renderer2, RendererFactory2} from '@angular/core';
import { MatSnackBar } from '@angular/material/snack-bar';

@Injectable({
  providedIn: 'root'
})
export class ClipboardService {
  private renderer: Renderer2;

  constructor (
      private rendererFactory: RendererFactory2,
      private snackBar: MatSnackBar,
      ) {
    // Get an instance of Angular's Renderer2
    this.renderer = this.rendererFactory.createRenderer(null, null);
  }

  copy(value: string) {

    // create an temporary text input field to contain the value and select the value
    const input = this.renderer.createElement('input');
    this.renderer.setProperty(input, 'value', value);
    this.renderer.appendChild(document.body, input);
    input.focus();
    input.select();
    input.setSelectionRange(0, 99999); // For mobile devices

    // Copy the selected text inside the text field to the clipboard
    document.execCommand('copy');
    this.snackBar.open(`Copied "${value}" to clipboard`, undefined, { duration: 1000 });

    // remove the temporary text input field
    this.renderer.removeChild(document.body, input);
  }
}

You then simply inject it into any component that needs it:

import { ClipboardService } from 'clipboard.service';

constructor( private clipboard: ClipboardService ) {} 

and then copy whatever values you want to the clipboard simply as:
this.clipboard.copy(value);