Project overview

Project type: High level concept creation, complex workflows, Machine Learning
Role: UX Designer
Timeline: Q3 2017
Partners: Research, Visual Designers, Product Owners, Engineers
Outcome: Recommendations for quicker estimation.
Platforms: desktop web, tablet - web.
Status: Complete

My Design Process

DISCOVER

Context

  • Mitchell Cloud Estimating allows auto body shop estimators to write comprehensive, accurate estimates quickly for vehicles that have been in accidents.

  • Mitchell is working to develop machine learning capabilities, specifically around the ability to recommend parts and labor operations. The goal of this technology is to provide decision support & recommendations within Mitchell Cloud Estimating.  Eventually, this technology is expected to be able to automate the majority of the estimate writing process.

  • This is a big opportunity space for Mitchell, as we were the first in the auto collision industry to attempt to incorporate machine learning into estimating.

This GIF shows an example of the current estimate writing process, where the user (estimator) manually selects parts and labor to add to the estimate.

The process is complex and highly prone to user error. It is also time consuming.

Goals

Business Goals

1. Reduce cost of the estimating process for both carriers and repair facilities by

  1. Reducing the number of estimate drafts and

  2. Improving final estimate quality

  3. Reducing cycle time allowing for quick settling or repair decisions on low-cost estimates/claims by the consumer (by way of providing an initial estimate)

2. Position Mitchell Estimating as a best-in-class smart estimating solution

3. Enabling greater consumer control over the claims process by leaning into the photo based estimating trend

4. Position Mitchell as a technology leader

Design Goals

1. The role of recommendations should be clear.  Machines can be wrong, and the Estimator is still the expert (for now).  The design should be transparent, authentic, and not oversold.

2. Users should be able to give feedback to the system if they feel it is warranted/important, and should understand the impact of their feedback.

3. This a partnership - computers and humans are good at different things; computers should support human decision-making. Balance the workload intelligently.

4. Machine Learning is a core part of the estimating system, and should be viewed as such by the user. No bolt-on solutions.

Competitive Analysis

I did a comparative analysis on products such as Facebook, Netflix, YouTube, and Spotify that have established recommendation display concepts.

Screenshots from Facebook Web. 2017

Screenshot from Spotify Desktop App 2017

DESIGN

Workflow for feedback feature

I ran several iterations through the development team in order to create an ideal workflow for the feedback mechanism. This was to ensure that the workflows were implementable and covered the entire breadth of what a user might want to submit. 

AI Suggested Lines Concept

Several concept sketches of where the AI suggestions appear within the the context of an estimate, where the user can add, edit or remove the suggestions.

Soft Guidance Concept

Lightweight suggestions within the parts panel, that prompts the user to add suggested parts to the estimate. The expertise icon appears at all levels.

Photo Based Estimating Concept

Introducing a new tab entitled “Photos” that showcases suggestions and recommendations within the context of the user uploaded photos.

Photo Based Estimating - Mobile Concept

These are examples of the same workflow for smaller viewports, such as mobile devices and tablets.

Concept Validation Study

The concepts were validated with both customers (insurance) and end-users (auto body shop estimators) where the idea of guided estimating was well received. Some of the feedback included:

  • It would be useful for users to filter lines to only see suggestions.

  • Users like to know the confidence percentage.

  • Some users felt distracted by the noise on the Parts UI and some didn’t understand at first what the number next to Expertise icon meant (the number of recommendations inside a particular category).

  • Users wanted to set what kind of suggestions/lines they would like to see. (Settings feature)

DEVELOP

Photo Based Estimating Prototype

High fidelity designs to showcase the Photo Based Estimating workflow. This version incorporated feedback from the previous concept validation study.

Soft Guidance Concept

Based on feedback, to reduce the noise level on screen, the suggestions were hidden until the user decides to view them by clicking/tapping the Expertise icon. Users are then navigated to the correct part, based on selection.

Quick access to Settings

Included more options within the in-estimate Settings feature to filter out unwanted suggestions.

Lessons Learned

  • The current AI was frequently incorrect - we need to determine whether we have picked the wrong algorithm or failed to include key inputs.

  • It’s important to introduce this feature to experienced estimators in the right way. Many mistrusted the AI, even before observing it make mistakes.

  • The algorithm needs feedback in order to get better. Several users said they are likely to just ignore the feature after a few mistakes rather than try to improve it.